Electronic device and control method thereof

Abstract

This electronic device comprises a memory for storing instructions, a display, a speaker, a microphone, and at least one processor including processing circuitry, wherein the instructions, when executed individually or collectively by the at least one processor: acquire a first audio signal via the microphone; when a preset sound is identified on the basis of the first audio signal, output an ultrasonic wave via the speaker; acquire a second audio signal including a reflected wave corresponding to the ultrasonic wave via the microphone; and when a user is identified as existing around the electronic device on the basis of the second audio signal, perform a preset function.

Description

Electronic device and method of controlling the same

[0001] The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device that uses ultrasonic waves to identify the proximity of a user and a control method thereof.

[0002] To conserve power, electronic devices can only perform certain functions when a user is nearby. For example, a light sensor might illuminate only when a user passes by. Electronic devices can also perform certain functions only when a user is recognized.

[0003] Electronic devices can use various methods to recognize users. When using image sensors to recognize users, image analysis can take time. While ultrasonic sensors can recognize users, electronic devices without them cannot.

[0004] Recognizing users solely using audio signals acquired via a microphone can result in lower accuracy. Recognizing users solely based on audio signals within the audible frequency range can result in lower recognition accuracy than recognizing users using ultrasonic sensors.

[0005] When outputting ultrasound through an ultrasonic sensor, there is a problem that animals or infants can hear the ultrasound.

[0006] The present disclosure is designed to improve the above-described problems, and an object of the present disclosure is to provide an electronic device and a control method thereof that detects a user using both a first audio signal that does not include ultrasonic waves and a second audio signal that includes ultrasonic waves.

[0007] According to one embodiment, an electronic device includes at least one processor including a memory storing instructions, a speaker, a microphone, and processing circuitry, wherein the instructions, when individually or collectively executed by the at least one processor, control to output an ultrasonic wave through the speaker when a preset sound is identified based on a first audio signal acquired through the microphone, and control to perform a function corresponding to the presence of a user when a user is identified as being present around the electronic device based on a second audio signal including a reflected wave corresponding to the ultrasonic wave acquired through the microphone.

[0008] Pre-configured sounds may include artificial sounds based on human behavior.

[0009] The above instructions, when individually or collectively executed by the at least one processor, can obtain an average sound pressure of the first audio signal, and if the average sound pressure is equal to or greater than a preset sound pressure, determine whether to identify the preset sound based on the first audio signal.

[0010] The instructions, when individually or collectively executed by the at least one processor, identify whether the electronic device is in an ultrasonic output disabled state when the preset sound is identified based on the first audio signal, and if the electronic device is not in the ultrasonic output disabled state, output the ultrasonic sound through the speaker, and the ultrasonic output disabled state may include a state in which at least one of the speaker or the microphone is activated.

[0011] The instructions, when individually or collectively executed by the at least one processor, may acquire the first audio signal while the electronic device is operating in a low power state, and change the low power state to a standby state if the electronic device is not in the ultrasonic output disabled state.

[0012] The instructions, when individually or collectively executed by the at least one processor, may obtain a target audio signal by filtering noise from the second audio signal based on a noise threshold value, and identify the presence of the user around the electronic device based on the ultrasonic wave and a reflected wave corresponding to the ultrasonic wave obtained from the target audio signal.

[0013] The instructions, when individually or collectively executed by the at least one processor, obtain a phase difference value based on the first phase of the ultrasonic wave and the second phase of the reflected wave obtained from the target audio signal, and if the phase difference value is greater than or equal to a phase threshold value, it is possible to identify that the user is present around the electronic device.

[0014] The electronic device further includes a display, and the instructions, when individually or collectively executed by the at least one processor, control the display to display a preset screen when it is identified that the user is present around the electronic device, and the preset screen may include at least one of a time UI, a weather UI, a schedule UI, an indoor environment UI, and a guide UI.

[0015] The above ultrasonic wave is a first ultrasonic wave, the reflected wave obtained from the target audio signal is a first reflected wave, and when the instructions are individually or collectively executed by the at least one processor, after displaying the preset screen, a second ultrasonic wave is output through the speaker, a third audio signal including a second reflected wave corresponding to the second ultrasonic wave is obtained through the microphone, and when it is determined that the user is present around the electronic device based on the third audio signal, the display can be controlled to continuously display the preset screen.

[0016] The instructions, when individually or collectively executed by the at least one processor, obtain a first average value of a first threshold number of phase difference values ​​corresponding to a first sensitivity from the second audio signal, and if the first average value is greater than or equal to the phase threshold value, control the display to display the preset screen, obtain a second average value of a second threshold number of phase difference values ​​corresponding to a second sensitivity from the third audio signal, and if the second average value is less than the phase threshold value, control the display not to display the preset screen, and the first threshold number may be greater than the second threshold number.

[0017] According to one embodiment, a method of controlling an electronic device including a speaker and a microphone includes the steps of: outputting an ultrasonic wave through the speaker when a preset sound is identified based on a first audio signal acquired through the microphone; and performing a function corresponding to the presence of a user when a user is identified around the electronic device based on a second audio signal including a reflected wave corresponding to the ultrasonic wave acquired through the microphone.

[0018] Pre-configured sounds may include artificial sounds based on human behavior.

[0019] The above control method may include a step of obtaining an average sound pressure of the first audio signal and, if the average sound pressure is equal to or greater than a preset sound pressure, a step of determining whether or not to identify the preset sound based on the first audio signal.

[0020] The control method includes a step of identifying whether the electronic device is in an ultrasonic output disabled state when the preset sound is identified based on the first audio signal, and a step of outputting the ultrasonic sound through the speaker if the electronic device is not in the ultrasonic output disabled state, wherein the ultrasonic output disabled state may include a state in which at least one of the speaker or the microphone is activated.

[0021] The control method may include a step of acquiring the first audio signal while the electronic device is operating in a low power state, and a step of changing the low power state to a standby state if the electronic device is not in the ultrasonic output disabled state.

[0022] The control method may include a step of obtaining a target audio signal by filtering noise from the second audio signal based on a pre-stored noise threshold value, and a step of identifying whether the user is present around the electronic device based on the ultrasonic wave and a reflected wave corresponding to the ultrasonic wave obtained from the target audio signal.

[0023] The control method may include a step of obtaining a phase difference value based on a first phase of the ultrasonic wave and a second phase of the reflected wave, and a step of identifying that the user is present around the electronic device if the phase difference value is greater than or equal to a phase threshold value.

[0024] The step of performing the above function may include displaying a preset screen when it is identified that the user is present around the electronic device, and the preset screen may include at least one of a time UI, a weather UI, a schedule UI, an indoor environment UI, and a guide UI.

[0025] The above ultrasonic wave is a first ultrasonic wave, the reflected wave is a first reflected wave, and the control method may include a step of outputting a second ultrasonic wave through the speaker after displaying the preset screen, a step of obtaining a third audio signal including a second reflected wave corresponding to the second ultrasonic wave through the microphone, and a step of continuously displaying the preset screen when it is determined that the user is present around the electronic device based on the third audio signal.

[0026] The control method includes a step of obtaining a first average value of a first threshold number of phase difference values ​​corresponding to a first sensitivity from the second audio signal, a step of displaying the preset screen if the first average value is greater than or equal to the phase threshold value, a step of obtaining a second average value of a second threshold number of phase difference values ​​corresponding to a second sensitivity from the third audio signal, and a step of displaying the preset screen if the second average value is less than the phase threshold value, wherein the first threshold number may be greater than the second threshold number.

[0027] According to one embodiment of the present disclosure, an electronic device may include a microphone, a speaker, at least one processor, and a memory storing instructions that, when executed by the at least one processor, cause the electronic device to (a) obtain a first audio signal through the microphone, (b) output an ultrasonic wave through the speaker when a predetermined sound is identified based on the first audio signal obtained through the microphone, (c) obtain a second audio signal including a reflected wave corresponding to the ultrasonic wave output through the speaker through the microphone, and (d) perform a function corresponding to the presence of a user when the presence of a user is identified around the electronic device based on the second audio signal obtained through the microphone.

[0028] FIG. 1 is a drawing for explaining an operation of detecting a user around an electronic device according to one embodiment.

[0029] FIG. 2 is a block diagram illustrating an electronic device according to one embodiment.

[0030] FIG. 3 is a block diagram illustrating a specific configuration of the electronic device of FIG. 2, according to one embodiment.

[0031] FIG. 4 is a diagram illustrating a three-step operation for identifying user proximity, according to one embodiment.

[0032] FIG. 5 is a drawing for explaining a screen provided according to a preset event, according to one embodiment.

[0033] FIG. 6 is a drawing for explaining a first detection operation according to one embodiment.

[0034] FIG. 7 is a drawing for explaining a second detection operation according to one embodiment.

[0035] FIG. 8 is a drawing for explaining a third detection operation according to one embodiment.

[0036] FIG. 9 is a diagram for explaining an operation of processing noise according to one embodiment.

[0037] FIG. 10 is a diagram for explaining an operation for processing general noise according to one embodiment.

[0038] FIG. 11 is a diagram for explaining an operation of processing noise according to a microphone, according to one embodiment.

[0039] FIG. 12 is a diagram for explaining noise and ultrasound according to one embodiment.

[0040] FIG. 13 is a diagram for explaining an operation of selecting one microphone from a two-channel microphone according to one embodiment.

[0041] FIG. 14 is a diagram for explaining an operation of selecting one microphone from a two-channel microphone according to one embodiment.

[0042] FIG. 15 is a diagram for explaining a plurality of states operating in an electronic device according to one embodiment.

[0043] FIG. 16 is a diagram for explaining an operation for processing noise according to a state change, according to one embodiment.

[0044] FIG. 17 is a diagram for explaining noise according to a state change, according to one embodiment.

[0045] FIG. 18 is a diagram for explaining an operation for processing noise according to a state change, according to one embodiment.

[0046] FIG. 19 is a diagram for explaining an operation of applying different sensitivities to each detection operation according to one embodiment.

[0047] Figure 20 is a drawing for explaining sensitivity according to one embodiment.

[0048] FIG. 21 is a drawing for explaining an operation of applying different sensitivities to each detection operation according to one embodiment.

[0049] FIG. 22 is a drawing for explaining an operation of adjusting sensitivity according to one embodiment.

[0050] FIG. 23 is a drawing for explaining negative pressure measurement data according to one embodiment.

[0051] FIG. 24 is a diagram for explaining conditions for performing a noise test according to one embodiment.

[0052] FIG. 25 is a diagram for explaining an operation of obtaining noise information according to one embodiment.

[0053] FIG. 26 is a diagram for explaining a process of obtaining noise information according to one embodiment.

[0054] FIG. 27 is a diagram for explaining noise information for multiple bands according to one embodiment.

[0055] FIG. 28 is a diagram for explaining an operation of analyzing noise information for multiple bands according to one embodiment.

[0056] FIG. 29 is a diagram for explaining an operation of performing a noise test according to one embodiment.

[0057] Figure 30 is a drawing for explanation according to one embodiment.

[0058] Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.

[0059] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0060] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0061] The expression "at least one of A and / or B" should be understood to mean either "A" or "B" or "A and B".

[0062] The expression "and / or" includes all combinations of one or more of the related items.

[0063] As used herein, the expressions “at least one of A and B,” “at least one of A, and B,” “at least one of A and / or B,” “at least one of A, and / or B,” etc. include any of the following: A, B, (A and B).

[0064] As used herein, the expressions “at least one of A, B, and C,” “at least one of A, B, and C,” “at least one of A, B, and / or C,” “at least one of A, B, and / or C,” and the like include any of the following: A, B, C, (A and B), (A and C), (B and C), (A and B and C).

[0065] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0066] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).

[0067] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0068] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be integrated into at least one module and implemented as at least one processor, excluding any "modules" or "parts" that need to be implemented as specific hardware.

[0069] In this specification, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0070] An embodiment of the present disclosure will be described in more detail with reference to the attached drawings below.

[0071] FIG. 1 is a drawing for explaining an operation of detecting a user around an electronic device according to one embodiment.

[0072] Referring to FIG. 1, an electronic device (100) can detect a user present in the surrounding space. The electronic device (100) can confirm the presence of a user through various methods.

[0073] According to one embodiment, the electronic device (100) can identify a user through an image. The electronic device (100) may include an image sensor (e.g., a camera). The electronic device (100) may capture a surrounding space of the electronic device (100) through the image sensor. The electronic device (100) may acquire a captured image through the image sensor. The electronic device (100) may identify whether a human object is included in the captured image.

[0074] According to one embodiment, the electronic device (100) can identify a user through an ultrasonic signal. The electronic device (100) may include an ultrasonic sensor. The electronic device (100) may output an ultrasonic signal for transmission into the surrounding space of the electronic device (100) through the ultrasonic sensor. The electronic device (100) may obtain an ultrasonic signal for reception through the ultrasonic sensor. The ultrasonic signal for reception may be described as a reflected wave. The electronic device (100) may determine whether the user is close based on the obtained ultrasonic signal for reception.

[0075] According to one embodiment, the electronic device (100) can identify a user through an ultrasonic signal without having a separate ultrasonic sensor. The electronic device (100) can output an ultrasonic wave for transmission through a speaker. The electronic device (100) can obtain an ultrasonic wave for reception through a microphone. The electronic device (100) can obtain an audio signal recorded through the microphone. The electronic device (100) can obtain an ultrasonic wave (or a reflected wave) for reception based on the audio signal. The electronic device (100) can determine whether a user is close (or present) based on the ultrasonic wave (or reflected wave) for reception.

[0076] FIG. 2 is a block diagram illustrating an electronic device according to one embodiment.

[0077] Referring to FIG. 2, the electronic device (100) may include various devices including a display (140). The electronic device (100) may be an electronic whiteboard, a TV, a desktop PC, a laptop, a smartphone, a tablet PC, a server, etc. The above-described examples are merely examples for describing the electronic device and are not necessarily limited to the above-described devices.

[0078] At least one processor (120) can perform overall control operations of the electronic device (100). At least one processor (120) can perform a function of controlling overall operations of the electronic device (100).

[0079] An electronic device (100) may include a memory (110) for storing instructions, a display (140), a speaker (170), a microphone (180), and at least one processor (120) including processing circuitry.

[0080] At least one processor (120) may obtain a first audio signal through a microphone (180), output ultrasonic waves through a speaker (170) when a preset sound is identified based on the first audio signal, obtain a second audio signal including a reflected wave corresponding to the ultrasonic waves through the microphone (180), and perform a preset function when it is identified that a user is present around the electronic device (100) based on the second audio signal.

[0081] At least one processor (120) can perform a preset function depending on whether a user is present. The preset function may include at least one of a function for providing information to the user and a function for providing notifications to the user. When a user is present around the electronic device (100), at least one processor (120) can display a preset screen. The preset screen is described in FIG. 5.

[0082] At least one processor (120) may perform a detection operation to determine whether a user is present around the electronic device (100). The detection operation may be performed in three stages. The detection operation may include a first detection operation, a second detection operation, and a third detection operation.

[0083] The action of identifying whether a user is present in the vicinity of an electronic device (100) may be described as an action of identifying whether a user is located in the vicinity of an electronic device (100). The action of identifying whether a user is present in the vicinity of an electronic device (100) may be described as an action of identifying whether a user is present within a threshold distance from an electronic device (100).

[0084] The first detection action may be an action that determines the proximity of a user based on a general audio signal without ultrasonic waves. The first detection action may be a prerequisite for the second detection action.

[0085] The second detection operation may be an operation to determine whether a user is approaching by outputting an ultrasonic wave when a preset sound is identified in the first detection operation. If a user is detected by the second detection operation, at least one processor (120) may perform a preset function.

[0086] The third detection operation may be an operation to detect a user while a preset function is being performed. In order for the preset function to be performed, the user must be continuously identified around the electronic device (100). If the user is identified through the third detection operation, at least one processor (120) may continue to perform the preset function. If the user is not identified through the third detection operation, at least one processor (120) may terminate the preset function.

[0087] Descriptions of the first detection operation, the second detection operation, and the third detection operation are described in Fig. 4.

[0088] At least one processor (120) can acquire a first audio signal via a microphone (180) without outputting ultrasound. The first audio signal can be used in a first sensing operation. At least one processor (120) can identify whether a preset sound is included in the first audio signal.

[0089] Pre-configured sounds may include artificial sounds based on human behavior.

[0090] The preset sound may include at least one of a human voice or an artificial sound generated by a human action. The preset sound may be a sound indicating the presence of a person. For example, the preset sound may include at least one of a voice, footsteps, clapping, laughter, conversation, crying, a door opening and closing, a faucet running, a yawn, or a cough.

[0091] As a first detection operation, at least one processor (120) may obtain an average sound pressure of a first audio signal, and if the average sound pressure is greater than or equal to a preset sound pressure, determine whether a preset sound is identified based on the first audio signal.

[0092] At least one processor (120) can acquire an average sound pressure of the first audio signal during a period in which the first audio signal was acquired. It can be determined whether the average sound pressure is equal to or greater than a preset sound pressure. If the sound pressure is lower than the preset sound pressure, it may be difficult to identify the preset sound. If the average sound pressure is equal to or greater than the preset sound pressure, the at least one processor (120) can determine whether the first audio signal includes the preset sound.

[0093] The operations associated with the first detection operation are described in Fig. 6.

[0094] At least one processor (120) identifies whether the electronic device (100) is in an ultrasonic output disabled state when a preset sound is identified based on the first audio signal, and if the electronic device (100) is not in an ultrasonic output disabled state, it can output ultrasonic waves through the speaker (170). The ultrasonic output disabled state may include a state in which at least one of the speaker (170) or the microphone (180) is activated. The state may be described as a mode.

[0095] When a preset sound is identified based on the first audio signal, at least one processor (120) can perform a second detection operation. The second detection operation is described in FIG. 7.

[0096] At least one processor (120) can identify the state of the electronic device (100). At least one processor (120) can identify whether the state of the electronic device (100) is an ultrasonic output disabled state.

[0097] The ultrasonic output disabled state may include a state in which the speaker (170) is activated. For example, when an audio signal corresponding to content is output through the speaker (170), the electronic device (100) may be in an ultrasonic output disabled state.

[0098] The ultrasonic output disabled state may include a state in which the microphone (180) is activated. For example, when performing a voice recognition function, the electronic device (100) may be in an ultrasonic output disabled state.

[0099] The second detection operation must necessarily perform a function of outputting ultrasonic waves. At least one processor (120) may not perform the second detection operation when ultrasonic output is disabled.

[0100] If the electronic device (100) is not in a state where ultrasonic output is disabled, at least one processor (120) can output ultrasonic waves through the speaker (170).

[0101] The electronic device (100) may operate in a low-power state while performing the first sensing operation. While performing the first sensing operation, a preset screen may not be displayed. In the low-power state, at least one processor (120) may analyze the first audio signal through the first sensing operation.

[0102] At least one processor (120) can acquire a first audio signal while the electronic device (100) is operating in a low power state. At least one processor (120) can change the low power state to a standby state if the electronic device (100) is not in an ultrasonic output disabled state.

[0103] The low power state may indicate a state in which power supply to at least a portion of the electronic device (100) is cut off while power supply to the display (140) is cut off. The low power state may be a state in which only preset functions are performed. The low power state may be a state in which power is supplied only to a module performing a preset function among multiple functions performed by the electronic device (100). For example, the low power state may be a state in which power is supplied only to a function performing the first detection operation of FIG. 6. In the low power state, the second detection operation and the third detection operation described in FIGS. 7 and 8 cannot be performed.

[0104] A low-power state may be a state in which only preset functions are performed. The low-power state may be a state in which power is supplied only to modules that perform preset functions among the multiple functions performed by the electronic device (100). For example, the low-power state may be a state in which power is supplied only to a function that performs the first detection operation of FIG. 6. In the low-power state, the second detection operation and the third detection operation described in FIGS. 7 and 8 cannot be performed.

[0105] The standby state performs more functions than the low-power state, but some functions may be limited. The second and third sensing operations may be performed in the standby state. The standby state may be a state for displaying a preset screen. The electronic device (100) may supply power to the display (140) in the standby state to display a preset screen.

[0106] At least one processor (120) can obtain a target audio signal by filtering out noise from a second audio signal based on a pre-stored noise threshold value. At least one processor (120) can identify a reflected wave corresponding to an ultrasonic wave from the target audio signal. At least one processor (120) can identify whether a user is present around the electronic device (100) based on the ultrasonic wave and the reflected wave. A description related to this is described in FIG. 10.

[0107] Specific descriptions of the noise filtering operation are described in FIGS. 9 through 18. A method for processing general noise is described in FIG. 10. A method for processing noise based on microphone characteristics is described in FIGS. 11 through 14. A method for processing noise according to state transitions is described in FIGS. 15 through 18.

[0108] At least one processor (120) can obtain a phase difference value based on the first phase of the ultrasonic wave and the second phase of the reflected wave. If the phase difference value is greater than or equal to a phase threshold value, the at least one processor (120) can identify that a user is present around the electronic device (100).

[0109] The first phase may be the fundamental phase of the ultrasonic wave output through the speaker (170). The first phase may be a preset value. The second phase may correspond to a reflected wave. The reflected wave may represent a signal reflected by a person (user) or an object. When the ultrasonic wave is reflected, the phase may change. Therefore, the first phase and the second phase may differ.

[0110] At least one processor (120) can obtain a first phase of ultrasound. At least one processor (120) can extract (or identify) a reflected wave from a second audio signal. At least one processor (120) can obtain a phase difference value based on a difference value between the first phase and the second phase. At least one processor (120) can identify whether the phase difference value is greater than or equal to a phase threshold value. The phase threshold value may be a value indicating whether a user is close.

[0111] If the phase difference value is greater than or equal to the phase threshold, at least one processor (120) can identify that a user has been detected around the electronic device (100).

[0112] If the phase difference value is less than the phase threshold, at least one processor (120) can identify that no user has been detected around the electronic device (100).

[0113] At least one processor (120) can control the display (140) to display a preset screen when it is determined that a user is present around the electronic device (100). The preset screen may include at least one of a time UI, a weather UI, a schedule UI, an indoor environment UI, and a guide UI. A description related thereto is provided in FIG. 5.

[0114] After displaying the preset screen, at least one processor (120) may perform a third detection operation. The third detection operation may be an operation for determining whether the user continues to be identified around the electronic device (100) while the preset screen is displayed. The third detection operation is described in FIG. 8.

[0115] In the second detection operation, the ultrasonic wave may be a first ultrasonic wave, the reflected wave may be a first reflected wave, and the phase difference value may be a first phase difference value.

[0116] According to the third detection action, at least one processor (120) may display a preset screen and then output a second ultrasonic wave through the speaker (170). At least one processor (120) may obtain a third audio signal including a second reflected wave corresponding to the second ultrasonic wave through the microphone (180). If it is determined that a user is present around the electronic device (100) based on the third audio signal, at least one processor (120) may control the display (140) to continuously display the preset screen.

[0117] At least one processor (120) can identify whether a user is continuously detected in the vicinity of the electronic device (100). At least one processor (120) can output a second ultrasonic wave through the speaker (170).

[0118] In one embodiment, the first ultrasonic wave output in the second sensing operation and the second ultrasonic wave output in the third sensing operation may be the same.

[0119] In one embodiment, the first ultrasonic wave output in the second sensing operation and the second ultrasonic wave output in the third sensing operation may be different.

[0120] At least one processor (120) may acquire a new third audio signal after outputting the second ultrasonic wave. Based on the third audio signal, the at least one processor (120) may determine whether a user is present around the electronic device (100). The third audio signal may include a second reflected wave corresponding to the second ultrasonic wave. If the user is still present, the phase of the second ultrasonic wave may be different from the phase of the second reflected wave.

[0121] At least one processor (120) can acquire a third phase of the second ultrasound. At least one processor (120) can acquire a fourth phase of the second reflected wave. At least one processor (120) can acquire a second phase difference value based on a difference value between the third phase and the fourth phase. At least one processor (120) can determine whether the second phase difference value is greater than or equal to a phase threshold value.

[0122] If the second phase difference value is greater than or equal to the phase threshold value, at least one processor (120) can identify that a user is present around the electronic device (100).

[0123] If the second phase difference value is less than the phase threshold, at least one processor (120) can identify that the user is not present around the electronic device (100).

[0124] At least one processor (120) can detect a user by adjusting sensitivity. At least one processor (120) can apply different sensitivities in the second sensing operation and the third sensing operation.

[0125] At least one processor (120) can obtain a first average value of a first threshold number of phase difference values ​​corresponding to a first sensitivity from a second audio signal.

[0126] At least one processor (120) can control the display (140) to display a preset screen if the first average value is greater than or equal to a phase threshold value.

[0127] At least one processor (120) can obtain a second average value of a second threshold number of phase difference values ​​corresponding to a second sensitivity from a third audio signal. At least one processor (120) can control the display (140) to not display a preset screen if the second average value is less than the phase threshold value.

[0128] The first critical number may be greater than the second critical number.

[0129] The second detection action may be an action to detect a user while the preset screen is off. The third detection action may be an action to detect a user while the preset screen is on.

[0130] The first sensitivity used in the second sensing operation may be less sensitive than the second sensitivity used in the third sensing operation, as this may waste power if a preset screen is displayed due to an error even when the user is not present.

[0131] The second sensitivity used in the third sensing operation may be more sensitive than the first sensitivity used in the second sensing operation, because if a preset screen that has been displayed once turns off due to an error, the user may experience great inconvenience.

[0132] Descriptions regarding sensitivity are described in FIGS. 19 to 22.

[0133] When outputting ultrasonic waves through a speaker (170) and receiving reflected waves through a microphone (180), a separate ultrasonic sensor is not required. The operations performed by the electronic device (100) can also be applied to various types of devices that do not include an ultrasonic sensor.

[0134] Even when using an ultrasonic sensor, noise processing methods or user detection methods can be applied in the same manner. For example, the electronic device (100) may include an ultrasonic sensor. The electronic device (100) may output ultrasonic waves through the ultrasonic sensor. The electronic device (100) may receive reflected waves through the ultrasonic sensor.

[0135] The electronic device (100) can detect a user with high accuracy using only a speaker (170) and a microphone (180), without the need for an ultrasonic sensor or motion sensor. By using the speaker (170) instead of an ultrasonic sensor, the influence of animals or infants can be minimized. The accuracy of user detection can be improved by performing noise processing operations.

[0136] According to one embodiment, all operations performed in the electronic device (100) may be performed in the electronic device (100). The first sensing operation of FIG. 6, the second sensing operation of FIG. 7, and the third sensing operation of FIG. 8 may all be performed in the electronic device (100).

[0137] According to one embodiment, the first detection operation may be performed in the electronic device (100), and the second detection operation and the third detection operation may be performed in an external server. The electronic device (100) may transmit a second audio signal (or a third audio signal) to the external server. The external server may receive the second audio signal (or the third audio signal) from the electronic device (100). The external server may determine whether a user is present based on the received second audio signal (or the third audio signal). The external server may perform a noise filtering function on the received second audio signal (or the third audio signal). The operations of FIGS. 9 to 18 related to the noise processing operation may be performed in the same manner in the external server.

[0138] According to one embodiment, the electronic device (100) can detect a user using artificial intelligence technology. The electronic device (100) can store an artificial intelligence model. The electronic device (100) can input an audio signal to the artificial intelligence model. The electronic device (100) can obtain output data indicating whether or not a user has been detected based on the artificial intelligence model. The electronic device (100) can display a preset screen according to user detection based on the output data.

[0139] For example, the artificial intelligence model may be a model that performs all of the first sensing action, the second sensing action, and the third sensing action.

[0140] For example, the artificial intelligence model may be a model that performs a second sensing action and a third sensing action.

[0141] The electronic device (100) can input a second audio signal (or a third audio signal) obtained after outputting ultrasound to an artificial intelligence model. Based on the input second audio signal (or third audio signal), the electronic device (100) can obtain result data indicating whether a user is present around the electronic device (100) as output data.

[0142] The electronic device (100) can utilize artificial intelligence (AI) utilizing a machine learning algorithm.

[0143] An artificial intelligence system is a computer system that implements human-level intelligence, and can be a system in which the machine learns and makes judgments on its own, and its recognition rate improves with use.

[0144] Artificial intelligence technology can be composed of machine learning (deep learning) technology that uses algorithms that classify / learn the characteristics of input data on their own, and element technologies that use machine learning algorithms to simulate the cognitive and judgment functions of the human brain.

[0145] The element technologies may include, for example, at least one of: linguistic understanding technology that recognizes human language / characters; visual understanding technology that recognizes objects as if they were human vision; inference / prediction technology that judges information and logically infers and predicts; knowledge expression technology that processes human experience information into knowledge data; and motion control technology that controls autonomous driving of vehicles and movements of robots.

[0146] FIG. 3 is a block diagram illustrating a specific configuration of the electronic device of FIG. 2, according to one embodiment.

[0147] Referring to FIG. 3, the electronic device (100) may include at least one of a memory (110), at least one processor (120), a communication interface (130), a display (140), an operation interface (150), an input / output interface (160), a speaker (170), a microphone (180), and a camera (190).

[0148] The memory (110) may be implemented as an internal memory such as a ROM (e.g., an electrically erasable programmable read-only memory (EEPROM)) or RAM included in at least one processor (120), or may be implemented as a separate memory from at least one processor (120). The memory (110) may be implemented as a memory embedded in the electronic device (100) or as a memory detachable from the electronic device (100) depending on the purpose of data storage. For example, data for driving the electronic device (100) may be stored in a memory embedded in the electronic device (100), and data for expanding functions of the electronic device (100) may be stored in a memory detachable from the electronic device (100).

[0149] In the case of memory embedded in the electronic device (100), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), etc.), hard drive, or solid state drive (SSD), and in the case of memory that can be attached or detached to the electronic device (100), it may be implemented in the form of a memory card (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multi-media card (MMC), etc.), external memory that can be connected to a USB port (e.g., USB memory), etc.

[0150] The memory (110) can store at least one instruction. Based on the instruction stored in the memory (110), at least one processor (120) can perform various operations.

[0151] At least one processor (120) may be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON) that processes digital signals. However, the present invention is not limited thereto, and may include one or more of a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a graphics-processing unit (GPU), a communication processor (CP), or an advanced reduced instruction set computer (RISC) machines (ARM) processor, or may be defined by the relevant terminology. At least one processor (120) may be implemented as a system on chip (SoC) having a built-in processing algorithm, a large scale integration (LSI), or may be implemented in the form of a field programmable gate array (FPGA). At least one processor (120) may perform various functions by executing computer executable instructions stored in a memory.

[0152] The communication interface (130) is a configuration that performs communication with various types of external devices according to various types of communication methods. The communication interface (130) may include a wireless communication module or a wired communication module. Each communication module may be implemented in the form of at least one hardware chip.

[0153] A wireless communication module may be a module that communicates wirelessly with an external device. For example, the wireless communication module may include at least one of a Wi-Fi module, a Bluetooth module, an infrared communication module, or other communication modules.

[0154] Wi-Fi and Bluetooth modules can communicate via Wi-Fi and Bluetooth, respectively. When using a Wi-Fi or Bluetooth module, various connection information, such as the service set identifier (SSID) and session key, is first transmitted and received. This information is then used to establish a communication connection before various other information can be transmitted and received.

[0155] Infrared communication modules perform communication based on infrared communication (IrDA, infrared Data Association) technology, which transmits data wirelessly over short distances using infrared light, which is between visible light and millimeter waves.

[0156] In addition to the above-described communication method, other communication modules may include at least one communication chip that performs communication according to various wireless communication standards such as zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc.

[0157] A wired communication module may be a module that communicates with an external device via a wire. For example, the wired communication module may include at least one of a Local Area Network (LAN) module, an Ethernet module, a paired cable, a coaxial cable, a fiber optic cable, or an Ultra Wide-Band (UWB) module.

[0158] According to one embodiment, the communication interface (130) may utilize the same communication module (e.g., a Wi-Fi module) to communicate with an external device such as a remote control device and an external server.

[0159] According to one embodiment, the communication interface (130) may utilize different communication modules to communicate with external devices, such as remote control devices, and external servers. For example, the communication interface (130) may utilize at least one of an Ethernet module or a Wi-Fi module to communicate with an external server, and may also utilize a Bluetooth module to communicate with an external device, such as a remote control device. However, this is merely one embodiment, and the communication interface (130) may utilize at least one of various communication modules when communicating with multiple external devices or external servers.

[0160] The display (140) may be implemented as a variety of displays such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display panel (PDP), etc. The display (140) may also include a driving circuit, a backlight unit, etc., which may be implemented as a form such as an a-si TFT (amorphous silicon thin film transistor), an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. The display (140) may be implemented as a touch screen combined with a touch sensor, a flexible display, a three-dimensional display (3D display, three-dimensional display), etc. According to an embodiment of the present disclosure, the display (140) may include not only a display panel that outputs an image, but also a bezel that houses the display panel. In particular, according to an embodiment of the present disclosure, the bezel may include a touch sensor for detecting user interaction.

[0161] The operating interface (150) may be implemented as a device such as a button, a touch pad, a mouse, and a keyboard, or as a touch screen capable of performing the above-described display function and operating input function. The button may be a mechanical button, a touch pad, a wheel, or any other type of button formed in any area of ​​the front, side, or back of the main body of the electronic device (100).

[0162] The input / output interface (160) may be any one of HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), and DVI (Digital Visual Interface). The input / output interface (160) may input / output at least one of audio and video signals. Depending on the implementation example, the input / output interface (160) may include a port that inputs / outputs only audio signals and a port that inputs / outputs only video signals as separate ports, or may be implemented as a single port that inputs / outputs both audio signals and video signals. The electronic device (100) may transmit at least one of the audio and video signals to an external device (e.g., an external display device or an external speaker) through the input / output interface (160). An output port included in the input / output interface (160) can be connected to an external device, and the electronic device (100) can transmit at least one of an audio and video signal to the external device through the output port.

[0163] The input / output interface (160) can be connected to a communication interface. The input / output interface (160) can transmit information received from an external device to the communication interface or transmit information received through the communication interface to the external device.

[0164] The speaker (170) may be a component that outputs various audio data as well as various notification sounds or voice messages.

[0165] The microphone (180) is a component that receives a user's voice or other sounds and converts them into audio data. The microphone (180) can receive the user's voice in an activated state. For example, the microphone (180) can be formed integrally on the upper side, the front side, the side side, etc. of the electronic device (100). The microphone (180) can include various components such as a microphone that collects the user's voice in analog form, an amplifier circuit that amplifies the collected user's voice, an A / D conversion circuit that samples the amplified user's voice and converts it into a digital signal, and a filter circuit that removes noise components from the converted digital signal.

[0166] The camera (190) is a device configured to capture a subject and generate a captured image, and the captured image includes both moving images and still images. The camera (190) can acquire images for at least one external device and can be implemented with a camera, lens, infrared sensor, or the like.

[0167] The camera (190) may include a lens and an image sensor. The type of lens may include a general-purpose lens, a wide-angle lens, a zoom lens, etc., and may be determined according to the type, characteristics, usage environment, etc. of the electronic device (100). The image sensor may include a complementary metal oxide semiconductor (CMOS) and a charge-coupled device (CCD).

[0168] According to one embodiment, the electronic device (100) may include a display (140). The electronic device (100) may directly display an acquired image or content on the display (140).

[0169] According to one embodiment, the electronic device (100) may not include a display (140). The electronic device (100) may be connected to an external display device and may transmit images or content stored in the electronic device (100) to the external display device.

[0170] The electronic device (100) can transmit an image or content to an external display device along with a control signal for controlling the display of the image or content on the external display device. The external display device can be connected to the electronic device (100) via a communication interface (130) or an input / output interface (160). For example, the electronic device (100) may not include a display, such as a set-top box (STB).

[0171] The electronic device (100) may include only a small display capable of displaying simple information such as text information. The electronic device (100) may transmit images or content to an external display device via a communication interface (130) wired or wirelessly, or transmit the images or content to the external display device via an input / output interface (160).

[0172] There may be an embodiment in which the electronic device (100) performs an action corresponding to a user voice signal received through a microphone (180).

[0173] According to one embodiment, the electronic device (100) can control the display (140) based on a user voice signal received through the microphone (180). For example, when a user voice signal for displaying content A is received, the electronic device (100) can control the display (140) to display content A.

[0174] According to one embodiment, the electronic device (100) can control an external display device connected to the electronic device (100) based on a user voice signal received through the microphone (180). The electronic device (100) can generate a control signal for controlling the external display device so that an operation corresponding to the user voice signal is performed on the external display device, and transmit the generated control signal to the external display device. The electronic device (100) can store a remote control application for controlling the external display device. In addition, the electronic device (100) can transmit the generated control signal to the external display device using at least one communication method among Bluetooth, Wi-Fi, and infrared. For example, when a user voice signal for displaying content A is received, the electronic device (100) can transmit a control signal for controlling the display of content A on the external display device to the external display device. The electronic device (100) may refer to various terminal devices on which a remote control application can be installed, such as a smartphone or an AI speaker.

[0175] According to one embodiment, the electronic device (100) may use a remote control device to control an external display device connected to the electronic device (100) based on a user voice signal received through a microphone (180). The electronic device (100) may transmit a control signal to the remote control device for controlling the external display device so that an operation corresponding to the user voice signal is performed on the external display device. In addition, the remote control device may transmit the control signal received from the electronic device (100) to the external display device. For example, when a user voice signal for displaying content A is received, the electronic device (100) may transmit a control signal to the remote control device for controlling the display of content A on the external display device, and the remote control device may transmit the received control signal to the external display device.

[0176] The electronic device (100) can receive a user voice signal in various ways.

[0177] According to one embodiment, the electronic device (100) can receive a user voice signal through a microphone (180) included in the electronic device (100).

[0178] According to one embodiment, the electronic device (100) may receive a user voice signal from an external device including a microphone. The external device may refer to a remote control device or a smartphone, etc. The received user voice signal may be a digital voice signal, but may also be an analog voice signal depending on the implementation example. The electronic device (100) may receive the user voice signal via a wireless communication method such as Bluetooth or Wi-Fi.

[0179] The electronic device (100) can convert the user voice signal in various ways.

[0180] According to one embodiment, the electronic device (100) can obtain text information corresponding to a user voice signal from an external server. The electronic device (100) can transmit the user voice signal (audio signal or digital signal) to the external server. The external server may refer to a voice recognition server. The voice recognition server can convert the user voice signal into text information using STT (Speech To Text). Then, the external server can transmit text information corresponding to the converted user voice signal to the electronic device (100).

[0181] According to one embodiment, the electronic device (100) can independently acquire text information corresponding to a user voice signal. The electronic device (100) can also directly apply a STT (Speech To Text) function to a digital voice signal to convert it into text information and transmit the converted text information to an external server.

[0182] An external server can transmit information to the electronic device (100) in a variety of ways.

[0183] According to one embodiment, an external server may transmit text information corresponding to a user voice signal to an electronic device (100). The external server may be a server that performs a voice recognition function that converts a user voice signal into text information.

[0184] According to one embodiment, the external server may transmit at least one of text information corresponding to the user voice signal or search result information corresponding to the text information to the electronic device (100). The external server may be a server that performs a search result providing function that provides search result information corresponding to the text information in addition to a voice recognition function that converts the user voice signal into text information. For example, the external server may be a server that performs both a voice recognition function and a search result providing function. In another example, the external server may perform only a voice recognition function and the search result providing function may be performed by a separate server. The external server may transmit text information to a separate server to obtain a search result and obtain a search result corresponding to the text information from the separate server.

[0185] The electronic device (100) can communicate with external devices and external servers in various ways.

[0186] According to one embodiment, communication modules for communication with external devices and external servers may be implemented identically. For example, the electronic device (100) may communicate with the external device using a Bluetooth module, and the external server may also communicate using a Bluetooth module.

[0187] According to one embodiment, communication modules for communication with external devices and external servers may be implemented separately. For example, the electronic device (100) may communicate with external devices using a Bluetooth module and with external servers using an Ethernet modem or Wi-Fi module.

[0188] FIG. 4 is a diagram illustrating a three-step operation for identifying user proximity, according to one embodiment.

[0189] Referring to FIG. 4, the electronic device (100) can identify the proximity of a user based on a general audio signal (S410). The general audio signal may represent an audio signal that does not include ultrasonic waves. The general audio signal may only include signals within the human audible frequency range. The general audio signal may be a signal output that does not include signals in the ultrasonic band.

[0190] The electronic device (100) can identify the presence of a user based on a general audio signal. The electronic device (100) can identify the proximity of a user based on the general audio signal. The electronic device (100) can identify whether a user is present within a threshold distance from the electronic device (100) based on the general audio signal. The electronic device (100) can identify whether a user is approaching based on the general audio signal. The proximity can indicate whether the user is approaching the electronic device (100) or moving away from the electronic device (100).

[0191] For example, a general audio signal may be output through a speaker (170) of an electronic device (100). Step S410 may be described as a first detection operation.

[0192] If a user is identified as being in proximity based on a general audio signal (S410-Y), the electronic device (100) can identify whether the user is in proximity based on an ultrasonic audio signal (S420). The ultrasonic audio signal may represent an audio signal including ultrasonic waves. The ultrasonic audio signal may include a signal in the ultrasonic band.

[0193] The electronic device (100) can identify the presence of a user based on an ultrasonic audio signal. The electronic device (100) can identify the proximity of a user based on the ultrasonic audio signal. The electronic device (100) can identify whether a user is present within a threshold distance from the electronic device (100) based on the ultrasonic audio signal. The electronic device (100) can identify whether a user is approaching based on the ultrasonic audio signal. The proximity can indicate whether the user is approaching the electronic device (100) or moving away from the electronic device (100).

[0194] For example, an ultrasonic audio signal may be output through a speaker (170) of an electronic device (100). Step S420 may be described as a second sensing operation.

[0195] If a user is identified as being in proximity based on an ultrasonic audio signal (S420-Y), the electronic device (100) may display a preset screen (S425). The preset screen may be a screen related to a preset service (e.g., a daily board screen) provided by the electronic device (100). The preset screen may be described as a home screen, a service screen, a home widget, etc. provided to the user. Details related to the preset screen are described in FIG. 5.

[0196] After displaying a preset screen, the electronic device (100) can identify whether the user is approaching based on an ultrasonic audio signal (S430). The ultrasonic audio signal may represent an audio signal including ultrasonic waves. The ultrasonic audio signal may include a signal in the ultrasonic band.

[0197] The electronic device (100) can identify the presence of a user based on an ultrasonic audio signal. The electronic device (100) can identify the proximity of a user based on the ultrasonic audio signal. The electronic device (100) can identify whether a user is present within a threshold distance from the electronic device (100) based on the ultrasonic audio signal. The electronic device (100) can identify whether a user is approaching based on the ultrasonic audio signal. The proximity can indicate whether the user is approaching the electronic device (100) or moving away from the electronic device (100).

[0198] For example, an ultrasonic audio signal may be output through a speaker (170) of an electronic device (100). Step S420 may be described as a third sensing operation.

[0199] If the user is identified as being in proximity (S430-Y), the electronic device (100) can perform steps S425 and S430. If the user is identified as not being in proximity (S430-N), the electronic device (100) can turn off the display (140). The electronic device (100) can control the display (140) to be turned off. The electronic device (100) can operate in a low power state (low power mode).

[0200] FIG. 5 is a drawing for explaining a screen provided according to a preset event, according to one embodiment.

[0201] Referring to FIG. 5, the electronic device (100) may display a preset screen (500) according to a preset event. The preset event may include at least one of an event in which a user is identified around the electronic device (100), an event in which the user's movement is identified, and an event in which the user is identified as being within a threshold distance from the electronic device (100).

[0202] The preset screen (500) may include at least one piece of information. The preset screen (500) may include at least one piece of information among time information (501), weather information (502), schedule information (503), indoor environment information (504), and guide information (505).

[0203] Time information can include the current time and current date.

[0204] Weather information may include weather corresponding to the current location of the electronic device (100).

[0205] Schedule information may include previously registered schedules corresponding to the current date. For example, schedule information may include birthday information.

[0206] Indoor environment information may include at least one of indoor temperature, indoor humidity, and indoor illuminance of the space in which the electronic device (100) is placed.

[0207] Guide information may include information to guide a user to a specific action. For example, the guide information may include at least one of a guide action for changing a favorite item and a guide action for connecting to an external device.

[0208] FIG. 6 is a drawing for explaining a first detection operation according to one embodiment.

[0209] Referring to FIG. 6, the electronic device (100) may turn off the display (140) (S605). When the electronic device (100) operates in a low-power state (or low-power mode), the display (140) may be turned off. The low-power state may refer to a state in which power supply to at least some components of the electronic device (100) is cut off while power supply to the display (140) is cut off. While operating in the low-power state, power is not supplied to all components of the electronic device (100). While operating in the low-power state, the electronic device (100) may perform a function of detecting the proximity of a user.

[0210] When the display (140) is turned off, the electronic device (100) can identify whether a proximity detection function of the user is running (S610). The proximity detection function may be a function for determining whether a user is present in the vicinity of the electronic device (100). The proximity detection function may be a function for performing at least one of a first detection operation, a second detection operation, and a third detection operation. The proximity detection function may be performed according to the user's settings. In response to a user input that executes the proximity detection function, the electronic device (100) can activate the proximity detection function. In response to a user input that does not execute the proximity detection function, the electronic device (100) can deactivate the proximity detection function.

[0211] When the user's proximity detection function is running (S610 -Y), the electronic device (100) can obtain a first audio signal (S615). The electronic device (100) can obtain the first audio signal through the microphone (180). The electronic device (100) can perform a recording function through the microphone (180). The electronic device (100) can obtain the first audio signal including ambient sounds through the microphone (180). The electronic device (100) can analyze the first audio signal to determine whether a user is present.

[0212] The electronic device (100) can determine whether a preset sound pressure level or higher is identified in the first audio signal (S610).

[0213] For example, the electronic device (100) can determine whether a preset sound pressure level or higher is identified in the first audio signal.

[0214] For example, the electronic device (100) can obtain the average sound pressure of the first audio signal. The electronic device (100) can identify whether the average sound pressure is greater than or equal to a preset sound pressure.

[0215] If the preset sound pressure is not identified in the first audio signal (S620-N), the electronic device (100) can obtain a new audio signal.

[0216] When a preset sound pressure is identified in the first audio signal (S620-Y), the electronic device (100) can determine whether a preset sound is identified in the first audio signal (S625).

[0217] The preset sound may include at least one of a human voice, a sound made by a human, or a sound indicating that a human is approaching the electronic device (100). The preset sound is not limited thereto.

[0218] If a preset sound is not identified in the first audio signal (S625-N), the electronic device (100) can acquire a new audio signal.

[0219] When a preset sound is identified in the first audio signal (S625-Y), the electronic device (100) can detect the proximity of a user using ultrasonic waves (second detection operation) (S630). The electronic device (100) can operate in a standby state (or standby mode) to perform the second detection operation.

[0220] FIG. 7 is a drawing for explaining a second detection operation according to one embodiment.

[0221] Referring to FIG. 7, when a user is identified as being close by the first detection operation, the electronic device (100) can perform a second detection operation.

[0222] The electronic device (100) can identify whether the electronic device (100) is in an ultrasonic output disabled state (S705). The ultrasonic output disabled state can be described as an ultrasonic output disabled event.

[0223] In one embodiment, the ultrasonic output disabled state may include a case where at least one of the current speaker (170) or microphone (180) is activated.

[0224] When the speaker (170) is activated, it may be difficult to output a specific ultrasound. When the speaker (170) is activated, it may be that the user is currently using the electronic device (100). When the microphone (180) is activated, it may be difficult to receive a reflected wave corresponding to the ultrasound. When the microphone (180) is activated, it may be that the user is currently using the electronic device (100).

[0225] For example, when a voice recognition function is performed, the electronic device (100) may determine that the speaker (170) or microphone (180) is activated.

[0226] For example, when an electronic device (100) displays content through a display (140), the electronic device (100) may determine that the speaker (170) is activated.

[0227] In one embodiment, the ultrasonic output failure state may be a state in which a preset sound pressure level in the ultrasonic band is identified. The second detection operation must output sound pressure in the ultrasonic band. If a preset sound pressure level in the ultrasonic band is already being output around the electronic device (100), the accuracy of the second detection operation may decrease.

[0228] The electronic device (100) can obtain the average sound pressure of the ultrasonic band from the first audio signal acquired in a state where ultrasonic waves are not output. If the average sound pressure of the ultrasonic band is higher than a preset sound pressure, the electronic device (100) can determine that the electronic device (100) is currently in a state where ultrasonic output is not possible.

[0229] The preset negative pressure described in step S705 and the preset negative pressure described in step S620 of FIG. 6 may differ. For clarity, the preset negative pressure of step S620 of FIG. 6 may be referred to as the first threshold negative pressure. The preset negative pressure of step S705 may be referred to as the second threshold negative pressure.

[0230] If it is identified that the ultrasonic output is not possible (S705-Y), the electronic device (100) can output ultrasonic waves (S710). The electronic device (100) can operate in a standby state (or standby mode). The electronic device (100) can output ultrasonic waves while operating in the standby state. For example, the electronic device (100) can output ultrasonic waves through the speaker (170). The electronic device (100) can output an audio signal including ultrasonic waves. The ultrasonic waves output may be pre-stored signals to determine whether a user is approaching.

[0231] After outputting the ultrasonic wave, the electronic device (100) can obtain a second audio signal (S715). The electronic device (100) can obtain the second audio signal through the microphone (180). The electronic device (100) can perform a recording function for a preset period of time from the time when the ultrasonic wave is output through the microphone (180). The electronic device (100) can obtain the second audio signal by performing the recording function.

[0232] The electronic device (100) can identify the presence of a user based on a second audio signal. The electronic device (100) can identify a reflected wave corresponding to an ultrasonic wave in the second audio signal (S720). The ultrasonic wave may be the ultrasonic wave output by the electronic device (100) in step S710. The ultrasonic wave may be described as an output ultrasonic wave. The ultrasonic wave output in step S710 may be reflected by a person. The electronic device (100) can identify the presence of a person by identifying the ultrasonic wave (reflected wave) reflected by the person.

[0233] If the presence of a person is identified based on the reflected wave (S725-Y), the electronic device (100) can display a preset screen (S730). The preset screen is described in FIG. 5.

[0234] If the presence of a person is not identified based on the reflected wave (S725-N), the electronic device (100) can identify whether a first threshold time has elapsed (S735). The electronic device (100) can identify whether the first threshold time has elapsed from the time at which the ultrasonic wave was output.

[0235] If the first threshold time has not elapsed (S735-N), the electronic device (100) can perform steps S710 to S735.

[0236] When the first threshold time elapses (S735-Y), the electronic device (100) can turn off the display (140) (S740). The electronic device (100) can change the state of the electronic device (100) from a standby state (or standby mode) in which the electronic device (100) operates for ultrasonic output to a low power state (or low power mode). In a specific embodiment, step S740 may be omitted. If the second detection operation is performed while the display (140) is turned off, step S740 may not be performed because the display (140) is already turned off. Step S740 is related to the third detection operation. A description related thereto is described in FIG. 8.

[0237] FIG. 8 is a drawing for explaining a third detection operation according to one embodiment.

[0238] Referring to FIG. 8, the electronic device (100) can calculate the elapsed time to determine whether to continue displaying a preset screen.

[0239] The electronic device (100) can display a preset screen (S805). When the preset screen is displayed, the electronic device (100) can initialize the elapsed time (S810).

[0240] The electronic device (100) can identify whether a second threshold time has elapsed from the time the preset screen is displayed (S815). If the second threshold time has elapsed (S815-Y), the electronic device (100) can turn off the display (140) (S820). The electronic device (100) can operate in a low-power state (or low-power mode).

[0241] If the second threshold time has not elapsed (S815-N), the electronic device (100) can determine whether the user's proximity detection function is running (S825). If the user's proximity detection function is not running (S825-N), the electronic device (100) can repeat steps S815 and S825 until the second threshold time has elapsed. If the second threshold time has elapsed, the electronic device (100) can turn off the display (140).

[0242] If the user's proximity detection function is running (S825-Y), the electronic device (100) can determine whether an event executing a preset function is identified (S830). An event executing a preset function may include an event indicating that a user is using the electronic device (100). An event executing a preset function may include an event executing a function that presupposes the user's proximity. For example, the preset function may include at least one of a voice recognition function and a camera shooting function.

[0243] When an event in which a preset function is running is identified (S830-Y), the electronic device (100) can continue to display the preset screen and initialize the elapsed time.

[0244] If an event in which a preset function is being executed is not identified (S830-N), the electronic device (100) can identify whether a user has been detected using ultrasonic waves (S835). Step S835 may be described as a third detection operation. The third detection operation may be identical to the second detection operation. The third detection operation may correspond to steps S705, S710, S715, S720, S725, S730, S735, and S745 of FIG. 7. In order to distinguish between the audio signal obtained in the third detection operation and the audio signal obtained in the second detection operation, the second audio signal may be described as a third audio signal.

[0245] The second detection action may be an action to detect a user while the preset screen is not displayed. The third detection action may be an action to detect a user while the preset screen is displayed.

[0246] FIG. 9 is a diagram for explaining an operation of processing noise according to one embodiment.

[0247] Referring to FIG. 9, the electronic device (100) can process noise in a received audio signal. The electronic device (100) can perform various noise processing operations depending on the cause of the noise. The electronic device (100) can store noise processing information (910). The noise processing information can include information indicating the operation of the electronic device (100) to filter noise.

[0248] The electronic device (100) can filter noise using a noise threshold value. A description related to this is described in FIG. 10.

[0249] For example, when noise occurs in a normal situation, the electronic device (100) can filter the noise using a noise threshold.

[0250] For example, if noise is generated by a hardware characteristic of a microphone (180), the electronic device (100) can filter the noise using a noise threshold corresponding to the hardware characteristic. A description related to this is described in FIG. 11.

[0251] For example, if noise occurs in relation to a channel of the microphone (180), the electronic device (100) can filter out the noise using a noise threshold value corresponding to each channel of the microphone (180). The electronic device (100) can perform a user proximity detection function using only a microphone corresponding to one of the multiple channels. A description related to this is described in FIGS. 11 to 14.

[0252] For example, if noise occurs during a state transition, the electronic device (100) can filter out the noise using a state-specific noise threshold. A description related to this is provided in FIGS. 15 to 18.

[0253] FIG. 10 is a diagram for explaining an operation for processing general noise according to one embodiment.

[0254] Steps S1005, S1010, S1015, S1025, S1030, S1035, and S1040 of FIG. 10 may correspond to steps S705, S710, S715, S725, S730, S735, and S740 of FIG. 7. Duplicate explanations are omitted.

[0255] When a second audio signal is acquired, the electronic device (100) can obtain a target audio signal by filtering noise by applying a noise threshold value to the second audio signal (S1021). The electronic device (100) can filter noise from the second audio signal based on the noise threshold value. The noise threshold value may be a preset value. The noise threshold value may represent a frequency component representing noise. The electronic device (100) can obtain a target audio signal by filtering (or removing) noise from the second audio signal using the noise threshold value. The target audio signal may be a signal acquired after the noise filtering function is performed.

[0256] The electronic device (100) can identify a reflected wave corresponding to an ultrasonic wave based on a target audio signal (S1022). The electronic device (100) can identify a reflected wave corresponding to an ultrasonic wave (output) from a noise-filtered target audio signal. Thereafter, the electronic device (100) can perform steps S1025, S1030, S1035, and S1040.

[0257] FIG. 11 is a diagram for explaining an operation of processing noise according to a microphone, according to one embodiment.

[0258] Referring to FIG. 11, the electronic device (100) can process microphone noise. The electronic device (100) can store microphone noise threshold information (1110) that includes different noise thresholds for each microphone. If the microphone is implemented with multiple channels, the microphone noise threshold information (1110) can include noise thresholds corresponding to each channel.

[0259] For example, the first microphone (A) may be a single channel. The electronic device (100) may store a noise threshold value (th1) corresponding to the first microphone (A).

[0260] For example, the second microphone (B) may be a single channel. The electronic device (100) may store a noise threshold value (th2) corresponding to the second microphone (B). The noise threshold value (th1) and the noise threshold value (th2) may be the same or different.

[0261] For example, the third microphone (C) may be two-channel. The electronic device (100) may store noise threshold values ​​(th3, th4) corresponding to each channel (L, R) of the third microphone (C). The noise threshold value corresponding to the first channel (L) may be th3, and the noise threshold value corresponding to the second channel (R) may be th4. th3 and th4 may be the same or different.

[0262] For example, the fourth microphone (D) may have three channels. The electronic device (100) may store noise threshold values ​​(th5, th6, th7) corresponding to each channel (L, C, R) of the fourth microphone (D). The noise threshold value corresponding to the first channel (L) may be th5, the noise threshold value corresponding to the second channel (C) may be th6, and the noise threshold value corresponding to the third channel (R) may be th7. At least some of th5, th6, and th7 may be the same or different.

[0263] The electronic device (100) can apply different noise thresholds to each microphone (180) based on the microphone noise threshold information (1110). The electronic device (100) can obtain identification information of the microphone (180). The electronic device (100) can obtain a noise threshold corresponding to the identification information of the microphone (180) among a plurality of noise thresholds included in the microphone noise threshold information (1110). The electronic device (100) can obtain a target audio signal based on the obtained noise threshold. The electronic device (100) can determine whether a user is close based on the target audio signal.

[0264] FIG. 12 is a diagram for explaining noise and ultrasound according to one embodiment.

[0265] The electronic device (100) may include a microphone (180) having two channels (L, R).

[0266] Referring to embodiment (1210) of FIG. 12, the electronic device (100) can obtain an audio signal obtained from a microphone (L) of a first channel. The audio signal may include noise.

[0267] An electronic device (100) can extract noise from an audio signal. The electronic device (100) can obtain the magnitude of the extracted noise. The audio signal may additionally include natural sounds, ultrasound, etc. in addition to noise. The electronic device (100) can extract noise from the audio signal and identify (or calculate) the magnitude of the noise.

[0268] For example, the electronic device (100) can obtain the noise size as VL based on the audio signal obtained from the microphone (L) of the first channel.

[0269] Referring to embodiment (1220) of FIG. 12, the electronic device (100) may acquire an audio signal acquired from a microphone (R) of a second channel. The audio signal may include noise. The operation of extracting noise and calculating the noise size may be identical to embodiment (1210) of FIG. 12.

[0270] For example, the electronic device (100) can obtain the noise size in VR based on an audio signal obtained from a microphone (R) of the second channel.

[0271] The VL may be smaller than the VR. Since hardware performance may vary by channel, noise levels may vary by channel. The electronic device (100) can determine whether a user is in proximity based on the microphone (L) corresponding to a channel with a smaller noise level among multiple channels.

[0272] The audio signal may include a signal for a reflected wave corresponding to the ultrasonic wave output in step S710 of FIG. 7. The electronic device (100) may acquire a time point (t1) identified by the reflected wave. The electronic device (100) may identify a noise level based on a threshold time range (t2 to t3) based on the time point (t1) identified by the reflected wave. The operation of identifying the noise level may be performed for each of an audio signal acquired from a microphone (L) of a first channel and an audio signal acquired from a microphone (R) of a second channel.

[0273] FIG. 13 is a diagram for explaining an operation of selecting one microphone from a two-channel microphone according to one embodiment.

[0274] Referring to FIG. 13, the electronic device (100) can obtain a first sub-audio signal from a first microphone (L channel) (S1311). The electronic device (100) can extract noise from the first sub-audio signal. Based on the extracted noise, the electronic device (100) can obtain a first noise magnitude of the first sub-audio signal (S1312). The electronic device (100) can obtain a first sound pressure of the first sub-audio signal (S1313). The first sound pressure can represent an average sound pressure of the first sub-audio signal.

[0275] The electronic device (100) can obtain a second sub-audio signal from a second microphone (R channel) (S1321). The electronic device (100) can extract noise from the second sub-audio signal. The electronic device (100) can obtain a second noise magnitude of the second sub-audio signal based on the extracted noise (S1322). The electronic device (100) can obtain a second sound pressure of the second sub-audio signal (S1323). The second sound pressure can represent an average sound pressure of the second sub-audio signal.

[0276] Sound pressure (first sound pressure, second sound pressure) may include various sound components, not just noise. The noise magnitude may only indicate the magnitude of the noise component. Sound pressure may indicate the magnitude of various components included in an audio signal. In the example of Fig. 12, the sound pressure may indicate the average sound pressure of an audio signal that includes noise, natural sounds, ultrasound, and more.

[0277] The electronic device (100) can determine whether the first noise level exceeds the second noise level (S1330). If the first noise level does not exceed the second noise level (S1330-N), the electronic device (100) can perform a user proximity detection function using the first microphone (L channel) (S1335). The electronic device (100) can perform the second detection operation or the third detection operation using the first microphone (L channel).

[0278] If the first noise level exceeds the second noise level (S1330-Y), the electronic device (100) can identify whether the first sound pressure exceeds the second sound pressure (S1340). If the first sound pressure exceeds the second music (S1340-Y), the electronic device (100) can perform a user proximity detection function using the first microphone (L channel) (S1335). The electronic device (100) can perform the second detection operation or the third detection operation using the first microphone (L channel).

[0279] If the first negative pressure does not exceed the second negative pressure (S1340-N), the electronic device (100) can perform a user proximity detection function using the second microphone (R channel) (S1345). The electronic device (100) can perform the second detection operation or the third detection operation using the second microphone (R channel).

[0280] In one embodiment, the noise magnitude may be described as noise pressure.

[0281] In one embodiment, the noise magnitude may represent a frequency range or a signal-to-noise ratio.

[0282] According to one embodiment, the electronic device (100) may perform a user proximity detection function using one of the plurality of microphones based solely on noise size.

[0283] According to one embodiment, the electronic device (100) may perform a user proximity detection function using only one microphone among a plurality of microphones using sound pressure.

[0284] The electronic device (100) can perform a user proximity detection function by using a microphone that measures small noise or large sound pressure.

[0285] The embodiment of Fig. 13 can be equally applied to the testing process. The testing process is described in Figs. 24 to 29.

[0286] FIG. 14 is a diagram for explaining an operation of selecting one microphone from a two-channel microphone according to one embodiment.

[0287] Steps S1411, S1412, S1413, S1421, S1422, and S1423 of FIG. 14 may correspond to steps S1311, S1312, S1313, S1321, S1322, and S1323 of FIG. 13. Duplicate explanations are omitted.

[0288] The electronic device (100) can obtain a first weight corresponding to the noise level (S1430). The electronic device (100) can obtain a second weight corresponding to the sound pressure (S1435). The first and second weights may be different. Each weight may include a scaling factor. The scaling factor applied to each weight may be different. The smaller the noise level and the higher the sound pressure, the more accurately the audio signal can be analyzed. For example, the first scaling factor included in the first weight may be a negative number, and the second scaling factor included in the second weight may be a positive number.

[0289] The electronic device (100) can obtain a first representative value of the first microphone (L channel) by applying a first weight to the first noise size and a second weight to the first sound pressure (S1440).

[0290] The electronic device (100) can obtain a second representative value of the second microphone (R channel) by applying a first weight to the second noise size and a second weight to the second sound pressure (S1445).

[0291] The electronic device (100) can compare the first representative value and the second representative value. The electronic device (100) can determine whether the first representative value exceeds the second representative value (S1450). If the first representative value exceeds the second representative value (S1450 -Y), the electronic device (100) can perform a user proximity detection function using the first microphone (L channel) (S1455). The electronic device (100) can perform the second detection operation or the third detection operation using the first microphone (L channel).

[0292] If the first representative value does not exceed the second representative value (S1450-N), the electronic device (100) can perform a user proximity detection function using the second microphone (R channel) (S1460). The electronic device (100) can perform the second detection operation or the third detection operation using the second microphone (R channel).

[0293] FIG. 15 is a diagram for explaining a plurality of states operating in an electronic device according to one embodiment.

[0294] Referring to the embodiment (1500) of FIG. 15, the electronic device (100) can operate in one of an off state (or off mode), a low power state (or low power mode), a standby state (or standby mode), and a normal state (or normal mode).

[0295] Off mode may indicate a mode in which the power of the electronic device (100) is cut off. Off mode may be a mode in which the power required to perform the remaining functions is cut off, except for the function of receiving a command to turn on the power.

[0296] The low power mode may be a mode that performs only preset functions. The low power mode may be a mode in which power is supplied only to modules that perform preset functions among the multiple functions performed by the electronic device (100). For example, the low power mode may be a mode in which power is supplied only to a function that performs the first detection operation of FIG. 6. In the low power mode, the second detection operation and the third detection operation described in FIGS. 7 and 8 cannot be performed.

[0297] Standby mode performs more functions than low-power mode, but some functions may be limited. Secondary and tertiary sensing operations may be performed in standby mode. Standby mode may be a mode for displaying a preset screen. The electronic device (100) may supply power to the display (140) to display a preset screen in standby mode.

[0298] The normal mode may be a mode in which all functions of the electronic device (100) are performed. The electronic device (100) may supply power according to a user command or a preset control command.

[0299] In one embodiment, the first sensing operation may be performed even in standby mode.

[0300] In one embodiment, the first detection operation, the second detection operation, the third detection operation, etc. may be performed even in the normal mode.

[0301] Based on power consumption, low power mode may consume a first amount of power, standby mode may consume a second amount of power, and normal mode may utilize a third amount of power. The second amount of power may be greater than the first amount of power. The third amount of power may be greater than the second amount of power.

[0302] The electronic device (100) can change modes based on preset conditions. The mode can be changed based on preset conditions or user input. When the mode is changed, a change in power supply may occur. As the hardware configuration of the electronic device (100) to which power is supplied changes, noise may occur.

[0303] For example, the device may change from off mode to low power mode, and from low power mode to standby mode. Audio signals recorded in standby mode may contain noise.

[0304] For example, the device may change from off mode to normal mode, and from normal mode to standby mode. Audio signals recorded in standby mode may contain noise.

[0305] For example, it may change from off mode to standby mode. Audio signals recorded in standby mode may contain noise.

[0306] FIG. 16 is a diagram for explaining an operation for processing noise according to a state change, according to one embodiment.

[0307] Referring to FIG. 16, the electronic device (100) may store noise processing information (1610) according to a state (or mode) change. The noise processing information (1610) according to a state change may include a noise threshold corresponding to an event related to the state change. The noise may vary depending on the state change. The electronic device (100) may apply different noise thresholds depending on the state change process.

[0308] For example, when changing from off mode to standby mode, the electronic device (100) may determine the noise threshold as the first threshold (th1).

[0309] For example, when changing from low power mode to standby mode, the electronic device (100) may determine the noise threshold as the second threshold (th2).

[0310] For example, when changing from normal mode to standby mode, the electronic device (100) may determine the noise threshold as the third threshold (th3).

[0311] The noise generated when changing from off mode to standby mode may be greater than the noise generated when changing from low power mode to standby mode.

[0312] The noise level generated when changing from off mode to standby mode may be greater than the noise level generated when changing from normal mode to standby mode.

[0313] The second threshold value (th2) and the third threshold value (th3) may be smaller than the first threshold value (th1). The electronic device (100) may obtain a noise threshold value corresponding to a mode change based on noise processing information (1610) according to the mode change.

[0314] FIG. 17 is a diagram for explaining noise according to a state change, according to one embodiment.

[0315] Referring to embodiment (1710) of Fig. 17, noise generated when changing from an off state to a standby state is shown. In embodiment (1710), the magnitude of noise generated in the standby state may be V1.

[0316] Referring to embodiment (1720) of Fig. 17, noise generated when changing from a low power state to a standby state is shown. In embodiment (1720), the magnitude of noise generated in the standby state may be V2.

[0317] The noise size (V1) generated when changing from the off state to the standby state may be greater than the noise size (V2) generated when changing from the normal state to the standby state.

[0318] Referring to embodiment 1730 of FIG. 17, noise generated when changing from an off state to a standby state, from a standby state to a low-power state, and from a low-power state to a standby state is illustrated. Although the audio signal is acquired in the same manner in the standby state, the magnitude of the noise may be calculated differently depending on the previous state. When performing a second detection operation or a third detection operation in the standby state, the electronic device (100) may use a noise threshold value differently depending on the previous state.

[0319] FIG. 18 is a diagram for explaining an operation for processing noise according to a state change, according to one embodiment.

[0320] Steps S1810, S1815, S1821, and S1822 of FIG. 18 may correspond to steps S1010, S1015, S1021, and S1022 of FIG. 10. Duplicate explanations are omitted.

[0321] The electronic device (100) can operate in a standby state for ultrasonic output. The electronic device (100) can identify a previous state of the standby state (S1816). The electronic device (100) can store history information of the state being performed. The electronic device (100) can identify a previous state of the standby state based on history information related to state changes.

[0322] The electronic device (100) can identify a noise threshold corresponding to a previous state (S1817). The electronic device (100) can identify a state in which it was operating prior to the standby state and identify a noise threshold corresponding to the identified state. The electronic device (100) can identify a noise threshold corresponding to the previous state based on noise processing information (1610) according to a state change of FIG. 16.

[0323] Once the noise threshold is identified, the electronic device (100) can perform steps S1821 and S1822.

[0324] FIG. 19 is a diagram for explaining an operation of applying different sensitivities to each detection operation according to one embodiment.

[0325] Referring to FIG. 19, the electronic device (100) may store sensitivity processing information (1910). The sensitivity processing information may include information indicating the sensitivity with which a user will be detected in a user detection operation. The higher the sensitivity, the easier it is for the user to be detected. The lower the sensitivity, the less easily the user may be detected. The sensitivity may vary depending on the type of detection operation.

[0326] The electronic device (100) can determine whether a user is approaching based on a first sensitivity (s1) in the first detection operation. The first sensitivity (s1) may be included in an insensitive group. The first detection operation may correspond to detecting a user while the display (140) is turned off. If the sensitivity in the first detection operation is relatively high, power must be supplied to the display (140) too easily. To improve power efficiency, the electronic device (100) may utilize the first sensitivity (s1) in the first detection operation to make it difficult for the user to be detected.

[0327] The electronic device (100) can determine whether a user is approaching based on the second sensitivity (s2) in the second detection operation. The second sensitivity (s2) may be included in the insensitive group or the intermediate group. The second detection operation may correspond to detecting a user by outputting ultrasonic waves while the display (140) is turned off. The electronic device (100) can utilize the second sensitivity (s2) to prevent a preset screen from being displayed too easily on the display (140).

[0328] The electronic device (100) can determine whether a user is approaching based on the third sensitivity (s3) in the third detection operation. The third sensitivity may be included in a sensitivity group. The third detection operation may correspond to detecting a user while the display (140) is turned on. The third detection operation may correspond to detecting a user while a preset screen is displayed on the display (140). To prevent a preset screen that is already being displayed from being easily turned off, the electronic device (100) can utilize the third sensitivity (s3) to easily detect a user.

[0329] Figure 20 is a drawing for explaining sensitivity according to one embodiment.

[0330] Referring to FIG. 20, the electronic device (100) can store a sensitivity table (2010). The sensitivity table (2010) can include at least one of sound pressure sensitivity or ultrasonic sensitivity according to a classification level.

[0331] Sound pressure sensitivity may indicate a threshold sound pressure to be compared in a sound pressure comparison operation. Ultrasonic sensitivity may include the number of reflections acquired that are used to identify the proximity of a user. The electronic device (100) may identify the proximity of a user using a threshold number of reflections.

[0332] The electronic device (100) can obtain a phase difference value based on the (output) ultrasonic wave and the (received) reflected wave. The electronic device (100) can store the phase difference value in a queue. The ultrasonic sensitivity can be the size (critical number) of the queue.

[0333] For example, the first sensitivity (sensitivity rating) may include at least one of a first sound pressure sensitivity (e.g., 30 dB) or a first ultrasonic sensitivity (e.g., queue size 3).

[0334] For example, a second sensitivity (medium grade) may include at least one of a second sound pressure sensitivity (e.g., 40 dB) or a second ultrasonic sensitivity (e.g., a queue size of 5).

[0335] For example, a third sensitivity (insensitivity rating) may include at least one of a third sound pressure sensitivity (e.g., 50 dB) or a third ultrasound sensitivity (e.g., queue size 7).

[0336] The first sensitivity may be less than the second sensitivity. The second sensitivity may be less than the third sensitivity.

[0337] The first sound pressure sensitivity may be less than the second sound pressure sensitivity. The second sound pressure sensitivity may be less than the third sound pressure sensitivity.

[0338] The first ultrasonic sensitivity may be less than the second ultrasonic sensitivity. The second ultrasonic sensitivity may be less than the third ultrasonic sensitivity.

[0339] Referring to the embodiment (2020) of FIG. 20, the ultrasonic sensitivity corresponding to the first sensitivity (s1, sensitivity) may include three queue sizes.

[0340] The ultrasonic sensitivity corresponding to the second sensitivity (s2, medium) can include five queue sizes.

[0341] Ultrasonic sensitivity corresponding to the third sensitivity (s3, insensitive) can include seven queue sizes.

[0342] When the sensitivity changes from the first sensitivity to the second sensitivity in the second sensing operation (or the third sensing operation), the electronic device (100) must additionally obtain two more phase difference values.

[0343] When the sensitivity changes from the second sensitivity to the third sensitivity in the second detection operation (or the third detection operation), the electronic device (100) must additionally obtain two more phase difference values.

[0344] According to one embodiment, the electronic device (100) may utilize only one of the acoustic pressure sensitivity or the ultrasonic sensitivity depending on the type of sensing operation.

[0345] FIG. 21 is a drawing for explaining an operation of applying different sensitivities to each detection operation according to one embodiment.

[0346] Steps S2110, S2120, S2125, S2130, and S2135 of FIG. 21 may correspond to steps S410, S420, S425, S430, and S435 of FIG. 4. Duplicate explanations are omitted.

[0347] The electronic device (100) can utilize the first sensitivity when performing the first detection operation. The electronic device (100) can identify whether a user is approaching based on a sound pressure (threshold sound pressure) corresponding to the first sensitivity and a general audio signal (first audio signal) (S2110).

[0348] The electronic device (100) can utilize the second sensitivity when performing the second detection operation. The electronic device (100) can identify the proximity of the user based on the queue size (critical number) corresponding to the second sensitivity and the ultrasonic audio signal (second audio signal).

[0349] The electronic device (100) can utilize the third sensitivity when performing the third detection operation. The electronic device (100) can identify the proximity of a user based on a queue size (critical number) corresponding to the third sensitivity and an ultrasonic audio signal (the third audio signal).

[0350] The queue sizes corresponding to the second sensitivity and the queue sizes corresponding to the third sensitivity may be different.

[0351] FIG. 22 is a drawing for explaining an operation of adjusting sensitivity according to one embodiment.

[0352] Referring to FIG. 22, the electronic device (100) can perform a second sensing operation or a third sensing operation. The electronic device (100) can obtain a plurality of phase difference values ​​based on the second audio signal (or the third audio signal) (S2205). The electronic device (100) can obtain the phase difference values ​​based on the output ultrasonic waves and the received reflected waves. The electronic device (100) can identify whether the plurality of phase difference values ​​are greater than or equal to a threshold number (S2215). The threshold number can indicate the size of the queue. The threshold number can be a value corresponding to a preset sensitivity.

[0353] If the number of phase difference values ​​is less than the threshold number (S2215-N), the electronic device (100) may repeatedly perform steps S2205, S2210, and S2215. The electronic device (100) may repeatedly perform steps S2205, S2210, and S2215 until the threshold number of phase difference values ​​is stored.

[0354] If the number of phase difference values ​​is greater than or equal to a threshold number (S2215-Y), the electronic device (100) can identify whether the signs (negative or positive) of each of the plurality of phase difference values ​​are the same (S2220).

[0355] The sign of the phase difference value may vary depending on the user's approaching direction. In a situation where the user approaches the electronic device (100), the electronic device (100) may obtain a negative phase difference value. In a situation where the user moves away from the electronic device (100), the electronic device (100) may obtain a positive phase difference value. The electronic device (100) may identify the user's approaching direction based on the sign of the phase difference value.

[0356] For example, if the signs of the three phase difference values ​​stored in the queue are +, +, +, the electronic device (100) can identify that the user is moving away from the electronic device (100).

[0357] For example, if the signs of the three phase difference values ​​stored in the queue are -, -, -, the electronic device (100) can identify that the user is approaching the electronic device (100).

[0358] The electronic device (100) can identify whether the signs of multiple phase difference values ​​are the same. The electronic device (100) can identify whether the signs of all phase difference values ​​are positive (or all negative).

[0359] If the signs of the plurality of phase difference values ​​are all the same (S2220-Y), the electronic device (100) can obtain an average value of the plurality of phase difference values. The electronic device (100) can identify whether the average value of the plurality of phase difference values ​​is greater than or equal to a first threshold value (S2225). The first threshold value may be a threshold value indicating the user's approach.

[0360] If the average of the plurality of phase difference values ​​is not greater than the first threshold value (S2225-N), it can be determined whether the first threshold time has elapsed from the time the preset screen is displayed (S2245). If the first threshold time has elapsed (S2245-Y), the electronic device (100) can turn off the display (140) (S2250). The electronic device (100) can operate in a low power state (or low power mode).

[0361] If the first threshold time has not elapsed (S2245-N), the electronic device (100) can obtain a new phase difference value. The electronic device (100) can repeat steps S2205 to S2250.

[0362] If the average value of multiple phase difference values ​​is greater than or equal to the first threshold value (S2225-Y), the electronic device (100) can identify that a user has been detected (S2230). The electronic device (100) can identify that a user is present around the electronic device (100). The electronic device (100) can display a preset screen on the display (140).

[0363] If the signs of the plurality of phase difference values ​​are not all the same (S2220-N), the electronic device (100) can identify the phase difference values ​​having inconsistent signs. The electronic device (100) can identify whether the phase difference values ​​having inconsistent signs are greater than or equal to a second threshold value (S2235). If there are a plurality of phase difference values ​​having inconsistent signs, it can identify whether the average value of the plurality is greater than or equal to the second threshold value. The first threshold value and the second threshold value may be different.

[0364] For example, consider a queue storing +1, +2, -1. A phase difference value with an inconsistent sign could be -1.

[0365] For example, consider a queue storing +1, -2, -1. A phase difference value with an inconsistent sign could be +1.

[0366] If the phase difference value with an inconsistent sign is greater than or equal to the second threshold value (S2235-Y), the electronic device (100) may increase the threshold number (the threshold number of step S2215) (S2240). Step S2240 may mean an operation for reducing sensitivity. If the signs of the phase difference value are inconsistent, the reliability of the data may be considered low. The electronic device (100) may increase the reliability of the data by increasing the accumulated number of phase difference values.

[0367] As the critical number increases, the electronic device (100) can accumulate new phase difference values ​​and include them in the queue. The electronic device (100) can repeat steps S2205 to S2250.

[0368] If the phase difference value with an inconsistent sign is less than the second threshold value (S2245-Y), the electronic device (100) may turn off the display (140) (S2250). The electronic device (100) may operate in a low power state.

[0369] In one embodiment, the absolute value of the phase difference value may be used in performing steps S2225 and S2235.

[0370] FIG. 23 is a drawing for explaining negative pressure measurement data according to one embodiment.

[0371] Table (2310) of FIG. 23 may represent sound pressure measurement data. Table (2310) may represent sound pressure data displayed in various situations. The proximity of a user can be primarily determined based solely on the sound pressure in an audio signal measured through a microphone (180). Operations related to this are described in step S620 of FIG. 6. The preset sound pressure can be determined based on the sound pressure of an audio signal generated by the user. The electronic device (100) may determine the preset sound pressure based on the sound pressure measurement data described in table (2310).

[0372] For example, the electronic device (100) can determine the average sound pressure of various sound pressure measurement data related to the user as the preset sound pressure.

[0373] FIG. 24 is a diagram for explaining conditions for performing a noise test according to one embodiment.

[0374] Referring to FIG. 24, the electronic device (100) may perform a noise test. The noise test may be an operation for determining a noise threshold. The noise test may be performed automatically. For example, the electronic device (100) may perform the noise test at preset intervals. The electronic device (100) may update the noise threshold at preset intervals.

[0375] The electronic device (100) can identify whether it is currently outputting ultrasonic waves (S2405). If it is not outputting ultrasonic waves (S2405-N), the electronic device (100) can identify whether it is in an initialization state (S2410). If it is in an initialization state (S2410-Y), the electronic device (100) can perform a noise test (S2435).

[0376] If it is not in the initialization state (S2410-N), the electronic device (100) can identify whether the current state is a low-power state (S2415). If it is in the low-power state (or low-power mode) (S2415), the electronic device (100) can identify whether the current time is included in a preset time (S2420). The preset time may indicate an update cycle.

[0377] If the current time is included in the preset time (S2420-Y), the electronic device (100) can identify whether the number of accumulated stored noise thresholds is greater than or equal to a threshold number (S2425). The threshold number in step S2425 may be different from the threshold number of the queue described above (S2215 of FIG. 22). The electronic device (100) can determine the average value of the three accumulated noise thresholds as the final noise threshold. The electronic device (100) can identify the number of accumulated stored noise thresholds.

[0378] If the accumulated stored noise threshold is less than the threshold number (S2425-N), the electronic device (100) can identify whether a first time has elapsed since the most recent test time (S2430). The most recent test time may indicate the completion (or end) time of the most recently performed test.

[0379] When the first hour has elapsed from the last test end time (S2430-Y), the electronic device (100) can perform a noise test (S2435).

[0380] If the accumulated stored noise threshold value is greater than or equal to a threshold value (S2425-Y), the electronic device (100) can determine whether a second period of time has elapsed since the most recent test time (S2440). The second period of time may be longer than the first period of time. For example, the first period of time may be one day, and the second period of time may be one week. This is because if the noise threshold value has already been sufficiently accumulated, there is no need to frequently update the noise threshold value.

[0381] When the second time has elapsed from the last test time (S2440-Y), the electronic device (100) can perform a noise test (S2435).

[0382] FIG. 25 is a diagram for explaining an operation of obtaining noise information according to one embodiment.

[0383] Referring to FIG. 25, the electronic device (100) can start test recording (S2505). The electronic device (100) can start test recording without outputting ultrasonic waves.

[0384] The electronic device (100) may operate in a standby state (or standby mode) for test recording (S2510). The electronic device (100) may identify whether a preset period of time (the third time) has elapsed from the time of test recording (S2515). If the preset period of time (the third time) has elapsed from the time of test recording (S2515-Y), the electronic device (100) may terminate the test recording (S2520).

[0385] The electronic device (100) can obtain a test audio signal based on a test recording (S2525). The electronic device (100) can obtain (or extract) noise information from the test audio signal (S2530).

[0386] FIG. 26 is a diagram for explaining a process of obtaining noise information according to one embodiment.

[0387] Referring to FIG. 26, the electronic device (100) can obtain a test audio signal (2610). The test audio signal (2610) can represent the magnitude of sound pressure over time.

[0388] The electronic device (100) can convert a test audio signal (2610) into a spectrogram signal (2620). The spectrogram can show changes in frequency components over time.

[0389] The electronic device (100) can obtain a Mel spectrogram signal (2630) from a spectrogram signal (2620). The Mel spectrogram can represent auditory frequency perception characteristics over time by converting the axis of the spectrogram into a Mel scale. The Mel scale can be a scale for converting frequencies according to human auditory perception.

[0390] For example, at low frequencies, even a small change in frequency can be perceived as a significant difference. At high frequencies, a significant change in frequency is required to perceive the difference. The Mel scale can be used to quantify this perception, representing the relationship between physical changes in frequency and the human auditory experience.

[0391] In the Mel Spectogram signal (2630), the same frequency change can be represented by the same magnitude change.

[0392] FIG. 27 is a diagram for explaining noise information for multiple bands according to one embodiment.

[0393] Referring to FIG. 27, the electronic device (100) can obtain a Mel spectrogram signal (2710). The Mel spectrogram signal (2710) can correspond to the Mel spectrogram signal (2630) of FIG. 26.

[0394] The electronic device (100) can analyze noise based on a Mel Spectogram signal (2710). The Mel Spectogram signal (2710) can include signals for various frequency bands (b0, b1, b2, b3, b4, b5). Noise can be extracted based on the remaining bands (b1, b2, b3, b4, b5) excluding the ultrasonic band (b0) among the various frequency bands.

[0395] The electronic device (100) can calculate the average sound pressure (mel-SPL) of the signal existing in each of a plurality of preset bands (b0, b1, b2, b3, b4, b5). The electronic device (100) can calculate the average sound pressure as the noise size of each band.

[0396] The electronic device (100) can obtain the average sound pressure of the ultrasonic band (b0) as an ultrasonic size.

[0397] The electronic device (100) can obtain the average sound pressure of the first band (b1) as the first noise magnitude. The electronic device (100) can obtain the average sound pressure of the second band (b2) as the second noise magnitude. The electronic device (100) can obtain the average sound pressure of the third band (b3) as the third noise magnitude. The electronic device (100) can obtain the average sound pressure of the fourth band (b4) as the fourth noise magnitude. The electronic device (100) can obtain the average sound pressure of the fifth band (b5) as the fifth noise magnitude.

[0398] The electronic device (100) can identify whether an electronic device that outputs ultrasonic waves exists around the electronic device (100) by comparing the ultrasonic waves and noise waves.

[0399] FIG. 28 is a diagram for explaining an operation of analyzing noise information for multiple bands according to one embodiment.

[0400] Referring to the embodiment (2810) of FIG. 28, the electronic device (100) can compare the sizes of signals acquired from each of the multiple bands.

[0401] For example, the magnitude of the signal corresponding to each of the preset plurality of bands (b0, b1, b2, b3, b4, b5) may be (1,1,1,1,2,1). The electronic device (100) may obtain the signal magnitude (1) of the ultrasonic band (b0) as the first magnitude. The electronic device (100) may obtain the largest signal magnitude (2) among the remaining bands as the second magnitude. Since the first magnitude (1) is smaller than the second magnitude (2), the electronic device (100) may identify that there is no external electronic device outputting ultrasonic waves around the electronic device (100). The electronic device (100) may determine that the obtained noise information is highly reliable.

[0402] For example, the magnitude of the signal corresponding to each of the preset plurality of bands (b0, b1, b2, b3, b4, b5) may be (10, 1, 1, 1, 2, 1). The electronic device (100) may obtain the signal magnitude (10) of the ultrasonic band (b0) as the first magnitude. The electronic device (100) may obtain the largest signal magnitude (2) among the remaining bands as the second magnitude. Since the first magnitude (10) is greater than the second magnitude (2), the electronic device (100) may identify that an external electronic device that outputs ultrasonic waves exists around the electronic device (100). The electronic device (100) may determine that the obtained noise information has low reliability.

[0403] FIG. 29 is a diagram for explaining an operation of performing a noise test according to one embodiment.

[0404] Referring to FIG. 29, the electronic device (100) can obtain noise information (S2905). The electronic device (100) can obtain the average sound pressure of the ultrasonic band (b0) as the first magnitude (or first signal magnitude) (S2910).

[0405] The electronic device (100) can obtain the largest average sound pressure among the bands other than the ultrasonic band (b1, b2, b3, b4, b5) as the second size (or second signal size) (S2915).

[0406] The electronic device (100) can obtain a difference value obtained by subtracting the second size from the first size (S2920). The electronic device (100) can identify whether the difference value is greater than or equal to a threshold value (S2925). If the difference value is greater than or equal to the threshold value (S2925-Y), the electronic device (100) can identify that there is an external electronic device outputting ultrasonic waves in the vicinity (S2930). The electronic device (100) can re-perform a noise test (S2935). The electronic device (100) can obtain new noise information based on the re-performed noise test. The electronic device (100) can repeat steps S2905 to S2935.

[0407] If the difference value is less than the threshold value (S2925-N), the electronic device (100) can obtain a noise threshold value based on the first size of the ultrasonic band (b0) (S2940). The electronic device (100) can determine the first size as the noise threshold value.

[0408] The electronic device (100) can obtain a noise test point in time (S2945). The noise test point in time can indicate the point in time when the test is completed (or terminated).

[0409] The electronic device (100) can operate in a low power state (S2950). As the noise test is completed, the electronic device (100) can change from a standby state (or standby mode) to a low power state (or low power mode).

[0410] Figure 30 is a drawing for explanation according to one embodiment.

[0411] Referring to FIG. 30, a method for controlling an electronic device including a speaker and a microphone includes a step of obtaining a first audio signal through the microphone (S3005), a step of outputting an ultrasonic wave through the speaker when a preset sound is identified based on the first audio signal (S3010), a step of obtaining a second audio signal including a reflected wave corresponding to the ultrasonic wave obtained through the microphone (S3015), and a step of performing a preset function when a user is identified as being present around the electronic device based on the second audio signal (S3020).

[0412] The preset sound may include at least one of a human voice or an artificial sound generated by a human action.

[0413] The control method may include a step of obtaining an average sound pressure of the first audio signal and, if the average sound pressure is greater than or equal to a preset sound pressure, a step of determining whether the preset sound is identified based on the first audio signal.

[0414] The control method includes a step of identifying whether the electronic device is in an ultrasonic output disabled state when the preset sound is identified based on the first audio signal, and a step of outputting the ultrasonic sound through the speaker if the electronic device is not in the ultrasonic output disabled state, wherein the ultrasonic output disabled state may include a state in which at least one of the speaker or the microphone is activated.

[0415] The control method may include a step of acquiring the first audio signal while the electronic device is operating in a low power state, and a step of changing the low power state to a standby state if the electronic device is not in the ultrasonic output disabled state.

[0416] The control method may include a step of obtaining a target audio signal by filtering noise from the second audio signal based on a pre-stored noise threshold value, a step of identifying a reflected wave corresponding to the ultrasonic wave from the target audio signal, and a step of identifying whether the user is present around the electronic device based on the ultrasonic wave and the reflected wave.

[0417] The control method may include a step of obtaining a phase difference value based on a first phase of the ultrasonic wave and a second phase of the reflected wave, and a step of identifying that the user is present around the electronic device if the phase difference value is greater than or equal to a phase threshold value.

[0418] The step of performing the above-described preset function may include displaying a preset screen when it is identified that the user is present around the electronic device, and the preset screen may include at least one of a time UI, a weather UI, a schedule UI, an indoor environment UI, and a guide UI.

[0419] The above ultrasonic wave is a first ultrasonic wave, the reflected wave is a first reflected wave, and the control method may include a step of outputting a second ultrasonic wave through the speaker after displaying the preset screen, a step of obtaining a third audio signal including a second reflected wave corresponding to the second ultrasonic wave through the microphone, and a step of continuously displaying the preset screen when it is determined that the user is present around the electronic device based on the third audio signal.

[0420] The control method comprises the steps of obtaining a first average value of a first threshold number of phase difference values ​​corresponding to a first sensitivity from the second audio signal, a step of displaying the preset screen if the first average value is greater than or equal to the phase threshold value, a step of obtaining a second average value of a second threshold number of phase difference values ​​corresponding to a second sensitivity from the third audio signal, and a step of displaying the preset screen if the second average value is less than the phase threshold value, wherein the first threshold number is greater than the second threshold number.

[0421] The methods according to the embodiments of the present disclosure described above can be implemented in the form of an application that can be installed on an existing electronic device.

[0422] The methods according to the embodiments of the present disclosure described above can be implemented only with a software upgrade or a hardware upgrade for an existing electronic device.

[0423] The above-described embodiments of the present disclosure may also be performed through an embedded server provided in an electronic device, or an external server of at least one of the electronic device and the display device.

[0424] According to an example embodiment of the present disclosure, the above-described embodiments may be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device may include an electronic device according to the disclosed embodiments, which is a device that can call instructions stored in the storage medium and operate according to the called instructions. When the instructions are executed by a processor, the processor may directly or under the control of the processor use other components to perform a function corresponding to the instructions. The instructions may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain signals and is tangible, but does not distinguish between whether data is stored semi-permanently or temporarily in the storage medium.

[0425] According to one embodiment of the present disclosure, the method according to one embodiment described above may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0426] Each of the components (e.g., modules or programs) according to the embodiments described above may be composed of one or more entities, and some of the sub-components described above may be omitted, or other sub-components may be further included in the embodiment. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the respective components prior to integration. Operations performed by a module, program or other component according to the embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0427] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the scope of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea of ​​the present disclosure.

Claims

In electronic devices, Memory that stores instructions; speaker; Mike; and At least one processor comprising processing circuitry, The above instructions, when individually or collectively executed by the at least one processor, When a preset sound is identified based on the first audio signal acquired through the microphone, the speaker is controlled to output ultrasonic waves, An electronic device that, when it is determined that a user is present around the electronic device based on a second audio signal including a reflected wave corresponding to the ultrasonic wave acquired through the microphone, controls the electronic device to perform a function corresponding to the presence of the user. In the first paragraph, The preset sound is, An electronic device that contains artificial sounds based on human behavior. In the first paragraph, The above instructions, when individually or collectively executed by the at least one processor, Obtain the average sound pressure of the first audio signal, An electronic device that determines whether to identify the preset sound based on the first audio signal if the average sound pressure is greater than or equal to a preset sound pressure. In the first paragraph, The above instructions, when individually or collectively executed by the at least one processor, When the preset sound is identified based on the first audio signal, the electronic device identifies whether or not the ultrasonic output is disabled. If the electronic device is not in a state where the ultrasonic output is disabled, the ultrasonic waves are output through the speaker, The above ultrasonic output is not possible, An electronic device comprising at least one of the speaker or the microphone in an activated state. In paragraph 4, The above instructions, when individually or collectively executed by the at least one processor, Acquiring the first audio signal while the electronic device is operating in a low power state; An electronic device that changes the low power state to a standby state if the electronic device is not in the ultrasonic output disabled state. In the first paragraph, The above instructions, when individually or collectively executed by the at least one processor, Obtaining a target audio signal by filtering noise from the second audio signal based on a noise threshold value, An electronic device that identifies the presence of a user around the electronic device based on a reflected wave corresponding to the ultrasonic wave obtained from the ultrasonic wave and the target audio signal. In paragraph 6, The above instructions, when individually or collectively executed by the at least one processor, Obtain a phase difference value based on the first phase of the ultrasound and the second phase of the reflected wave obtained from the target audio signal, An electronic device that identifies the presence of the user around the electronic device when the phase difference value is greater than or equal to a phase threshold value. In paragraph 7, including display; The above instructions, when individually or collectively executed by the at least one processor, When the presence of the user is identified around the electronic device, the display is controlled to display a preset screen, The above preset screen is, An electronic device comprising at least one of a time UI, a weather UI, a schedule UI, an indoor environment UI, and a guide UI. In paragraph 8, The above ultrasound is the first ultrasound, The reflected wave obtained from the target audio signal is a first reflected wave, The above instructions, when individually or collectively executed by the at least one processor, After displaying the above preset screen, a second ultrasonic wave is output through the speaker, Obtaining a third audio signal including a second reflected wave corresponding to the second ultrasonic wave through the microphone, An electronic device that controls the display to continuously display the preset screen when it is determined that the user is present around the electronic device based on the third audio signal. In paragraph 9, The above instructions, when individually or collectively executed by the at least one processor, Obtaining a first average value of a first critical number of phase difference values ​​corresponding to a first sensitivity from the second audio signal, If the first average value is greater than or equal to the phase threshold value, the display is controlled to display the preset screen, Obtain a second average value of the second critical number of phase difference values ​​corresponding to the second sensitivity from the third audio signal, If the second average value is less than the phase threshold value, the display is controlled so as not to display the preset screen; An electronic device wherein the first threshold number is greater than the second threshold number. In a method for controlling an electronic device including a speaker and a microphone, A step of outputting ultrasonic waves through the speaker when a preset sound is identified based on a first audio signal acquired through the microphone; A control method comprising: a step of performing a function corresponding to the presence of a user when it is determined that a user is present around the electronic device based on a second audio signal including a reflected wave corresponding to the ultrasonic wave acquired through the microphone. In Article 11, The preset sound is, A control method comprising artificial sounds based on human behavior. In Article 11, The above control method is, A step of obtaining an average sound pressure of the first audio signal; and A control method, comprising: a step of determining whether to identify the preset sound based on the first audio signal if the average sound pressure is greater than or equal to a preset sound pressure. In Article 11, The above control method A step of identifying whether the electronic device is in an ultrasonic output disabled state when the preset sound is identified based on the first audio signal; and If the electronic device is not in a state where the ultrasonic output is disabled, a step of outputting the ultrasonic wave through the speaker is included; The above ultrasonic output is not possible, A control method comprising: at least one of the speaker or the microphone being activated. In Article 14, The above control method is, A step of acquiring the first audio signal while the electronic device is operating in a low power state; and A control method, comprising: a step of changing the low power state to a standby state if the electronic device is not in the ultrasonic output disabled state;

Images