Electronic device for providing user authentication function, and operation method and storage medium thereof

Abstract

According to one embodiment, an electronic device (101, 201) may comprise a communication circuit (190, 390), at least one processor (120, 320), and a memory (130, 330). According to one embodiment, the memory may be configured to store instructions configured to, when executed by the at least one processor, instruct the electronic device to: output guidance requesting authentication information in response to receiving an execution request for a first operation; receive, through the communication circuit, the authentication information and a sound signal acquired as a result of the manipulation of an external electronic device via a microphone of the external electronic device in response to the guidance; extract a sound feature from the sound signal; confirm whether a user manipulating the external electronic device is a registered user on the basis of the extracted sound feature; and execute the first operation on the basis of confirming the registered user and the authentication information.

Description

Electronic device for providing user authentication function, method of operation thereof and storage medium

[0001] One embodiment disclosed in this document relates to an electronic device for providing a user authentication function, a method of operating the same, and a storage medium.

[0002] A display device is a type of output device that converts electrical information into visual information and displays it to the user. This device can include not only televisions (hereinafter referred to as TVs) and computer monitors, but also various portable terminals such as smartphones and tablet PCs. Users can remotely control the display device using an external electronic device such as a remote control.

[0003] With the recent increase in various broadcast channels, including personal and cable broadcasting, in addition to terrestrial broadcasting, and the widespread adoption of OTT applications, display devices can now authenticate users by receiving a password (e.g., a PIN) via an external electronic device when purchasing content or performing actions requiring user authentication. If display devices could perform authentication that enhances security while not compromising usability when performing actions requiring user authentication, user convenience would be greatly enhanced.

[0004] According to one embodiment, the electronic device (101, 201) may include a communication circuit (190, 390), at least one processor (120, 320), and a memory (130, 330). According to one embodiment, the memory may be configured to store instructions that, when executed by the at least one processor, cause the electronic device to output a guide requesting authentication information in response to receiving a request for execution of a first operation.

[0005] According to one embodiment, the instructions may be configured to cause the electronic device to receive, through the communication circuit, an acoustic signal and the authentication information obtained according to the operation of the external electronic device through a microphone of the external electronic device in response to the guidance.

[0006] According to one embodiment, the instructions may be configured to cause the electronic device to extract acoustic features from the acoustic signal.

[0007] In one embodiment, the instructions may be configured to cause the electronic device to determine, based on the extracted acoustic features, whether a user operating the external electronic device is a registered user.

[0008] According to one embodiment, the instructions may be configured to store instructions that cause the electronic device to execute the first operation based on the authentication information and the registered user.

[0009] According to one embodiment, a method for providing a user authentication function in an electronic device may include an operation of outputting a guide requesting authentication information in response to receiving a request for execution of a first operation.

[0010] According to one embodiment, the method may include an operation of receiving, in response to the guidance, an acoustic signal and the authentication information obtained according to an operation of the external electronic device through a microphone of the external electronic device.

[0011] According to one embodiment, the method may include an operation of extracting an acoustic feature from the acoustic signal.

[0012] According to one embodiment, the method may include an operation of determining whether a user operating the external electronic device is a registered user based on the extracted acoustic features.

[0013] According to one embodiment, the method may include an operation of executing the first operation based on verifying the authentication information and the registered user.

[0014] According to one embodiment, in a non-transitory storage medium storing instructions, the instructions, when executed by at least one processor (120, 320) of an electronic device (101, 201), are configured to cause the electronic device to perform at least one operation, wherein the at least one operation may include an operation of outputting a guide requesting authentication information in response to receiving a request for execution of a first operation.

[0015] According to one embodiment, the at least one operation may include receiving, in response to the guidance, an acoustic signal and the authentication information obtained according to the operation of the external electronic device through a microphone of the external electronic device.

[0016] According to one embodiment, the at least one operation may include an operation of extracting an acoustic feature from the acoustic signal.

[0017] According to one embodiment, the at least one operation may include an operation of determining, based on the extracted acoustic features, whether a user operating the external electronic device is a registered user.

[0018] In one embodiment, the at least one action may include executing the first action based on the authentication information and verifying the registered user.

[0019] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0020] FIG. 2 is a diagram illustrating an electronic device and an external electronic device according to one embodiment.

[0021] Figure 3 is an internal block diagram of an electronic device and an external electronic device according to one embodiment.

[0022] FIG. 4 is a flowchart illustrating an operation of an electronic device for providing a user authentication function according to an embodiment.

[0023] FIG. 5 is a drawing for explaining a user authentication method according to operation of an external electronic device according to one embodiment.

[0024] FIG. 6 is a signal flow diagram transmitted and received between an external electronic device and an electronic device according to one embodiment.

[0025] FIG. 7 is a diagram for explaining the operation of an external electronic device corresponding to the output of a guide requesting input of authentication information in an electronic device according to one embodiment.

[0026] FIG. 8 is a signal flow diagram transmitted and received between an external electronic device, an electronic device, and a server according to one embodiment.

[0027] FIG. 9 is a diagram illustrating a method for controlling the operation of an electronic device without a display according to one embodiment.

[0028] Fig. 10 is a drawing for explaining an operation of extracting acoustic features according to one embodiment.

[0029] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0030] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100), according to one embodiment. Referring to FIG. 1 , in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0031] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0032] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0033] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0034] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0035] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0036] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0037] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0038] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0039] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0040] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0041] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0042] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0043] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0044] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0045] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0046] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0047] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0048] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0049] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0050] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0051] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In one embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0052] In the detailed description below, reference numerals in the drawings may be used interchangeably or omitted for components that can be easily understood through the preceding embodiments, and their detailed descriptions may also be omitted. The electronic device (101) according to one embodiment disclosed in this document may be implemented by selectively combining components of different embodiments, and components of one embodiment may be replaced by components of another embodiment. For example, it should be noted that the present invention is not limited to specific drawings or embodiments.

[0053] Before explaining the present invention, a case in which user authentication is required in a display device used in the present invention will be explained.

[0054] Display devices offer a variety of services (or content) through over-the-air broadcasting, personal broadcasting, cable broadcasting, and even OTT applications. Users of display devices can access these services by entering a password (e.g., a PIN) using an external electronic device (e.g., a remote control). By entering this password, users authenticate themselves and can view or purchase the desired service through the display device.

[0055] As described above, while users can access desired services by entering a password, this security method based on password input may not be sufficient for user authentication. For example, in situations where users must watch age-restricted channels or pay for paid services, the password input method may be vulnerable to shoulder surfing. For example, when a user enters a password using an external input device, the password may be exposed to others in the user's vicinity (e.g., children). Therefore, security could be further enhanced if the user could be additionally authenticated in cases where the password is entered by an unauthorized user.

[0056] In one embodiment, an electronic device for providing a user authentication function, an operating method thereof, and a storage medium can be provided so as to perform authentication without hindering usability while enhancing security when performing an operation requiring user authentication.

[0057] According to one embodiment, the security of an authentication means can be enhanced by performing authentication using an external electronic device that provides a microphone function in an electronic device.

[0058] According to one embodiment, in an electronic device, the user is not required to undergo a separate additional authentication procedure when entering a password through an external electronic device, thereby improving user convenience.

[0059] FIG. 2 is a diagram illustrating an electronic device and an external electronic device according to one embodiment.

[0060] Referring to FIG. 2, an electronic device (201) according to an embodiment may be a display device. In FIG. 2, a display device such as a TV is illustrated as an example of the electronic device (201), but this is merely exemplary and not limiting. The electronic device (201) may be a device implemented in various forms including a display. In addition, one embodiment may be easily implemented in a display device such as a TV, but is not limited thereto. In addition, the electronic device (201) may be fixed or mobile, and may be implemented in various forms such as a smart home appliance even if it does not have a display. For example, a smart home appliance may be a device that requires the input of authentication information for user authentication required to control the smart home appliance.

[0061] An external electronic device (204) according to an embodiment may be a device capable of transmitting a control command to an electronic device (201). Here, the control command may be a command that causes the controllable electronic device (201) to perform a specific operation. In FIG. 2, a remote control device for remotely controlling the operation of the electronic device (201) is illustrated as an example of the external electronic device (204), but this is exemplary and not limiting. For example, the external electronic device (204) may be implemented as various types of devices capable of controlling the electronic device (201) using an input interface (e.g., a key input interface, a touch input interface). For example, the external electronic device (204) may include an electronic device such as a smartphone capable of displaying an input interface provided by a remote control application.

[0062] An electronic device (201) according to an embodiment can communicate with an external electronic device (204). For example, the electronic device (201) can transmit and receive data with the external electronic device (204) using short-range communication. Short-range communication may include, but is not limited to, Bluetooth, Wi-Fi, NFC, infrared, Zigbee, etc.

[0063] According to one embodiment, as illustrated in FIG. 2, if the channel currently being displayed on the electronic device (201) is a channel with restricted viewing, input of authentication information may be required to display the restricted viewing channel. Here, the restricted viewing channel may refer to a channel that can be viewed only by those who require protection (e.g., adults) when viewing video content, excluding those who require protection (e.g., children). Those who require protection may be set by the user. For example, the user may designate a specific person as a protected person, and when operating an external electronic device (204), the user may set the external electronic device (204) to be verified and authenticated. FIG. 2 illustrates a case where a guide requesting input of authentication information is output while a channel with restricted viewing is displayed, but the present invention is not limited thereto. For example, situations in which input of authentication information is required for user authentication may vary, such as when purchasing content or downloading an application on the electronic device (201).

[0064] In one embodiment, when the authentication information is input, the user inputting the authentication information can be additionally authenticated by utilizing not only the input authentication information but also the pattern of an acoustic signal generated by the operation of an external electronic device (204), which is a means for inputting the authentication information. For example, the user can be authenticated by comparing the pattern of an acoustic signal obtained by pressing an input module (e.g., a key) of the external electronic device (204).

[0065] In one embodiment, a learning model that learns the pattern characteristics of user-specific acoustic signals that are generated when an input module (e.g., a key) of an external electronic device (204) is pressed may be utilized to authenticate a user. User authentication using such a learning model may be performed within the electronic device (201) according to the present disclosure, or may be performed via a separate server (208) (e.g., an intelligent server or cloud app) and / or system.

[0066] Figure 3 is an internal block diagram of an electronic device and an external electronic device according to one embodiment.

[0067] Referring to FIG. 3, the electronic device (201) may include at least one processor (320), memory (330), and / or communication circuit (390). According to one embodiment, the electronic device (201) may further include a display (360). The electronic device (201) of FIG. 3 may be the electronic device (101) of FIG. 1. Here, not all components illustrated in FIG. 3 are essential components of the electronic device (201), and the electronic device (201) may be implemented with more or fewer components than the components illustrated in FIG. 3.

[0068] Additionally, the external electronic device (204) may include a processor (301), an input module (302), a communication circuit (303) and / or a microphone (304).

[0069] Referring to FIG. 3, the processor (320) can recognize the presence of an external electronic device (204) and perform a communication connection with the external electronic device (204) using short-range communication (e.g., BLE communication) supported by the communication circuit (390). The processor (320) can receive a control command from the external electronic device (204) and perform operations according to the control command.

[0070] According to one embodiment, the processor (320) may determine whether an operation requiring user authentication is requested. Here, the operation requiring user authentication may include, but is not limited to, displaying a restricted channel (or displaying an adult channel), displaying adult content, and / or purchasing content or downloading an application.

[0071] When an operation requiring user authentication is requested, the processor (320) may output a prompt requesting the input of authentication information. For example, in the case of an electronic device (201) including a display (360), a prompt requesting the input of authentication information for user authentication may be displayed on the display (360). In the case of an electronic device (201) without a display (360), a prompt requesting the input of authentication information for user authentication may be output in voice form through a speaker (not shown).

[0072] The processor (320) may request activation of the microphone (304) of the external electronic device (204) to receive an acoustic signal obtained by operation of the external electronic device (204) together with authentication information. In response to the request, the processor (301) of the external electronic device (204) may change the state of the microphone (304) from a deactivated state (or off state) to an activated state (or on state).

[0073] According to one embodiment, the processor (320) may output a guide requesting input of the authentication information and simultaneously transmit a request for activating the microphone (304) to the external electronic device (204) through the communication circuit (390). In addition, the processor (320) may transmit a request for activating the microphone (304) to the external electronic device (204) through the communication circuit (390) after outputting the guide requesting input of the authentication information.

[0074] Meanwhile, in the above description, the microphone (304) of the external electronic device (204) is activated when a request to activate the microphone (304) is received from the electronic device (201), but it may also be activated according to the user's selection. For example, in order to increase the recognition rate of the acoustic signal when acquiring the acoustic signal through the microphone (304), the microphone (304) may be activated when a user input is received. The processor (301) of the external electronic device (204) may also activate the microphone (304) in response to receiving the user input through a designated voice command, a designated key input, or a dedicated hardware key (e.g., a key corresponding to a microphone indicator).

[0075] According to one embodiment, the processor (320) may receive, through the communication circuit (390), an acoustic signal corresponding to an operation sound for entering authentication information acquired by the external electronic device (204) while outputting a guide requesting input of the authentication information. For example, the processor (320) may receive, through the communication circuit (390), at least a portion of authentication information corresponding to the button press whenever at least one button of the input module (302) (e.g., a key) of the external electronic device (204) is pressed. The processor (320) may obtain, in real time, an acoustic signal according to the button press corresponding to the authentication information, together with the authentication information corresponding to the button press. In addition, the processor (320) may also receive the acoustic signal separately from receiving the authentication information corresponding to the button press.

[0076] According to one embodiment, the processor (320) may perform user authentication by using not only the authentication information entered during user authentication but also an acoustic signal according to the operation of an external electronic device (204) that inputs the authentication information.

[0077] According to one embodiment, the memory (330) can store information for communication with an external electronic device (204) and data transmitted and received. According to one embodiment, the memory (330) can store a control program for controlling the electronic device (201), a UI related to an application provided by a manufacturer or downloaded from an external source, images for providing the UI, user information, documents, databases, or related data.

[0078] According to one embodiment, the memory (330) may store instructions that, when executed, control the processor (320) to perform various operations. According to one embodiment, the memory (330) may be operatively connected to the communication circuit (390), the display (360), and the processor (320), and may output a guide requesting authentication information to authenticate a user operating an external electronic device (204), and, in response to the guide, receive an acoustic signal and the authentication information acquired according to the operation of the external electronic device (204) through the microphone (304) of the external electronic device (204) through the communication circuit (390), extract an acoustic feature from the acoustic signal, and, based on the extracted acoustic feature, determine whether the user operating the external electronic device is a registered user, and, based on the authentication information and the confirmed registered user, may store instructions that are set to execute an operation.

[0079] In one embodiment, the memory (330) may store a learning model for authenticating a user by comparing a pattern of an acoustic signal obtained by operating an external electronic device (204) with a pattern for a registered user.

[0080] In one embodiment, the processor (320) may further authenticate a user entering authentication information by utilizing a pattern of an acoustic signal generated by the operation of an external electronic device (204) when the user enters authentication information. For example, the processor (320) may authenticate a user by comparing a pattern of an acoustic signal obtained by pressing an input module (e.g., a key) of the external electronic device (204).

[0081] In one embodiment, the processor (320) can use a model learned about user-specific pattern characteristics of operating the external electronic device (204) to determine whether the user operating the external electronic device (204) is a registered user. For example, the processor (320) can use a learning model learned about pattern characteristics of user-specific acoustic signals pressing an input module (e.g., a key) of the external electronic device (204) to input authentication information.

[0082] The processor (320) can be controlled to process input data according to predefined operation rules or learning models stored in the memory (330). The predefined operation rules or learning models are characterized by being created through learning. Here, being created through learning means that a predefined operation rule or artificial intelligence model set to perform a desired characteristic (or purpose) is created by learning a basic artificial intelligence model using a plurality of learning data by a learning algorithm. This learning may be performed in the electronic device (201) according to the present disclosure itself, or may be performed through a separate server (208) and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the above-described examples.

[0083] In one embodiment, the learning model can learn user-specific pattern features using an acoustic signal acquired through a microphone (304) of an external electronic device (204) to identify whether a user entering authentication information is a registered user.

[0084] According to one embodiment, the operation of authenticating a user using a learning model may be performed through a separate server (208) and / or system. Accordingly, the server (208) may receive an acoustic signal generated by the operation of an external electronic device (204). For example, the electronic device (201) may receive an acoustic signal generated by the operation of the external electronic device (204) from the external electronic device (204) through the communication circuit (390) and provide the received acoustic signal to the server (208). The server (208) may transmit a result corresponding to whether the user is a registered user using the learning model for the acoustic signal received from the electronic device (201). The processor (320) may perform an operation corresponding to the authentication information based on the authentication information and the authentication result.

[0085] In one embodiment, when an external electronic device (204) is connected to a server (208) for communication, an acoustic signal generated by the operation of the external electronic device (204) may be transmitted to the server (208) through the communication circuit (303). For example, the external electronic device (204) may provide an acoustic signal to the server (208) and authentication information may be provided to the electronic device (201), and the authentication result from the server (208) may be transmitted to the electronic device (201).

[0086] The processor (320) or the server (208) can use a model learned about user-specific pattern characteristics of operating the external electronic device (204) to determine whether an acoustic signal generated by the operation of the external electronic device (204) is for a registered user. For example, the learning model may be an artificial intelligence model trained through learning that takes the pattern of an acoustic signal generated by the operation of the external electronic device (204) as an input value and the result of identifying whether the user is a registered user as an output value. For example, the processor (320) or the server (208) may also learn about user-specific pattern characteristics by using an acoustic signal acquired through the microphone (304) whenever an operation of the input module (302) of the external electronic device (204) occurs, even if the operation is not for inputting authentication information.

[0087] The processor (320) can authenticate a user by verifying whether the authentication information received from the external electronic device (204) matches the established authentication information. Furthermore, the processor (320) can further authenticate the user based on the authentication result obtained through a learning model in addition to the authentication information. Here, the authentication result may be based on a pattern of an acoustic signal acquired through a microphone (304) when the user inputs authentication information through an input module (e.g., a key) (302) of the external electronic device (204).

[0088] According to one embodiment, an acoustic signal generated by the operation of an external electronic device (204) may be acquired through a microphone (304) included in the external electronic device (204). For example, when a user operates the external electronic device (204), for example, when pressing a button of an input module (e.g., a key) (302), a subtle impact sound (or operation sound) may be generated. The processor (301) of the external electronic device (204) may collect an acoustic signal of such operation through the microphone (304).

[0089] For example, when the processor (320) is requested to perform an operation requiring user authentication, it may display a screen requesting the input of authentication information and request the activation of a microphone (304) to receive an acoustic signal corresponding to a sound occurring in the vicinity of the external electronic device (204).

[0090] The processor (320) may receive an acoustic signal acquired through the microphone (304) while displaying the screen through the display (360). In one embodiment, the starting point of the acoustic signal acquired through the microphone (304) may be the time when the first key of the authentication information (e.g., PIN number) is pressed, and the ending point of the acoustic signal may be the time when the last key of the authentication information (e.g., PIN number) is pressed. In one embodiment, the starting point and the ending point of the acoustic signal may be based on the time when the authentication information is input, but may not be limited thereto.

[0091] Meanwhile, during the activation of the microphone (304), not only the operation sound according to the operation of the external electronic device (204), but also voices including noise generated around the external electronic device (204) can be received through the microphone (304).

[0092] According to one embodiment, the processor (320) may identify a voice for at least a portion of an acoustic signal acquired through the microphone (304). Data used to identify the voice may be separated or extracted from the acoustic signal. For example, the processor (320) may convert the acoustic signal into digital data in units of preset intervals, amplify the digital data, and then identify whether it is a human voice. If the acoustic signal corresponds to a human voice, the processor (320) may detect the start and end points of the voice. Here, the processor (320) may identify whether the voice corresponds to the voice of a registered user in a voice recognition database stored in the memory (330). At this time, the processor (320) may perform speaker recognition using at least one feature value among the waveform, format, or pitch of the acoustic signal input from the microphone (304).

[0093] As described above, the processor (320) can detect not only the operation sound resulting from the operation of the external electronic device (204) based on the acoustic signal received through the microphone (304), but also the speaker recognition result. Accordingly, the processor (320) can utilize the speaker recognition result as well as the acoustic pattern recognition result to perform user authentication.

[0094] According to one embodiment, the processor (320) may extract acoustic features from an acoustic signal obtained by operating an external electronic device (204). To this end, the processor (320) may extract acoustic features corresponding to the acoustic signal by converting the acoustic signal into a frequency domain signal and applying a specified filter to the frequency domain signal. According to one embodiment, linear frequency ceptral coefficients (LFCC) may be used to extract acoustic features. In addition to the above, various filters (or functions) such as power spectral density (PSD) and mel-filter bank cepstrum coefficients (MFCC) may be used to extract acoustic features, and the types thereof are not limited thereto.

[0095] For example, when operating the button of the input module (302) while holding the external electronic device (204) in the hand, an adult may hold the external electronic device (204) in a form that mostly surrounds it, but a child may have a smaller area in contact with the external electronic device (204) than an adult. Therefore, the acoustic signal obtained according to the operation of the external electronic device (204) may vary depending on the area of ​​the grip. In addition, when operating the button of the input module (302), the acoustic signal obtained according to the operation of the external electronic device (204) may vary depending on the area of ​​the finger touching the button. Therefore, the acoustic signal collected through the microphone (304) according to the operation of the external electronic device (204) may have a user-specific pattern characteristic.

[0096] The extracted acoustic features can be used as input for a model trained on user-specific pattern features. In one embodiment, the training model may be trained on registered users to identify whether they are registered users. Furthermore, the training model may be trained on a user-specific basis (e.g., adults, children).

[0097] According to one embodiment, the learning model used to identify a registered user may be trained on the electronic device (201) itself, or may be trained through a separate server (208) (e.g., an intelligent server) and / or system. Furthermore, the learning model may be deployed where user authentication is required.

[0098] According to one embodiment, the electronic device (201) or server (208) may operate in a manner of fine tuning by additionally performing learning by collecting data about actual users as needed based on the learned model.

[0099] According to one embodiment, the processor (320) may determine whether user authentication is successful based on the output result of the learning model. For example, if the output result of the learning model indicates that the user is not a registered user, the processor (320) may restrict the execution of an operation by considering the input authentication information as an input by an unauthorized user even if the input authentication information matches the established authentication information. For example, in response to an input requesting the display of a restricted channel, if the user is identified as an unauthorized user (e.g., a child or a minor), the processor (320) may not display the restricted channel. On the other hand, if the output result of the learning model indicates a registered user, the processor (320) may consider the input authentication information as an input by an authorized user even if the input authentication information matches the established authentication information, and may execute the operation. For example, in response to an input requesting the display of a restricted channel, if the user is identified as an authorized user (e.g., an adult), the processor (320) may display the restricted channel on the display (360).

[0100] According to one embodiment, the electronic device (101, 201) may include a communication circuit (190, 390), at least one processor (120, 320), and a memory (130, 330). According to one embodiment, the memory may be configured to store instructions that, when executed by the at least one processor, cause the electronic device to output a guide requesting authentication information in response to receiving a request for execution of a first operation.

[0101] According to one embodiment, the instructions may be configured to cause the electronic device to receive, through the communication circuit, an acoustic signal and the authentication information obtained according to the operation of the external electronic device through a microphone of the external electronic device in response to the guidance.

[0102] According to one embodiment, the instructions may be configured to cause the electronic device to extract acoustic features from the acoustic signal.

[0103] In one embodiment, the instructions may be configured to cause the electronic device to determine, based on the extracted acoustic features, whether a user operating the external electronic device is a registered user.

[0104] According to one embodiment, the instructions may be configured to store instructions that cause the electronic device to execute the first operation based on the authentication information and the registered user.

[0105] According to one embodiment, the instructions may be configured to cause the electronic device to determine whether the user is a registered user from the extracted acoustic features using a model learned about user-specific pattern features that operate the external electronic device.

[0106] According to one embodiment, the acoustic signal obtained by operation of the external electronic device may be an acoustic signal obtained by pressing at least one button of an input module of the external electronic device.

[0107] According to one embodiment, the instructions may be configured to cause the electronic device to transmit the acoustic signal to a server through the communication circuit, receive from the server a result corresponding to whether the user is a registered user using a model learned about user-specific pattern characteristics for operating the external electronic device through the communication circuit, and execute the first operation based on the authentication information and the received result.

[0108] According to one embodiment, the instructions may be configured to cause the electronic device to convert the acoustic signal into a frequency domain signal and extract an acoustic feature corresponding to the acoustic signal by applying a specified filter to the frequency domain signal.

[0109] In one embodiment, the instructions may be configured to cause the electronic device to transmit a request for microphone activation of the external electronic device to the external electronic device through the communication circuit in response to the electronic device outputting a guide requesting the authentication information.

[0110] According to one embodiment, the instructions may be configured to receive an acoustic signal obtained by operation of the external electronic device while the electronic device outputs a guide requesting the authentication information.

[0111] According to one embodiment, the external electronic device may include a remote control device for controlling the operation of the electronic device.

[0112] In one embodiment, the instructions may be configured to restrict execution of the first operation based on the electronic device verifying that the electronic device is not the registered user.

[0113] FIG. 4 is a flowchart illustrating an operation of an electronic device for providing a user authentication function according to an embodiment. Referring to FIG. 4, the operation method may include operations 405 to 425. Each operation of the operation method of FIG. 4 may be performed by an electronic device (e.g., the electronic device 101 of FIG. 1, the electronic device 201 of FIGS. 2 and 3), or at least one processor of the electronic device (e.g., the processor 120 of FIG. 1 and the processor 320 of FIG. 3). In an embodiment, at least one of operations 405 to 425 may be omitted, the order of some operations may be changed, or other operations may be added. In order to help understand the description of FIG. 4, the description will be made with reference to FIG. 5. FIG. 5 is a diagram illustrating a user authentication method according to the operation of an external electronic device according to an embodiment.

[0114] Referring to FIG. 4, in operation 405, the electronic device (201) may output a guide requesting authentication information in response to receiving a request for execution of the first operation. For example, the guide requesting authentication information may include a request to enter a personal identification number (PIN). For example, if the electronic device (201) includes a display (360), the electronic device (201) may display the guide requesting authentication information on the display (360). For example, the request for execution of the first operation may include, but is not limited to, an operation requiring user authentication, such as displaying a restricted viewing channel (or an adult channel), displaying adult content, purchasing content, and / or downloading an application.

[0115] According to one embodiment, the electronic device (201) may transmit a request to the external electronic device (204) to activate the microphone (304) of the external electronic device (204) in response to outputting a guide requesting the authentication information. According to one embodiment, the external electronic device (204) may include a remote control device for controlling the operation of the electronic device (201).

[0116] According to one embodiment, in operation 410, in response to the guidance, the electronic device (201) may receive an acoustic signal obtained according to the operation of the external electronic device (204) and the authentication information through the microphone (304) of the external electronic device (204). According to one embodiment, the electronic device (201) may receive an acoustic signal obtained according to the operation of the external electronic device (204) while outputting the guidance requesting the authentication information. According to one embodiment, the acoustic signal obtained according to the operation of the external electronic device (204) may be an acoustic signal obtained according to the pressing of at least one button of the input module (302) of the external electronic device (204).

[0117] For example, as illustrated in FIG. 5, the external electronic device (204) can receive (500) a key operation sound through a microphone (304) and transmit an acoustic signal corresponding to the key operation sound to the electronic device (201).

[0118] According to one embodiment, in operation 415, the electronic device (201) may extract acoustic features from the acoustic signal. According to one embodiment, the electronic device (201) may extract acoustic features corresponding to the acoustic signal by converting the acoustic signal into a frequency domain signal and applying a specified filter to the frequency domain signal.

[0119] For example, as illustrated in FIG. 5, the electronic device (201) may perform an operation of extracting (505) acoustic features from an acoustic signal received from the external electronic device (204). According to one embodiment, when the acoustic signal is provided to the server (208) through the electronic device (201) from the external electronic device (204) or is provided directly to the server (208) from the external electronic device (204), the server (208) may perform an operation of extracting acoustic features.

[0120] According to one embodiment, in operation 420, the electronic device (201) may determine whether a user operating the external electronic device (204) is a registered user based on the extracted acoustic features. For example, as illustrated in FIG. 5, the electronic device (201) may perform an inference operation (510) using a learning model, and user authentication using the learning model may be performed in the electronic device (201) itself according to the present disclosure, or may be performed through a separate server (208) (e.g., an intelligent server or a cloud app) and / or system.

[0121] According to one embodiment, the operation of determining whether a user operating the external electronic device (204) is a registered user may include an operation of determining whether the user is a registered user from the extracted acoustic features using a model learned for pattern features of each user operating the external electronic device (204).

[0122] According to one embodiment, in operation 425, the electronic device (201) may execute the first operation based on the authentication information and the verification of the registered user. For example, as illustrated in FIG. 5, the electronic device (201) may perform user authentication (515) based on the results using the learning model.

[0123] According to one embodiment, the electronic device (201) may transmit the acoustic signal to the server (208) and receive a result corresponding to whether the user is a registered user by using a model learned about user-specific pattern characteristics of operating the external electronic device (204) from the server (208). The electronic device (201) may execute the first operation based on the authentication information and the received result. On the other hand, the electronic device (201) may restrict execution of the first operation based on confirmation that the user is not a registered user. For example, restricting execution of the first operation may mean restricting operations such as displaying a restricted viewing channel (or displaying an adult channel), displaying adult content, and / or purchasing content, and / or downloading an application.

[0124] FIG. 6 is a signal flow diagram transmitted and received between an external electronic device and an electronic device according to an embodiment. Each operation of the operating method of FIG. 6 may be performed by an electronic device (e.g., the electronic device 101 of FIG. 1, the electronic device 201 of FIGS. 2 and 3), an external electronic device (e.g., the external electronic device 204 of FIGS. 2 and 3), at least one processor of the electronic device (e.g., the processor 120 of FIG. 1 and the processor 320 of FIG. 3), and at least one processor of the external electronic device (e.g., the processor 301 of FIG. 3). In an embodiment, at least one of operations 605 to 645 may be omitted, the order of some operations may be changed, or other operations may be added. In order to help understand the description of FIG. 6, the description will be made with reference to FIG. 7. FIG. 7 is a diagram for explaining an operation of an external electronic device corresponding to the output of a guide requesting input of authentication information in an electronic device according to an embodiment.

[0125] Referring to FIG. 6, in operation 605, the electronic device (201) may be connected to an external electronic device (204) for communication. Thereafter, when performing an operation requiring user authentication, the electronic device (201) may, in operation 610, notify that user authentication is required in order to receive authentication information (e.g., a password or PIN number) through the external electronic device (204).

[0126] For example, as illustrated in FIG. 7, when a user wishes to purchase content, the act of purchasing the content may require user authentication. Accordingly, the electronic device (201) may display a prompt (710) requesting the user to enter a purchase authentication number for purchasing the content.

[0127] In response to the guidance regarding a situation requiring user authentication, the external electronic device (204) may activate the microphone (304) in operation 615, and may receive authentication information for user authentication from the user through the input module (302) of the external electronic device (204) in operation 620. The external electronic device (204) may obtain an acoustic signal according to the input of authentication information through the microphone (304) in operation 625. For example, as illustrated in FIG. 7, the user may input a purchase authentication number (720) using the input module (302) of the external electronic device (204).

[0128] In operation 630, the external electronic device (204) may transmit the acquired acoustic signal to the electronic device (201). In operation 635, the electronic device (201) may extract acoustic features based on the acoustic signal, and in operation 640, may perform user authentication using a learning model. In operation 645, the electronic device (201) may perform an operation or restrict an operation based on the authentication result. For example, the electronic device (201) may perform or restrict a content purchase operation based on the authentication result.

[0129] For example, as illustrated in FIG. 7, if the electronic device (201) obtains an authentication result indicating that the received purchase authentication number is identical to a preset purchase authentication number and that the sound pattern characteristic obtained by pressing the purchase authentication number corresponds to a registered user, the electronic device (201) may perform an action corresponding to a content purchase. On the other hand, if the electronic device (201) obtains an authentication result indicating that the received purchase authentication number is identical to a preset purchase authentication number but that the sound pattern characteristic obtained by pressing the purchase authentication number does not correspond to a registered user, the electronic device (201) may perform an action corresponding to a content purchase failure.

[0130] FIG. 8 is a signal flow diagram transmitted and received between an external electronic device, an electronic device, and a server according to an embodiment. Operations 805 to 825 of FIG. 8 are identical to operations 605 to 625 of FIG. 6 , and thus, detailed descriptions thereof will be omitted. To facilitate understanding of the description of FIG. 8 , reference will be made to FIG. 9 . FIG. 9 is a diagram illustrating a method for controlling the operation of an electronic device without a display according to an embodiment.

[0131] Referring to FIG. 9, the electronic device (201) may be implemented in various forms, such as a smart home appliance (801), even if it does not have a display. For example, the smart home appliance (801) may be a device that requires input of authentication information for user authentication required to control the smart home appliance. For example, when performing an operation requiring user authentication in the smart home appliance (801), the external electronic device (204) may provide a voice notification through a speaker that user authentication is required in order to input authentication information (e.g., a password or PIN number). The user may input the authentication information using the input module (302) of the external electronic device (204). The external electronic device (204) may obtain an acoustic signal according to the input of the authentication information through the microphone (304).

[0132] In one embodiment, in operation 830, the external electronic device (204) may transmit an acoustic signal acquired through the electronic device (201) to the server (208) (or another electronic device (802)). If the external electronic device (204) is connected to the server (208) in communication, the acquired acoustic signal may be transmitted to the server (208) without passing through the electronic device (201). Here, the other electronic device (802) may be an electronic device such as a user's smartphone, and any device that can perform user authentication using a learning model on behalf of the electronic device (201) may be used.

[0133] According to one embodiment, in operation 835, the server (208) may extract acoustic features based on an acoustic signal, and in operation 840, perform user authentication using a learning model. In operation 845, the server (208) may transmit the authentication result to the electronic device (201), and in operation 850, the electronic device (201) may perform an operation or restrict an operation based on the authentication result.

[0134] Fig. 10 is a drawing for explaining an operation of extracting acoustic features according to one embodiment.

[0135] Referring to FIG. 10, in response to a key input (1005) via a remote control (or remote control device) (e.g., an external electronic device (204) of FIGS. 2 and 3), the remote control may provide (1010) an acoustic signal acquired via a microphone included in the remote control to a target electronic device (e.g., an electronic device (201) of FIGS. 2 and 3, a smart home appliance (801) of FIG. 8). Here, the target electronic device may refer to a device that can be controlled by a remote control, such as a TV or a smart home appliance. The target electronic device may determine whether inference using an acoustic signal is possible. For example, if there is a learning model (e.g., a learned child protection model) learned for user-specific sound pattern features, inference using an acoustic signal may be possible.

[0136] If the target electronic device stores a learned learning model (e.g., a learned parental control model), the target electronic device can extract acoustic features from an acoustic signal (1020), perform an inference operation (1025) for user authentication using the learned learning model (e.g., a learned parental control model), and output an inference result of performing user authentication based on the acoustic features. The result of performing user authentication may include information indicating whether the user is an adult or a minor. Accordingly, if information indicating that the user is an adult is output, the target electronic device can perform an operation corresponding to a successful user authentication. On the other hand, if information indicating that the user is a minor is output, the target electronic device can perform an operation corresponding to a failure of user authentication.

[0137] On the other hand, if the target electronic device does not store a learned learning model (e.g., a learned parental control model), an acoustic signal acquired through a microphone included in the remote control may be transmitted to a server (e.g., a server (208) of FIG. 2, a server (208) of FIGS. 8 and 9) (or another electronic device (e.g., another electronic device (802) of FIGS. 8 and 9)). The server or other electronic device may extract acoustic features from the acoustic signal (1030), perform an inference operation (1035) for user authentication on the acoustic features using the learned learning model (e.g., a learned parental control model), and output the inference result. The result of performing the user authentication may include information indicating whether the user is an adult or a minor. The result of performing the user authentication may be provided to the target electronic device, and if information indicating that the user is an adult is output, the target electronic device may perform an operation corresponding to a successful user authentication. On the other hand, if information indicating that the user is a minor is output, the target electronic device may perform an operation corresponding to a failure of the user authentication.

[0138] According to one embodiment, the security of an authentication means can be enhanced by performing authentication using an external electronic device that provides a microphone function in an electronic device.

[0139] According to one embodiment, in an electronic device, when entering authentication information through an external electronic device, the user is not required to undergo a separate additional authentication procedure, so that user convenience can be improved.

[0140] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0141] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0142] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0143] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0144] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0145] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0146] According to one embodiment, in a non-transitory storage medium storing instructions, the instructions, when executed by at least one processor (120, 320) of an electronic device (101, 201), are configured to cause the electronic device to perform at least one operation, wherein the at least one operation may include an operation of outputting a guide requesting authentication information in response to receiving a request for execution of a first operation.

[0147] According to one embodiment, the at least one operation may include receiving, in response to the guidance, an acoustic signal and the authentication information obtained according to the operation of the external electronic device through a microphone of the external electronic device.

[0148] According to one embodiment, the at least one operation may include an operation of extracting an acoustic feature from the acoustic signal.

[0149] According to one embodiment, the at least one operation may include an operation of determining, based on the extracted acoustic features, whether a user operating the external electronic device is a registered user.

[0150] In one embodiment, the at least one action may include executing the first action based on the authentication information and verifying the registered user.

Claims

1. In the electronic device (101, 201), Communication circuit (190, 390); At least one processor (120, 320); and Contains memory (130, 330), The above memory, when executed by the at least one processor, causes the electronic device to: In response to receiving a request to execute the first action, output a guide requesting authentication information, In response to the above guidance, an audio signal and the authentication information obtained according to the operation of the external electronic device are received through the microphone of the external electronic device through the communication circuit, Extracting acoustic features from the above acoustic signal, Based on the extracted acoustic features, it is determined whether the user operating the external electronic device is a registered user, An electronic device storing instructions set to execute the first operation based on the authentication information and the verification of the registered user.

2. In the first paragraph, the instructions cause the electronic device to: An electronic device configured to determine whether the user is a registered user from the extracted acoustic features using a model learned for user-specific pattern features that operate the external electronic device.

3. In the first or second paragraph, the sound signal obtained by the operation of the external electronic device is An electronic device, wherein the acoustic signal is obtained by pressing at least one button of an input module of the external electronic device.

4. In any one of the first to third paragraphs, the instructions cause the electronic device to: Transmitting the sound signal to the server through the above communication circuit, Using a model learned about user-specific pattern characteristics for operating the external electronic device from the server, a result corresponding to whether the user is a registered user is received through the communication circuit, An electronic device configured to execute the first operation based on the above authentication information and the received result.

5. In any one of paragraphs 1 to 4, the instructions cause the electronic device to: Converting the above acoustic signal into a frequency domain signal, An electronic device configured to extract an acoustic feature corresponding to the acoustic signal by applying a specified filter to the frequency domain signal.

6. In any one of paragraphs 1 to 5, the instructions cause the electronic device to: An electronic device configured to transmit a request for activating a microphone of the external electronic device to the external electronic device through the communication circuit in response to outputting a guide requesting the above authentication information.

7. In any one of paragraphs 1 to 6, the instructions cause the electronic device to: An electronic device configured to receive an acoustic signal obtained by operation of the external electronic device while outputting a guide requesting the above authentication information.

8. In any one of paragraphs 1 to 7, the external electronic device, An electronic device comprising a remote control device for controlling the operation of the electronic device.

9. In any one of paragraphs 1 to 8, the instructions cause the electronic device to: An electronic device configured to restrict execution of the first action based on verification that the user is not a registered user.

10. In any one of paragraphs 1 to 9, the guidance requesting the authentication information is as follows: An electronic device that includes a personal identification number (PIN) entry request.

11. A method for providing a user authentication function in an electronic device, An action for outputting a guide requesting authentication information in response to receiving a request for execution of the first action; In response to the above guidance, an action of receiving an acoustic signal and the authentication information obtained according to the operation of the external electronic device through a microphone of the external electronic device; An operation of extracting acoustic features from the above acoustic signal; An operation of determining whether a user operating the external electronic device is a registered user based on the extracted acoustic features; and A method for providing a user authentication function, comprising an action of executing the first action based on the authentication information and the registered user.

12. In paragraph 11, the operation of checking whether the user operating the external electronic device is a registered user is as follows: A method for providing a user authentication function, comprising an operation of confirming whether the user is a registered user from the extracted acoustic features using a model learned for user-specific pattern features that operate the external electronic device.

13. In the 11th or 12th paragraph, the acoustic signal obtained by the operation of the external electronic device is A method for providing a user authentication function, wherein the acoustic signal is obtained by pressing at least one button of an input module of the external electronic device.

14. In any one of paragraphs 11 to 13, An action of transmitting the above sound signal to a server; An operation of receiving a result corresponding to whether the user is a registered user using a model learned about user-specific pattern characteristics for operating the external electronic device from the server; and A method for providing a user authentication function, comprising an operation of executing the first operation based on the authentication information and the received result.

15. In a non-transitory storage medium storing instructions, the instructions are configured to cause the electronic device (101, 201) to perform at least one operation when executed by at least one processor (120, 320), wherein the at least one operation is: An action for outputting a guide requesting authentication information in response to receiving a request for execution of the first action; In response to the above guidance, an action of receiving an acoustic signal and the authentication information obtained according to the operation of the external electronic device through a microphone of the external electronic device; An operation of extracting acoustic features from the above acoustic signal; An operation of determining whether a user operating the external electronic device is a registered user based on the extracted acoustic features; and A storage medium comprising an operation for executing the first operation based on the authentication information and the registered user.

Images