Electronic device and method for controlling electronic device

Abstract

Provided are an electronic device and a method for controlling the electronic device. The electronic device comprises: at least one motion sensor; at least one biometric sensor; a memory for storing at least computer program; and at least one processor communicatively connected to the at least one motion sensor, the at least one biometric sensor, and the memory. The processor may execute the at least one computer program, including computer-executable instructions for operating the electronic device, to: acquire a first signal via the at least one motion sensor when a request for user authentication is received; acquire a second signal via the at least one biometric sensor; acquire, on the basis of the first signal, motion information indicating the type of a motion of the user; acquire, on the basis of the second signal, biometric information indicating a biometric feature of the user according to the motion of the user; input the motion information and the biometric information to a user authentication model; and perform user authentication according to whether the type of the motion and the biometric feature match authentication information.

Description

Electronic devices and methods for controlling electronic devices

[0001] The present disclosure relates to an electronic device and a method for controlling the electronic device, and more particularly, to an electronic device capable of performing user authentication and a method for controlling the electronic device.

[0002] Recently, the increased portability and connectivity of electronic devices has led to increased security risks. For example, recent portable electronic devices, including smartphones and smartwatches, offer easy access to information stored not only on the electronic device but also on various external devices connected to it. This increases the risk of sensitive information, such as user privacy or health, being leaked.

[0003] Additionally, electronic devices have become capable of connecting to email and social media, and even performing various functions related to finance and payments, which significantly increases security risks.

[0004] In order to ensure the security of electronic devices, technologies are being developed to perform user authentication using fingerprint recognition, iris recognition, facial recognition, etc. instead of password input according to related technologies. However, it is pointed out that the limitations of existing technologies are still insufficient to sufficiently deal with techniques that threaten security, such as spoofing or phishing.

[0005] In addition, conventional technologies have limitations in that they cannot be applied to devices with limited display size, such as smart watches, or devices that are difficult to install with separate sensors, such as high-performance cameras and fingerprint recognition sensors.

[0006] The above information is provided solely as background information to aid in understanding the present disclosure. No determination has been made, nor is any assertion made, regarding whether any of the above information constitutes prior art in connection with the present disclosure.

[0007] One aspect of the present disclosure is to overcome the problems and / or disadvantages described above and to provide at least the advantages described below. Accordingly, one aspect of the present disclosure is to provide an electronic device and a method for controlling the electronic device capable of performing accurate and efficient user authentication.

[0008] Additional aspects will be disclosed in part in the following description, and in part will become apparent from the following description or may be learned by practice of the embodiments presented.

[0009] According to one aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one motion sensor, at least one biometric sensor, a memory storing one or more computer programs, and at least one processor communicatively connected to the at least one motion sensor, the at least one biometric sensor, and the memory, wherein the processor executes the one or more computer programs including computer execution instructions for operating the electronic device, so that when a request for user authentication is received, the electronic device acquires a first signal through the at least one motion sensor, acquires a second signal through the at least one biometric sensor, acquires motion information indicating a type of motion of the user based on the first signal, and acquires biometric information indicating a biometric characteristic of the user according to the motion of the user based on the second signal, inputs the motion information and the biometric information into a user authentication model, and performs the user authentication based on whether the type of motion and the biometric characteristic match the authentication information.

[0010] Here, the processor executes one or more computer programs including computer execution instructions for operating the electronic device, thereby unlocking the electronic device and updating the authentication information based on the motion information and the biometric information, if the user is identified as a registered user as a result of the user authentication.

[0011] Meanwhile, the processor executes the one or more computer programs including computer execution instructions for operating the electronic device, so that when the first signal begins to be received through the at least one motion sensor, the processor acquires the second signal through the at least one biometric sensor.

[0012] Meanwhile, the at least one biosensor comprises a bone conduction sensor, and the biometric information comprises information on acoustic characteristics transmitted from at least one of the user's bones and joints according to the user's motion.

[0013] Meanwhile, the at least one biosensor comprises a bioimpedance sensor, and the bioinformation comprises information on electrical characteristics transmitted from at least one of the user's muscles, tissues, and joints according to the user's motion.

[0014] Meanwhile, the at least one biometric sensor comprises a heartbeat sensor, and the biometric information comprises information on heartbeat characteristics transmitted from the user's heart while the user performs the motion.

[0015] Here, the processor obtains stress information indicating a stress level of the user based on the biometric information by executing the one or more computer programs including computer execution instructions for operating the electronic device, and the user authentication model performs the user authentication based on whether the stress level, the type of the motion, and the heartbeat characteristic match the authentication information.

[0016] Meanwhile, the processor performs preprocessing to remove at least one of noise and trend of the first signal and the second signal by executing the one or more computer programs including computer execution instructions for operating the electronic device, and obtains the motion information and the bio-information based on the preprocessed first signal and the preprocessed second signal.

[0017] Meanwhile, one or more sensors, including a wearable part and at least one motion sensor and at least one biometric sensor, are coupled to the wearable part.

[0018] Meanwhile, the electronic device further includes a communication unit, and the processor controls the communication unit to execute one or more computer programs including computer execution instructions for operating the electronic device, so that when the user authentication is performed and the user is identified as an unregistered user, the electronic device is locked and a message is transmitted to an external device connected to the electronic device to inform that authentication has been performed by an unregistered user.

[0019] Meanwhile, the electronic device further includes an output unit, and the processor controls the output unit to keep the electronic device locked and output a guidance message for requesting additional authentication when the user is identified as an unregistered user as a result of the user authentication by executing one or more computer programs including computer execution instructions for operating the electronic device.

[0020] According to another aspect of the present disclosure, a method for controlling an electronic device is disclosed. The method for controlling an electronic device includes, when a request for user authentication is received, the steps of: obtaining a first signal through at least one motion sensor, obtaining a second signal through the at least one biometric sensor, obtaining motion information indicating a type of motion of the user based on the first signal, obtaining biometric information indicating a biometric characteristic of the user according to the motion of the user based on the second signal, and inputting the motion information and the biometric information into a user authentication model, and performing the user authentication based on whether the type of motion and the biometric characteristic match the authentication information.

[0021] Here, if the user authentication is performed and the user is identified as a registered user, the method further includes a step of unlocking the electronic device and a step of updating the authentication information based on the motion information and the biometric information.

[0022] Meanwhile, the step of acquiring the second signal acquires the second signal through the at least one biometric sensor when the first signal begins to be received through the at least one motion sensor.

[0023] Meanwhile, the at least one biosensor comprises a bone conduction sensor, and the biometric information comprises information on acoustic characteristics transmitted from at least one of the user's bones and joints according to the user's motion.

[0024] Meanwhile, the at least one biosensor comprises a bioimpedance sensor, and the bioinformation comprises information on electrical characteristics transmitted from at least one of the user's muscles, tissues, and joints according to the user's motion.

[0025] Meanwhile, the at least one biometric sensor comprises a heartbeat sensor, and the biometric information comprises information on heartbeat characteristics transmitted from the user's heart while the user performs the motion.

[0026] Here, the control method of the electronic device further includes a step of obtaining stress information indicating a stress level of the user based on the biometric information, and the user authentication model performs the user authentication based on whether the stress level, the type of the motion, and the heartbeat characteristic match the authentication information.

[0027] Meanwhile, the control method of the electronic device further includes a step of performing preprocessing to remove at least one of noise and trend of the first signal and the second signal, and a step of obtaining the motion information and the bio-information based on the preprocessed first signal and the preprocessed second signal.

[0028] Meanwhile, the method for controlling the electronic device further includes, when the user authentication is performed and the user is identified as an unregistered user, a step of maintaining the lock of the electronic device, a step of transmitting a message to inform that authentication has been performed by an unregistered user through an external device connected to the electronic device, and a step of outputting a guidance message to request additional authentication.

[0029] According to another aspect of the present disclosure, one or more non-transitory computer-readable storage media are provided that store computer-executable instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform operations. The operations include, when a request for user authentication is received, obtaining a first signal through at least one motion sensor, obtaining a second signal through the at least one biometric sensor, obtaining motion information indicating a type of motion of the user based on the first signal, obtaining biometric information indicating a biometric characteristic of the user according to the motion of the user based on the second signal, and inputting the motion information and the biometric information into a user authentication model, and performing the user authentication based on whether the type of motion and the biometric characteristic match the authentication information.

[0030] Other aspects, advantages and salient features of the present disclosure will become apparent to those skilled in the art from the following detailed description of various embodiments of the present disclosure taken in conjunction with the accompanying drawings.

[0031] The above and other aspects, features and advantages of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0032] FIG. 1 is a block diagram briefly showing the configuration of an electronic device according to an embodiment of the present disclosure;

[0033] FIG. 2 is a block diagram showing a plurality of modules according to one embodiment of the present disclosure;

[0034] FIG. 3 is a block diagram showing a plurality of modules according to one embodiment of the present disclosure;

[0035] FIG. 4 is a block diagram showing the configuration of an electronic device according to an embodiment of the present disclosure;

[0036] FIG. 5 and FIG. 6 are flowcharts showing a control method of an electronic device according to various embodiments of the present disclosure, and

[0037] FIG. 7 is a flowchart illustrating a control method of an electronic device according to an embodiment of the present disclosure.

[0038] The same reference numbers are used throughout the drawing to represent the same elements.

[0039] The following description, with reference to the accompanying drawings, is provided to assist in a comprehensive understanding of various embodiments of the present disclosure as defined by the claims and their equivalents. While numerous specific details are included to aid this understanding, they are to be considered merely exemplary. Accordingly, those skilled in the art will recognize that various changes and modifications can be made to the various embodiments described herein without departing from the scope and spirit of the present disclosure. Furthermore, descriptions of well-known functions and structures may be omitted for clarity and conciseness.

[0040] The terms and words used in this specification and claims are not limited to their bibliographic meanings, but are used by the inventors solely to facilitate a clear and consistent understanding of the present invention. Therefore, it will be apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustrative purposes only and is not intended to limit the present disclosure, which is defined by the appended claims and their equivalents.

[0041] Unless the context clearly dictates otherwise, the singular forms "a," "an," and "the" should be understood to include plural referents. Thus, for example, reference to "a component surface" includes reference to one or more such surfaces.

[0042] In describing the present disclosure, if it is determined that a specific description of a related known function or configuration may unnecessarily obscure the gist of the present disclosure, a detailed description thereof will be omitted.

[0043] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concepts of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to further faithfully and completely convey the technical concepts of the present disclosure to those skilled in the art.

[0044] Additionally, the terminology used in this disclosure is only used to describe specific embodiments of the present disclosure and is not intended to limit the scope of the rights.

[0045] In this disclosure, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a corresponding feature (e.g., a component such as a number, function, operation, or part), and do not exclude the presence of additional features.

[0046] In this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A or / and B” can include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” can all refer to (1) including at least one A, (2) including at least one B, or (3) including both at least one A and at least one B.

[0047] The expressions “first,” “second,” “first,” or “second,” etc., used in this disclosure can describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0048] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that said component may be directly coupled to said other component, or may be coupled via another component (e.g., a third component).

[0049] On the other hand, when it is said that a component (e.g., a first component) is "directly connected" or "directly connected" to another component (e.g., a second component), it can be understood that no other component (e.g., a third component) exists between said component and said other component.

[0050] The expression "configured to" as used in the present disclosure may be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" may not necessarily mean only "specifically designed to" in terms of hardware.

[0051] Instead, in some contexts, the phrase "a device configured to" may mean that the device, in conjunction with other devices or components, is "capable of" performing A, B, and C. For example, the phrase "a processor configured (or set) to perform A, B, and C" may refer to a dedicated processor (e.g., an embedded processor) for performing those operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in memory.

[0052] In the embodiments, a 'module' or 'part' performs at least one function or operation, and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, a plurality of 'modules' or 'parts' may be integrated into at least one module and implemented as at least one processor, except for a 'module' or 'part' that needs to be implemented as a specific hardware.

[0053] Meanwhile, the various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0054] It should be recognized that the blocks and combinations of the flowcharts in each flowchart can be performed by one or more computer programs containing computer-executable instructions. The entirety of one or more computer programs may be stored in a single memory device, or the one or more computer programs may be divided into different portions stored in multiple different memory devices.

[0055] Any function or operation described herein may be processed by a single processor or a combination of processors. A single processor or a combination of processors is a circuit that performs processing and includes circuits such as an application processor (AP, e.g., a central processing unit (CPU)), a communication processor (CP, e.g., a modem), a graphical processing unit (GPU), a neural processing unit (NPU), an artificial intelligence (AI) chip, a wireless-fidelity (Wi-Fi) chip, and a Bluetooth chip. TM A chip, a global positioning system (GPS) chip, a near field communication (NFC) chip, a connection chip, a sensor controller, a touch controller, a fingerprint sensor controller, a display driver integrated circuit (IC), an audio codec (CODEC) chip, a universal serial bus (USB) controller, a camera controller, an image processing IC, a microprocessor unit (MPU), a system on chip (SoC), an IC or a similar chip.

[0056] Hereinafter, with reference to the attached drawings, an embodiment according to the present disclosure will be described so that a person having ordinary knowledge in the technical field to which the present disclosure pertains can easily carry out the present disclosure.

[0057] FIG. 1 is a block diagram schematically illustrating the configuration of an electronic device according to an embodiment of the present disclosure, and FIG. 2 is a block diagram illustrating a plurality of modules according to an embodiment of the present disclosure. The following description will be made with reference to FIG. 1 and FIG. 2 together.

[0058] Referring to FIG. 1, the "electronic device (100)" according to the present disclosure refers to a device capable of performing user authentication. In particular, the electronic device (100) may be a wearable device that can be worn on a user's body. For example, the electronic device (100) may be implemented as one of a smart watch, a smart band, smart glasses, a head-mounted display (HMD), and smart clothing. However, there is no particular limitation on the type of the electronic device (100).

[0059] "User authentication" refers to the process of identifying whether a specific user is a legitimate user of an electronic device (100). For example, user authentication may include identifying whether the user being authenticated is a registered user of the electronic device (100) or a user authorized to access applications or information provided by the electronic device (100). The term "user authentication" may be replaced with terms such as "user verification" or "user identification."

[0060] Referring to FIG. 1, an electronic device (100) may include a motion sensor (110), a biometric sensor (120), a memory (130), and a processor (140).

[0061] A 'motion sensor (110)' refers to a sensor capable of detecting a signal according to the motion of an electronic device (100). The electronic device (100) may include at least one motion sensor (110). For example, the at least one motion sensor (110) may include at least one of an accelerometer and a gyroscope, but is not limited thereto.

[0062] At least one motion sensor (110) can obtain a signal indicating the motion of the electronic device (100). For example, an accelerometer can receive a signal indicating a rate of change in the speed of the electronic device (100) that changes over time, and a gyroscope can receive a signal indicating a rotational speed and direction of the electronic device (100). Hereinafter, a signal obtained through at least one motion sensor (110) is referred to as a “first signal,” and the term “first signal” can be replaced with a term such as “motion signal.”

[0063] Meanwhile, if the electronic device (100) is implemented as a wearable device that can be worn on the user's body, the motion of the electronic device (100) may correspond to the user's motion. Therefore, the terms "user's motion" and "electronic device (100) motion" may be interchangeable hereinafter.

[0064] Hereinafter, the term 'motion' is used as a general term to refer to an action that causes a change in the position of a user or an electronic device (100) over time. Therefore, the term 'motion' may be replaced with the term 'action' or 'movement'. In a broad sense, the term 'motion' may be used to mean 'posture' meaning the static state of a user's body after a motion is performed, or 'activity' meaning a set of a series of motions.

[0065] A 'biometric sensor (120)' refers to a sensor capable of detecting signals related to a user's biometric information. The electronic device (100) may include at least one biometric sensor (120), and the at least one biometric sensor (120) may include at least one of a bone conduction sensor (121), a bioimpedance sensor (122), and a heartbeat sensor (123).

[0066] A bone conduction sensor (121) can receive a sound signal corresponding to the vibration of at least one of the user's bones and joints. The sound signal received through the bone conduction sensor (121) can represent the characteristics of a sound transmitted through at least one of the user's bones and joints.

[0067] The bone conduction sensor (121) may be replaced with an acoustic sensor or microphone, but since an acoustic sensor or microphone is generally optimized for detecting sound transmitted through the air, when an acoustic sensor or microphone is used instead of the bone conduction sensor (121), it may be preferable to use an acoustic sensor or microphone that is optimized for acquiring characteristics of sound transmitted from at least one of a bone and a joint.

[0068] A bioimpedance sensor (122) can transmit electrical signals to the user's body components and, in response to the transmitted signals, receive electrical signals representing the resistance and reactance of the user's muscles, tissues, and joints. The signals received through the bioimpedance sensor (122) can represent physiological characteristics such as the user's body fat mass, body water mass, muscle mass, and intracellular fluid content.

[0069] The heartbeat sensor (123) can receive a heartbeat signal (HBS) generated according to the contraction and relaxation of the user's heart. In addition, the heartbeat sensor (123) can also receive an electrocardiogram (ECG) signal indicating the electrical activity of the heart and a heart rate variability (HRV) signal indicating the variability of the heartbeat. The signal received through the heartbeat sensor (123) can indicate characteristics related to the user's heart health, exercise performance, stress management, etc.

[0070] In addition, the biosensor (120) may include various sensors such as a PPG sensor (Photoplethysmography Sensor) and an electrocardiogram sensor that can measure pulse and blood flow by detecting changes in light reflected from the skin surface and measure oxygen saturation and heart rate.

[0071] Hereinafter, a signal acquired through at least one biosensor (120) is referred to as a 'second signal' to distinguish it from a first signal acquired through at least one motion sensor (110), and the term 'second signal' may be replaced with a term such as 'biological signal'.

[0072] At least one instruction regarding the electronic device (100) may be stored in the memory (130). In addition, an operating system (O / S) for driving the electronic device (100) may be stored in the memory (130). In addition, various software programs or applications for operating the electronic device (100) according to various embodiments of the present disclosure may be stored in the memory (130). In addition, the memory (130) may include a semiconductor memory such as a flash memory or a magnetic storage medium such as a hard disk.

[0073] Specifically, the memory (130) may store various software modules for operating the electronic device (100) according to various embodiments of the present disclosure, and the processor (140) may control the operation of the electronic device (100) by executing the various software modules stored in the memory (130). For example, the memory (130) may be accessed by the processor (140), and data may be read / written / modified / deleted / updated, etc. by the processor (140).

[0074] Meanwhile, in the present disclosure, the term memory (130) may be used to mean memory (130), read only memory (ROM), random access memory (RAM) in the processor (140), or a memory card (e.g., micro SD (secure digital) card, memory stick) mounted in the electronic device (100).

[0075] More specifically, the memory (130) may store various information, such as information about a user authentication model, authentication information about a user, information about a plurality of modules, motion information, biometric information, stress information, information about an algorithm for processing and analyzing signals received through at least one motion sensor (110) and at least one biometric sensor (120), etc.

[0076] In addition, various information necessary within the scope of achieving the purpose of the present disclosure may be stored in the memory (130), and the information stored in the memory (130) may be updated as received from an external device or input by a user.

[0077] The processor (140) controls the overall operation of the electronic device (100). Specifically, the processor (140) is connected to the configuration of the electronic device (100) including a motion sensor (110), a biometric sensor (120), and a memory (130), and can control the overall operation of the electronic device (100) by executing at least one instruction stored in the memory (130) as described above.

[0078] The processor (140) may be implemented in various ways. For example, the processor (140) may be implemented as at least one of an application specific integrated circuit (ASIC), an embedded processor, a microprocessor, hardware control logic, a hardware finite state machine (FSM), and a digital signal processor (DSP). Meanwhile, the term "processor (140)" in the present disclosure may be used to mean a central processing unit (CPU), a graphic processing unit (GPU), and a microprocessor unit (MPU).

[0079] In particular, the processor (140) can perform user authentication based on signals received through various types of sensors. The processor (140) can perform various processing steps related to user authentication using a plurality of modules, as illustrated in FIG. 2. The plurality of modules may be comprised of hardware modules or software modules, and at least one of the modules may be implemented via a neural network.

[0080] Meanwhile, the following description assumes that all of the multiple modules are implemented on-device through the processor (140) of the electronic device (100). However, it should be understood that at least one of the multiple modules may be implemented through an external device or server.

[0081] Referring to FIG. 2, the plurality of modules may include a motion information acquisition module (141), a biometric information acquisition module (142), a user authentication module (146), and an authentication information update module (147). In addition, the biometric information acquisition module (142) may include an acoustic feature acquisition module (143), an electrical feature acquisition module (144), and a heartbeat feature acquisition module (145). Hereinafter, various embodiments of the present disclosure implemented through a processor (140) and a plurality of modules will be described.

[0082] When a request for user authentication is received, the processor (140) can obtain a first signal through at least one motion sensor (110) and a second signal through at least one biometric sensor (120).

[0083] A "request for user authentication" may be received based on a user input to initiate user authentication, or may be received based on the occurrence of a predefined event requiring user authentication. For example, a request for user authentication may be received based on a user touch input to unlock a locked electronic device (100). Additionally, a request for user authentication may be received based on the occurrence of a predefined event requiring user authentication in the electronic device (100), such as receiving a phone call, checking a message, or making a payment.

[0084] In one or more embodiments of the present disclosure, when a first signal is received through at least one motion sensor (110), the processor (140) may obtain a second signal through at least one biometric sensor (120). For example, the processor (140) may obtain the second signal through at least one biometric sensor (120) for a predetermined period of time (e.g., from the time when reception of the first signal begins to occur until reception of the first signal ends) after the first signal begins to be received through at least one motion sensor (110) according to the user's motion. In other words, the processor (140) may receive the second signal through at least one biometric sensor (120) while the user's motion is performed in order to detect a change in the user's biometric information according to the user's motion.

[0085] The first signal and the second signal can be received in analog or digital form, and the processor (140) can obtain motion information and bio-information by performing various signal processing and analysis on the received first signal and second signal as described below.

[0086] In one or more embodiments of the present disclosure, the processor (140) performs preprocessing to remove at least one of noise and / or trend of the first signal and / or the second signal (i.e., de-noising and de-trending), and, based on the preprocessed first signal and the preprocessed second signal, can obtain motion information and biometric information as described below.

[0087] For example, the processor (140) can remove noise from the first signal and the second signal by calculating a moving average of the first signal and the second signal, or by using a low-pass filter or a high-pass filter on the first signal and the second signal. In addition, a wavelet transform, as described below, can be used to remove noise from the first signal and the second signal. In addition, the processor (140) can remove trends from the first signal and / or the second signal by applying polynomial approximation, spline approximation, or empirical mode decomposition (EMD) to the first signal and the second signal, or by using regression modeling.

[0088] The processor (140) can obtain motion information indicating the type of user's motion based on the first signal. As illustrated in FIG. 2, when the first signal is obtained through the motion sensor (110), the processor (140) can input the first signal to the motion information acquisition module (141) to obtain motion information indicating the type of user's motion. The 'motion information' may be a vector or tensor indicating the type of user's motion, but is not limited thereto.

[0089] The 'motion information acquisition module (141)' refers to a module that can acquire motion information indicating the type of user's motion by analyzing the first signal acquired through the motion sensor (110).

[0090] In one or more embodiments of the present disclosure, the motion information acquisition module (141) may extract statistical features for the first signal and acquire motion information based on the extracted statistical features. For example, the motion information acquisition module (141) may acquire motion information by analyzing features such as the mean, variance, maximum, minimum, and standard deviation of the first signal by applying a statistical model, such as a Gaussian model or a sparse mode, to the first signal.

[0091] In one or more embodiments of the present disclosure, the motion information acquisition module (141) can extract frequency domain features for the first signal and acquire motion information based on the extracted frequency domain features. For example, the motion information acquisition module (141) can apply a Blackman-Harris window and a short-time Fourier transform (STFT) to the first signal to acquire a spectrogram corresponding to the first signal. The motion information acquisition module (141) can convert the acquired spectrogram into 8 bits and input the 8-bit converted spectrogram into a trained neural network model, thereby acquiring information on frequency domain features of the first signal. In addition, the motion information acquisition module (141) can acquire motion information corresponding to information on frequency domain features of the first signal.

[0092] In one or more embodiments of the present disclosure, the motion information acquisition module (141) may acquire motion information by combining statistical features for the first signal and frequency domain features for the first signal. For example, the motion information acquisition module (141) may acquire a first vector representing statistical features for the first signal and a second vector representing frequency domain features for the first signal, and may perform an operation such as concatenation, averaging, or weighted averaging of the first and second vectors, thereby acquiring a third vector comprehensively representing the statistical features for the first signal and the frequency domain features for the first signal. In addition, the motion information acquisition module (141) may input the acquired third vector into a neural network model to acquire motion information corresponding to the third vector.

[0093] Meanwhile, 'obtaining motion information indicating the type of user's motion' may include a process of identifying which type of user's motion indicated by the characteristics of the first signal corresponds to among a plurality of predefined motion types. For example, the plurality of predefined motion types may include a swing motion, a grip motion, a shake motion, and the like, and may also include various other types.

[0094] The processor (140) can obtain biometric information representing the user's biometric characteristics according to the user's motion based on the second signal. As illustrated in FIG. 2, when the second signal is obtained through the biometric sensor (120), the processor (140) inputs the second signal to the biometric information acquisition module (142) to obtain biometric information representing the user's biometric characteristics according to the user's motion. The 'biometric information' may be a vector or tensor representing the user's biometric characteristics, but is not limited thereto. The 'biometric features' refer to unique characteristics of an individual user that can identify the user, and may include the user's physical characteristics and physiological characteristics.

[0095] The 'biometric information acquisition module (142)' refers to a module that can acquire biometric information representing the user's biometric characteristics according to the user's motion by performing an analysis on a second signal acquired through a biometric sensor (120). The biometric information acquisition module (142) may include an acoustic feature acquisition module (143), an electrical feature acquisition module (144), and a heartbeat feature acquisition module (145).

[0096] In one or more embodiments of the present disclosure, when a second signal is received through the bone conduction sensor (121), the acoustic feature acquisition module (143) may apply the Blackman-Harris window and short-time Fourier transform as described above to the second signal (sound signal) to acquire a spectrogram corresponding to the first signal. Here, the acquired spectrogram may be a Mel spectrogram acquired by transforming the frequency domain by modeling human hearing characteristics. The acoustic feature acquisition module (143) may input the acquired spectrogram into a learned neural network model to acquire information on acoustic features transmitted from at least one of the user's bones and joints according to the user's motion as a type of biometric information.

[0097] In one or more embodiments of the present disclosure, when a second signal is received through the bioimpedance sensor (122), the electrical feature acquisition module (144) may apply a discrete wavelet transform (DWT) to the second signal (electrical signal) to acquire a wavelet spectrum. Here, the discrete wavelet transform refers to a technique for extracting frequency and time information of a signal by decomposing the signal in the time-frequency domain and expressing it as a wavelet coefficient and a scale coefficient. The electrical feature acquisition module (144) may convert the acquired wavelet spectrum into 8 bits and input the spectrogram converted into 8 bits into a learned neural network model, thereby acquiring information on electrical features transmitted from at least one of the user's muscles, tissues, and joints according to the user's motion as a type of biometric information.

[0098] In one or more embodiments of the present disclosure, when a second signal is received through a heartbeat sensor (123), a heartbeat feature acquisition module (145) may apply a Pan-Tompkins algorithm to the second signal (heartbeat signal) to detect R-peaks, which are main peaks of a heartbeat, segment the second signal according to the interval between the R-peaks, and normalize the segments of the segmented second signal using an L1 or L2 norm. Thereafter, the heartbeat feature acquisition module (145) may extract spectral features of the second signal that has been normalized, and may acquire a wavelet spectrum by applying a discrete wavelet transformation or a stationary wavelet transformation (SWT).

[0099] Meanwhile, the heartbeat feature acquisition module (145) can perform self-similarity feature extraction on the second signal to acquire information on the fractal dimension representing the repetitive pattern and complexity of the signal. In addition, the heartbeat feature acquisition module (145) can also acquire information on the fractal dimension for various scales or multiple components of the second signal by applying the MFA (Multifractal Analysis) technique or SSA (Singular Spectrum Analysis) technique to the information on the fractal dimension.

[0100] The heartbeat feature acquisition module (145) can acquire information on heartbeat features transmitted from the user's heart while the user performs a motion by combining information on wavelet spectra and fractal dimensions as described above. Here, the combination can be performed based on operations such as concatenation, averaging, or weighted averaging, as described above.

[0101] The processor (140) can input motion information and biometric information into a user authentication model, and perform user authentication depending on whether the type of motion and biometric characteristics match the authentication information. As illustrated in FIG. 2, when motion information is acquired through a motion information acquisition module (141) and biometric information is acquired through a biometric information acquisition module (142), the processor (140) can input the motion information and biometric information into a user authentication module (146) to perform user authentication. Here, the 'user authentication module (146)' refers to a module that performs user authentication, and may include a user authentication model including a neural network.

[0102] A "user authentication model" refers to a neural network model trained to perform user authentication. The user authentication model may be trained to identify whether input motion information and biometric information match previously constructed authentication information. Specifically, the processor (140) may combine a first vector corresponding to motion information and a second vector corresponding to biometric information to obtain a third vector, and input the obtained third vector into the user authentication model. When the third vector is input, the user authentication model may identify whether a feature indicated by the third vector matches a feature of previously constructed authentication information, and output information regarding the identification result.

[0103] First, a user authentication model can be pre-trained to recognize multiple types of pre-defined motions. The pre-trained user authentication model can be transmitted to an electronic device (100) or distributed to a user by allowing the user to access a server providing the user authentication model, after which a user registration process can be performed.

[0104] When a user registration process is initiated, the user may perform at least one motion (e.g., a shaking motion of shaking the electronic device (100)) among a plurality of predefined types of motions. When at least one motion is performed, the processor (140) may obtain a first signal through at least one motion sensor (110) and a second signal through at least one biometric sensor (120) while each of the at least one motions is performed. The processor (140) may obtain motion information indicating a type of the user's motion based on the first signal, and may obtain biometric information indicating a biometric characteristic of the user according to the user's motion based on the second signal.

[0105] Thereafter, the processor (140) may train a user authentication model to generate authentication information by matching each of a plurality of types of motions with corresponding biometric information. For example, a few-shot calibration may be performed on the user authentication model to match the biometric information of an individual user. Accordingly, the “authentication information” may include information on the type of motion performed by the user during the user authentication process among a plurality of predefined types of motions and the biometric characteristics of the user corresponding thereto. The term “authentication information” may be replaced with terms such as “user authentication data” or “user authentication template.”

[0106] Meanwhile, the phrase "matching" the type of motion and biometric characteristics with the authentication information may include not only cases where the type of motion and biometric characteristics "match" with the authentication information, but also cases where the type of motion and biometric characteristics are "similar" to the authentication information. The expression "matching" the type of motion and biometric characteristics with the authentication information may be replaced with expressions such as "corresponding" the type of motion and biometric characteristics with the authentication information.

[0107] In one or more embodiments of the present disclosure, the user authentication module (146) may compare the type of motion identified based on the first signal and the biometric characteristic identified based on the second signal with the type of motion and biometric characteristic included in the authentication information, and obtain a score based on the comparison result. If the obtained score is equal to or greater than a preset threshold value, the user authentication module (146) may identify the user as a registered user and output a result value indicating authentication success. On the other hand, if the obtained score is less than the preset threshold value, the user authentication module (146) may identify the user as an unregistered user and output a result value indicating authentication failure.

[0108] If the user authentication is performed and the user is identified as a registered user (i.e., user authentication is successful), the processor (140) may unlock the electronic device (100). When the electronic device (100) is unlocked, the processor (140) may allow the user to access information stored in the electronic device (100), such as schedules, text messages, contacts, etc., or use applications, such as phone applications, payment applications, financial applications, etc. It goes without saying that the applications that can be accessed when the electronic device (100) is unlocked and the functions of the electronic device (100) may change depending on the settings of the developer or the user.

[0109] If the user authentication is performed and the user is identified as an unregistered user (i.e., user authentication fails), the processor (140) may lock the electronic device (100) and request additional authentication from the user. In addition, if the user authentication fails a threshold number of times in succession, the processor (140) may delete at least some data stored in the electronic device (100) or initialize the electronic device (100).

[0110] Meanwhile, if the score according to user authentication is less than the first threshold value that serves as the criterion for authentication success and greater than or equal to the second threshold value that serves as the criterion for authentication failure, the processor (140) may request re-authentication from the user. For example, the processor (140) may provide a message and / or user interface for requesting re-authentication, such as authentication through an external device connected to the electronic device (100), authentication through the user's contact information such as an email address, or authentication using other biometric information not used in the authentication process.

[0111] Meanwhile, as illustrated in FIG. 2, when user authentication is completed through the user authentication module (146), the processor (140) can update authentication information through the authentication information update module (147). The 'authentication information update module (147)' refers to a module that can update authentication information using motion information and biometric information used for user authentication.

[0112] In one or more embodiments of the present disclosure, if the user is identified as a registered user as a result of user authentication, the processor (140) may update authentication information based on motion information and biometric information. In other words, if the motion information and biometric information used for user authentication are identified as reliable information, the processor (140) may update (or calibrate) the authentication information using the motion information and biometric information used for user authentication.

[0113] On the other hand, if the user authentication is performed and the user is identified as an unregistered user, the processor (140) may maintain the lock of the electronic device (100) and may not update the authentication information using the motion information and biometric information used for user authentication.

[0114] According to the various embodiments described above with reference to FIGS. 1 and 2, when a motion for user authentication is performed, the electronic device (100) can perform user authentication based on the type of motion and the user's biometric characteristics according to the motion.

[0115] Furthermore, since the electronic device (100) performs user authentication by combining the user's motion type and the user's biometric characteristics, the accuracy of user authentication can be improved. For example, the electronic device (100) can perform highly accurate and convenient multi-modal user authentication by combining the characteristics of bone conduction signals, bioimpedance signals, and heartbeat signals, which are highly unique to each user, with the user's motions performed in real time.

[0116] In addition, the electronic device (100) does not require special sensors such as fingerprint recognition sensors, high-performance cameras, etc., and can efficiently perform user authentication without consuming a large amount of power because it collects various information through sensors after the user's motion begins.

[0117] In addition, since the electronic device (100) performs user authentication by identifying the user's motion and the user's biometric presence (i.e., liveness checking), high security can be secured, and especially when the electronic device (100) is implemented as a wearable device (e.g., a smart watch, etc.), the electronic device (100) can be easily utilized as an authentication means for other devices of the user (e.g., a laptop, a vehicle, a digital door lock, etc.) or various services requiring user authentication (e.g., a payment service, a financial service, etc.).

[0118] Furthermore, if the electronic device (100) identifies that the motion information and biometric information used for user authentication are reliable information, the reliability of user authentication can be further improved each time the number of user authentications increases by updating the authentication information using the motion information and biometric information used for user authentication.

[0119] FIG. 3 is a block diagram illustrating a plurality of modules according to one embodiment of the present disclosure.

[0120] Referring to FIG. 3, a plurality of modules according to the present disclosure may include a noise adjustment module (148), a motion information acquisition module (141), a heartbeat feature acquisition module (145), a stress information acquisition module (149), a user authentication module (146), and an authentication information update module (147). In addition, the processor (140) may perform various processing steps related to user authentication using the plurality of modules as illustrated in FIG. 3. The plurality of modules may be composed of hardware modules or software modules, and at least one of the plurality of modules may be implemented through a neural network.

[0121] As described above, when a first signal is acquired through the motion sensor (110), the processor (140) can input the first signal to the motion information acquisition module (141) to acquire motion information indicating the type of the user's motion. Specifically, the motion information acquisition module (141) can identify which type of motion the user corresponds to among a plurality of predefined motion types based on the first signal received through at least one motion sensor (110).

[0122] In particular, in one or more embodiments that perform user authentication using a heartbeat signal, the term 'motion' may be used to mean 'activity' that refers to a set of a series of motions. For example, the user's motion may include activities such as 'walking', 'running', 'sitting', and 'sleeping'. For example, the motion information acquisition module (141) may identify, based on the first signal, whether the user's motion corresponds to any activity among activities such as 'walking', 'running', 'sitting', and 'sleeping'.

[0123] Meanwhile, when a heartbeat signal is acquired through a heartbeat sensor (123) included in a biosensor (120), the processor (140) inputs the acquired heartbeat signal into a noise adjustment module (148) to adjust noise included in the heartbeat signal.

[0124] The noise adjustment module (148) refers to a module capable of adjusting noise included in a heartbeat signal. Specifically, when a request for noise adjustment is received, the noise adjustment module (148) can update at least one parameter related to the noise of the heartbeat signal so as to accurately extract heartbeat characteristics. The noise adjustment module (148) may include at least one of a frequency parameter update module (148-1), a statistical parameter update module (148-2), and a trend parameter update module (148-3). In the description of the present disclosure, "adjusting" noise may include "removing" and "changing" noise of the heartbeat signal.

[0125] The frequency parameter update module (148-1) refers to a module capable of updating frequency parameters related to noise in a heartbeat signal. For example, operations performed by the frequency parameter update module (148-1) may be performed in the following order.

[0126] The frequency parameter update module (148-1) can stabilize the heartbeat signal by removing high-frequency components of the heartbeat signal. In other words, the frequency parameter update module (148-1) can minimize irregular fluctuations and emphasize the main characteristics of the heartbeat signal by removing high-frequency components that indicate rapid changes in the heartbeat signal and thereby removing signal trends.

[0127] The frequency parameter update module (148-1) performs a short-term Fourier transform on a heartbeat signal from which high-frequency components have been removed, thereby expressing the heartbeat signal expressed as time-series data in a spectral form that can be analyzed in the frequency domain.

[0128] The frequency parameter update module (148-1) can remove frequency components other than the band of interest from a heartbeat signal expressed in spectrum form by performing bandpass filtering.

[0129] After bandpass filtering is performed, the frequency parameter update module (148-1) can estimate the power spectral density (PSD), which represents the energy distribution of the signal for each sub-band. Furthermore, the frequency parameter update module (148-1) can adjust the noise of the heartbeat signal by updating the frequency parameters based on the power frequency density.

[0130] The statistical parameter update module (148-2) refers to a module capable of updating statistical parameters related to noise in a heartbeat signal. For example, the operations performed by the statistical parameter update module (148-2) may be performed in the following order.

[0131] The statistical parameter update module (148-2) can stabilize the heartbeat signal by removing low-frequency components of the heartbeat signal. In other words, the statistical parameter update module (148-2) can remove low-frequency components representing slow changes in the heartbeat signal, thereby removing impulses from the signal, thereby smoothing the heartbeat signal and reducing noise.

[0132] The statistical parameter update module (148-2) can apply an ARMA (autoregressive moving average model) model to a heartbeat signal with low-frequency components removed. The ARMA model is a type of statistical model used to analyze and predict time-series data, and refers to a technique for explaining the characteristics of time-series data by combining autoregressive parameters and moving average parameters.

[0133] Specifically, the statistical parameter update module (148-2) can estimate autoregressive parameters representing the relationship between the data of the current time and the data of the previous time using a technique such as the autoregressive parameter least squares method. In addition, the statistical parameter update module (148-2) can estimate moving average parameters representing the relationship between the data of the current time and the data of the previous time using a technique such as the least squares method or the maximum likelihood method. The statistical parameter update module (148-2) can adjust the noise of the heartbeat signal by updating the autoregressive parameters and the moving average parameters.

[0134] The trend parameter update module (148-3) refers to a module capable of updating trend parameters related to noise in a heartbeat signal. Here, the trend parameter refers to a parameter indicating the trend, or tendency, of the heartbeat signal, and may be replaced with terms such as "trend parameter." For example, operations performed by the trend parameter update module (148-3) may be performed in the following order.

[0135] The trend parameter update module (148-3) can decompose a heartbeat signal into N trend components and residuals using the empirical mode decomposition (EMD) technique. Here, EMD refers to a technique for decomposing a heartbeat signal expressed as time series data into trend components of various frequency bands.

[0136] The trend parameter update module (148-3) can separate the trend component from the heartbeat signal by removing N trend components estimated through the EMD technique from the heartbeat signal.

[0137] The trend parameter update module (148-3) can identify whether the residual, which is the remainder after the trend component is separated from the heartbeat signal, resembles white noise. Specifically, the trend parameter update module (148-3) can identify whether the residual resembles white noise using a standard statistical test technique, such as the Kolmogorov-Smirnov test.

[0138] If the residuals do not resemble white noise, the trend parameter update module (148-3) can increase the number of modes N and apply N-level EMD again to decompose the heartbeat signal into N+1 trend components. For example, if the residuals do not resemble white noise, the trend parameter update module (148-3) can repeatedly decompose the trend components to a desired level.

[0139] On the other hand, if the residual is similar to white noise, the trend parameter update module (148-3) can adjust the noise of the heartbeat signal by estimating intrinsic mode functions (IMFs), which are functions representing a specific trend corresponding to a specific frequency band of the signal, and modeling the statistical characteristics of the residual based on the estimated IMFs and updating the trend parameters.

[0140] As described above, when the noise included in the heartbeat signal is adjusted, the processor (140) inputs the noise-adjusted heartbeat signal into the heartbeat feature acquisition module (145) and the stress information acquisition module (149), thereby obtaining information on the heartbeat feature and stress information. However, according to an embodiment of the present disclosure, the heartbeat signal may be input into the heartbeat feature acquisition module (145) and the stress information acquisition module (149), without performing a noise adjustment process on the heartbeat signal, thereby obtaining information on the heartbeat feature and stress information.

[0141] The heartbeat feature acquisition module (145) refers to a module capable of acquiring information about heartbeat features transmitted from the user's heart while the user performs a motion. The process of acquiring information about heartbeat features has been described with reference to FIGS. 1 and 2 , so a duplicate description of the same content will be omitted.

[0142] The stress information acquisition module (149) refers to a module capable of acquiring stress information based on a heartbeat signal. Here, "stress information" collectively refers to information indicating the user's stress level. In particular, the stress information may indicate the user's stress level based on the user's motion. For example, the operations performed by the stress information acquisition module (149) may be performed in the following order.

[0143] The stress information acquisition module (149) may perform preprocessing to remove at least one of noise and / or trends contained in the heartbeat signal. The preprocessing process for removing noise and trends has been described above in the description of the preprocessing processes for the first and second signals, and therefore, a duplicate description of the same content will be omitted.

[0144] The stress information acquisition module (149) can detect R-peaks using an electrocardiogram processing algorithm such as the Pan-Tompkins algorithm after performing a preprocessing process. Here, the R-peak is a signal representing a major event of a heartbeat in a heartbeat signal and can serve as a reference point for the heartbeat signal.

[0145] The stress information acquisition module (149) can estimate the variability of heartbeats by calculating the intervals between R-peaks (i.e., R-to-R durations). For example, the stress information acquisition module (149) can estimate heart rate variability (HRV), which indicates the variability of heartbeat intervals depending on the user's posture, activity, and emotions.

[0146] The stress information acquisition module (149) can identify the user's stress level corresponding to the heartbeat signal based on the estimated heartbeat variability. Specifically, the stress information acquisition module (149) can acquire stress information indicating the user's stress level by utilizing a regression model that predicts the stress level.

[0147] As described above, when motion information, information on heartbeat characteristics, and stress information are acquired, the processor (140) inputs the motion information, information on heartbeat characteristics, and stress information into the user authentication module (146), and performs user authentication based on whether the type of motion, heartbeat characteristics, and stress level match the authentication information.

[0148] The user authentication module (146) refers to a module that performs user authentication as described above, and may include a user authentication model including a neural network. However, in the description of FIGS. 1 and 2, embodiments in which the user authentication module (146) performs user authentication based on motion information and biometric information were described, but the user authentication module (146) may also perform user authentication using motion information, information on heartbeat characteristics, and stress information.

[0149] The user authentication module (146) can compare the type of motion identified based on the first signal, the stress level identified based on the second signal (specifically, the heartbeat signal), and the heartbeat characteristic identified based on the heartbeat signal with the type of motion, the stress level, and the heartbeat characteristic included in the authentication information, and obtain a score based on the comparison result. Then, if the obtained score is equal to or greater than a preset threshold value, the user authentication module (146) can identify the user as a registered user and output a result value indicating authentication success. On the other hand, if the obtained score is less than the preset threshold value, the user authentication module (146) can identify the user as an unregistered user and output a result value indicating authentication failure.

[0150] The matching process between pieces of information performed in the user authentication module (146) may be performed by converting each piece of information into a vector and determining whether the characteristics represented by each vector are similar. In addition, the user authentication module (146) may obtain a first vector corresponding to a type of motion identified based on a first signal, a second vector corresponding to a stress level identified based on a second signal, and a third vector corresponding to a heartbeat characteristic identified based on the second signal, and may perform the matching process by combining the first vector, the second vector, and the third vector to obtain a fourth vector, and comparing the fourth vector with a vector stored in the authentication information.

[0151] In one or more embodiments of the present disclosure, the processor (140) may utilize a dynamic time warping (DTW) technique in the process of comparing a heartbeat signal received through a heartbeat sensor (123) with a heartbeat signal included in authentication information. Here, the dynamic time warping technique refers to a technique that compensates for a temporal difference between two time series data and aligns and compares similar heartbeat patterns. Specifically, the dynamic time warping technique may include a process of calculating a distance matrix between two heartbeat signals to be compared, calculating an optimal alignment path based on the distance matrix, and aligning the two heartbeat signals according to the calculated optimal alignment path.

[0152] Meanwhile, as illustrated in FIG. 3, when user authentication is completed through the user authentication module (146), the processor (140) can update authentication information through the authentication information update module (147). As described above, the authentication information update module (147) refers to a module that can update authentication information using motion information and biometric information used for user authentication.

[0153] If the user is identified as a registered user as a result of user authentication, the processor (140) may update authentication information based on motion information, information about heartbeat characteristics, and stress information. In other words, if the motion information, information about heartbeat characteristics, and stress information used for user authentication are identified as reliable information, the processor (140) may update authentication information using the motion information, information about heartbeat characteristics, and stress information used for user authentication.

[0154] On the other hand, if the user authentication is performed and the user is identified as an unregistered user, the processor (140) may maintain the lock of the electronic device (100) and may not update the authentication information using the motion information, information about heartbeat characteristics, and stress information used for user authentication.

[0155] Meanwhile, updating of authentication information through the authentication information update module (147) can be repeatedly performed whenever user authentication is performed, and furthermore, collection of signals through the motion sensor (110) and biometric sensor (120) can be repeated at preset intervals. Here, collecting signals at preset intervals means that signals can be collected even when user authentication is not performed. In particular, by periodically acquiring heartbeat signals and updating authentication information, authentication information can be effectively personalized to suit the unique biometric characteristics of each user.

[0156] Meanwhile, although the description of FIG. 3 described an embodiment of acquiring motion information based on a first signal acquired through a motion sensor (110), the present disclosure is not limited thereto. For example, the processor (140) may identify the type of user's motion by using not only the first signal but also information about heartbeat characteristics and stress information. For example, the processor (140) may identify that the user's motion corresponds to 'sleeping' based on whether the first signal and heartbeat characteristics are similar to characteristics indicating the activity of 'sleeping' and whether the user's stress level is below a threshold level.

[0157] Meanwhile, in FIG. 3, only the heartbeat sensor (123) among the biosensors (120) is illustrated, but the bone conduction signal and bioimpedance signal obtained through the bone conduction sensor (121) and the bioimpedance sensor (122) may be used in the process of identifying the type of motion of the user or in the process of identifying the user's stress level.

[0158] According to the various embodiments described above with reference to FIG. 3, the electronic device (100) performs user authentication by combining a heartbeat signal with high uniqueness for the user with the motion of the user with high real-time accuracy, thereby performing user authentication conveniently, transparently, and with high accuracy even under various external conditions, such as when the user is exercising or in a dark environment.

[0159] FIG. 4 is a block diagram showing the configuration of an electronic device according to an embodiment of the present disclosure.

[0160] Referring to FIG. 4, the electronic device (100) may further include a motion sensor (110), a biometric sensor (120), a memory (130), and a processor (140), as well as a wearable unit (150), a communication unit (160), an input unit (170), and an output unit (180). However, in carrying out the present disclosure, new components may be added or some components may be omitted in addition to the components illustrated in FIGS. 1 and 4.

[0161] The wearable part (150) refers to a configuration that allows a user to wear the electronic device (100). For example, if the electronic device (100) is implemented as a smart watch, the wearable part (150) may include a watch strap (strap or band) and a buckle, etc., for fixing the smart watch to the user's wrist.

[0162] When the electronic device (100) is implemented as a smart glass, the wearable part (150) may include temples and nose pads, etc. In addition, any component of the electronic device (100), such as the back of a smart watch, that may come into contact with at least some of the user's body parts when the user wears the electronic device (100), may correspond to the wearable part (150) according to the present disclosure.

[0163] In one or more embodiments of the present disclosure, at least one sensor included in at least one motion sensor (110) and at least one biometric sensor (120) may be coupled to a wearable part (150). For example, when the electronic device (100) is implemented as a smart watch, a bone conduction sensor (121), a bioimpedance sensor, a heart rate sensor (123), etc. may be coupled to the back of the watch and / or the watch strap so as to be in contact with the user's skin. For example, the wearable part (150) may be a smart strap developed to expand the function of a smart watch or provide additional functions.

[0164] The communication unit (160) includes a circuit and can perform communication with an external device. Specifically, the processor (140) can receive various data or information from an external device connected via the communication unit (160) and can also transmit various data or information to the external device.

[0165] The communication unit (160) may include at least one of a WiFi module, a Bluetooth module, a wireless communication module, an NFC module, and a UWB (Ultra Wide Band) module. Specifically, the WiFi module and the Bluetooth module may each perform communication in the WiFi or Bluetooth manner. When using a WiFi module or a Bluetooth module, various connection information, such as an SSID (service set identifier), may be first transmitted and received, and then communication may be established using this, after which various pieces of information may be transmitted and received.

[0166] In addition, the wireless communication module can perform communication according to various communication standards such as IEEE (Institute of Electrical and Electronics Engineers), Zigbee, 3G (3rd generation), 3GPP (3rd generation partnership project), LTE (long term evolution), 5G (5th generation), etc. In addition, the NFC module can perform communication in the NFC (Near Field Communication) method using the 13.56MHz band among various RF-ID (radio frequency identification) frequency bands such as 135kHz, 13.56MHz, 433MHz, 860~960MHz, 2.45GHz, etc. In addition, the UWB module can accurately measure ToA (time of arrival), which is the time it takes for a pulse to reach a target, and AoA (angle of arrival), which is the pulse arrival angle at the transmitting device, through communication between UWB antennas, and accordingly, precise distance and location recognition is possible within an error range of several tens of centimeters indoors.

[0167] In one or more embodiments of the present disclosure, if the processor (140) determines that the user is not a registered user as a result of performing user authentication, the processor (140) may control the communication unit (160) to keep the electronic device (100) locked and transmit a message to an external device connected to the electronic device (100) to inform that authentication has been performed by a user who is not a registered user.

[0168] The input unit (170) includes a circuit, and the processor (140) can receive a user command to control the operation of the electronic device (100) through the input unit (170). Specifically, the input unit (170) can be configured with components such as a microphone, a camera, and a remote control signal receiving unit. In addition, the input unit (170) can also be implemented in a form included in a display as a touch screen. In particular, the microphone can receive a voice signal and convert the received voice signal into an electrical signal.

[0169] In one or more embodiments of the present disclosure, the processor (140) may receive user input, such as user input for initiating user authentication, user input for user registration, etc., through the input unit (170).

[0170] The output unit (180) includes a circuit, and the processor (140) can output various functions that the electronic device (100) can perform through the output unit (180). In addition, the output unit (180) can include at least one of a display, a speaker, and an indicator.

[0171] The display can output image data under the control of the processor (140). Specifically, the display can output an image previously stored in the memory (130) under the control of the processor (140). In particular, the display according to one embodiment of the present disclosure can also display a user interface stored in the memory (130). The display can be implemented as an LCD (Liquid Crystal Display Panel), an OLED (Organic Light Emitting Diodes), etc., and in some cases, the display can also be implemented as a flexible display, a transparent display, etc. However, the display according to the present disclosure is not limited to a specific type.

[0172] The speaker can output audio data under the control of the processor (140), and the indicator can be turned on under the control of the processor (140).

[0173] In one or more embodiments of the present disclosure, if the user authentication is performed and the user is identified as an unregistered user, the processor (140) may control the output unit (180) to maintain the lock of the electronic device (100) and output a guidance message for requesting additional authentication.

[0174] FIG. 5 and FIG. 6 are flowcharts illustrating a control method of an electronic device (100) according to various embodiments of the present disclosure.

[0175] Referring to FIG. 5, the electronic device (100) may receive a request for user authentication (S510). When the request for user authentication is received, the electronic device (100) may acquire a first signal through at least one motion sensor (110) and a second signal through at least one biometric sensor (120) (S520).

[0176] In one or more embodiments of the present disclosure, a request for user authentication may be received based on user input to initiate user authentication, or may be received based on the occurrence of an event defined as requiring user authentication.

[0177] In one or more embodiments of the present disclosure, when a first signal is received through at least one motion sensor (110), the electronic device (100) can obtain a second signal through at least one biometric sensor (120). For example, the electronic device (100) can obtain the second signal through at least one biometric sensor (120) for a predetermined period of time after the first signal starts to be received through at least one motion sensor (110) according to the user's motion.

[0178] When the first signal is acquired, the electronic device (100) can acquire motion information indicating the type of user's motion based on the first signal (S530). Acquiring motion information indicating the type of user's motion may include a process of identifying which of a plurality of predefined motion types the user's motion indicated by the characteristics of the first signal corresponds to.

[0179] When the second signal is acquired, the electronic device (100) can acquire biometric information representing the biometric characteristics of the user according to the user's motion based on the second signal (S540). At least one biometric sensor (120) can include at least one of a bone conduction sensor (121), a bioimpedance sensor (122), and a heartbeat sensor (123), and the electronic device (100) can receive the second signal through each of the bone conduction sensor (121), the bioimpedance sensor (122), and the heartbeat sensor (123). In addition, the electronic device (100) can acquire, based on the second signal, information about an acoustic characteristic transmitted from at least one of the user's bones and joints according to the user's motion, information about an electrical characteristic transmitted from at least one of the user's muscles, tissues, and joints according to the user's motion, and information about a heartbeat characteristic transmitted from the user's heart while the user's motion is performed.

[0180] The electronic device (100) can input motion information and biometric information into a user authentication model and perform user authentication based on whether the type of motion and biometric characteristics match the authentication information (S550).

[0181] A "user authentication model" refers to a neural network model trained to perform user authentication. The user authentication model may be trained to identify whether input motion information and biometric information match previously constructed authentication information. Specifically, the electronic device (100) may combine a first vector corresponding to motion information and a second vector corresponding to biometric information to obtain a third vector, and input the obtained third vector into the user authentication model. When the third vector is input, the user authentication model may identify whether a feature indicated by the third vector matches a feature of previously constructed authentication information, and output information regarding the identification result.

[0182] In one or more embodiments of the present disclosure, the electronic device (100) may compare the type of motion identified based on the first signal and the biometric characteristic identified based on the second signal with the type of motion and biometric characteristic included in the authentication information, and obtain a score based on the comparison result. Then, if the obtained score is equal to or greater than a preset threshold value, the electronic device (100) may identify the user as a registered user and output a result value indicating authentication success. On the other hand, if the obtained score is less than the preset threshold value, the electronic device (100) may identify the user as an unregistered user and output a result value indicating authentication failure.

[0183] FIG. 6 is a flowchart for explaining the operation of an electronic device (100) according to the result of user authentication performed according to step S550 of FIG. 5.

[0184] Referring to FIG. 6, if the user is identified as a registered user as a result of user authentication (S610-Y), the electronic device (100) can unlock the electronic device (100) (S620). When the electronic device (100) is unlocked, the electronic device (100) can allow the user to access information stored in the electronic device (100), such as schedules, text messages, and contacts, or to use applications, such as phone applications, payment applications, and financial applications.

[0185] Additionally, if the user is identified as a registered user as a result of user authentication (S610-Y), the electronic device (100) can update authentication information based on motion information and biometric information (S630). In other words, if the motion information and biometric information used for user authentication are identified as reliable information, the electronic device (100) can update (or calibrate) the authentication information using the motion information and biometric information used for user authentication.

[0186] If the user authentication is performed and it is identified that the user is not a registered user (S610-N), the electronic device (100) can maintain the lock of the electronic device (100) (S640). In addition, if the user authentication is performed and it is identified that the user is not a registered user (S610-N), the electronic device (100) can transmit a message to an external device connected to the electronic device (100) to inform that authentication has been performed by a user who is not a registered user (S650), and can output a guidance message to request additional authentication (S660). In addition, if the user authentication is performed and it is identified that the user is not a registered user (S610-N), the electronic device (100) may not update the authentication information using the motion information and biometric information used for the user authentication.

[0187] FIG. 7 is a flowchart illustrating a control method of an electronic device (100) according to one embodiment of the present disclosure.

[0188] Referring to FIG. 7, the electronic device (100) may receive a request for user authentication (S710). When the request for user authentication is received, the electronic device (100) may obtain a first signal through the motion sensor (110) and a heartbeat signal through the heartbeat sensor (123) (S720).

[0189] When a first signal is acquired through the motion sensor (110), the electronic device (100) can acquire motion information based on the first signal (S730). Specifically, the electronic device (100) can identify which type of motion the user corresponds to among a plurality of predefined motion types based on the first signal received through at least one motion sensor (110). For example, the electronic device (100) can identify which activity the user's motion corresponds to among activities such as 'walking', 'running', 'sitting', and 'sleeping' based on the first signal.

[0190] When a heartbeat signal is acquired through a heartbeat sensor (123), the electronic device (100) can adjust noise included in the heartbeat signal (S740). Specifically, the electronic device (100) can stabilize the heartbeat signal by adjusting at least one of a frequency parameter, a statistical parameter, and a trend parameter related to the heartbeat signal.

[0191] The electronic device (100) can obtain information on heartbeat characteristics and stress information based on the noise-adjusted heartbeat signal (S750). Specifically, the electronic device (100) can obtain information on heartbeat characteristics based on the noise-adjusted heartbeat signal, and can also obtain stress information indicating the user's stress level based on the noise-adjusted heartbeat signal. The electronic device (100) can input motion information, biometric information, and stress information into a user authentication model, and perform user authentication based on whether the type of motion, heartbeat characteristics, and stress level match the authentication information (S760).

[0192] Specifically, the electronic device (100) may compare the type of motion identified based on the first signal, the stress level identified based on the heartbeat signal, and the heartbeat characteristic identified based on the heartbeat signal with the type of motion, the stress level, and the heartbeat characteristic included in the authentication information, and obtain a score based on the comparison result. Then, if the obtained score is equal to or greater than a preset threshold value, the electronic device (100) may identify the user as a registered user and output a result value indicating authentication success. On the other hand, if the obtained score is less than the preset threshold value, the electronic device (100) may identify the user as an unregistered user and output a result value indicating authentication failure.

[0193] Meanwhile, if the user is identified as a registered user as a result of user authentication, the electronic device (100) may update authentication information based on motion information, information about heartbeat characteristics, and stress information. On the other hand, if the user is identified as an unregistered user as a result of user authentication, the electronic device (100) may maintain the lock of the electronic device (100) and may not update authentication information using the motion information, information about heartbeat characteristics, and stress information used for user authentication.

[0194] Meanwhile, the control method of the electronic device (100) according to the above-described embodiment may be implemented as a program and provided to the electronic device (100). In particular, the program including the control method of the electronic device (100) may be stored and provided in a non-transitory computer readable medium.

[0195] Specifically, in a non-transitory computer-readable recording medium including a program for executing a method for controlling an electronic device (100), the method for controlling an electronic device (100) may include, when a request for user authentication is received, a step of obtaining a first signal through at least one motion sensor (110) and a step of obtaining a second signal through at least one biometric sensor (120), a step of obtaining motion information indicating a type of motion of the user based on the first signal, a step of obtaining biometric information indicating a biometric characteristic of the user according to the motion of the user based on the second signal, and a step of inputting the motion information and the biometric information into a user authentication model and performing user authentication based on whether the type of motion and the biometric characteristic match the authentication information.

[0196] In the above, a method for controlling an electronic device (100) and a computer-readable recording medium including a program for executing the method for controlling an electronic device (100) have been briefly described, but this is only to omit redundant descriptions, and it goes without saying that various embodiments of the electronic device (100) can also be applied to a method for controlling an electronic device (100) and a computer-readable recording medium including a program for executing the method for controlling an electronic device (100).

[0197] According to various embodiments of the present disclosure as described above, when a motion for user authentication is performed, the electronic device (100) can perform user authentication based on the type of motion and the user's biometric characteristics according to the motion.

[0198] Furthermore, since the electronic device (100) performs user authentication by combining the user's motion type and the user's biometric characteristics, the accuracy of user authentication can be improved. For example, the electronic device (100) can perform highly accurate and convenient multi-modal user authentication by combining the characteristics of bone conduction signals, bioimpedance signals, and heartbeat signals, which are highly unique to each user, with the user's motions performed in real time.

[0199] In addition, the electronic device (100) does not require special sensors such as fingerprint recognition sensors, high-performance cameras, etc., and can efficiently perform user authentication without consuming a large amount of power because it collects various information through sensors after the user's motion begins.

[0200] In addition, since the electronic device (100) performs user authentication by identifying the user's motion and the user's biometric presence (i.e., liveness checking), high security can be secured, and especially when the electronic device (100) is implemented as a wearable device (e.g., a smart watch, etc.), the electronic device (100) can be easily utilized as an authentication means for other devices of the user (e.g., a laptop, a vehicle, a digital door lock, etc.) or various services requiring user authentication (e.g., a payment service, a financial service, etc.).

[0201] In addition, if the electronic device (100) identifies that the motion information and biometric information used for user authentication are reliable information, the reliability of user authentication can be further improved each time the number of user authentications increases by updating the authentication information using the motion information and biometric information used for user authentication.

[0202] Furthermore, the electronic device (100) performs user authentication by combining a highly unique heartbeat signal for the user with the user's motion with high real-time accuracy, thereby enabling convenient, transparent, and highly accurate user authentication even under various external conditions, such as when the user is exercising or in a dimly lit environment.

[0203] The artificial intelligence-related function according to the present disclosure is operated through the processor (140) and memory (130) of the electronic device (100).

[0204] The processor (140) may be composed of one or more processors (140). At this time, the one or more processors (140) may include at least one of a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), and an NPU (Neural Processing Unit), but is not limited to the examples of the processors (140) described above.

[0205] The CPU is a general-purpose processor (140) capable of performing not only general calculations but also artificial intelligence calculations. Its multi-layer cache structure allows for the efficient execution of complex programs. The CPU is advantageous in a serial processing method, which enables organic linking of previous and subsequent calculation results through sequential calculations. The general-purpose processor (140) is not limited to the examples described above, except in cases where it is specifically referred to as a CPU.

[0206] A GPU is a processor (140) for large-scale operations such as floating point operations used in graphic processing, and can perform large-scale operations in parallel by integrating a large number of cores. In particular, a GPU may be advantageous compared to a CPU in parallel processing methods such as convolution operations. In addition, a GPU may be used as a co-processor (140) to supplement the functions of a CPU. The processor (140) for large-scale operations is not limited to the examples described above, except in cases where it is specifically referred to as a GPU.

[0207] An NPU is a processor (140) specialized in artificial intelligence operations using an artificial neural network, and each layer constituting the artificial neural network can be implemented with hardware (e.g., silicon). At this time, since the NPU is designed specifically according to the required specifications of the company, it has a lower degree of freedom compared to a CPU or GPU, but it can efficiently process the artificial intelligence operations requested by the company. Meanwhile, as a processor (140) specialized in artificial intelligence operations, the NPU can be implemented in various forms such as a Tensor Processing Unit (TPU), an Intelligence Processing Unit (IPU), a Vision Processing Unit (VPU), etc. The artificial intelligence processor (140) is not limited to the above-described examples, except in cases where it is specified as the above-described NPU.

[0208] Additionally, one or more processors (140) may be implemented as a System on Chip (SoC). In this case, the SoC may further include, in addition to one or more processors (140), a memory (130), and a network interface such as a bus for data communication between the processor (140) and the memory (130).

[0209] When a plurality of processors (140) are included in a SoC (System on Chip) included in an electronic device (100), the electronic device (100) may perform operations related to artificial intelligence (e.g., operations related to learning or inference of an artificial intelligence model) by using some of the plurality of processors (140). For example, the electronic device (100) may perform operations related to artificial intelligence by using at least one of a GPU, an NPU, a VPU, a TPU, and a hardware accelerator specialized in artificial intelligence operations such as convolution operations and matrix multiplication operations among the plurality of processors (140). However, this is merely an example, and it is of course possible to process operations related to artificial intelligence by using a CPU or a general-purpose processor (140).

[0210] In addition, the electronic device (100) can perform operations related to functions related to artificial intelligence by utilizing multiple cores (e.g., dual cores, quad cores, etc.) included in one processor (140). More specifically, the electronic device (100) can perform artificial intelligence operations, such as convolution operations and matrix multiplication operations, in parallel by utilizing multiple cores included in the processor (140).

[0211] One or more processors (140) are controlled to process input data according to predefined operation rules or artificial intelligence models stored in the memory (130). The predefined operation rules or artificial intelligence models are characterized by being created through learning.

[0212] Here, "created through learning" means that a predefined set of behavioral rules or an AI model with desired characteristics is created by applying a learning algorithm to a large number of learning data. This learning may be performed on the device itself, where the AI ​​according to the present disclosure is implemented, or through a separate server / system.

[0213] An artificial intelligence model may be composed of multiple neural network layers. At least one layer has at least one weight value and performs its operation through the operation result of the previous layer and at least one defined operation. Examples of neural networks include a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, and a transformer. The neural networks in the present disclosure are not limited to the above-described examples unless otherwise specified.

[0214] A learning algorithm is a method for training a target device (e.g., a robot) using a large amount of learning data, thereby enabling the target device to make decisions or predictions on its own. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Unless otherwise specified, the learning algorithms in this disclosure are not limited to the aforementioned examples.

[0215] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0216] According to one embodiment of the present disclosure, the method according to various embodiments disclosed in the present document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smartphones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be at least temporarily stored or temporarily created in a device-readable storage medium, such as a manufacturer's server, an application store's server, or a memory (130) of an intermediary server.

[0217] In addition, each of the components (e.g., modules or programs) according to the various embodiments of the present disclosure as described above may be composed of a single or multiple entities, and some of the sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the respective components prior to integration.

[0218] Furthermore, according to various embodiments, operations performed by a module, program or other component may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0219] Meanwhile, the terms "part" or "module" used in the present disclosure include units composed of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A "part" or "module" may be an integrally composed component, a minimum unit performing one or more functions, or a portion thereof. For example, a module may be composed of an application-specific integrated circuit (ASIC).

[0220] Various embodiments of the present disclosure may be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device may include an electronic device (e.g., an electronic device (100)) according to the disclosed embodiments, which is a device capable of calling instructions stored in the storage medium and operating according to the called instructions.

[0221] When the above instruction is executed by the processor, the processor may perform the function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or interpreter.

[0222] It will be appreciated that the various embodiments of the present disclosure, in accordance with the claims and description herein, may be implemented in the form of hardware, software, or a combination of hardware and software.

[0223] Such software may be stored on a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium stores one or more computer programs (software modules), and the one or more computer programs include computer-executable instructions that, when executed by one or more processors of the electronic device, cause the electronic device to perform the disclosed method.

[0224] Such software may be stored in a storage device, whether erasable or rewritable, in volatile or non-volatile storage, for example, a read only memory (ROM), or in a random access memory (RAM), a memory chip, device or integrated circuit, or in an optically or magnetically readable medium, for example, a compact disc (CD), a digital versatile disc (DVD), a magnetic disk or magnetic tape. It will be appreciated that the storage device and the storage medium are various embodiments of non-transitory machine-readable storage suitable for storing a computer program or a computer program comprising instructions that, when executed, implement various embodiments of the present disclosure. Accordingly, various embodiments provide a program comprising code for implementing an apparatus or method as claimed in any of the claims of this specification, and a non-transitory machine-readable storage medium storing such a program.

[0225] While the present disclosure has been illustrated and described with reference to various embodiments, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.

Claims

1. In electronic devices, At least one motion sensor; At least one biometric sensor; A memory storing one or more computer programs; and comprising at least one motion sensor, at least one biometric sensor and at least one processor in communication with the memory; The processor executes one or more computer programs including computer execution instructions that operate the electronic device. When a request for user authentication is received, a first signal is acquired through the at least one motion sensor, and a second signal is acquired through the at least one biometric sensor, Obtain motion information indicating the type of user's motion based on the first signal, Based on the second signal, biometric information representing the user's biometric characteristics according to the user's motion is obtained, An electronic device that inputs the above motion information and the above biometric information into a user authentication model and performs the user authentication based on whether the type of the motion and whether the above biometric characteristic matches the authentication information.

2. In paragraph 1, The processor executes one or more computer programs including computer execution instructions that operate the electronic device. If the user authentication is performed and it is identified that the user is a registered user, the electronic device is unlocked. An electronic device that updates the authentication information based on the motion information and the biometric information.

3. In paragraph 1, The processor executes one or more computer programs including computer execution instructions that operate the electronic device. An electronic device that acquires a second signal through at least one biometric sensor when the first signal begins to be received through at least one motion sensor.

4. In paragraph 1, wherein said at least one biosensor comprises a bone conduction sensor, An electronic device wherein the biometric information includes information about acoustic features transmitted from at least one of the user's bones and joints according to the user's motion.

5. In paragraph 1, wherein said at least one biosensor comprises a bioimpedance sensor, An electronic device wherein the biometric information includes information about electrical characteristics transmitted from at least one of the user's muscles, tissues, and joints according to the user's motion.

6. In paragraph 1, wherein said at least one biometric sensor comprises a heart rate sensor, An electronic device wherein the biometric information includes information about heartbeat characteristics transmitted from the user's heart while the user performs the motion.

7. In paragraph 6, The processor executes one or more computer programs including computer execution instructions that operate the electronic device. Based on the above biometric information, stress information indicating the user's stress level is obtained, The above user authentication model is, An electronic device that performs said user authentication based on whether said stress level, said type of motion and said heart rate characteristic match said authentication information.

8. In paragraph 1, The processor executes one or more computer programs including computer execution instructions that operate the electronic device. Preprocessing is performed to remove at least one of noise and trends of the first signal and the second signal, An electronic device that obtains the motion information and the bio-information based on the preprocessed first signal and the preprocessed second signal.

9. In paragraph 1, including the wearing part; An electronic device comprising at least one sensor, the at least one motion sensor and the at least one biometric sensor, coupled to the wearable.

10. In paragraph 1, including the Department of Communications; The processor executes one or more computer programs including computer execution instructions that operate the electronic device. If the user authentication is performed and it is determined that the user is not a registered user, the electronic device is locked. An electronic device controlling the communication unit to transmit a message to notify that authentication has been performed by a user who is not registered as an external device connected to the electronic device.

11. In paragraph 1, further comprising an output section; The processor executes one or more computer programs including computer execution instructions that operate the electronic device. If the user authentication is performed and it is determined that the user is not a registered user, the electronic device is locked. An electronic device that controls the output unit to output a guidance message requesting additional authentication.

12. In a method for controlling an electronic device, When a request for user authentication is received, the step of obtaining a first signal through at least one motion sensor and obtaining a second signal through the at least one biometric sensor; A step of obtaining motion information indicating the type of user's motion based on the first signal; A step of obtaining biometric information representing the user's biometric characteristics according to the user's motion based on the second signal; and A method for controlling an electronic device, comprising: a step of inputting the motion information and the biometric information into a user authentication model, and performing the user authentication according to whether the type of the motion and the biometric characteristic match the authentication information; 13. In paragraph 12, If the user authentication is performed and the user is identified as a registered user, a step of unlocking the electronic device; and A method for controlling an electronic device, further comprising: a step of updating the authentication information based on the motion information and the biometric information; 14. In paragraph 12, The step of obtaining the second signal is: A control method of an electronic device that acquires a second signal through at least one biometric sensor when the first signal begins to be received through at least one motion sensor.

15. In paragraph 12, wherein said at least one biosensor comprises a bone conduction sensor, A control method of an electronic device, wherein the biometric information includes information about acoustic characteristics transmitted from at least one of the user's bones and joints according to the user's motion.

Images