Rest state-based user authentication device using electromyogram signal and inertia measurement device signal and method thereof

Abstract

The present invention relates to a technology for rest state-based user authentication by using an electromyogram signal and an inertia measurement device signal and provides a rest state-based user authentication device using an electromyogram signal and an inertia measurement device signal, which extracts a user's unique electromyogram signal and inertia measurement device signal during a rest state in which a specific operation is not performed, and inputs the extracted signal into a linear discriminant analysis and ensemble classification device to identify and authenticate the user, and a method thereof.

Description

A resting state-based user authentication device and method using electromyography signals and inertial measurement device signals

[0001] The present invention relates to a technology for authenticating a user based on a resting state using an electromyography signal and an inertial measurement device signal, and more particularly, to a device and method for authenticating a user based on a resting state using an electromyography signal and an inertial measurement device signal, which extracts a user's individual electromyography signal and an inertial measurement device signal during a resting state in which a specific movement is not performed, and inputs the extracted signal into a linear discriminant analysis and an ensemble classifier to identify and authenticate the user.

[0002] [National Research and Development Project Supporting This Invention]

[0003] [Project ID] 2710003363

[0004] [Project Number] RS-2023-00302489

[0005] [Ministry Name] Ministry of Science and ICT

[0006] [Name of Project Management (Specialist) Institution] National Research Foundation of Korea

[0007] [Research Project Name] STEAM Research

[0008] [Research Project Name] Hyper-realistic Sensitivity Based on a Precision Encoding / Decoding Model of Tactile Sensory Pathways

[0009] Development of tactile signal generation / regeneration technology

[0010] [Name of the project performing organization] Korea University Sejong Industry-Academic Cooperation Foundation

[0011] Research Period: August 1, 2023 - December 31, 2027

[0012] [National Research and Development Project Supporting This Invention]

[0013] [Project ID] 2710006658

[0014] [Project Number] RS-2023-00258971

[0015] [Ministry Name] Ministry of Science and ICT

[0016] [Name of Project Management (Specialist) Agency] Information and Communications Technology Planning and Evaluation Institute

[0017] [Research Project Name] Training of Information, Communication, and Broadcasting Innovation Talents

[0018] [Research Project Name] Large-Scale AI-Based Smart City Health Using Multimodal Biodata

[0019] Care technology development

[0020] [Name of the project performing organization] Korea University Sejong Industry-Academic Cooperation Foundation

[0021] [Research Period] July 1, 2023 - December 31, 2030

[0022] [National Research and Development Project Supporting This Invention]

[0023] [Project ID] 2710020012

[0024] [Project Number] RS-2024-00397674

[0025] [Ministry Name] Ministry of Science and ICT

[0026] [Name of Project Management (Specialist) Institution] National Research Foundation of Korea

[0027] [Research Project Name] Bio-Medical Technology Development (R&D)

[0028] [Research Project Name] NeuroAdap: A Rehabilitation Approach to Promoting Neuroplasticity in Stroke Patients

[0029] A personalized, AI-based, brain-machine closed-loop interface system

[0030] development

[0031] [Name of the project performing organization] Korea Advanced Institute of Science and Technology

[0032] Research Period: April 1, 2024 - December 31, 2026

[0033] Biometrics technology is a technology that authenticates individuals by utilizing unique characteristics such as fingerprints and irises. It is attracting attention as an information security technology that provides high reliability and convenience while complementing the shortcomings of existing identification methods such as one-time passwords and security cards.

[0034] Recent biometric technologies are actively conducting research on biometric authentication using electrical biosignals such as electrocardiography (ECG), electroencephalography (EEG), and electromyogram (EMG).

[0035] Electrical biometric signals can provide more robust security compared to fingerprint and iris-based biometrics because they are difficult to synthesize and imitate due to their hidden characteristics.

[0036] In particular, electromyography is a noninvasive electrical signal that detects muscle activity by placing surface electrodes on the skin, measuring motor nerve activity induced by muscle contraction.

[0037] Compared to other biosignals, electromyography (EMG) has the characteristic of significantly varying depending on age, movement, muscle strength, etc., and because it can be easily measured on the muscle surface, it can be utilized in wrist or hand gesture-based user authentication systems.

[0038] When using EMG signals alone, there are limitations in that they are non-periodic and highly variable, but this can be overcome by using inertial measurement device signals that can reflect the characteristics of position changes during gesture performance.

[0039] However, since conventional electromyography-based personal authentication technology uses specific wrist and hand gestures as illustrated in Fig. 1, studies on user authentication using electromyography-based specific wrist and hand gestures have raised the issue that, although accurate, they are unnatural and unintuitive when using gestures that are not frequently used in daily life.

[0040] Human muscles have different muscle activity levels for each individual, just like fingerprints, and the electromyography and muscle activity patterns induced during a resting state also differ from person to person.

[0041] Currently, although the electromyography signal at rest can also be a useful feature for biometric authentication, there are few reports of user authentication studies utilizing it.

[0042] Accordingly, it is necessary to develop a technology that can authenticate users in a resting state without requiring specific gestures by utilizing multimodal biosignal measurement equipment that can simultaneously measure electromyography and inertial measurement device signals.

[0043] In order to solve the stated problem, the technical problem to be achieved by the present invention is to provide a resting state-based user authentication device and method using electromyography signals and inertial measurement device signals, which can extract a user's individual electromyography signal and inertial measurement device signal during a resting state in which no specific movement is performed, and input the extracted signal into a linear discriminant analysis and an ensemble classifier to identify and authenticate the user.

[0044] The technical problem to be achieved by the present invention is to provide a resting state-based user authentication device and method using electromyography signals and inertial measurement device signals, which can improve convenience and practicality by allowing the user to perform personal authentication in a comfortable state without making a specific gesture and without requiring long-term repetitive training for a specific gesture.

[0045] The technical problem to be achieved by the present invention is to provide a resting state-based user authentication device and method using electromyography signals and inertial measurement device signals, which can authenticate users without temporal and spatial constraints by applying real-time analysis.

[0046] The technical problem to be achieved by the present invention is to provide a user authentication device and method based on a resting state using electromyography signals and inertial measurement device signals, which can easily identify an individual without a separate action in applications such as virtual reality and games that utilize electromyography signals and inertial measurement device signals.

[0047] The technical problems to be solved by the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0048] In order to achieve the above technical problem, a user authentication device using an electromyography signal and an inertial measurement device signal according to an embodiment of the present invention includes an electromyography measurement module for measuring an electromyography signal generated from a movement of a user's muscles in a resting state; an inertial measurement module for measuring an inertial measurement device signal including an acceleration and angular velocity signal of a moving object based on a sensor in the resting state; a measurement value preprocessing unit for performing a preprocessing process for reducing data loss, standard deviation, noise, and disturbance due to communication transmission on the electromyography signal and the inertial measurement device signal; a user information registration unit for registering user information based on the preprocessed electromyography signal and the inertial measurement device signal; a user information processing unit for identifying or authenticating an individual user based on the preprocessed electromyography signal and the inertial measurement device signal; and an artificial intelligence model for learning and classifying an individual user identification and individual user authentication by receiving the electromyography signal and the inertial measurement device signal and using the artificial intelligence model.

[0049] The above measurement value preprocessing unit includes an outlier removal unit that removes outliers using a standard deviation for data loss occurring in wireless communication according to the EMG signal and the inertial measurement device signal; an external noise removal unit that removes external noise introduced when measuring the EMG signal and the inertial measurement device signal; a time series feature extraction unit that extracts time series features for the EMG signal and the inertial measurement device signal; and a motion outlier removal unit that removes motion outlier data based on the extracted time series feature data, thereby allowing measurement of the EMG signal and the inertial measurement device signal in a resting state.

[0050] The above outlier removal unit can remove trials judged to be outliers using 2 standard deviations based on the number of samples measured in each trial.

[0051] The above external noise removal unit can remove external noise by applying a band-pass filter to a frequency range of 1 to 240 Hz of the measured electromyography signal and by applying a band-pass filter to a frequency range of 0.1 to 15 Hz of the measured inertial measurement device signal.

[0052] The above time series feature extraction unit can extract the time series feature using at least one of a root mean square error, a median absolute deviation, a variance, a zero crossing, a waveform length, and a mean absolute value.

[0053] The above motion outlier removal unit can remove outlier data that is determined to have moved based on two standard deviations of the time series features in order to increase the reliability of the feature data extracted by the time series feature extraction unit.

[0054] The user information processing unit may include a user identification unit that identifies a user based on the electromyography signal and the inertial measurement device signal; and a user authentication unit that authenticates a user based on the electromyography signal and the inertial measurement device signal.

[0055] The artificial intelligence model includes a linear discriminant classification model that identifies a registered user individual by inputting the electromyography signal and the inertial measurement device signal; and an ensemble classification model that authenticates a registered user individual by inputting the electromyography signal and the inertial measurement device signal.

[0056] The above artificial intelligence model can apply 5-fold cross-validation to prevent overfitting.

[0057] In addition, in order to achieve the above-described technical task, according to an embodiment of the present invention, a user authentication method using an electromyography signal and an inertial measurement device signal comprises the steps of: measuring an electromyography signal generated from a movement of a user's muscles in a resting state; measuring an inertial measurement device signal including an acceleration and an angular velocity of a moving object based on a sensor in the resting state; performing a preprocessing process for reducing data loss, standard deviation, noise, and disturbance due to communication transmission on the electromyography signal and the inertial measurement device signal; registering user information based on the preprocessed electromyography signal and the inertial measurement device signal; and receiving the preprocessed electromyography signal and the inertial measurement device signal and learning and classifying the user individual identification and user individual authentication using an artificial intelligence model.

[0058] The step of executing the above preprocessing process includes the step of removing outliers using a standard deviation for data loss occurring in wireless communication according to the EMG signal and the inertial measurement device signal; the step of removing external noise introduced when measuring the EMG signal and the inertial measurement device signal; the step of extracting time-series features for the EMG signal and the inertial measurement device signal; and the step of removing movement outlier data based on the extracted time-series feature data, thereby allowing the EMG signal and the inertial measurement device signal to be measured in a resting state.

[0059] The step of removing the above outliers can remove trials judged to be outliers using 2 standard deviations based on the number of samples measured in each trial.

[0060] The step of removing the external noise may remove the external noise by applying a band-pass filter to the frequency range of 1 to 240 Hz of the measured electromyography signal and applying a band-pass filter to the frequency range of 0.1 to 15 Hz of the measured inertial measurement device signal.

[0061] The step of extracting the time series feature may extract the time series feature using at least one of root mean square, median absolute deviation, variance, zero crossing, waveform length, and mean absolute value.

[0062] The step of removing the above movement outlier data can remove outlier data that is judged to have moved based on 2 standard deviations of the time series feature in order to increase the reliability of the feature data extracted from the time series feature extraction unit.

[0063] The above user identification or authentication may include a user identification unit that identifies a user based on the electromyography signal and the inertial measurement device signal; and a user authentication unit that authenticates a user based on the electromyography signal and the inertial measurement device signal.

[0064] The artificial intelligence model may include a linear discriminant classification model that identifies a registered user individual by inputting the electromyography signal and the inertial measurement device signal; and an ensemble classification model that authenticates a registered user individual by inputting the electromyography signal and the inertial measurement device signal.

[0065] The above artificial intelligence model can apply 5-fold cross-validation to prevent overfitting.

[0066] According to an embodiment of the present invention, a user's personal electromyography signal and inertial measurement device signal can be extracted during a resting state when no specific movement is performed, and the extracted signals can be input into a linear discriminant analysis and an ensemble classifier to enable personal identification and authentication.

[0067] In addition, according to an embodiment of the present invention, convenience and practicality can be improved because a user can perform personal authentication in a comfortable state without having to make a specific gesture and without requiring long-term repetitive training for a specific gesture.

[0068] Additionally, according to an embodiment of the present invention, user authentication can be achieved without temporal or spatial constraints by applying real-time analysis.

[0069] In addition, according to an embodiment of the present invention, personal identification can be easily performed without separate operation in applications such as virtual reality and games that utilize electromyography signals and inertial measurement device signals.

[0070] The effects of the present invention are not limited to the above-described effects, and should be understood to include all effects that can be inferred from the composition of the invention described in the description or claims of the present invention.

[0071] Figure 1 is an example diagram of a method for taking a specific gesture for conventional personal authentication.

[0072] FIG. 2 is a block diagram illustrating the configuration of a resting state-based user authentication device using electromyography signals and inertial measurement device signals according to an embodiment of the present invention.

[0073] FIG. 3 is an exemplary diagram showing signals measured by an electromyography measurement module and an inertial measurement module according to an embodiment of the present invention.

[0074] Fig. 4 is a block diagram showing a detailed configuration of a measurement value preprocessing unit according to an embodiment of the present invention.

[0075] Figure 5 is a block diagram showing a detailed configuration of a user information processing unit according to an embodiment of the present invention.

[0076] FIG. 6 is a block diagram illustrating the relationship between a user information processing unit and an artificial intelligence model according to an embodiment of the present invention.

[0077] FIG. 7 is a graph showing the average classification accuracy of individual identification by posture and measurement modality according to an embodiment of the present invention.

[0078] FIG. 8 is a graph showing the average classification accuracy of individual authentication by posture and measurement modality according to an embodiment of the present invention.

[0079] FIG. 9 is a flowchart illustrating a resting state-based user authentication method using electromyography signals and inertial measurement device signals according to an embodiment of the present invention.

[0080] Figure 10 is a flowchart illustrating a detailed method of preprocessing measurement values ​​according to an embodiment of the present invention.

[0081] Hereinafter, the present invention will be described with reference to the attached drawings. However, the present invention can be implemented in various different forms and is therefore not limited to the embodiments described herein. In the drawings, irrelevant parts have been omitted for clarity of description, and similar parts have been designated with similar reference numerals throughout the specification.

[0082] Throughout the specification, when a part is said to be "connected (connected, contacted, or coupled)" to another part, this includes not only cases where it is "directly connected," but also cases where it is "indirectly connected" with another part in between. Furthermore, when a part is said to "include" a component, this does not exclude other components, but rather implies that it may include other components, unless otherwise specifically stated.

[0083] The terminology used herein is merely used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, it should be understood that the terms "comprises" or "has" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0084] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0085] FIG. 2 is a block diagram illustrating the configuration of a resting state-based user authentication device using electromyography signals and inertial measurement device signals according to an embodiment of the present invention.

[0086] FIG. 3 is an exemplary diagram showing signals measured by an electromyography measurement module and an inertial measurement module according to an embodiment of the present invention.

[0087] As illustrated in FIG. 2, a resting state-based user authentication device (100) using an electromyography signal and an inertial measurement device signal may include an electromyography measurement module (110), an inertial measurement module (120), a measurement value preprocessing unit (130), a user information registration unit (140), a user information processing unit (150), and an artificial intelligence model (160).

[0088] The electromyography measurement module (110) can measure electromyography (EMG) signals generated before and after the user's muscle movement.

[0089] Electromyography (EMG) refers to a curve that records the action potential of muscles, and EMG can be measured in various ways, including the surface extraction method, which attaches electrodes to the surface of the human skin and measures the action potential of the muscles at the attached location, and the needle electrode method, which extracts data by inserting a needle-shaped electrode into the muscle.

[0090] The electromyography measurement module (110) can be attached, worn, or removed from the right forearm muscle or part of the body using an arm band, wrist band, or other fixing device.

[0091] Conventionally, trigger signals were measured by having a user wear an electromyography (EMG) device on their arm and make a predetermined gesture, such as extending their hand as shown in Fig. 1, moving or twisting their wrist or palm in a specific direction, or making a fist. Conventional gestures, however, were unnatural and unintuitive, as they utilized gestures that were less commonly used in everyday life.

[0092] On the other hand, the present invention can measure electromyography signals by having the user wear a device equipped with an electromyography measurement module (110) on his or her arm, and while the user is at rest, that is, in a posture where the arm is kept still without moving.

[0093] For example, as a result of examining the resting state paradigm illustrated in Fig. 2, the resting state used in the experiment was performed 20 times each in a sitting state (sit) and a standing state (stand). In the sitting state (sit), the subjects were instructed to place their arms comfortably on the desk with their elbows fixed, and in the standing state (stand), the subjects were instructed to relax their arms and naturally extend them toward the ground. The gesture to be made on the monitor for 2 seconds before performing the gesture was to relax their arms and naturally extend them toward the ground, as illustrated in Fig. 2, and after a beep sound was played, the subjects were to refrain from movement for 5 seconds and assume a resting state.

[0094] For example, the electromyography measurement module (110) is equipped with an 8-channel electromyography sensor, and has the advantage of not requiring a separate gesture and being able to comfortably detect electromyography signals in a resting state, as the electromyography and muscle activity patterns generated in a resting state are different for each person.

[0095] As a result of measuring the user's muscle signal using an 8-channel electromyography sensor by the electromyography measurement module (110), it can be confirmed that the 8-channel electromyography sensors detect different signals, as shown in FIG. 3.

[0096] The inertial measurement module (120) can measure the speed, direction, gravity, acceleration, etc. of a moving object based on a sensor that measures inertial force (IMU, Inertia Measurement Unit).

[0097] For example, the inertial measurement module (120) may include a 9-axis sensor (a 3-axis acceleration sensor, a 3-axis angular velocity sensor, a 3-axis geomagnetic sensor) or a 3-axis acceleration and 3-axis angular velocity sensor.

[0098] It can be confirmed that the inertial measurement module (120) detects a three-channel acceleration signal corresponding to a three-axis acceleration sensor and a three-channel angular velocity signal corresponding to a three-axis angular velocity sensor through each sensor, as shown in FIG. 3.

[0099] The electromyography measurement module (110) and the inertial measurement module (120) can detect detection signals simultaneously or with a time difference for each sensor, and the detected signal data can be transmitted in real time to the measurement value preprocessing unit (130) described later via Bluetooth wireless communication.

[0100] Electromyography signals and inertial measurement device signals exhibit different patterns and signals for each individual user, making them suitable for use as biometric security technologies that can authenticate individual users.

[0101] The measurement value preprocessing unit (130) can receive signals measured from the electromyography measurement module (110) and the inertial measurement module (120) and execute a preprocessing process to reduce data loss, standard deviation, noise, disturbance, etc. due to communication transmission before analyzing the two signals.

[0102] For example, the measurement value preprocessing unit (130) performs a preprocessing process based on the measured electromyography signal and inertial measurement device signal in a resting state for 5 seconds as shown in FIG. 2.

[0103] The detailed configuration and method of the measurement value preprocessing unit (130) will be described in detail later with reference to FIG. 4.

[0104] The user information registration unit (140) can register user information based on preprocessed electromyography signals and inertial measurement device signals.

[0105] User information is information that can distinguish an individual, like a fingerprint, using the above-mentioned electromyography signal and the above-mentioned inertial measurement device signal.

[0106] The user information processing unit (150) can identify or authenticate an individual user based on the preprocessed electromyography signal and inertial measurement device signal.

[0107] The detailed configuration and interaction method of the user information processing unit (150) will be described in detail later with reference to FIG. 5.

[0108] The artificial intelligence model (160) can learn and classify user personal identification and personal authentication by receiving electromyography signals and inertial measurement device signals processed in the measurement value preprocessing unit (140).

[0109] For example, an artificial intelligence learning algorithm may use linear discriminant analysis for user identification and authentication, and an ensemble classifier algorithm for user authentication.

[0110] The detailed configuration and interaction method of the user information processing unit (150) and the artificial intelligence model (160) will be described in detail later with reference to FIG. 6.

[0111] Therefore, the resting state-based user authentication device (100) using the electromyography signal and inertial measurement device signal of the present invention can extract the electromyography signal and inertial measurement device signal unique to the individual generated during the resting state when no specific movement is performed and use them as inputs to linear discriminant analysis (LDA) and an ensemble classifier (RUSBoosted tree) to identify and authenticate the individual.

[0112] In addition, the present invention has the advantage of improving convenience and practicality by allowing the user to perform personal authentication in a comfortable state without having to make a specific gesture and without requiring long-term repetitive training for a specific gesture.

[0113] Fig. 4 is a block diagram showing a detailed configuration of a measurement value preprocessing unit according to an embodiment of the present invention.

[0114] As illustrated in FIG. 4, the measurement value preprocessing unit (130) may include an outlier removal unit (131), an external noise removal unit (132), a time series feature extraction unit (133), and a motion outlier removal unit (134).

[0115] The measurement value preprocessing unit (130) receives the electromyography signal and inertial measurement device signal measured in the resting state for 5 seconds as shown in FIG. 2 via wireless communication and performs data preprocessing of the signals.

[0116] The outlier removal unit (131) can remove trials judged as outliers by using 2 standard deviations based on the number of samples measured in each trial, taking into account data loss that occurs due to the characteristics of Bluetooth transmission during signal measurement.

[0117] If there are outliers, the learning algorithm may learn incorrect patterns or reduce the accuracy of the model. To prevent this, it is desirable to remove or replace outliers to improve model learning or enhance model performance.

[0118] The external noise removal unit (132) can remove external noise introduced when measuring electromyography signals and inertial measurement device signals.

[0119] For example, the EMG signals measured in eight channels can be filtered by applying a band-pass filter for the frequency range of 1 to 240 Hz, and the acceleration and angular velocity (gyroscope) signals of an inertial measurement device measured along three axes (X, Y, Z) can be filtered by applying a band-pass filter for the frequency range of 0.1 to 15 Hz to remove external noise.

[0120] The time series feature extraction unit (133) can extract time series features for electromyography signals and inertial measurement device signals.

[0121] For example, a time series feature extraction method can extract time series features using at least one of root mean square, median absolute deviation, variance, zero crossing, waveform length, and mean absolute value.

[0122] The motion outlier removal unit (134) can remove outlier data that is judged to have moved based on two standard deviations of time series features (e.g., root mean square, median absolute deviation, variance, etc.) to increase the reliability of feature data extracted from the time series feature extraction unit (133).

[0123] Accordingly, the present invention has the effect of enabling accurate pattern learning in a classification learning algorithm by performing a preprocessing process including outlier removal, external noise removal, time series feature extraction, and movement outlier removal on electromyography signals and inertial measurement device signals measured in a resting state, thereby improving model accuracy and model performance.

[0124] Figure 5 is a block diagram showing a detailed configuration of a user information processing unit according to an embodiment of the present invention.

[0125] The user information processing unit (150) may include a user identification unit (151) and a user authentication unit (152).

[0126] The user identification unit (151) can identify an individual user based on the preprocessed electromyography signal and the inertial measurement device signal.

[0127] For example, the user identification unit (151) may use a user identification artificial intelligence algorithm, including linear discriminant analysis (LDA), to identify individual users.

[0128] The user authentication unit (152) can authenticate an individual user based on the preprocessed electromyography signal and inertial measurement device signal.

[0129] For example, the user authentication unit (152) may use a user authentication artificial intelligence algorithm including an ensemble classifier (RUSBoosted tree, Random Under-Sampling Boosting tree) for user personal authentication.

[0130] User identification is performed by inputting user identification data into an artificial intelligence algorithm and classifying each data into a separate class based on linear discriminant analysis, and user authentication is performed by classifying specific data into one class and the rest into a separate class based on an ensemble classifier.

[0131] FIG. 6 is a block diagram illustrating the relationship between a user information processing unit and an artificial intelligence model according to an embodiment of the present invention.

[0132] As illustrated in FIG. 6, the user information processing unit (150) and the artificial intelligence model (160) are interconnected to learn and classify input data using the artificial intelligence model for user personal identification and user personal authentication.

[0133] The artificial intelligence model (160) may include a linear discriminant classification model (161) and an ensemble classification model (162).

[0134] For example, the artificial intelligence model (160) learns a classifier by generating feature vectors of three types: a composite of EMG signals and inertial measurement unit signals (EMG + IMU), EMG signals, and inertial measurement unit (IMU) signals, to compare the user identification accuracy when EMG signals and inertial measurement unit signals are used alone and when EMG signals and inertial measurement unit signals are used together.

[0135] User identification can be performed through the user identification unit (151).

[0136] User personal identification can be classified by assigning each data to a separate class using a linear discriminant classification model (161) of an artificial intelligence model (160).

[0137] User authentication can be performed through the user authentication unit (152).

[0138] User personal authentication can be classified by assigning specific data to one class and assigning the remaining data to a separate class using an ensemble classification model (162) of an artificial intelligence model (160).

[0139] For example, when 36 subjects are in a resting state, electromyography signals and inertial measurement device signals are measured, and when identifying a user based on the electromyography signals and inertial measurement device signals, a linear discriminant classification model is used to classify each data by assigning a class to it, and when authenticating a user, an ensemble classification model is used to find which of the 36 classes a specific data matches, so that a user can authenticate a specific data.

[0140] At this time, the artificial intelligence model (160) can calculate the user authentication accuracy by applying 5-fold cross-validation to prevent the overfitting problem.

[0141] As a result of inputting the EMG signal and the inertial measurement device signal into the artificial intelligence model to perform user identification and user authentication, the user identification accuracy and user authentication accuracy can be improved depending on the posture and measurement modality, as shown in FIGS. 7 and 8.

[0142] FIG. 7 is a graph showing the average classification accuracy of individual identification by posture and measurement modality according to an embodiment of the present invention.

[0143] FIG. 8 is a graph showing the average classification accuracy of individual authentication by posture and measurement modality according to an embodiment of the present invention.

[0144] Figures 7 and 8 show the average classification accuracy of user individual identification and average classification accuracy of user individual authentication according to three types by measuring EMG signals and inertial measurement device signals in a resting state, i.e., a sitting state (sit) and a standing state (stand), and then synthesizing the measured EMG signals and inertial measurement device signals (EMG + IMU), EMG signals, and inertial measurement device (IMU) signals.

[0145] Referring to Figure 7, the average classification accuracy of user individual identification according to posture and measurement modality is shown. Regardless of posture, when the EMG signal and the inertial measurement device signal were used together, the user individual authentication performance in the sitting posture (sit) and standing posture (stand) was the highest at 80.58% and 75.20%, respectively.

[0146] Referring to Fig. 8, the average classification accuracy of user personal authentication according to posture and measurement modality is shown. When the EMG signal and the inertial measurement device signal were used together in all postures, the average accuracy was the highest at 90.63% and 89.87%, respectively, as in the user personal identification classification results, confirming high user personal authentication performance.

[0147] Therefore, the present personal authentication device can measure electromyography signals and inertial measurement device signals in a resting state and use the data synthesized from the two signals to achieve the highest authentication accuracy and performance in artificial intelligence-based user personal identification and user personal authentication.

[0148] In addition, the present invention can enable user authentication without spatiotemporal constraints by applying real-time analysis, and can easily authenticate users without separate actions in applications such as virtual reality and games by utilizing electromyography signals and inertial measurement device signals measured in a resting state.

[0149] FIG. 9 is a flowchart illustrating a resting state-based user authentication method using electromyography signals and inertial measurement device signals according to an embodiment of the present invention.

[0150] As illustrated in FIG. 9, the user authentication device can measure the user's electromyography signal and inertial measurement device signal in a resting state and perform user personal authentication through artificial intelligence learning.

[0151] In step (S110), the user authentication device can simultaneously measure the user's electromyography signal and inertial measurement device signal in a resting state using the electromyography measurement module and the inertial measurement module.

[0152] The user authentication device can be attached to the user's right forearm or body part to measure electromyography (EMG) signals generated before and after the user's muscle movement while at rest, i.e., with the arm comfortably lowered.

[0153] For example, the electromyography measurement module of the user authentication device is equipped with an 8-channel electromyography sensor, and since the electromyography and muscle activity patterns generated in a resting state are different for each person, it has the advantage of not requiring a separate gesture and being able to comfortably detect electromyography signals in a resting state.

[0154] For example, the inertial measurement module of the user authentication device may include a 9-axis sensor (a 3-axis acceleration sensor, a 3-axis angular velocity sensor, a 3-axis geomagnetic sensor) or a 3-axis acceleration and 3-axis angular velocity sensor.

[0155] The inertial measurement module can detect a three-channel acceleration signal corresponding to a three-axis acceleration sensor and a three-channel angular velocity signal corresponding to a three-axis angular velocity sensor through each sensor.

[0156] The electromyography module and inertial measurement module can detect detection signals simultaneously or with a time difference for each sensor, and the detected signal data can be transmitted in real time to the measurement value preprocessing unit described later via Bluetooth wireless communication.

[0157] Electromyography (EMG) signals and inertial measurement device (IMD) signals exhibit different patterns and signals for each individual, making them suitable for use as biometric security technologies that can authenticate individual users.

[0158] In step (S120), the user authentication device may receive the measured electromyography signal and inertial measurement device signal and execute a preprocessing process to reduce data loss, standard deviation, noise, disturbance, etc. due to communication transmission before analyzing the two signals.

[0159] The detailed method of preprocessing the above EMG signal and the above inertial measurement device signal will be described in detail later with reference to FIG. 10.

[0160] In step (S130), the user authentication device can register user information based on the preprocessed electromyography signal and inertial measurement device signal.

[0161] User information is information that can distinguish an individual, like a fingerprint, using the above-mentioned electromyography signal and the above-mentioned inertial measurement device signal.

[0162] In step (S140), the user authentication device can identify and authenticate an individual user using an artificial intelligence model.

[0163] At this time, the user authentication device may use a user identification artificial intelligence algorithm, including linear discriminant analysis (LDA), for user identification.

[0164] For example, user identification is performed by inputting user identification data into an artificial intelligence algorithm and classifying each data into a separate class based on a linear discriminant classification model.

[0165] Additionally, the user authentication device may use a user authentication artificial intelligence algorithm, including an ensemble classifier (RUSBoosted tree, Random Under-Sampling Boosting tree), for user authentication.

[0166] For example, user authentication performs classification based on an ensemble classifier by assigning certain data to one class and the rest to a separate class.

[0167] The user authentication device can learn a classifier by generating feature vectors of three types: a composite of EMG and IMU signals (EMG + IMU), EMG signals, and inertial measurement unit signals (IMU) signals, to compare the identification accuracy when using EMG signals and inertial measurement unit signals alone and together.

[0168] As a result of verifying the accuracy of user identification classification by inputting EMG signals and inertial measurement device signals into the artificial intelligence model, the authentication performance in the sitting posture (sit) and standing posture (stand) was the highest at 80.58% and 75.20%, respectively, when EMG signals and inertial measurement device signals were used together rather than individually, regardless of posture.

[0169] In addition, the accuracy of user personal authentication classification was confirmed by inputting EMG signals and inertial measurement device signals into the artificial intelligence model. As with the personal identification classification results, when EMG signals and inertial measurement device signals were used together in all postures, the average results were 90.63% and 89.87%, respectively, showing the highest authentication performance.

[0170] Accordingly, the present invention can extract individual-specific electromyography signals and inertial measurement device signals generated during a resting state when no specific movement is performed, and use them as inputs for linear discriminant analysis (LDA) and an ensemble classifier (RUSBoosted tree) to identify and authenticate a user.

[0171] In addition, this personal authentication device can measure electromyography signals and inertial measurement device signals in a resting state and use data synthesized from the two signals to achieve the highest authentication accuracy and performance in artificial intelligence-based personal identification and personal authentication.

[0172] In addition, the present invention can enable user authentication without spatiotemporal constraints by applying real-time analysis, and can easily identify individuals without separate actions in applications such as virtual reality and games by utilizing electromyography signals and inertial measurement device signals measured in a resting state.

[0173] In addition, the present invention has the advantage of improving convenience and practicality by allowing the user to perform personal authentication in a comfortable state without having to make a specific gesture and without requiring long-term repetitive training for a specific gesture.

[0174] Figure 10 is a flowchart illustrating a detailed method of preprocessing measurement values ​​according to an embodiment of the present invention.

[0175] In step (S121), the user authentication device can remove trials judged as outliers by using 2 standard deviations based on the number of samples measured in each trial, taking into account data loss that occurs due to the characteristics of Bluetooth transmission when measuring a signal.

[0176] If there are outliers, the learning algorithm may learn incorrect patterns or reduce the accuracy of the model. To prevent this, it is desirable to remove or replace outliers to improve model learning or enhance model performance.

[0177] In step (S122), the user authentication device can remove external noise introduced when measuring electromyography signals and inertial measurement device signals.

[0178] For example, the EMG signals measured in eight channels can be filtered by applying a band-pass filter for the frequency range of 1 to 240 Hz, and the acceleration and angular velocity (gyroscope) signals of an inertial measurement device measured along three axes (X, Y, Z) can be filtered by applying a band-pass filter for the frequency range of 0.1 to 15 Hz to remove external noise.

[0179] In step (S123), the user authentication device can extract time series features for the electromyography signal and the inertial measurement device signal.

[0180] For example, a time series feature extraction method can extract the time series feature using at least one of a root mean square error, a median absolute deviation, a variance, a zero crossing, a waveform length, and a mean absolute value.

[0181] In step (S124), the user authentication device can remove outlier data that is judged to have moved by two standard deviations of the time series feature means (e.g., root mean square, median absolute deviation, variance, etc.) to increase the reliability of the extracted time series feature data.

[0182] The present invention has the effect of learning an accurate pattern through a classification learning algorithm or improving the accuracy of a model and the performance of a model by performing a preprocessing process including outlier removal, external noise removal, time series feature extraction, and movement outlier removal on measured electromyography signals and inertial measurement device signals.

[0183] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single entity may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.

[0184] The scope of the present invention is indicated by the claims set forth below, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

[0185] The mode for carrying out the invention is described together with the best mode for carrying out the invention.

[0186] A resting state-based user authentication device and method using electromyography signals and inertial measurement device signals according to an embodiment of the present invention can provide personal identification and authentication by extracting a user's personal electromyography signal and inertial measurement device signal during a resting state in which no specific movement is performed and inputting the extracted signal into a linear discriminant analysis and an ensemble classifier. In applications such as virtual reality and games that utilize electromyography signals and inertial measurement device signals, personal identification can be easily performed without a separate operation.

Claims

1. In a user authentication device using electromyography signals and inertial measurement device signals, An electromyography (EMG) measurement module that measures EMG signals generated from the user's muscle movements while at rest; An inertial measurement module for measuring an inertial measurement device signal including acceleration and angular velocity signals of a moving object based on a sensor in the above resting state; A measurement value preprocessing unit that performs a preprocessing process to reduce data loss, standard deviation, noise, and disturbance due to communication transmission for the above-mentioned EMG signal and the above-mentioned inertial measurement device signal; A user information registration unit that registers user information based on the preprocessed electromyography signal and the inertial measurement device signal; A user information processing unit that identifies or authenticates a user based on the preprocessed electromyography signal and the inertial measurement device signal; and An artificial intelligence model that learns and classifies user identification and user authentication by receiving the above-mentioned electromyography signal and the above-mentioned inertial measurement device signal and using the artificial intelligence model. A user authentication device including:

2. In paragraph 1, The above measurement value preprocessing unit is, An outlier removal unit that removes outliers using a standard deviation for data loss occurring in wireless communication according to the above electromyography signal and the above inertial measurement device signal; An external noise removal unit that removes external noise introduced when measuring the above electromyography signal and the above inertial measurement device signal; A time series feature extraction unit that extracts time series features for the above EMG signal and the above inertial measurement device signal; and Including a motion outlier removal unit that removes motion outlier data based on the above-extracted time series feature data, A user authentication device that measures the electromyography signal and the inertial measurement device signal in a resting state.

3. In paragraph 2, The above outlier removal unit is a user authentication device that removes trials judged to be outliers using 2 standard deviations based on the number of samples measured in each trial.

4. In paragraph 2, The above external noise removal unit, Apply a band-pass filter to the frequency range of 1 to 240 Hz of the above-mentioned measured EMG signal, A user authentication device that removes external noise by applying a band-pass filter to the frequency range of 0.1 to 15 Hz of the measured inertial measurement device signal.

5. In paragraph 2, The above time series feature extraction unit, A user authentication device that extracts the time series features using at least one of root mean square, median absolute deviation, variance, zero crossing, waveform length, and mean absolute value.

6. In paragraph 2, The above motion outlier removal unit is, A user authentication device that removes outlier data that is judged to have moved by two standard deviations of the time series feature in order to increase the reliability of the feature data extracted from the above time series feature extraction unit.

7. In paragraph 1, The above user information processing unit, A user identification unit that identifies a user based on the electromyography signal and the inertial measurement device signal; and A user authentication unit that authenticates an individual user based on the above electromyography signal and the above inertial measurement device signal. A user authentication device including:

8. In paragraph 1, The above artificial intelligence model is, A linear discriminant classification model that identifies a registered user individual by inputting the above electromyography signal and the above inertial measurement device signal; and An ensemble classification model that authenticates registered users by inputting the above-mentioned electromyography signal and the above-mentioned inertial measurement device signal. A user authentication device including:

9. In paragraph 8, The above artificial intelligence model is, A user authentication device that applies five-fold cross-validation to prevent overfitting problems.

10. In a user authentication method using electromyography signals and inertial measurement device signals, A step of measuring electromyography signals generated from the movement of the user's muscles in a resting state; A step of measuring an inertial measurement device signal including acceleration and angular velocity of a moving object based on a sensor in the above resting state; A step of executing a preprocessing process for reducing data loss, standard deviation, noise, and disturbance due to communication transmission for the above electromyography signal and the above inertial measurement device signal; A step of registering user information based on the preprocessed electromyography signal and the inertial measurement device signal; and A step of learning and classifying user identification and user authentication using an artificial intelligence model by receiving the preprocessed electromyography signal and the inertial measurement device signal. A user authentication method including:

11. In paragraph 10, The step of executing the above preprocessing process is: A step of removing outliers using a standard deviation for data loss occurring in wireless communication according to the above electromyography signal and the above inertial measurement device signal; A step of removing external noise introduced when measuring the above electromyography signal and the above inertial measurement device signal; A step of extracting time series features for the above electromyography signal and the above inertial measurement device signal; and Including a step of removing motion outlier data based on the above extracted time series feature data, A user authentication method comprising measuring the above-described electromyography signal and the above-described inertial measurement device signal in a resting state.

12. In paragraph 11, A user authentication method in which the step of removing the above outliers removes trials judged to be outliers using 2 standard deviations based on the number of samples measured in each trial.

13. In paragraph 11, The step of removing the above external noise is: Apply a band-pass filter to the frequency range of 1 to 240 Hz of the above-mentioned measured EMG signal, A user authentication method for removing external noise by applying a band-pass filter to the frequency range of 0.1 to 15 Hz of the measured inertial measurement device signal.

14. In paragraph 11, The step of extracting the above time series features is: A user authentication method for extracting time series features using at least one of root mean square, median absolute deviation, variance, zero crossing, waveform length, and mean absolute value.

15. In paragraph 11, The step of removing the above motion outlier data is: A user authentication method for removing outlier data that is judged to have moved by two standard deviations of the time series feature in order to increase the reliability of the feature data extracted from the above time series feature extraction unit.

16. In paragraph 10, The above user identification or authentication is, A user identification unit that identifies a user based on the electromyography signal and the inertial measurement device signal; and A user authentication unit that authenticates an individual user based on the above electromyography signal and the above inertial measurement device signal. A user authentication method including:

17. In paragraph 10, The above artificial intelligence model is, A linear discriminant classification model that identifies a registered user individual by inputting the above electromyography signal and the above inertial measurement device signal; and An ensemble classification model that authenticates registered users by inputting the above-mentioned electromyography signal and the above-mentioned inertial measurement device signal. A user authentication method including:

18. In paragraph 17, The above artificial intelligence model is, A user authentication method that applies 5-fold cross-validation to prevent overfitting.

Images