Local user authentication using neural and neuro-mechanical fingerprints

Abstract

This application relates to local user authentication using neural and neuro-mechanical fingerprints. According to one embodiment, a method for locally verifying the identity of a user using an electronic device is disclosed. The method includes regenerating a neuro-mechanical fingerprint (NFP) in response to a micro-motion signal sensed at a body part. A match percentage of the neuro-mechanical fingerprint is determined in response to a plurality of authorized user calibration parameters. The match percentage is determined without using a calibration NFP previously used to generate the user calibration parameters. Access to the electronic device and its software applications is then controlled by the match percentage. If the match percentage is greater than or equal to an access match level, access to the electronic device is granted. If the match percentage is less than the access match level, access is denied. Subsequent access requires further regenerating of the NFP and determining its match percentage in response.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is a divisional application of the application patent application entitled "LOCAL USER AUTHENTICATION WITH NEURO-MECHANICAL FINGERPRINTS" having application number 201680019058.8, filing date February 4, 2016.

[0003] FIG. 1

[0004] This patent application claims the benefit of U.S. Provisional Patent Application No. 62 / 112,153, filed February 04, 2015, entitled "LOCAL USER AUTHENTICATION WITH NEURO-MECHANICAL FINGERPRINTS" by inventors Martin Zizi et al., and Non-Provisional Patent Application No. 15 / 013,875, entitled "LOCAL USER AUTHENTICATION WITH NEURO-MECHANICAL FINGERPRINTS" by inventors Martin Zizi et al. TECHNICAL FIELD

[0005] The present embodiments relate generally to user identification and authentication. BACKGROUND

[0006] Wide area network connections via the Internet interconnect many electronic devices, such as computers and mobile smart phones, with remote servers and remote storage devices, so that cloud computer services can be provided. More electronic devices are ready to be interconnected via the Internet as wireless transmitters / receivers are added to them.

[0007] Access to some electronic devices and databases by users is typically through a login name and password. As more portable electronic devices are used, such as laptop computers and mobile smart phones, in highly mobile computing environments, the correct authentication of people and devices becomes very important to determine authorized use and to reduce the risk of miscommunication of data. For example, as more mobile health electronic devices are introduced, the privacy of health data captured by the mobile health devices becomes very important. As more banking and payments are conducted using mobile electronic devices, authorized use becomes very important.

[0008] Authentication of a user using a local electronic device is typically performed by a remote server. Software applications executed by the local electronic device typically save the user's login name and password to make the electronic device and its software applications easier to use. Protecting local access to the user's electronic device has become increasingly important to protect the user and his / her login name and password in the event that the electronic device is lost or stolen. With electronic devices now being used to make payments such as credit card transactions, protection of local access to a user's electronic device has become even more important.

[0009] Reference will now be made to FIG. 1 , which describes various known behavioral and biometric recognition methods. Behavioral recognition methods relate to a user's behavior or his / her habits. Known biometric recognition methods relate to physical characteristics of a user, such as fingerprints, eye iris scans, veins, facial scans, and DNA.

[0010] Biometrics are used to better authenticate a user in order to provide more protection to a mobile electronic device. Biometrics have pursued both biometric-based methods (i.e., physical characteristics such as fingerprints, eye iris scans, veins, facial scans, DNA, etc.) and habit (behavioral) methods (typing or keystroke, handwriting signature, voice or tone variations). For example, a biometric aspect of the user, such as a fingerprint, is used to locally authenticate the user and restrict access to the electronic device. As another example, a hand shape or vein position can be used in conjunction with image analysis to locally authenticate the user and restrict access to the electronic device.

[0011] Behavioral aspects of a user can also be used to better authenticate a user in order to provide more protection to a mobile electronic device. Behavioral aspects relate to a personal profile of a user's unique habits, tastes, and behaviors. For example, the way a user signs his name is a unique behavioral aspect that can be used to verify the identity of the user.

[0012] However, neither known biometric nor known behavioral aspects of user recognition are foolproof. For example, known user data used for comparison can be stolen from a server and used without the user's knowledge. Additional hardware is typically required to capture biometrics from a user, which results in increased cost. With known added hardware, the risk can increase where a user can be forced by another user to involuntarily capture biometric data. Behavioral aspects can be somewhat intrusive and raise privacy concerns. Typically, sensing of behavioral aspects is performed using software, which requires the system to remain on, consuming energy that is typically stored in a rechargeable battery. Furthermore, biometrics using behavioral aspects typically require increased expense to maintain a remote centralized database. SUMMARY

[0013] Using biometrics with known biometric and / or known behavioral aspects of a user for recognition has some drawbacks.

[0014] A user authentication solution is needed that is easy to use, decentralized, so it can be used locally; substantially fail-safe to a skimmer, low power consuming for battery applications, and respects the user's privacy by protecting the user's data so it can be easily adopted.

[0015] The user authentication solution disclosed herein provides fail-safe authentication in a decentralized mobile environment while protecting the user's privacy and data. The user authentication solution disclosed herein employs different forms of tamper-resistant biometrics.

[0016] The user authentication solution disclosed herein can employ 3D sensors and signal processing to capture signals representing the user's neuromuscular micro-function that are translated into well-defined micro-movements ("micro-movement signals"). Next, signal processing algorithms and feature extraction are used to capture unique signal features in the micro-movement signals. These unique signal features associated with the user's neuromuscular micro-movements can be used to uniquely identify the user, somewhat akin to a fingerprint. Therefore, these unique signal features associated with the user's neuromuscular micro-movements are referred to herein as "neuromechanical fingerprints" (NFPs). The neuromechanical fingerprints can be used for local user authentication at a mobile electronic device, such as a laptop or smartphone, or at other types of electronic devices. Using the user's NFPs captured by the sensors and extracted by the algorithms, the user's habits or patterned behavior are avoided from being profiled and stored.

[0017] Embodiments of a user authentication solution that employs sensors and signal processing algorithms to generate and reproduce an authorized user's neuromechanical fingerprint (NFP) are disclosed herein through the accompanying drawings and detailed description.

[0018] 1. A method for locally authenticating an authorized user and controlling access to an electronic device, the method comprising: generating a neuromechanical fingerprint of a user in response to micro-movement signals sensed at a body part of the user using the electronic device; generating a match percentage of the neuromechanical fingerprint in response to authorized user calibration parameters; and controlling user access to the electronic device in response to the match percentage.

[0019] 2. The method of claim 1, wherein the neuro-mechanical fingerprint is a set of values of a plurality of unique features extracted from the micro-motion signals associated with neuromuscular micro-motions in the body part of the user. 3. The method of claim 2, wherein the body part of the user is a finger. 4. The method of claim 2, wherein the body part of the user is a hand. 5. The method of claim 1, wherein the match percentage is generated without a prior calibration neuro-mechanical fingerprint used to generate the user calibration parameters. 6. The method of claim 1, wherein if the match percentage is greater than or equal to an access match level, the user is identified as an authorized user and the authorized user is granted access to the electronic device. 7. The method of claim 6, further comprising, in response to the match percentage being less than or equal to an autonomous re-calibration level, notifying the authorized user to autonomously re-calibrate the authorized user calibration parameters. 8. The method of claim 7, further comprising, in response to the authorized user selecting to autonomously re-calibrate the authorized user calibration parameters, re-calibrating the authorized user calibration parameters, wherein the autonomous re-calibration level has a match percentage greater than the access match level. 9. The method of claim 7, further comprising, in response to the match percentage being less than or equal to a non-autonomous re-calibration level, re-calibrating the authorized user calibration parameters, wherein the non-autonomous re-calibration level has a match percentage greater than the access match level and less than the match percentage of the autonomous re-calibration level. 10. The method of claim 6, further comprising, sensing the micro-motion signals at the body part of the user.

[0020] 11. A method comprising: extracting a first neuro-mechanical fingerprint (NFP) associated with an authorized user; generating an authorized user calibration parameter associated with the authorized user in response to the first neuro-mechanical fingerprint; extracting a second neuro-mechanical fingerprint associated with an unknown user or the authorized user; determining a match percentage of the second neuro-mechanical fingerprint in response to the authorized user calibration parameter; and controlling access to an electronic device in response to the match percentage. 12. The method of claim 11, wherein the first NFP is a calibration NFP and the second NFP is a regenerated NFP associated with the authorized user, such that access to the electronic device is granted to the authorized user in response to the match percentage. 13. The method of claim 12, wherein the calibration NFP is a first set of values of a plurality of unique features extracted from a first micro-motion signal associated with the neuromuscular micro-motions in the hand of the authorized user; and the regenerated NFP is a second set of values of the plurality of unique features extracted from a second micro-motion signal associated with the neuromuscular micro-motions in the hand of the authorized user. 14. The method of claim 12, wherein the calibration NFP is a first set of values of a plurality of unique features extracted from a first micro-motion signal associated with the neuromuscular micro-motions in a finger of the authorized user; and the regenerated NFP is a second set of values of the plurality of unique features extracted from a second micro-motion signal associated with the neuromuscular micro-motions in the finger of the authorized user. 15. The method of claim 11, wherein the first NFP is a calibration NFP associated with the authorized user and the second NFP is a regenerated NFP associated with the unknown user, such that an attempt by the unknown user to access the electronic device is denied in response to the match percentage. 16. The method of claim 15, wherein the calibration NFP is a first set of values of a plurality of unique features extracted from a first micro-motion signal associated with the neuromuscular micro-motions in the hand of the authorized user; and the regenerated NFP is a second set of values of the plurality of unique features extracted from a second micro-motion signal associated with the neuromuscular micro-motions in the hand of the unknown user. 17. The method of claim 15, wherein the calibration NFP is a first set of values of a plurality of unique features extracted from a first micro-motion signal associated with the neuromuscular micro-motions in a finger of the authorized user; and the regenerated NFP is a second set of values of the plurality of unique features extracted from a second micro-motion signal associated with the neuromuscular micro-motions in a finger of the unknown user.

[0021] 18. An electronic device comprising: one or more sensors to sense motion in a body part of a user and generate a motion signal in three dimensions, the motion signal including a micro-motion signal associated with the neuromuscular micro-motions in the body part of the user; a storage device to store machine readable instructions and authorized user calibration parameters; a processor coupled to the sensors and the storage device, the processor to execute the machine readable instructions stored in the storage device to perform processes including: sampling the motion signal; filtering the sampled motion signal using a bandpass filter response associated with a frequency range of the micro-motion signal associated with the neuromuscular micro-motions; extracting a set of values for a plurality of unique features from the micro-motion signal, the set of values for the plurality of unique features forming a neuro-mechanical fingerprint of the user; and controlling user access to the electronic device in response to the neuro-mechanical fingerprint and the authorized user calibration parameters. 19. The electronic device of claim 18, wherein the one or more sensors are one or more touchpad sensors to receive one or more touches of a finger to generate one or more control signals in response to user access to the electronic device to further control the electronic device. 20. The electronic device of claim 19, wherein the one or more touchpad sensors are a plurality of touchpad sensors to receive a number to further control the electronic device. 21. The electronic device of claim 20, wherein the number is a personal identification number (PIN) to further control user access to the electronic device. 22. The electronic device of claim 20, wherein the electronic device is a smartphone and the number is a phone number to further control the smartphone to place a phone call. 23. The electronic device of claim 18, wherein one of the one or more sensors is an image scanner to capture a user image of a finger, hand, eye iris, or face and further control user access to the electronic device. 24. The electronic device of claim 18, wherein one of the one or more sensors is a touchpad sensor to receive a signature to further control user access to the electronic device. 25. The electronic device of claim 18, wherein the processor executes further machine readable instructions stored in the storage device to perform further processes including: generating a match percentage of the neuro-mechanical fingerprint in response to authorized user calibration parameters; wherein the user access is further controlled in response to the match percentage.26. The electronic device of clause 25, wherein the processor executes further machine- readable instructions stored in the storage device to perform further processes including comparing the match percentage to an access match level and denying access to an unauthorized user if the associated match percentage is less than the access match level. 27. The electronic device of clause 25, wherein the processor executes further machine- readable instructions stored in the storage device to perform further processes including comparing the match percentage to an access match level and granting access to an authorized user if the associated match percentage is greater than or equal to the access match level. 28. The electronic device of clause 27, wherein the processor executes further machine- readable instructions stored in the storage device to perform further processes including notifying the authorized user to autonomously recalibrate the authorized user calibration parameters in response to the match percentage being less than or equal to an autonomous recalibration level. 29. The electronic device of clause 27, wherein the processor executes further machine- readable instructions stored in the storage device to perform further processes including recalibrating the authorized user calibration parameters in response to the match percentage being less than or equal to a non-autonomous recalibration level. 30. The electronic device of clause 18, wherein the one or more motion sensors are one or more accelerometers to sense three-dimensional motion in the user's hand and the processor executes further machine-readable instructions stored in the storage device to perform further processes including compensating for gravity and compensating for an axis orientation of the electronic device that is different from a fixed ground axis.

[0022] 31. A physiological authentication controller for controlling access to an electronic device, the physiological authentication controller comprising: a signal processing module for generating a neuro-mechanical fingerprint of a user; an authentication classifier module communicatively coupled with the signal processing module, the authentication classifier module for generating a match percentage in response to the neuro-mechanical fingerprint and an authorized user calibration parameter; and an authentication controller module communicatively coupled with the authentication classifier module, the authentication controller module for controlling access to the electronic device in response to the match percentage and an access match level.32. The physiological authentication controller of claim 31, further comprising: a storage device communicatively coupled with the authentication classifier module and the authentication controller module, the storage device storing the authorized user calibration parameter and the access match level.33. The physiological authentication controller of claim 32, wherein the storage device further stores a recalibration level; and the authentication controller module causes an authorized user to recalibrate the authentication classifier module with an updated authorized user calibration parameter in response to the match percentage being less than or equal to the recalibration level.34. A method of generating a unique physiological identification of a user, the method comprising: sensing three-dimensional motion of a body part of a user to generate a three-dimensional motion signal, the three-dimensional motion signal including gross motion associated with large motions of the body part and neuromuscular micro-motions associated with motor control of the brain and quality control of the neuromuscular system; sampling the three-dimensional motion signal at a predetermined sampling frequency over a predetermined sampling period; filtering out unwanted signals in the three-dimensional sampled motion signal to form a three-dimensional sampled micro-motion signal representing the neuromuscular micro-motions of the user; and extracting a plurality of values of a plurality of predetermined features in the three-dimensional sampled micro-motion signal to form a neuro-mechanical fingerprint unique to the user.35. The method of claim 34, further comprising: signal processing the three-dimensional sampled micro-motion signal to determine the plurality of predetermined extracted features from the three-dimensional sampled micro-motion signal.36. The method of claim 34, wherein the signal processing is inverse spectral analysis of the three-dimensional sampled micro-motion signal from which the predetermined extracted features are selected.37. The method of claim 34, wherein the predetermined extracted features are frequencies at which the first N peaks occur in the micro-motion signal in the inverse spectral analysis of the three-dimensional sampled micro-motion signal, where N is greater than or equal to three.38. The method of claim 34, wherein the predetermined extracted features are times at which the first N peaks occur in the micro-motion signal in the inverse spectral analysis of the three-dimensional sampled micro-motion signal, where N is greater than or equal to three.39. The method of claim 34, wherein the first N peaks represent a large variance in peak-to-peak amplitude of the micro-motion signal.40. The method of claim 34, wherein the signal processing is a chaos algorithm that selects the three-dimensional sampled micro-motion signal from which to extract features. 41. The method of claim 34, wherein the unwanted signals include gross motion signals associated with the large motions of the body part. 42. The method of claim 34, wherein the unwanted signals include one or more tremor signals associated with a neurological disorder of the user. 43. The method of claim 34, wherein the body part is a finger. 44. The method of claim 34, wherein the motions of the body part are sensed by a sensor having an axis with a changeable orientation and producing an orientation angle relative to a fixed axis, and the method further comprises compensating the sampled micro-motion signals for the orientation angle of the sensor axis relative to the fixed axis. 45. The method of claim 44, wherein the sensor is an accelerometer affected by the gravitational force of the Earth, and the method further comprises compensating the sampled micro-motion signals for the gravitational effect. 46. The method of claim 45, wherein the body part is a hand.

[0023] 47. A method of authenticating a user, the method comprising: receiving a first user identifier of a user while using a sensor to sense motions of the user to produce a motion signal; in response to the motion signal, producing a neuromechanical fingerprint unique to the user; evaluating the neuromechanical fingerprint of the user against a database of N user calibration parameters respectively associated with N authorized users to determine a maximum matching percentage; and in response to the maximum matching percentage, determining whether the user is an authorized user of a system or an application by comparing the maximum matching percentage to an authorized user percentage. 48. The method of claim 47, further comprising: in response to authenticating the first user identifier and the neuromechanical fingerprint of the user, granting access to the system or application. 49. The method of claim 47, further comprising: in response to authenticating the first user identifier and the neuromechanical fingerprint of the user, producing a token; and transmitting the token to a server to obtain access to the server. 50. The method of claim 47, wherein the first user identifier of the user is a personal identification number. 51. The method of claim 47, wherein the first user identifier of the user is a fingerprint.

[0024] 52. A method comprising: selecting one or more touch sensitive devices of a user interface of an electronic device; in response to the selection of the one or more touch sensitive devices of the user interface, authenticating a user of the electronic device using a neuro-mechanical fingerprint; and in response to the authentication of the user and the selection of the one or more touch sensitive devices of the user interface, controlling the electronic device. 53. The method of solution 52, wherein the selection of the one or more touch sensitive devices of the user interface includes at least one of a selection of a power button, a selection of a function button, and a selection of a plurality of buttons to enter a number. 54. The method of solution 53, wherein the selection of the plurality of buttons is to enter a personal identification number to further authenticate the user of the electronic device. 55. The method of solution 53, wherein the selection of the plurality of buttons is to enter a telephone number to control the electronic device to make a telephone call. 56. The method of solution 52, wherein the selection of the one or more touch sensitive devices of the user interface is a selection of a function button; the function button includes an image scanner; and the method further comprises authenticating the user of the electronic device using an image of a finger. 57. The method of solution 52, wherein the authentication of the user of the electronic device using the neuro-mechanical fingerprint includes: sensing three-dimensional motion in a finger of the user using at least one of the one or more touch sensitive devices of the user interface to generate a three-dimensional motion signal; in response to the three-dimensional motion signal, generating the neuro-mechanical fingerprint; evaluating the neuro-mechanical fingerprint of the user to determine a maximum match percentage; and comparing the maximum match percentage to an authorized user percentage to authenticate the user of the electronic device.

[0025] 58. A method of logging into a system or application, the method comprising: authenticating a user using a first authentication means; further authenticating the user in response to a neuromechanical fingerprint generated from motion of the user captured by a sensor, the neuromechanical fingerprint being unique to the user; and granting access to the system or application in response to the first authentication means and the neuromechanical fingerprint authenticating the user. 59. The method of claim 58, wherein further authenticating the user using the neuromechanical fingerprint comprises: sensing three-dimensional motion of a body part of a user to sense a three- dimensional motion signal; generating the neuromechanical fingerprint unique to the user in response to the three-dimensional motion signal; receiving the neuromechanical fingerprint as a login identification; evaluating the neuromechanical fingerprint of the user against a database of N user calibration parameters respectively associated with N authorized users to determine a maximum matching percentage; and determining whether the user is an authorized user of a system or an application in response to the maximum matching percentage by comparing the maximum matching percentage to an authorized user percentage. 60. The method of claim 59, further comprising: identifying the user by the association of the N authorized users with the N user calibration parameters in response to the maximum matching percentage; and allowing the user identified as one of the N authorized users to use the system or the application. 61. The method of claim 58, wherein the first authentication means is an image scanner and the user is further authenticated by an image of a finger, a hand, an iris of an eye, or a face. 62. The method of claim 58, wherein the first authentication means is a touchpad sensor and the user is further authenticated by one or more keystrokes or a signature. 63. The method of claim 58, wherein the first authentication means is a microphone and the user is further authenticated by voice. BRIEF DESCRIPTION OF DRAWINGS

[0026] FIG. 2A to 2D FIG. 1 is a background diagram illustrating various behavioral and physiological identification methods.

[0027] FIG. 3A FIG. 2 illustrates an example of a Poincare phase scatter plot showing gravity- corrected three-dimensional accelerometer data captured from different users.

[0028] FIG. 3B FIG. 3 is a diagram illustrating the use of a server and storage local area network (cloud storage device) for remote authentication of a user via the Internet.

[0029] FIG. 4 FIG. 4 is a diagram illustrating local authentication of a user at a local electronic device.

[0030] FIG. 5is a chart illustrating authentication techniques that can be used to provide multi-factor authentication in conjunction with NFP authentication.

[0031] FIG. 6 is a chart to compare various biometric recognition techniques with NFP authentication.

[0032] FIG. 7 is a table of tremor types and associated frequencies and conditions.

[0033] FIG. 8A to 8C is a functional block diagram of an example of an electronic device that includes a sensor to capture micro-motion signals from a user and an NFP authentication system to control access in response to the micro-motion signals.

[0034] FIG. 9 is an NFP authentication system with different sensors to sense micro-motion from a finger or from a hand.

[0035] FIG. 10A is a waveform plot of acceleration measured at a user's hand to show the difference between gross motion and micro-motion.

[0036] FIG. 10B is a diagram illustrating device orientation and world orientation.

[0037] FIG. 11A is a diagram illustrating the use of principal vectors to convert 3D data samples in a data set from device orientation to world orientation.

[0038] FIG. 11B is a functional block diagram of a bandpass filter to filter out signals outside the frequency range of the desired tremor to capture micro-motion signals.

[0039] FIG. 11A is a functional block diagram of a high pass filter and a low pass filter to achieve the same result as FIG. 12A the bandpass filter.

[0040] FIG. 12B is a functional block diagram of a gravity high pass filter to filter out the effects of gravity.

[0041] FIG. 13A is a diagram illustrating coordinate transformation of data samples in a data set to move the center of gravity to the origin of the three axes in order to eliminate the effects of gravity.

[0042] FIG. 13B is a waveform plot of a raw, unfiltered sample set of three-dimensional acceleration data signals from a 3D accelerometer.

[0043] FIG. 13A is a 3D micro-motion signal in FIG. 14AThe waveforms shown are the original signal waveforms after preprocessing, bandpass filtering, and gravity compensation.

[0044] FIG. 14B This is a functional block diagram of the NFP certification controller.

[0045] FIG. 14A yes FIG. 15 The functional block diagram of the NFP certification classifier is shown in the figure.

[0046] FIG. 14A It is by FIG. 16 The diagram shows the access and recalibration controls performed by the authentication controller.

[0047] FIG. 17A It is a plot of the power spectral density of micromotion signals associated with tremor.

[0048] FIG. 13B Through the FIG. 17B The diagram shows a CEPSTRUM waveform generated by performing CEPSTRUM analysis on the micro-motion waveform.

[0049] FIG. 17A yes FIG. 18A to 18B The image shows a magnified view of the portion of the CEPSTRUM waveform plotted in the image.

[0050] FIG. 19A to 19B This illustrates instances of different NFPs for a single axis based on the frequencies of two different users.

[0051] FIG. 20 This illustrates instances of different NFPs for a single axis based on the time of two different users.

[0052] INTRODUCTION This describes a Hidden Markov (HMM) model that can be used as an NFP classifier model to determine the percentage of matches from NFP and authorized user-calibrated parameters. Detailed Implementation

[0053] Numerous specific details are set forth in the following detailed description of the embodiments to provide a thorough understanding. However, it will be apparent to those skilled in the art that the embodiments can be practiced without these specific details. In other instances, well-known laws, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0054] The embodiments include methods, apparatus, and systems for forming and utilizing neuromechanical fingerprints (NFPs) to identify and authenticate users.

[0055] FIG. 2A to 2D

[0056] Certain user motions are habitual or part of a user's motion repertoire. Signing a document by a user is, for example, a habitual motion in the context of a user's repertoire of motions. Typically analyzed signature motions are large motions or gross motions made by a user using a writing instrument. For example, from the large motions of a signature, one can determine, for example, using the eyes, whether the writer is left-handed or right-handed.

[0057] While these large motions can be useful, there are also micro-motions (very small motions) made by a user when signing, making other motions, or simply at rest and not making motions. These micro-motions are neuro-derived or neural-based and are not visible to the eye. These micro-motions of a user are due to each person's unique neuromuscular anatomy and can also be referred to herein as neuro-derived micro-motions. These micro-motions are also related to the process of motor control from a person's motor cortex all the way to his / her hand. Using one or more sensors, signal processing algorithms, and / or filters, electronic signals ("motion signals" and "micro-motion signals") can be captured that include the neuro-derived micro-motions of a user. Of particular interest is the micro-motion electronic signal that represents the micro-motions of a user within the motion signals.

[0058] The micro-motions of a user are related to the cortical and subcortical control of motor activity in the brain or elsewhere in the nervous system of the human body. Like a mechanical filter, a person's specific musculoskeletal anatomy can influence the micro-motions of a user and contribute to the motion signals that include the micro-motions of a user. The motion signals captured from a user can also reflect part of the proprioceptive control loop that includes the brain and proprioceptors present in the human body of a user.

[0059] In analyzing the motion signals properly for the micro-motion signals that represent the micro-motions of a user, the resulting data can yield unique and stable physiological identifiers, more specifically neural identifiers, that can be used as unwritten signatures. These unique identifiers are the neuro-mechanical fingerprints of a user. The neuro-mechanical fingerprints can also be referred to herein as NeuroFingerPrints (NFPs).

[0060] User motion can be captured by various electronic sensors, including gross motion of the user and micro-motion of the user. For example, electronic touch sensors or electronic accelerometers can be used to capture gross motion and micro-motion of the user. U.S. Patent Application No. 13 / 823,107, by Geoff Klein, filed January 5, 2012, incorporated herein by reference, describes how accelerometers can be used to capture time-varying motion data of a user as the user operates an electronic device and to produce a motion profile that can be used to recognize the user. U.S. Patent Application No. 13 / 823,107 utilizes a profile or motion habit. The embodiments disclosed herein neither establish nor rely on a motion profile. The embodiments disclosed herein extract signals related to the quality control mechanisms of the nervous system. U.S. Patent Application No. 13 / 823,107 ignores the user's neuro-derived micro-motion that is of interest to the embodiments disclosed herein. Further, in producing a neuro-mechanical fingerprint, gross motion signals are suppressed or filtered out to capture micro-motion signal components. When using a three-dimensional accelerometer to capture micro-motion of a user, gross motion signals need to be filtered out from the micro-motion signal data. Vectors due to gravity are also ignored in U.S. Patent Application No. 13 / 823,107. When using a three-dimensional accelerometer, gravity is compensated for or filtered out in producing micro-motion signals.

[0061] Referring now to FIG. 2A to 2C , examples of three-dimensional Poincare phase scatter plots are shown for four different users. Each three-dimensional Poincare phase scatter plot shows a pattern 200A-200D of gravity-corrected three-dimensional accelerometer data.

[0062] Raw accelerometer data is obtained for a user using their hand to make gross motions using the same electronic device over the same time period. The electronic device has a three-dimensional accelerometer sensor to capture raw accelerometer data in three dimensions, X, Y, and Z.

[0063] Signal processing is performed on the raw accelerometer data to filter or suppress unwanted signals, correct for gravity, and extract signals (micro-motion signals) representing neuro-derived micro-motion of the user's hand or finger. The three-dimensional x(t), y(t), z(t) of the micro-motion signals can be further processed into phase x(t), y(t), z(t) and plotted in a three-dimensional Poincare phase scatter plot over a sampling time period for the user.

[0064] As can be seen in NFP neural algorithmAs can be readily seen, the patterns 200A to 200D in each of the Poincare phase scatter plots generated from the user's neuro-derived motion are substantially different. For example, the centroids 202A to 202D of each pattern 200A to 200D are different. Other characteristics of each pattern 200A to 200D are also different for each user. Thus, the patterns of neuro-derived motion are unique to each user and can be used to uniquely identify the user.

[0065] The unique patterns 200A to 200D and NFP in the generated Poincare phase plots are generally stable. Thus, generally each time the user touches or moves the sensor, the unique patterns 200A to 200D and NFP can be repeatedly sensed over a sampling time period and then compared to the initially calibrated NFP using an algorithm to authenticate the identity of the user. However, if there is a progressive neuromuscular disease, the user's unique patterns 200A to 200D are less stable. If so, the neural algorithm can be periodically recalibrated to compensate for disease progression. For example, an elderly person with a neuromuscular disease can easily recalibrate the neural algorithm each week depending on the drift cutoff for the initial calibration to the NFP. As another example, a user who is being treated with a motion-altering drug can also periodically recalibrate the NFP neural algorithm. For most users, recalibrating the NFP neural algorithm after the initial calibration will be rare.

[0066] The analysis of the motion signals to micro-motions employs a neural algorithm to obtain a neural mechanical fingerprint (NFP) of the user. Thus, this neural algorithm can be referred to herein as the NFP neural algorithm.

[0067] Touch recognition

[0068] The NFP neural algorithm is not an algorithm that uses neural network training, classifiers, or deep learning. The NFP neural algorithm is an algorithm that specifically collects, isolates, and analyzes signals from the human nervous system and its interaction with the body parts connected to the nervous system, such as the muscle system or gland cells, skin. Activities of interest to analyze the micro-motions signals include motion control originating from our nervous system, sensory inputs, and stress responses that are unique to each person's anatomy and nervous system.

[0069] Consider a user who, for example, moves his or her hand. For example, a motion algorithm can record the physical motion over time and analyze its speed, distance, direction, and force. A neural algorithm collects, isolates, and analyzes the neural activity over time related to the hand motion from electronic signals, such as motion signals including micro-motions signals, without using a brain scanner to perform a brain scan.

[0070] While kinematic motion of a body part can be analyzed using an electronic accelerometer for micro-motions, touch by one or more fingers to an electronic touch sensor, such as a touchpad, can be analyzed, for example. The touch sensor can be used to generate a micro-motion signal representing micro-motions over time as a finger touches the touch sensor. A touch sensor can generate a three-dimensional signal representing X and Y positions of a finger relative to the touch sensor and a Z position representing pressure applied to the touch sensor. Micro-motions can be included as part of each of the three-dimensional position signals. With a touch sensor, gravity is generally not a factor that needs to be corrected for.

[0071] By focusing on micro-motion signals rather than gross motion signals, a touch sensor can be used in conjunction with a neural algorithm to better mimic a human cognitive interface in a machine. This can improve human-machine interfaces. Consider, for example, a human cognitive interface between a husband and wife or between people who are close to one another. When a husband touches his wife's arm, she often can recognize from the feel of that touch that it is her husband touching her because she is familiar with his touch. If the touch feels unique, then a person often can recognize from that unique feel what is touching him / her.

[0072] FIG. 1

[0073] The technique of using a touch sensor to sense and capture micro-motions and using a neural algorithm to analyze NFPs can be referred to herein as touch recognition. A system that includes touch recognition to authenticate and / or identify a user can be referred to herein as a touch recognition system. With respect to Local and remote user authentication Touch recognition using NeuroFingerPrint is a physiological user identifier. Touch recognition is not an anatomical user identifier. Touch recognition is not a behavioral user identifier. In contrast to anatomical identification methods that are fixed attributes of a user, physiological identification methods relate to the functional body of a user. Behavioral identification methods relate to the behavior or habits of a user.

[0074] Using a touch sensor and a neural algorithm in user authentication has inherent advantages. The user interface for user authentication can be intuitive and related to spontaneous behavior of a human user with only minimal training if any training is needed at all. The user experience of the user authentication process can be smooth. The user does not need to perform any specific user authentication tasks beyond those normally associated with use of the user's electronic device. For example, after calibration, the user only needs to hold his / her smart phone, for example, to make a phone call, so that the accelerometer and neural algorithm perform the user authentication process.

[0075] The NeuroFingerPrint can be captured non-intrusively by touch sensors and NFP neural algorithms. The NeuroFingerPrint can be isolated from other user motions. Thus, one or more touch sensors used to capture keystrokes or button presses can also be used to capture micro-motions of a user and generate micro-motion signals and NeuroFingerPrints in response thereto. For example, a user can simply need to select one or more buttons (e.g., dial a phone number to make a call, enter a personal identification number, select a function button, or a power button) such that one or more touch sensors underneath and neural algorithms perform a user authentication process. With little or no change in the user interface, one or more touch sensors associated with a keypad or button can be used to capture micro-motions and generate micro-motion signals as a user presses the keypad or button. For example, a NeuroFingerPrint (NFP) can be generated by a NeuroFingerPrint (NFP) neural algorithm while a user enters a personal identification number (PIN) of the user with no change in the user experience. Alternatively, a NeuroFingerPrint (NFP) can be generated by a NeuroFingerPrint (NFP) neural algorithm while a power button or function button (e.g., home button) is pressed. In this case, the NeuroFingerPrint (NFP) can be used for user authentication without a PIN number. Alternatively, a NeuroFingerPrint (NFP) can be used in conjunction with a PIN number for user authentication with little or no change in the user interface.

[0076] FIG. 3A to 3B

[0077] Referring now to FIG. 3A . FIG. 3B Remote authentication of a user is described. FIG. 3B Local authentication of a user is described.

[0078] Touch recognition facilitates local authentication of a user at a local electronic device, such as shown in FIG. 3A . NFP user authentication does not require use of a large remote database. An NFP user authentication system can be self-contained within the local electronic device, such as a mobile smart phone. As with remote authentication, login IDs and passwords for user authentication do not need to be sent over the Internet to a server to authenticate a user. Touch recognition is non-centralized and can be self-contained within the local electronic device.

[0079] In the case of remote authentication of a user, such as shown in FIG. 3B , a user is open to risks associated therewith. In the case of local authentication, such asFIG. 3B As demonstrated in the background, a user can be isolated from the Internet during user authentication to increase security. A token representing local authentication of the user can be passed out of the local electronic device to a local server or a remote server via, for example Multi-factor authentication and NeuroFingerPrint The Internet cloud as demonstrated in the background passes a token representing local authentication of the user from the local electronic device to a local server or a remote server.

[0080] While touch recognition facilitates local authentication, touch recognition can be easily combined with a remote authentication technique or another local authentication technique to provide an additional level of security.

[0081] FIG. 4

[0082] Referring now to Comparison of biometric identification technologies A user authentication system can employ multiple factors to authenticate a user. Multi-factor authentication combines two or more authentication techniques. Previously, multi-factor authentication was not user-friendly and so it was avoided. User-friendly authentication methods are otherwise generally preferred, but result in a trade-off of security.

[0083] As referred to herein, touch recognition or recognition using an accelerometer can be combined with a remote authentication technique or another local authentication technique such that multi-factor authentication is more user-friendly.

[0084] Touch recognition can be combined with a password, a personal identification number, a pattern, or another thing that a user knows or remembers. Alternatively, touch recognition can be combined with a token, a smart card, a mobile token, an OTP token, or some other user authentication device that a user has. Alternatively, touch recognition can be combined with an anatomical or behavioral user biometric. For example, neural-based touch recognition can be easily combined with fingerprint recognition to provide a user-friendly multi-factor authentication system.

[0085] The banking industry is particularly interested in avoiding additional burden on a user when user authentication is required. Furthermore, the banking industry is interested in providing a high performance user authentication system that is reliable and difficult to circumvent. Neural-based touch recognition can be easily combined with other authentication techniques that cause minimal burden on a user. Furthermore, the neural-based touch recognition disclosed herein can be substantially reliable and extremely difficult to circumvent.

[0086] FIG. 5

[0087] Referring now to Privacy protection, a chart illustrating a comparison matrix of various biometric identification technologies. A relative ranking is used to compare user authentication using NFP to other biometric user identification technologies. The relative ranking is based on seven grading criteria. Each criterion can be graded as high (H), medium (M), or low (L) for each biometric identification technology. From left to right, the seven grading criteria are persistence, collectability, performance, acceptability, circumvention, uniqueness, and universality. NFP ranks higher than other biometric identification technologies for the following reasons.

[0088] NFP is a high-ranking universal technology. Every person will have a unique nervous system from which NFP can be captured.

[0089] NFP is a highly unique user authentication technology that distinguishes users. NFP reflects a user's motor control over the user's body. Motor control is related to brain activity that controls muscle tension in the hand. Even twins will have different motor control because they will have experienced different life experiences and have different motor skill training. For example, twins will learn to ride a bicycle differently, learning different motor control over the muscles that will be reflected in different NFP.

[0090] NFP is a relatively highly persistent or stable technology because it is related to each person's anatomy. Evolutionary processes that can affect the stability of NFP, such as neurological diseases, are rare. Even in those rare cases, neurological evolutionary processes are typically slow enough that the NFP user authentication system can be recalibrated. NFP for touch recognition is different for a user's left hand and a user's right hand. However, the NFP is specific to each user. A user can be consistent in which hand / finger he / she uses to hold / touch an accelerometer-enabled device / touch sensor-enabled device. Alternatively, multiple NFPs for the same user can be calibrated using the NFP user authentication system. In this case, touch of either hand or multiple fingers can authenticate the user of the electronic device.

[0091] Some neurologically friendly drugs can change the scanned NFP. However, the NFP user authentication system can be recalibrated in order to compensate.

[0092] Alcohol can temporarily change the scanned NFP so that it does not match the calibrated NFP. This can be advantageously used to prohibit driving a car under the influence of alcohol. For example, NFP touch technology can be implemented in a car key or a car start button. If the scanned NFP is not affected by alcohol, then the NFP touch technology allows the key or start button to turn on the engine to drive the car. If the scanned NFP is affected by the user's drinking, then the NFP touch technology can prohibit the key or start button from starting the car engine.

[0093] NFPs are relatively easy to collect or capture because accelerometers have become common in mobile devices. Accelerometer values are frequently sampled using a clock every microsecond or so to capture the user's micro-movements. In addition, accelerometers are relatively sensitive. Accelerometers can typically sense sub-micro level accelerations so that, for example, they can capture micro-movements of a user's hand. Various three-dimensional accelerometers can be used to capture micro-movements of a user's hand. Various touch sensors can be used to capture micro-movements and type (whether capacitive or resistive is irrelevant).

[0094] The performance of NFP authentication using NFPs is high. Using non-optimal mathematical algorithms as the neural algorithm can achieve a recognition success rate between 93% and 97%. More appropriate and more accurate mathematical algorithms that consider time and trajectory variables can achieve a higher recognition success rate to reach 100% recognition of the correct user. In the event that the NFP authentication system fails to recognize the correct user, an alternative user authentication system, such as a PIN, can be used in parallel with the NFP authentication. In contrast, most biometrics (except for real fingerprint or iris scans) have low performance with a higher failure rate that can range between 18% and 20%, providing a successful recognition rate between 80% and 82%.

[0095] NFP technology should have a high level of acceptability because it is a user-friendly authentication method. NFP user authentication systems can be quite transparent or smooth. However, acceptability is also governed by how the NFP user authentication system is designed and sold, as well as how it is supported by appropriate security measures and independent audits.

[0096] NFPs are difficult to circumvent, so they have a high circumvention factor rating. When the NFP authentication system resides in an electronic device and is not available to the Internet, hacking, electronic spoofing, or reverse engineering is almost impossible to gain access.

[0097] Initially, a calibration NFP file is electronically generated for a user by the NFP user authentication. The calibration NFP file is typically encrypted and saved locally on the electronic device and thus is not available to the Internet. However, when stored elsewhere (such as a storage device in a storage area network or a storage device associated with an authentication server), the calibration NFP file should be encrypted to provide higher security.

[0098] NFP user authentication does not simply perform a file comparison to grant access. The NFP must be regenerated from the user's body in real time to gain access. This prevents hackers from using stolen files to gain access to and authorization for electronic devices by using a stand-in to mislead input or electronic deception or phishing for information. To circumvent NFP user authentication at the service level, a hacker would have to perform data acquisition simultaneously and in parallel with an authorized user while the user is accessing a service that requires authentication or login. To circumvent NFP user authentication at the device level, a hacker would have to mimic the neurological control of an authorized user to obtain NFP data and then key it into a device to gain access. A hacker can steal calibrated NFP values from an authorized user's device, but then the hacker would have to incredulously reproduce the user's micro-movement input into a second electronic device using the same calibrated NFP authentication system. Thus, electronic devices with NFP authentication systems can be extremely difficult to circumvent. Mobile electronic devices with NFP authentication systems can eventually be trusted as gatekeepers that provide real-time verified identification of a user.

[0099] Only when a user utilizes an electronic device, a scanned NFP is subsequently generated in real time, where the calibrated NFP only adjusts the calculation. Without a correct user generating a calibrated NFP file, the NFP authentication system denies local access to the electronic device. If the correct user is not a living body, then the NFP authentication system will deny access because micro-movement signals of the correct user cannot be generated. Thus, circumvention of NFP authentication technology is difficult. In contrast, classic fingerprints have a high level of difficulty for circumvention, however, even classic fingerprints can be stolen and can be reverse engineered to log into a fingerprint authentication system.

[0100] In summary, NFP technology for biometrics for user authentication systems ranks high compared to other biometric technologies.

[0101] Micro-motions and tremors

[0102] Mobile electronic devices are being used in conjunction with health and fitness applications. In some cases, mobile electronic devices can be used as a user interface to connected medical devices. Rules and laws in several countries regulate the privacy of a user's medical condition. Thus, the protection of a user's medical condition and medical data has become increasingly important.

[0103] NFP authentication systems can be used to help protect user data that can be stored or input into an electronic device. NFP authentication systems can be used to help comply with medical data laws and regulations that are in effect in several countries.

[0104] NFP data generated from micro-movements due to a user's nervous system can be considered a form of medical data. NFP authentication systems can be implemented to effectively protect calibrated NFP data stored in an electronic device or elsewhere when used for authentication purposes.

[0105] FIG. 6

[0106] NFPs are generated in response to micro-movements associated with a type or form of tremor. Tremor is an involuntary, rhythmic muscle movement that causes oscillations in one or more parts of the human body. Tremor can or can not be visible to the naked eye. Visible tremor is most common in middle-aged and older adults. Visible tremor is sometimes seen as a disorder in a part of the brain that controls one or more muscles or a specific area of the body, such as the hands and / or fingers.

[0107] Most tremors occur in the hands. Thus, tremor with micro-movements can be sensed when holding a device with an accelerometer or by finger touch to a touchpad sensor.

[0108] There are different types of tremor. The most common form or type of tremor occurs in healthy individuals. Typically, healthy individuals do not notice this type of tremor because the movements are too small and can occur while performing other movements. The micro-movements of interest associated with a type of tremor are too small to be visible to the naked eye.

[0109] Tremor can be activated under various conditions (resting, postural, kinetic) and can generally be classified as resting tremor, kinetic tremor, postural tremor, or movement or intention tremor. Resting tremor is a tremor that occurs when the affected body part is not active but is against gravity. Kinetic tremor is a tremor due to voluntary muscle activation and includes numerous tremor types, including postural tremor, movement or intention tremor, and task-specific tremor. Postural tremor is related to positioning a body part against gravity (such as extending an arm away from the body). Movement or intention tremor is related to goal-directed movements and non-goal-directed movements. An example of a kinetic tremor is the movement of a finger to one's nose, often used to detect drivers under the influence of alcohol while driving. Another example of a kinetic tremor is the movement of lifting a cup of water from a table. Task-specific tremor occurs during very specific movements, such as when using a pen or pencil to write on paper.

[0110] Tremor, whether visible to the eye or not, is seen to originate from some pool of oscillatory neurons, some brain structure, some sensory reflex mechanism, and / or some neuro-mechanical coupling and resonance within the nervous system.

[0111] While numerous tremors have been described as being physiological (without any disease) or pathological, it is believed that tremor amplitude is not very useful for classifying them. However, tremor frequency is of interest. Tremor frequency allows it to be used in a useful way to extract the signal of interest and to produce a unique NFP for each user.

[0112] Electronic device with NFP authentication A table illustrating various tremor types, their activation conditions, and the expected motion frequency attributed to a given tremor type.

[0113] Numerous pathological states, such as Parkinson's disease (3 to 7 Hz), cerebellar disease (3 to 5 Hz), dystonia (4 to 7 Hz), various neuropathies (4 to 7 Hz), cause motion / signals to be of lower frequency, for example, 7 Hertz (Hz) and lower. Since pathological states are not common to all users, motion / signals of these frequencies are not useful for producing NFPs and need to be filtered out. However, some embodiments disclosed herein are used to specifically focus on making those pathological signals a way to record, monitor, follow the disease to determine if the health condition is sound or deteriorating.

[0114] Other tremors, such as physiological, essential, orthostatic, and enhanced physiological tremor, can occur in normal health conditions. These tremors are not in themselves a disease. Thus, they are generally present in the population at large. Physiological tremor, as well as other tremors that are common to all users, are of interest because they produce micro-motions at frequencies in the range of 3 to 30 Hz or 4 to 30 Hz. The tremors can be activated when a muscle is used to counteract gravity to support a body part. Thus, holding an electronic device in one's hand to counteract gravity to support the hand and arm can produce a physiological tremor that can be sensed by an accelerometer. Touching a touchpad of an electronic device with a finger of a hand that is used to counteract gravity to support it can produce a physiological tremor that can be easily sensed by a finger touchpad sensor.

[0115] Essential tremor of the motion type can occur and be sensed when a user has to enter a PIN or login ID to gain access to a device or cell phone. The frequency range of essential tremor is between 4 to 12 Hz, which can be reduced to a frequency range of 8 to 12 Hz to avoid sensing tremors attributed to very rare pathological states.

[0116] For physiological tremor (or enhanced physiological tremor, supra with greater amplitude), the coherence between different body sides is low. That is, the physiological tremor on the left body side is very incoherent with the physiological tremor on the right body side. Thus, it is expected that the tremor in the user's left hand or left finger will be different from the tremor in the right hand or right finger. Thus, the NFP authentication system will need the user to be consistent in using the same side hand or finger for authentication; or alternatively, multiple authorized user calibration parameter sets (one for each hand or one for each finger) will be used to extract the NFP.

[0117] Motion with higher frequencies of interest can be considered noise. Thus, it is desirable to filter out signals with frequencies above the maximum in the desired range (e.g., 12 Hz or 30 Hz) in the original motion signal. Thus, the frequency signal range from 8 Hz to 12 Hz and / or 8 Hz to 30 Hz contains useful information about the micro-motions that can be used to generate the NFP.

[0118] The original signal captured by a finger touchpad sensor in an electronic device or by an accelerometer of a handheld electronic device can have several unwanted signal frequencies in it. Thus, one type of filtering responsive to filtering out signals outside of the desired frequency range can be used to obtain the micro-motion signal from the original electronic signal. Alternatively, isolation / extraction means for signals in the desired frequency range can be used to obtain the micro-motion signal from the original electronic signal. For example, a finite impulse response bandpass filter (e.g., 8 to 30 HZ bandpass) can be used to select the low signal frequency range of interest in the original electronic signal sensed by the touchpad or accelerometer. Alternatively, a low pass filter (e.g., 30 Hz cutoff) and a high pass filter (e.g., 8 Hz cutoff) or a high pass filter (e.g., 8 Hz cutoff) and a low pass filter (e.g., 30 Hz cutoff) can be combined in series to achieve similar results.

[0119] FIG. 7

[0120] Referring now to FIG. 8A to 8C , a functional block diagram of an electronic device 700 with NFP authentication is illustrated. The electronic device 700 can be, for example, a smart cellular phone or other handheld type of portable or mobile electronic device that requires access control. The electronic device 700 includes one or more three-dimensional (3D) sensors that can be used to capture original 3D electronic signals that include original 3D micro-motion signals that can be used for NFP authentication.

[0121] The electronic device 700 can include the following devices coupled together: a processor 701, a storage device (SD) 702, a power source 703 (e.g., a rechargeable battery), a button / pad 704, a keypad 705, a three-dimensional (3D) accelerometer 706 (optional), a display device 707, one or more radios 708, and one or more antennas 709. If the electronic device does not have a three-dimensional (3D) accelerometer 706, then one or more of the button / pad 704, the keypad 705, and the display device 707 are touch sensitive, thus they can be used to capture 3D raw electronic signals that include 3D micro-motion signals. In the case that the electronic device 700 is a smart phone, the electronic device includes a microphone and a speaker coupled to the processor. The speaker can be used to capture voice samples as an additional authentication means.

[0122] The storage device 702 is preferably a non-volatile type of storage device that stores data, instructions, and (possibly) other information (e.g., user calibration parameters) in a non-volatile manner, thus it is not lost when the electronic device goes into a sleep state to conserve power or is turned off completely. The storage device 702 stores software application instructions 712 for the NFP authentication system, NFP instructions and (possibly) NFP data 714, and user doodle pad data 716 for a user. The NFP instructions and NFP data 714 are separated from the user doodle pad data 716 and the software application instructions 712 for security reasons so that it is not accessible to all users. The storage device 702 is coupled to the processor 701 so that data and instructions can be read by the processor and executed to perform the functions of the software application. The NFP instructions and NFP data 714 (if any) are read by the processor to execute the NFP authentication controller module and perform the functions needed to provide NFP authentication.

[0123] The one or more radios 708 are coupled to and between the processor 701 and the one or more antennas 709. The one or more radios 708 can wirelessly receive and transmit data via a wireless network. The one or more radios 708 can include a Wi-Fi radio for a local area wireless (Wi-Fi) network, a cellular radio for a cellular telephone network, and a Bluetooth radio for a Bluetooth wireless connection. Software applications can be executed by the electronic device that require authentication to a remote server. The NFP authentication system is used to grant access to the electronic device itself. However, the NFP authentication system can also be used by software applications to verify or authenticate the identity of an authorized user.

[0124] To recharge the battery or provide an alternative power source, a power connector and / or a combined power / communication connector (e.g., a Universal Serial Bus connector) 722 can be included as part of the electronic device 700. The connector 722 can be coupled to the processor 701 for data communication and to the power source 703 to recharge the battery and / or to provide an alternative power source to the electronic device 700. For wired connectivity, the electronic device 700 can further include a network interface controller and a connector 724 coupled to the processor, such as an Ethernet controller and an RJ-45 port connector.

[0125] Most portable or mobile electronic devices now include a 3D accelerometer 706. Traditionally, the 3D accelerometer 706 has been used to determine the orientation of the electronic device. However, the 3D accelerometer 706 can also be used to capture micro-motions of the hand of the user holding the electronic device 700. In this case, the 3D accelerometer data is captured by the accelerometer 706, sampled, pre-processed, and then provided to the NFP authentication controller.

[0126] If the accelerometer is not available in the electronic device, then one or both of the button / pad 704 and the keypad 705 are touch sensitive in three dimensions including X, Y, and Z in order to be used for NFP authentication. The Z axis is the axis normal to the button and keypad and pressure can be applied by a finger to select the underlying function of the button or key of the keypad in order to control the electronic device. The button / pad 704 can be, for example, a power button used to turn the electronic device on / off. The button / pad 704 can be, for example, a home button that causes the electronic device to enter a "home" or initial user interface state.

[0127] One or both of the touch sensitive button / pad 704 and / or the touch sensitive keypad 705 can be used to capture three-dimensional raw micro-motion signals that can be used for NFP authentication.

[0128] If the display device 707 is a touch sensitive display device, then the keypad 705 or button 704 can be displayed on the touch sensitive display device 707 from which three-dimensional raw micro-motion signals can be captured.

[0129] The NFP authentication controller 810A-C, 810 in FIGS. 14A, 14B can be formed by the processor 701 executing instructions re-calle FIG. 8A from firmware / software 712 stored in the storage device 702.

[0130] Referring now to FIG. 8B , a functional block diagram showing part of an NFP authentication system with a three-dimensional touch sensitive button / pad 704. The NFP authentication system further includes an NFP authentication controller 810A coupled to a touch sensor 801 of the three-dimensional touch sensitive button / pad 704.

[0131] The three-dimensional touch sensitive button / pad 704 includes a 3D touch sensor 801 proximate the surface of the pad 704 to sense position as well as X, Y finger position and changes in finger pressure Z. The three-dimensional touch sensitive button / pad 704 typically includes a function button switch 814 to generate a function control signal sent to the processor. The 3D touch sensor 801 generates raw three-dimensional displacement signals including micro-motions sensed at the user's finger 890 and indicated by the 3D arrow 850.

[0132] After some pre-processing, the NFP authentication controller 810A receives data samples representing the micro-motions sensed by the touch sensor 801 and generates the NFP of the user. In response to the NFP and stored user calibration parameters trained by and associated with the authorized user, the NFP authentication controller 810A classifies the NFP and generates a match percentage value. In response to the match percentage value and a predetermined acceptable match percentage, the NFP authentication controller 810A can generate an access grant signal 1449 granting access to the electronic device to the authorized user touching the touch sensor.

[0133] Referring now to FIG. 8C , a functional block diagram of an NFP authentication system with a three-dimensional touch sensitive keypad 705 is shown. The touch sensitive keypad 705 includes a matrix of an M x N array of 3D touch sensors 801AA-801MN, each of which can sense position as well as X, Y finger position and changes in finger pressure Z of the finger 890 and generate raw micro-motion data. Each pad can include a function button switch 814 to generate a plurality of function control signals while capturing raw micro-motion sensor data by the 3D touch sensors 801AA-801MN.

[0134] The signals from the M x N array of 3D touch sensors 801AA-801MN are pre-processed and sampled, with the sampled data coupled to the NFP authentication controller 810B. In granting access, the NFP authentication controller 810B behaves similarly to the NFP authentication controller 810A. However, the NFP authentication controller 810B can be trained to support slightly different signals from different touch sensors in the matrix of the M x N array of 3D touch sensors 801AA-801MN.

[0135] Referring now to FIG. 7 , a functional block diagram of an NFP authentication system with a three-dimensional accelerometer 706 in the electronic device 700 is shown. The user hand 899 holds the electronic device 700 including the three-dimensional accelerometer 706. The three-dimensional accelerometer 706 is coupled to the NFP authentication controller 810C.

[0136] While the electronic device 700 is stably held in his / her hand 899, micro-motions in the user's hand 899 and indicated by the 3D arrow 852 are sensed by the 3D accelerometer 706. As the electronic device 700 held in the user's hand is moved to adjust position (e.g., from a pocket to the user's ear), undesired micro-motions in addition to the micro-motions are also sensed by the 3D accelerometer 706. As explained further herein, these undesired micro-motions are suppressed, filtered out, or eliminated in the desired signal.

[0137] The raw three-dimensional accelerometer signals generated by the 3D accelerometer 706 are pre-processed and subjected to compensation for gravity. The raw three-dimensional accelerometer signals are sampled into data samples of a data set so that digital filtering can be performed using digital signal processing and digital transforms are performed using a digital processor (e.g., the processor 701 shown in Signal processing The data samples of the accelerometer data are coupled into the NFP authentication controller.

[0138] The NFP authentication controller 810C receives the data samples representing the micro-motions sensed by the accelerometer 706 and generates the NFP of the user. In response to the NFP and stored user calibration parameters trained by and associated with the authorized user, the NFP authentication controller 810C classifies the NFP and generates a match percentage value. In response to the match percentage value and a predetermined acceptable match percentage, the NFP authentication controller 810C can generate an access grant signal 1449 granting access to the electronic device to the authorized user holding the electronic device.

[0139] FIG. 14A

[0140] Referring now temporarily to Signal sampling To obtain the NFP 1460 from the sampled micro-motion signals 1450 captured by the sensors and sampled by the sampler (analog-to-digital converter), a number of signal processing steps are performed by a processor in the electronic device. The signal processing steps performed on each dimension in the sampled micro-motion signals 1450 are performed by a signal processing and feature extraction module 1401 of the NFP authentication system 810. These one or more signal processing algorithms performed by the module 1401 can be generally referred to herein as NFP algorithms and methods.

[0141] Generally, a sequence of events is performed to obtain the NFP of the user and then assess its authenticity.

[0142] The raw data files (in X, Y, and Z directions) over a predetermined sampling period are obtained from the 3D accelerometer or one or more touchpad sensors, as the case can be.

[0143] The raw data file is sampled at a predetermined sampling frequency over a predetermined time span (e.g., 5, 10, 20, or 30 seconds) to capture signals of interest compatible with the desired further filtering (e.g., 250 Hz (4 milliseconds between samples), 330 Hz, 200 Hz, or as low as 60 Hz (twice the 30 Hz frequency of interest)).

[0144] The sampled signals are processed to produce micro-movement signals having specific frequency components of interest. The sampled signals are filtered using a bandpass filter having a frequency range between 7 to 8 Hz, 7 to 12 Hz, or 8 to 30 Hz. Micro-movement signals in this frequency range are most useful to distinguish users. The bandpass filter can also suppress large amplitude signals attributable to voluntary or involuntary movements that can be captured by the sensor (e.g., accelerometer). If an accelerometer is used as the sensor, the effects of gravity are compensated for or removed by the signal processing. If an accelerometer is used as the sensor, the sampled signals are made position invariant or orientation invariant by the signal processing. The micro-movement signals are position invariant sampled signals that can be consistently used to extract feature values for comparison between users.

[0145] Additional signal processing is performed on the micro-movement signals to produce signal-processed waveform signals from which values of NFPs can be extracted. Predetermined features whose values are to be extracted to represent NFPs are selected. Values of NFPs can be extracted directly from the signal-processed waveform signals, each micro-movement signal, and / or from both after the additional signal processing is performed. Regardless, unique values representing unique NFPs of a user are extracted that will be different from other NFPs produced by other users.

[0146] A user's NFPs can be used in a number of different applications. A classifier (a variety of data mining techniques can be used as the classifier) is used to produce a match result value (e.g., a percentage). The classifier is trained / calibrated using an initial NFP (a calibration NFP) that produces an authorized user calibration parameter such that a calibration match result level is achieved. Thereafter, the classifier can be used in a user mode in conjunction with the authorized user calibration parameter. The classifier in the user mode produces a match result value to authenticate a user as an authorized user or an unauthorized user. In response to a predetermined access match level, an authentication controller can determine whether a user is an authorized user based on the match result value.

[0147] A number of these signal processing steps are further detailed herein.

[0148] FIG. 9

[0149] Reference is now made to Signal normalization to unrelated signals, showing hand acceleration waveforms 900 of hand acceleration signals of a single axis (X, Y, or Z) as a function of time. A portion 901 of the hand acceleration waveforms 900 is magnified as waveforms 900T, as shown. While analog signal waveforms can be shown in the figures, it should be understood that analog signal waveforms can be sampled as a function of time and represented by a sequence of numbers at discrete periodic time stamps ("digital waveforms"). While accelerometers sense acceleration as a function of time, if the sensor instead senses displacement as a function of time, the displacement can be converted to acceleration by double differentiating the displacement signal as a function of time.

[0150] Hand acceleration is sampled for each axis over a predetermined sampling time period 905, e.g., a 10, 20, or 30 second time span. The sampling frequency is selected so that it is compatible with the subsequent filtering. For example, the sampling frequency can be 250 Hz (4 milliseconds between samples). Alternatively, the sampling frequency can be, e.g., 330 Hz or 200 Hz. The sampling is performed by a sampling analog-to-digital converter on the analog signal to produce samples S1 to SN represented by numbers at time stamps T1 to TN during the given predetermined sampling time period. Assuming a 20 second sampling time period and a 250 Hz sampling frequency, then over a time period for a total of 15k samples, the acceleration data set will contain 3 (3 axes) times 5000 samples.

[0151] FIG. 10A

[0152] The generation of the NFP is based on three-dimensional uncorrelated signals. The 3D accelerometer used to sense tremor is part of the electronic device held by the user's hand. As shown in FIG. 10B The device 700 and device axes Xd, Yd, Zd can be held by the user's hand in different orientations relative to the world W and world axes Xw, Yw, Zw, as shown in

[0153] Each data set of raw sensor data from the 3D accelerometer of device axes Xd, Yd, Zd for a given sampling time period is analyzed in a 3D feature space. Three eigenvectors and eigenvalues are determined for each axis. The eigenvalues of each eigenvector are compared to determine the largest eigenvalue that identifies the largest eigenvector. Then, the points of the data set are rotated (transforming the points and their data values in space) such that the largest eigenvector is aligned and along a predetermined axis. The predetermined axis is constant for all data sets of 3D raw sensor data originating from the same electronic device 700. The predetermined axis can even be constant for any device that implements the NFP algorithm using a 3D accelerometer. The predetermined axis can be, for example, the Zwworld axis. Aligned along the predetermined axis, the largest eigenvector of each data set transforms the X, Y, Z components of the 3D points in the raw sensor data such that they are uncorrelated and rotationally invariant.

[0154] For example, Signal suppression / extraction / filtering A 3D point 1011A of acceleration is illustrated with respect to an eigenvector 1010A. The data set that forms the eigenvector 1010A, including the point 1011A, is rotated in 3D space to an eigenvector 1010B that is aligned with the Zwworld axis. Thus, the point 1011A and its X, Y, Z component values are transformed in 3D space to a point 1011B and its X', Y', Z' component values.

[0155] FIG. 9

[0156] This NFP algorithm specifically extracts tremor or micro-movements from the transformed sensor data signals related to the cerebral cortex, its subcortical parts, cerebellum, motor control of the central nervous system, with or without influence of peripheral structures (e.g., muscles, bones, glands, etc.), and suppresses unwanted parts of the transformed sensor data signals.

[0157] Thus, it is desirable to generate or extract a micro-movement signal from the transformed hand acceleration signal. However, the transformed hand acceleration signal can have several undesirable signals within it that can be suppressed, removed, or filtered out. For example, the hand that holds the electronic device typically moves with large movements, such as moving the electronic device from a person's pocket / purse or pocket so that the keypad is accessible and the display screen is visible.

[0158] Large acceleration swings 902 can occur due to such large movements, as FIG. 9These large swings in the signal from the user's gross motion (micro motion) can be used as a behavioral profile, as shown in U.S. Patent Application No. 13 / 823,107, filed January 5, 2012, by Geoff Klein, entitled "METHOD AND SYSTEM FOR UNOBTRUSIVE MOBILE DEVICE USER RECOGNITION." However, since these large swings are not related to the micro motion associated with neuromuscular tremor, it is desirable in this case to suppress or remove these large swings during signal processing and generation of the user's unique NFP.

[0159] Unwanted signal components, such as those caused by large swings 902, can be suppressed. The large swings can come from vibrations associated with the building or structure in which the user is located or from motions actually performed by the user while holding an electronic device with an acceleration sensor.

[0160] Most buildings and structures resonate at frequencies between 3 Hz and 6 Hz. Thus, vibrations from the building and structure are outside the desired frequency range (e.g., outside the range of 8 Hz to 30 Hz) and are unlikely to resonate and contaminate the desired signals having the range. Signals in the range of 3 Hz to 6 Hz will then be filtered out.

[0161] Gross scale motion (macro motion) by the user includes jumping with the electronic device, moving one's arm with the electronic device, turning around with the electronic device, walking, running, jogging, and other gross body movements with the electronic device. Gross scale motion (macro motion) by the user is generally not conducive to generating NFPs. NFP algorithms are not based on hand or body gestures, and NFPs are not based on a motion repertoire / library.

[0162] Gross scale motion (macro motion) by the user forms, for example, FIG. 11A Gross scale signal swings 902, as shown in the middle. Gross scale motion by the user is unlikely to be repeated in an attempt to reproduce the NFP during an authentication procedure. Gross scale motion by the user can adversely bias the algorithm or calculations in generating the NFP. Thus, gross scale signals from gross scale motion are partially suppressed or filtered out from the acquired data by bandpass filtering performed for the desired frequency range. Gross scale signals from gross scale motion are highly correlated over relatively long periods of time. Micro motion due to the nervous system is not closely correlated. Thus, subsequent signal processing to decorrelate the three-dimensional signal will suppress gross scale signals from gross scale motion.

[0163] Alternative ways of suppressing / filtering out large scale signals from the desired signal, due to large scale motion of the user, can also be used. The electronic signal can be analyzed and then classified / identified as small signals from small signal amplitudes of micro-motions and large signals. The analysis can be in the form described in the paper "Time Series Classification Using Gaussian Mixture Models of Reconstructed Phase Spaces" by Richard J. Povinelli et al. published in the IEEE Transactions on Knowledge and Data Engineering, Vol. 16, June 6, 2004. Alternatively, the large signals due to voluntary motion can be separated by using the BPF-Kalman filter as described in the paper attached in the Appendix: "Estimation of Physiological Tremor from Accelerometers for Real-Time Applications" by Kalyana C. Veluvolu et al. published in Sensors, Vol. 11, pp. 3020-3036, 2011.

[0164] The amplified hand acceleration waveform 900T is more representative of an acceleration signal that contains tremor from which micro-motions (micro-accelerations) signals of interest can be generated. The waveform 900T has several frequency components outside the frequency range of interest. For example, the frequency range of interest in the signal is from 8 Hz to 12 Hz and / or from 8 Hz to 30 Hz. These frequency ranges are associated with known tremors that most people should typically have.

[0165] Referring now to FIG. 11B , a band pass filter (BPF) with a filter response is shown, with a lower cut-off (LC) frequency and an upper cut-off (UC) frequency at the ends of the desired frequency range. The BPF is in the form of a digital finite impulse response (FIR) band pass filter used to filter a digital signal. This BPF filters out the undesirable frequency signal components from the raw electronic signal captured by the sensor to produce the micro-motions acceleration signal of interest in the desired frequency range. Three BPFs can be used in parallel, one for each axis signal. Alternatively, one BPF can be time shared between each axis.

[0166] Referring now to Gravity correction to accelerometer sensor, a digital high pass filter (HPF) with LC frequency and a digital low pass filter (LPF) with UC frequency can be combined in series to achieve a result similar to a digital FIR band pass filter in producing a micro-motion acceleration signal of interest. Alternatively, a low pass filter with UC frequency and a high pass filter with LC frequency can be combined in series to achieve a similar composite signal output.

[0167] Additional signal processing is performed on the micro-motion signal using a processor to produce an NFP.

[0168] FIG. 12A

[0169] It is desirable to remove the effect of gravity on the three-dimensional accelerometer data captured by the accelerometer. As such, the NFP produced is substantially independent of the device orientation. A dedicated three-dimensional touchpad sensor can be unaffected by gravity. In this case, the three-dimensional data captured by the touchpad sensor as a user's finger touches it need not follow these steps.

[0170] The gravitational acceleration is a constant. Therefore, the expected effect of gravity on the spectrum is a fixed vector near zero frequency. Gravity acts as a zero frequency DC component, so one would expect that it would achieve leakage towards low signal frequencies (below 1 Hz).

[0171] Referring now to FIG. 12B A high pass filter with a cutoff frequency below 1 Hz (e.g., 0.25 Hz) can be used to compensate for gravity and improve the accuracy of recognition using the NFP. The micro-motion signal with the effect of gravity is coupled into a gravity high pass filter (HPF) with a high pass filter response with a cutoff (CO) frequency less than 1 Hz but greater than 0 Hz. The signal output from the gravity HPF is a micro-motion signal without the effect of gravity.

[0172] Referring now to Filtered micro-motion waveform Another way to eliminate the effect of gravity is to transform the coordinates of the 3D acceleration data points (ADP1 through ADPN) of each data set. First, the XYZ coordinates of the center of gravity CG of the sampled data set of 3D acceleration points (X, Y, Z signals) over a predetermined sampling period are determined. Then, the coordinates of the 3D acceleration points in a given data set are subjected to a translation transformation such that the center of gravity is at the (X=0, Y=0, Z=0) coordinate or axis origin. In this case, the entire data set is subjected to a translation transformation.

[0173] Yet another way to eliminate the effect of gravity on the acceleration signal would be to correct for the phase shift induced in the 3D accelerometer data in an analytical manner for the gravity vector. The gravity vector can be determined from the original 3D acceleration signal. Then, the 3D accelerometer data can be attenuated along one direction in response to the gravity vector and enhanced along the opposite direction of the gravity vector.

[0174] FIG. 13A

[0175] After filtering and rejection to remove unwanted signals and transformation to compensate for gravity or relationship, the desired micro-motion signal is formed.

[0176] Referring now to FIG. 13B , a waveform plot shows three axes of raw acceleration waveform data 1301-1303 offset from each other due to gravity and orientation differences. FIG. 13B Acceleration waveform data for three axes is illustrated after rejection / removal / filtering of unwanted signals from captured acceleration sensor data. FIG. 13A to 13B Acceleration waveform data in FIG. 2A to 2C Four seconds of sampling are shown, but a longer or shorter period of sampling can be used.

[0177] There are three axes of micro-motion signal corresponding to three axes of acceleration sensed by a 3D accelerometer or three axes of displacement sensed by a 3D touchpad sensor. The three axes of micro-motion signal define a drawable three-dimensional point x(t), y(t), z(t). The user's three-dimensional micro-motion signal x(t), y(t), z(t) can be further processed into phase(t), y(t), z(t) and plotted in a three-dimensional Poincare phase scatter plot, such as shown in NFP authentication controller

[0178] Three-dimensional Poincare phase scatter plots for different users show that micro-motion signals have unique patterns that can be used to identify and authenticate users. A unique pattern for each user can be obtained from the Poincare phase scatter plot. Another unique pattern for each user can be obtained from the time series of micro-motion itself without any phase information. However, it is easier to use a signal processor and digital signal processing algorithms to extract the unique pattern from the signal itself.

[0179] FIG. 14A

[0180] Referring now to FIG. 15 , a block diagram of an NFP authentication controller 810 is shown. The NFP authentication controller 810 includes a signal processing and feature extractor module 1401 (which can be split into two separate modules), an NFP authentication classifier module 1402, and an authorization (authentication) controller module 1404. The NFP authentication controller 810 can further include an optional secondary authentication module 1406 for multi-factor authentication, such as by a keypad. One or more of the modules can be implemented by software / firmware instructions executed by a processor, hardwired electronic circuitry, or a combination of each.

[0181] ​The NFP authentication controller 810 can further include a non-volatile storage device 1453 coupled to the classifier 1402 and an authorization (authentication) controller module 1404 to store data such as user calibration parameters 1466; an access match (AM) level 1456A, a mandatory re-calibration (MR) level 1456B, a voluntary re-calibration (VR) level 1456C (collectively referred to as match percentage levels 1456); and an authentication enable bit (EN) 1455. Alternatively, the non-volatile storage device 1453 can be external to the NFP authentication controller 810, but remain internal to the electronic device as a secure independent non-volatile storage device or as a secure portion of a larger non-volatile storage device.

[0182] The NFP authentication controller 810 receives the three dimensions of micro-motion data samples 1450 in each data set within each sampling period. The micro-motion data samples 1450 are coupled into the signal processing and feature extractor module 1401. The signal processing and feature extractor module 1401 performs signal processing and signal analysis on the micro-motion data samples 1450 to extract a plurality of extracted features 1460X, 1460Y, 1460Z for each of the respective three dimensions (X, Y, Z). The extracted features 1460X, 1460Y, 1460Z collectively represent a NeuroFingerPrint (NFP) 1460 coupled into the NFP authentication classifier module 1402.

[0183] The NFP authentication classifier module 1402 receives the NFP 1460 (extracted features from the micro-motion data samples 1450) and generates a match percentage (MP) signal output 1465. In a user mode, the match percentage signal 1465 is coupled into the authentication controller 1404. In a calibration or training mode, the match percentage signal 1465 is used by a processor in the electronic device to evaluate the match percentage signal 1465 in response to a selection of initial user calibration parameters 1466. The duration of the training / calibration of the NFP authentication classifier module and the generation of the initial user calibration parameters 1466 is expected to be between 5 and 10 seconds. In the user mode, it is expected to take less than 5 seconds to sense motion in a body part of a user and determine whether to grant or deny the access.

[0184] In the calibration or training mode, an authorized user can generate one or more sets of user calibration parameters 1466 so that the NFP authentication system operates under different conditions. For example, a user can want to hold the electronic device in either hand and has been granted access. Between the left and right hands, or between different fingers, the tremor will be different. The user can want to calibrate the NFP authentication system to both the left and right hands or multiple fingers. Also, in some cases, more than one user will use the electronic device. In this case, multiple people can be authorized users and will need to make and store multiple calibrations. Thus, the storage device 1453 can store multiple sets of user calibration parameters for the same authorized user or different authorized users.

[0185] An authentication enable bit (EN) 1455 is coupled into the authentication controller 1404. The authentication enable bit (EN) 1455 can be used to enable the authentication controller 1404 after the initial calibration or training mode. After the initial calibration or training mode, the authentication enable bit (EN) 1455 is set and the authentication controller 1404 is enabled. The authentication enable bit (EN) 1455 is not reset in the user mode or re-calibration mode after the initial calibration or training mode. The authentication controller 1404 is enabled by the enable bit 1455 in order to enhance security unless the entire electronic device is erased along with the enable bit.

[0186] In the user mode, the match percentage signal 1465 is used by the authentication controller 1404 to evaluate whether to grant access to the electronic device and its software applications. The match percentage signal 1465 is evaluated with the access match level 1456A to generate an access granted (AG) signal 1499. If the match percentage signal 1465 is greater than or equal to the access match level 1456A, an access granted (AG) signal 1499 at a logic level is generated to signal that access has been granted. If the match percentage signal 1465 is less than the access match level 1456A, the access granted (AG) signal 1499 is not generated and access is not granted. The access granted (AG) signal 1499 is coupled to the processor in order to enable the authorized user to control and operate the functions of the electronic device.

[0187] If the optional secondary authentication mode is used, a secondary match (SM) signal 1468 is coupled into the authentication controller 1404. In this case, the authentication controller 1404 further evaluates the secondary match signal 1468 as to whether to generate the access granted (AG) signal 1499. The authentication controller 1404 can use AND logic to require both conditions to be met before generating the access granted signal. Alternatively, the authentication controller 1404 can use OR logic to require both conditions to be met before generating the access granted signal.

[0188] In response to the non-action reactivation signal 1470 from the processor, the authentication controller 1404 maintains the access grant signal 1499 in an active state (as long as the user is using the electronic device) and avoids a sleep state or timeout to enter a protected state. If the sleep state or timeout occurs, a pulse is applied to the reactivation signal 1470 by the processor. In response to the pulsed reactivation signal, the authorization (authentication) controller module 1404 deactivates the access grant signal 1499 so that the user must re-authenticate himself / herself using the NFP authentication system of the electronic device to gain access.

[0189] Referring now to FIG. 14A , a graph of the match percentage level (MP) 1456 compared to the match percentage signal 1465 along the X-axis is shown. The Y-axis indicates the access grant (AG) signal 1499 generated by the authentication controller 1404 and whether access is granted to a suspect user.

[0190] In the user mode, the match percentage signal 1465 is used by the authentication controller 1404 to evaluate whether to grant access to the electronic device and its software applications. The match percentage signal 1465 is evaluated against the access match level 1456A. If the match percentage signal 1465 is at or above the access match level 1456A, the access grant (AG) signal 1499 is generated by the authentication controller 1404. If the match percentage signal 1465 is below the access match level 1456A, the access grant (AG) signal 1499 is not generated by the authentication controller 1404.

[0191] Referring now to FIG. 14A to 14B and 15 , the authentication controller 1404 can also generate one or more recalibration signals to inform the user to recalibrate the NFP authentication system by regenerating the user calibration parameters 1466 for an authorized user before the NFP authentication classifier 1402 is unable to generate a level of the match percentage signal 1465 at or above the access match level 1456A.

[0192] In the training or calibration mode, training / calibrating the NFP authentication classifier 1402 by the initial user calibration parameters 1466 causes the NFP authentication classifier 1402 to produce a match percentage signal 1465 at a level that can be or close to 100% (e.g., 98%) of the calibration level 1457. After training, in the user mode, the NFP produced by the user over time can change as his / her body ages, becomes ill, or for other reasons affecting the physiological condition of the body. Thus, the match percentage signal 1465 can decrease from the calibration level 1457 over time. Using periodic re-calibration to reset the user calibration parameters 1466 causes the match percentage signal 1465 to return to the calibration level 1457. Depending on the user's age, health, and other physiological conditions of the body, the user can or can not need periodic re-calibration more or less often.

[0193] In the user mode, the NFP authentication classifier 1402 is used by the authentication controller 1404 to evaluate whether re-calibration is needed. The determination of re-calibration is in response to a mandatory re-calibration (MR) level 1456B and a voluntary re-calibration (VR) level 1456C. The mandatory re-calibration (MR) level 1456B is less than both the voluntary re-calibration (VR) level 1456C and the calibration level 1457. The voluntary re-calibration (VR) level 1456C is less than the calibration level 1457.

[0194] Re-calibration requires first verifying the user as an authorized user, first having to authorize access to the electronic device to enter the user mode and perform re-calibration. If the device is stolen by an unauthorized user, the unauthorized user will not be granted access to re-calibrate the device.

[0195] If the match percentage signal 1465 falls below the voluntary re-calibration (VR) level 1456C, a voluntary re-calibration signal 1471 is generated by the authentication controller 1404. In this case, the electronic device notifies the user through its user interface that he / she should pause and take some time to voluntarily re-calibrate the NFP authentication system by regenerating the user calibration parameters 1466. It is expected that most users will voluntarily choose to re-calibrate the NFP authentication system. However, some authorized users will choose to wait, forget the warning, or completely ignore the warning.

[0196] For those authorized users who do not voluntarily recalibrate the NFP authentication system, they can be forced to go through a mandatory recalibration process. If the match percentage signal 1465 is further reduced and falls below the mandatory recalibration (MR) level 1456B, a mandatory recalibration signal 1472 is generated by the authentication controller 1404. In this case, after the authorized user is verified by the NFP authentication classifier 1402 and granted access by the authentication controller 1404, the electronic device immediately enters a recalibration mode, through its user interface, informing the user to prepare and perform a recalibration procedure by continuing to hold the device properly or properly touching the button. The user is further informed that the NFP authentication system is performing a recalibration and continues to wait until the recalibration is completed and the match percentage signal 1465, with the regenerated user calibration parameters 1466, has successfully recalibrated the NFP authentication system to be at or above the calibration level 1457.

[0197] If the electronic device is not used for a period of time, it is possible that the match percentage signal 1465 is further reduced and falls below the access match (AM) level 1456A. In this case, the authorized user can be denied access. Therefore, the selection of the value of the access match (AM) level 1456A is important so that the authorized user is not easily denied access to the electronic device, while the unauthorized user is denied access. It is desirable to produce an NFP signal 1460 that is less sensitive to changes in the user (e.g., aging, illness) so that recalibration is less frequent, generated by the module 1401 and coupled into the NFP authentication classifier 1402. Therefore, it is desirable to select the signal processing and feature extraction algorithms of the module 1401 so that the sensitivity is low in producing the NFP and is less likely to change over time.

[0198] An example setting of the match percentage levels (MP) 1456 in increasing order is that the access match (AM) level 1456A is 85%, the mandatory recalibration (MR) level 1456B is 90%, and the voluntary recalibration (VR) level 1456C is 95%. Therefore, the user is informed by the user interface of the electronic device to perform a voluntary recalibration before a mandatory recalibration is needed to be performed.

[0199] In the case where the user is required to perform a mandatory recalibration, if the electronic device is actively used, the NFP authentication system should avoid denying access to the electronic device. If the electronic device is left for a period of time (e.g., one or more years), it can be necessary for the user to wipe the electronic device clean, reinitialize the NFP authentication system, and reload the applications and / or data, for example, from a backup.

[0200] Referring now to Signal processing and feature extraction to generate NFP, showing a model for the NFP authentication classifier 1402. According to one embodiment, the model is a linear regression analysis model. In alternative embodiments, the regression analysis model for the NFP authentication classifier 1402 can be non-linear. During a training or calibration mode, features of the micro-movement signals from an authorized user are extracted by the signal processing and feature extraction module 1401 to produce a calibration NFP 1460, including calibration NFP information 1460X, 1460Y, 1460Z for each axis.

[0201] The calibration NFP information 1460X, 1460Y, 1460Z for each axis is placed in a single row NFP matrix 1452. The appropriate values of the user calibration parameters 1466 are unknown. The values of the user calibration parameters 1466 can be set randomly into a single column calibration matrix 1454. Matrix multiplication is performed by the processor to multiply the single row NFP matrix 1452 and the single column calibration matrix 1454 to obtain a match percentage value 1465.

[0202] During calibration / training, the processor finds the user calibration parameters 1466 to be placed in the single column calibration matrix 1454 to produce a predetermined value of the match percentage 1465. This predetermined value is referred to as the calibration level 1457. For example, the calibration level 1457 can be set to 90%. In this case, the processor finds the values of the authorized user calibration parameters 1466 such that when multiplied with the NFP 1460, the match percentage 1465 output from the classifier is 90% or greater.

[0203] Once the authorized user calibration parameters 1466 are set and stored in the electronic device, the NFP classifier 1402 uses the authorized user calibration parameters 1466 to multiply subsequent regenerated NFPs produced from unknown users or authorized users.

[0204] If a person subsequently fails to produce a regenerated NFP substantially similar to the calibration NFP, the value of the match percentage 1465 produced by the classifier 1402 when the user calibration parameters 1466 are multiplied with the different regenerated NFP will be low. The authorization (authentication) controller module 1404 can be set such that a low value of the match percentage 1465 denies access to unknown persons who produce significantly different regenerated NFPs.

[0205] If a person subsequently produces an NFP (a regenerated NFP substantially similar to the calibration NFP), the value of the match percentage 1465 produced by the classifier 1402 when the user calibration parameters 1466 are multiplied with the regenerated NFP will be high. The authorization (authentication) controller module 1404 can be set such that a high value of the match percentage 1465 above an access match level 1456A grants access to the electronic device.

[0206] FIG. 13A

[0207] Referring now to FIG. 13B , raw unfiltered samples of three-dimensional acceleration data from a 3D accelerometer are shown. For the X, Y, and Z axes of the electronic device, raw signals 1301-1303 are captured for each of the three dimensions. With some pre-processing, band-pass filtering, and gravity compensation, FIG. 13B The micro-motion signal waveforms 1311-1313 shown in may be formed from the raw signals 1301-1303, respectively. Further signal processing can then be performed on the micro-motion signal waveforms 1311-1313 to form NFPs for a given sample of acceleration data. There are various ways to produce NFPs from micro-motion signals. Once calibrated / trained, the NFP authentication system in the electronic device uses a consistent method of producing the NFPs.

[0208] Three-dimensional micro-motion signals are related to common tremors that exist in users. There are patterns hidden in the time domain of the micro-motion signals that are unique to each user as they originate from his / her neuromuscular function. It is desirable to highlight these patterns by performing signal processing on the micro-motion signals and then extracting features from the micro-motion signals to form NFPs. The calibrated NFPs can then be used during a training or calibration process to produce authorized user calibration parameters. For the purpose of user authentication, the authorized user calibration parameters can then be used to classify a reproduced NFP and distinguish between a known authorized user and an unknown unauthorized user.

[0209] However, before the NFPs can be used for authentication, the NFPs are used to train or calibrate a model of a classifier by producing a set of authorized user calibration parameters. Subsequently, this set of authorized user calibration parameters is used in conjunction with the classifier model to evaluate future NFPs (reproduced NFPs) captured by the sensor from unknown individuals. Various signal processing algorithms can be used to extract data from the micro-motion signals as NFPs.

[0210] The patterns hidden in the micro-motion signals can be detected using inverse spectral analysis, for example, such as FIG. 16 the micro-motion signal waveforms 1311-1313 shown in . The inverse spectral analysis is performed in order to recover unknown signal components that are related to time variations.

[0211] One type of available inverse spectral analysis is cepstrum analysis. Cepstrum analysis is a tool used to detect periodicity in a spectrum. The cepstrum analysis can be used to detect repeating patterns, their periodicity, and frequency intervals.

[0212] Generally, the cepstrum is the result of an inverse Fourier transform (IFT) of the log of an estimated spectrum of a signal.

[0213] The power cepstrum of a signal can be defined as the square magnitude of the inverse Fourier transform of the log of the square magnitude of the Fourier transform of the signal, as shown by the following equation.

[0214]

[0215] First, the Fourier spectrum of a micro-motion signal (such as the micro-motion signal shown in FIG. 13) is taken using a fast Fourier transform. Mathematically, this can be done by performing a Fourier transform using the following equation

[0216]

[0217] where ξ represents the real frequency (in hertz) value and the independent variable x represents time, the transformed variable. Using software or digital circuitry that can digitally sample a signal, the discrete-time Fourier transform (DFT) represented by the following equation can be used

[0218]

[0219] The result of the Fourier transform of a micro-motion signal is a spectral density (power spectrum or Fourier spectrum). FIG. 17A to 17B An example of a spectral density presented for tremor at a hand, acceleration as a function of time, is illustrated. Note that the expected peak (P) signal component of physiological tremor is at a peak frequency (PF) of about 10 to 12 Hz. However, there is still much useful hidden pattern (HP) information in the spectral density curve for tremor.

[0220] The spectral density (power spectrum) is a composite of the component frequencies of a signal. That is, there are numerous signal frequencies that combine together to form the spectral density curve. It is desirable to effectively separate out the signal frequencies to show the patterns. The signal composite illustrated by the spectral density is equivalent to a signal convolution that is the product of the constituent signals.

[0221] Taking the log of the convolution effectively transforms the spectral density from a sum to a product of the constituent signals. Furthermore, the log of the spectral density (or the log of the square of the spectral density) highlights the lower amplitude frequency components in the spectral density curve where hidden pattern (HP) information can exist. Taking the log of the spectral density effectively compresses the large signal amplitudes of the spectral density signal and expands the smaller amplitudes of the spectral density signal.

[0222] However, it is difficult to see the periodicity and patterns of the signal waveform produced by the log transform. Therefore, performing an inverse Fourier transform (IFT) on the log signal waveform makes the unknown frequency components and hidden patterns visible. The inverse Fourier transform of the log transform separates the components of the composite signal in the spectral density of the micro-motion signal waveform.

[0223] FIG. 13B illustratedFIG. 17A to 17B The cepstrum presented in the micro-motion waveform shown in FIG. 6B. This synthetic waveform is produced after performing an inverse Fourier transform (IFT) of the log spectrum. The cepstrum produces a series of roughly resolved unknown signal components in the synthetic waveform. The values of the pattern of features can be extracted from each axis (dimension) in the synthetic waveform and used as the NFP to identify the user. Note that, FIG. 17A to 17B The horizontal axis of the cepstrum waveform shown in FIG. 6B is frequency and not a measure of time in the time domain.

[0224] In general, an inverse Fourier transform is an integral that can be integrated over the variable values of a function g. In this case, the inverse Fourier transform can be represented by the following equation:

[0225]

[0226] To recover the discrete-time data sequence x[n], an inverse discrete Fourier transform (IDFT) can be used.

[0227]

[0228] Using an inverse fast Fourier transform (IFFT), the equation becomes

[0229]

[0230] where a sequence of N samples f(n) is indexed by n = 0 to N - 1, and the discrete Fourier transform (DFT) is defined as F(k), where k = 0 to n - 1.

[0231]

[0232] FIG. 17A An example of a three-dimensional cepstrum waveform signal 1700 is illustrated. The waveform signal 1700 is a result of performing a cepstrum analysis on a micro-motion signal, each dimension of the micro-motion signal having one cepstrum signal waveform, resulting in cepstrum signal waveforms 1701, 1702, 1703. The cepstrum waveform signal 1700 has different features associated with it that can be readily used to identify a user. For example, the first N peaks of the cepstrum waveform signal 1700 can be predetermined features used to extract from each cepstrum waveform signal 1700 to identify a user. The peaks are selected for their high variance, thus they are different features. For each user, the values of the inverse frequency and amplitude of the first N peaks can be used as the NFP of the respective user. The values of the inverse frequency and amplitude of the N peaks can be used as the NFP input into the classifier model in response to an authorized user calibration parameter to produce a match percentage (MP).

[0233] Other features can be selected and have their values extracted from the cepstrum of the micro-motion signal. Other signal processing can be performed so that additional features or patterns in the micro-motion signal become apparent and can be used to generate the NFP. For the NFP authentication system, the consistency of which features are used to extract values is key. The features should be predetermined. The same features (e.g., the first N peaks) should be used in conjunction with the calibration NFP during the calibration / training mode and when forming the regenerated NFP from a new sample set in the user mode.

[0234] For example, as can be seen in FIG. 18A to 18B , the first five peaks P1, P2, P3, P4, P5 on each axis at their respective quefrequency (or quefrency) and amplitude (e.g., from 3 axes / 3 curves and 5 peaks, a total of 3 x 5 amplitude values) have large variations that can be used to differentially identify users. The amplitudes and quefrequencies of the first five peaks P1, P2, P3, P4, P5 will be significantly different when captured from different users. Thus, the amplitudes and quefrequencies extracted for the first five peaks P1, P2, P3, P4, P5 can be used to distinguish user identity, for example as shown in FIG. 17A . While the first five peaks are used in this example, it is not a constraint or limitation on the embodiments, as fewer peaks, additional peaks, or multiple other features can be selected to form the NFP.

[0235] In the case where the first five peaks P1, P2, P3, P4, P5 are the predetermined features used to extract values from the cepstrum signal waveform, the NFP can be regenerated repeatedly for the same user with minor variations. If a different user generates his / her NFP, the extracted values of the predetermined features will vary substantially. The substantial variation between the NFP of an authorized user and the NFP of a different user can be used to grant access to the authorized user and deny access to the different user.

[0236] The first N features extracted from the cepstrum waveform can be used to regenerate the NFP and for the NFP authentication classifier to classify a match to the calibrated parameters of an authorized user. Selecting earlier features in the cepstrum (e.g., the first N (e.g., N is 3 to 5) peaks or other coefficients) can reduce the sampling period, but still represent a large portion of the variation in the signal. For example, these first N coefficients can be extended to 128 extracted features by planning N features through PCA analysis, and thus become a linear combination of 128 extracted features.

[0237] In a sampling period of 5, 10, or 20 seconds, the values extracted from the cepstrum waveform are the locations (quefrequencies) of the predetermined features and their respective amplitudes. If the predetermined features are, for example, the first five peaks, the values of the amplitudes and quefrequencies of each axis are extracted as the NFP. This method aggregates the time in the sampling period.

[0238] The cepstrum can be analyzed in near real time, rather than aggregating over time in the sampling period. For example, in this case, a peak at one quefrequency can be selected and plotted over time. For example, a peak PI at quefrequency 8 in FIG. 18A to 18B

[0239] However, analysis of 128 quefrequencies is a challenge. The analysis can be made simpler and reduce computation time by examining where the largest changes in the waveform are. The first 3, 5, or 10 quefrequency positions can be employed and principal component analysis (PCA) performed in 3, 5, or 10 dimensions of the data space. Then, nearest neighbor analysis can be performed, for example, on this reduced data space to extract NFP values.

[0240] Another way of feature extraction and generating NFP values can be done without 3D plotting of data points for each quefrequency over time. Quefrequency, amplitude, and time can be taken as true coordinates of a vector. N quefrequencies can be selected to plot amplitude and time as dimensions. Then, these vectors can be plotted for resolved frequencies at any given time. This produces a time series of the extracted features that can be analyzed to extract NFP values.

[0241] Another way of feature extraction and generating NFP values is to extract the frequency (quefrequency), amplitude (as a reminder, the raw signal of interest and relevance is bandpass between 16 and 30+ Hz, at 50 per second) at different times. Over a few seconds, a series of numbers is produced that can be used as extracted values of NFP. Values at different time points, rather than different quefrequencies, can be extracted and used as NFP.

[0242] While cepstrum signal processing is described herein, other signal processing algorithms can be used on the micro-movement signals so that other patterns emerge in the different waveforms. Different features can be extracted from the waveform signals and used to define NFPs for NFP authentication classifiers. Different extracted features can be used as input to a model in response to an authorized user calibration parameter to produce a matching percentage (MP) of NFP.

[0243] Referring now to FIG. 18A to 18B and 19A-19B, examples of a single axis of different NFPs of different users are shown. Additional dimensions can be provided by two or more additional dimensions.

[0244] FIG. 19A to 19B ​An example of a different NFP based on the inverse frequency of a single axis of two different users is illustrated. In FIG. 19A to 19B In each of, one or more of the five extracted peaks P1-P5 occur at different inverse frequencies F1-F5 with different amplitudes. Thus, the NFP of each user is different. FIG. 14B An example of a different NFP based on the time of a single axis of two different users is illustrated. In Training / calibration In each of, one or more of the five peaks P1-P5 occur at different times T1-T5 with different amplitudes. Thus, the NFP of each user is different.

[0245] The first five peaks of the cepstrum signal waveform are predetermined features extracted from the different micro-motion signals of each user for each user. The location and amplitude values of the first five peaks distinguish one user from another, similar to how the teeth and cuts in a door key distinguish the key. The authorized user calibration parameters behave somewhat like the tumblers of a lock that engage the teeth and cuts in a door key. If the teeth and cuts in the door key are incorrect, the tumblers of the door lock will not engage properly to unlock. With more than one dimension (e.g., 3 axes) used for feature extraction, the NFP becomes more unique. FIG. 14B An added dimension AD 1460A of the values of the features extracted beyond the three dimensions of the micro-motion signals 1460X, 1460Y, 1460Z to form the NFP 1460 is illustrated.

[0246] While 3D acceleration or 3D displacement values can be used, time can be another added dimension. At one point in time, the 3D values will have one amplitude, while at another point in time, they will have another amplitude.

[0247] Alternative ways of cepstral signal processing and generating NFP.

[0248] Referring now to FIG. 2A to 2C The one or more dimensions in the NFP 1460 and the user calibration parameters, if trained, are coupled into the NFP authentication classifier 1402. The algorithm used for the NFP authentication classifier is a data mining algorithm, such as a linear regression algorithm, a non-linear regression algorithm, a linear quadratic algorithm, a quadratic equation algorithm, a nearest neighbor classifier, or a k x N nearest neighbor classifier. The linear regression algorithm is shown as a row matrix 1452 with xM elements or columns multiplied by a column matrix 1454 with xM rows. The row matrix 1452 represents the NFP 1460 with the values of the extracted features for each of the one or more dimensions. The column matrix 1454 represents the authorized user calibration parameters 1466. In this case, the multiplication provides the values of the match percentage. During training / calibration, the user calibration parameters 1466 are adjusted to form a calibration level into a match percentage (MP) output 1465.

[0249] The NFP authentication classifier 1402 is trained / calibrated by an authorized user prior to use. Once the initial training / calibration is complete, an unauthorized user cannot train or recalibrate the classifier 1402. The micro-motion signals are captured by the accelerometer during one or more sampling periods with the electronic device in the user's hand or the user pressing the touch sensor. Calibration to the accelerometer can occur with the left or right hand or both hands. Calibration to the touch sensor can occur with one or more fingers of each hand.

[0250] The micro-motion signals are processed using signal processing algorithms and features are extracted from the composite waveform as NFP calibration samples, referred to herein as calibration NFPs. During training / calibration, the NFP calibration samples are used to find authorized user calibration parameters 1466 associated with the authorized user and to train / calibrate the classifier to produce a desired calibration match percentage level. After the authorized user calibration parameters 1466 have been produced and securely stored, the calibration NFPs are discarded for security reasons. Since the calibration NFPs are not saved, a NFP needs to be regenerated for the user in user mode (which is referred to as a regenerated NFP) to gain access to the electronic device. After the device authentication has been reestablished to lock the device from being accessed, the NFP needs to be regenerated by the authorized user.

[0251] The regenerated NFP is produced using another sample of neuro-mechanical micro-motions from the user and by recalling the stored authorized user calibration parameters from storage. The regenerated NFP is then used for authentication purposes to gain access to the electronic device. Using the regenerated NFP, the authorized user calibration parameters can provide a high match percentage so that access to the electronic device can be granted again. Like the calibration NFPs are discarded, the regenerated NFP is not saved after it is temporarily used to measure authentication. The regenerated NFP is subsequently discarded for security reasons and a next regenerated NFP is produced and evaluated for its match percentage with the authorized user calibration parameters. If the regenerated NFP is classified as having a match percentage greater than or equal to the access match level, access to the electronic device is granted.

[0252] Depending on the algorithm used by the classifier 1402, the training of the NFP authentication classifier can be linear or not linear (i.e., non-linear).

[0253] To calibrate the classifier 1402, a predetermined value is selected for the calibration match percentage level. The predetermined value for the calibration match percentage level can be 100% or less. However, the predetermined value for the calibration match percentage level should be higher than the access match (AM) level desired to be granted.

[0254] In view of the NFP calibration samples, the processor in the electronic device browses the test sequence of user calibration parameters to produce a match percentage equal to or greater than a predetermined value of the calibration match percentage level. Prior to selecting an initial user calibration parameter that produces a match percentage equal to or greater than the calibration match percentage level, the user can be required to produce additional micro-motions (representing NFPs) by continuing to hold the device or continuing to press the touch sensitive button. These additional NFPs are used to verify that the classifier is properly trained / calibrated by the user calibration parameters 1466.

[0255] After calibration, experiments were conducted to determine the error rate in authorizing access to authorized users. Using the cepstral signal analysis of the micro-motion signals, the trained classifier in the handheld electronic device with the accelerometer sensor had a 7% error rate in recognizing authorized users. At this time, 93% of the authorized users were correctly recognized. Thus, the access match (AM) level needs to be set to less than 93% (e.g., 80% for example) so that the authorized users are always authorized access to the electronic device.

[0256] When the cepstral signal analysis of the micro-motion signals is used in conjunction with the touch sensitive keypad in the electronic device, the error results appear to improve. With a small sample size of users entering their PIN numbers on the touch sensitive keypad, a 100% recognition rate of authorized users was achieved using the method of cepstral signal analysis. Nevertheless, the access match (AM) level needs to be set to less than 100% (e.g., 85% for example) so that the authorized users are always authorized access to the electronic device.

[0257] FIG. 20

[0258] Many existing time-domain methods for signal classification tasks are based on the presence of a fairly simple underlying pattern or template (either known a priori or learned from the data). However, for real signals (such as cardiac, speech or electric motor systems), this simple pattern rarely exists due to the complexity of the underlying process. Indeed, frequency-based techniques are based on the presence of spectral patterns. From a random process point of view, the frequency-based signal processing techniques used on micro-motion signals will only capture the first and second order characteristics of the system.

[0259] The cepstral analysis described herein can extract some time-dependent information from the accelerometer data, but this method is still related to frequency dispersion. Since there is more information in the micro-motion signals than in the normal frequency, other signal processing techniques and feature extraction methods can be able to improve the strength of the NFPs so that they are less sensitive and less likely to change over time.

[0260] Alternatively, NFPs can be generated, for example, using chaos analysis. Using chaos analysis, predetermined features can be extracted about the cepstrum description. However, these predetermined features are not based on frequency, but have a time trajectory. Example features for extraction using chaos analysis are the center of gravity of a strange attractor with its fractal dimension and Lyapunov exponent. A data scan over several seconds (e.g., 5, 10, 20 seconds) can be used to obtain values of these features from the micro-movement signal.

[0261] Another example using chaos analysis is to use the waveform signal generated from the logarithm of the spectrum of each axis without IFT. From the log waveform signal, amplitude and time can be taken as true coordinates of a vector and plotted for N (e.g., N equals 128) resolved frequencies at any given time. The center of gravity, dimension, and Lyapunov exponent can be extracted as time series of values representing the NFP.

[0262] According to one embodiment, the NFP signal features that can be extracted and coupled into the NFP authentication classifier can be phase scatter plots or Poincare scatter plots (see, for example, Poincare scatter plots 200A-200D shown in FIG. 2A to 2D According to one embodiment, the NFP signal features that can be extracted and coupled into the NFP authentication classifier can be phase scatter plots or Poincare scatter plots (see, for example, Poincare scatter plots 200A-200D shown in

[0263] In yet another embodiment, a Hidden Markov (Markov) model analysis is used as a signal processing module to obtain more feature information as NFPs to be coupled into a selected classifier. A Markov model is a stochastic model used to model a random changing system, where future states depend only on the present state and not on the sequence of its previous events. Typically, a Markov chain (a state space of one or more states) is used as the Markov model. The transition from one state to another is a memoryless stochastic process. The next state depends only on the current state and not on the sequence of its previous events. A Hidden Markov model is a Markov chain whose states are only partially observable.

[0264] Referring now to Method of use , a state machine 2000 implementing the Hidden Markov model analysis is described. The state machine 2000 and Hidden Markov model are used to extract features from the signal processed waveform of the micro-movement signal and perform a classification (or authenticate a user) to grant or deny access.

[0265] The state machine 2000 includes states 2001-2004 and extracted features Fl 2010A- Fn 2010N (e.g., FIG. 8A to 8CState 2001 is an accept state, state 2002 is a recalibrate state, state 2003 is a reject state, and state 2004 is a reset / initiate state.

[0266] There are state transitions that can occur between states 2001-2004 of state machine 2000. State transition al is from accept state 2001 to recalibrate state 2002. State transition a2 is from recalibrate state 2002 to reject state 2003. State transition a3 is from recalibrate state 2002 to accept state 2001. State transition a4 is from accept state 2001 to reject state 2003. Reject state 2003 can also transition to reset / initiate state 2004.

[0267] There are output probabilities from states 2001-2003 to features Fl 2010A-Fn 2010N. There are output probabilities Oal-Oan from accept state 2001 to features Fl 2010A-Fn 2010N. There are output probabilities Orl-Orn from recalibrate state 2002 to features Fl 2010A-Fn 2010N. There are output probabilities Od1-Odn from reject state 2003 to features Fl 2010A-Fn 2010N.

[0268] Using a hidden Markov model, an attempt is made to get the right answer without knowing the internal details of the system being analyzed. The answer is states 2001-2003, which are very complex and based on chaotic inputs. The N features Fl 2010A-Fn 2010N that are extracted are the only measurable features. The features can be diverse, such as the center of mass or a series of Lyapunov exponents, any parameter that can be extracted from a data trajectory, each of which is related to a probability of advantage specific to the three states 2001-2003 (e.g., Oal-Oan, Orl-Orn, Od1-Odn). Exit the state machine 2002 with a transition to a fourth state 2004. The fourth state 2004 corresponds to a system reinitialization, such as a full reset.

[0269] In yet another embodiment, a new signal classification method is used to perform dynamic system signal analysis based on a reconstructed phase space (RPS). Dynamic invariants can be used as features that will be extracted after the signal analysis. This method is expected to capture more information, resulting in better user recognition.

[0270] Applications for NFP authentication

[0271] As Advantages of NFP authenticationAs demonstrated in the middle, a person holds an electronic device or touches one or more touch sensitive pads to authenticate himself / herself. Within a few seconds of the sampling period, 3D sensors in the electronic device (3 sensors positioned on different axes) sense vibrations or micro-motions in the user's hand or fingers due to physiological conditions of the user's body. The 3D sensors simultaneously produce three electronic signals that can be sampled and converted into digital form. Digital samples are pre-processed and analyzed using signal processing techniques to produce NFPs in real-time for the user who just held the electronic device or touched the touch sensor.

[0272] In response to authorized user calibration parameters, the NFPs can then be used to assess whether a person holding or touching the electronic device is an authorized user. In response to authorized user calibration parameters, the NFPs can be used as keys to encrypt and decrypt data files. Alternatively, valid NFPs can be used to authenticate whether a file was signed, written or created by the authorized user.

[0273] User calibration parameters can be stored in a local archive or local storage in the electronic device. Alternatively, user calibration parameters can be stored in a remote archive in storage associated with a server or in a storage area network in an internet cloud. In the absence of the authorized user's hand or finger, the authorized user calibration parameters are useless. In the absence of the authorized user's hand or finger generating his / her NFPs, access to the electronic device is denied.

[0274] CONCLUSION

[0275] An NFP authentication system can be used to control access to an electronic device. An NFP authentication system can also be used to control access to functions associated with the electronic device and software applications executed by the electronic device. An NFP authentication system can be used to control access to a remote electronic device (i.e., where local device sensing is practiced to determine access to a remote electronic device).

[0276] Examples of functions that can have access controlled by an NFP authentication system include, but are not limited to, login, user-protected access, electronic transactions, and any other local function that requires explicit authentication.

[0277] For example, security applications for the NFP authentication system include computer and software logins, e-commerce, e-banking, and anti-fraud applications. Examples of human controlled applications for the NFP authentication system include domotic authentication and protection (home security systems), car security, and professional access to restricted areas. Examples of medical applications for the NFP authentication system include diagnostic assistance to health professionals (neuromuscular a.o.), treatment monitoring of patients, and patient and doctor medical authentication to databases storing medical records (records, data routing,...). Examples of health and wellness applications for the NFP authentication system include protection of the user's fitness records. Examples of gaming applications for the NFP authentication system include security features for virtual reality games and / or simulators.

[0278] Biometric encryption combined touch recognition technology with real data storage. Data is encrypted using NFP so that it is unreadable unless it is the authorized user / owner of the data. Backdoor access is not available. However, other authorized users as trusted persons can also cause the authorized user calibration parameters to be generated so that in case of problems related to the original user / owner, they are granted access.

[0279] Touch recognition technology can be embedded in certain applications. For example, touch recognition technology can be used for access control panels so that when a surface is touched by a human hand or human finger, access to a vehicle, building (e.g., a home or office), or area (e.g., a floor) can be granted or denied in response to a user NFP without the use of a key (keyless). Similarly, touch recognition technology can be used for a wireless remote key that is held in a user's hand or touched by a finger so that access can be granted or denied in response to a user NFP without the use of a key. Touch recognition technology can be combined with fingerprint authentication. In this case, an image scanner and a touch-sensitive pad can be used together to capture both a user's fingerprint and a user's NFP simultaneously for multiple authentication in granting or denying access.

[0280] Using additional signal processing algorithms, the generation of NFPs can be adapted to monitor the health and wellness of a user. Changes in the reproduction of NFPs can reduce the matching percentage indicating a degradation of the user's health. The changes can be used to discover and / or monitor neurodegenerative diseases (e.g., Alzheimer's, Parkinson's) or for efficacy monitoring. The changes can be used to obtain a measure of heart rate variability, stress and emotional state correlations can be measured using an accelerometer by extracting the shock waves of the heart pulse when the user holds the device. The changes can be used as a physiological security feature that can be used for virtual reality glasses and goggles for the video game industry, for example.

[0281] The government can provide online services (e.g., voting) and touch technology can be used to verify the identity of authorized users. Schools and educators can provide online testing. Touch technology can be used to verify the identity of students taking online tests or even tests that can be taken at school. Electronic online banking can become more secure with banks being able to use touch technology to verify the identity of their customers.

[0282] NFP technology can protect medical records by better authentication of patients and doctors as well as encryption of medical records. A user can authenticate his / her interaction with the healthcare system (providers, insurance companies, medical records companies...). The user can store his / her medical records in an encrypted format in the cloud or on any device where the NFP is the key.

[0283] ​

[0284] There are several advantages to using NFP for touch recognition. Touch recognition does not require a centralized database. Access to the electronic device is controlled and shielded locally by the local NFP user authentication system.

[0285] An additional advantage of NFP is that one can be uniquely identified as the correct user of an electronic device but remain anonymous, providing strong user privacy protection. NFPs are generated in response to brain / neural system related signals generated by sensors. Then, NFPs can provide foolproof unique user identification (even twins have different muscle strength).

[0286] Touch recognition employs neural algorithms to generate NFPs from micro-movement signals associated with micro-movements of fingers on the hand. This avoids the background related to behavioral biometrics associated with either a lineage or a motion lineage. Touch recognition is user friendly and can provide smooth login and authentication.

[0287] Neural algorithms can be developed as software and added to previous existing electronic devices with an application programming interface (API) or mobile software development kit. It can be added as part of the system login.

[0288] Neural algorithms do not need to continuously monitor sensors. Neural algorithms can be in a sleep state until the electronic device is woken from a sleep state or the sensor senses a touch. Training time and recognition are fast (only a few seconds). Thus, neural algorithms can save power and use battery power that is typically available for mobile device electronics intelligently.

[0289] ​

[0290] In an embodiment in which embodiments are implemented in software, an element of an embodiment is essentially the code, or portions of code, executable by one or more processors for performing and implementing tasks and functions. A program or code segment, can be stored, for example, in a processor readable medium or storage device coupled to or at least in communication with the one or more processors. The processor readable medium can include any medium or storage device capable of storing information. Examples of processor readable media include, but are not limited to, electronic circuitry, semiconductor memory devices, read-only memory (ROM), flash memory, erasable programmable read-only memory (EPROM), floppy disks, CD-ROMs, optical disk, hard disks, or solid-state drives. A program or code segment can be downloaded or transferred, for example, over a computer network (e.g., the Internet, an intranet, etc.) between storage devices.

[0291] While this specification contains many details, these should not be construed as limitations on the scope of or content of the application, but as descriptions of particular embodiments of the application. Certain features that are, for clarity, described above in the context of separate embodiments, can also be provided in combination in a single embodiment. Conversely, various features that are, for brevity, described above in the context of a single embodiment, can also be provided separately or in any suitable subcombination. In addition, while a feature can have been described above as having particular

[0292] Accordingly, while certain exemplary embodiments have been particularly described and shown, these should not be construed as limiting the application, but rather the scope of the application should be interpreted according to the appended claims.

Claims

1. A method for authenticating a user on an electronic device, system, or software application, the method comprising: The first user identifier of the receiving user; Sensors are used to sense the movement of the user's body parts to generate three-dimensional motion signals, which include large-amplitude motion signals associated with large movements of the body parts and micro-motion signals associated with the user's brain's motor control and the quality control of the neuromuscular system. The sampled motion signal is filtered using a bandpass filter, the frequency response of which is correlated with the frequency range of the micro-motion signal; A set of values ​​for multiple unique features is extracted from the filtered and sampled motion signal, and the set of values ​​for the multiple unique features forms a user-unique neuromechanical fingerprint based on the motion signal. The neuromechanical fingerprint of the user is evaluated within a database of N user calibration parameters associated with N authorized users to determine the maximum percentage of matches; and In response to the maximum match percentage, the user is determined to be an authorized user of the electronic device, the system, or the software application by comparing the maximum match percentage with the percentage of authorized users. and Based on the neuromechanical fingerprint and the authorized N user calibration parameters, the user's access to the electronic device, the system, or the software application is controlled.

2. The method of claim 1, wherein controlling the user's access comprises: In response to authenticating the user's first user identifier and the neuromechanical fingerprint, access to the electronic device, the system, or the software application is granted.

3. The method according to claim 1, wherein the system is a server and controlling the user's access includes: A token is generated in response to the first user identifier and the neuromechanical fingerprint used to authenticate the user. and The token is sent to the server to obtain access to the server.

4. The method according to claim 1, wherein the first user identifier of the user is a personal identification number.

5. The method according to claim 1, wherein the first user identifier of the user is a fingerprint.

6. A method comprising: One or more touch-sensitive devices using the user interface of an electronic device are used to receive the movement of a user’s body parts to generate three-dimensional motion signals, the motion signals including large-amplitude motion signals associated with large movements of the body parts and micro-motion signals associated with the user’s brain’s motor control and the quality control of the neuromuscular system. The motion signal is sampled; The sampled motion signal is filtered using a bandpass filter, the frequency response of which is correlated with the frequency range of the micro-motion signal; A set of values ​​for multiple unique features is extracted from the filtered and sampled motion signal, and the set of values ​​for the multiple unique features forms the user's neuromechanical fingerprint; The user of the electronic device is authenticated using the neuromechanical fingerprint; and Access to the electronic device is controlled based on the user's authentication, authorized user calibration parameters, and the selection of one or more touch-sensitive devices in the user interface.

7. The method of claim 6, wherein the one or more touch-sensitive devices of the user interface include at least one of a power button, a function button, and a plurality of buttons for entering numbers.

8. The method of claim 7, wherein the number is a personal identification number for further authentication of the user of the electronic device.

9. The method of claim 7, wherein the number is a telephone number to control the electronic device to make a phone call.

10. The method of claim 6, wherein The one or more touch-sensitive devices of the user interface are function buttons, and the body part is a finger; The function button includes an image scanner; and The method further includes authenticating the user of the electronic device using an image of the finger.

11. The method of claim 10, wherein authenticating the user of the electronic device using the neuromechanical fingerprint comprises: At least one of the one or more touch-sensitive devices of the user interface is used to sense three-dimensional motion in the user's finger to generate the three-dimensional motion signal; The neuromechanical fingerprint is generated in response to the three-dimensional motion signal; The user's neuromechanical fingerprint is evaluated to determine the maximum percentage of matches; and The user of the electronic device is authenticated by comparing the maximum matching percentage with the percentage of authorized users.

12. A method for logging into an electronic device, system, or software application, the method comprising: Authenticate users using the first authentication component; In response to further authentication of the user using a neuromechanical fingerprint, wherein further authentication of the user using the neuromechanical fingerprint includes: The three-dimensional motion of a user’s body parts is sensed to generate motion signals, which include large-amplitude motion signals associated with large movements of the body parts and micro-motion signals associated with the user’s brain’s motor control and the quality control of the neuromuscular system. The motion signal is sampled; The sampled motion signal is filtered using a bandpass filter, the frequency response of which is correlated with the frequency range of the micro-motion signal; A set of values ​​for multiple unique features is extracted from the filtered and sampled motion signal, and the set of values ​​for the multiple unique features forms the user's unique neuromechanical fingerprint. and Access to the electronic device, the system, or the software application is controlled based on authenticating the user using the first authentication component and authenticating the user using the neuromechanical fingerprint and authorized user calibration parameters.

13. The method of claim 12, further comprising: The neuromechanical fingerprint is received as a login identifier; The neuromechanical fingerprint of the user is evaluated within a database of N user calibration parameters associated with N authorized users to determine the maximum percentage of matches; and In response to the maximum match percentage, the user is determined to be an authorized user of the system or the application by comparing the maximum match percentage with the percentage of authorized users.

14. The method of claim 13, further comprising: In response to the maximum matching percentage, the user is identified by the correlation between the N authorized users and the N user calibration parameters; and The user identified as one of the N authorized users is permitted to use the electronic device, the system, or the software application.

15. The method of claim 12, wherein the first authentication component is an image scanner and the user is further authenticated by an image of a finger, hand, iris of the eye, or face.

16. The method of claim 12, wherein the first authentication element is a touch pad sensor and the user is further authenticated by one or more keystrokes or signatures.

17. The method of claim 12, wherein the first authentication component is a microphone and further authenticates the user via voice.

Images