Authentication using augmented reality facial expression and body motion tracking

Abstract

In some embodiments, a system includes a processor and a non-transitory computer-readable medium coupled to the processor, where the non-transitory computer-readable medium includes code that: requests a comparison of user expressions from a user of an application of a user device; mapping, at an authentication server, the comparative user expression to a comparative mixed shape coefficient and a comparative mixed shape coefficient value; requesting an authentication user expression from the authentication server to the user of the application on the user device; mapping, at the authentication server, the authenticated user expression to an authentication mixed shape coefficient and an authentication mixed shape coefficient value; and authenticating the user of the user device using the authenticating user expression and the comparing user expression.

Description

Background Art

[0001] The background description provided herein is for the purpose of generally presenting the context of the present disclosure. To the extent described in this background section, the work of the presently named inventors and various aspects described that may not otherwise be prior art at the time of filing are neither explicitly nor implicitly admitted to be prior art for the present disclosure.

[0002] With the rapid adoption of mobile financial technology solutions, second-factor authentication is becoming increasingly important in authenticating users of mobile device applications. Traditional application authentication methods, such as one-time passwords (OTPs) or two-factor authentication methods, present unique challenges based on, for example, the application user's geographic location and local regulations imposed by local governments. Security issues associated with existing authentication methods can be particularly significant for specially-abled users of applications, as they are more likely to be exploited during the authentication process. Therefore, there is a need to improve existing authentication applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] Figure 1A A block diagram of a system according to some embodiments is shown.

[0004] Figure 1B Shown according to some embodiments include Figure 1A Block diagram of the authentication network of the system.

[0005] Figure 2 Shown according to some embodiments Figure 1A Block diagram of the user expression-based authentication unit of the system.

[0006] Figure 3 FIG. 1 is a flow chart illustrating a method for mapping based on user expressions according to some embodiments.

[0007] Figure 4 Flowchart showing an authentication method based on user expression according to some embodiments.

[0008] Figure 5A A table of user expression blend shape coefficients is shown in accordance with some embodiments.

[0009] Figure 5B Shown according to some embodiments Figure 5A Continuation of the user expression blend shape coefficients table.

[0010] Figure 6 Showing that according to some embodiments, Figure 4 The user expression-based authentication method utilizes an image sequence of the user's left eye blinking user expression.

[0011] Figure 7 Showing that according to some embodiments, Figure 4 The user expression-based authentication method utilizes an image sequence of the user's jaw-opening user expression. DETAILED DESCRIPTION

[0012] Figure 1A A block diagram of an exemplary system 100 for implementing embodiments consistent with the present disclosure is shown. In some embodiments, the system 100 includes an input / output (IO) interface 101, a processor 102, a storage interface 104, a network interface 103, and a memory 105. In some embodiments, the system 100 may be, for example, a server, such as an authentication server for implementing the embodiments described herein. In some embodiments, the memory 105 may include an operating system (OS) 107, a process 120, and a user expression-based authentication unit 130. In some non-limiting embodiments or aspects, the system 100 provides a user expression-based authentication unit 130, which is configured to implement a user expression-based authentication method configured to authenticate a user of a user device, as further described herein.

[0013] In some embodiments, the processor 102 may include at least one data processor for executing program components for dynamic resource allocation at runtime. The processor 102 may include special processing units, such as an integrated system (bus) controller, a memory management control unit, a floating point unit, a graphics processing unit, a digital signal processing unit, etc. In some embodiments, the processor 102 may be arranged to communicate with one or more input / output (I / O) devices (not shown) via an I / O interface 101. The I / O interface 101 may employ communication protocols / methods such as, but not limited to, audio, analog, digital, mono, RCA, stereo, IEEE-1394, serial bus, universal serial bus (USB), infrared, PS / 2, BNC, coaxial, component, composite, digital video interface (DVI), high-definition multimedia interface (HDMi), RF antenna, S-Video, VGA, IEEE 802.1n / b / g / n / x, Cellular (e.g., Code Division Multiple Access (CDMA), High Speed ​​Packet Access (HSPA+), Global System for Mobile Communications (GSM), Long Term Evolution (LTE), etc.

[0014] In some embodiments, the system 100 can communicate with one or more I / O devices using the I / O interface 101. For example, an input device (not shown) can be an antenna, keyboard, mouse, joystick, (infrared) remote control, camera, card reader, fax machine, dongle, biometric reader, microphone, touch screen, touch pad, trackball, stylus, scanner, storage device, transceiver, video device / source, etc. An output device (not shown) can be a printer, fax machine, video display (e.g., cathode ray tube (CRT), liquid crystal display (LCD), light emitting diode (LED), plasma, plasma display panel (PDP), organic light emitting diode display (OLED)), audio speaker, etc.

[0015] In some embodiments, the processor 102 may be arranged to communicate with a communication network or other type of network via a network interface 103. The network interface 103 may communicate with the communication network. The network interface 103 may employ connection protocols including, but not limited to, direct connection, Ethernet (e.g., twisted pair 10 / 100 / 1000Base T), Transmission Control Protocol / Internet Protocol (TCP / IP), Token Ring, IEEE 802.11a / b / g / n / x, etc. The communication network may include, but not limited to, direct interconnection, e-commerce network, peer-to-peer (P2P) network, local area network (LAN), wide area network (WAN), wireless network (e.g., using wireless application protocol), the Internet, Etc. Using the network interface 103 and a communication network, the system 100 can communicate with one or more service operators.

[0016] In some non-limiting embodiments or aspects, the processor 102 may be arranged to communicate with a memory 105 (e.g., RAM, ROM, etc.) via a storage interface 104. In some embodiments, the storage interface 104 may be connected to a memory 105 including, but not limited to, a memory drive, a removable optical drive, etc., using a connection protocol such as Serial Advanced Technology Attachment (SATA), Integrated Drive Electronics (IDE), IEEE-1394, Universal Serial Bus (USB), Fibre Channel, Small Computer System Interface (SCSI), etc. The memory drive may also include a drum, a magnetic disk drive, a magneto-optical disk drive, an optical disk drive, a redundant array of independent disks (RAID), a solid-state memory device, a solid-state hard disk, etc.

[0017] In some embodiments, the memory 105 may store a collection of program or database components, including but not limited to a user interface, an operating system 107, a web server, etc. In some non-limiting embodiments or aspects, the system 100 may store user / application data, such as data, variables, records, etc., as described in the present disclosure. Such a database may be implemented as a fault-tolerant, relational, scalable, secure database, such as Oracle or Sybase.

[0018] In some embodiments, operating system 107 may facilitate resource management and operation of system 100. Examples of operating systems include, but are not limited to, Apple OS UNIX-like system distributions (e.g., Berkeley Software (BSD), OPENBSD, etc.), DISTRIBUTIONS (e.g., RED wait), ( / 7 / 8, 10, etc.), GOOGLE TM Android TM 、 OS, etc.

[0019] In some non-limiting embodiments or aspects, the system 100 may implement a program component stored in a web browser (not shown). The web browser (not shown) may be a hypertext viewing application such as Microsoft INTERNET GOOGLE TM CHROME TM 、 Secure web browsing can be provided using Hypertext Transfer Protocol Secure (HTTPS), Secure Sockets Layer (SSL), Transport Layer Security (TLS), etc. Web browsers can use technologies such as AJAX, DHTML, Application Programming Interface (API) and other facilities.

[0020] In addition, one or more computer-readable storage media can be utilized to implement embodiments consistent with the present disclosure. In some embodiments, a computer-readable storage medium refers to any type of physical memory that can store information or data that can be read by a processor. Therefore, a computer-readable storage medium can store instructions executed by one or more processors, including instructions that cause the processor to execute steps or stages consistent with the embodiments described herein. The term "computer-readable medium" should be understood to include tangible items and does not include carrier waves and transient signals, such as non-transient ones. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, non-volatile memory, hard drive, compact disc (CD) ROM, digital video disc (DVD), flash drive, disk, and any other known physical storage medium.

[0021] Figure 1B A block diagram of an authentication network 180 according to some embodiments is shown. In some embodiments, the authentication network 180 is configured to utilize the system 100 as an authentication server to authenticate a user of an application 191 on a user device 190. In some embodiments, the authentication network 180 includes a user device 190 communicatively coupled to the system 100 via a communication network. In some embodiments, the user device 190 may be a computer having a computer system similar to Figure 1A The computer system of the configuration of the system 100 (excluding the authentication unit 130 based on user expression) can be a mobile phone, tablet computer or other type of computer system. In some embodiments, the user device 190 includes a capture device 195 (such as a camera) and an application 191. In some embodiments, the application 191 is a software application configured to perform at least a portion of the operations described herein. In some embodiments, the application 191 can be, for example, a financial technology application that is configured to authenticate the user of the user device 190 using the method described herein. In some embodiments, the system 100 is an authentication server that is configured to use the method described herein with reference to Figures 2 to 7 The method is further described to authenticate the user of the user device 190.

[0022] Figure 2 Shown according to some embodiments Figure 1A191 or the like. In some embodiments, the user expression based authentication unit 130 is an executable code configured to authenticate a user of the application 191 using a captured user expression or sequence of user expressions of the user of the user device 190. In some embodiments, the user expression may be an expression or motion of the user of the user device 190, which is captured using the capture device 195 on the user device 190 and provided to the authentication server for authenticating the user to use the application 191 or the user device 190. In some embodiments, for example, the user expression may be a facial expression (e.g., a smile, a frown, etc.) or a motion (e.g., a body movement, etc.) of the user captured by the capture device 195. In some embodiments, the user expression based authentication unit 130 is configured to map or encode the user expression to blend shape coefficients and blend shape coefficient values ​​(described in further detail herein) in the memory 105, and to authenticate the user of the user device 190 using the user expression and the blend shape coefficient values ​​(e.g., with the aid of Figure 4 ).

[0023] In some embodiments, the user expression-based authentication unit 130 includes a user expression request unit 211, a user expression comparison unit 212, and a blend shape coefficient mapping unit 213. In some embodiments, the user expression request unit 211 is executable code configured to request a user expression or a sequence of user expressions from the user of the user device 190 to be stored in the memory 105. In some embodiments, each type of user expression associated with the user is stored in the user expression blend shape coefficient table 221 in the memory 105 (with the aid of Figure 5A and Figure 5B ). In some embodiments, the user expression blend shape coefficient table 221 includes user expressions mapped to blend shape coefficients 251, blend shape coefficient values ​​252, and scaling factors 253. In some embodiments, the blend shape coefficients in the blend shape coefficients 251 are named representations of user expressions expressed by the user of the user device 190. In some embodiments, the blend shape coefficient values ​​of the blend shape coefficient values ​​252 are numerical values ​​assigned to the blend shape coefficients by the blend shape coefficient mapping unit 213 for authentication purposes by the user expression-based authentication unit 130. For example, in some embodiments, the blend shape coefficient values ​​252 may be integers greater than or equal to zero, the enumeration of which depends on the number of blend shape coefficients (e.g., by means of Figure 5A and Figure 5BIn some embodiments, the scaling factor in scaling factor 253 is a numerical value assigned to the blend shape coefficient to determine whether the user expression is a valid user expression (e.g., a valid facial expression or a valid motion). For example, in some embodiments, scaling factor 253 can be a decimal value ranging from zero to one, depending on the assigned value of the user expression-based authentication unit 130 (e.g., by means of Figure 5A and Figure 5B ).

[0024] In some embodiments, the scaling factor threshold 271 is a threshold value set by, for example, a developer of the application 191, the system 100, or more specifically the user expression-based authentication unit 130, and is configured to be used as a threshold value utilized by the user expression-based authentication unit 130 to indicate whether a user expression captured by the user device 190 is a valid user expression. For example, in some embodiments, for a scaling factor threshold of 0.3, a user expression of a user received by the user expression-based authentication unit 130 may not be evaluated by the user expression-based authentication unit 130 as a valid user expression unless the scaling factor value is greater than the scaling factor threshold of 0.3. In some embodiments, the blending shape coefficients (e.g., Figure 3 ) and assigned to the user during authentication (as shown in Figure 4 The mixed shape coefficients of the authentication user expression or authentication user expression sequence provided by the user expression-based authentication unit 130 are utilized by the user expression-based authentication unit 130 to authenticate the user of the user device 190.

[0025] In some embodiments, the blend shape coefficient mapping unit 213 is executable code configured to map user expressions to blend shape coefficients and blend shape coefficient values, as in, for example, Figure 5A and Figure 5B, is shown in user expression blend shape coefficient table 221. In some embodiments, blend shape coefficient mapping unit 213 is configured to map the comparison user expression to a comparison blend shape coefficient, a comparison blend shape coefficient value, and a comparison scaling factor. In some embodiments, blend shape coefficient mapping unit 213 is configured to map the authentication user expression to an authentication blend shape coefficient, an authentication blend shape coefficient value, and an authentication scaling factor. In some embodiments, the comparison user expression is a user expression received from user device 190 and stored in memory 105 for comparison by user expression comparison unit 212 with an authentication user expression received during authentication of a user of application 191 of user device 190. In some embodiments, the authentication user expression is a user expression received from user device 190 during user authentication of application 191 of user device 190, the user expression being used to authenticate the user of application 191 of the user device. In some embodiments, the authentication user expression (or associated authentication blend shape coefficient value) is compared by user expression comparison unit 212 with a stored comparison user expression (or associated comparison blend shape coefficient value) to authenticate the user of application 191 of the user device.

[0026] In some embodiments, the user expression comparison unit 212 is executable code configured to determine whether an authentication user expression or an authentication user expression sequence matches a comparison user expression or a comparison user expression sequence. In some embodiments, the user expression comparison unit 212 is configured to compare a stored comparison user expression or a stored comparison user expression sequence with the authentication user expression or the authentication user expression sequence. In some embodiments, the user expression comparison unit 212 may be configured to compare a comparison coefficient value mapped to the comparison user expression (or a comparison coefficient value mapped to the comparison user expression sequence) with an authentication coefficient value mapped to the authentication user expression (or an authentication coefficient value mapped to the authentication user expression sequence). In some embodiments, the user expression comparison unit 212 is configured to determine whether the authentication user expression sequence matches the comparison user expression sequence provided as a password by the user of the user device 190. In some embodiments, the user expression request unit 211, the user expression comparison unit 212, and the blend shape coefficient mapping unit 213 are collectively configured to perform user expression-based authentication to authenticate the user of the user device 190, as further described herein.

[0027] Figure 3Flowchart illustrating a user expression-based mapping method 300 according to some embodiments. In some embodiments, the user expression-based mapping method 300 is a method utilized by the user expression-based authentication unit 130 to map a user expression to a blending shape coefficient 251, a blending shape coefficient value 252, and a scaling factor 253 for authenticating a user of the user device 190. In some embodiments, the user expression-based authentication unit 130 is configured to perform the user expression-based mapping method 300. The methods, process steps, or stages shown in the accompanying drawings can be implemented as independent routines or processes, or as part of a larger routine or process. It should be noted that in other embodiments, each process step or stage depicted can be implemented as a device, method, or system including a processor that executes a set of instructions. In some embodiments, the user expression-based mapping method 300 is described with reference to the figures described herein.

[0028] In some embodiments, at operation 305, a user of user device 190 selects application 191 (e.g., a financial application, etc.) and logs into application 191 at operation 310. In some embodiments, after logging into application 191 at operation 310, user expression-based authentication unit 130 determines, via application 191, whether the user has previously logged into application 191 at operation 315. In some embodiments, user expression-based authentication unit 130 determines whether the user has previously logged into application 191 by evaluating a cookie stored on user device 190 or by evaluating a record associated with the user stored on system 100 (which may be, for example, an authentication server). In some embodiments, at operation 320, when user expression-based authentication unit 130 determines that the user has previously logged into the application at operation 315, operation 320 continues with normal application flow on application 191 because the user has previously been authenticated.

[0029] In some embodiments, at operation 325, when the user expression-based authentication unit 130 determines, via, for example, the application 191, that the user has not previously logged into the application 191 (for example, the user is logging into the application 191 for the first time), the user expression request unit 211 of the user expression-based authentication unit 130 requests the user to capture or record the user's user expression for the purpose of authentication of the application and / or the user device 190, via, for example, a prompt on the application 191 of the user device 190. In some embodiments, the user expression request unit 211 requests the user to capture or record the user's user expression using the capture device 195 of the user device 190, via, for example, a prompt on the application 191 of the user device 190.

[0030] In some embodiments, as previously described, the user expression requested by the user expression request unit 211 via the application 191 is a user's facial expression or body movement, which is used by the authentication server to authenticate the user of the application 191 on the user device 190. For example, in some embodiments, the user expression may be pouting, puffing cheeks, sneering nose, or opening jaw, or other user expressions, such as Figure 5A and Figure 5B The user expression blend shape coefficients are shown in Table 221. Figure 5A and Figure 5B , as shown in the example of the user expression blend shape coefficient table 221 shown in . In some embodiments, the user expression request unit 211, via the application 191, can be configured to indicate to the user that a user expression needs to be captured using the user device 190. In some embodiments, the user expression request unit 211 can request the user to capture a random expression and then map the user expression to a corresponding blend shape coefficient and blend shape coefficient value. In some embodiments, the user expression-based authentication unit 130 is configured to identify special characteristics of the user of the user device 190 and provide the services provided herein accordingly. For example, in some embodiments, the user expression-based authentication unit 130 can identify that the user is a special ability user, for example, through voluntary information provided by the user of the user device 190, and thus notify the user based on the characteristics of the special ability user. For example, if the user is blind, the user expression-based authentication unit 130 can request the user expression by vibrating the user device or via an audio component on the user device 190. In some embodiments, operation 325 proceeds to operation 350.

[0031] In some embodiments, at operation 350, the user expression-based authentication unit 130 receives a user expression from the user device 190 and determines whether the capture device 195 has captured the appropriate user expression. In some embodiments, a prompt may be provided to the user of the application 191 to determine whether the requested user expression has been captured. In some embodiments, when the user expression-based authentication unit 130 determines that the requested user expression has not been captured, operation 350 proceeds to operation 325 to prompt the user to capture a user expression until the requested user expression is captured or the application 191 exits the user from the application 191. In some embodiments, when the user expression-based authentication unit 130 determines that the requested user expression has been captured, operation 350 proceeds to operation 360.

[0032] In some embodiments, at operation 360, the user expression-based authentication unit 130 maps, encodes, or associates the user expression received from the user device 190 with the blending shape coefficients, blending shape coefficient values, and scaling factors assigned and / or enumerated by the user expression-based authentication unit 130. In some embodiments, as previously described, the blending shape coefficients, blending shape coefficient values, and scaling factors are assigned by the user expression-based authentication unit 130 to the user expression used to authenticate the user expression. For example, Figure 5A As shown in the user expression blend shape coefficient table 221, when the user expression received by the user expression-based authentication unit 130 is recognized as one or more puffed cheeks, the blend shape coefficient mapping unit 213 assigns the user expression a blend shape coefficient "cheekPuff," a blend shape coefficient value of 5, and a scaling factor depending on the validity level of the received user expression. In some embodiments, after the user expression is mapped to the blend shape coefficient value at operation 360, the user expression-based authentication unit 130 stores the user expression and the blend shape coefficient value in the memory 105 of the authentication server at operation 370.

[0033] In some embodiments, after the user expression, blending shape coefficient, blending shape coefficient value and scaling factor have been stored by the user expression-based authentication unit 130, the user expression, blending shape coefficient, blending shape coefficient value and scaling factor can be utilized by the user expression-based authentication unit 130 to authenticate the user using the user expression-based authentication method described herein.

[0034] Figure 4 Flowchart showing a method 400 for authentication based on user expressions according to some embodiments. In some embodiments, the method 400 for authentication based on user expressions is a method utilized by the user expression-based authentication unit 130 to authenticate a user of an application 191 of a user device 190. The methods, process steps, or stages shown in the accompanying drawings may be implemented as independent routines or processes, or as part of a larger routine or process. Note that in other embodiments, each process step or stage depicted may be implemented as a device, method, or system including a processor that executes a set of instructions. In some embodiments, the user expression-based authentication unit 130 is configured to perform the method 400 for authentication based on user expressions according to some embodiments. In some embodiments, the method 400 for authentication based on user expressions is described with reference to the diagrams described herein.

[0035] In some embodiments, at operation 405, a user of user device 190 opens application 191 and logs into application 191 at operation 410. In some embodiments, after logging into application 191 at operation 410, at operation 415, user expression-based authentication unit 130 determines whether multi-factor authentication (MFA) or two-factor authentication (2FA) is required. In some embodiments, user expression-based authentication unit 130 determines whether MFA or 2FA is required by evaluating the user's account information of user device 190 to determine whether the user of user device 190 selected the MFA or 2FA option during the initial login to application 191. In some embodiments, user expression-based authentication unit 130 may require MFA or 2FA based on user expression-based authentication unit 130's identification of a potential fraudulent transaction involving a user associated with application 191 and / or user device 190. In some embodiments, application 191 may require the use of MFA or 2FA, and thus, by default, application 191 may utilize MFA or 2FA. In some embodiments, when the user expression based authentication unit 130 determines that MFA or 2FA is not required, the user expression based authentication unit 130 allows the user to utilize the application 191 at operation 420 without further authentication.

[0036] In some embodiments, at operation 425, when the user expression-based authentication unit 130 determines that MFA or 2FA is required, the user expression request unit 211 of the user expression-based authentication unit 130 requests the user, for example, via a prompt on the application 191 of the user device 190, to capture an authentication user expression or an authentication user expression sequence using the capture device 195 in order to authenticate the user using the user expression-based authentication unit 130. For example, in some embodiments, the user expression-based authentication unit 130 may indicate to the user, "For authentication purposes, please provide an authentication user expression." In some embodiments, the user then captures an authentication user expression (or an authentication user expression sequence) using the capture device 195 of the user device 190, and the user expression-based authentication unit 130 receives the captured authentication user expression (or authentication user expression sequence) from the user device 190. In some embodiments, the user expression-based authentication unit 130 may specify one or more specific authentication user expressions that the user is required to capture using the capture device 195. For example, the user expression-based authentication unit 130 may specify that a left eye blink is required to be captured as a user expression, that an open mouth is required to be captured as a user expression, and so on. In some embodiments, after capturing the authentication user expression from the user of the user device 190, the captured authentication user expression is provided to the user expression comparison unit 212 of the user expression-based authentication unit 130. In some embodiments, operation 425 proceeds to operation 430.

[0037] In some embodiments, at operation 430, the blend shape coefficient mapping unit 213 maps the one or more authentication user expressions to one or more authentication blend shape coefficients, one or more authentication blend shape coefficient values, and one or more authentication scaling factors, following a process similar to the process for mapping the comparison user expressions to comparison blend shape coefficients, blend shape coefficient values, and comparison scaling factors. In some embodiments, after performing the mapping operation, operation 430 proceeds to operation 440.

[0038] In some embodiments, at operation 440, after receiving the authentication user expression or the authentication user expression sequence, the user expression comparison unit 212 of the user expression-based authentication unit 130 evaluates the authentication user expression and determines whether the authentication user expression or the authentication user expression sequence matches the comparison user expression of the user stored in the memory 105.

[0039] In some embodiments, the user expression comparison unit 212 compares the authentication blend shape coefficient value associated with the received authentication user expression with the authentication blend shape coefficient value stored in the user expression blend shape coefficient table 221 (with the aid of Figure 5A and Figure 5B The comparison mixing coefficient values ​​associated with the comparison user expression in the example shown in FIG) are compared to determine whether the authentication user expression (or authentication user expression sequence) matches the comparison user expression (or comparison user expression sequence).

[0040] In some embodiments, at operation 445, when the user expression comparison unit 212 determines that the authentication user expression or the authentication user expression sequence does not match the comparison user expression represented in the user expression blend shape coefficient table 221, the user expression-based authentication unit 130 does not use the authentication user expression to authenticate the user, and continues to request one or more additional authentication user expressions for authenticating the user using the user expression request unit 211. In some embodiments, operations 445, 425, and 440 may be repeated for a limited number of examples indicated by the user expression-based authentication unit 130.

[0041] In some embodiments, at operation 450, when the user expression comparison unit 212 determines that the authentication user expression or the authentication user expression sequence matches the user expression represented in the user expression blending shape coefficient table 221, the user expression-based authentication unit 130 authenticates the user and continues to allow the user to utilize the application 191 or the user device 190.

[0042] Figure 5A and Figure 5BFIG2 shows a user expression blend shape coefficient table 221 according to some embodiments. In some embodiments, as previously described, the user expression blend shape coefficient table 221 represents the user expression mapped to the blend shape coefficient 251, the blend shape coefficient value 252, and the scaling factor 253. In some embodiments, for Figure 5A and Figure 5B , there are fifty-one user expressions that have been mapped to fifty-one blend shape coefficients 510, fifty-one blend shape coefficient values ​​515, and fifty-one scaling factors 520 by the user expression based authentication unit 130. For illustration purposes, the possible scaling factor range (e.g., a range of 0 and 1) for each blend shape coefficient is shown in the user expression blend shape coefficient table 221 as scaling factor 520.

[0043] Figure 6 A sequence of images 610 representing user expressions used to authenticate the user is shown according to some embodiments. Figure 6 In the example shown in , the user expression is a left eye blink user expression and is mapped to the eyeblinkleft blend shape coefficients and includes a visual representation of the associated scaling factor 620. Figure 6 In the example shown in FIG, the scaling factor 620 is mapped by the user expression based authentication unit 130 to a value of 1 for a left eye blink user expression (e.g., eyeblinkleft blend shape coefficient), and the blend shape coefficient value is mapped to a value of 8. In some embodiments, the user expression based authentication unit 130 utilizes Figure 6 to authenticate the user of the user device 190 using the methods described herein.

[0044] Figure 7 A sequence of images 710 showing user expressions used to authenticate the user is shown in accordance with some embodiments. Figure 7 In the example shown in , the user expression is an open jaw user expression and is mapped to the jaw open blend shape coefficients and includes a visual representation of an associated scaling factor 720. Figure 7 , the scaling factor 720 is mapped by the user expression based authentication unit 130 to a value of 1 for the open jaw user expression (e.g., the jawOpen blend shape coefficient), and the blend shape coefficient value is mapped to a value of 24. In some embodiments, the user expression based authentication unit 130 utilizes Figure 7 to authenticate the user of the user device 190 using the methods described herein.

[0045] In an alternative embodiment, the user expression-based authentication unit 130 may be configured to allow the user to select a specific user expression sequence as a password or for MFA / 2FA via the application 191 (e.g., the user selects or sets a stored user expression sequence of user expressions as a password in the authentication server). In some embodiments, the user expression sequence (represented using blend shape coefficient values) may be utilized by the user expression-based verification unit 130 for verification and authentication purposes (e.g., as a password or for MFA / 2FA). For example, the user may utilize the application 191 to select an open mouth, closed mouth, and left eye blink as a user expression sequence as a password or MFA / 2FA. In some embodiments, when the user expression request unit 211 requests the user to enter an MFA / 2FA or password, the user must provide the correct user expression sequence as the MFA / 2FA or password (which is compared with the stored MFA / 2FA or password by the user expression comparison unit 212 as previously described herein).

[0046] In another example, a user uses application 191 to select a user expression sequence as a password, the password corresponding to the eyeBlinkLeft blend shape coefficient, the mouthSmileLeft blend shape coefficient, the mouthSmileRight blend shape coefficient, and the jawOpen blend shape coefficient. In some embodiments, after the user expression is selected by the user of application 191, the user expression is provided to an authentication server (e.g., system 100). In some embodiments, after receiving the selected user expression sequence, the authentication server maps each user expression of the user expression sequence to its corresponding blend shape coefficient value (e.g., [8, 43, 44, 24]) and stores the user expression sequence as the password in the authentication server. In some embodiments, when the authentication server authenticates the password, the authentication server uses the values ​​mapped by the authentication server to verify or authenticate each user expression.

[0047] In some embodiments, the user can utilize any sequence of expressions between 0 and 50 as the user expression for the password. In some embodiments, using the methods described herein, the system 100 is able to provide a greater number of password combinations than conventional computer systems that typically utilize password numbers ranging from 0 to 9.

[0048] In some embodiments, the application 191 may utilize augmented reality (AR) technology to track the user's facial expressions and body movements (which may be used for authentication purposes instead of traditional passwords or OTPs).

[0049] In some embodiments, in addition to the user's facial expressions for authentication purposes, the user may use a combination of body movements and / or facial expressions for authentication. In some embodiments, the use of a combination of body movements and / or facial expressions makes fraudulent authentication by fraudsters more difficult because the fraudster and / or computer system may not be able to guess the combination the user uses for the password. In some embodiments, augmented reality (AR) technology, such as a head-mounted device (HMD), can be used to identify and track the user's movements using, for example, a rear camera of a mobile device. In some embodiments, AR applies the detected movements to a three-dimensional (3D) character model in real time. In some embodiments, body movements can be verified using, for example, the position of the joints that make up the user's skeleton and the position of the joints in relation to other joints.

[0050] In some embodiments, a scaling factor value of 0 is considered "neutral" (eg, a user expression captured by user device 190 is not considered a valid user expression), and a scaling factor value greater than 0.3 is considered a valid user expression.

[0051] In some embodiments, a computer-implemented method includes: requesting a comparison user expression from an authentication server to a user of an application on a user device; mapping the comparison user expression to a comparison blend shape coefficient and a comparison blend shape coefficient value at the authentication server; requesting an authentication user expression from the authentication server to a user of an application on the user device; mapping the authentication user expression to an authentication blend shape coefficient and an authentication blend shape coefficient value at the authentication server; and authenticating the user of the user device using the authentication user expression and the comparison user expression.

[0052] In some embodiments of the computer-implemented method, authenticating the user of the user device using the authentication user expression and the comparison user expression includes comparing the authentication user expression with the comparison user expression stored in the authentication server.

[0053] In some embodiments of the computer-implemented method, the user of the user device is authenticated when the authentication user expression matches the comparison user expression.

[0054] In some embodiments of the computer-implemented method, the comparison blend shape coefficient values ​​are integer values ​​mapped to the comparison blend shape coefficients, and the authentication blend shape coefficient values ​​are integers mapped to the authentication blend shape coefficients.

[0055] In some embodiments, the computer-implemented method further comprises mapping the comparative user expressions to a scaling factor.

[0056] In some embodiments, the computer-implemented method further comprises utilizing a scaling factor to determine whether the comparison user expression is a valid user expression.

[0057] In some embodiments of the computer-implemented method, the scaling factor is a decimal value in the range of zero to one.

[0058] In some embodiments of the computer-implemented method, authenticating the user of the user device using the authentication user expression and the comparison user expression includes comparing the authentication blend shape coefficient value with the comparison blend shape coefficient value stored in the authentication server.

[0059] In some embodiments, a system includes: a processor; and a non-transitory computer-readable medium coupled to the processor, the non-transitory computer-readable medium including code that: requests a comparison user expression from a user of an application on a user device; maps the comparison user expression to a comparison blend shape coefficient and a comparison blend shape coefficient value; requests an authentication user expression from a user of an application on the user device; maps the authentication user expression to an authentication blend shape coefficient and an authentication blend shape coefficient value; and authenticates the user of the user device using the authentication user expression and the comparison user expression.

[0060] In some embodiments of the system, a non-transitory computer-readable medium includes code for authenticating a user of the user device using the authentication user expression and the comparison user expression including comparing the authentication user expression to the comparison user expression.

[0061] In some embodiments of the system, the non-transitory computer-readable medium includes code for authenticating a user of the user device when the authentication user expression matches the comparison user expression.

[0062] In some embodiments of the system, the comparison blend shape coefficient values ​​are integer values ​​that map to comparison blend shape coefficients.

[0063] In some embodiments of the system, the certified blend shape coefficient values ​​are integers that map to certified blend shape coefficients.

[0064] In some embodiments of the system, a non-transitory computer-readable medium includes code for mapping the comparison user expression to a scaling factor.

[0065] In some embodiments of the system, a scaling factor is used to determine whether a comparison user expression is a valid user expression.

[0066] In some embodiments of the system, the scaling factor is a decimal value ranging from zero to one.

[0067] In some embodiments of the system, the authenticated user expression is compared to stored comparison user expressions using artificial intelligence or machine learning techniques.

[0068] In some embodiments, a user expression-based authentication unit includes: a user expression request unit; a blended shape coefficient mapping unit coupled to the user expression request unit; and a user expression comparison unit coupled to the blended shape coefficient mapping unit, wherein the user expression-based authentication unit authenticates a user of an application on a user device based on an authentication comparison between a comparison user expression and an authenticated user expression performed using a comparison blended shape coefficient value and an authenticated blended shape coefficient value.

[0069] In some embodiments of the user expression based authentication unit, the blend shape coefficient mapping unit maps the comparison user expression to the authentication blend shape coefficient value and maps the authentication user expression to the authentication blend shape coefficient value.

[0070] In some embodiments of the user expression based authentication unit, the user of the application is authenticated when the compared blend shape coefficient values ​​match the authentication blend shape coefficient values.

Claims

1. A computer-implemented method comprising: a request from the authentication server to the user of the application on the user device to compare the user's expression; mapping the comparison user expressions to comparison blend shape coefficients and comparison blend shape coefficient values ​​at the authentication server; requesting, from the authentication server, an authentication user expression from the user of the application on the user device; mapping the authenticated user expression to authenticated blend shape coefficients and authenticated blend shape coefficient values ​​at the authentication server; as well as The user of the user device is authenticated using the authentication user expression and the comparison user expression.

2. The computer-implemented method of claim 1 , wherein: Authenticating the user of the user device using the authentication user expression and the comparison user expression includes comparing the authentication user expression with the comparison user expression stored in the authentication server.

3. The computer-implemented method of claim 2, wherein: When the authentication user expression matches the comparison user expression, the user of the user device is authenticated.

4. The computer-implemented method of claim 3, wherein: The comparison blend shape coefficient values ​​are integer values ​​that map to the comparison blend shape coefficients, and the authentication blend shape coefficient values ​​are integers that map to the authentication blend shape coefficients.

5. The computer-implemented method of claim 4 , further comprising: The comparison user expression is mapped to a scaling factor.

6. The computer-implemented method of claim 5 , further comprising: The scaling factor is used to determine whether the comparison user expression is a valid user expression.

7. The computer-implemented method of claim 6, wherein: The scaling factor is a decimal value ranging from zero to one.

8. The computer-implemented method of claim 1 , wherein: Authenticating the user of the user device using the authentication user expression and the comparison user expression includes comparing the authentication blend shape coefficient value with the comparison blend shape coefficient value stored in the authentication server.

9. A system comprising: processor; as well as a non-transitory computer-readable medium coupled to the processor, the non-transitory computer-readable medium comprising code, the code: requesting a comparison of user expressions from a user of an application on a user device; mapping the comparison user expressions to comparison blend shape coefficients and comparison blend shape coefficient values; requesting authentication of a user expression from the user of the application on the user device; Mapping the authenticated user expression to an authenticated blend shape coefficient and an authenticated blend shape coefficient value; as well as The user of the user device is authenticated using the authentication user expression and the comparison user expression.

10. The system of claim 9, wherein: The non-transitory computer readable medium includes code, the code: Authenticating the user of the user device using the authentication user expression and the comparison user expression includes comparing the authentication user expression with the comparison user expression.

11. The system of claim 10, wherein: The non-transitory computer readable medium includes code, the code: When the authentication user expression matches the comparison user expression, the user of the user device is authenticated.

12. The system of claim 11, wherein: The comparison blend shape coefficient values ​​are integer values ​​that map to the comparison blend shape coefficients.

13. The system of claim 12, wherein: The certified blend shape coefficient values ​​are integers that map to the certified blend shape coefficients.

14. The system of claim 13, wherein: The non-transitory computer readable medium includes code, the code: The comparison user expression is mapped to a scaling factor.

15. The system of claim 14, wherein: The scaling factor is used to determine whether the comparison user expression is a valid user expression.

16. The system of claim 15, wherein: The scaling factor is a decimal value ranging from zero to one.

17. The system of claim 16, wherein: The authenticated user expression is compared with the stored comparison user expression using artificial intelligence or machine learning techniques.

18. An authentication unit based on user expression, comprising: User expression request unit; a blend shape coefficient mapping unit coupled to the user expression request unit; as well as and a user expression comparison unit coupled to the blend shape coefficient mapping unit, wherein the user expression based authentication unit authenticates a user of an application on a user device based on an authentication comparison between the comparative user expression and the authenticated user expression using the comparative blend shape coefficient value and the authenticated blend shape coefficient value.

19. The user expression-based authentication unit according to claim 18, wherein: The blend shape coefficient mapping unit maps the comparison user expression to the authentication blend shape coefficient value, and maps the authentication user expression to the authentication blend shape coefficient value.

20. The user expression-based authentication unit according to claim 19, wherein: When the comparison blend shape coefficient value matches the authentication blend shape coefficient value, the user of the application is authenticated.

Images