Unlocking method, device, computer equipment and storage medium based on facial expression

Abstract

The present application relates to a facial expression-based unlocking method, device, computer equipment and storage medium. The method comprises: displaying an expression unlocking page; displaying an unlocking node sequence on the expression unlocking page; displaying a real-time acquired facial image in the facial preview area of ​​the expression unlocking page; generating an unlocking state identifier at the unlocking node to be processed in the unlocking node sequence based on the facial expression in the corresponding facial image; when an unlocking state identifier is generated at each unlocking node in the unlocking node sequence and the facial expression in each facial image matches the corresponding target expression, the unlocking is successful. The use of this method can effectively improve information security.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a facial expression-based unlocking method, apparatus, computer equipment and storage medium. Background Art

[0002] With the popularization of smart devices and the continuous development of information security, users have higher and higher requirements for information security. Therefore, the application of security locks is becoming more and more popular among users, such as device power-on password locks, application login security locks, and payment password locks, etc.

[0003] Common security lock unlocking solutions usually require entering a corresponding password to unlock, or entering a user's fingerprint to unlock, or collecting a face image for face recognition to unlock. For face recognition unlocking, a third party can unlock the lock using a photo or face model, thus creating a potential information security risk. Summary of the invention

[0004] Based on this, it is necessary to provide a facial expression-based unlocking method, device, computer equipment and storage medium that can improve information security in response to the above technical problems.

[0005] A facial expression-based unlocking method, the method comprising:

[0006] Display the expression unlock page;

[0007] Displaying the unlocking node sequence on the expression unlocking page;

[0008] Displaying the facial image collected in real time in the facial preview area of ​​the expression unlocking page;

[0009] At an unlocking node to be processed in the unlocking node sequence, generating an unlocking state identifier based on a facial expression in the corresponding facial image;

[0010] When an unlocking state identifier is generated at each unlocking node in the unlocking node sequence, and the facial expression in each facial image matches the corresponding target expression, the unlocking is successful.

[0011] In one embodiment, the facial image includes a face and a hand; the method further comprises:

[0012] During the facial expression recognition, performing gesture recognition on the hands in the facial image;

[0013] The step of generating an unlocking status mark at a corresponding unlocking node in the unlocking progress area each time facial expression recognition is completed includes:

[0014] Each time facial expression recognition and gesture recognition are completed, an unlocking state mark is generated at the corresponding unlocking node in the unlocking progress area.

[0015] In one embodiment, the method further comprises:

[0016] When the facial expressions in the at least two facial images match the corresponding target expressions, determining the acquisition time interval between the at least two facial images; or,

[0017] Determine an unlocking time interval between the unlocking node to be processed and the last unlocking node processed;

[0018] When the collection time interval or the unlocking time interval satisfies the corresponding time interval condition, the step of generating an unlocking state identifier at the unlocking node to be processed is performed.

[0019] In one embodiment, the step of facial expression recognition further comprises:

[0020] Extracting eyebrow feature points and eyelid feature points from the facial image;

[0021] determining a fifth distance between the eyebrow feature point and the eyelid feature point;

[0022] The eyebrow posture is determined according to the magnitude relationship between the fifth distance and the preset distance.

[0023] A facial expression-based unlocking device, the device comprising:

[0024] Display module, used to display the expression unlocking page;

[0025] A first display module, used to display an unlocking node sequence on the expression unlocking page;

[0026] A second display module, used to display the facial image collected in real time in the facial preview area of ​​the expression unlocking page;

[0027] A generating module, configured to generate an unlocking state identifier at an unlocking node to be processed in the unlocking node sequence based on a facial expression in the corresponding facial image;

[0028] The unlocking module is used to unlock successfully when an unlocking state mark is generated at each unlocking node in the unlocking node sequence and the facial expression in each facial image matches the corresponding target expression.

[0029] In one embodiment, the unlocking node sequence is displayed in the unlocking progress area of ​​the expression unlocking page. The generating module is further used to perform facial expression recognition on the facial images corresponding to the unlocking nodes to be processed in the unlocking node sequence in sequence according to the order of the unlocking nodes in the unlocking node sequence; when the facial expression recognition is completed each time, an unlocking state mark is generated at the corresponding unlocking node in the unlocking progress area.

[0030] In one embodiment, the device further includes: a generating module, a superposition module and a determining module; wherein:

[0031] The generating module is used to generate an expression model diagram corresponding to the facial expression each time the facial expression recognition is completed;

[0032] The superimposition module is used to superimpose and display the expression model diagram on the corresponding facial image in the facial preview area;

[0033] The determination module is used to determine the expression model diagram and the expression of the corresponding unlocking node. Figure 1 When the facial expression in the facial image matches the corresponding target expression, it is determined that the facial expression in the facial image matches the corresponding target expression.

[0034] In one embodiment, the facial image includes a face and a hand. The device further includes: a recognition module; wherein:

[0035] The recognition module is used to perform gesture recognition on the hands in the facial image during the process of facial expression recognition;

[0036] The generating module is further used to generate an unlocking status mark at a corresponding unlocking node in the unlocking progress area each time facial expression recognition and gesture recognition are completed.

[0037] In one embodiment, each unlocking node corresponds to at least two different target expressions. The generating module is further configured to perform facial expression recognition on at least two facial images corresponding to the unlocking node to be processed; when the facial expressions in the at least two facial images match the corresponding target expressions, an unlocking state mark is generated at the unlocking node to be processed.

[0038] In one embodiment, the determination module is further used to determine the acquisition time interval between the at least two facial images when the facial expressions in the at least two facial images match the corresponding target expressions; or determine the unlocking time interval between the unlocking node to be processed and the last processed unlocking node;

[0039] The generating module is further configured to generate an unlocking state identifier at the unlocking node to be processed when the collection time interval or the unlocking time interval satisfies a corresponding time interval condition.

[0040] In one embodiment, the recognition module is also used to extract eye feature points from the facial image; among the eye feature points, determine the first distance between the upper eyelid feature point and the lower eyelid feature point, and determine the second distance between the left eye corner feature point and the right eye corner feature point; determine the eye posture based on the relationship between the ratio between the first distance and the second distance and at least one preset interval.

[0041] In one embodiment, the recognition module is also used to extract lip feature points from the facial image; among the lip feature points, the lip posture is determined according to the height difference between the lip center feature point and the lip corner feature point; or, among the lip feature points, the lip posture is determined according to the third distance between the upper lip feature point and the lower lip feature point; or, among the lip feature points, the lip posture is determined according to the relationship between the ratio of the third distance to the fourth distance and at least one preset interval; wherein the fourth distance is the distance between the left lip corner feature point and the right lip corner feature point.

[0042] In one embodiment, the recognition module is also used to extract eyebrow feature points and eyelid feature points from the facial image; determine the fifth distance between the eyebrow feature points and the eyelid feature points; and determine the eyebrow posture based on the size relationship between the fifth distance and the preset distance.

[0043] In one embodiment, a face collection frame is displayed in the face preview area. The device further includes: a detection module and a prompt module; wherein:

[0044] The detection module is used to detect whether the facial key points in the facial image are located within the facial acquisition frame;

[0045] The generating module is further configured to generate an unlocking state identifier at each unlocking node based on the facial expression in the corresponding facial image in the order of the unlocking nodes in the unlocking node sequence if the facial key points in the facial image are located within the facial acquisition frame;

[0046] The prompt module is used to issue a prompt message for adjusting the acquisition orientation if the facial key points in the facial image are not located within the facial acquisition frame.

[0047] In one embodiment, the facial image is an image obtained by collecting the object to be tested. The recognition module is further used to perform face recognition on the facial image corresponding to each unlocking node according to the order of the unlocking nodes in the unlocking node sequence to obtain a face recognition result;

[0048] The unlocking module is also used to successfully unlock when an unlocking status mark is generated at each unlocking node, the facial expression in each facial image matches the corresponding target expression, and it is determined based on the face recognition result that the object to be tested is consistent with the target object.

[0049] In one embodiment, the device further comprises: a cancellation module; wherein:

[0050] The prompt module is further configured to issue a prompt message indicating unlocking failure when an unlocking state identifier is generated at each unlocking node in the unlocking node sequence, but the facial expression in each facial image does not match the corresponding target expression;

[0051] The cancel module is used to cancel the display of the unlock state identifier and return to execute the step of generating an unlock state identifier based on the facial expression in the corresponding facial image at the unlock node to be processed in the unlock node sequence.

[0052] In one embodiment, the device further includes: an acquisition module, a pause module and an alarm module; wherein:

[0053] The acquisition module is used to acquire the cumulative number of unlocking failures when unlocking fails;

[0054] The pause module is used to pause the unlocking process when the accumulated number of times reaches a preset number of times;

[0055] The alarm module is used to obtain the reserved communication signal and send alarm information to the reserved communication signal.

[0056] In one embodiment, the device further comprises: an input module and a combination module; wherein:

[0057] The second display module is used to display the expression entry page;

[0058] The input module is used to sequentially input the expression images corresponding to the expression identifiers in the expression input page through the expression input page;

[0059] The entry module is used to enter the expression combination page;

[0060] The combination module is used to combine the expression identifiers in response to a combination operation triggered on the expression combination page to obtain the expression unlocking sequence.

[0061] In one embodiment, the expression unlocking page includes an expression prompt area. The second display module is further configured to display a prompt image corresponding to the unlocking node to be processed in the expression prompt area in response to an expression prompt operation triggered in the expression prompt area during the process of unlocking the unlocking node to be processed in the unlocking node sequence.

[0062] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0063] Display the expression unlock page;

[0064] Displaying the unlocking node sequence on the expression unlocking page;

[0065] Displaying the facial image collected in real time in the facial preview area of ​​the expression unlocking page;

[0066] At an unlocking node to be processed in the unlocking node sequence, generating an unlocking state identifier based on a facial expression in the corresponding facial image;

[0067] When an unlocking state identifier is generated at each unlocking node in the unlocking node sequence, and the facial expression in each facial image matches the corresponding target expression, the unlocking is successful.

[0068] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0069] Display the expression unlock page;

[0070] Displaying the unlocking node sequence on the expression unlocking page;

[0071] Displaying the facial image collected in real time in the facial preview area of ​​the expression unlocking page;

[0072] At an unlocking node to be processed in the unlocking node sequence, generating an unlocking state identifier based on a facial expression in the corresponding facial image;

[0073] When an unlocking state identifier is generated at each unlocking node in the unlocking node sequence, and the facial expression in each facial image matches the corresponding target expression, the unlocking is successful.

[0074] The above-mentioned facial expression-based unlocking method, device, computer equipment and storage medium configure an unlocking node sequence consisting of multiple unlocking nodes on the expression unlocking page. When unlocking each unlocking node, it is necessary to perform expression recognition on the facial image collected in real time, and each unlocking node corresponds to a specific target expression. Only when the facial expressions in the facial images corresponding to all unlocking nodes match the target expressions corresponding to the corresponding unlocking nodes, the entire unlocking process is completed, thereby effectively avoiding unlocking due to stolen images or face models, and effectively improving information security. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 FIG. 1 is an application environment diagram of an unlocking method based on facial expression in one embodiment;

[0076] Figure 2 is a flowchart of a facial expression-based unlocking method in one embodiment;

[0077] Figure 3 A schematic diagram of an expression unlocking page in one embodiment;

[0078] Figure 4 A schematic diagram of entering an expression unlocking page by triggering a face detection control in one embodiment;

[0079] Figure 5 A schematic diagram showing a prompt for adjusting the acquisition orientation in an embodiment;

[0080] Figure 6 A schematic diagram of recognizing a facial image and superimposing an expression model diagram obtained by recognition on the facial expression in one embodiment;

[0081] Figure 7 is a schematic diagram of facial feature points in one embodiment;

[0082] Figure 8 A schematic diagram of a flow chart of unlocking steps by combining expression and face in one embodiment;

[0083] Fig. 9 A schematic diagram of a flow chart of unlocking steps by combining expressions and gestures in one embodiment;

[0084] Fig.10 A schematic diagram of a flow chart of steps of unlocking each unlocking node by using at least two facial expressions in one embodiment;

[0085] Fig.11 A schematic diagram of a flow chart of a step of entering an expression picture in an embodiment;

[0086] Fig.12 A schematic diagram of entering an expression entry page through a face entry control in one embodiment;

[0087] Fig.13 A schematic diagram of entering an expression image after expression recognition in one embodiment;

[0088] Fig.14 A schematic diagram of combining and sorting expression identifiers in one embodiment;

[0089] Fig.15 is a schematic diagram of expression combination and expression recognition in one embodiment;

[0090] Fig.16 is a flowchart of an unlocking method based on facial expression in another embodiment;

[0091] Fig.17 is a structural block diagram of an unlocking device based on facial expression in one embodiment;

[0092] Fig.18 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0093] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0094] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0095] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0096] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphics processing so that the computer processes the images into images that are more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0097] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0098] The solution provided in the embodiments of the present application involves artificial intelligence computer vision technology and other technologies, which are specifically described by the following embodiments:

[0099] The facial expression-based unlocking method provided in this application can be applied to Figure 1 In the application environment shown. In the application environment, a terminal 102 and a server 104 are included. The terminal 102 can collect the facial image of the object to be tested in real time through a built-in camera or an external camera, and then recognize the facial expression of the facial image. When the recognition is completed, an unlocking state mark is generated at the corresponding unlocking node to prompt the unlocking state of the unlocking node. When an unlocking state mark is generated at each unlocking node in the unlocking node sequence, and the facial expression in each facial image matches the corresponding target expression, the entire unlocking node sequence is unlocked successfully.

[0100] Among them, the object to be tested may refer to the user to be tested or other objects to be tested (such as animals). In the subsequent embodiments, the object to be tested is taken as the user to be tested as an example for explanation. The unlocking node sequence can be regarded as a security lock with a multi-digit password, and each unlocking node corresponds to a digit of password, which is decoded by facial expressions. The unlocking state includes: a state in which the unlocking operation has not been performed and a state in which the unlocking operation has been performed. The state in which the unlocking operation has been performed includes: a state in which the unlocking operation has been performed and the unlocking node has been successfully unlocked, and a state in which the unlocking operation has been performed but the unlocking node has not been successfully unlocked.

[0101] In addition, the terminal 102 can also pre-enter the expression map of the target object. Different unlocking nodes can enter different expression maps (i.e., corresponding to different target expressions), or two or three different unlocking nodes can enter the same expression map (i.e., two or three different unlocking nodes correspond to the same target expression). Each unlocking node corresponds to a password, and the unlocking node is unlocked (i.e., decoded) by the corresponding facial expression. The expression map of the target object pre-entered by the terminal 102 can be saved locally or saved in the server 104. During the unlocking process, the facial expression recognized from the facial image of the object to be tested is compared with the saved expression map by the server 104 to determine whether the facial expression of the object to be tested is consistent with the target expression of the target object.

[0102] Among them, the terminal 102 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc.; in addition, it can also be an access control device, a gate, etc., but is not limited to this.

[0103] The server 104 may be an independent physical server or a server cluster composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud servers, cloud databases, cloud storage, and content delivery networks (CDN).

[0104] The terminal 102 and the server 104 may be connected via Bluetooth, USB (Universal Serial Bus) or a network or other communication connection methods, which are not limited in this application.

[0105] In one embodiment, Figure 2 As shown, a facial expression-based unlocking method is provided, and the method is applied to Figure 1 The terminal 102 in the example is used as an example to illustrate, and the following steps are included:

[0106] S202, displaying the expression unlocking page.

[0107] The expression unlocking page may refer to a page that uses facial expressions to unlock, or a page that verifies the facial expressions of the object to be tested, such as an expression unlocking page that unlocks the terminal to enter the user operation interface, or an expression unlocking page that unlocks when entering the fund management page of an application or other pages with privacy information (such as the chat history page in a social application), or an expression unlocking page that uses facial expressions to make payments in a payment scenario.

[0108] For example, Figure 2As shown, the expression unlocking page includes a face preview area, an unlocking progress area, and an expression prompt area. Among them, the face preview area is used to display the real-time collected facial image; the unlocking progress area is used to display the unlocking node sequence. When an unlocking operation is performed on a certain unlocking node, the unlocking state mark of the unlocking operation is displayed at the position of the unlocking node. The expression prompt area can be used to display the prompt image of the corresponding unlocking node. For example, if the first unlocking node is unlocked, the expression prompt area can display the prompt image associated with the first unlocking node. For example, if the pre-recorded expression corresponding to the first unlocking node is a smiling expression, the prompt image can be a blue sky; if the pre-recorded expression corresponding to the first unlocking node is an open mouth expression, the prompt image can be a sunflower.

[0109] In one embodiment, the terminal may display the expression unlocking page when detecting an unlocking instruction, or detecting an operation instruction to enter a page with privacy information, or detecting an operation instruction to enter a fund management page, or detecting a payment instruction. In addition, when a facial recognition instruction triggered on a face management page is obtained, the expression unlocking page is entered. Wherein, face may refer to a person's face, or the face of other objects.

[0110] For example, if the subject wants to enter the terminal's operation page, the terminal will enter the expression unlocking page when receiving the unlocking instruction. Or, if the subject wants to use online payment, the terminal will enter the expression unlocking page when detecting the payment instruction. Or, Figure 4 As shown, when the subject to be tested clicks or touches the face recognition control on the face management page, the expression unlocking page is entered.

[0111] S204, displaying the unlocking node sequence on the expression unlocking page.

[0112] The unlocking node sequence may refer to a node sequence consisting of nodes that need to be unlocked (i.e., unlocking nodes). The unlocking nodes in the unlocking node sequence may be nodes with a sequence. In other words, when unlocking, each unlocking node in the unlocking node sequence is unlocked in sequence. The unlocking node sequence may correspond to a security lock, or to a string of passwords, and each unlocking node in the unlocking node sequence is unlocked by a corresponding facial expression.

[0113] In one embodiment, when the expression unlocking page is displayed, the unlocking node sequence is displayed in the unlocking progress area. In the unlocking progress area, a pointer, such as an arrow or a ">" symbol, may be displayed between each unlocking node in the unlocking node sequence. Figure 4 shown.

[0114] S206, displaying the facial image collected in real time in the facial preview area of ​​the expression unlocking page.

[0115] The face may generally refer to the face, chin, lips, eyes, nose, eyebrows, forehead and ears of the object to be tested. The face preview area may display a face acquisition frame to capture face blocks in the face image through the face acquisition frame, thereby avoiding the problem of high computational complexity caused by recognizing the entire face image.

[0116] In one embodiment, the terminal captures the object to be tested in real time through a built-in camera or an external camera connected to the terminal to obtain a facial image of the object to be tested, and then displays the real-time captured facial image in the facial preview area of ​​the expression unlocking page. The facial image may include only the face of the object to be tested, or may include the face and hands of the object to be tested.

[0117] In one embodiment, after S206, the method may further include: the terminal detects whether the facial key points in the facial image are located within the facial acquisition frame; if so, executing S208; if not, issuing a prompt message to adjust the acquisition orientation.

[0118] Among them, the facial key points may be the ears, chin and forehead of the object to be tested, etc. If the ears, chin and forehead in the facial image are within the facial acquisition frame, it can be indicated that the entire face is within the facial acquisition frame.

[0119] For example, Figure 5 As shown, when the facial key points in the detected facial image are not located in the facial acquisition frame (such as Figure 5 When the face is within the black rectangular frame in the face capture frame, a prompt message "Please adjust the capture direction" is displayed near the face capture frame so that the entire face in the face image is within the face capture frame.

[0120] S208, at the unlocking node to be processed in the unlocking node sequence, generating an unlocking state identifier based on the facial expression in the corresponding facial image.

[0121] The unlocking node to be processed may refer to a node in the unlocking node sequence that has not been unlocked. In this embodiment, the unlocking node to be processed may also refer to a node in the unlocking node sequence that has not been unlocked and currently needs to be unlocked.

[0122] The unlock status mark can be used to indicate that the unlock node to be processed has been successfully unlocked, or to indicate that the unlock node to be processed has been unlocked, but it is not certain whether it has been successfully unlocked. According to the above two meanings of the unlock status mark, S208 can be divided into the following two scenarios:

[0123] Scenario 1: The unlock status indicator indicates that the unlock node to be processed has been unlocked, but it is not certain whether the unlocking has been successfully completed.

[0124] In one embodiment, the unlocking node sequence is displayed in the unlocking progress area of ​​the expression unlocking page. S206 may specifically include: the terminal performs facial expression recognition on the facial images corresponding to the unlocking nodes to be processed in the unlocking node sequence in sequence according to the order of the unlocking nodes in the unlocking node sequence; when the facial expression recognition is completed each time, an unlocking status mark is generated at the corresponding unlocking node in the unlocking progress area. Among them, the unlocking status mark can be used to indicate that the unlocking operation has been performed on the unlocking node to be processed. The unlocking status mark includes: a state mark that the unlocking operation has been performed and the unlocking node has been successfully unlocked, or a state mark that the unlocking operation has been performed but the unlocking node has not been successfully unlocked.

[0125] For the unlocking step of a certain unlocking node to be processed in the unlocking node sequence, the steps may include: when the terminal unlocks the unlocking node to be processed in the unlocking node sequence, the terminal performs facial expression recognition on the currently collected facial image, compares the recognized facial expression with the target expression corresponding to the unlocking node to be processed, and after obtaining the comparison result, generates an unlocking state mark at the unlocking node to be processed. After the unlocking state mark is generated, the unlocking node to be processed is converted into a processed unlocking node, and then the remaining unlocking nodes to be processed are unlocked.

[0126] For example, Figure 6 As shown in the figure on the left, each unlocking node (i.e., node 1-6) in the unlocking node sequence has not been unlocked. When it is detected that the entire face in the facial image is located within the facial acquisition frame, the first unlocking node in the unlocking node sequence is first unlocked, and the unlocking process includes: performing facial expression recognition on the facial image displayed in the facial preview area, that is, performing facial expression recognition on the facial block in the facial acquisition frame, and obtaining a recognition result of the facial expression; wherein the facial expression is an expression of opening the mouth and squinting the left eye. After obtaining the recognition result of the facial expression, the recognized facial expression is compared with the target expression corresponding to the first unlocking node. When the comparison result is obtained, an unlocking state mark is generated at the position of the first unlocking node, indicating that the unlocking operation has been performed on the first unlocking node. Similarly, the other unlocking nodes in the unlocking node sequence are unlocked.

[0127] In one embodiment, when the terminal completes facial expression recognition each time, it generates an expression model graph corresponding to the facial expression; it overlays and displays the expression model graph on the corresponding facial image in the face preview area, and then compares the expression model graph with the expression graph of the corresponding unlocking node. After obtaining the comparison result, it generates an unlocking state mark at the unlocking node to be processed to indicate that the facial image corresponding to the unlocking node to be processed has been recognized. The expression model graph can refer to Figure 6 The black dots in the

[0128] Scenario 2: The unlock status indicator indicates that the unlock node to be processed has been successfully unlocked.

[0129] In one embodiment, each time the terminal completes facial expression recognition, it generates an expression model graph corresponding to the facial expression; superimposes and displays the expression model graph on the corresponding facial image in the face preview area; and when the expression model graph matches the expression of the corresponding unlock node, Figure 1 When the facial expression in the facial image matches the corresponding target expression, an unlock state identifier is generated at the unlock node to be processed in the unlock node sequence.

[0130] Among them, the expression model diagram can refer to a graphic generated based on the recognized facial expression, which can be used to represent the facial expression obtained by recognizing the object to be tested. In addition, the expression model diagram can also be used to indicate that the facial image corresponding to the unlocking node to be processed has undergone facial expression recognition.

[0131] Expression model diagram and corresponding unlocked node expressions Figure 1 When the facial expression in the facial image matches the facial expression image pre-recorded in the unlocking node to be processed, the unlocking node to be processed can be successfully unlocked.

[0132] For example, Figure 6 As shown, when the first unlocking node is unlocked, the facial expression in the facial image is recognized. After the facial key points in the facial image are recognized, an expression model image matching the facial expression of opening the mouth and squinting the left eye is generated and then superimposed on the facial expression in the facial image. Then, the expression model image is compared with the pre-recorded expression image corresponding to the first unlocking node. If they are consistent, an unlocking state mark is generated at the position of the first unlocking node.

[0133] In the process of unlocking each unlocking node, a prompt image may be used to prompt what expression corresponds to the unlocking node that currently needs to be unlocked.

[0134] In one embodiment, the expression unlocking page includes an expression prompt area; the method further includes: in the process of unlocking the unlocking node to be processed in the unlocking node sequence, in response to the expression prompt operation triggered in the expression prompt area, displaying a prompt image corresponding to the unlocking node to be processed in the expression prompt area. Thus, when the user forgets what facial expression the unlocking solution corresponds to, the prompt image can be used to associate the corresponding facial expression. It should be noted that for the target object who enters the target expression, the prompt image can be used to associate with images of joy, anger, sorrow, and happiness. For other users, the specific meaning of the prompt image may not be clear, such as using a sunflower image to associate with an open mouth.

[0135] like Figure 5As shown, when the first unlocking node is unlocked, if the subject forgets what facial expression it is, the sunflower image can be used to determine what facial expression the first unlocking node corresponds to, and then the facial expression is made to unlock the first unlocking node.

[0136] S210, when an unlocking state identifier is generated at each unlocking node in the unlocking node sequence, and the facial expression in each facial image matches the corresponding target expression, the unlocking is successful.

[0137] The target expression may refer to an expression in a pre-recorded expression map. For example, for a security lock with 6 unlocking nodes, each unlocking node may have one or more expression maps pre-recorded, and the expression in the expression map is the target expression.

[0138] When the recognized facial expression matches the expression graph corresponding to the corresponding unlocking node to be processed, it means that the facial expression matches the corresponding target expression; conversely, when the recognized facial expression matches the corresponding target expression, it means that the facial expression matches the expression graph corresponding to the corresponding unlocking node to be processed.

[0139] In addition, when the generated expression model graph matches the expression graph corresponding to the corresponding unlocking node to be processed, it means that the expression model graph matches the corresponding target expression; conversely, when the generated expression model graph matches the corresponding target expression, it means that the expression model graph matches the expression graph corresponding to the corresponding unlocking node to be processed.

[0140] In one embodiment, during the process of unlocking each unlocking node, the terminal records the time interval corresponding to each unlocking node when unlocking is completed, and then calculates the total time interval. When an unlocking state identifier is generated at each unlocking node in the unlocking node sequence, and the facial expression in each facial image matches the corresponding target expression, and the total time interval is less than the preset time interval, the unlocking is successful.

[0141] When an unlocking status mark is generated at each unlocking node in the unlocking node sequence, but the facial expressions in one or more facial images do not match the corresponding target expressions, the terminal fails to unlock. At this time, the unlocking can be performed again, that is, return to execute S204 to S210 until the cumulative number of unlocking successes or unlocking failures reaches a preset number, and then the unlocking is suspended.

[0142] In one embodiment, when an unlock status mark is generated at each unlock node in the unlock node sequence, but the facial expression in at least one facial image does not match the corresponding target expression, the terminal issues a prompt message indicating that the unlocking has failed; the display of the unlock status mark is canceled, and execution of S204 to S210 is returned.

[0143] In one embodiment, when unlocking fails, the accumulated number of unlocking failures is obtained; when the accumulated number reaches a preset number, the unlocking process is suspended; a reserved communication signal is obtained, and an alarm message is sent to the reserved communication signal.

[0144] Among them, the above-mentioned cumulative number may refer to the total number of unlocking failures in this round or the unlocking failures within a preset time period. The preset time period can be set according to the actual situation and is not specifically limited in this embodiment. The above-mentioned reserved communication signal may refer to: a communication identifier reserved by the target object (such as the user who enters the emoticon) in the application account of the terminal for receiving alarm information or emergency contact, such as a reserved mobile phone number, email account or other instant messaging account. It should be noted that the cumulative number of unlocking failures refers to the total number of consecutive unlocking failures. If one of the unlocking attempts is successful, the cumulative number of unlocking failures is set to zero.

[0145] For example, taking the reserved communication number as the reserved mobile phone number, if the cumulative number of unlocking failures within 5 minutes reaches the preset number of 5, the unlocking process is paused, and an alarm message is sent to the reserved mobile phone number to alert the target user corresponding to the reserved mobile phone number, so as to inform the target user to reset the password to avoid malicious unlocking by others, or to inform the target user to obtain the password again.

[0146] For example, face recognition on a facial image can yield the following: Figure 7 The recognition results of facial feature points shown in the figure are marked with numbers for the convenience of explanation below. For example, Figure 7 1 to 17 shown in the figure represent facial edge feature points, 18 to 22 and 23 to 27 represent the user's left eyebrow feature points and right eyebrow feature points respectively, 28 to 36 represent the user's nose feature points, 37 to 42 represent the user's left eye feature points, 43 to 48 represent the user's right eye feature points, and 49 to 68 represent the user's lip feature points. It should be pointed out that the above are only examples, and in optional embodiments, only some or more feature points can be identified from the above facial feature points, or each feature point can be marked in other ways, all of which belong to the scope of the embodiments of the present invention.

[0147] Facial feature point recognition technology is usually divided into two categories according to the different criteria it adopts and the different features it recognizes:

[0148] (1) Methods based on local features

[0149] In one embodiment, the local feature-based method can use local geometric features of the face, such as the relative positions and relative distances of some facial organs (eyes, nose, mouth, etc.) to describe the face. Its feature components usually include the Euclidean distance, curvature and angle between feature points, etc., which can achieve an efficient description of the significant features of the face.

[0150] For example, the integral projection method is used to locate facial feature points, and the Euclidean distance between feature points is used as the feature component to identify the multi-dimensional facial feature point vector for classification. The feature components mainly include: the vertical distance between the eyebrows and the center of the eyes; multiple description data of the eyebrow arc; the width of the nose and the vertical position of the nose; the position of the nostrils and the width of the face, etc. Through the recognition of the above facial feature points, a 100% correct recognition rate can be achieved in the recognition process.

[0151] In an optional embodiment of the present invention, the method based on local features can also be an empirical description of the general characteristics of facial feature points. For example, facial images have some obvious basic features, such as the facial area usually includes facial feature points such as eyes, nose and lips, and their brightness is generally lower than that of the surrounding area; the eyes are roughly symmetrical, and the nose and mouth are distributed on the symmetry axis, etc.

[0152] (2) Holistic approach

[0153] Here, the holistic method treats the facial image as a whole and performs some transformation processing on it to identify features. This method takes into account the overall attributes of the face and also retains the topological relationship between facial parts and the information of each part itself.

[0154] Since the dimensionality of facial images is usually very high and the distribution of facial images in high-dimensional space is not compact, it is not conducive to classification and the computational complexity is also very large. Subspace analysis can be used to find a linear or nonlinear space transformation according to certain goals, compressing the original high-dimensional data into a low-dimensional subspace, making the distribution of data in this subspace more compact and reducing the computational complexity.

[0155] In addition, a set of rectangular grid nodes can be placed on the facial image, and the characteristics of each node are described by the multi-scale wavelet features at the node, and the connection relationship between the nodes is represented by the geometric distance, thus forming a facial representation based on a two-dimensional topological graph. In the face recognition process, recognition is performed based on the similarity between the nodes and connections in the two images.

[0156] In addition to the above-mentioned subspace analysis method and elastic graph matching method, the overall method also includes a method based on a neural network, etc. In the embodiment of the present invention, there is no restriction on the type of overall method.

[0157] In the above embodiment, an unlocking node sequence consisting of multiple unlocking nodes is configured on the expression unlocking page. When unlocking each unlocking node, it is necessary to perform expression recognition on the facial image collected in real time, and each unlocking node corresponds to a specific target expression. Only when the facial expressions in the facial images corresponding to all unlocking nodes match the target expressions corresponding to the corresponding unlocking nodes, the entire unlocking process is completed, thereby effectively avoiding unlocking due to stolen images or face models, and effectively improving information security.

[0158] In one embodiment, in addition to unlocking by facial expression, unlocking can also be performed by combining facial expression and face recognition. Figure 8 As shown, the method may also include:

[0159] S802, performing facial expression recognition on facial images corresponding to the unlocking nodes to be processed in the unlocking node sequence in sequence according to the order of the unlocking nodes in the unlocking node sequence.

[0160] Facial expressions can be expressed by different parts of the face performing corresponding actions or being in corresponding postures, such as Figure 5 and Figure 6 The expression shown is with the mouth open and the left eye squinted.

[0161] In one embodiment, the step of facial expression recognition includes: the terminal extracts eye feature points from the facial image; among the eye feature points, determines a first distance between an upper eyelid feature point and a lower eyelid feature point, and determines a second distance between a left eye corner feature point and a right eye corner feature point; and determines the eye posture based on a relationship between a ratio between the first distance and the second distance and at least one preset interval.

[0162] Among them, the left eye corner feature point and the right eye corner feature point refer to the left eye corner feature point and the right eye corner feature point of the same eye respectively. For example, for the left eye, the left eye corner feature point refers to the left eye corner feature point of the left eye, and the right eye corner feature point refers to the right eye corner feature point of the left eye.

[0163] For example, if Figure 7 As shown, the terminal calculates the first distance between the upper eyelid feature point 38 and the lower eyelid feature point 42, and calculates the second distance between the left eye corner feature point 37 and the right eye corner feature point 40 according to the facial feature points acquired by the face recognition technology. When the ratio between the first distance and the second distance is 0, it is determined that the subject to be tested narrows the left eye. When the ratio between the first distance and the second distance is less than 0.2, it is determined that the subject to be tested blinks left. When the ratio between the first distance and the second distance is greater than 0.2 and less than 0.6, it is determined that the subject to be tested stares with the left eye.

[0164] For lip gestures, the following methods can be used for recognition:

[0165] Method 1: Recognize lip gestures based on the height of lip feature points.

[0166] In one embodiment, the step of facial expression recognition also includes: the terminal extracts lip feature points from the facial image; among the lip feature points, the lip posture is determined according to the height difference between the lip center feature point and the lip corner feature point.

[0167] For example, Figure 7 As shown, the terminal determines the height of the lip center feature point 63 and the height of the lip corner feature point 49 (or 55), and then calculates the height difference between the lip center feature point 63 and the lip corner feature point 49 (or 55). If the height difference is positive (that is, the height of the lip center feature point 63 is higher than the height of the lip corner feature point 49), the lip posture is judged to be a smile.

[0168] Method 2: Recognize lip gestures based on the distance between the feature points of the upper and lower lips.

[0169] In one embodiment, among the lip feature points, the terminal determines the lip gesture according to a third distance between the upper lip feature point and the lower lip feature point.

[0170] For example, based on the facial feature points obtained by face recognition technology, the third distance between the upper lip feature point and the lower lip feature point is compared with the distance threshold, and when the third distance reaches the distance threshold, it is determined that the lip posture is open. Figure 7 As shown, the third distance between the upper lip feature point 63 and the lower lip feature point 67 is compared with the distance threshold. If it is greater than or equal to the distance threshold, it is identified that the subject to be tested is opening its mouth.

[0171] In another embodiment, the terminal calculates a fourth distance between a left lip corner feature point and a right lip corner feature point, and when the third distance is greater than or equal to the fourth distance, determines that the lip posture is open.

[0172] Method 3: Recognize lip gestures based on the ratio of the distance between the upper and lower lips to the distance between the left and right lip corner feature points.

[0173] In one embodiment, among the lip feature points, the lip posture is determined based on the relationship between the ratio of the third distance to the fourth distance and at least one preset interval; wherein the fourth distance is the distance between the left lip corner feature point and the right lip corner feature point.

[0174] For example, Figure 7As shown, the terminal determines the third distance between the upper lip feature point 63 and the lower lip feature point 67, and determines the fourth distance between the left lip corner feature point 49 and the right lip corner feature point 55, and the lip posture is determined by the relationship between the ratio between the third distance and the fourth distance and at least one preset interval. For example, when the ratio between the third distance and the fourth distance is in the first preset interval, the lip posture is determined to be open mouth; in addition, when the ratio between the third distance and the fourth distance is in the second preset interval, the lip posture is determined to be closed mouth. Among them, the values ​​of the first preset interval are all greater than the values ​​of the second preset interval.

[0175] In one embodiment, the step of facial expression recognition also includes: extracting eyebrow feature points and eyelid feature points from the facial image; determining a fifth distance between the eyebrow feature points and the eyelid feature points; and determining the eyebrow posture based on the size relationship between the fifth distance and the preset distance.

[0176] For example, Figure 7 As shown, the distance between the eyebrow feature point and the eyelid feature point 38 is calculated, and when the distance is greater than a preset distance, it is determined that the subject to be measured is raising his eyebrows.

[0177] S804, performing face recognition on the facial image corresponding to each unlocking node according to the order of the unlocking nodes in the unlocking node sequence to obtain a face recognition result.

[0178] The process of face recognition may refer to S210 of the above embodiment.

[0179] S806, when an unlocking status mark is generated at each unlocking node, and the facial expression in each facial image matches the corresponding target expression, and it is determined according to the face recognition result that the object to be tested is consistent with the target object, the unlocking is successful.

[0180] When using facial expressions for unlocking, face recognition can also be performed to determine whether the face matches the face of the pre-recorded target object. If they match, it means that the object to be tested and the target object are the same object. The facial image is an image obtained by capturing the object to be tested.

[0181] When an unlocking status mark is generated at each unlocking node, and the facial expression in each facial image matches the corresponding target expression, and the object to be tested and the target object are determined to be inconsistent according to the face recognition result, the unlocking fails. When the object to be tested and the target object are determined to be inconsistent according to the face recognition result, it means that the currently detected object to be tested and the target object are not the same object, and the unlocking fails at this time.

[0182] In the above embodiment, facial expressions are combined with human faces for unlocking, which can further improve the security of the unlocking node sequence and improve information security.

[0183] In one embodiment, in addition to unlocking by facial expression, facial expression and gesture can also be combined to unlock, such as Fig. 9 As shown, the method may also include:

[0184] S902, performing facial expression recognition on facial images corresponding to the unlocking nodes to be processed in the unlocking node sequence in sequence according to the order of the unlocking nodes in the unlocking node sequence.

[0185] Facial expressions can be expressed by different parts of the face performing corresponding actions or being in corresponding postures, such as Figure 5 and Figure 6 The expression shown is with the mouth open and the left eye narrowed.

[0186] The recognition of facial expressions may refer to S802 of the above embodiment.

[0187] S904: During the facial expression recognition process, gesture recognition is performed on the hands in the facial image.

[0188] In one embodiment, the terminal performs convolution processing on the facial image through a neural network model, thereby extracting gesture features in the facial image and determining a specific gesture based on the gesture features.

[0189] The neural network model may be a network model for extracting gesture features, specifically a two-dimensional convolutional neural network model. The two-dimensional network model may be one of the network branches of the machine learning model.

[0190] S906, when facial expression recognition and gesture recognition are completed each time, an unlocking state mark is generated at the corresponding unlocking node in the unlocking progress area.

[0191] In one embodiment, the terminal can also determine whether the facial expression is consistent with the corresponding target expression after each facial expression recognition is completed; in addition, the terminal can also determine whether the gesture is consistent with the corresponding target gesture after each gesture recognition is completed. If the facial expression is consistent with the corresponding target expression and the gesture is consistent with the corresponding target gesture, an unlock status mark is generated at the corresponding unlock node in the unlock progress area.

[0192] S908, when an unlocking state identifier is generated at each unlocking node, and the facial expression in each facial image matches the corresponding target expression, and each recognized gesture is consistent with the corresponding target gesture, the unlocking is successful.

[0193] In the above embodiment, unlocking by combining facial expressions with hand gestures can further improve the security of the unlocking node sequence and enhance information security.

[0194] In one embodiment, each unlocking node can be unlocked by at least two facial expressions, and each unlocking node corresponds to at least two different target expressions; Fig.10 As shown, S208 may specifically include:

[0195] S1002, performing facial expression recognition on at least two facial images corresponding to the unlocked node to be processed.

[0196] When unlocking the unlocking node to be processed, facial expression recognition may be performed on at least two facial images corresponding to the unlocking node to be processed.

[0197] For example, when unlocking node 1, facial image 1 is first collected, and then facial expression recognition is performed on facial image 1 to obtain facial expression 1; then facial expression recognition is performed on the collected facial image 1 to obtain facial expression 2.

[0198] The process of facial expression recognition may refer to S802 in the above embodiment.

[0199] S1004: When the facial expressions in at least two facial images match the corresponding target expressions, an unlock state identifier is generated at the unlock node to be processed.

[0200] The facial expressions in at least two facial images may be the same or different.

[0201] In one embodiment, when the facial expressions in at least two facial images match the corresponding target expressions, the terminal determines the acquisition time interval between the at least two facial images; or determines the unlocking time interval between the unlocking node to be processed and the last processed unlocking node; when the acquisition time interval or the unlocking time interval meets the corresponding time interval condition, S1004 is executed. This can avoid wasting too much time when unlocking the same unlocking node or unlocking different unlocking nodes, improve the unlocking efficiency, and also prevent others from trying to unlock a certain unlocking node by multiple attempts with different facial expressions.

[0202] S1006, when an unlocking state identifier is generated at each unlocking node and the facial expression in each facial image matches the corresponding target expression, the unlocking is successful.

[0203] In the above embodiment, each unlocking node is unlocked using at least two facial expressions, which can further improve the security of the unlocking node sequence and improve information security.

[0204] In one embodiment, a specific expression image may be pre-recorded for each unlocking node, and the step of recording the expression image may include:

[0205] S1102, displaying the expression entry page.

[0206] The expression entry page may include an expression identifier or may not include an expression identifier. The expression identifier is used to indicate a corresponding expression type, and different expression identifiers correspond to different expression types.

[0207] In one embodiment, when the terminal obtains the expression entry instruction triggered on the expression management page, it switches the page to the expression entry page containing the expression identifier. Fig.12 As shown, when a click or touch operation is detected on the face entry control on the expression management page, the expression entry page containing the expression identifier is displayed. The upper part of the expression entry page is used to display the collected expression pictures, and the lower part is used to display the expression identifier.

[0208] S1104, sequentially inputting expression images corresponding to the expression identifiers in the expression input page through the expression input page.

[0209] In one embodiment, when the expression entry page contains expression identifiers, the terminal collects expression images of the target object in sequence according to the arrangement order of the expression identifiers, and then enters the collected expression images in sequence. Fig.12 As shown, in the expression entry page, the expression image is entered for the first expression identifier first, and then the expressions are entered for the subsequent expression identifiers in sequence.

[0210] In one embodiment, when the expression entry page does not contain an expression identifier, the terminal recognizes the collected expression image to obtain a target expression; and displays a corresponding expression identifier in the expression identifier preview area of ​​the expression entry page according to the target expression. Fig.13 As shown, the open-mouth facial image is recognized to obtain the open-mouth facial expression, and then an expression identifier is generated according to the open-mouth facial expression, and is displayed in the expression identifier preview area of ​​the expression entry page.

[0211] S1106, enter the expression combination page.

[0212] The expression combination page may be a page for combining expression identifiers.

[0213] S1108, in response to a combination operation triggered on the expression combination page, the expression identifiers are combined to obtain an expression unlocking sequence.

[0214] In one embodiment, in the expression combination page, the terminal can choose to sort the expression identifiers in the expression entry page, and construct an expression unlocking sequence based on the sorted expression identifiers; or, the terminal first combines the expression identifiers in the expression entry page, and then sorts them, and constructs an expression unlocking sequence based on the combined and sorted expression identifiers.

[0215] For example, Fig.14 As shown in , the target object can combine the expression identifiers in the expression combination page in pairs to obtain the corresponding expression combination. Fig.14 As shown in the expression combination page on the right, in the first row, the normal expression is combined with the left-glaring expression.

[0216] In the above embodiment, the corresponding expression images are pre-recorded according to the expression identifiers, and then the expression identifiers are combined and sorted to generate an expression unlocking sequence, so that when unlocking, the expression unlocking node is unlocked according to the corresponding facial expression. Moreover, when unlocking each unlocking node, it is necessary to compare the currently recognized facial expression with the pre-recorded corresponding expression image. Only when the two match can the unlocking node be unlocked, thereby effectively improving the security of the unlocking node.

[0217] As an example, let’s take a face image as an example. Fig.15 and Fig.16 As shown, in this embodiment, the function of the smart password lock is realized mainly by face recognition + combined expression + expression sequence + prompt image in the form of prompt, which can well realize the privacy of the user. Combining expressions can increase the difficulty of expression recognition, using expression sequence can increase the complexity of unlocking, and using prompt image can help users remember the password combination and avoid password leakage. Among them, the expression sequence corresponds to the above-mentioned unlocking node sequence.

[0218] In this embodiment, users can customize the combination of different expressions, set expression sequences, add prompt images and pictures (or animations) to create a combination of password locks that can meet the user's encryption and decryption needs and increase personal privacy protection. The specific implementation steps for the definition and decryption of the password lock include:

[0219] S1, recognize facial feature points through expression recognition to obtain corresponding facial expressions.

[0220] S2, redefine and implement the logic of expression triggering.

[0221] 1) Blinking: Based on the facial feature points obtained by face recognition technology, the distance between the central feature points of the upper and lower eyelids is divided by the distance between the feature points of the left and right eye corners to obtain the ratio between the two. When the ratio between the two is less than the set distance threshold, it is judged as blinking. Figure 7 As shown, the distance between the center feature point 38 of the left upper eyelid and the center feature point 42 of the left lower eyelid is divided by the distance between the left and right eye corner feature points 37 and 40. If the ratio is less than 0.2, it is determined that the user blinks the left eye.

[0222] 2) Staring: Based on the facial feature points obtained by face recognition technology, the distance between the central feature points of the upper and lower eyelids is divided by the distance between the feature points of the left and right eye corners to obtain the ratio between the two. When the ratio between the two is greater than the set ratio threshold, it is judged as staring. Figure 7 , the distance between the central feature point 38 of the left upper eyelid and the central feature point 42 of the left lower eyelid is divided by the distance between the left and right eye corner feature points 37 and 40. If the ratio is less than 0.6, it is judged as blinking the left eye.

[0223] 3) Pouting: Based on the facial feature points obtained by face recognition technology, a pouting ratio threshold is set. The distance between the upper and lower feature points of the lips is divided by the distance between the left and right feature points of the lips. When the quotient reaches the set pouting ratio threshold, it is judged as pouting. Figure 7 , divide the distance between the upper and lower center feature points 67 and 63 of the lips by the distance between the left and right feature points 49 and 55 of the lips. If the quotient is greater than 1.0, it is judged as pouting.

[0224] 4) Smile: Based on the facial feature points obtained by face recognition technology, when the center feature point of the lips is lower than the height of the left and right feature points of the lips, it is judged as a smile. Figure 7 As shown, when the position of the central feature point 63 of the lips is lower than the positions of the left and right feature points 49 and 55 of the lips, it is judged as a smile.

[0225] 5) Angry: According to the facial feature points obtained by face recognition technology, when the center feature point of the lips is higher than the height of the left and right feature points of the lips, it is judged as angry. As shown in the figure, when the position of the center feature point 63 of the lips is higher than the positions of the left and right feature points 49 and 55 of the lips, it is judged as angry.

[0226] 6) Open mouth: Based on the facial feature points obtained by face recognition technology, the unit distance between the upper and lower lips is calculated and weighted averaged, and a distance threshold for opening the mouth is set. When the distance between the feature points of the upper and lower lips is greater than the distance threshold, it is judged as opening the mouth. Figure 7 As shown, when the distance between the upper lip center feature point 63 and the lower lip terminal feature point 67 is greater than 0.2 times the distance between the left and right feature points 49 and 55 of the lips, it is judged that the mouth is open.

[0227] 7) Raising eyebrows: according to the facial feature points obtained by face recognition technology, a distance threshold for raising eyebrows is set. When the distance between the eyebrow feature point and the upper eyelid feature point is greater than the distance threshold, it is judged as raising eyebrows.

[0228] 8) Nodding: Based on the facial feature points obtained by face recognition technology, the angle of up and down rotation of the head in the previous frame of the face image and the angle of up and down rotation of the head in the current frame of the face image are calculated, and the angle difference between the two is calculated. When the angle difference exceeds 10 degrees, it is judged as a nod.

[0229] S3, combine expressions

[0230] Users can combine the above expressions in pairs as needed, such as blinking the left eye + opening the mouth to form an expression combination, and blinking the right eye + pouting the mouth to form an expression combination.

[0231] S4, sorting different expression combinations

[0232] Users can arrange the set expression combinations in a certain order and build a password group based on the sorted expression combinations, such as: Expression combination 1 (wink + nod): Expression combination 2 (wink left eye + open mouth): Expression combination 3 (wink + pout): Expression combination 4 (smile): Expression combination 5 (raise eyebrow): Expression combination 6 (stare).

[0233] S5, set prompt image and animation

[0234] For each expression combination, you can set a hint image or animation that can help with memory without revealing the expression. For example, you can set a blue sky to remember a smile, set a rainy day to remember anger, set a palm to remember a nod, etc. This association scheme is set according to the user's own preferences. Try to avoid too many obvious hints to prevent leaking the password combination.

[0235] S6, set over-limit protector

[0236] Set the verification time for each expression combination, such as 5s, and set the total number of recognition errors. If the recognition error operation is performed five times, the password lock will be locked and cannot be opened for a certain period of time, and an early warning message will be sent to a fixed email address and mobile phone number.

[0237] S7, unlock

[0238] First, face recognition is performed, and then expression recognition is performed. When a certain unlocking node does not know the corresponding expression, a prompt image is displayed. After a single facial expression is recognized, the expression combination is then recognized to determine whether the expression combination corresponding to the unlocking node is correct. After each expression combination is correctly recognized, the entire expression sequence is recognized to determine whether the order of each expression combination is correct. Finally, the cumulative number of timeouts or failures is judged. If the cumulative number of timeouts or failures reaches 5 times, the unlocking is suspended, and a warning message is sent to a fixed email address and mobile phone number.

[0239] The solution of this embodiment can help users flexibly set password combinations for access control, file encryption, entering smart terminal operation pages and payment needs. It has low technical difficulty, strong privacy, and is easy to operate and remember.

[0240] It should be understood that although Figure 2 , 8 The steps in the flowchart of -11 are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 , 8 -At least part of the steps in 11 may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0241] In one embodiment, Fig.17 As shown, a facial expression-based unlocking device is provided, which can be a software module or a hardware module, or a combination of the two to become a part of a computer device. The device specifically includes: a display module 1702, a first display module 1704, a second display module 1706, a generation module 1708 and an unlocking module 1710, wherein:

[0242] Display module 1702, used to display the expression unlocking page;

[0243] The first display module 1704 is used to display the unlocking node sequence on the expression unlocking page;

[0244] The second display module 1706 is used to display the real-time collected facial image in the facial preview area of ​​the expression unlocking page;

[0245] A generating module 1708, configured to generate an unlocking state identifier based on a facial expression in a corresponding facial image at an unlocking node to be processed in the unlocking node sequence;

[0246] The unlocking module 1710 is used to unlock successfully when an unlocking state identifier is generated at each unlocking node in the unlocking node sequence and the facial expression in each facial image matches the corresponding target expression.

[0247] In the above embodiment, an unlocking node sequence consisting of multiple unlocking nodes is configured on the expression unlocking page. When unlocking each unlocking node, it is necessary to perform expression recognition on the facial image collected in real time, and each unlocking node corresponds to a specific target expression. Only when the facial expressions in the facial images corresponding to all unlocking nodes match the target expressions corresponding to the corresponding unlocking nodes, the entire unlocking process is completed, thereby effectively avoiding unlocking due to stolen images or face models, and effectively improving information security.

[0248] In one embodiment, the unlocking node sequence is displayed in the unlocking progress area of ​​the expression unlocking page. The generating module is further used to perform facial expression recognition on the facial images corresponding to the unlocking nodes to be processed in the unlocking node sequence in sequence according to the order of the unlocking nodes in the unlocking node sequence; when the facial expression recognition is completed each time, an unlocking state mark is generated at the corresponding unlocking node in the unlocking progress area.

[0249] In one embodiment, the device further includes: a generating module, a superposition module and a determining module; wherein:

[0250] A generation module, used for generating an expression model diagram corresponding to the facial expression each time the facial expression recognition is completed;

[0251] An overlay module, used for overlaying and displaying the expression model diagram on the corresponding facial image in the facial preview area;

[0252] Determine module, used to determine the expression model graph and the expression of the corresponding unlock node Figure 1 When the facial expression in the facial image matches the corresponding target expression, it is determined that the facial expression in the facial image matches the corresponding target expression.

[0253] In one embodiment, the facial image includes a face and a hand. The device further includes: a recognition module; wherein:

[0254] A recognition module, used for performing gesture recognition on the hands in the facial image during the process of facial expression recognition;

[0255] The generating module is also used to generate an unlocking state mark at the corresponding unlocking node in the unlocking progress area each time the facial expression recognition and gesture recognition are completed.

[0256] In the above embodiment, unlocking by combining facial expressions with hand gestures can further improve the security of the unlocking node sequence and enhance information security.

[0257] In one embodiment, each unlocking node corresponds to at least two different target expressions. The generating module is further configured to perform facial expression recognition on at least two facial images corresponding to the unlocking node to be processed; when the facial expressions in the at least two facial images match the corresponding target expressions, an unlocking state mark is generated at the unlocking node to be processed.

[0258] In one embodiment, the determination module is further used to determine the acquisition time interval between the at least two facial images when the facial expressions in the at least two facial images match the corresponding target expressions; or determine the unlocking time interval between the unlocking node to be processed and the last processed unlocking node;

[0259] The generating module is further used to generate an unlocking state mark at the unlocking node to be processed when the collection time interval or the unlocking time interval meets the corresponding time interval condition.

[0260] In the above embodiment, each unlocking node is unlocked using at least two facial expressions, which can further improve the security of the unlocking node sequence and improve information security.

[0261] In one embodiment, the recognition module is also used to extract eye feature points from the facial image; among the eye feature points, determine the first distance between the upper eyelid feature point and the lower eyelid feature point, and determine the second distance between the left eye corner feature point and the right eye corner feature point; determine the eye posture based on the relationship between the ratio between the first distance and the second distance and at least one preset interval.

[0262] In one embodiment, the recognition module is also used to extract lip feature points from the facial image; among the lip feature points, the lip posture is determined according to the height difference between the lip center feature point and the lip corner feature point; or, among the lip feature points, the lip posture is determined according to the third distance between the upper lip feature point and the lower lip feature point; or, among the lip feature points, the lip posture is determined according to the relationship between the ratio of the third distance to the fourth distance and at least one preset interval; wherein the fourth distance is the distance between the left lip corner feature point and the right lip corner feature point.

[0263] In one embodiment, the recognition module is also used to extract eyebrow feature points and eyelid feature points from the facial image; determine the fifth distance between the eyebrow feature points and the eyelid feature points; and determine the eyebrow posture based on the size relationship between the fifth distance and the preset distance.

[0264] In one embodiment, a face collection frame is displayed in the face preview area. The device also includes: a detection module and a prompt module; wherein:

[0265] A detection module, used to detect whether the facial key points in the facial image are located within the facial acquisition frame;

[0266] The generating module is further configured to generate an unlocking state identifier based on the facial expression in the corresponding facial image at each unlocking node in the unlocking node sequence according to the order of the unlocking nodes in the unlocking node sequence if the facial key points in the facial image are located within the facial acquisition frame;

[0267] The prompt module is used to issue a prompt message for adjusting the acquisition direction if the facial key points in the facial image are not located in the facial acquisition frame.

[0268] In one embodiment, the facial image is an image obtained by collecting the object to be tested. The recognition module is further used to perform face recognition on the facial image corresponding to each unlocking node according to the order of the unlocking nodes in the unlocking node sequence to obtain a face recognition result;

[0269] The unlocking module is also used to successfully unlock when an unlocking state mark is generated at each unlocking node, the facial expression in each facial image matches the corresponding target expression, and it is determined based on the face recognition result that the object to be tested is consistent with the target object.

[0270] In the above embodiment, facial expressions are combined with human faces for unlocking, which can further improve the security of the unlocking node sequence and improve information security.

[0271] In one embodiment, the device further comprises: a cancellation module; wherein:

[0272] The prompt module is further used to issue a prompt message indicating unlocking failure when an unlocking state identifier is generated at each unlocking node in the unlocking node sequence, but the facial expression in each facial image does not match the corresponding target expression;

[0273] The cancel module is used to cancel the display of the unlock state mark and return to the step of generating the unlock state mark based on the facial expression in the corresponding facial image at the unlock node to be processed in the unlock node sequence.

[0274] In one embodiment, the device further includes: an acquisition module, a pause module and an alarm module; wherein:

[0275] The acquisition module is used to obtain the cumulative number of unlocking failures when unlocking fails;

[0276] A pause module is used to pause the unlocking process when the cumulative number of times reaches a preset number;

[0277] The alarm module is used to obtain the reserved communication signal and send alarm information to the reserved communication signal.

[0278] In one embodiment, the device further comprises: an input module and a combination module; wherein:

[0279] The second display module is used to display the expression input page;

[0280] An input module, used to input expression images corresponding to expression identifiers in the expression input page in sequence through the expression input page;

[0281] Entry module, used to enter the expression combination page;

[0282] The combination module is used to respond to the combination operation triggered on the expression combination page, combine the expression identifiers, and obtain the expression unlocking sequence.

[0283] In one embodiment, the expression unlocking page includes an expression prompt area. The second display module is further used to display a prompt image corresponding to the unlocking node to be processed in the expression prompt area in response to an expression prompt operation triggered in the expression prompt area during the process of unlocking the unlocking node to be processed in the unlocking node sequence.

[0284] In the above embodiment, the corresponding expression images are pre-recorded according to the expression identifiers, and then the expression identifiers are combined and sorted to generate an expression unlocking sequence, so that when unlocking, the expression unlocking node is unlocked according to the corresponding facial expression. Moreover, when unlocking each unlocking node, it is necessary to compare the currently recognized facial expression with the pre-recorded corresponding expression image. Only when the two match can the unlocking node be unlocked, thereby effectively improving the security of the unlocking node.

[0285] For the specific definition of the unlocking device based on facial expression, please refer to the definition of the unlocking method based on facial expression above, which will not be repeated here. Each module in the above-mentioned unlocking device based on facial expression can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0286] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.18 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a facial expression-based unlocking method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0287] Those skilled in the art will understand that Fig.18 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0288] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0289] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0290] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.

[0291] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0292] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0293] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A facial expression-based unlocking method, characterized in that: The method comprises: Displaying an expression unlocking page, the expression unlocking page comprising a face preview area for displaying a real-time collected face image, an unlocking progress area for displaying an unlocking node sequence, and an expression prompt area for displaying a prompt image corresponding to each unlocking node; A plurality of unlocking nodes are displayed in the unlocking progress area of ​​the expression unlocking page to form an unlocking node sequence, wherein adjacent unlocking nodes in the unlocking node sequence have unlocking direction guidance marks between them, and there are unlocking nodes in the unlocking node sequence that have the same expression image; A facial image captured in real time is displayed in a facial preview area of ​​the expression unlocking page; in the process of unlocking the unlocking node to be processed in the unlocking node sequence, in response to an expression prompt operation triggered in the expression prompt area, a prompt image corresponding to the unlocking node to be processed is displayed in the expression prompt area of ​​the expression unlocking page, wherein the prompt image and the expression image corresponding to the unlocking node to be processed are different image contents; When the facial expression recognition of the facial image is completed, the expression model diagram corresponding to the facial expression is superimposed and displayed on the facial image displayed in the facial preview area; When the expression model graph is consistent with the expression graph of the unlocking node to be processed, determining that the facial expression in the facial image matches the corresponding target expression, and generating an unlocking state identifier at the unlocking node to be processed in the unlocking node sequence, wherein the unlocking state identifier is used to indicate that the unlocking node to be processed has been successfully unlocked; When an unlocking state identifier is generated at each unlocking node in the unlocking node sequence, it is determined that the unlocking is successful.

2. The method according to claim 1, characterized in that Each unlocking node in the unlocking node sequence corresponds to at least two different target expressions; the method further comprises: Facial expression recognition is performed on at least two facial images corresponding to the unlocking node to be processed, and when the facial expressions in the at least two facial images match the corresponding target expressions, an unlocking state mark is generated at the unlocking node to be processed.

3. The method according to claim 1, characterized in that The facial image includes a face and a hand; the method further includes: During the facial expression recognition, performing gesture recognition on the hands in the facial image; The generating an unlocking state identifier at the unlocking node to be processed in the unlocking node sequence comprises: When the facial expression matches the corresponding target expression and the recognized gesture is consistent with the corresponding target gesture, an unlocking state mark is generated at the corresponding unlocking node in the unlocking progress area.

4. The method according to claim 2 or 3, characterized in that: The steps of facial expression recognition include: Extracting eye feature points from the facial image; Among the eye feature points, determining a first distance between an upper eyelid feature point and a lower eyelid feature point, and determining a second distance between a left eye corner feature point and a right eye corner feature point; The eye gesture is determined according to a relationship between a ratio of the first distance to the second distance and at least one preset interval.

5. The method according to claim 4, characterized in that The step of facial expression recognition also includes: Extracting lip feature points from the facial image; Among the lip feature points, the lip posture is determined according to the height difference between the lip center feature point and the lip corner feature point; or, Among the lip feature points, the lip posture is determined according to a third distance between the upper lip feature point and the lower lip feature point; or, Among the lip feature points, the lip posture is determined based on the relationship between the ratio between the third distance and the fourth distance and at least one preset interval; wherein the third distance is the distance between the upper lip feature point and the lower lip feature point, and the fourth distance is the distance between the left lip corner feature point and the right lip corner feature point.

6. The method according to claim 1, characterized in that A face collection frame is displayed in the face preview area; after the face image collected in real time is displayed in the face preview area of ​​the expression unlocking page, the method further includes: Detecting whether facial key points in the facial image are located within the facial acquisition frame; If yes, unlock each unlocked node according to the order of unlocked nodes in the unlocked node sequence; If not, a prompt message is issued to adjust the collection direction.

7. The method according to claim 1, characterized in that The facial image is an image obtained by collecting the object to be tested; the method also includes: According to the order of the unlocking nodes in the unlocking node sequence, face recognition is performed on the facial image corresponding to each unlocking node to obtain a face recognition result; When an unlocking status mark is generated at each unlocking node, determining that the unlocking is successful includes: When an unlocking status identifier is generated at each unlocking node and it is determined according to the face recognition result that the object to be detected is consistent with the target object, it is determined that the unlocking is successful.

8. The method according to any one of claims 1 to 3, 6 to 7, characterized in that: The method further comprises: Display the expression entry page; Entering expression images corresponding to the expression identifiers in the expression entry page in sequence through the expression entry page; Enter the emoticon combination page; In response to a combination operation triggered on the expression combination page, the expression identifiers are combined to obtain an expression unlocking sequence.

9. The method according to claim 2, characterized in that: The method further comprises: When the facial expressions in the at least two facial images match the corresponding target expressions, determining the acquisition time interval between the at least two facial images; or, Determine an unlocking time interval between the unlocking node to be processed and the last unlocking node processed; When the acquisition time interval or the unlocking time interval satisfies the corresponding time interval condition, the step of generating an unlocking state identifier at the unlocking node to be processed when the facial expressions in the at least two facial images match the corresponding target expressions is performed.

10. The method according to any one of claims 1 to 3, 6 to 7, characterized in that: The step of facial expression recognition also includes: Extracting eyebrow feature points and eyelid feature points from the facial image; determining a fifth distance between the eyebrow feature point and the eyelid feature point; The eyebrow posture is determined according to the magnitude relationship between the fifth distance and the preset distance.

11. An unlocking device based on facial expression, characterized in that: The device comprises: A display module, used to display an expression unlocking page, wherein the expression unlocking page includes a face preview area for displaying a real-time collected face image, an unlocking progress area for displaying an unlocking node sequence, and an expression prompt area for displaying a prompt image corresponding to each unlocking node; A first display module is used to display a plurality of unlocking nodes in the unlocking progress area of ​​the expression unlocking page to form an unlocking node sequence, wherein adjacent unlocking nodes in the unlocking node sequence have unlocking direction guidance marks between them, and there are unlocking nodes with the same expression image in the unlocking node sequence; A second display module is configured to display a facial image collected in real time in a facial preview area of ​​the expression unlocking page; in a process of unlocking an unlocking node to be processed in the unlocking node sequence, in response to an expression prompt operation triggered in the expression prompt area, display a prompt image corresponding to the unlocking node to be processed in the expression prompt area of ​​the expression unlocking page, wherein the prompt image and the expression image corresponding to the unlocking node to be processed are different image contents; A superimposition module, for superimposing and displaying an expression model diagram corresponding to the facial expression on the facial image displayed in the facial preview area when the facial expression recognition of the facial image is completed; a determination module, configured to determine that the facial expression in the facial image matches the corresponding target expression when the expression model graph is consistent with the expression graph of the unlocking node to be processed in the unlocking node sequence; A generating module, configured to generate an unlocking status mark at an unlocking node to be processed in the unlocking node sequence, wherein the unlocking status mark is used to indicate that the unlocking node to be processed has been successfully unlocked; The unlocking module is used to determine that the unlocking is successful when an unlocking state mark is generated at each unlocking node in the unlocking node sequence.

12. The device according to claim 11, characterized in that Each unlocking node in the unlocking node sequence corresponds to at least two different target expressions; The generating module is further used to perform facial expression recognition on at least two facial images corresponding to the unlocking node to be processed, and when the facial expressions in the at least two facial images match the corresponding target expressions, an unlocking state mark is generated at the unlocking node to be processed.

13. The device according to claim 11, characterized in that The facial image includes a face and a hand; The generating module is also used to perform gesture recognition on the hands in the facial image during the process of facial expression recognition; when the facial expression matches the corresponding target expression and the recognized gesture is consistent with the corresponding target gesture, an unlocking status mark is generated at the corresponding unlocking node in the unlocking progress area.

14. The device according to claim 12 or 13, characterized in that The device also includes: A recognition module is used to extract eye feature points from the facial image; among the eye feature points, determine a first distance between an upper eyelid feature point and a lower eyelid feature point, and determine a second distance between a left eye corner feature point and a right eye corner feature point; and determine an eye posture based on a relationship between a ratio between the first distance and the second distance and at least one preset interval.

15. The device according to claim 14, characterized in that The recognition module is further used to extract lip feature points from the facial image; among the lip feature points, determine the lip posture according to the height difference between the lip center feature point and the lip corner feature point; or, among the lip feature points, determine the lip posture according to the third distance between the upper lip feature point and the lower lip feature point; Alternatively, in the lip feature points, the lip posture is determined based on the relationship between the ratio of the third distance to the fourth distance and at least one preset interval; wherein the third distance is the distance between the upper lip feature point and the lower lip feature point, and the fourth distance is the distance between the left lip corner feature point and the right lip corner feature point.

16. The device according to claim 11, characterized in that A face collection frame is displayed in the face preview area; the device also includes: A detection module, used to detect whether the facial key points in the facial image are located within the facial acquisition frame; The generating module is further configured to, if yes, unlock each unlocked node according to the order of unlocked nodes in the unlocked node sequence; The prompt module is used to send a prompt message to adjust the collection direction if no.

17. The device according to claim 11, characterized in that The facial image is an image acquired by capturing the object to be tested; A recognition module, configured to perform face recognition on a facial image corresponding to each unlocking node according to the order of the unlocking nodes in the unlocking node sequence, to obtain a face recognition result; The unlocking module is further used to determine that the unlocking is successful when an unlocking state identifier is generated at each unlocking node and it is determined according to the face recognition result that the object to be tested is consistent with the target object.

18. The device according to any one of claims 11 to 13, 16 to 17, characterized in that The device also includes: The second display module is used to display the expression input page; An input module, used to input expression images corresponding to expression identifiers in the expression input page in sequence through the expression input page; and enter an expression combination page; The combination module is used to combine the expression identifiers in response to the combination operation triggered on the expression combination page to obtain an expression unlocking sequence.

19. The device according to claim 12, characterized in that The device also includes: A determination module, configured to determine a collection time interval between the at least two facial images when the facial expressions in the at least two facial images match the corresponding target expressions; or determine an unlocking time interval between the unlocking node to be processed and a last processed unlocking node; The generating module is further configured to, when the collection time interval or the unlocking time interval satisfies the corresponding time interval condition, execute the step of generating an unlocking state identifier at the unlocking node to be processed when the facial expressions in the at least two facial images match the corresponding target expressions.

20. The device according to any one of claims 11 to 13, 16 to 17, characterized in that The device also includes: The recognition module is used to extract eyebrow feature points and eyelid feature points from the facial image; determine the fifth distance between the eyebrow feature points and the eyelid feature points; and determine the eyebrow posture according to the size relationship between the fifth distance and a preset distance.

21. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

22. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images