An encrypted unlocking system and method based on multi-scenario expression recognition

Abstract

The invention discloses a multi-scenario encryption unlocking system based on expression recognition, comprising a camera module, a data acquisition module, a data storage module, a data processing module and an unlocking application module; the data acquisition module shoots a user through the camera module, obtains a face image and uses the face image of the user to establish identity information, and prompts the user to collect the user's expression sequence in sequence to set an expression password; the data storage module stores relevant information; the data processing module analyzes the expression sequence, performs liveness detection, feature extraction and face recognition on the face image, compares the identity information, and then performs expression recognition on the face image to obtain an instruction, and obtains a comparison result by comparing the instruction with the user's expression password; unlocking is completed through the comparison result, and the system is applied in different scenarios; the system of the invention has strong reusability and can effectively cope with possible attacks in different scenarios, thereby ensuring the information security and experience of the user.

Description

Technical Field

[0001] The present invention relates to the research field of expression recognition, and in particular to an encryption unlocking system and method based on multi-scenario expression recognition. Background Art

[0002] Existing face recognition systems

[0003] Existing face recognition systems have been widely used in access management, access control and attendance, monitoring management, and photo search. Today, face recognition technology is relatively mature. Real-time images are collected by cameras, and face detection is performed on them to obtain face images. Then, preprocessing is performed and artificial neural networks are used for face detection and feature extraction. Finally, the classification results of a certain type of expression are output through feature matching and retrieval.

[0004] Existing password unlocking system

[0005] In order to protect the privacy of computers, mobile phones, and even software, various forms of password unlocking methods have been developed and applied, and unlocking has become faster, more personalized, and more user-friendly. The initial digital password unlocking was just to set a simple number combination, and then to gesture unlocking, which is essentially to record the connection button according to the drawn path. With the development of technology, the unlocking program has become increasingly diversified, and a variety of unlocking functions that are more interesting than digital and graphic passwords have been developed, such as fingerprint unlocking, face unlocking, voice unlocking, iris unlocking, etc. However, the security of these identification methods is relative, and there is no absolutely safe identification method. The trade-off between security and convenience depends not only on the level of technological development, but also on the acceptance of consumers.

[0006] Existing cloud server technology

[0007] Cloud servers are basic computing components composed of CPU, memory, operating system, and cloud hard disk. By deploying software that consumes computing unit resources on the corresponding cloud service, it provides computing, storage, network and other services for various user needs. Nowadays, cloud servers are safe, reliable, and elastically scalable computing services. Users can quickly create component cloud servers without purchasing hardware, and have efficient operation and maintenance. Management and expansion are simpler and more convenient than physical servers. They can meet different usage scenarios and multiple needs of users, and are widely used by small and medium-sized enterprises and individual users in the Internet industry.

[0008] Existing equipment and technology for facial expression recognition

[0009] The facial expression features are extracted through facial camera, and different facial expressions are recognized using trained classifiers. Facial expressions refer to various emotional states expressed through changes in facial muscles, eye muscles, and mouth muscles. The seven main human emotions: anger, happiness, sadness, surprise, disgust, fear, and nature are all represented by corresponding facial expressions in a formulaic way. After the recognition of expressions, the current mood of the protagonist in the picture is analyzed through the four steps of face detection, face registration, feature extraction, and expression classification.

[0010] Existing equipment and technology for face liveness detection

[0011] In order to attack the face recognition system, criminals collect facial images of legitimate users and create photos, videos, masks and other fake faces to crack the system. Liveness detection technology improves the security of face recognition systems by identifying whether the acquired facial images are from live faces or fake faces. Currently, liveness detection is mainly divided into two categories: one is liveness detection based on image classification models; the other is cooperative liveness detection, which requires the user to cooperate in turning the head, blinking and other actions.

[0012] Existing facial recognition unlocking and payment systems are not capable of resisting different types of attack techniques, and the research on security vulnerabilities is incomplete. Intruders use some kind of artifact to imitate real users, or make the algorithm unable to recognize the correct user's face to attack the system; the existing digital passwords are too simple. There are many low-cost and easy-to-implement cracking methods for single digital passwords. They are simple in form and highly insecure, and are increasingly unable to meet user needs; existing password systems rarely use facial expressions to unlock. Although facial recognition technology has made rapid progress in recent years, the development of further technology to unlock using facial expressions is not yet mature and has not been widely used; existing facial expression password systems only use a single facial expression for decoding, which is less complex and not rich in content and form; (6) Most existing facial recognition liveness detection technologies require the user to cooperate in completing actions such as raising the head, blinking, closing the eyes, or even reading text aloud, which takes a long time and has a poor user experience; or they require expensive instruments such as infrared cameras and multi-spectral cameras for detection. Although the accuracy rate is high, the cost is too high and cannot be widely popularized. Summary of the invention

[0013] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide an encryption unlocking system and method based on multi-scenario expression recognition.

[0014] The first object of the present invention is to provide a multi-scenario encryption and unlocking system based on expression recognition;

[0015] The second object of the present invention is to provide a multi-scenario encryption unlocking method based on expression recognition. The first object of the present invention is achieved by the following technical solutions:

[0016] A multi-scenario encryption unlocking system based on expression recognition, including a camera module, a data acquisition module, a data storage module, a data processing module, and an unlocking application module;

[0017] The data acquisition module takes a photo of the user through the camera module, obtains a face image and uses the face image of the user to establish identity information, and prompts the user to sequentially collect the user's expression sequence to set an expression password;

[0018] The data storage module stores the facial image, identity information and the corresponding user's expression password collected by the data collection module;

[0019] The data processing module analyzes the expression sequence, extracts features and recognizes the face images of living faces in the face images, compares the identity information in the data storage module, and then recognizes the expression of the face images to obtain a digital or letter instruction, and obtains a comparison result by comparing the instruction with the expression password of the user in the data storage module;

[0020] The unlocking application module completes encryption and unlocking through comparison results, and then applies the system in different subsequent scenarios.

[0021] Furthermore, the camera module is a terminal camera module, which can capture the user's facial image in a timely manner.

[0022] Furthermore, the data acquisition module obtains the face image through the camera module, uses the user's face image to establish and save identity information in the data storage module, and after establishing the identity information, collects the user's expression pictures according to the password sequence, and then sets the expression password.

[0023] Further, the data processing module includes a living body detection module, a face recognition module, and a facial expression recognition module;

[0024] The living body detection module preprocesses the whole image, which includes a background image and a face image. The preprocessing includes: cropping, aligning, and segmenting the whole image, and transforming and superimposing the space of the whole image, by operating from the time domain to the frequency domain, the spatial domain, or changing its color space; after preprocessing, the neural network is used to extract deep features of the whole image, and whether the face in the whole image is from a living body is determined based on its features. The judgment and detection of the whole image includes whether there are abnormal light and shadow, reflection of mobile phone frames, paper, displays, plastic materials, and abnormal skin texture;

[0025] The face recognition module preprocesses the static face image, and the preprocessing is: using an artificial neural network to perform face detection, determine the face position, size, and posture, accurately identify the position of different features of the face, and perform face alignment, and then perform grayscale and geometric normalization on the face image after a series of data enhancement operations; after preprocessing, the network model is used to extract the deep features of the face in the face image, that is, the face picture data is modeled and analyzed, the feature data of the extracted face image is searched, matched and compared with the feature template stored in the data storage module, and the identity information of the face is judged according to the degree of similarity to determine whether the user corresponding to the face image is entered into the system;

[0026] The facial expression recognition module preprocesses the facial expression image of the face image obtained by the data acquisition module, and the preprocessing is: using an artificial neural network to perform face detection, determine the face position, size, and posture, and accurately identify the positions of different facial features, and perform face alignment, and then perform grayscale and geometric normalization on the face image after a series of data enhancement operations; after preprocessing, the facial expression image is frame aggregated, and the deep learning algorithm is used to extract the deep features of the facial expression, and multiple frames are combined, and the facial expression image is input into the CNN, and the expression classification result is output, and the expression classification result is converted into a digital instruction.

[0027] Furthermore, the unlocking application module controls whether the terminal is unlocked according to the comparison results of the identity information and expression password of the data processing module and the data storage module. If the identity information does not match, the user is prompted through the terminal that the identity information does not exist and the identity information should be set first; if the expression password is incorrect, the user is prompted to re-enter it; if the identity information and expression password are correct, the corresponding application is completed according to the scenario.

[0028] The second object of the present invention is achieved by the following technical solutions:

[0029] A multi-scenario encryption unlocking method based on expression recognition comprises the following steps:

[0030] The user is photographed through the camera module to obtain a face image and use the face image of the user to establish identity information, and the user is prompted to collect the user's expression sequence in order to set an expression password;

[0031] Analyze the expression sequence, extract and identify the features of the face images that are live faces, compare the identity information in the data storage module, and then identify the face image to obtain a digital or letter instruction, and obtain a comparison result by comparing the instruction with the user's expression password in the data storage module;

[0032] The encryption and unlocking are completed through the comparison results, and the system is applied in different subsequent scenarios.

[0033] Furthermore, the expressions of the expression sequence include: anger, happiness, sadness, surprise, disgust, fear, and naturalness. The password length is set according to the corresponding identification of the expression, and the password is set according to the corresponding identification.

[0034] Furthermore, the setting of the password through its corresponding identification specifically includes setting a number or letter instruction for the expression, and setting the password and the password length through the number or letter instruction.

[0035] Furthermore, the feature extraction and recognition of the living face image portion in the overall image includes liveness detection, face recognition, and facial expression recognition;

[0036] The liveness detection is specifically as follows: preprocessing the whole image, the whole image includes a background image and a face image, the preprocessing is as follows: cropping, aligning, segmenting the whole image, and transforming and superimposing the space of the whole image, by operating from the time domain to the frequency domain, the spatial domain or changing its color space; after preprocessing, using a neural network to extract deep features of the whole image, judging whether the face in the whole image is from a living body according to its features, and judging whether the whole image has abnormal light and shadow, reflection of the mobile phone frame, paper, display, plastic material, and abnormal skin texture;

[0037] The face recognition is specifically as follows: preprocessing a static face image, wherein the preprocessing is as follows: using an artificial neural network to perform face detection, determine the face position, size, and posture, accurately identify the positions of different features of the face, and perform face alignment, and then perform grayscale and geometric normalization on the face image after a series of data enhancement operations; after preprocessing, extracting the deep features of the face in the face image through a network model, that is, performing a modeling and analysis process on the face picture data, searching, matching, and comparing the feature data of the extracted face image with the feature template stored in the data storage module, and judging whether the user corresponding to the face image has been entered into the system based on the degree of similarity of the face identity information;

[0038] The facial expression recognition is specifically as follows: preprocessing the facial expression image of the face image obtained by the data acquisition module, wherein the preprocessing is as follows: using an artificial neural network to perform face detection, determine the face position, size, and posture, accurately identify the positions of different features of the face, and perform face alignment, and then perform grayscale and geometric normalization on the face image after a series of data enhancement operations; after preprocessing, frame aggregation is performed on the facial expression image, deep learning algorithms are used to extract deep features of facial expressions, multiple frames are combined, the facial expression image is input into a CNN, expression classification results are output, and the expression classification results are converted into digital instructions.

[0039] Furthermore, the encryption and unlocking are completed through the comparison results, and the system is applied in different subsequent scenarios, specifically: whether the terminal is unlocked is controlled according to the comparison results of the identity information and the expression password of the data processing module and the data storage module. If the identity information does not match, the user is prompted through the terminal that the identity information does not exist and please set the identity information first; if the expression password is incorrect, the user is prompted to re-enter it; if the identity information and the expression password are correct, the corresponding application is completed according to the scenario.

[0040] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0041] 1. The present invention is convenient and easy to implement, and can meet the needs of various occasions. The present invention can capture the user's facial expression image through the camera of a terminal such as a computer, mobile phone, access control, etc. and convert it into a password, which has high reusability and is simple and convenient.

[0042] 2. The present invention can capture facial images of the human body through a camera, recognize facial expressions and convert them into digital instructions, recognize the user's facial expressions in real time online, increase the complexity of the password, reduce the risk of the password being cracked, and add fun.

[0043] 3. The present invention uses liveness detection to detect the background and face in facial expression pictures, prevents various 2D and 3D prosthetic attacks, ensures the security of user information in various scenarios, and improves the overall security of the system.

[0044] 4. The data processing and corresponding algorithm modules of the present invention are described by executable program codes and then read out and executed by the cloud server background. There is no need for complex hardware equipment to process data, which saves the space occupied by the product, has efficient data processing, and improves the user experience.

[0045] 5. The entire system of the present invention supports wireless communication, and data can be wirelessly transmitted to the storage module, which facilitates data storage, processing and data analysis operations, and has fast data preservation and will not be lost, thus enhancing the security and reliability of data storage. The data can be subsequently modified, processed and other operations.

[0046] 6. The present invention can be used for access control, computer, mobile phone unlocking, employee daily attendance punching, online payment and other scenarios. The system has strong reusability and can effectively deal with possible attacks in different scenarios to ensure user information security and experience BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a system structure block diagram of an encryption unlocking system based on multi-scenario expression recognition according to the present invention;

[0048] Figure 2is a structural block diagram of a data processing module in Embodiment 1 of the present invention;

[0049] Figure 3 The present invention is a flowchart of an encrypted unlocking method based on multi-scenario expression recognition. DETAILED DESCRIPTION

[0050] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0051] Embodiment 1:

[0052] An encrypted unlocking system based on multi-scenario expression recognition, such as Figure 1 As shown, it includes a camera module, a data acquisition module, a data storage module, a data processing module, and an unlocking application module;

[0053] The data acquisition module takes a photo of the user through the camera module, obtains a face image and uses the face image of the user to establish identity information, and prompts the user to sequentially collect the user's expression sequence to set an expression password;

[0054] The data storage module stores the facial image, identity information and the corresponding user's expression password collected by the data collection module;

[0055] The data processing module analyzes the expression sequence, extracts features and recognizes the face images of living faces in the face images, compares the identity information in the data storage module, and then recognizes the expression of the face images to obtain a digital or letter instruction, and obtains a comparison result by comparing the instruction with the expression password of the user in the data storage module;

[0056] The unlocking application module completes encryption and unlocking through comparison results, and then applies the system in different subsequent scenarios.

[0057] The details are as follows:

[0058] Data acquisition module:

[0059] The data acquisition module is used to collect the user's identity, face image data, facial expression password data information, and transmit it to the data storage module, so that the computing module can analyze and process these data. Users use terminals such as computers, mobile phones, software, and access control cameras to take facial images. The data acquisition module collects user face images and transmits them to the data storage module to establish an identity information file. After the file is established, the acquisition module collects the user's expression pictures according to the password sequence, and then transmits the expression picture data to the computing module for recognition.

[0060] Data storage module:

[0061] The data storage module receives the user identity information and facial image data collected by the data acquisition module, as well as the password instructions obtained by the calculation module when recognizing the expression password set by the user, and provides comparison data with the recognition result of the calculation module for subsequent user unlocking to determine whether the identity and password are correct.

[0062] Data processing module:

[0063] The data processing module is mainly composed of three parts: face recognition module, facial expression recognition module and liveness detection module. Figure 2 As shown, after the algorithm recognizes the face and expression and confirms that it is not an illegal prosthesis attack, it is responsible for completing key functions such as feature comparison and retrieval between the acquired face image and the face image in the data storage module, as well as whether the password matches.

[0064] Face recognition module

[0065] This module is responsible for preprocessing static images, using artificial neural networks for face detection, and aligning faces based on face landmarks detected. After data enhancement, the image is grayscale and geometrically normalized. After preprocessing, the facial features in the image are extracted, and the facial data is modeled and analyzed to extract the corresponding facial features, and then the features are retrieved, matched and compared to determine whether the user in the image is entered into the system; specifically: the face recognition module preprocesses the static face image, and the preprocessing is: using an artificial neural network to perform face detection, determine the face position, size, and posture, and accurately identify the positions of different facial features, including the corners of the eyes, pupils, mouth, and nose, and perform face alignment, and then grayscale and geometric normalization of the face image after a series of data enhancement operations, and the series of data enhancement operations include filtering, translation, flipping, and rotation; after preprocessing, the deep features of the face in the face image are extracted through the network model, that is, the face picture data is modeled and analyzed, the feature data of the extracted face image is searched, matched and compared with the feature template stored in the data storage module, and the identity information of the face is judged according to the degree of similarity to determine whether the user corresponding to the face image is entered into the system.

[0066] Facial expression recognition module

[0067] After the facial expression recognition module receives the facial expression picture from the acquisition module, it first preprocesses the expression image, and also aligns the facial positioning points. After data enhancement, the image is grayscale and geometrically normalized. After preprocessing, frame aggregation is performed, and the deep learning algorithm is used to extract the deep features of facial expressions. Multiple frames are combined, and the facial image is input into CNN to output the classification results of a certain type of expression. 7 different emotions: anger, happiness, sadness, surprise, disgust, fear, and nature. The emotions are represented by numbers 1-7, and other symbols can also be used to represent them. The classification results are converted into digital instructions;

[0068] Liveness detection module

[0069] The liveness detection module first preprocesses the image. The overall image includes a background image and a face image. The preprocessing is: cropping, aligning, and segmenting the overall image, and transforming and superimposing the space of the overall image, by operating from the time domain to the frequency domain, spatial domain, or changing its color space; after preprocessing, the neural network is used to extract features of the overall image, classify it according to its features, and judge whether the face in the overall image is from a living body. The judgment is to detect whether there are abnormal light and shadow, mobile phone frame, paper, display, plastic material reflection, and abnormal skin texture in the overall image.

[0070] Unlock application module:

[0071] The application module controls whether the terminal is unlocked based on the identity and password comparison results of the calculation module and the storage module. If the identity is incorrect, the terminal prompts the user that the identity information does not exist and asks the user to set the identity information first; if the ID card and password are incorrect, the user is prompted to re-enter; if the identity and password are correct, the corresponding application is completed according to the scenario. For example, whether the access control, computer, mobile phone, software, etc. can be opened; whether the online payment is successful; whether the attendance system records the punch-in information.

[0072] Example 2

[0073] An encrypted unlocking method based on multi-scenario expression recognition, such as Figure 3 As shown, the following steps are included:

[0074] The user is photographed through the camera module to obtain a face image and use the face image of the user to establish identity information, and the user is prompted to collect the user's expression sequence in order to set an expression password;

[0075] Analyze the expression sequence, extract and identify the features of the face images that are live faces, compare the identity information in the data storage module, and then identify the face images to obtain digital instructions, and compare the digital instructions with the user's expression password in the data storage module to obtain the comparison result;

[0076] The encryption and unlocking are completed through the comparison results, and the system is applied in different subsequent scenarios.

[0077] Furthermore, the expressions of the expression sequence include: anger, happiness, sadness, surprise, disgust, fear, and naturalness. The password length is set according to the corresponding identification of the expression, and the password is set according to the corresponding identification.

[0078] Furthermore, the setting of a corresponding identifier for the expression specifically includes setting a numeric or letter instruction for the expression, and setting a password and a password length through the numeric or letter instruction.

[0079] Furthermore, the facial images of living faces are subjected to feature extraction and recognition, including face recognition, facial expression recognition, and liveness detection;

[0080] The liveness detection is specifically as follows: preprocessing the whole image, the whole image includes a background image and a face image, the preprocessing is as follows: cropping, aligning, segmenting the whole image, and transforming and superimposing the space of the whole image, by operating from the time domain to the frequency domain, the spatial domain or changing its color space; after preprocessing, using a neural network to extract deep features of the whole image, judging whether the face in the whole image is from a living body according to its features, and judging whether the whole image has abnormal light and shadow, reflection of the mobile phone frame, paper, display, plastic material, and abnormal skin texture;

[0081] The face recognition is specifically as follows: preprocessing a static face image, wherein the preprocessing is as follows: using an artificial neural network to perform face detection, determine the face position, size, and posture, accurately identify the positions of different features of the face, and perform face alignment, and then perform grayscale and geometric normalization on the face image after a series of data enhancement operations; after preprocessing, extracting the deep features of the face in the face image through a network model, that is, performing a modeling and analysis process on the face picture data, searching, matching, and comparing the feature data of the extracted face image with the feature template stored in the data storage module, and judging whether the user corresponding to the face image has been entered into the system based on the degree of similarity of the face identity information;

[0082] The facial expression recognition is specifically as follows: preprocessing the facial expression image of the face image obtained by the data acquisition module, wherein the preprocessing is as follows: using an artificial neural network to perform face detection, determine the face position, size, and posture, accurately identify the positions of different features of the face, and perform face alignment, and then perform grayscale and geometric normalization on the face image after a series of data enhancement operations; after preprocessing, frame aggregation is performed on the facial expression image, deep learning algorithms are used to extract deep features of facial expressions, multiple frames are combined, the facial expression image is input into a CNN, expression classification results are output, and the expression classification results are converted into digital instructions.

[0083] Furthermore, the encryption and unlocking are completed through the comparison results, and the system is applied in different subsequent scenarios, specifically: whether the terminal is unlocked is controlled according to the comparison results of the identity information and the expression password of the data processing module and the data storage module. If the identity information does not match, the user is prompted through the terminal that the identity information does not exist and please set the identity information first; if the expression password is incorrect, the user is prompted to re-enter it; if the identity information and the expression password are correct, the corresponding application is completed according to the scenario.

[0084] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A multi-scenario encryption unlocking system based on expression recognition, characterized in that: It includes camera module, data acquisition module, data storage module, data processing module and unlocking application module; The data acquisition module takes a photo of the user through the camera module, obtains a face image and uses the face image of the user to establish identity information, and prompts the user to sequentially collect the user's expression sequence to set an expression password; The data acquisition module obtains a face image through a camera module, uses the face image of the user to establish and save identity information in a data storage module, and after establishing the identity information, collects expression pictures of the user according to the password sequence, and then sets an expression password; The data storage module stores the facial image, identity information and expression password of the corresponding user collected by the data collection module; The data processing module analyzes the expression sequence, extracts features and recognizes the face images of living faces in the face images, compares the identity information in the data storage module, and then recognizes the expression of the face images to obtain a digital or letter instruction, and obtains a comparison result by comparing the instruction with the expression password of the user in the data storage module; The data processing module includes a living body detection module, a face recognition module, and a facial expression recognition module; The living body detection module preprocesses the whole image, which includes the background image and the face image. The preprocessing includes: cropping, aligning, and segmenting the whole image, and transforming and superimposing the space of the whole image, by operating from the time domain to the frequency domain, the spatial domain, or changing its color space; after preprocessing, the neural network is used to extract deep features of the whole image, and it is judged whether the face in the whole image is from a living body according to its features, and whether the whole image is abnormal in light and shadow, mobile phone frame, paper, display, plastic material reflection, and skin texture abnormality; The face recognition module preprocesses the static face image, and the preprocessing is: using an artificial neural network to perform face detection, determine the face position, size, and posture, accurately identify the position of different features of the face, and perform face alignment, and then perform grayscale and geometric normalization on the face image after a series of data enhancement operations; after preprocessing, the network model is used to extract the deep features of the face in the face image, that is, the face picture data is modeled and analyzed, the feature data of the extracted face image is searched, matched and compared with the feature template stored in the data storage module, and the identity information of the face is judged according to the degree of similarity to determine whether the user corresponding to the face image is entered into the system; The facial expression recognition module pre-processes the facial expression image of the face image acquired by the data acquisition module, and the pre-processing includes: using an artificial neural network to perform face detection, determine the position, size, and posture of the face, accurately identify the positions of different features of the face, and perform face alignment, and then perform grayscale and geometric normalization on the face image after a series of data enhancement operations; After preprocessing, the facial expression images are frame aggregated, and the deep features of facial expressions are extracted using deep learning algorithms. Multiple frames are combined, and the facial expression images are input into CNN to output the expression classification results, which are then converted into digital instructions. The unlocking application module completes encryption and unlocking through comparison results, and then applies the system in different scenarios; The unlocking application module controls whether the terminal is unlocked according to the comparison results of the identity information and the expression password of the data processing module and the data storage module. If the identity information does not match, the user is prompted through the terminal that the identity information does not exist and the identity information should be set first; if the expression password is incorrect, the user is prompted to re-enter it; if the identity information and the expression password are correct, the corresponding application is completed according to the scenario.

2. According to the multi-scenario encryption and unlocking system based on expression recognition according to claim 1, it is characterized in that: The camera module is a terminal camera module, which can collect the user's facial image in time.

3. A multi-scenario encryption unlocking method based on expression recognition, characterized in that: A multi-scenario encryption and unlocking system based on expression recognition according to any one of claims 1 to 2 is provided, comprising the following steps: The user is photographed through the camera module to obtain a face image and use the face image of the user to establish and save identity information, and the user is prompted to collect the user's expression sequence in order to set an expression password; After establishing the identity information, the user's expression pictures are collected according to the password sequence, and then the expression password is set; Analyze the expression sequence, extract and identify the features of the face images that are live faces, compare the identity information in the data storage module, and then identify the face image to obtain a digital or letter instruction, and obtain a comparison result by comparing the instruction with the user's expression password in the data storage module; Extract and identify the live face image part in the whole image, including liveness detection, face recognition, and facial expression recognition; The liveness detection is specifically as follows: preprocessing the whole image, the whole image includes a background image and a face image, the preprocessing is as follows: cropping, aligning, segmenting the whole image, and transforming and superimposing the space of the whole image, by operating from the time domain to the frequency domain, the spatial domain or changing its color space; after preprocessing, using a neural network to extract deep features of the whole image, judging whether the face in the whole image is from a living body according to its features, judging whether the whole image has abnormal light and shadow, reflection of the mobile phone frame, paper, display, plastic material, and abnormal skin texture; The face recognition is specifically as follows: preprocessing a static face image, wherein the preprocessing is as follows: using an artificial neural network to perform face detection, determine the face position, size, and posture, accurately identify the positions of different features of the face, and perform face alignment, and then perform grayscale and geometric normalization on the face image after a series of data enhancement operations; after preprocessing, extracting the deep features of the face in the face image through a network model, that is, performing a modeling and analysis process on the face picture data, searching, matching, and comparing the feature data of the extracted face image with the feature template stored in the data storage module, and judging whether the user corresponding to the face image has been entered into the system based on the degree of similarity of the face identity information; The facial expression recognition is specifically as follows: preprocessing the facial expression image of the face image acquired by the data acquisition module, wherein the preprocessing is as follows: using an artificial neural network to perform face detection, determine the face position, size, and posture, accurately identify the positions of different features of the face, and perform face alignment, and then perform grayscale and geometric normalization on the face image after a series of data enhancement operations; after the preprocessing, frame aggregation is performed on the facial expression image, and deep learning algorithms are used to extract deep features of facial expressions, multiple frames are combined, and the facial expression image is input into CNN, and expression classification results are output, and the expression classification results are converted into digital instructions; The encryption and unlocking are completed through the comparison results, and the system is applied in different subsequent scenarios. Specifically: whether the terminal is unlocked is controlled based on the comparison results of the identity information and expression password of the data processing module and the data storage module. If the identity information does not match, the user is prompted through the terminal that the identity information does not exist and please set the identity information first; if the expression password is incorrect, the user is prompted to re-enter it; if the identity information and expression password are correct, the corresponding application is completed according to the scenario.

4. According to the multi-scenario encryption unlocking method based on expression recognition according to claim 3, it is characterized in that: The expressions of the expression sequence include: anger, happiness, sadness, surprise, disgust, fear, and nature. The password length is set according to the corresponding identification of the expression, and the password is set according to the corresponding identification.

5. According to the multi-scenario encryption unlocking method based on expression recognition according to claim 4, it is characterized in that: The password is set by its corresponding identification, specifically setting a number or letter instruction for the expression, and setting the password and the password length by the number or letter instruction.

Images