Biometric feature recognition method and apparatus, electronic device, and storage medium

Abstract

The application discloses a biometric feature recognition method and device, an electronic device and a storage medium, and relates to the field of biometric recognition, and the method comprises the following steps: receiving a biometric image sequence uploaded by a client, wherein the biometric image sequence refers to an image sequence extracted after video collection of a target object; analyzing expression labels, intention emotions and biometric feature comparison results in the biometric image sequence; determining a recognition confidence based on at least one of the expression labels, the intention emotions and the biometric feature comparison results; and outputting an abnormality notification result of biometric security recognition of the target object to the client when the recognition confidence is greater than a preset confidence threshold. The application solves the technical problem of insufficient safety of biometric feature recognition through action liveness detection in the related art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of biometric recognition, and in particular, to a biometric feature recognition method and device, an electronic device, and a storage medium. BACKGROUND

[0002] In recent years, with the upgrading of various terminals and the increasing demand for information security in various scenarios, face recognition has been widely used in various fields due to its non-replicable, non-contact, scalable and convenient characteristics. In particular, it is commonly used in online identity authentication scenarios. Face recognition has become an inevitable part of information security verification.

[0003] In the application process of face recognition technology, there are two major links, namely action liveness detection and face recognition comparison. That is, after the normal face is determined by liveness detection, face recognition comparison is performed to determine the identity of the object to be recognized. It can be seen that face liveness detection is an important link to improve the security and reliability of the face authentication system, and the face image determines the final recognition result.

[0004] In related technologies, the most widely used mobile APP and other online channel scenarios of face recognition mainly realize liveness detection through the interaction between the application and the user by issuing action instructions. The user is required to cooperate with the given random action instructions, such as blinking, turning the head, opening the mouth, etc. Then, the motion information of the face region is recognized from the collected video, and it is determined whether the motion information is consistent with the given instruction to determine whether it is a live body. At the same time, a front face photo is selected from the collected content for face recognition comparison.

[0005] Action liveness detection technology is widely used in biometric recognition scenarios. It can effectively prevent attacks such as static photos, video playback, and static head models. However, it also has the following disadvantages: first, due to the small number of randomly generated action instructions, with the emergence of various photo activation tools and AI face changing software, the cost and difficulty of implementing liveness detection attacks are becoming lower and lower. It is easy to generate actions corresponding to the action instructions in advance through video to perform biometric recognition and deceive business personnel; second, in some special scenarios, action liveness detection cannot effectively identify whether the person is willing to perform face recognition operation; third, in action liveness detection, the user not only needs to face the screen, but also needs to make corresponding actions according to the screen instructions. For some scenarios, such as when the user wears a mask or does not wear glasses, it is easy to cause a poor user experience.

[0006] To address the above problems, no effective solutions have been proposed so far. SUMMARY

[0007] Embodiments of the present application provide a biometric feature recognition method and device, an electronic device, and a storage medium to at least solve the technical problem of insufficient security of biometric feature recognition through action liveness detection in related technologies.

[0008] According to an aspect of an embodiment of the present application, a biometric feature recognition method is provided, applied to a server, including: receiving a biometric image sequence uploaded by a client, wherein the biometric image sequence is an image sequence extracted after video collection of a target object; analyzing expression labels, intent emotions, and biometric feature comparison results in the biometric image sequence; determining a recognition confidence based on at least one of the expression labels, the intent emotions, and the biometric feature comparison results; and outputting an abnormality notification result of biometric security recognition of the target object to the client if the recognition confidence is greater than a preset confidence threshold.

[0009] Optionally, the step of analyzing the expression labels, intent emotions, and biometric feature comparison results in the biometric image sequence includes: inputting the biometric image sequence into a specified deep learning model to output the expression labels corresponding to each biometric image in the biometric image sequence; inputting the expression labels into a specified analysis model to output weights corresponding to each expression label; determining the intent emotions of the target object by combining the weights of the expression labels and normal distribution information of all expression labels; and extracting feature vector values corresponding to each biometric image using a biometric feature extraction algorithm, comparing the feature vector values with original registered features, and obtaining the biometric feature comparison results.

[0010] Optionally, a target loss function value used by a neural network in the specified deep learning model is calculated using a normalized loss function and an island loss function, wherein the normalized loss function uses calculation elements including a training sample number, a label classification number, and an image element of an output vector, wherein the training sample number is a total number of training samples formed by historical biometric images and corresponding historical expression labels in a historical time period, the label classification number is a total number of all classified expression labels, and the output vector is a vector corresponding to the output expression label of the neural network; and the island loss function uses calculation elements including the training sample number, a sample feature vector, an expression label set, an expression label center, and a first preset weight value.

[0011] Optionally, before the expression labels are input into the specified analysis model to output weights corresponding to each expression label, the method further includes: training a mixture model including multiple Gaussian distribution functions using an expectation maximization algorithm to obtain the specified analysis model.

[0012] Optionally, the step of determining the intention emotion of the target object in combination with the weight of the expression label and the normal distribution information of all expression labels comprises: determining an intention confidence value corresponding to the biological image sequence in combination with the weight of the expression label and the normal distribution information of all expression labels; in the case that the intention confidence value is within the confidence interval of the specified abnormal emotion, it is confirmed that the mental state of the target object is abnormal, and the intention emotion of the target object is abnormal; in the case that the intention confidence value is within the confidence interval of the specified normal emotion, it is confirmed that the mental state of the target object is normal, and the intention emotion of the target object is normal.

[0013] Optionally, the step of extracting the feature vector value corresponding to each biological image by using a biological feature extraction algorithm comprises: in the case that the biological feature is a face feature, extracting a face frame in each biological image by using a specified convolutional neural network; taking the face frame as a reference, mapping the face appearing in each biological image to a multi-dimensional space vector by using a specified face vector model to obtain the feature vector value.

[0014] Optionally, the specified convolutional neural network uses a multi-task convolutional neural network, and the multi-task convolutional neural network comprises three sub-networks: a first sub-network for generating a candidate window, a second sub-network for filtering the candidate window, and a third sub-network for generating a final bounding box and a face key point.

[0015] According to another aspect of the embodiment of the present application, a biological feature recognition method applied to a client is further provided, comprising: when detecting that a target object is aligned to an acquisition frame, extracting images meeting biological recognition detection requirements in a video stream according to a specified frame rate to obtain an extracted image set, wherein the video stream refers to a video of the target object collected by a shooting device; obtaining a specified part front area of each image in the extracted image set, and cutting a specified part area image to obtain a biological image sequence; and sending the biological image sequence to a server, wherein the server analyzes expression labels, intention emotions and biological feature comparison results in the biological image sequence after receiving the biological image sequence, and determines a biological safety recognition result of the target object.

[0016] Optionally, after cutting a specified part area image to obtain a biological image sequence, the method further comprises: compressing the biological image sequence by using a run-length encoding strategy; and encrypting the compressed biological image sequence by using a specified encryption algorithm.

[0017] Optionally, the step of analyzing the expression label, the intention emotion, and the biometric feature comparison result in the biological image sequence comprises: the server inputs the biological image sequence into a specified deep learning model to output the expression label corresponding to each biological image in the biological image sequence; the server inputs the expression label into a specified analysis model to output the weight corresponding to each expression label, and determines the intention emotion of the target object by combining the weight of the expression label and the normal distribution information of all expression labels; and the server extracts the feature vector value corresponding to each biological image by using a biometric feature extraction algorithm, and compares the feature vector value with the original registered feature to obtain the biometric feature comparison result.

[0018] Optionally, the front region of the specified part at least comprises a front face region.

[0019] According to another aspect of the embodiments of the present application, a biometric feature recognition system is further provided, comprising: a client connected with a shooting device, which executes any one of the biometric feature recognition methods applied to the client after receiving a video stream transmitted by the shooting device; and a server connected with the client, which executes any one of the biometric feature recognition methods applied to the server.

[0020] According to another aspect of the embodiments of the present application, a biometric feature recognition device applied to a server is further provided, comprising: a receiving unit configured to receive a biological image sequence, wherein the biological image sequence is an image sequence extracted after video collection of a target object; an analyzing unit configured to analyze an expression label, an intention emotion, and a biometric feature comparison result in the biological image sequence; a determining unit configured to determine a recognition confidence based on at least one of the expression label, the intention emotion, and the biometric feature comparison result; and an output unit configured to output an abnormality notification result of biometric security recognition of the target object to a client when the recognition confidence is greater than a preset confidence threshold.

[0021] Optionally, the analyzing unit comprises: a first input module configured to input the biological image sequence into a specified deep learning model to output the expression label corresponding to each biological image in the biological image sequence; a first determining module configured to input the expression label into a specified analysis model to output the weight corresponding to each expression label, and determine the intention emotion of the target object by combining the weight of the expression label and the normal distribution information of all expression labels; and a first extraction module configured to extract the feature vector value corresponding to each biological image by using a biometric feature extraction algorithm, and compare the feature vector value with the original registered feature to obtain the biometric feature comparison result.

[0022] Optionally, the target loss function value used by the neural network in the specified deep learning model is calculated by using a normalized loss function and an island loss function, wherein the normalized loss function uses calculation elements including a number of training samples, a number of label categories, and image elements of an output vector, wherein the number of training samples is a total number of training samples formed by historical biological images and corresponding historical expression labels in a historical time period, the number of label categories is a total number of all to-be-classified expression labels, and the output vector is a vector corresponding to an output expression label of the neural network; the island loss function uses calculation elements including the number of training samples, a sample feature vector, an expression label set, an expression label center, and a first preset weight value.

[0023] Optionally, the analysis unit further includes a first training module configured to train a mixture model including a plurality of Gaussian distribution functions by using an expectation maximization algorithm to obtain the specified analysis model.

[0024] Optionally, the first determination module includes a first determination submodule configured to determine an intention confidence value corresponding to the biological image sequence in combination with a weight of the expression label and normal distribution information of all expression labels; a first confirmation submodule configured to confirm that the psychological state of the target object is abnormal and the intention emotion of the target object is abnormal if the intention confidence value is within a confidence interval of a specified abnormal emotion; and a second confirmation submodule configured to confirm that the psychological state of the target object is normal and the intention emotion of the target object is normal if the intention confidence value is within a confidence interval of a specified normal emotion.

[0025] Optionally, the first extraction module includes a first extraction submodule configured to extract a face frame in each biological image by using a specified convolutional neural network if the biological feature is a face feature; and a first mapping submodule configured to map a face appearing in each biological image to a multidimensional space vector by using a specified face vector model to obtain the feature vector value, with the face frame as a reference.

[0026] Optionally, the specified convolutional neural network uses a multi-task convolutional neural network, and the multi-task convolutional neural network includes three subnetworks: a first subnetwork for generating a candidate window, a second subnetwork for filtering the candidate window, and a third subnetwork for generating a final bounding box and face key points.

[0027] According to another aspect of the embodiments of the present application, there is also provided a biometric feature recognition device applied to a client, comprising: a detection unit configured to extract images meeting biometric recognition detection requirements in a video stream according to a specified frame rate when detecting that a target object is aligned with a capture frame, to obtain an extracted image set, wherein the video stream refers to a video of the target object captured by a shooting device; an acquisition unit configured to acquire a specified part front region of each image in the extracted image set, to intercept a specified part region image, and to obtain a biometric image sequence; and a sending unit configured to send the biometric image sequence to a server, wherein the server analyzes expression labels, intention emotions and biometric feature comparison results in the biometric image sequence after receiving the biometric image sequence, and determines a biometric security recognition result of the target object.

[0028] Optionally, the biometric feature recognition device further comprises: a first compression module configured to compress the biometric image sequence by using a run-length encoding strategy; and a first encryption module configured to encrypt the compressed biometric image sequence by using a specified encryption algorithm.

[0029] Optionally, the biometric feature recognition device further comprises: a second input module configured to input the biometric image sequence to a specified deep learning model by the server, to output the expression labels corresponding to each biometric image in the biometric image sequence; a second determination module configured to input the expression labels to a specified analysis model by the server, to output weights corresponding to each expression label, and to determine the intention emotions of the target object by combining the weights of the expression labels and normal distribution information of all expression labels; and a second extraction module configured to extract feature vector values corresponding to each biometric image by using a biometric feature extraction algorithm by the server, to compare the feature vector values with original registered features, and to obtain the biometric feature comparison results.

[0030] Optionally, the specified part front region at least includes a face front region.

[0031] According to another aspect of the embodiments of the present application, there is also provided a computer readable storage medium comprising a stored computer program, wherein the computer program, when executed, controls a device in which the computer readable storage medium is located to perform any of the biometric feature recognition methods described above.

[0032] According to another aspect of the embodiments of the present application, there is also provided an electronic device comprising one or more processors and a memory, the memory being configured to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement any of the biometric feature recognition methods described above.

[0033] In the present disclosure, the following steps are adopted to receive a biological image sequence uploaded by a client, wherein the biological image sequence refers to an image sequence extracted after video collection of a target object; expression labels, intention emotions and biological feature comparison results in the biological image sequence are analyzed; a recognition confidence is determined based on at least one of the expression labels, the intention emotions and the biological feature comparison results; and in the case that the recognition confidence is greater than a preset confidence threshold, an abnormality notification result of biological safety recognition of the target object is output to the client.

[0034] In the present disclosure, when biological feature recognition is performed, three verifications including expression detection, intention emotion analysis and face feature comparison are included, and living body detection is performed through expressions and intention emotions, which not only can prevent attacks such as photo activation, but also can analyze intention emotion abnormality in user face recognition, thereby improving the safety and reliability of face recognition, and further solving the technical problem of insufficient safety of biological feature recognition through motion living body detection in the related art.

[0035] The biological feature recognition method in the present disclosure does not depend on specific device hardware, can be applied on a client device such as a mobile terminal (for example, a mobile phone, a tablet) and the like, does not need to increase user cost, and has a basis for wide application and promotion.

[0036] In the present disclosure, in the face recognition process, the user only needs to cooperate with the face collection frame facing the device screen, the whole process is simple and convenient, and does not need to be greatly cooperated with actions, thereby greatly improving the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0037] The accompanying drawings, which are included to provide a further understanding of the application and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application. In the drawings:

[0038] Figure 1 is a biological feature recognition method flowchart applied to a server according to an embodiment of the present application;

[0039] Figure 2 is a biological feature recognition method flowchart applied to a client according to an embodiment of the present application;

[0040] Figure 3 is a biological feature recognition system structure block diagram according to an embodiment of the present application;

[0041] Figure 4 is a biological feature recognition device schematic diagram applied to a server according to an embodiment of the present application;

[0042] Figure 5is a schematic diagram of an optional biometric recognition device applied to a client according to an embodiment of the present application;

[0043] Figure 6 is a hardware structure block diagram of an electronic device (or a mobile device) of a biometric recognition method according to an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the personnel in the technical field better understand the present application scheme, the technical scheme in the embodiment of the present application will be described clearly and completely in combination with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, not all. Based on the embodiment in the present application, all other embodiments obtained by the person skilled in the art without creative labor should belong to the protection scope of the present application.

[0045] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0046] It should be noted that the biometric recognition method and device in the present disclosure can be used in the field of biometric recognition, and can also be used in any field other than the field of biometric recognition in the case of living body detection and face recognition. The application field of the biometric recognition method and device in the present disclosure is not limited.

[0047] It should be noted that the related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, analyzed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties. For example, an interface is provided between the system and the related user or institution. Before obtaining the related information, the interface needs to send a request for obtaining to the aforementioned user or institution, and after receiving the consent information fed back by the aforementioned user or institution, the related information is obtained.

[0048] The application can be applied to various biometric recognition devices / apparatuses / products / systems (for example, various mobile terminals, PC terminals, chat software, and financial APPs), and is particularly suitable for biometric recognition, liveness detection, and facial information recognition in the business processing process of a financial institution, and includes three verifications of expression detection, intent emotion analysis, and facial feature comparison. Liveness detection is performed through micro-expression and intent emotion, which can not only prevent attacks such as photo activation and judge the similarity between the collected image and the user, but also analyze the abnormal situation of the intent emotion of the user during facial recognition, thereby improving the security and reliability of facial recognition.

[0049] The application will be described in detail below in conjunction with various embodiments.

[0050] Embodiment one

[0051] According to the embodiment of the application, a biometric recognition method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in a different order.

[0052] When performing facial recognition, liveness detection needs to be performed first. Liveness detection is used to determine whether the biological information is obtained from a legal user with biological characteristics (especially by detecting whether the user object in front of the video detection terminal is a user object, distinguishing whether the user object is a live user, and preventing interference by photo synthesis technology). Liveness detection is mainly performed by recognizing physiological information on the live body, and physiological information is used as a live body feature to distinguish biological non-live body features such as flat photos, silica gel, and plastic non-living substances.

[0053] Figure 1 is a flowchart of a biometric recognition method applied to a server according to an embodiment of the application, as shown in Figure 1 The method comprises the following steps:

[0054] Step S101, receiving a biological image sequence uploaded by a client, wherein the biological image sequence is an image sequence extracted after video collection of a target object;

[0055] Step S102, analyzing expression labels, intent emotions, and biological feature comparison results in the biological image sequence;

[0056] Step S103, determining a recognition confidence level based on at least one of the expression labels, intent emotions, and biological feature comparison results;

[0057] Step S104, in the case where the recognition confidence is greater than the preset confidence threshold, an abnormal notification result of biological safety recognition of the target object is output to the client.

[0058] Through the above steps, the biological image sequence uploaded by the client is received, wherein the biological image sequence refers to an image sequence extracted after video collection of the target object; the expression label, the intention emotion, and the biological feature comparison result in the biological image sequence are analyzed; the recognition confidence is determined based on at least one of the expression label, the intention emotion, and the biological feature comparison result; and in the case where the recognition confidence is greater than the preset confidence threshold, an abnormal notification result of biological safety recognition of the target object is output to the client. In the embodiment, three verifications including expression detection, intention emotion analysis, and face feature comparison are included, and micro-expression and intention emotion are used for living body detection, which can not only prevent attacks such as photo activation, but also analyze the intention emotion abnormality of the user during face recognition, thereby improving the safety and reliability of face recognition, and further solving the technical problem of insufficient safety of biological feature recognition through motion living body detection in the related art.

[0059] The above steps will be described in detail below.

[0060] It should be noted that the server is taken as the implementation subject in the embodiment of the present application, and the server types include but are not limited to: local server, cloud server (the cloud server integrates resources through virtualization and cluster technology, and generates the required resources of the host through a cloud control platform), and the server can realize micro-expression analysis, intention analysis processing, face comparison processing, and result judgment functions. The server is connected with the client to realize interaction, the local SDK (software development kit) is integrated in the client terminal, the face recognition security enhancement service is deployed in the server, and the client program is called. After the client initiates a face recognition request, the camera device is driven to collect user face image sequence information, and then the server is called to send the processed and extracted image information to the server (or to the server) for face recognition judgment processing. Finally, the recognition result returned by the server is received, so as to cooperatively complete the whole face recognition process and ensure the safety of face recognition application.

[0061] It should be noted that before receiving the biological image sequence, the face information needs to be collected first. When the client collects the face image information, the face detection SDK (software development kit) is encapsulated, the camera device is driven, the shooting device (for example, a mobile phone camera or a specified camera) is opened to obtain real-time video stream, the multi-task convolutional neural network is used to obtain the face frame in the current image, and it is judged whether the angle, distance, and occlusion of the face to be recognized meet the requirements. If the detection is passed, it means that the user's face is ready, otherwise, the user is prompted to face the screen, approach (away) the screen, and remove the occlusion according to the angle, distance, and occlusion.

[0062] When the face is ready, the client drives the camera device, opens the camera to obtain a real-time video stream, and extracts a front face image (corresponding to the above-mentioned image meeting the biological recognition detection requirement in the video stream according to the specified frame rate) meeting the face recognition detection requirement at a frame rate (set according to the actual terminal configuration and network transmission speed) when the user aims at the face collection frame. The front face image contains the facial muscle micro changes and facial expression information of the face in the specified time sequence.

[0063] In the embodiment, after obtaining the face image information, the facial key information in the face image needs to be intercepted. The local binary pattern algorithm can be selected to locate the front face position of the face image, intercept the image of the region where the face is located, discard the image that does not meet the image quality requirement, form a standardized image sequence, and reduce the transmission of redundant information.

[0064] Meanwhile, in order to ensure the accurate and safe transmission of information, the image sequence also needs to be compressed and encrypted. In the embodiment of the application, after intercepting the specified part region image and obtaining the biological image sequence, the client compresses the biological image sequence by using the run-length encoding strategy; and the client encrypts the compressed biological image sequence by using a specified encryption algorithm.

[0065] It should be noted that the run-length encoding strategy and the encryption algorithm can be selected by the user. In the embodiment, the image sequence can be compressed by using the run-length encoding (RLE) to reduce the transmission data amount; and the data can be encrypted by using the asymmetric encryption algorithm SM2 to improve the information transmission security.

[0066] After encoding and encryption, the client transmits the encrypted biological image sequence to the server, and the server completes the biological feature recognition operation by receiving the encrypted biological image sequence.

[0067] In step S101, the biological image sequence uploaded by the client is received, wherein the biological image sequence refers to the image sequence extracted after video collection of the target object.

[0068] The image sequence information collected and obtained by the client is transmitted to the server, and the information contained in the image sequence is analyzed and processed by the server.

[0069] In step S102, the expression label, the intention emotion and the biological feature comparison result in the biological image sequence are analyzed.

[0070] In the embodiment of the present application, in step S102, the biological image sequence is input into a specified deep learning model, and an expression label corresponding to each biological image in the biological image sequence is output; the expression label is input into a specified analysis model, and a weight corresponding to each expression label is output; the intention emotion of the target object is determined by combining the weight of the expression label and the normal distribution information of all expression labels; and a biological feature extraction algorithm is used to extract a feature vector value corresponding to each biological image, and the feature vector value is compared with the original registered feature to obtain a biological feature comparison result.

[0071] It should be noted that the expression label involved in the embodiment of the present application is obtained based on facial micro-expression. Micro-expression is a short and involuntary facial expression that is inadvertently revealed when a person tries to suppress or repress real emotions. Micro-expression can reflect the instantaneous psychological changes of a user after stress and is an important clue for identifying the real intention of the user. Micro-expression has a relatively short duration, generally only 40ms-200ms, and is often difficult to detect with the naked eye. However, it can be effectively detected and identified by using computer vision processing technology. The server obtains the expression label by using a deep learning model after receiving the biological image sequence. The expression label is self-defined according to requirements and includes but is not limited to sadness, anger, happiness, surprise, fear, and neutrality.

[0072] In the embodiment of the present application, the target loss function value used by the neural network in the specified deep learning model is calculated by using a normalized loss function and an island loss function. The calculation elements of the normalized loss function include the number of training samples, the number of label classifications, and the image elements of the output vector. The number of training samples is the total number of training samples formed by historical biological images and corresponding historical expression labels in a historical time period. The number of label classifications is the total number of all classified expression labels. The output vector is a vector corresponding to the output expression label of the neural network. The calculation elements of the island loss function include the number of training samples, the sample feature vector, the expression label set, the expression label center, and the first preset weight value.

[0073] The normalized loss function calculation formula is as follows:

[0074]

[0075] wherein m represents the number of training samples, n represents the number of expression label classifications, Z i represents the i-th element of the output vector of the trained network, and Z i represents the j-th element of the output vector of the trained network, and L s is the normalized loss function value.

[0076] On the basis of the normalized loss function, there is also an island loss function, and the loss function calculation formula is as follows:

[0077]

[0078] wherein m represents the number of training samples, x i represents the feature vector of the i-th sample, C yi represents the feature center of the expression class y i , L IL represents the island loss function value, λ1 is a weight value, N is the expression label set, c j and c k represent the j-th and k-th expression label center, +1 makes the function value range between 0 and 2, and the closer to 0 indicates the greater the class difference.

[0079] Therefore, the target loss function value calculation formula of the specified deep learning model is as follows:

[0080] L = L S + λL IL ;

[0081] wherein λ is a parameter for balancing the normalized loss function and the island loss function, L is the target loss function value of the specified deep learning model, L s is the normalized loss function value, and L IL represents the island loss function value.

[0082] In the embodiment of the present application, after the micro-expression is processed to obtain the expression label, the intention and emotion of the target object need to be analyzed, the expression label obtained in the above step is input into the specified analysis model, and the weight corresponding to each expression label is output, the intention and emotion of the target object are determined by combining the weight of the expression label and the normal distribution information of all expression labels. Before the expression label is input into the specified analysis model, the weight corresponding to each expression label is output, and the mixture model containing a plurality of Gaussian distribution functions is trained by using the expectation maximization algorithm to obtain the specified analysis model.

[0083] It should be noted that the mixture model of Gaussian distribution function contains a plurality of Gaussian distribution functions, and the calculation result is obtained by weighted summation of each Gaussian probability density function. When constructing the model, different micro-expressions will have different degrees of influence in the model. By applying the Gaussian mixture model and using the expectation maximization algorithm for training, the weight and model probability value of each micro-expression are obtained.

[0084] In the embodiment of the present application, the step of determining the intention emotion of the target object in combination with the weight of the expression label and the normal distribution information of all expression labels comprises: determining the intention confidence value corresponding to the biological image sequence in combination with the weight of the expression label and the normal distribution information of all expression labels; in the case that the intention confidence value is in the confidence interval of the specified abnormal emotion, it is confirmed that the psychological state of the target object is abnormal and the intention emotion of the target object is abnormal; in the case that the intention confidence value is in the confidence interval of the specified normal emotion, it is confirmed that the psychological state of the target object is normal and the intention emotion of the target object is normal.

[0085] It should be noted that in the embodiment, the micro-expression image sequence collected when the user registers is taken as the comparison reference, and the micro-expression image sequence of the user in the face recognition process is compared. If the measured feature data exceeds the given confidence level (corresponding to the confidence interval of the abnormal emotion), it is judged that the psychological state of the object to be detected has an abnormal intention emotion, which indicates that the face recognition scene of the user at this time has a risk, and the current face recognition is not passed.

[0086] In the embodiment of the present application, the step of extracting the feature vector value corresponding to each biological image by using the biological feature extraction algorithm comprises: in the case that the biological feature is a face feature, a specified convolutional neural network is used to extract the face frame in the biological image; taking the face frame as a reference, a specified face vector model is used to map the face appearing in the biological image to a multi-dimensional space vector to obtain the feature vector value.

[0087] In the embodiment of the present application, the specified convolutional neural network adopts a multi-task convolutional neural network, and the multi-task convolutional neural network comprises three sub-networks: a first sub-network for generating a candidate window, a second sub-network for filtering the candidate window, and a third sub-network for generating a final boundary frame and a face key point.

[0088] It should be noted that after analyzing and processing the expression label and the intention emotion, the face features need to be analyzed and processed, the face feature vector value of the input face original image is extracted through the face feature extraction algorithm, and is compared with the original registered face features to judge the similarity, so as to judge whether the user performing the biometric feature recognition is the user himself, the face frame in the current image is obtained through the convolutional neural network, the multi-task neural network model mainly adopts three cascaded networks, adopts the idea of candidate frame plus classifier, and performs fast and efficient face detection, which are P-Net (corresponding to the first sub-network) for quickly generating candidate window, R-Net (corresponding to the second sub-network) for high-precision candidate window filtering and selection, and O-Net (corresponding to the third sub-network) for generating final boundary frame and face key point, then the face image extracted in the current image is mapped to a multi-dimensional space vector, and the network directly outputs a 128-dimensional vector space, that is, the feature vector value. Finally, the extracted feature vector and the face feature vector generated when the user registers are subjected to inner product operation, and when the similarity result of the two is greater than the set threshold, it is considered that the user performing the live face recognition and the user registering are the same person, and the final face feature comparison similarity result is returned after calculation.

[0089] In step S103, the recognition confidence is determined based on at least one of the expression label, the intention emotion and the biometric feature comparison result.

[0090] In step S104, if the recognition confidence is greater than the preset confidence threshold, the abnormality notification result of the biometric security recognition of the target object is output to the client.

[0091] It should be noted that according to the security level requirement, the confidence threshold is set in advance, and the final biometric feature recognition result is obtained by combining the expression label, the intention emotion and the processing result of the face feature comparison returned by the server. The recognition result includes but is not limited to: whether the face is consistent with the user himself, whether the user has life characteristics, and whether the user is in a safe face recognition scene.

[0092] Through the above embodiment, the expression detection, intention emotion analysis and face feature comparison triple verification are included, the micro-expression and intention emotion are used for live detection, which can not only prevent photo activation and other attacks, but also judge the similarity between the collected image and the user himself, and analyze the intention emotion abnormality of the user face recognition, improve the security and reliability of the face recognition, and solve the technical problem of insufficient security of biometric feature recognition through motion live detection in the related art.

[0093] The following provides an embodiment of a biometric feature recognition method applied to a client.

[0094] Figure 2 is an optional biometric recognition method flowchart applied to a client according to an embodiment of the present application, as shown in the figure, comprising the following steps: Figure 2

[0095] Step S201, when detecting that the target object is aligned with the collection frame, extracting images meeting the biometric recognition detection requirements in the video stream according to a specified frame rate to obtain an extracted image set, wherein the video stream refers to a video of the target object collected by a shooting device;

[0096] Step S202, obtaining a specified part front area of each image in the extracted image set, intercepting the specified part area image to obtain a biological image sequence;

[0097] Step S203, sending the biological image sequence to a server, wherein the server analyzes the expression label, the intention emotion and the biometric feature comparison result in the biological image sequence after receiving the biological image sequence, and determines the biometric security recognition result of the target object.

[0098] Through the above steps, when detecting that the target object is aligned with the collection frame, extracting images meeting the biometric recognition detection requirements in the video stream according to a specified frame rate to obtain an extracted image set, wherein the video stream refers to a video of the target object collected by a shooting device; obtaining a specified part front area of each image in the extracted image set, intercepting the specified part area image to obtain a biological image sequence; sending the biological image sequence to a server, wherein the server analyzes the expression label, the intention emotion and the biometric feature comparison result in the biological image sequence after receiving the biological image sequence, and determines the biometric security recognition result of the target object. In this embodiment, three verifications including expression detection, intention emotion analysis and face feature comparison are included, and micro-expression and intention emotion are used for living body detection, which can not only prevent attacks such as photo activation, but also analyze the abnormal situation of intention emotion when the user performs face recognition, thereby improving the security and reliability of face recognition, and further solving the technical problem of insufficient security of biometric feature recognition by action living body detection in the related art.

[0099] The above steps will be described in detail below.

[0100] In the embodiment of the present application, the client establishes a connection with the server, and the client includes functions such as face detection, data collection, image extraction, data compression and data encryption. The client is applied to various terminals such as mobile terminals, PC terminals and image collection devices, and the client terminal includes a video recording device and a display device.

[0101] ​Step S201, when detecting that the target object is aligned with the collection frame, extracting images meeting the requirements of biological recognition detection in the video stream according to a specified frame rate to obtain an extracted image set, wherein the video stream refers to a video of the target object collected by a shooting device.

[0102] It should be noted that the local SDK (software development kit) is collected on the client terminal, the face detection SDK is encapsulated, the camera device is driven, the camera is opened to obtain real-time video stream, the face frame in the current image is obtained through the multi-task convolutional neural network, and it is judged whether the angle, distance and occlusion of the face to be recognized meet the requirements. If the detection is passed, it means that the user's face is ready, otherwise, according to the angle, distance and occlusion, the user is prompted to face the screen, approach (away) the screen, and remove the occlusion.

[0103] It should be noted that when the user aligns the face collection frame, the camera is opened to obtain real-time video stream, and the front face image meeting the face recognition detection requirements is extracted according to the frame rate. The above front face image contains the small changes of facial muscles and facial expression information of the face in the specified time sequence.

[0104] Step S202, obtaining a specified part front area of each image in the extracted image set, cutting the specified part area image to obtain a biological image sequence.

[0105] It should be noted that after obtaining the face image, the image of the area where the face is located is cut off, and the images that do not meet the image quality requirements are discarded to form a standardized image sequence, reducing the transmission of redundant information.

[0106] In the embodiment of the application, after cutting the specified part area image to obtain the biological image sequence, the method further comprises: compressing the biological image sequence by using a run-length encoding strategy; and encrypting the compressed biological image sequence by using a specified encryption algorithm. In this way, the amount of data transmission is reduced and the security of information transmission is improved.

[0107] It should be noted that the run-length encoding strategy and the encryption algorithm can be selected by the user. In this embodiment, the image sequence can be compressed by using run-length encoding (RLE) to reduce the amount of data transmission, and the data can be encrypted by using the asymmetric encryption algorithm SM2 to improve the security of information transmission.

[0108] Step S203, sending the biological image sequence to a server, wherein the server analyzes the expression label, the intention emotion and the biological feature comparison result in the biological image sequence after receiving the biological image sequence, and determines a biological security recognition result of the target object.

[0109] After encoding and encryption, the client transmits the encrypted biological image sequence to the server, and the server completes the biological feature recognition operation by receiving the encrypted biological image sequence.

[0110] In the embodiment of the application, when analyzing the expression label, the intention emotion and the biological feature comparison result in the biological image sequence, the server comprises: inputting the biological image sequence into a specified deep learning model to output an expression label corresponding to each biological image in the biological image sequence; inputting the expression label into a specified analysis model to output a weight corresponding to each expression label, determining the intention emotion of the target object by combining the weight of the expression label and the normal distribution information of all expression labels; and extracting a feature vector value corresponding to each biological image by using a biological feature extraction algorithm, comparing the feature vector value with the original registered feature to obtain a biological feature comparison result.

[0111] In the embodiment of the application, the front region of the specified part at least comprises a front region of a face.

[0112] Through the above embodiment, the client processes the face image and then uploads it to the server after encryption, and performs live detection through micro-expression and intention emotion, which not only can prevent attacks such as photo activation, but also can judge the similarity between the collected image and the person, analyze the intention emotion abnormality of the user during face recognition, improve the security and reliability of face recognition, and further solve the technical problem of insufficient security of biological feature recognition by action live detection in the related art.

[0113] The application will be described in more detail below with reference to another alternative embodiment.

[0114] Embodiment two

[0115] The embodiment provides a biological feature recognition system, each implementation module of the biological feature recognition system corresponds to each implementation step in the embodiment one.

[0116] Figure 3 is a structural block diagram of an alternative biological feature recognition system according to the embodiment of the application, as shown in Figure 3 The biological feature recognition system comprises:

[0117] The client is connected with the shooting device, and after receiving the video stream transmitted by the shooting device, executes any one of the biological feature recognition methods.

[0118] The server is connected with the client, and executes any one of the biological feature recognition methods, or the cloud server is connected with the client, and executes any one of the biological feature recognition methods.

[0119] The application will be described in more detail below with reference to another alternative embodiment.

[0120] The embodiment of the present application provides a biometric feature recognition system based on micro-expression and intention analysis, wherein the client contains functions such as face detection, data collection, image extraction and data compression encryption; the server (or cloud server) is connected with the client through a network, and contains functions such as micro-expression analysis, intention analysis processing, face comparison processing and recognition result processing.

[0121] The local SDK is collected in the client terminal, the face recognition security enhancement service is deployed in the server, and the client program is called. After the client initiates a face recognition request, the camera device is driven to collect user face image sequence information, then the server (or cloud server) is called to upload the processed and extracted image information to the server for face recognition judgment processing, and finally the recognition result returned by the server (or cloud server) is received, so that the whole face recognition process is completed, and the safety of the face recognition application is ensured.

[0122] The user initiates a face recognition request through the client, the client collects user face image sequence information through the control device camera after the face to be detected is ready, and finally uploads the processed and extracted image information to the server (or cloud server) for face recognition judgment. After receiving the image information data packet uploaded by the client, the server (or cloud server) analyzes and processes the micro-expression and intention emotion in the image information, and completes face feature extraction and comparison judgment, and finally returns the face recognition processing result, so as to improve the safety of online face recognition application, which specifically includes the following steps:

[0123] Step one: the user initiates a face recognition request, the client verifies the parameters of the device camera, and returns the corresponding error information directly if the verification fails, and enters the next step if the verification is passed.

[0124] By encapsulating the face detection SDK, driving the camera device, opening the camera to obtain real-time video stream, and obtaining the face frame in the current image through a multi-task convolutional neural network, it is judged whether the angle, distance and occlusion of the face to be recognized meet the requirements. If the detection is passed, it means that the user's face is ready, otherwise, according to the angle, distance and occlusion, the user is prompted to face the screen, approach (away) the screen and remove the occlusion.

[0125] Step two: the client calls the device camera to guide the user to align the face collection frame, and collects the face image according to the set collection frequency parameter after the face to be detected is ready, to form a face image sequence frame. The number of image frames can be set by parameterization.

[0126] Drive the camera device, open the camera to obtain real-time video stream, when the user aims at the face collection frame, extract the front face image meeting the face recognition detection requirements according to the frame rate, wherein the front face image contains the small changes of face muscle and facial expression information of the face in the specified time sequence.

[0127] The face key information in the face image is intercepted, the local binary pattern algorithm is used to locate the front face position of the collected face image, the image of the region where the face is located is intercepted, the images not meeting the image quality requirements are discarded, the standardized image sequence is formed, and the transmission of redundant information is reduced.

[0128] Step three, the client detects the picture quality of the collected face image sequence frame, processes to form a standardized image sequence, compresses and encrypts the processed image information, and then uploads it to the server.

[0129] The image sequence is compressed by run-length encoding to reduce the transmission data amount; the data is encrypted by the asymmetric encryption algorithm SM2 to improve the information transmission security.

[0130] Step four, after receiving the face image sequence data packet uploaded by the client, the server processes and analyzes the image information, judges the micro-expression action change, detects the abnormal situation of intention and emotion, and compares the similarity of face features, so as to complete the face recognition detection and judgment, and finally returns the recognition result to the client.

[0131] The server first analyzes and processes the micro-expression information and small changes of facial muscles in the input face image sequence, adopts a deep learning model for model training and construction, customizes different expression labels according to application needs, such as sadness, anger, surprise, fear, disgust, neutral and happiness, adopts a structure based on a multi-layer convolutional neural network, and adopts a normalized exponential function combined with an island loss function for loss function calculation, wherein:

[0132] The normalized loss function calculation formula is as follows:

[0133]

[0134] Wherein, m represents the number of training samples, n represents the number of expression label classifications, Z i represents the i-th element of the trained network output vector, Z i represents the j-th element of the trained network output vector, and L s is the normalized loss function value.

[0135] On the basis of the normalized loss function, there is also an island loss function, and the loss function calculation formula is as follows:

[0136]

[0137] wherein m represents the number of training samples, x i represents the feature vector of the i-th sample, C yi represents the feature center of the expression class y i , L IL represents the island loss function value, λ1 is a weight value, N is the expression label set, c j and c k represent the j-th and k-th expression label centers, +1 makes the function value range between 0 and 2, and the closer to 0 indicates the greater the class difference.

[0138] Therefore, the target loss function value calculation formula of the specified deep learning model is as follows:

[0139] L = L S + λL IL ;

[0140] wherein λ is a parameter for balancing the normalized loss function and the island loss function, L is the target loss function value of the specified deep learning model, L s is the normalized loss function value, and L IL represents the island loss function value.

[0141] Secondly, it is also necessary to analyze and process the intention emotion, analyze the intention emotion appearing in the input face image sequence, and judge whether the user face recognition scene is risky. On the basis of the generated micro-expression data, an analysis model is constructed based on the clustering algorithm Gaussian mixture model. This method contains multiple Gaussian distribution functions, and the final calculation result is obtained by weighted summation of each Gaussian probability density function. When constructing, different micro-expressions will have different degrees of influence in the model. By applying the Gaussian mixture model, the expectation maximization algorithm is used for training, so as to obtain the weight and model probability value of each micro-expression. At the same time, combined with the normal distribution principle, specifically, 68.2% of the points fall within ±1 standard deviation, about 95.5% of the points fall within ±2 standard deviations, and about 99.7% of the points fall within ±3 standard deviations. After obtaining the calculation result, the micro-expression image sequence collected when the user registers the face is taken as the benchmark for comparison, and the micro-expression image sequence of the user in the face recognition process is compared. If the measured feature data exceeds a set confidence level, it is judged that the psychological state of the to-be-detected object has an abnormal intention emotion.

[0142] Finally, the user's face features need to be analyzed and processed, the input face original image is extracted by a face feature extraction algorithm to obtain a face feature vector value, and the face feature vector value is compared with the original registered face feature to determine the similarity. A face frame in the current image is obtained through a multi-task convolutional neural network, and the multi-task neural network model mainly adopts three cascaded networks (using the idea of candidate frame plus classifier for fast and efficient face detection): P-Net for quickly generating a candidate window, R-Net for high-precision candidate window filtering and selection, and O-Net for generating a final boundary frame and face key points. The face in the current image is extracted through a face training model and mapped to a multi-dimensional space vector, and the output is a 128-dimensional vector space to obtain a face feature vector. The extracted feature vector is subjected to inner product operation with the face feature vector generated when the user registers, and when the similarity result of the two is greater than a set threshold, it is considered that the face in the live face photo and the face in the registered face photo are the same face, and the final face feature comparison similarity result is returned after calculation.

[0143] The micro-expression analysis, intent analysis and face similarity analysis results are fused, and the final recognition result is determined based on a preset confidence threshold, and the result includes whether the face is consistent with the user himself, whether the user has life characteristics, whether the user is in a safe face recognition scene, and finally the server returns the recognition result to the client.

[0144] Step five, the client receives the server processing result, prompts the user whether the final face recognition processing is passed, and completes the processing process.

[0145] In the embodiment of the application, three verifications of expression detection, intent emotion analysis and face feature comparison are included, live body detection is performed through micro-expression and intent emotion, not only can prevent attacks such as photo activation, but also can analyze the intent emotion abnormality of the user during face recognition, improve the safety and reliability of face recognition, and the biological feature recognition method in the disclosure does not depend on specific device hardware, can be applied to mobile phones and other client devices, does not need to increase the cost of users, has the basis for wide application, and in the face recognition process, the user only needs to cooperate with the face collection frame facing the device screen, the whole process is simple and convenient, without large-scale action cooperation, greatly improving the user experience.

[0146] The application will be described below in combination with another alternative embodiment.

[0147] Example Three

[0148] The biological feature recognition device provided in the embodiment includes each implementation unit corresponding to each implementation step in the first embodiment.

[0149] Figure 4 This is a schematic diagram of an optional biometric identification device applied to a server according to an embodiment of the present invention, such as... Figure 4 As shown, the biometric identification device includes: a receiving unit 41, an analysis unit 42, a determination unit 43, and an output unit 44, wherein,

[0150] The receiving unit 41 is used to receive biological image sequences uploaded by the client, wherein the biological image sequence refers to the image sequence extracted after video acquisition of the target object;

[0151] Analysis unit 42 is used to analyze facial expression tags, intention emotions, and biometric feature comparison results in biological image sequences;

[0152] The determining unit 43 is used to determine the recognition confidence based on at least one of the facial expression labels, intention emotions, and biometric comparison results;

[0153] Output unit 44 is used to output the abnormal notification result of biosafety identification of the target object to the client when the identification confidence level is greater than the preset confidence level threshold.

[0154] The aforementioned biometric identification device receives a biometric image sequence uploaded by a client via a receiving unit. This biometric image sequence refers to the image sequence extracted after video capture of the target object. An analysis unit analyzes the facial expression tags, intent / emotion, and biometric feature comparison results within the biometric image sequence. A determination unit 43 determines the identification confidence level based on at least one of the facial expression tags, intent / emotion, and biometric feature comparison results. An output unit 44, if the identification confidence level exceeds a preset confidence threshold, outputs an anomaly notification result to the client indicating that biometric identification of the target object is not performed. In this embodiment, a triple verification process—facial expression detection, intent / emotion analysis, and facial feature comparison—is implemented. Liveness detection using micro-expressions and intent / emotion not only defends against attacks such as photo animation but also analyzes abnormal intent / emotion during user facial recognition, improving the security and reliability of facial recognition. This addresses the technical problem of insufficient security in biometric identification based on motion liveness detection in related technologies.

[0155] Optionally, the analysis unit comprises: a first input module, configured to input the biological image sequence into a specified deep learning model, and output the expression label corresponding to each biological image in the biological image sequence; a first determination module, configured to input the expression label into a specified analysis model, and output the weight corresponding to each expression label; and a first extraction module, configured to extract the feature vector value corresponding to each biological image by using a biological feature extraction algorithm, and compare the feature vector value with the original registered feature to obtain the biological feature comparison result.

[0156] Optionally, the target loss function value used by the neural network in the specified deep learning model is calculated by using a normalized loss function and an island loss function, wherein the calculation elements of the normalized loss function include the number of training samples, the number of label categories, and the image elements of the output vector, the number of training samples is the total number of training samples formed by historical biological images and corresponding historical expression labels in a historical time period, the number of label categories is the total number of all to-be-classified expression labels, and the output vector is a vector corresponding to the output expression label of the neural network; and the calculation elements of the island loss function include the number of training samples, the sample feature vector, the expression label set, the expression label center, and a first preset weight value.

[0157] Optionally, the analysis unit further comprises a first training module, configured to train a mixture model comprising a plurality of Gaussian distribution functions by using an expectation maximization algorithm to obtain the specified analysis model.

[0158] Optionally, the first determination module comprises: a first determination submodule, configured to determine the intention confidence value corresponding to the biological image sequence in combination with the weight of the expression label and the normal distribution information of all expression labels; a first confirmation submodule, configured to confirm that the psychological state of the target object is abnormal and the intention emotion of the target object is abnormal if the intention confidence value is within a confidence interval of a specified abnormal emotion; and a second confirmation submodule, configured to confirm that the psychological state of the target object is normal and the intention emotion of the target object is normal if the intention confidence value is within a confidence interval of a specified normal emotion.

[0159] Optionally, the first extraction module comprises: a first extraction submodule, configured to extract a face frame in the biological image by using a specified convolutional neural network if the biological feature is a face feature; and a first mapping submodule, configured to map the face appearing in the biological image to a multidimensional space vector by using a specified face vector model with the face frame as a reference to obtain the feature vector value.

[0160] Optionally, the specified convolutional neural network adopts a multi-task convolutional neural network, and the multi-task convolutional neural network comprises three sub-networks: a first sub-network for generating a candidate window, a second sub-network for filtering the candidate window, and a third sub-network for generating a final bounding box and a face key point.

[0161] Figure 5 is a schematic diagram of an optional biometric recognition device applied to a client according to an embodiment of the present application. The biometric recognition device is applied to a client, as shown in Figure 5 The biometric recognition device comprises a detection unit 51, an acquisition unit 52, and a sending unit 53.

[0162] The detection unit 51 is configured to extract images meeting biometric recognition detection requirements in the video stream according to a specified frame rate when detecting that the target object is aligned with the collection frame, to obtain a set of extracted images, wherein the video stream refers to a video of the target object collected by a shooting device.

[0163] The acquisition unit 52 is configured to acquire a specified part front area of each image in the set of extracted images, to intercept a specified part area image, and to obtain a biological image sequence.

[0164] The sending unit 53 is configured to send the biological image sequence to a server, wherein the server analyzes an expression label, an intention emotion, and a biometric feature comparison result in the biological image sequence after receiving the biological image sequence, and determines a biometric security recognition result of the target object.

[0165] The biometric recognition device, through the detection unit 51, extracts images meeting biometric recognition detection requirements in the video stream according to a specified frame rate when detecting that the target object is aligned with the collection frame, to obtain a set of extracted images, wherein the video stream refers to a video of the target object collected by a shooting device. Through the acquisition unit 52, a specified part front area of each image in the set of extracted images is acquired, a specified part area image is intercepted, and a biological image sequence is obtained. Through the sending unit 53, the biological image sequence is sent to a server, wherein the server analyzes an expression label, an intention emotion, and a biometric feature comparison result in the biological image sequence after receiving the biological image sequence, and determines a biometric security recognition result of the target object. In this embodiment, micro-expression and intention emotion are used for live detection, which not only can prevent attacks such as photo activation, but also can analyze abnormal situations of intention emotion when a user performs face recognition, to improve the security and reliability of face recognition, and to solve the technical problem of insufficient security of biometric feature recognition through motion live detection in related technologies.

[0166] Optionally, the biometric recognition device further comprises: a first compression module configured to compress the sequence of biometric images using a run-length encoding strategy; and a first encryption module configured to encrypt the compressed sequence of biometric images using a specified encryption algorithm.

[0167] Optionally, the biometric recognition device further comprises: a second input module configured to input, by the server, the sequence of biometric images into a specified deep learning model to output an expression label corresponding to each biometric image in the sequence of biometric images; a second determination module configured to input, by the server, the expression label into a specified analysis model to output a weight corresponding to each expression label, and determine an intended emotion of the target object based on the weight of the expression label and normal distribution information of all expression labels; and a second extraction module configured to extract, by the server, a feature vector value corresponding to each biometric image using a biometric feature extraction algorithm, and compare the feature vector value with the original registered feature to obtain a biometric feature comparison result.

[0168] Optionally, the front region of the specified part at least includes a front region of a face.

[0169] The biometric recognition device described above can further comprise a processor and a memory, and the receiving unit 41, the analysis unit 42, the determination unit 43, the output unit 44, etc. are all stored in the memory as program units, and the corresponding functions are realized by the processor executing the program units stored in the memory.

[0170] The processor described above comprises a core, and the core retrieves the corresponding program units from the memory. The core can be one or more, and the biometric feature is recognized by adjusting the core parameters.

[0171] The memory described above can include a non-permanent memory in a computer readable medium, a random access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory comprises at least one memory chip.

[0172] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, which comprises a stored computer program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to perform any of the biometric recognition methods described above when the computer program is executed.

[0173] According to another aspect of the embodiments of the present application, an electronic device is also provided, which comprises one or more processors and a memory, and the memory is configured to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement any of the biometric recognition methods described above.

[0174] The application further provides a computer program product suitable for executing a program for initializing the following method steps when executed on a data processing device: receiving a biological image sequence uploaded by a client, wherein the biological image sequence refers to an image sequence extracted after video acquisition of a target object; analyzing an expression label, an intention emotion, and a biological feature comparison result in the biological image sequence; determining a recognition confidence based on at least one of the expression label, the intention emotion, and the biological feature comparison result; and outputting an abnormality notification result of biological safety recognition of the target object to the client when the recognition confidence is greater than a preset confidence threshold.

[0175] Figure 6 is a hardware structure block diagram of an electronic device (or a mobile device) according to an embodiment of a biological feature recognition method of the application. As shown in Figure 6 , the electronic device can include one or more (shown in the figure as 602a, 602b, …, 602n) processors 602 (the processor 602 can include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 604 for storing data. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply and / or a camera. Those skilled in the art can understand that Figure 6 the structure shown is only schematic, and does not limit the structure of the above-mentioned electronic device. For example, the electronic device can include more or fewer components than those shown in Figure 6 , or have a different configuration from Figure 6 .

[0176] The above-mentioned serial numbers of the embodiments of the application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0177] In the above-mentioned embodiments of the application, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0178] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other ways. Among them, the above-mentioned device embodiments are only schematic, for example, the division of the units can be a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between units or modules, which can be electrical or other forms.

[0179] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0180] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0181] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical scheme of the present application or the part of the present application which contributes to the prior art or the whole or part of the technical scheme can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes various media that can store program codes, such as a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0182] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A method for identifying biometric features, characterized in that, Applied to servers, including: Receive biological image sequences uploaded by the client, wherein the biological image sequence refers to the image sequence extracted after video capture of the target object; Analyzing the facial expression labels, intention / emotion, and biometric feature comparison results in the biological image sequence includes: inputting the biological image sequence into a specified deep learning model, outputting the facial expression label corresponding to each biological image in the biological image sequence; inputting the facial expression label into a specified analysis model, outputting the weight corresponding to each facial expression label, and combining the weights of the facial expression labels with the normal distribution information of all facial expression labels to determine the intention / emotion of the target object; using a biometric feature extraction algorithm to extract the feature vector value corresponding to each biological image, and comparing the feature vector value with the original registered features to obtain the biometric feature comparison result; The recognition confidence level is determined based on at least one of the expression tags, the intended emotion, and the biometric comparison results. If the identification confidence level is greater than a preset confidence threshold, the abnormal notification result of the biosafety identification of the target object will be output to the client.

2. The identification method according to claim 1, characterized in that, The target loss function value used by the neural network in the specified deep learning model is calculated using a normalized loss function and an island loss function, wherein... The normalized loss function uses the following computational elements: the number of training samples, the number of label classifications, and the image elements of the output vector. The number of training samples is the total number of training samples formed by historical biological images and corresponding historical expression labels in the historical time period. The number of label classifications is the total number of all expression labels to be classified. The output vector is the vector corresponding to the output expression label of the neural network. The island loss function uses the following computational elements: the number of training samples, the sample feature vector, the set of expression labels, the expression label center, and the first preset weight value.

3. The identification method according to claim 1, characterized in that, Before inputting the emoji tags into the specified analysis model and outputting the weights corresponding to each emoji tag, the process also includes: The specified analysis model is obtained by training a mixture model containing multiple Gaussian distribution functions using the expectation-maximization algorithm.

4. The identification method according to claim 1, characterized in that, The step of determining the intended emotion of the target object by combining the weights of the emoji tags and the normal distribution information of all emoji tags includes: By combining the weights of the facial expression tags and the normal distribution information of all facial expression tags, the intent confidence value corresponding to the biological image sequence is determined; If the confidence value of the intent is within the confidence interval of the specified abnormal emotion, then it is confirmed that the psychological state of the target object is abnormal and the intent emotion of the target object is abnormal. If the confidence value of the intention is within the specified confidence interval of normal emotion, then it is confirmed that the psychological state of the target object is normal and the intention and emotion of the target object are normal.

5. The identification method according to claim 1, characterized in that, The step of extracting the feature vector value corresponding to each biological image using a biometric feature extraction algorithm includes: When the biometric feature is a facial feature, a specified convolutional neural network is used to extract the facial bounding box in each biometric image; Using the face bounding box as a reference, a specified face vector model is used to map the faces appearing in each biological image to a multi-dimensional space vector to obtain the feature vector value.

6. The identification method according to claim 5, characterized in that, The specified convolutional neural network is a multi-task convolutional neural network, which includes three sub-networks: a first sub-network for generating candidate windows, a second sub-network for filtering candidate windows, and a third sub-network for generating the final bounding box and facial key points.

7. A method for identifying biometric features, characterized in that, Applied to the client side, including: When the target object is detected to be aligned with the acquisition frame, images that meet the biometric detection requirements are extracted from the video stream according to the specified frame rate to obtain an extracted image set. The video stream refers to the video of the target object acquired by the shooting device. Obtain the frontal region of a specified part of each image in the extracted image set, crop the image of the specified region, and obtain a biological image sequence; The biological image sequence is sent to the server, wherein, after receiving the biological image sequence, the server analyzes the facial expression tags, intention emotions, and biometric feature comparison results in the biological image sequence, and determines the biometric security identification result of the target object; The server analyzes the facial expression tags, intention emotions, and biometric feature comparison results in the biological image sequence, including: the server inputs the biological image sequence into a specified deep learning model and outputs the facial expression tag corresponding to each biological image in the biological image sequence; the server inputs the facial expression tag into a specified analysis model and outputs the weight corresponding to each facial expression tag, and determines the intention emotion of the target object by combining the weight of the facial expression tag and the normal distribution information of all facial expression tags; the server uses a biometric feature extraction algorithm to extract the feature vector value corresponding to each biological image, and compares the feature vector value with the original registered features to obtain the biometric feature comparison result.

8. The identification method according to claim 7, characterized in that, After extracting an image of a specified region to obtain a biological image sequence, the process also includes: The biological image sequence is compressed using a run-length encoding strategy; The compressed biological image sequence is encrypted using a specified encryption algorithm.

9. The identification method according to any one of claims 7 and 8, characterized in that, The designated front area includes at least the front area of ​​a human face.

10. A biometric identification system, characterized in that, include: The client is connected to the shooting device and, after receiving the video stream transmitted by the shooting device, executes the biometric identification method according to any one of claims 7 to 9; The server connects to the client and executes the biometric identification method according to any one of claims 1 to 6.

11. A biometric identification device, characterized in that, Applied to servers, including: A receiving unit is used to receive a biological image sequence, wherein the biological image sequence refers to an image sequence extracted after video acquisition of a target object; The analysis unit is used to analyze the facial expression tags, intention emotions, and biometric feature comparison results in the biological image sequence; The analysis unit includes: a first input module, used to input the biological image sequence into a specified deep learning model and output the expression label corresponding to each biological image in the biological image sequence; a first determination module, used to input the expression label into a specified analysis model, output the weight corresponding to each expression label, and determine the intention emotion of the target object by combining the weight of the expression label and the normal distribution information of all expression labels; and a first extraction module, used to extract the feature vector value corresponding to each biological image using a biometric feature extraction algorithm, and compare the feature vector value with the original registered features to obtain the biometric feature comparison result. The determining unit is configured to determine the recognition confidence level based on at least one of the expression tag, the intention emotion, and the biometric comparison result; The output unit is used to output the abnormal notification result of the biosafety identification of the target object to the client when the identification confidence level is greater than a preset confidence threshold.

12. A biometric identification device, characterized in that, Applied to the client side, including: The detection unit is used to extract images that meet the biometric detection requirements from the video stream at a specified frame rate when the target object is detected to be aligned with the acquisition frame, thereby obtaining an extracted image set. The video stream refers to the video of the target object acquired by the shooting device. The acquisition unit is used to acquire the frontal region of a specified part of each image in the extracted image set, crop the image of the specified region, and obtain a biological image sequence. A sending unit is configured to send the biological image sequence to a server, wherein, after receiving the biological image sequence, the server analyzes the facial expression tags, intention emotions, and biometric feature comparison results in the biological image sequence, and determines the biometric security identification result of the target object; The biometric identification device further includes: a second input module, used by the server to input the biometric image sequence into a specified deep learning model and output the expression label corresponding to each biometric image in the biometric image sequence; a second determination module, used by the server to input the expression label into a specified analysis model and output the weight corresponding to each expression label, and combine the weight of the expression label and the normal distribution information of all expression labels to determine the intention emotion of the target object; and a second extraction module, used by the server to extract the feature vector value corresponding to each biometric image using a biometric extraction algorithm, and compare the feature vector value with the original registered features to obtain the biometric comparison result.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the biometric identification method according to any one of claims 1 to 9.

14. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the biometric identification method according to any one of claims 1 to 9.

Images