Identity verification method, device, equipment and storage medium based on face recognition

Abstract

The identity authentication method, apparatus, device and storage medium based on face recognition provided in the present application relate to the field of face recognition technology. In the present application, first, the face image data to be identified is obtained by collecting face information of the object to be identified; secondly, feature mining and feature enhancement are performed on the face image data to be identified, and a first face feature vector corresponding to the face image data to be identified is output; then, feature mining and feature enhancement are performed on the target face image data corresponding to the pre-stored target object, and a second face feature vector corresponding to the target face image data is output; finally, a comparative analysis is performed based on the first face feature vector and the second face feature vector, and a target comparative analysis result is output, and a target identity authentication result is determined based on the target comparative analysis result. Based on the above content, the problem of relatively low reliability of identity authentication in the prior art can be improved.

Description

Technical Field

[0001] The present application relates to the field of face recognition technology, and more specifically, to an identity authentication method and apparatus, device, and storage medium based on face recognition. Background Art

[0002] Facial recognition technology, as a biometric identification technique, has been widely used in recent years, particularly in the fields of identity verification and security. Facial recognition is a technique that verifies and authenticates a person's identity by analyzing the characteristic features in facial images or videos. This technology, which combines advances in various fields such as computer vision, machine learning, and image processing, has gradually become a core technology in fields such as intelligent security and smart cities. For example, in public places such as airports, train stations, and subways, facial recognition technology is used for identity verification, replacing traditional manual security checks and ID verification. This not only improves efficiency but also reduces the errors and safety risks associated with manual operations. However, the inventors have discovered that existing facial recognition-based identity verification technologies suffer from relatively low reliability. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide an identity authentication method and apparatus, device and storage medium based on face recognition to improve.

[0004] To achieve the above objectives, this application adopts the following technical solutions:

[0005] An identity authentication method based on face recognition, comprising:

[0006] Acquire facial image data to be identified formed by collecting facial information of a subject to be identified, wherein the facial image data to be identified includes at least one frame of facial image to be identified;

[0007] Performing feature mining and feature enhancement on the facial image data to be identified using a feature mining model included in a target facial recognition network, and outputting a first facial feature vector corresponding to the facial image data to be identified, wherein the target facial recognition network is a trained neural network;

[0008] Using the feature mining model, performing feature mining and feature enhancement on pre-stored target facial image data corresponding to the target object, and outputting a second facial feature vector corresponding to the target facial image data, wherein the target facial image data includes at least one frame of target facial image;

[0009] Utilizing the feature comparison model included in the target face recognition network, a comparative analysis is performed based on the first face feature vector and the second face feature vector, a target comparative analysis result is output, and a target identity authentication result is determined based on the target comparative analysis result, wherein the target comparative analysis result is used to reflect whether the object to be identified belongs to the target object, and the target identity authentication result is used to reflect whether the object to be identified has passed the identity authentication.

[0010] In a preferred embodiment of the present application, in the above-mentioned identity authentication method based on face recognition, the step of utilizing the feature mining model included in the target face recognition network to perform feature mining and feature enhancement on the face image data to be identified, and outputting the first face feature vector corresponding to the face image data to be identified, includes:

[0011] Using a convolution unit in a feature mining model included in a target face recognition network, respectively perform convolution processing on each frame of the face image to be recognized in the face image data to be recognized, and output a first face convolution vector corresponding to each frame of the face image to be recognized, wherein the face image data to be recognized includes multiple frames of face images to be recognized;

[0012] Utilizing the association mining unit in the feature mining model, performing association mining on a plurality of first face convolution vectors corresponding to a plurality of frames of face images to be recognized included in the face image data to be recognized, according to association relationships between the plurality of frames of face images to be recognized, and outputting a first face association vector corresponding to each of the first face convolution vectors;

[0013] Superimposing the multiple first face association vectors corresponding to the multiple first face convolution vectors, and outputting a first face aggregation vector corresponding to the face image data to be recognized;

[0014] performing a plurality of different noise processing on the first face aggregation vectors respectively, and outputting a plurality of first face noise vectors corresponding to the face image data to be recognized;

[0015] The multiple first face noise vectors are spliced ​​to form a first face feature vector corresponding to the face image data to be recognized.

[0016] In a preferred embodiment of the present application, in the above-mentioned identity authentication method based on face recognition, the step of using the association mining unit in the feature mining model to perform association mining on the multiple first face convolution vectors corresponding to the multiple frames of face images to be identified included in the face image data to be identified according to the association relationship between the multiple frames of face images to be identified, and outputting the first face association vector corresponding to each of the first face convolution vectors includes:

[0017] For each frame of the facial image to be recognized in the multiple frames of facial images to be recognized included in the facial image data to be recognized, segmenting the first facial convolution vector corresponding to the facial image to be recognized to form multiple first facial local vectors corresponding to the facial image to be recognized;

[0018] performing association mining on a plurality of first face local vectors corresponding to a plurality of frames of face images to be recognized included in the face image data according to association relationships between the plurality of frames of face images to be recognized using an association mining unit in the feature mining model, and outputting an associated first face local vector corresponding to each of the first face local vectors;

[0019] For each frame of the facial image to be recognized in the multiple frames of facial images to be recognized included in the facial image data to be recognized, combining multiple associated first facial local vectors corresponding to the multiple first facial local vectors corresponding to the facial image to be recognized to form a first facial combination vector corresponding to the facial image to be recognized, wherein the combining process and the segmentation process are mutually inverse processes;

[0020] For each frame of the face image to be identified, based on the first face combination vector corresponding to the face image to be identified, the first face convolution vector corresponding to the face image to be identified is subjected to focused feature mining processing to form a first face association vector corresponding to the first face convolution vector.

[0021] In a preferred embodiment of the present application, in the above-mentioned identity authentication method based on face recognition, the step of using the association mining unit in the feature mining model to perform association mining on multiple first face local vectors corresponding to multiple frames of face images to be identified included in the face image data to be identified according to the association relationship between the multiple frames of face images to be identified, and outputting the associated first face local vector corresponding to each of the first face local vectors, includes:

[0022] Arranging the plurality of first face local vectors corresponding to each frame of the face image to be recognized in the plurality of frames of the face image to be recognized included in the face image data to be recognized according to a sequence relationship between their distribution positions in the corresponding first face convolution vectors and the corresponding face images to be recognized, to form a plurality of first face local vector sequences, wherein any two first face local vectors in the same first face local vector have the same distribution positions in the corresponding first face convolution vector;

[0023] For each of the first partial face vectors, association mining is performed on the first partial face vector according to the first partial face vector sequence in which the first partial face vector is located, and an associated first partial face vector corresponding to the first partial face vector is output.

[0024] In a preferred embodiment of the present application, in the above-mentioned identity authentication method based on face recognition, the step of performing association mining on each first face local vector based on the first face local vector sequence in which the first face local vector is located, and outputting the associated first face local vector corresponding to the first face local vector, includes:

[0025] For a first first face partial vector in the first face partial vector sequence, taking the first face partial vector as the corresponding associated first face partial vector;

[0026] For each other first human face local vector other than the first first human face local vector in the first human face local vector sequence, based on the associated first human face local vector corresponding to the previous first human face local vector, the other first human face local vector is subjected to focused feature mining processing to form the associated first human face local vector corresponding to the other first human face local vector.

[0027] In a preferred embodiment of the present application, in the above-mentioned identity authentication method based on face recognition, the step of applying a plurality of different noise processing to the first face aggregation vector and outputting a plurality of first face noise vectors corresponding to the face image data to be identified includes:

[0028] generating, based on the vector size of the first face aggregation vector, a plurality of random noise vectors having the same vector size, wherein, in each of the random noise vectors, the number of non-zero vector parameters is less than the number of zero vector parameters;

[0029] Each of the multiple random noise vectors is superimposed with the first face aggregation vector to form multiple first face noise vectors corresponding to the face image data to be identified; or, the multiple random noise vectors are sorted, and the random noise vector sorted first is superimposed with the first face aggregation vector to form a first face noise vector corresponding to the random noise vector sorted first, and, for each other random noise vector other than the random noise vector sorted first, self-attention processing is performed on the first face noise vector corresponding to the previous random noise vector of the other random noise vector to obtain the corresponding self-attention vector, and, the self-attention vector and the other random noise vector are superimposed to form the first face noise vector corresponding to the other random noise vector.

[0030] In a preferred embodiment of the present application, in the above-mentioned identity authentication method based on face recognition, the step of using the feature mining model to perform feature mining and feature enhancement on the pre-stored target face image data corresponding to the target object, and outputting a second face feature vector corresponding to the target face image data, includes:

[0031] Using the convolution unit in the feature mining model, convolution processing is performed on each frame of the target facial image data, and a second facial convolution vector corresponding to each frame of the target facial image is output, wherein the target facial image data includes multiple frames of target facial images;

[0032] Utilizing the association mining unit in the feature mining model, performing association mining on a plurality of second face convolution vectors corresponding to the plurality of target face images included in the target face image data according to association relationships between the plurality of target face images, and outputting a second face association vector corresponding to each of the second face convolution vectors;

[0033] Superimposing multiple second face association vectors corresponding to the multiple second face convolution vectors, and outputting a second face aggregation vector corresponding to the target face image data;

[0034] performing a plurality of different noise processing on the second face aggregation vectors respectively, and outputting a plurality of second face noise vectors corresponding to the target face image data;

[0035] The multiple second facial noise vectors are spliced ​​to form a second facial feature vector corresponding to the target facial image data.

[0036] This application also provides an identity verification device based on face recognition, comprising:

[0037] A face image acquisition module is used to acquire face image data to be identified formed by collecting face information of a subject to be identified, wherein the face image data to be identified includes at least one frame of face image to be identified;

[0038] a first feature mining module, configured to perform feature mining and feature enhancement on the facial image data to be identified using a feature mining model included in a target facial recognition network, and output a first facial feature vector corresponding to the facial image data to be identified, wherein the target facial recognition network is a trained neural network;

[0039] a second feature mining module, configured to perform feature mining and feature enhancement on pre-stored target facial image data corresponding to the target object using the feature mining model, and output a second facial feature vector corresponding to the target facial image data, wherein the target facial image data includes at least one frame of target facial image;

[0040] A verification result determination module is used to utilize the feature comparison model included in the target face recognition network to perform a comparative analysis based on the first face feature vector and the second face feature vector, output a target comparative analysis result, and determine a target identity authentication result based on the target comparative analysis result, wherein the target comparative analysis result is used to reflect whether the object to be identified belongs to the target object, and the target identity authentication result is used to reflect whether the object to be identified has passed the identity authentication.

[0041] Based on the above, the present application further provides an electronic device, including:

[0042] memory for storing computer programs;

[0043] A processor connected to the memory is used to execute the computer program stored in the memory to implement the above-mentioned identity authentication method based on face recognition.

[0044] Based on the above, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is run, it executes the various steps of the above-mentioned face recognition-based identity authentication method.

[0045] The identity authentication method, apparatus, device and storage medium based on face recognition provided by the present application first obtain the face image data to be identified formed by collecting face information of the object to be identified; secondly, perform feature mining and feature enhancement on the face image data to be identified, and output a first face feature vector corresponding to the face image data to be identified; then, perform feature mining and feature enhancement on the target face image data corresponding to the pre-stored target object, and output a second face feature vector corresponding to the target face image data; finally, perform comparative analysis based on the first face feature vector and the second face feature vector, output the target comparative analysis result, and determine the target identity authentication result based on the target comparative analysis result. Based on the above content, since the face image data to be identified and the target face image data will be feature mined and feature enhanced respectively, the obtained first face feature vector and second face feature vector both have better face semantic representation capabilities, and therefore, the reliability of the target identity authentication result obtained based on the first face feature vector and the second face feature vector can be guaranteed, thereby improving the problem of relatively low reliability of identity authentication in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings.

[0047] Figure 1This is a structural block diagram of the electronic device provided in an embodiment of the present application.

[0048] Figure 2 A flowchart of the face recognition-based identity authentication method provided in an embodiment of the present application.

[0049] Figure 3 A schematic diagram of the noise application process provided in an embodiment of the present application.

[0050] Figure 4 A block diagram of an identity verification device based on face recognition provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0052] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0053] like Figure 1 As shown, an embodiment of the present application provides an electronic device, wherein the electronic device may include a memory, a processor, and an identity verification device based on face recognition.

[0054] In detail, the memory and the processor are electrically connected directly or indirectly to realize data transmission or interaction. For example, the memory and the processor can be electrically connected through one or more communication buses or signal lines. The face recognition-based identity authentication device includes at least one software function module stored in the memory in the form of software or firmware. The processor is used to execute the executable computer program stored in the memory, for example, the software function module and computer program included in the face recognition-based identity authentication device, so as to realize the face recognition-based identity authentication method provided in the embodiment of the present application.

[0055] Optionally, the memory may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0056] Furthermore, the processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0057] I understand. Figure 1 The structure shown is for illustration only. The electronic device may also include Figure 1 More or fewer components than shown, or with Figure 1 The different configurations shown, for example, may further include a communication unit for exchanging information with other devices (such as an image acquisition device).

[0058] Combine Figure 2 The embodiment of the present application further provides a face recognition-based identity authentication method applicable to the above-mentioned electronic device. The method steps defined in the process related to the face recognition-based identity authentication method can be implemented by the electronic device.

[0059] The following will Figure 2 The specific process shown is explained in detail.

[0060] Step S110: acquiring facial image data of an object to be identified formed by collecting facial information of the object to be identified.

[0061] In an embodiment of the present application, the electronic device can obtain facial image data to be identified by collecting facial information of the object to be identified, such as from a connected image acquisition device. The facial image data to be identified includes at least one frame of facial image to be identified, that is, it can include one frame of facial image to be identified, or it can include multiple frames of continuous facial images to be identified. It should be noted that the facial image to be identified can be either the original image captured by the image acquisition device or a facial image formed by further segmentation based on the original image.

[0062] Step S120 , utilizing the feature mining model included in the target face recognition network to perform feature mining and feature enhancement on the face image data to be recognized, and outputting a first face feature vector corresponding to the face image data to be recognized.

[0063] In an embodiment of the present application, after obtaining the facial image data to be recognized, the electronic device may utilize a feature mining model included in a target facial recognition network to perform feature mining and feature enhancement on the facial image data to be recognized, and output a first facial feature vector corresponding to the facial image data to be recognized. The target facial recognition network is a trained neural network, i.e., trained using sample facial image data and corresponding data labels.

[0064] Step S130 , utilizing the feature mining model to perform feature mining and feature enhancement on the pre-stored target facial image data corresponding to the target object, and outputting a second facial feature vector corresponding to the target facial image data.

[0065] In an embodiment of the present application, the electronic device can utilize the feature mining model to perform feature mining and feature enhancement on the target facial image data corresponding to the pre-stored target object, and output a second facial feature vector corresponding to the target facial image data. Wherein, the target facial image data includes at least one frame of target facial image, that is, it can include one frame of target facial image, or it can include multiple frames of continuous target facial images. It should be noted that the target facial image can be either the original image captured by the image acquisition device, or it can be a facial image formed by further segmentation based on the original image. In addition, there can be multiple target objects, so that feature mining and feature enhancement can be performed on the target facial image data corresponding to each target object respectively.

[0066] Step S140, using the feature comparison model included in the target face recognition network, performing a comparative analysis based on the first face feature vector and the second face feature vector, outputting a target comparative analysis result, and determining a target identity verification result based on the target comparative analysis result.

[0067] In an embodiment of the present application, after obtaining the first facial feature vector and the second facial feature vector, the electronic device can use the feature comparison model included in the target facial recognition network to perform a comparative analysis based on the first facial feature vector and the second facial feature vector, output a target comparative analysis result, and determine a target identity verification result based on the target comparative analysis result. The target comparative analysis result is used to reflect whether the object to be identified belongs to the target object, and the target identity verification result is used to reflect whether the object to be identified has passed the identity verification. For example, when the target comparative analysis result reflects that the object to be identified belongs to the target object, the target identity verification result is used to reflect that the object to be identified has passed the identity verification; when the target comparative analysis result reflects that the object to be identified does not belong to the target object, the target identity verification result is used to reflect that the object to be identified has failed the identity verification. In addition, in some embodiments, the similarity (such as cosine similarity) between the first facial feature vector and the second facial feature vector can be calculated. In this way, when the similarity is greater than a preset similarity (such as 0.8, 0.9, etc.), it can be determined that the object to be identified belongs to the target object; when the similarity is not greater than the preset similarity, it can be determined that the object to be identified does not belong to the target object. Alternatively, in other embodiments, the first facial feature vector and the second facial feature vector can be spliced ​​together, and then the spliced ​​vector can be fully connected. Furthermore, the fully connected vector can be classified and output (such as through classification functions such as softmax to predict the probability of belonging to the target object) to obtain the corresponding target comparison analysis results.

[0068] Based on the above content, since the facial image data to be identified and the target facial image data will be feature mined and enhanced respectively, the obtained first facial feature vector and second facial feature vector will have better facial semantic representation capabilities. Therefore, the reliability of the target identity authentication results obtained based on the first facial feature vector and the second facial feature vector can be guaranteed, thereby improving the problem of relatively low reliability of identity authentication in the existing technology.

[0069] It should be noted that for step S120 , the specific method of performing feature mining and feature enhancement on the facial image data to be identified is not limited and can be selected according to actual needs.

[0070] For example, in an alternative embodiment, in order to ensure that the output first facial feature vector has better semantic representation ability, the above-mentioned step S120 can further include step S121, step S122, step S123, step S124 and step S125, and the contents of each step are described as follows.

[0071] Step S121, using the convolution unit in the feature mining model included in the target face recognition network, convolution processing is performed on each frame of the face image to be recognized in the face image data to be recognized, and a first face convolution vector corresponding to each frame of the face image to be recognized is output.

[0072] In an embodiment of the present application, a convolution unit in a feature mining model included in a target face recognition network can be used to perform convolution processing on each frame of the face image data to be recognized, and output a first face convolution vector corresponding to each frame of the face image data to be recognized. Where the face image data to be recognized includes multiple frames of face images to be recognized, multiple corresponding first face convolution vectors can be obtained.

[0073] Step S122, using the association mining unit in the feature mining model, the multiple first face convolution vectors corresponding to the multiple frames of face images to be identified included in the face image data to be identified are associated mined according to the association relationship between the multiple frames of face images to be identified, and the first face association vector corresponding to each of the first face convolution vectors is output.

[0074] In an embodiment of the present application, after obtaining multiple first face convolution vectors, the association mining unit in the feature mining model can be used to perform association mining on the multiple first face convolution vectors corresponding to the multiple frames of face images to be identified included in the face image data to be identified, according to the association relationship between the multiple frames of face images to be identified (such as the order of collection, etc.), and output the first face association vector corresponding to each of the first face convolution vectors. In this way, the semantic information carried by the first face association vector can be richer.

[0075] Step S123: Superimpose the multiple first face association vectors corresponding to the multiple first face convolution vectors to output a first face aggregation vector corresponding to the face image data to be recognized.

[0076] In an embodiment of the present application, multiple first face association vectors corresponding to the multiple first face convolution vectors can be superimposed to output the first face aggregation vector corresponding to the face image data to be identified. Exemplarily, mean superposition can be performed.

[0077] Step S124 , performing a plurality of different noise application processes on the first face aggregation vectors, and outputting a plurality of first face noise vectors corresponding to the face image data to be recognized.

[0078] In this embodiment of the present application, the first face aggregation vector can be subjected to various noise processing techniques, thereby outputting multiple first face noise vectors corresponding to the facial image data to be recognized, such that the multiple first face noise vectors are mutually distinct. In this way, by applying noise, more realistic facial information can be simulated to a certain extent (some information may be lost during image acquisition).

[0079] Step S125 , concatenating the multiple first facial noise vectors to form a first facial feature vector corresponding to the facial image data to be recognized.

[0080] In an embodiment of the present application, after forming multiple first face noise vectors, the multiple first face noise vectors can be spliced ​​to form a first face feature vector corresponding to the face image data to be identified; or, after splicing, pooling processing can be performed to obtain the first face feature vector corresponding to the face image data to be identified.

[0081] It can be understood that in the above-mentioned step S122, the specific method of performing association mining on the multiple first face convolution vectors corresponding to the multiple frames of face images to be identified included in the face image data to be identified according to the association relationship between the multiple frames of face images to be identified is not limited and can be selected according to actual needs. For example, in an alternative embodiment, in order to improve the reliability of association mining, the above-mentioned step S122 can further include step S122a, step S122b, step S122c and step S122d, and the specific content of each step is described as follows.

[0082] Step S122a: For each frame of facial images to be recognized in the multiple frames of facial images to be recognized included in the facial image data to be recognized, segment the first facial convolution vector corresponding to the facial image to be recognized to form multiple first facial local vectors corresponding to the facial image to be recognized.

[0083] In an embodiment of the present application, for each frame of facial images to be recognized in the multiple frames of facial images to be recognized included in the facial image data to be recognized, the first facial convolution vector corresponding to the facial image to be recognized can be segmented to form multiple first facial local vectors corresponding to the facial image to be recognized; illustratively, the sizes of the first facial local vectors can be the same, and in addition, the first facial local vectors can be combined together to form a corresponding first facial convolution vector. For example, the size of the first facial convolution vector is n*m, and it can be segmented to form 4 (n / 2)*(m / 2) first facial local vectors.

[0084] Step S122b, using the association mining unit in the feature mining model, performs association mining on the multiple first face local vectors corresponding to the multiple frames of face images to be identified included in the face image data to be identified according to the association relationship between the multiple frames of face images to be identified, and outputs the associated first face local vector corresponding to each of the first face local vectors.

[0085] In an embodiment of the present application, after the first local face vector is segmented and formed, the association mining unit in the feature mining model can be used to perform association mining on the multiple first local face vectors corresponding to the multiple frames of face images to be identified included in the face image data to be identified, according to the association relationship between the multiple frames of face images to be identified, and output the associated first local face vector corresponding to each of the first local face vectors. In this way, the granularity of the association mining can be updated, thereby ensuring that the reliability of the association mining can be higher.

[0086] Step S122c, for each frame of facial images to be identified in the multiple frames of facial images to be identified included in the facial image data to be identified, multiple associated first facial local vectors corresponding to the multiple first facial local vectors corresponding to the facial image to be identified are combined to form a first facial combination vector corresponding to the facial image to be identified.

[0087] In this embodiment of the present application, for each of the multiple frames of facial images to be recognized included in the facial image data to be recognized, multiple associated first facial local vectors corresponding to the multiple first facial local vectors corresponding to the facial image to be recognized are combined to form a first facial combination vector corresponding to the facial image to be recognized. The combination process is an inverse process to the segmentation process.

[0088] Step S122d: For each frame of the face image to be identified, based on the first face combination vector corresponding to the face image to be identified, focus feature mining processing is performed on the first face convolution vector corresponding to the face image to be identified to form a first face association vector corresponding to the first face convolution vector.

[0089] In an embodiment of the present application, after forming the first face combination vector, for each frame of the face image to be identified, focused feature mining processing can be performed on the first face convolution vector corresponding to the face image to be identified based on the first face combination vector corresponding to the face image to be identified, thereby forming a first face association vector corresponding to the first face convolution vector. Exemplarily, the first face combination vector can be multiplied by the transposed vector of the first face convolution vector to form a corresponding vector association parameter, and the first face convolution vector can be weighted based on the vector association parameter to obtain the first face association vector corresponding to the first face convolution vector.

[0090] It can be understood that in the above-mentioned step S122b, the specific method of performing association mining on the multiple first facial local vectors corresponding to the multiple frames of facial images to be identified included in the facial image data to be identified according to the association relationship between the multiple frames of facial images to be identified is not limited and can be selected according to actual needs. For example, in an alternative embodiment, in order to improve the reliability of the associated first facial local vectors obtained by association mining, the above-mentioned step S122b can further include step b1 and step b2, and the specific content of each step is described as follows.

[0091] Step b1, arrange the multiple first face local vectors corresponding to each frame of the face image to be identified in the multiple frames of the face image to be identified included in the face image data to be identified according to the distribution position in the corresponding first face convolution vector and the sequence relationship between the corresponding face image to be identified, to form multiple first face local vector sequences.

[0092] In an embodiment of the present application, multiple first facial local vectors corresponding to each of the multiple facial images to be recognized included in the facial image data to be recognized can be arranged according to the order of their distribution positions in the corresponding first facial convolution vectors and the corresponding facial images to be recognized, forming a sequence of multiple first facial local vectors, such as {a first facial local vector corresponding to the first image to be recognized, a first facial local vector corresponding to the second image to be recognized, a first facial local vector corresponding to the third image to be recognized, ...}. Any two first facial local vectors within the same first facial local vector have the same distribution position in the corresponding first facial convolution vector.

[0093] Step b2: for each of the first partial face vectors, perform association mining on the first partial face vector according to the first partial face vector sequence in which the first partial face vector is located, and output an associated first partial face vector corresponding to the first partial face vector.

[0094] In an embodiment of the present application, for each of the first facial local vectors, association mining can be performed on the first facial local vector based on the first facial local vector sequence in which the first facial local vector is located, and the associated first facial local vector corresponding to the first facial local vector is output, that is, association mining is performed on the first facial local vectors with the same distribution position, so that semantic information can be fused to obtain associated first facial local vectors with richer semantic information.

[0095] It is understandable that, in the above step b2, the specific method of performing association mining on the first face local vector based on the first face local vector sequence in which the first face local vector is located is not limited and can be selected according to actual needs. For example, in an alternative embodiment, in order to improve the reliability of association mining, the above step b2 may further include the following:

[0096] First, for the first first face local vector in the first face local vector sequence, the first face local vector may be used as the corresponding associated first face local vector;

[0097] Secondly, for each other first face local vector other than the first first face local vector in the first face local vector sequence, based on the associated first face local vector corresponding to the previous first face local vector, the other first face local vector is subjected to focused feature mining processing to form an associated first face local vector corresponding to the other first face local vector, that is, the semantic information carried by the associated first face local vector corresponding to the previous first face local vector is integrated into the other first face local vector to obtain the associated first face local vector.

[0098] It is understandable that, in the above step S124, the specific manner of performing a plurality of different noise processing on the first face aggregation vector is not limited and can be selected according to actual needs. For example, in an alternative embodiment, the above step S124 may include:

[0099] First, based on the vector size of the first face aggregation vector, a plurality of random noise vectors having the same vector size may be generated, wherein in each of the random noise vectors, the number of non-zero vector parameters is less than the number of zero vector parameters;

[0100] Secondly, each of the plurality of random noise vectors is superimposed with the first face aggregation vector to form a plurality of first face noise vectors corresponding to the face image data to be identified; or Figure 3, sorting the multiple random noise vectors, and superimposing the random noise vector ranked first with the first face aggregation vector to form a first face noise vector corresponding to the random noise vector ranked first, and, for each other random noise vector other than the random noise vector ranked first, performing self-attention processing on the first face noise vector corresponding to the previous random noise vector of the other random noise vector to obtain the corresponding self-attention vector, and, superimposing the self-attention vector and the other random noise vector to form the first face noise vector corresponding to the other random noise vector; exemplarily, the aforementioned superposition processing can be mean superposition.

[0101] It should be noted that for step S130 , the specific method of performing feature mining and feature enhancement on the target facial image data corresponding to the target object is not limited and can be selected according to actual needs.

[0102] For example, in an alternative embodiment, to ensure that the second facial feature vector corresponding to the output target facial image data has better semantic representation capabilities, the above-mentioned step S130 may further include the following contents (refer to the relevant description of step S120 above):

[0103] First, the convolution unit in the feature mining model can be used to perform convolution processing on each frame of the target facial image data, and output a second facial convolution vector corresponding to each frame of the target facial image, wherein the target facial image data includes multiple frames of target facial images, as described above.

[0104] Secondly, utilizing the association mining unit in the feature mining model, association mining is performed on a plurality of second face convolution vectors corresponding to the multiple frames of target face images included in the target face image data according to the association relationship between the multiple frames of target face images, and a second face association vector corresponding to each of the second face convolution vectors is output, as described above.

[0105] Then, a plurality of second face association vectors corresponding to the plurality of second face convolution vectors are superimposed to output a second face aggregation vector corresponding to the target face image data;

[0106] Then, performing a plurality of different noise processing on the second face aggregation vectors respectively, and outputting a plurality of second face noise vectors corresponding to the target face image data, as described above;

[0107] Finally, the multiple second facial noise vectors are spliced ​​to form a second facial feature vector corresponding to the target facial image data.

[0108] Combine Figure 4 The present application also provides a face recognition-based identity authentication device applicable to the aforementioned electronic device. The face recognition-based identity authentication device may include a face image acquisition module, a first feature mining module, a second feature mining module, and a verification result determination module.

[0109] In detail, the face image acquisition module can be used to acquire face image data to be identified by collecting face information of the object to be identified, wherein the face image data to be identified includes at least one frame of face image to be identified. In the embodiment of the present application, the face image acquisition module can be used to execute Figure 2 As shown in step S110, for the relevant content of the face image acquisition module, reference can be made to the above description of step S110.

[0110] In detail, the first feature mining module can be used to utilize the feature mining model included in the target face recognition network to perform feature mining and feature enhancement on the face image data to be recognized, and output a first face feature vector corresponding to the face image data to be recognized, wherein the target face recognition network is a trained neural network. In the embodiment of the present application, the first feature mining module can be used to perform Figure 2 As shown in step S120, for the relevant content of the first feature mining module, please refer to the description of step S120 above.

[0111] In detail, the second feature mining module can be used to use the feature mining model to perform feature mining and feature enhancement on the target face image data corresponding to the pre-stored target object, and output a second face feature vector corresponding to the target face image data, wherein the target face image data includes at least one frame of target face image. In the embodiment of the present application, the second feature mining module can be used to perform Figure 2 As shown in step S130, for the relevant content of the second feature mining module, reference can be made to the above description of step S130.

[0112] In detail, the verification result determination module can be used to use the feature comparison model included in the target face recognition network to perform a comparison analysis based on the first face feature vector and the second face feature vector, output a target comparison analysis result, and determine a target identity verification result based on the target comparison analysis result, wherein the target comparison analysis result is used to reflect whether the object to be identified belongs to the target object, and the target identity verification result is used to reflect whether the object to be identified has passed the identity verification. In an embodiment of the present application, the verification result determination module can be used to perform Figure 2As shown in step S140, for the relevant content of the verification result determination module, reference can be made to the above description of step S140.

[0113] In an embodiment of the present application, corresponding to the above-mentioned face recognition-based identity authentication method applied to the electronic device, a computer-readable storage medium is also provided, in which a computer program is stored, and when the computer program is run, the various steps of the face recognition-based identity authentication method are executed.

[0114] Among them, the steps executed when the aforementioned computer program is running will not be described here one by one. Please refer to the previous explanation of the identity authentication method based on face recognition.

[0115] In summary, the identity authentication method and apparatus, device and storage medium based on face recognition provided by the present application first obtain the face image data to be identified formed by collecting face information of the object to be identified; secondly, feature mining and feature enhancement are performed on the face image data to be identified, and a first face feature vector corresponding to the face image data to be identified is output; then, feature mining and feature enhancement are performed on the target face image data corresponding to the pre-stored target object, and a second face feature vector corresponding to the target face image data is output; finally, a comparative analysis is performed based on the first face feature vector and the second face feature vector, and a target comparative analysis result is output, and a target identity authentication result is determined based on the target comparative analysis result. Based on the above content, since the face image data to be identified and the target face image data will be feature mined and feature enhanced respectively, the obtained first face feature vector and second face feature vector both have better face semantic representation capabilities, and therefore, the reliability of the target identity authentication result obtained based on the first face feature vector and the second face feature vector can be guaranteed, thereby improving the problem of relatively low reliability of identity authentication in the prior art.

[0116] In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device and method embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0117] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0118] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or partly contributed to the prior art or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, electronic device, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk. It should be noted that, in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without further constraints, an element defined by the phrase "comprises a..." does not preclude the existence of additional identical elements in the process, method, article or apparatus that includes the element.

[0119] The foregoing description is merely a preferred embodiment of the present application and is not intended to limit the present application. Persons skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. An identity authentication method based on face recognition, characterized in that: include: Acquire facial image data to be identified formed by collecting facial information of a subject to be identified, wherein the facial image data to be identified includes at least one frame of facial image to be identified; The method comprises the following steps: using the convolution unit in the feature mining model included in the target face recognition network, performing convolution processing on each frame of the face image to be recognized in the face image data to be recognized, and outputting a first face convolution vector corresponding to each frame of the face image to be recognized; performing association mining on a plurality of first face convolution vectors corresponding to a plurality of frames of the face image to be recognized included in the face image data to be recognized according to the association relationship between the plurality of frames of the face images to be recognized, and outputting each first face association vector; superimposing a plurality of first face association vectors corresponding to the plurality of first face convolution vectors, and outputting a first face aggregation vector; sorting a plurality of random noise vectors, and sorting the random noise vectors that are sorted first. The vector is superimposed with the first face aggregation vector to form a first face noise vector corresponding to the random noise vector ranked first, and for each other random noise vector other than the random noise vector ranked first, a first face noise vector corresponding to the previous random noise vector of the other random noise vector is self-attention processed to obtain a corresponding self-attention vector, and the self-attention vector is superimposed with the other random noise vector to form a first face noise vector corresponding to the other random noise vector; multiple first face noise vectors are spliced ​​to form a first face feature vector, wherein the target face recognition network is a trained neural network; Using the feature mining model, performing feature mining and feature enhancement on pre-stored target facial image data corresponding to the target object, and outputting a second facial feature vector corresponding to the target facial image data, wherein the target facial image data includes at least one frame of target facial image; Utilizing the feature comparison model included in the target face recognition network, a comparative analysis is performed based on the first face feature vector and the second face feature vector, a target comparative analysis result is output, and a target identity authentication result is determined based on the target comparative analysis result, wherein the target comparative analysis result is used to reflect whether the object to be identified belongs to the target object, and the target identity authentication result is used to reflect whether the object to be identified has passed the identity authentication.

2. The identity authentication method based on face recognition according to claim 1, characterized in that: The step of utilizing the association mining unit in the feature mining model to perform association mining on a plurality of first face convolution vectors corresponding to a plurality of frames of face images to be recognized included in the face image data to be recognized, according to the association relationship between the plurality of frames of face images to be recognized, and outputting a first face association vector corresponding to each of the first face convolution vectors includes: For each frame of the facial image to be recognized in the multiple frames of facial images to be recognized included in the facial image data to be recognized, segmenting the first facial convolution vector corresponding to the facial image to be recognized to form multiple first facial local vectors corresponding to the facial image to be recognized; performing association mining on a plurality of first face local vectors corresponding to a plurality of frames of face images to be recognized included in the face image data according to association relationships between the plurality of frames of face images to be recognized using an association mining unit in the feature mining model, and outputting an associated first face local vector corresponding to each of the first face local vectors; For each frame of the facial image to be recognized in the multiple frames of facial images to be recognized included in the facial image data to be recognized, combining multiple associated first facial local vectors corresponding to the multiple first facial local vectors corresponding to the facial image to be recognized to form a first facial combination vector corresponding to the facial image to be recognized, wherein the combining process and the segmentation process are mutually inverse processes; For each frame of the face image to be identified, based on the first face combination vector corresponding to the face image to be identified, the first face convolution vector corresponding to the face image to be identified is subjected to focused feature mining processing to form a first face association vector corresponding to the first face convolution vector.

3. The identity authentication method based on face recognition according to claim 2, characterized in that: The step of utilizing the association mining unit in the feature mining model to perform association mining on a plurality of first face local vectors corresponding to a plurality of frames of face images to be recognized included in the face image data to be recognized, according to the association relationship between the plurality of frames of face images to be recognized, and outputting an associated first face local vector corresponding to each of the first face local vectors includes: Arranging the plurality of first face local vectors corresponding to each frame of the face image to be recognized in the plurality of frames of the face image to be recognized included in the face image data to be recognized according to a sequence relationship between their distribution positions in the corresponding first face convolution vectors and the corresponding face images to be recognized, to form a plurality of first face local vector sequences, wherein any two first face local vectors in the same first face local vector have the same distribution positions in the corresponding first face convolution vector; For each of the first partial face vectors, association mining is performed on the first partial face vector according to the first partial face vector sequence in which the first partial face vector is located, and an associated first partial face vector corresponding to the first partial face vector is output.

4. The identity authentication method based on face recognition according to claim 3, characterized in that: The step of performing association mining on each first face local vector based on a first face local vector sequence in which the first face local vector is located, and outputting an associated first face local vector corresponding to the first face local vector, includes: For a first first face partial vector in the first face partial vector sequence, taking the first face partial vector as the corresponding associated first face partial vector; For each other first human face local vector other than the first first human face local vector in the first human face local vector sequence, based on the associated first human face local vector corresponding to the previous first human face local vector, the other first human face local vector is subjected to focused feature mining processing to form the associated first human face local vector corresponding to the other first human face local vector.

5. The identity authentication method based on face recognition according to any one of claims 1 to 4, characterized in that: The step of using the feature mining model to perform feature mining and feature enhancement on pre-stored target facial image data corresponding to the target object, and outputting a second facial feature vector corresponding to the target facial image data, includes: Using the convolution unit in the feature mining model, convolution processing is performed on each frame of the target facial image data, and a second facial convolution vector corresponding to each frame of the target facial image is output, wherein the target facial image data includes multiple frames of target facial images; Utilizing the association mining unit in the feature mining model, performing association mining on a plurality of second face convolution vectors corresponding to the plurality of target face images included in the target face image data according to association relationships between the plurality of target face images, and outputting a second face association vector corresponding to each of the second face convolution vectors; Superimposing multiple second face association vectors corresponding to the multiple second face convolution vectors, and outputting a second face aggregation vector corresponding to the target face image data; performing a plurality of different noise processing on the second face aggregation vectors respectively, and outputting a plurality of second face noise vectors corresponding to the target face image data; The multiple second facial noise vectors are spliced ​​to form a second facial feature vector corresponding to the target facial image data.

6. An identity verification device based on face recognition, characterized in that: include: A face image acquisition module is used to acquire face image data to be identified formed by collecting face information of a subject to be identified, wherein the face image data to be identified includes at least one frame of face image to be identified; The first feature mining module is used to use the convolution unit in the feature mining model included in the target face recognition network to perform convolution processing on each frame of the face image data to be recognized, and output the first face convolution vector corresponding to each frame of the face image to be recognized; perform association mining on multiple first face convolution vectors corresponding to multiple frames of the face image to be recognized included in the face image data to be recognized according to the association relationship between the multiple frames of the face images to be recognized, and output each first face association vector; superimpose multiple first face association vectors corresponding to the multiple first face convolution vectors, and output a first face aggregation vector; sort multiple random noise vectors, and sort them into the first The random noise vector of the face is superimposed with the first face aggregation vector to form a first face noise vector corresponding to the random noise vector ranked first, and for each other random noise vector other than the random noise vector ranked first, the first face noise vector corresponding to the previous random noise vector of the other random noise vector is self-attention processed to obtain a corresponding self-attention vector, and the self-attention vector is superimposed with the other random noise vector to form a first face noise vector corresponding to the other random noise vector; multiple first face noise vectors are spliced ​​to form a first face feature vector, wherein the target face recognition network is a trained neural network; a second feature mining module, configured to perform feature mining and feature enhancement on pre-stored target facial image data corresponding to the target object using the feature mining model, and output a second facial feature vector corresponding to the target facial image data, wherein the target facial image data includes at least one frame of target facial image; A verification result determination module is used to utilize the feature comparison model included in the target face recognition network to perform a comparative analysis based on the first face feature vector and the second face feature vector, output a target comparative analysis result, and determine a target identity authentication result based on the target comparative analysis result, wherein the target comparative analysis result is used to reflect whether the object to be identified belongs to the target object, and the target identity authentication result is used to reflect whether the object to be identified has passed the identity authentication.

7. An electronic device, characterized in that: include: memory for storing computer programs; A processor connected to the memory is used to execute the computer program stored in the memory to implement the face recognition-based identity authentication method described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when running, executes the identity authentication method based on face recognition according to any one of claims 1 to 5.

Images