Identity verification method and device based on face recognition, equipment and storage medium

Abstract

The invention provides an identity verification method and device based on face recognition, equipment and a storage medium, and relates to the technical field of face recognition. The method comprises the following steps: firstly, acquiring to-be-recognized face image data formed by performing face information acquisition on a to-be-recognized object; secondly, performing feature mining and feature enhancement on the to-be-recognized face image data, and outputting a first face feature vector corresponding to the to-be-recognized face image data; then, performing feature mining and feature enhancement on pre-stored target face image data corresponding to a target object, and outputting a second face feature vector corresponding to the target face image data; and finally, performing comparative analysis based on the first face feature vector and the second face feature vector, outputting a target comparative analysis result, and determining a target identity verification result based on the target comparative analysis result. Based on the content, the problem that the reliability of identity verification is relatively low in the prior art can be improved.

Description

Technical Field

[0001] The present application relates to the field of face recognition technology, and in particular to an identity authentication method and apparatus, device and storage medium based on face recognition. Background Art

[0002] As a biometric recognition technology, face recognition technology has been widely used in recent years, especially in the fields of identity authentication and security. Face recognition is a technology that confirms and verifies a person's identity by analyzing the feature points in a face image or video. This technology combines the technological developments in various fields such as computer vision, machine learning, and image processing, and has gradually become one of the core technologies in many fields such as smart security and smart cities. For example, in public places such as airports, railway stations, subways and other transportation hubs, face recognition technology is used for identity authentication, replacing traditional manual security checks and identity document verification. This not only improves efficiency, but also reduces errors and safety hazards caused by manual operations. However, the inventors have found that in the existing identity authentication technology based on face recognition, there is a relatively low reliability of identity authentication. Summary of the invention

[0003] In view of this, the purpose of this application is to provide an identity authentication method and apparatus, device and storage medium based on face recognition to improve.

[0004] To achieve the above objectives, this application adopts the following technical solutions:

[0005] An identity authentication method based on face recognition, comprising:

[0006] Acquire facial image data to be identified formed by collecting facial information of an object to be identified, wherein the facial image data to be identified includes at least one frame of facial image to be identified;

[0007] Using a feature mining model included in a target face recognition network, feature mining and feature enhancement are performed on the face image data to be recognized, and a first face feature vector corresponding to the face image data to be recognized is output, wherein the target face recognition network is a trained neural network;

[0008] Using the feature mining model, feature mining and feature enhancement are performed on pre-stored target facial image data corresponding to the target object, and a second facial feature vector corresponding to the target facial image data is output, wherein the target facial image data includes at least one frame of target facial image;

[0009] Utilizing the feature comparison model included in the target face recognition network, a comparison analysis is performed based on the first face feature vector and the second face feature vector, a target comparison analysis result is output, and a target identity authentication result is determined based on the target comparison analysis result, wherein the target comparison analysis result is used to reflect whether the object to be identified belongs to the target object, and the target identity authentication result is used to reflect whether the object to be identified has passed the identity authentication.

[0010] In a preferred embodiment of the present application, in the above-mentioned identity authentication method based on face recognition, the step of using the feature mining model included in the target face recognition network to perform feature mining and feature enhancement on the face image data to be identified, and outputting the first face feature vector corresponding to the face image data to be identified, includes:

[0011] Using the convolution unit in the feature mining model included in the target face recognition network, convolution processing is performed on each frame of the face image to be recognized in the face image data to be recognized, and a first face convolution vector corresponding to each frame of the face image to be recognized is output, wherein the face image data to be recognized includes multiple frames of face images to be recognized;

[0012] Utilizing the association mining unit in the feature mining model, performing association mining on a plurality of first face convolution vectors corresponding to a plurality of frames of face images to be recognized included in the face image data to be recognized according to the association relationship between the plurality of frames of face images to be recognized, and outputting a first face association vector corresponding to each of the first face convolution vectors;

[0013] Superimposing the multiple first face association vectors corresponding to the multiple first face convolution vectors, and outputting the first face aggregation vector corresponding to the face image data to be recognized;

[0014] Performing a plurality of different noise processing on the first face aggregation vectors respectively, and outputting a plurality of first face noise vectors corresponding to the face image data to be recognized;

[0015] The multiple first face noise vectors are concatenated to form a first face feature vector corresponding to the face image data to be identified.

[0016] In a preferred embodiment of the present application, in the above-mentioned identity authentication method based on face recognition, the step of using the association mining unit in the feature mining model to perform association mining on multiple first face convolution vectors corresponding to multiple frames of face images to be identified included in the face image data to be identified according to the association relationship between the multiple frames of face images to be identified, and outputting the first face association vector corresponding to each of the first face convolution vectors, includes:

[0017] For each frame of the facial image to be recognized in the multiple frames of the facial image to be recognized included in the facial image data to be recognized, segmenting the first facial convolution vector corresponding to the facial image to be recognized to form multiple first facial local vectors corresponding to the facial image to be recognized;

[0018] Utilizing the association mining unit in the feature mining model, performing association mining on a plurality of first face local vectors corresponding to a plurality of frames of face images to be recognized included in the face image data to be recognized according to the association relationship between the plurality of frames of face images to be recognized, and outputting an associated first face local vector corresponding to each of the first face local vectors;

[0019] For each frame of the face image to be recognized in the multiple frames of the face image to be recognized included in the face image data to be recognized, multiple associated first face local vectors corresponding to the multiple first face local vectors corresponding to the face image to be recognized are combined to form a first face combination vector corresponding to the face image to be recognized, wherein the combination process and the segmentation process are mutually inverse processes;

[0020] For each frame of face image to be identified, based on the first face combination vector corresponding to the face image to be identified, a focused feature mining process is performed on the first face convolution vector corresponding to the face image to be identified to form a first face association vector corresponding to the first face convolution vector.

[0021] In a preferred embodiment of the present application, in the above-mentioned identity authentication method based on face recognition, the step of using the association mining unit in the feature mining model to perform association mining on multiple first face local vectors corresponding to multiple frames of face images to be identified included in the face image data to be identified according to the association relationship between the multiple frames of face images to be identified, and outputting the associated first face local vector corresponding to each of the first face local vectors, includes:

[0022] Arrange the plurality of first face local vectors corresponding to each frame of the face image to be recognized in the plurality of frames of the face image to be recognized included in the face image data to be recognized according to the distribution position in the corresponding first face convolution vector and the order relationship between the corresponding face image to be recognized, to form a plurality of first face local vector sequences, wherein any two first face local vectors in the same first face local vector have the same distribution position in the corresponding first face convolution vector;

[0023] For each of the first partial face vectors, association mining is performed on the first partial face vector according to the first partial face vector sequence in which the first partial face vector is located, and an associated first partial face vector corresponding to the first partial face vector is output.

[0024] In a preferred embodiment of the present application, in the above-mentioned identity authentication method based on face recognition, the step of performing association mining on each of the first face local vectors according to the first face local vector sequence in which the first face local vector is located, and outputting the associated first face local vector corresponding to the first face local vector includes:

[0025] For a first first face local vector in the first face local vector sequence, taking the first face local vector as the corresponding associated first face local vector;

[0026] For each other first human face local vector other than the first first human face local vector in the first human face local vector sequence that does not belong to it, based on the associated first human face local vector corresponding to the previous first human face local vector, focused feature mining processing is performed on the other first human face local vector to form an associated first human face local vector corresponding to the other first human face local vector.

[0027] In a preferred embodiment of the present application, in the above-mentioned identity authentication method based on face recognition, the step of performing a plurality of different noise processing on the first face aggregation vector respectively and outputting a plurality of first face noise vectors corresponding to the face image data to be identified includes:

[0028] Based on the vector size of the first face aggregation vector, generating a plurality of random noise vectors having the same vector size, wherein in each of the random noise vectors, the number of non-zero vector parameters is less than the number of zero vector parameters;

[0029] Respectively superimpose each of the multiple random noise vectors with the first face aggregation vector to form multiple first face noise vectors corresponding to the face image data to be identified; or, sort the multiple random noise vectors, and superimpose the random noise vector sorted as the first with the first face aggregation vector to form a first face noise vector corresponding to the random noise vector sorted as the first, and, for each other random noise vector except the random noise vector sorted as the first, perform self-attention processing on the first face noise vector corresponding to the previous random noise vector of the other random noise vector to obtain the corresponding self-attention vector, and superimpose the self-attention vector and the other random noise vector to form the first face noise vector corresponding to the other random noise vector.

[0030] In a preferred embodiment of the present application, in the above-mentioned identity authentication method based on face recognition, the step of using the feature mining model to perform feature mining and feature enhancement on the target face image data corresponding to the pre-stored target object, and outputting the second face feature vector corresponding to the target face image data, includes:

[0031] Using the convolution unit in the feature mining model, convolution processing is performed on each frame of the target face image in the target face image data, and a second face convolution vector corresponding to each frame of the target face image is output, wherein the target face image data includes multiple frames of target face images;

[0032] Utilizing the association mining unit in the feature mining model, performing association mining on a plurality of second face convolution vectors corresponding to the plurality of frames of target face images included in the target face image data according to the association relationship between the plurality of frames of target face images, and outputting a second face association vector corresponding to each of the second face convolution vectors;

[0033] Superimposing multiple second face association vectors corresponding to the multiple second face convolution vectors, and outputting a second face aggregation vector corresponding to the target face image data;

[0034] Performing a plurality of different noise processing on the second face aggregation vectors respectively, and outputting a plurality of second face noise vectors corresponding to the target face image data;

[0035] The multiple second face noise vectors are concatenated to form a second face feature vector corresponding to the target face image data.

[0036] The present application also provides an identity verification device based on face recognition, comprising:

[0037] A face image acquisition module, used to acquire face image data to be identified formed by collecting face information of the object to be identified, wherein the face image data to be identified includes at least one frame of face image to be identified;

[0038] A first feature mining module is used to perform feature mining and feature enhancement on the face image data to be identified by using a feature mining model included in a target face recognition network, and output a first face feature vector corresponding to the face image data to be identified, wherein the target face recognition network is a trained neural network;

[0039] A second feature mining module is used to use the feature mining model to perform feature mining and feature enhancement on the pre-stored target face image data corresponding to the target object, and output a second face feature vector corresponding to the target face image data, wherein the target face image data includes at least one frame of target face image;

[0040] A verification result determination module is used to utilize the feature comparison model included in the target face recognition network to perform a comparison analysis based on the first face feature vector and the second face feature vector, output a target comparison analysis result, and determine a target identity authentication result based on the target comparison analysis result, wherein the target comparison analysis result is used to reflect whether the object to be identified belongs to the target object, and the target identity authentication result is used to reflect whether the object to be identified has passed the identity authentication.

[0041] Based on the above, the present application also provides an electronic device, including:

[0042] Memory for storing computer programs;

[0043] A processor connected to the memory is used to execute the computer program stored in the memory to implement the above-mentioned identity authentication method based on face recognition.

[0044] Based on the above, the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is run, the various steps of the above-mentioned identity authentication method based on face recognition are executed.

[0045] The identity authentication method and apparatus, device and storage medium based on face recognition provided by the present application first obtain the face image data to be identified formed by collecting face information of the object to be identified; secondly, perform feature mining and feature enhancement on the face image data to be identified, and output the first face feature vector corresponding to the face image data to be identified; then, perform feature mining and feature enhancement on the target face image data corresponding to the pre-stored target object, and output the second face feature vector corresponding to the target face image data; finally, perform comparative analysis based on the first face feature vector and the second face feature vector, output the target comparative analysis result, and determine the target identity authentication result based on the target comparative analysis result. Based on the above content, since the face image data to be identified and the target face image data will be feature mined and feature enhanced respectively, the first face feature vector and the second face feature vector obtained have better face semantic representation capabilities, therefore, the reliability of the target identity authentication result obtained based on the first face feature vector and the second face feature vector can be guaranteed, thereby improving the problem of relatively low reliability of identity authentication existing in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings.

[0047] Figure 1A structural block diagram of an electronic device provided in an embodiment of the present application.

[0048] Figure 2 A flowchart of an identity verification method based on face recognition provided in an embodiment of the present application.

[0049] Figure 3 A schematic diagram of the noise application process provided in an embodiment of the present application.

[0050] Figure 4 A block diagram of an identity verification device based on face recognition provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0052] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0053] like Figure 1 As shown, an embodiment of the present application provides an electronic device, wherein the electronic device may include a memory, a processor, and an identity verification device based on face recognition.

[0054] In detail, the memory and the processor are electrically connected directly or indirectly to achieve data transmission or interaction. For example, the memory and the processor can be electrically connected through one or more communication buses or signal lines. The face recognition-based identity authentication device includes at least one software function module stored in the memory in the form of software or firmware. The processor is used to execute an executable computer program stored in the memory, for example, the software function module and computer program included in the face recognition-based identity authentication device, so as to implement the face recognition-based identity authentication method provided in the embodiment of the present application.

[0055] Optionally, the memory can be, but is not limited to, random access memory (RAM), read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0056] Furthermore, the processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0057] Understandably, Figure 1 The structure shown is for illustration only, and the electronic device may also include Figure 1 More or fewer components as shown, or with Figure 1 The different configurations shown, for example, may also include a communication unit for exchanging information with other devices (such as an image acquisition device).

[0058] Combination Figure 2 The embodiment of the present application also provides a method for identity verification based on face recognition that can be applied to the above electronic device. The method steps defined in the process related to the identity verification method based on face recognition can be implemented by the electronic device.

[0059] The following will Figure 2 The specific process shown is explained in detail.

[0060] Step S110, acquiring face image data to be identified formed by collecting face information of the object to be identified.

[0061] In an embodiment of the present application, the electronic device can obtain facial image data to be identified formed by collecting facial information of the object to be identified, such as from a connected image acquisition device. The facial image data to be identified includes at least one frame of facial image to be identified, that is, it can include one frame of facial image to be identified, or it can include multiple frames of continuous facial images to be identified. It should be noted that the facial image to be identified can be either the original image acquired by the image acquisition device, or it can be a facial image formed by further segmentation based on the original image.

[0062] Step S120, using the feature mining model included in the target face recognition network, perform feature mining and feature enhancement on the face image data to be recognized, and output a first face feature vector corresponding to the face image data to be recognized.

[0063] In an embodiment of the present application, after obtaining the face image data to be identified, the electronic device can use the feature mining model included in the target face recognition network to perform feature mining and feature enhancement on the face image data to be identified, and output a first face feature vector corresponding to the face image data to be identified. The target face recognition network is a trained neural network, that is, it is trained accordingly through sample face image data and corresponding data labels.

[0064] Step S130, using the feature mining model, performing feature mining and feature enhancement on the target facial image data corresponding to the pre-stored target object, and outputting a second facial feature vector corresponding to the target facial image data.

[0065] In an embodiment of the present application, the electronic device can utilize the feature mining model to perform feature mining and feature enhancement on the target facial image data corresponding to the pre-stored target object, and output a second facial feature vector corresponding to the target facial image data. Wherein, the target facial image data includes at least one frame of target facial image, that is, it can include one frame of target facial image, and can also include multiple frames of continuous target facial images. It should be noted that the target facial image can be either the original image captured by the image acquisition device, or a facial image formed by further segmentation based on the original image. In addition, there can be multiple target objects, so that feature mining and feature enhancement can be performed on the target facial image data corresponding to each target object respectively.

[0066] Step S140, using the feature comparison model included in the target face recognition network, performing a comparison analysis based on the first face feature vector and the second face feature vector, outputting a target comparison analysis result, and determining a target identity authentication result based on the target comparison analysis result.

[0067] In an embodiment of the present application, after obtaining the first face feature vector and the second face feature vector, the electronic device can use the feature comparison model included in the target face recognition network to perform comparison analysis based on the first face feature vector and the second face feature vector, output the target comparison analysis result, and determine the target identity authentication result based on the target comparison analysis result. Wherein, the target comparison analysis result is used to reflect whether the object to be identified belongs to the target object, and the target identity authentication result is used to reflect whether the object to be identified has passed the identity authentication. For example, when the target comparison analysis result reflects that the object to be identified belongs to the target object, the target identity authentication result is used to reflect that the object to be identified has passed the identity authentication; when the target comparison analysis result reflects that the object to be identified does not belong to the target object, the target identity authentication result is used to reflect that the object to be identified has not passed the identity authentication. In addition, in some embodiments, the similarity (such as cosine similarity) between the first face feature vector and the second face feature vector can be calculated. In this way, when the similarity is greater than a preset similarity (such as 0.8, 0.9, etc.), it can be determined that the object to be identified belongs to the target object; when the similarity is not greater than the preset similarity, it can be determined that the object to be identified does not belong to the target object. Alternatively, in some other embodiments, the first facial feature vector and the second facial feature vector may be concatenated, and then the concatenated vector may be fully connected. Furthermore, the fully connected vector may be classified and output (such as through classification functions such as softmax to predict the probability of belonging to the target object) to obtain corresponding target comparison analysis results.

[0068] Based on the above content, since the facial image data to be identified and the target facial image data will be feature mined and enhanced respectively, the obtained first facial feature vector and the second facial feature vector will have better facial semantic representation capabilities. Therefore, the reliability of the target identity authentication results obtained based on the first facial feature vector and the second facial feature vector can be guaranteed, thereby improving the problem of relatively low reliability of identity authentication in the prior art.

[0069] It should be noted that for step S120, the specific method of performing feature mining and feature enhancement on the face image data to be identified is not limited and can be selected according to actual needs.

[0070] For example, in an alternative implementation, in order to ensure that the output first facial feature vector has better semantic representation ability, the above-mentioned step S120 may further include step S121, step S122, step S123, step S124 and step S125, and the contents of each step are described as follows.

[0071] Step S121, using the convolution unit in the feature mining model included in the target face recognition network, respectively perform convolution processing on each frame of the face image to be recognized in the face image data to be recognized, and output a first face convolution vector corresponding to each frame of the face image to be recognized.

[0072] In the embodiment of the present application, the convolution unit in the feature mining model included in the target face recognition network can be used to perform convolution processing on each frame of the face image data to be recognized, and output the first face convolution vector corresponding to each frame of the face image to be recognized. Wherein, the face image data to be recognized includes multiple frames of face images to be recognized, so that multiple corresponding first face convolution vectors can be obtained.

[0073] Step S122, utilizing the association mining unit in the feature mining model, performs association mining on multiple first face convolution vectors corresponding to multiple frames of face images to be identified included in the face image data to be identified, according to the association relationship between the multiple frames of face images to be identified, and outputs a first face association vector corresponding to each of the first face convolution vectors.

[0074] In an embodiment of the present application, after obtaining multiple first face convolution vectors, the association mining unit in the feature mining model can be used to perform association mining on the multiple first face convolution vectors corresponding to the multiple frames of face images to be identified included in the face image data to be identified, according to the association relationship between the multiple frames of face images to be identified (such as the order of collection, etc.), and output the first face association vector corresponding to each of the first face convolution vectors. In this way, the semantic information carried by the first face association vector can be richer.

[0075] Step S123, superimposing the multiple first face association vectors corresponding to the multiple first face convolution vectors, and outputting the first face aggregation vector corresponding to the face image data to be identified.

[0076] In an embodiment of the present application, multiple first face association vectors corresponding to the multiple first face convolution vectors can be superimposed to output a first face aggregation vector corresponding to the face image data to be identified. Exemplarily, mean superposition can be performed.

[0077] Step S124, performing a plurality of different noise application processes on the first face aggregation vectors respectively, and outputting a plurality of first face noise vectors corresponding to the face image data to be recognized.

[0078] In the embodiment of the present application, the first face aggregation vector can be processed with a plurality of different noises respectively, and a plurality of first face noise vectors corresponding to the face image data to be identified can be output, so that the plurality of first face noise vectors are different from each other. In this way, by applying noise, more realistic face information can be simulated to a certain extent (some information may be lost during image acquisition).

[0079] Step S125: concatenate the multiple first face noise vectors to form a first face feature vector corresponding to the face image data to be recognized.

[0080] In an embodiment of the present application, after forming multiple first face noise vectors, the multiple first face noise vectors can be spliced ​​to form a first face feature vector corresponding to the face image data to be identified; or, after splicing, pooling processing can be performed to obtain the first face feature vector corresponding to the face image data to be identified.

[0081] It can be understood that, in the above-mentioned step S122, the specific method of performing association mining on the multiple first face convolution vectors corresponding to the multiple frames of face images to be identified included in the face image data to be identified according to the association relationship between the multiple frames of face images to be identified is not limited and can be selected according to actual needs. For example, in an alternative embodiment, in order to improve the reliability of association mining, the above-mentioned step S122 can further include step S122a, step S122b, step S122c and step S122d, and the specific content of each step is described as follows.

[0082] Step S122a, for each frame of the face image to be identified in the multiple frames of the face image to be identified included in the face image data to be identified, segment the first face convolution vector corresponding to the face image to be identified to form multiple first face local vectors corresponding to the face image to be identified.

[0083] In an embodiment of the present application, for each frame of facial images to be identified in the multiple frames of facial images to be identified included in the facial image data to be identified, the first facial convolution vector corresponding to the facial image to be identified can be segmented to form multiple first facial local vectors corresponding to the facial image to be identified; illustratively, the sizes of the first facial local vectors can be the same, and in addition, the first facial local vectors can be combined together to form a corresponding first facial convolution vector. For example, the size of the first facial convolution vector is n*m, which can be segmented to form 4 (n / 2)*(m / 2) first facial local vectors.

[0084] Step S122b, utilizing the association mining unit in the feature mining model, performs association mining on multiple first local face vectors corresponding to multiple frames of face images to be identified included in the face image data to be identified, according to the association relationship between the multiple frames of face images to be identified, and outputs the associated first local face vector corresponding to each of the first local face vectors.

[0085] In an embodiment of the present application, after the first local face vector is segmented and formed, the association mining unit in the feature mining model can be used to perform association mining on multiple first local face vectors corresponding to multiple frames of face images to be identified included in the face image data to be identified, according to the association relationship between the multiple frames of face images to be identified, and output the associated first local face vector corresponding to each of the first local face vectors. In this way, the granularity of the association mining can be updated, thereby ensuring that the reliability of the association mining can be higher.

[0086] Step S122c, for each frame of the face image to be identified in the multiple frames of the face image to be identified included in the face image data to be identified, multiple associated first face local vectors corresponding to the multiple first face local vectors corresponding to the face image to be identified are combined to form a first face combination vector corresponding to the face image to be identified.

[0087] In the embodiment of the present application, for each frame of the face image to be identified in the multiple frames of the face image to be identified included in the face image data to be identified, multiple associated first face local vectors corresponding to the multiple first face local vectors corresponding to the face image to be identified are combined to form a first face combination vector corresponding to the face image to be identified. The combination process and the segmentation process are mutually inverse processes.

[0088] Step S122d, for each frame of the face image to be identified, based on the first face combination vector corresponding to the face image to be identified, a focused feature mining process is performed on the first face convolution vector corresponding to the face image to be identified to form a first face association vector corresponding to the first face convolution vector.

[0089] In an embodiment of the present application, after the first face combination vector is formed, for each frame of the face image to be identified, based on the first face combination vector corresponding to the face image to be identified, the first face convolution vector corresponding to the face image to be identified can be subjected to focused feature mining processing to form a first face association vector corresponding to the first face convolution vector. Exemplarily, the first face combination vector can be multiplied by the transposed vector of the first face convolution vector to form a corresponding vector association parameter, and, based on the vector association parameter, the first face convolution vector can be weighted to obtain the first face association vector corresponding to the first face convolution vector.

[0090] It can be understood that, in the above-mentioned step S122b, the specific method of performing association mining on the multiple first local face vectors corresponding to the multiple frames of face images to be identified included in the face image data to be identified according to the association relationship between the multiple frames of face images to be identified is not limited and can be selected according to actual needs. For example, in an alternative embodiment, in order to improve the reliability of the associated first local face vectors obtained by association mining, the above-mentioned step S122b can further include steps b1 and b2, and the specific contents of each step are described as follows.

[0091] Step b1, arrange the multiple first face local vectors corresponding to each frame of the face image to be identified in the multiple frames of the face image to be identified included in the face image data to be identified according to the distribution position in the corresponding first face convolution vector and the sequence relationship between the corresponding face image to be identified, so as to form multiple first face local vector sequences.

[0092] In an embodiment of the present application, the multiple first face local vectors corresponding to each frame of the face image to be recognized in the multiple frames of the face image to be recognized included in the face image data to be recognized can be arranged according to the distribution position in the corresponding first face convolution vector and the order relationship between the corresponding face images to be recognized, to form a sequence of multiple first face local vectors, such as {a first face local vector corresponding to the first frame of the image to be recognized, a first face local vector corresponding to the second frame of the image to be recognized, a first face local vector corresponding to the third frame of the image to be recognized, ...}. Among them, any two first face local vectors in the same first face local vector have the same distribution position in the corresponding first face convolution vector.

[0093] Step b2: for each of the first partial face vectors, perform association mining on the first partial face vector according to the first partial face vector sequence in which the first partial face vector is located, and output an associated first partial face vector corresponding to the first partial face vector.

[0094] In an embodiment of the present application, for each of the first local face vectors, association mining can be performed on the first local face vector based on the first local face vector sequence in which the first local face vector is located, and an associated first local face vector corresponding to the first local face vector is output, that is, association mining is performed on the first local face vectors with the same distribution position, so that semantic information can be fused to obtain associated first local face vectors with richer semantic information.

[0095] It can be understood that, in the above step b2, the specific manner of performing association mining on the first face local vector according to the first face local vector sequence in which the first face local vector is located is not limited and can be selected according to actual needs. For example, in an alternative implementation, in order to improve the reliability of association mining, the above step b2 can further include the following contents:

[0096] Firstly, for the first first face local vector in the first face local vector sequence, the first face local vector may be used as the corresponding associated first face local vector;

[0097] Secondly, for each other first human face local vector other than the first first human face local vector in the first human face local vector sequence, based on the associated first human face local vector corresponding to the previous first human face local vector, focused feature mining processing is performed on the other first human face local vector to form an associated first human face local vector corresponding to the other first human face local vector, that is, the semantic information carried by the associated first human face local vector corresponding to the previous first human face local vector is integrated into the other first human face local vector to obtain the associated first human face local vector.

[0098] It can be understood that, in the above step S124, the specific manner of performing a plurality of different noise application processes on the first face aggregation vector is not limited and can be selected according to actual needs. For example, in an alternative implementation, the above step S124 may include:

[0099] First, based on the vector size of the first face aggregation vector, a plurality of random noise vectors having the same vector size may be generated, wherein in each of the random noise vectors, the number of non-zero vector parameters is less than the number of zero vector parameters;

[0100] Secondly, each of the plurality of random noise vectors is superimposed with the first face aggregation vector to form a plurality of first face noise vectors corresponding to the face image data to be identified; or Figure 3, sorting the plurality of random noise vectors, and superimposing the random noise vector ranked first with the first face aggregation vector to form a first face noise vector corresponding to the random noise vector ranked first, and, for each other random noise vector other than the random noise vector ranked first, performing self-attention processing on the first face noise vector corresponding to the previous random noise vector of the other random noise vector to obtain the corresponding self-attention vector, and, superimposing the self-attention vector and the other random noise vector to form a first face noise vector corresponding to the other random noise vector; exemplarily, the aforementioned superposition processing may be mean superposition.

[0101] It should be noted that for step S130, the specific method of performing feature mining and feature enhancement on the target facial image data corresponding to the target object is not limited and can be selected according to actual needs.

[0102] For example, in an alternative implementation, to ensure that the second facial feature vector corresponding to the output target facial image data has better semantic representation capability, the above step S130 may further include the following contents (refer to the above description of step S120):

[0103] First, the convolution unit in the feature mining model can be used to perform convolution processing on each frame of the target face image in the target face image data, and output a second face convolution vector corresponding to each frame of the target face image, wherein the target face image data includes multiple frames of target face images, as described above;

[0104] Secondly, using the association mining unit in the feature mining model, multiple second face convolution vectors corresponding to the multiple frames of target face images included in the target face image data are subjected to association mining according to the association relationship between the multiple frames of target face images, and a second face association vector corresponding to each of the second face convolution vectors is output, as described above;

[0105] Then, a plurality of second face association vectors corresponding to the plurality of second face convolution vectors are superimposed to output a second face aggregation vector corresponding to the target face image data;

[0106] Afterwards, a plurality of different noise application processes are respectively performed on the second face aggregation vectors, and a plurality of second face noise vectors corresponding to the target face image data are output, as described above;

[0107] Finally, the multiple second face noise vectors are concatenated to form a second face feature vector corresponding to the target face image data.

[0108] Combination Figure 4 The embodiment of the present application also provides a face recognition-based identity authentication device applicable to the above electronic device. The face recognition-based identity authentication device may include a face image acquisition module, a first feature mining module, a second feature mining module, and a verification result determination module.

[0109] In detail, the face image acquisition module can be used to acquire face image data to be identified by collecting face information of the object to be identified, wherein the face image data to be identified includes at least one frame of face image to be identified. Figure 2 As shown in step S110, the relevant contents of the face image acquisition module can refer to the above description of step S110.

[0110] In detail, the first feature mining module can be used to use the feature mining model included in the target face recognition network to perform feature mining and feature enhancement on the face image data to be recognized, and output a first face feature vector corresponding to the face image data to be recognized, wherein the target face recognition network is a trained neural network. In the embodiment of the present application, the first feature mining module can be used to perform Figure 2 As shown in step S120, the relevant contents of the first feature mining module can refer to the above description of step S120.

[0111] In detail, the second feature mining module can be used to use the feature mining model to perform feature mining and feature enhancement on the target face image data corresponding to the pre-stored target object, and output a second face feature vector corresponding to the target face image data, wherein the target face image data includes at least one frame of the target face image. In the embodiment of the present application, the second feature mining module can be used to perform Figure 2 As shown in step S130, the relevant contents of the second feature mining module can refer to the above description of step S130.

[0112] In detail, the verification result determination module can be used to use the feature comparison model included in the target face recognition network to perform comparison analysis based on the first face feature vector and the second face feature vector, output the target comparison analysis result, and determine the target identity verification result based on the target comparison analysis result, wherein the target comparison analysis result is used to reflect whether the object to be identified belongs to the target object, and the target identity verification result is used to reflect whether the object to be identified has passed the identity verification. In an embodiment of the present application, the verification result determination module can be used to perform Figure 2As shown in step S140, the relevant contents of the verification result determination module can refer to the above description of step S140.

[0113] In an embodiment of the present application, corresponding to the above-mentioned face recognition-based identity authentication method applied to the electronic device, a computer-readable storage medium is also provided, in which a computer program is stored, and when the computer program is run, each step of the face recognition-based identity authentication method is executed.

[0114] Among them, the steps executed when the aforementioned computer program is running will not be described one by one here, and reference can be made to the explanation of the aforementioned identity authentication method based on face recognition in the previous article.

[0115] In summary, the identity authentication method and apparatus, device and storage medium based on face recognition provided by the present application first obtain the face image data to be identified formed by collecting face information of the object to be identified; secondly, perform feature mining and feature enhancement on the face image data to be identified, and output the first face feature vector corresponding to the face image data to be identified; then, perform feature mining and feature enhancement on the target face image data corresponding to the pre-stored target object, and output the second face feature vector corresponding to the target face image data; finally, perform comparative analysis based on the first face feature vector and the second face feature vector, output the target comparative analysis result, and determine the target identity authentication result based on the target comparative analysis result. Based on the above content, since the face image data to be identified and the target face image data will be feature mined and feature enhanced respectively, so that the obtained first face feature vector and the second face feature vector both have better face semantic representation capabilities, therefore, the reliability of the target identity authentication result obtained based on the first face feature vector and the second face feature vector can be guaranteed, thereby improving the problem of relatively low reliability of identity authentication existing in the prior art.

[0116] In several embodiments provided in the embodiments of the present application, it should be understood that the disclosed device and method can also be implemented in other ways. The device and method embodiments described above are merely schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0117] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0118] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk. It should be noted that, in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0119] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An identity authentication method based on face recognition, characterized in that: include: Acquire facial image data to be identified formed by collecting facial information of an object to be identified, wherein the facial image data to be identified includes at least one frame of facial image to be identified; Using a feature mining model included in a target face recognition network, feature mining and feature enhancement are performed on the face image data to be recognized, and a first face feature vector corresponding to the face image data to be recognized is output, wherein the target face recognition network is a trained neural network; Using the feature mining model, feature mining and feature enhancement are performed on pre-stored target facial image data corresponding to the target object, and a second facial feature vector corresponding to the target facial image data is output, wherein the target facial image data includes at least one frame of target facial image; Utilizing the feature comparison model included in the target face recognition network, a comparison analysis is performed based on the first face feature vector and the second face feature vector, a target comparison analysis result is output, and a target identity authentication result is determined based on the target comparison analysis result, wherein the target comparison analysis result is used to reflect whether the object to be identified belongs to the target object, and the target identity authentication result is used to reflect whether the object to be identified has passed the identity authentication.

2. The identity authentication method based on face recognition according to claim 1, characterized in that: The step of using the feature mining model included in the target face recognition network to perform feature mining and feature enhancement on the face image data to be recognized, and outputting the first face feature vector corresponding to the face image data to be recognized, comprises: Using the convolution unit in the feature mining model included in the target face recognition network, convolution processing is performed on each frame of the face image to be recognized in the face image data to be recognized, and a first face convolution vector corresponding to each frame of the face image to be recognized is output, wherein the face image data to be recognized includes multiple frames of face images to be recognized; Utilizing the association mining unit in the feature mining model, performing association mining on a plurality of first face convolution vectors corresponding to a plurality of frames of face images to be recognized included in the face image data to be recognized according to the association relationship between the plurality of frames of face images to be recognized, and outputting a first face association vector corresponding to each of the first face convolution vectors; Superimposing the multiple first face association vectors corresponding to the multiple first face convolution vectors, and outputting the first face aggregation vector corresponding to the face image data to be recognized; Performing a plurality of different noise processing on the first face aggregation vectors respectively, and outputting a plurality of first face noise vectors corresponding to the face image data to be recognized; The multiple first face noise vectors are concatenated to form a first face feature vector corresponding to the face image data to be identified.

3. The identity authentication method based on face recognition according to claim 2, characterized in that: The step of using the association mining unit in the feature mining model to perform association mining on a plurality of first face convolution vectors corresponding to a plurality of frames of face images to be recognized included in the face image data to be recognized according to the association relationship between the plurality of frames of face images to be recognized, and outputting a first face association vector corresponding to each of the first face convolution vectors comprises: For each frame of the facial image to be recognized in the multiple frames of the facial image to be recognized included in the facial image data to be recognized, segmenting the first facial convolution vector corresponding to the facial image to be recognized to form multiple first facial local vectors corresponding to the facial image to be recognized; Utilizing the association mining unit in the feature mining model, performing association mining on a plurality of first face local vectors corresponding to a plurality of frames of face images to be recognized included in the face image data to be recognized according to the association relationship between the plurality of frames of face images to be recognized, and outputting an associated first face local vector corresponding to each of the first face local vectors; For each frame of the face image to be recognized in the multiple frames of the face image to be recognized included in the face image data to be recognized, multiple associated first face local vectors corresponding to the multiple first face local vectors corresponding to the face image to be recognized are combined to form a first face combination vector corresponding to the face image to be recognized, wherein the combination process and the segmentation process are mutually inverse processes; For each frame of face image to be identified, based on the first face combination vector corresponding to the face image to be identified, a focused feature mining process is performed on the first face convolution vector corresponding to the face image to be identified to form a first face association vector corresponding to the first face convolution vector.

4. The identity authentication method based on face recognition according to claim 3, characterized in that: The step of using the association mining unit in the feature mining model to perform association mining on a plurality of first face local vectors corresponding to a plurality of frames of face images to be recognized included in the face image data to be recognized according to the association relationship between the plurality of frames of face images to be recognized, and outputting an associated first face local vector corresponding to each of the first face local vectors comprises: Arrange the plurality of first face local vectors corresponding to each frame of the face image to be recognized in the plurality of frames of the face image to be recognized included in the face image data to be recognized according to the distribution position in the corresponding first face convolution vector and the order relationship between the corresponding face image to be recognized, to form a plurality of first face local vector sequences, wherein any two first face local vectors in the same first face local vector have the same distribution position in the corresponding first face convolution vector; For each of the first partial face vectors, association mining is performed on the first partial face vector according to the first partial face vector sequence in which the first partial face vector is located, and an associated first partial face vector corresponding to the first partial face vector is output.

5. The identity authentication method based on face recognition according to claim 4, characterized in that: The step of performing association mining on each of the first face local vectors according to the first face local vector sequence in which the first face local vector is located, and outputting the associated first face local vector corresponding to the first face local vector comprises: For a first first face local vector in the first face local vector sequence, taking the first face local vector as the corresponding associated first face local vector; For each other first human face local vector other than the first first human face local vector in the first human face local vector sequence that does not belong to it, based on the associated first human face local vector corresponding to the previous first human face local vector, focused feature mining processing is performed on the other first human face local vector to form an associated first human face local vector corresponding to the other first human face local vector.

6. The identity authentication method based on face recognition according to claim 2, characterized in that: The step of performing a plurality of different noise processing on the first face aggregation vectors respectively and outputting a plurality of first face noise vectors corresponding to the face image data to be recognized comprises: Based on the vector size of the first face aggregation vector, generating a plurality of random noise vectors having the same vector size, wherein in each of the random noise vectors, the number of non-zero vector parameters is less than the number of zero vector parameters; Respectively superimpose each of the multiple random noise vectors with the first face aggregation vector to form multiple first face noise vectors corresponding to the face image data to be identified; or, sort the multiple random noise vectors, and superimpose the random noise vector sorted as the first with the first face aggregation vector to form a first face noise vector corresponding to the random noise vector sorted as the first, and, for each other random noise vector except the random noise vector sorted as the first, perform self-attention processing on the first face noise vector corresponding to the previous random noise vector of the other random noise vector to obtain the corresponding self-attention vector, and superimpose the self-attention vector and the other random noise vector to form the first face noise vector corresponding to the other random noise vector.

7. The identity authentication method based on face recognition according to any one of claims 1 to 6, characterized in that: The step of using the feature mining model to perform feature mining and feature enhancement on the target face image data corresponding to the pre-stored target object, and outputting the second face feature vector corresponding to the target face image data, comprises: Using the convolution unit in the feature mining model, convolution processing is performed on each frame of the target face image in the target face image data, and a second face convolution vector corresponding to each frame of the target face image is output, wherein the target face image data includes multiple frames of target face images; Utilizing the association mining unit in the feature mining model, performing association mining on a plurality of second face convolution vectors corresponding to the plurality of frames of target face images included in the target face image data according to the association relationship between the plurality of frames of target face images, and outputting a second face association vector corresponding to each of the second face convolution vectors; Superimposing multiple second face association vectors corresponding to the multiple second face convolution vectors, and outputting a second face aggregation vector corresponding to the target face image data; Performing a plurality of different noise processing on the second face aggregation vectors respectively, and outputting a plurality of second face noise vectors corresponding to the target face image data; The multiple second face noise vectors are concatenated to form a second face feature vector corresponding to the target face image data.

8. An identity verification device based on face recognition, characterized in that: include: A face image acquisition module, used to acquire face image data to be identified formed by collecting face information of the object to be identified, wherein the face image data to be identified includes at least one frame of face image to be identified; A first feature mining module is used to perform feature mining and feature enhancement on the face image data to be identified by using a feature mining model included in a target face recognition network, and output a first face feature vector corresponding to the face image data to be identified, wherein the target face recognition network is a trained neural network; A second feature mining module is used to use the feature mining model to perform feature mining and feature enhancement on the pre-stored target face image data corresponding to the target object, and output a second face feature vector corresponding to the target face image data, wherein the target face image data includes at least one frame of target face image; A verification result determination module is used to utilize the feature comparison model included in the target face recognition network to perform a comparison analysis based on the first face feature vector and the second face feature vector, output a target comparison analysis result, and determine a target identity authentication result based on the target comparison analysis result, wherein the target comparison analysis result is used to reflect whether the object to be identified belongs to the target object, and the target identity authentication result is used to reflect whether the object to be identified has passed the identity authentication.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor connected to the memory is used to execute the computer program stored in the memory to implement the identity authentication method based on face recognition as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed, executes the identity authentication method based on face recognition as described in any one of claims 1 to 7.

Images