Face recognition method, apparatus and electronic device

Abstract

The application provides a face recognition method, device and electronic equipment, which can improve the one-time pass rate of face recognition. The method comprises: acquiring a first face image and a target face image of a user; extracting images of facial feature regions from the first face image and the target face image; taking images of the facial feature regions that meet a threshold requirement in similarity as candidate facial feature region images; taking the first face image as a base image, and fusing the candidate facial feature region images in corresponding regions of the base image with the base image to generate a second face image; and identifying the user according to the second face image. In the embodiment of the application, the one-time pass rate can be improved, and the user experience can be improved, because the difference between part of the facial feature regions and the target image is large when a single image is identified.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to a method, apparatus and electronic device for face recognition. Background Technology

[0002] With the development of computer and internet technology, face recognition technology has brought great convenience to users in verifying their identity.

[0003] In related technologies, face recognition methods generally involve face detection and feature extraction from the entire face image, similarity measurement based on the extracted features, and face image verification based on a set threshold.

[0004] However, during the recognition process, users cannot perceive the timing of the shooting. Therefore, the captured user's facial image may show changes in expression such as closed eyes, or changes in lighting or facial angle. This may cause some areas of the image to differ significantly from the target image, thus reducing the recognition rate. Even if the user adjusts their posture multiple times, the verification may still fail, resulting in a poor user experience.

[0005] Therefore, improving the first-pass success rate of facial recognition technology is a technical problem that urgently needs to be solved. Summary of the Invention

[0006] This application provides a method, apparatus, and electronic device for facial recognition, which can improve the first-pass rate of facial recognition and enhance user experience.

[0007] In a first aspect, a face recognition method is provided. The method includes acquiring a first face image of a user and a target face image, the first face image comprising at least two face images of the user; extracting images of facial feature regions from the first face image and the target face image, the facial feature regions comprising at least two regions selected from the left eyebrow, right eyebrow, left eye, right eye, nose, and mouth; determining the similarity between the images of the facial feature regions in the first face image and corresponding regions in the target face image based on image feature information of the facial feature regions, and selecting images of the facial feature regions whose similarity meets a threshold requirement as candidate facial feature region images; using the first face image as a base image, fusing the candidate facial feature region images with the base image within corresponding regions of the base image to generate a second face image; and recognizing the user based on the second face image.

[0008] Therefore, in this embodiment, facial feature regions with high similarity to the target image in multiple face images can be fused and then identified, avoiding the low recognition rate caused by some regions in the original face image being significantly different from the target image. This can improve the first-pass rate of face recognition and enhance the user experience.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the step of extracting images of facial feature regions from the first face image and the target face image includes: constructing a facial feature detector using an adaptive enhancement algorithm, wherein the facial feature detector is a filtering cascaded classifier; inputting the first face image and the target face image into the facial feature detector, wherein the facial feature detector sequentially detects different facial feature regions from the face images; and extracting images of different facial feature regions from the first face image and the target face image according to the position and size of the facial feature regions.

[0010] Therefore, in this embodiment of the application, an adaptive enhancement algorithm is used to construct a facial feature detector, which can accurately obtain facial feature region images. The facial feature regions with high similarity to the target image in multiple face images are fused together, avoiding the low recognition rate caused by some regions in the original face image being significantly different from the target image. This can improve the first-pass rate of face recognition and enhance the user experience.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the step of extracting the image of the facial feature region from the first face image and the target face image includes: acquiring facial features, wherein the facial features are features of the first face image and the target face image; using the facial features as input to a convolutional neural network extraction model, using a set of parameters of the facial feature as weights of the convolutional neural network extraction model, acquiring the facial feature detected by the convolutional neural network extraction model in the facial features, wherein the set of parameters of the facial feature includes facial feature parameters of different facial features, and using the facial feature parameters of the corresponding facial features when detecting features of different facial features; and extracting the image of the facial feature region based on the facial feature.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the image feature information of the facial features region includes the facial feature contour lines within the image of the facial features region. The step of determining the similarity between the image of the facial features region in the first face image and the corresponding region image of the target face image based on the image feature information of the facial features region includes: determining the similarity between the image of the facial features region in the first face image and the corresponding region image of the target face image based on the facial feature contour lines within the image of the facial features region. The facial feature contour lines are closed two-dimensional curves enclosing the facial features region.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, determining the similarity between the image of the facial feature region in the first face image and the corresponding region image of the target face image based on the facial feature contour lines within the image of the facial feature region includes: calculating the curvature K(t,σ) of the facial feature contour lines.

[0014]

[0015]

[0016]

[0017]

[0018]

[0019] Where t is an arbitrary parameter, and x(t,σ) and y(t,σ) are the results of convolving the abscissa x(t) and ordinate y(t) of each point on the facial contour line with the Gaussian function G(t,σ), respectively. t (t,σ) and x tt (t,σ) are the first and second derivatives of x(t,σ) with respect to t, respectively. t (t,σ) and y tt (t,σ) are the first and second derivatives of y(t,σ) with respect to t, respectively; σ is the standard deviation of the Gaussian function; v is a parameter; Δv is the step size; x(v) and y(v) are functions of v with respect to the horizontal and vertical coordinates on the facial contour line, respectively; the extreme point set c1 on the facial contour line in the first face image and the extreme point set c2 on the facial contour line in the target face image are determined by curvature detection; the similarity is determined based on the Hausdorff distance between the extreme point set c1 and the extreme point set c2.

[0020] In conjunction with the first aspect, in some implementations of the first aspect, using the first face image as the base image includes: using the first face image containing the most candidate facial feature regions as the base image.

[0021] Secondly, a face recognition device is provided, the device comprising: an image acquisition module for acquiring a first face image of a user and a target face image, the first face image including at least two face images of the user; a processor for extracting images of facial feature regions from the first face image and the target face image, the facial feature regions including at least two regions selected from left eyebrow, right eyebrow, left eye, right eye, nose, and mouth; the processor is further configured to determine, based on image feature information of the facial feature regions, the similarity between the images of the facial feature regions in the first face image and the corresponding regions of the target face image, and to select images of the facial feature regions whose similarity meets a threshold requirement as candidate facial feature region images; the processor is further configured to use the first face image as a base image, and to fuse the candidate facial feature region images with the base image in corresponding regions of the base image to generate a second face image; the processor is further configured to recognize the user based on the second face image.

[0022] In conjunction with the second aspect, in some implementations of the second aspect, the processor is specifically configured to: construct a facial feature detector using an adaptive enhancement algorithm, wherein the facial feature detector is a filtering cascaded classifier; input the first face image and the target face image into the facial feature detector, wherein the facial feature detector sequentially detects different facial feature regions from the face images; and extract images of different facial feature regions from the first face image and the target face image based on the position and size of the facial feature regions.

[0023] In conjunction with the second aspect, in some implementations of the second aspect, the processor is specifically configured to: acquire facial features, wherein the facial features are features of the first face image and the target face image; use the facial features as input to a convolutional neural network extraction model, use the parameter set of facial feature as weights of the convolutional neural network extraction model, acquire the facial feature detected by the convolutional neural network extraction model in the facial features, wherein the parameter set of facial feature includes facial feature parameters of different facial features, and the facial feature parameters of the corresponding facial features are used when detecting features of different facial features; and extract the image of the facial feature region based on the facial feature.

[0024] In conjunction with the second aspect, in some implementations of the second aspect, the image feature information of the facial features region includes the facial feature contour lines within the image of the facial features region. Specifically, the processor is used to: determine the similarity between the image of the facial features region in the first face image and the corresponding region image of the target face image based on the image feature information of the facial features region. The facial feature contour lines are closed two-dimensional curves enclosing the facial features region.

[0025] In conjunction with the second aspect, in some implementations of the second aspect, the step of determining the similarity between the image of the facial feature region in the first face image and the corresponding region image of the target face image based on the facial feature contour lines within the image of the facial feature region is specifically configured to: calculate the curvature K(t,σ) of the facial feature contour lines.

[0026]

[0027]

[0028]

[0029]

[0030]

[0031] Where t is an arbitrary parameter, and x(t,σ) and y(t,σ) are the results of convolving the abscissa x(t) and ordinate y(t) of each point on the facial contour line with the Gaussian function G(t,σ), respectively. t (t,σ) and x tt (t,σ) are the first and second derivatives of x(t,σ) with respect to t, respectively. t (t,σ) and y tt (t,σ) are the first and second derivatives of y(t,σ) with respect to t, respectively; σ is the standard deviation of the Gaussian function; v is a parameter; Δv is the step size; x(v) and y(v) are functions of v with respect to the horizontal and vertical coordinates on the facial contour line, respectively; the extreme point set c1 on the facial contour line in the first face image and the extreme point set c2 on the facial contour line in the target face image are determined by curvature detection; the similarity is determined based on the Hausdorff distance between the extreme point set c1 and the extreme point set c2.

[0032] In conjunction with the second aspect, in some implementations of the second aspect, the processor is specifically used to: use the first face image containing the most candidate facial feature regions as the base image.

[0033] Thirdly, a computer-readable medium is provided having a computer program stored thereon, characterized in that, when the program is executed by a computer, the computer performs the face recognition method of the first aspect or any possible implementation thereof.

[0034] Fourthly, a computer program product is provided, which, when executed by a computer, implements the face recognition method in the first aspect or any possible implementation of the first aspect.

[0035] Fifthly, an electronic device is provided, including means for implementing face recognition as disclosed in the second aspect or any possible implementation of the second aspect.

[0036] A sixth aspect provides an electronic device comprising: a storage device having a computer program stored thereon; and a processing device for executing the computer program in the storage device to implement the steps of the first aspect or any possible implementation thereof.

[0037] In a seventh aspect, a chip system is provided, characterized in that the chip system includes a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface to implement the method of the first aspect and any of its implementations. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating a face recognition method according to an embodiment of the present invention.

[0039] Figure 2 This is a flowchart illustrating the method for extracting images of the facial features.

[0040] Figure 3 This is a schematic diagram illustrating the process of extracting an image of the facial features region using an extraction model according to an embodiment of this application.

[0041] Figure 4 This is a schematic diagram illustrating the process of extracting images of facial features using an extraction model according to another embodiment of this application.

[0042] Figure 5 This is a schematic flowchart of a method for determining similarity according to an embodiment of this application.

[0043] Figure 6This is a schematic flowchart illustrating a method for generating a second face image based on a convolutional neural network image fusion model.

[0044] Figure 7 This is a schematic block diagram of a face recognition device according to an embodiment of this application.

[0045] Figure 8 This is a schematic diagram of the hardware structure of a face recognition device according to an embodiment of this application.

[0046] Figure 9 This is a schematic block diagram of an electronic device according to one embodiment of this application. Detailed Implementation

[0047] The technical solutions in this application will now be described with reference to the accompanying drawings. It is understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0048] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0049] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0050] It should be noted that the concepts of "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0051] It should be noted that the terms "a" and "a plurality of" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0052] The names of the messages or information exchanged between multiple devices in the embodiments of this application are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0053] It should also be understood that the various implementation methods described in this specification can be implemented individually or in combination, and the embodiments of this application are not limited in this respect.

[0054] Facial recognition technology can be used for user authentication in areas such as shopping and security checks. Specifically, facial recognition can be used for unlocking, payment, and various entertainment applications. Smart terminal devices, such as mobile phones, tablets, computers, and televisions, are mostly equipped with cameras. After capturing images containing faces, these images can be used for face detection and recognition, and the results can then be used to perform other related applications. However, during the process of capturing a user's face image, the user is unaware of the timing of the capture. Changes in facial expressions, such as closing the eyes, or changes in lighting or facial angle can occur during the capture, potentially leading to significant differences between parts of the image and the target image, thus reducing the recognition rate. Even if the user adjusts their posture multiple times, verification may still fail, resulting in a poor user experience.

[0055] This application first provides a face recognition method and apparatus based on facial feature image fusion. It utilizes extracted facial feature images with higher fitting degrees to the target image, fuses them with the face image, and performs face recognition based on the processed image. Therefore, it can improve the first-pass yield rate of face recognition. Furthermore, based on this, this application provides a task execution method and electronic device for an electronic device, which can perform different operations using the recognition results of the above-mentioned face recognition method, such as verification, unlocking, and payment. The embodiments of this application are described in detail below with reference to specific accompanying drawings.

[0056] Figure 1 This is a flowchart illustrating a face recognition method according to an exemplary embodiment of the present disclosure. For ease of understanding, it is first combined with... Figure 1 This paper provides a brief introduction to face recognition methods based on the fusion of facial feature images.

[0057] like Figure 1 As shown, the method includes the following steps:

[0058] 100. Obtain the user's first face image and the target face image, wherein the first face image includes at least two face images of the user.

[0059] The first facial image can be captured by a terminal with a camera function. This facial image can be captured by one camera or by multiple cameras.

[0060] The target facial image can be an ID photo or pre-stored image information, which can be used for comparison and reference after image processing.

[0061] 200, extract images of facial feature regions from the first face image and the target face image, wherein the facial feature regions include at least two regions selected from the left eyebrow, right eyebrow, left eye, right eye, nose, and mouth.

[0062] In some embodiments, before extracting images of facial features from the first face image and the target face image, it may be verified whether the user in the first face image and the target face image is the same person.

[0063] Preliminary verification of the first face image can prevent subsequent image processing of non-target face users, thus improving verification efficiency.

[0064] In one possible implementation, the geometric features of facial features in a face image can be obtained based on an adaptive boosting algorithm. Key points of facial features can be detected in the face image to determine the location and range of facial features. The facial feature region can include at least two regions among the left eyebrow, right eyebrow, left eye, right eye, nose, and mouth.

[0065] Specifically, an adaptive augmentation method is used to construct a facial feature detector, and a cascaded classifier using a waterfall algorithm is organized into a filtering structure. Each node in the cascade is a strong classifier trained with adaptive augmentation. A threshold b is set at each node in the cascade, allowing almost all corresponding facial feature samples to pass through, while rejecting the vast majority of non-corresponding facial feature samples. The nodes are arranged from simple to complex, with later nodes being more complex and containing more weak classifiers. This minimizes the computational cost when rejecting images but specific regions, ensuring a high detection rate and a low rejection rate for the classifier. A face image is input into the facial feature detector, which sequentially detects different facial feature regions. Based on the position and size of these regions, at least two facial feature regions are extracted from the face image, yielding images of these regions.

[0066] In one possible implementation, the active shape model method can also be used to locate the facial features region of the face image, but this application does not limit this method.

[0067] In one possible implementation, an active shape model feature point localization algorithm is used to determine the shape models of different facial features in the first face image and the target face image;

[0068] Based on the shape models of different facial features, Gabor wavelet transform, PCA (Principal Component Analysis), and LDA (Linear Discriminant Analysis) are performed sequentially on different facial features to obtain the feature information of different facial feature images.

[0069] In practical applications, when using a feature point localization algorithm based on an active shape model to determine the shape model of facial features, initial localization is first performed in the image. Then, for each initially located feature point, the exact location of each feature point is searched in the image and corrected according to the grayscale model of each feature point. After multiple searches and corrections, the determined shape model can better reflect the facial features.

[0070] In some embodiments, the facial landmark recognition model is trained through the following steps:

[0071] Obtain a training sample set, which includes face image samples and pre-annotated facial landmark information samples for the face image samples;

[0072] For the training samples in the training sample set, perform the following steps:

[0073] The face image samples in the training sample are input into the face feature recognition model; the image features extracted by the feature extraction layer of the face feature recognition model are obtained as image feature samples, and the obtained image feature samples and the training sample are used to form a new training sample; using machine learning methods, the face image samples and image feature samples included in the training samples in the new training sample are used as input, and the face key point information samples corresponding to the input face image samples and image feature samples are used as the expected output to train the face key point recognition model.

[0074] Alternatively, as an embodiment, images of facial features can be extracted from the first face image and the target face image based on a convolutional neural network image extraction model.

[0075] The convolutional neural network extraction model includes a set of facial feature extraction parameters. The set of extraction parameters used in the convolutional neural network extraction model is different when extracting images of different facial feature regions.

[0076] A convolutional neural network (CNN) is a deep neural network with a convolutional structure. A CNN contains a feature extractor consisting of convolutional layers and subsampling layers, which can be viewed as a filter. A convolutional layer is a layer of neurons in a CNN that performs convolutional processing on the input signal. In a convolutional layer of a CNN, a neuron may only be connected to some of its neighboring neurons. A convolutional layer typically contains several feature planes, each composed of a series of rectangularly arranged neural units. Neural units on the same feature plane share weights, which are called the convolutional kernel. Shared weights can be understood as the way image information is extracted regardless of location. The convolutional kernel can be initialized as a matrix of random size, and during the training process of the CNN, the kernel can learn appropriate weights. Furthermore, the direct benefit of shared weights is that it reduces the connections between layers in the CNN, while also reducing the risk of overfitting.

[0077] The following are examples, not limitations, in combination with Figure 2 Specific examples are provided to describe in detail the image extraction method for the facial features region according to embodiments of this application. For instance... Figure 2 The method shown can be performed by a device for extracting facial features images.

[0078] Accordingly, the facial feature image extraction device (hereinafter referred to as the "extraction device") in the embodiments of this application can be a device with facial feature image extraction function, including but not limited to monitoring devices, smartphones, cameras, video cameras, etc. The embodiments of this application are not limited to this, as long as the extraction device can implement the method of facial feature image extraction. Optionally, the extraction device can also be a chip.

[0079] 210. Obtain facial features, wherein the facial features are the features of the first facial image and the target facial image.

[0080] In some implementations, the first face image and the target face image can be input into a neural network for feature extraction. The extracted features are called face features. This feature extraction network can be a general neural network used for feature extraction, or it can be a redesigned neural network capable of extracting features.

[0081] One form of representation for facial features is a matrix. If the matrix corresponding to a facial feature is a one-dimensional matrix, the facial feature is represented as a vector, which can be called a facial feature vector. In one example, a facial feature vector can be 256-dimensional, with each dimension being a 32-bit floating-point number (fp32).

[0082] 220. The facial features are used as input to the convolutional neural network extraction model, and the parameter set of facial features is used as the weight of the convolutional neural network extraction model. The facial features detected by the convolutional neural network extraction model in the facial features are obtained. The parameter set of facial features includes facial feature parameters of different facial features. When detecting features of different facial features, the facial feature parameters of the corresponding facial features are used.

[0083] In some implementations, the facial feature parameters in the facial feature parameter set can be feature parameters of facial feature image samples pre-configured according to facial feature types. The type of facial feature image sample can be manually calibrated, for example, pre-defined by professionals based on prior experience. Optionally, the type of the facial feature image sample can also be pre-determined by the extraction device or other equipment according to predefined rules; however, this application embodiment is not limited to this.

[0084] Optionally, facial features may include the left eyebrow, right eyebrow, left eye, right eye, nose, and mouth.

[0085] Specifically, in this embodiment, the input to the convolutional neural network (CNN) image extraction model (hereinafter referred to as the "extraction model") is the facial features extracted from various face images, and the output of the extraction model is the facial features of different facial regions in each face image. In other words, after a face image is input into the extraction model, it is processed by the various layers in the extraction model, and finally outputs the image of the facial regions in the face image.

[0086] Optionally, when the image type is left eye, it can correspond to the first set of facial feature parameters; when the image patch type is right eye, it can correspond to the second set of facial feature parameters, and so on. Accordingly, the first set of facial feature parameters, the second set of facial feature parameters, the third set of facial feature parameters, and so on, can be facial feature parameters corresponding to different facial regions.

[0087] 230. Extract the image of the facial features region based on the facial features.

[0088] In the embodiments of this application, after obtaining the output of the extraction model, the region corresponding to the facial feature in the face image can be determined based on the output of the extraction model, and the image of the facial feature region can be extracted from the face image according to the region where the facial feature is located.

[0089] It should be understood that the selection of the region corresponding to the facial feature in the face image can be the smallest image region containing the facial feature.

[0090] Therefore, in this embodiment, different facial feature parameters are used to extract different types of facial feature regions through the extraction model, avoiding the shortcomings of uniform processing of the entire face image. This allows for corresponding extraction for different facial feature regions, thereby improving processing efficiency and accuracy.

[0091] It should be understood that the extraction model in this application embodiment can be a shallow CNN model, but this application embodiment is not limited to this.

[0092] It should be understood that in the embodiments of this application, the extraction model may be pre-trained. For example, the extraction model may be pre-trained by the extraction device. Alternatively, the extraction model may be obtained by the extraction device from other devices, and the extraction model may be pre-trained by those other devices, which may be other extraction devices or devices specifically used for training extraction models, etc. The embodiments of this application are not limited to these. By training the extraction model, the feature parameters of different facial features can be obtained.

[0093] The following describes in detail the specific process of training the extraction model by the extraction device or by other devices, depending on the specific circumstances.

[0094] Scenario 1: The extraction model was trained by the extraction device.

[0095] In this embodiment of the application, before the extraction device extracts the image of the facial features region from the first face image and the target face image, the method further includes:

[0096] Train the extraction model.

[0097] Specifically, such as Figure 3 The method for training this extraction model, as shown, includes:

[0098] 310. Obtain different face image samples and predefined types of facial feature image samples.

[0099] It should be understood that the predefined type of facial feature sample can be a facial feature image sample whose type has been predefined. For example, the type of facial feature image sample can be manually labeled, such as being predefined by professionals based on prior experience. Optionally, the type of the facial feature image sample can also be predetermined by the extraction device or other equipment according to predefined rules, and the embodiments of this application are not limited thereto.

[0100] Specifically, such as Figure 3As shown, the extraction device first acquires different face image samples and predefined types of facial feature image samples. For example, the predefined types of facial feature image samples include left eyebrow objects, right eyebrow objects, left eye objects, right eye objects, nose objects, or mouth objects. The facial feature image samples can be randomly collected from a sample database to form a training set of 300-500 sample images. The facial object features (including left eyebrow objects, right eyebrow objects, left eye objects, right eye objects, nose objects, or mouth objects) of all sample images in the training set are manually labeled.

[0101] 320. The extraction model is trained using the different face image samples and predefined types of facial feature image samples, respectively.

[0102] The extraction device inputs different face image samples and predefined types of facial feature image samples into the extraction model for parameter training. Specifically, it can train the corresponding parameters in each layer of the extraction model. For example, it can train the convolution kernels in the convolutional layers and the weight parameters in the fully connected layers of the extraction model, as well as the bias values ​​corresponding to the convolution kernels and weight parameters, but this embodiment is not limited to this. After training the extraction model, in practical applications, the extraction device inputs the face image to be extracted into the extraction model. After real-time processing by the extraction model, it finally extracts images of different facial feature regions from the face image.

[0103] It should be understood that after the extraction model is trained, the extraction device can directly use the extraction model to extract the image of the facial features region when extracting face images in subsequent operations, without needing to train the extraction model again. Alternatively, the extraction device may only need to periodically train and improve the extraction model, rather than training the extraction model every time an image is extracted. The embodiments of this application are not limited to this.

[0104] Scenario 2: The extraction model was trained by other devices.

[0105] In this embodiment of the application, before the extraction device extracts the image of the facial features region from the first face image and the target face image, the method further includes:

[0106] Obtain the extraction model.

[0107] Specifically, the extraction device first obtains the extraction model from another device. In this case, the extraction model may be pre-trained by the other device, or the extraction model may be obtained by the other device from another device and trained by that other device. The embodiments of this application are not limited to this.

[0108] like Figure 4As shown, the extraction device first obtains the extraction model (which is a pre-trained model) from the other device. Then, in practical applications, the extraction device inputs the face image to be extracted into the extraction model. After real-time processing by the extraction model, the images of different facial feature regions in the face image are finally extracted.

[0109] 300. Based on the image feature information of the facial features region, determine the similarity between the image of the facial features region in the first face image and the corresponding region image of the target face image, and take the image of the facial features region whose similarity meets the threshold requirement as the candidate facial features region image.

[0110] Optionally, the image feature information may be feature points or feature vectors extracted from the image, and this application does not limit this.

[0111] Optionally, the similarity may be determined by selecting at least one from a group consisting of Euclidean distance (the true distance between two points in m-dimensional space, or the natural length of a vector), cosine similarity (which assesses the similarity between two vectors by calculating the cosine of the angle between them; vectors are plotted in a vector space, such as the most common two-dimensional space, based on their coordinate values), and relative entropy (Kullback-Leibler divergence, an asymmetric measure of the difference between two probability distributions).

[0112] It should be understood that the similarity threshold can be arbitrarily selected. Preferably, the image with the highest similarity between the image of the facial features region in the first face image and the corresponding region image of the target face image can be used as the candidate facial features region image.

[0113] Optionally, the similarity between the image of the facial features in the first face image and the image of the corresponding region of the target face image is determined based on the cosine value between the feature vectors of the image of the facial features region and the feature vector of the corresponding region image of the target face image.

[0114] Specifically, the cosine distance between the feature vectors of different facial features in the first face image and the feature vectors of different facial features in the target face image can be calculated using the cosine similarity formula, thereby obtaining the similarity between the different facial features in the first face image and the target face image.

[0115] The formula for calculating cosine similarity is as follows:

[0116]

[0117] Where n represents the feature vectors of facial features in n target face images; A iW represents the i-th term of the facial feature vector in the first face image; i A represents the i-th term of the facial feature vector in the target face image; T Let ||A|| represent the transpose of the facial feature vector in the first face image; ||A|| represents the 2-norm of the facial feature vector A in the first face image, which is the square root of the sum of squares of the elements in the facial feature vector A in the first face image; ||W|| represents the 2-norm of the facial feature vector W in the target face image, which is the square root of the sum of squares of the elements in the facial feature vector W in the target face image; cosθ represents the cosine similarity value between the facial feature vectors in the first face image and the facial feature vectors in the target face image. The closer cosθ is to 1, the higher the similarity between the facial feature vectors in the first face image and the facial feature vectors in the target face image.

[0118] In some embodiments, the image feature information of the facial features region includes facial feature contour lines within the image of the facial features region. The step of determining the similarity between the image of the facial features region in the first face image and the corresponding region image of the target face image based on the image feature information of the facial features region includes: determining the similarity between the image of the facial features region in the first face image and the corresponding region image of the target face image based on the facial feature contour lines within the image of the facial features region. The facial feature contour lines are closed two-dimensional curves enclosing the facial features region.

[0119] It should be understood that facial contour lines within images that can identify different facial feature regions can be used to determine the similarity between the image of the facial feature region in the first face image and the corresponding region image of the target face image based on the Hausdorff distance between feature points on the facial feature contour lines.

[0120] Specifically, the facial contour line is a closed two-dimensional curve that encloses the facial feature area.

[0121] Optionally, the image of the facial features region can be processed to obtain the facial feature outlines, which are the boundary lines of facial features with significant contrast differences and large gray-scale gradient changes in the image.

[0122] Specifically, Hausdorff distance is a maximum-minimum distance defined on two finite point sets A = {n1, n2, ..., a}. p} and B = {b1,b2,…,b q}, then the Hausdorff distance between A and B is defined as:

[0123] H(A,B)=max{h(A,B),h(B,A)}

[0124] In the formula, h(A,B) is the directed Hausdorff distance from point set A to point set B, and h(B,A) can be deduced by analogy.

[0125] The following are examples, not limitations, in combination with Figure 5 A specific example is described in detail below: a method for determining the similarity between an image of the facial features region in a first face image and the corresponding region image of a target face image, according to an embodiment of this application.

[0126] Figure 5 This is a schematic flowchart of a method for determining similarity according to an embodiment of this application, such as... Figure 5 As shown, Figure 5 The method shown can be executed by a processor. The method includes the following steps:

[0127] 510, Calculate the curvature of the facial contour lines.

[0128] In one possible implementation, the facial contour lines can be the contour lines of the lips, the contour lines of the eyes, or the boundary lines of the eyebrows; this application does not limit this to any particular type.

[0129] Specifically, points are taken on the facial contour line function r = [x(t), y(t)], where t is an arbitrary parameter, x(t) is the abscissa of each point on the facial contour line, and y(t) is the ordinate of each point on the facial contour line.

[0130] Convolve the x-coordinate x(t) and y-coordinate y(t) of each point on the facial contour line with the Gaussian function G(t,σ) to obtain r. σ =(x(t,σ),y(t,σ)), calculate the curvature of the facial contour lines after noise removal:

[0131]

[0132]

[0133]

[0134]

[0135]

[0136] Where K(t,σ) is a function with the parameter t of the contour and the standard deviation σ in the Gaussian function as independent variables, x t (t,σ) and x tt (t,σ) are the first and second derivatives of x(t,σ) with respect to t, respectively. t (t,σ) and y tt(t,σ) are the first and second derivatives of y(t,σ) with respect to t, respectively. σ is the standard deviation in the Gaussian function, v is a parameter, Δv is the step size, and x(v) and y(v) are functions of v and the horizontal and vertical coordinates on the facial contour line, respectively. As the standard deviation σ in the Gaussian function increases, the amount of detail retained also gradually decreases.

[0137] 520, determine the feature points on the outline of the facial features.

[0138] Search for the point P with the greatest curvature on the outline of the facial features. max Its curvature value is denoted as K. max And let P max =2; rotate P clockwise. max The next point is taken as the starting point P. Its curvature K is then compared with the curvature of the subsequent points. If the curvature of the subsequent point Pi is still less than the curvature K of the current point, the comparison continues iteratively until the curvature of a subsequent point is greater than the curvature of the current point P. At this point, the current point is a local extremum, denoted as LP. i Its curvature is denoted as K(LP). i If its curvature K(LP) i If ) < 0, then let LP i =-2; if K(LP) i If ) > 0, then let LP i =1;

[0139] When the local extreme point LP i When the value is -2, let the point after it be the starting point P. i+1 The curvature of a point is compared one by one with the curvature of its next point. This alternating comparison process is repeated until a subsequent point has a curvature that is greater than that of the current point P. i+1 If the curvature is small, then the current point P... i+1 It is a local extremum point, denoted as LP. i+1 Its curvature is denoted as K(LP). i+1 If its curvature K(LP) i+1 If ) < 0, then let LP i+1 =2; if its curvature is less than K(LP) i+1 If ) < 0, then let LP i+1 =-1;

[0140] If LP i If a point is equal to 2 and satisfies one of the following two conditions, then the point is considered a feature point:

[0141] 1. At point K(LP) i+1 )>value (value is the threshold), and the curvature of its two adjacent points is less than 0;

[0142] 2. At point K(LP)i+1 )>value, and is also greater than twice the minimum curvature of its two adjacent points;

[0143] If LP i+1 If a point is equal to 2 and satisfies one of the following two conditions, then the point is considered a feature point:

[0144] 1. At point K(LP) i+1 If the value is less than -value, and the curvature of its two adjacent points is greater than 0;

[0145] 2. At point K(LP) i+1 ) <-value, and is smaller than twice the maximum curvature of its two adjacent points.

[0146] Specifically, similarity matching of contour lines can be performed based on the curvature of feature points on the contour lines.

[0147] Take the extreme point set c1 on the facial feature contour line in the first face image and the extreme point set c2 on the facial feature contour line in the target face image, respectively. The Hausdorff distance between point sets c1 and c2 is:

[0148]

[0149] 530, similarity is determined based on Hausdorff distance.

[0150] Specifically, the smaller the Hausdorff distance H(c1,c2), the higher the similarity between the facial feature outline and the facial feature outline in the target face image.

[0151] Therefore, this embodiment makes full use of the geometric feature information of the facial contour lines when judging the similarity of facial features, thereby improving the recognition efficiency.

[0152] 400, using the first face image as a base image, the candidate facial feature region image is fused with the base image within the corresponding region of the base image to generate a second face image.

[0153] In one possible implementation, when selecting the base image, any one of these first face images can be randomly chosen as the base image. This embodiment does not limit this.

[0154] Optionally, the first face image containing the most candidate facial feature regions can be used as the base image.

[0155] In one possible implementation, when fusing candidate facial feature region images with a base image, the candidate facial feature region images and the base image can be matched based on the same feature information. For example, when fusing an image of an open eye region with a base image of a closed eye region, a mole or wrinkle in that region can be used as a location. This is beneficial for using the common feature information as a reference for fusion, resulting in a more accurate generated image.

[0156] As an alternative approach, feature points of facial features in the target face image and the first face image can be used to find corresponding feature point pairs in the images to be fused. Based on these feature point pairs, the matching and fusion position of the two images can be determined. The fusion position can be the overlapping area between the candidate facial feature region image and the base image. For two images without overlapping areas, the fusion position is determined by calculating relevant feature points. Feature points are points in an image that have a special labeling function and can be implemented using relevant digital image processing algorithms or neural network models.

[0157] In one possible implementation, the second face image can be generated by fusing the candidate facial feature region images at the corresponding facial feature region positions in the base image, which can be based on a convolutional neural network image fusion model.

[0158] Specifically, such as Figure 6 As shown, the method for generating a second face image based on a convolutional neural network image fusion model includes:

[0159] 610, Obtain the training dataset.

[0160] Specifically, the training dataset may include different face image samples and their corresponding facial feature image samples in different poses.

[0161] 620. Construct a convolutional neural network image fusion model.

[0162] Specifically, image features of different face images and their corresponding facial features in different poses can be obtained based on the feature information extracted in step 300; the extracted features are fused (they can be directly concatenated), and the fused features are input into the registration decoder network to obtain registration parameters; using the obtained registration parameters, the facial feature image is transformed to correspond to the region of the face image; the transformed facial feature image is input into the encoder network for encoding; the encoded transformed facial feature image and the encoded target face image are input into the fusion layer for fusion; the obtained fused data is input into the reconstruction decoder network to obtain the final fused image.

[0163] Specifically, the reconstructed decoder network may include a first convolutional kernel, a second convolutional kernel, a third convolutional kernel, and a fourth convolutional kernel; the first convolutional kernel, the second convolutional kernel, the third convolutional kernel, and the fourth convolutional kernel are connected in series; the size of the first convolutional kernel is 64*64*3*3*3; the size of the second convolutional kernel is 64*32*3*3*3; the size of the third convolutional kernel is 32*16*3*3*3; the size of the fourth convolutional kernel is 16*1*3*3*3; the parameters are defined as the number of input channels * the number of output channels * the length * the width * the height of the convolutional kernel.

[0164] 630. The convolutional neural network image fusion model is trained using the training dataset to obtain the image fusion model;

[0165] It should be understood that after training the convolutional neural network image fusion model, when fusing facial feature images and face images in subsequent processes, the fusion model can be directly used to fuse the images without needing to train the extraction model again. Alternatively, the fusion device may only need to periodically train and improve the fusion model, rather than training it every time an image is fused. The embodiments of this application are not limited to this.

[0166] 640. Input the candidate facial feature region image and the background image into the image fusion model to complete image fusion.

[0167] Specifically, for example, if an image of the eye region of a user with open eyes is fused with an image of a face with closed eyes, the user's face image with open eyes can be obtained by using this image fusion model. Other facial feature images can also be adjusted using this image fusion model; however, this application is not limited to this.

[0168] Therefore, this application embodiment addresses the problem of low first-pass rate of face recognition caused by issues such as closed eyes, eyeglass reflections, open mouth, and lighting problems in the original face image. It can comprehensively generate a face image with high similarity to the facial features of the target face image, thereby improving the first-pass rate of face recognition and enhancing the user experience.

[0169] 500, the user is identified based on the second facial image.

[0170] In one possible implementation, the face recognition process is performed by comparing feature information obtained from a second face image with the target face image. This application is not limited to this embodiment.

[0171] Optionally, the user identification based on the second facial image can be performed on other devices, and this application embodiment does not limit this.

[0172] Based on the above technical solution, at least the following technical effects can be achieved:

[0173] It can acquire user face images and select candidate facial feature regions that have a similarity to the target face image exceeding a set value. By fusing the candidate facial feature regions with the base image, a user face image with a high similarity to the target face image is obtained. This can improve the first-pass rate of face recognition and prevent low recognition accuracy caused by facial expressions such as closed eyes in the extracted face image, thereby improving the first-pass rate and enhancing the user experience.

[0174] This invention also provides a schematic block diagram of a face recognition device, such as... Figure 7 As shown, the device 700 includes:

[0175] The image acquisition module 710 is used to acquire a first face image of the user and a target face image, wherein the first face image includes at least two face images of the user.

[0176] Optionally, the image acquisition module 710 can be any device for acquiring images, such as a camera, a webcam, etc.

[0177] The processor 720 is configured to extract images of facial feature regions from the first face image and the target face image, the facial feature regions including at least two regions selected from the left eyebrow, right eyebrow, left eye, right eye, nose, and mouth.

[0178] Optionally, the processor 720 is further configured to determine the similarity between the image of the facial features region in the first face image and the corresponding region image of the target face image based on the image feature information of the facial features region, and to use the image of the facial features region whose similarity meets the threshold requirement as a candidate facial features region image.

[0179] Optionally, the processor 720 is also used for image fusion, specifically for: using the first face image as a base image, fusing the candidate facial feature region image with the base image in the corresponding region of the base image to generate a second face image;

[0180] Optionally, the processor 720 is also configured to identify the user based on the second face image.

[0181] In one possible implementation, the processor 720 is based on an adaptive enhancement algorithm and is specifically used to extract images of facial feature regions from the first face image and the target face image, including: constructing a facial feature detector using the adaptive enhancement algorithm, wherein the facial feature detector is a filtering cascaded classifier; inputting the first face image and the target face image into the facial feature detector, wherein the facial feature detector sequentially detects different facial feature regions from the face images; and extracting images of different facial feature regions from the first face image and the target face image according to the position and size of the facial feature regions.

[0182] Optionally, in the step of extracting the image of the facial feature region from the first face image and the target face image, the processor 720 is further configured to: acquire face features, wherein the face features are features of the first face image and the target face image; use the face features as input to a convolutional neural network extraction model, use the parameter set of facial feature features as weights of the convolutional neural network extraction model, acquire the facial feature features detected by the convolutional neural network extraction model in the face features, wherein the parameter set of facial feature features includes facial feature parameters of different facial features, and the facial feature parameters of the corresponding facial features are used when detecting features of different facial features; and extract the image of the facial feature region based on the facial feature features.

[0183] Optionally, the image feature information of the facial features region includes the facial feature contour lines within the image of the facial features region. The processor 720 is further configured to: determine the similarity between the image of the facial features region in the first face image and the corresponding region image of the target face image based on the image feature information of the facial features region; wherein the facial feature contour lines are closed two-dimensional curves enclosing the facial features region.

[0184] In one possible implementation, the processor 720 further specifically uses the step of determining the similarity between the image of the facial features region in the first face image and the corresponding region image of the target face image based on the facial feature contour lines within the image of the facial feature region.

[0185]

[0186]

[0187]

[0188]

[0189]

[0190] Where t is an arbitrary parameter, and x(t,σ) and y(t,σ) are the results of convolving the abscissa x(t) and ordinate y(t) of each point on the facial contour line with the Gaussian function G(t,σ), respectively. t (t,σ) and x tt (t,σ) are the first and second derivatives of x(t,σ) with respect to t, respectively. t (t,σ) and y tt (t,σ) are the first and second derivatives of y(t,σ) with respect to t, respectively; σ is the standard deviation of the Gaussian function; v is a parameter; Δv is the step size; x(v) and y(v) are functions of v with respect to the horizontal and vertical coordinates on the facial contour line, respectively; the extreme point set c1 on the facial contour line in the first face image and the extreme point set c2 on the facial contour line in the target face image are determined by curvature detection; the similarity is determined based on the Hausdorff distance between the extreme point set c1 and the extreme point set c2.

[0191] Optionally, the processor 720 may specifically be used to: use the first face image containing the most candidate facial feature regions as the base image.

[0192] Based on the above technical solution, at least the following technical effects can be achieved:

[0193] It can acquire user face images, select candidate facial feature regions with high similarity to the target face image, and fuse the candidate facial feature regions with the base image to obtain user face images with high similarity to the target face image. Using the processed user face images for face recognition can improve the first-pass rate of the face recognition process and prevent low recognition accuracy caused by facial expressions such as closed eyes in the extracted face images. This can improve the accuracy of recognition and enhance the user experience.

[0194] Figure 8 This is a schematic diagram of the hardware structure of a face recognition device according to an embodiment of this application. Figure 8 The face recognition device 800 shown includes a memory 801, a processor 802, a communication interface 803, and a bus 804. The memory 801, processor 802, and communication interface 803 are interconnected via the bus 804.

[0195] The memory 801 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 801 may store a program. When the program stored in the memory 801 is executed by the processor 802, the processor 802 and the communication interface 803 are used to execute the various steps of the face recognition device of this application embodiment.

[0196] The processor 802 may be a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), graphics processing unit (GPU), or one or more integrated circuits, used to execute relevant programs to achieve the functions required by the units in the image extraction device of this application embodiment, or to execute the face recognition method of the method embodiment of this application.

[0197] The processor 802 can also be an integrated circuit chip with signal processing capabilities. In implementation, each step of the face recognition method in this embodiment can be completed through integrated logic circuits in the processor 802 or through software instructions.

[0198] The processor 802 described above can also be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or the execution of a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 801. The processor 802 reads the information in memory 801 and, in conjunction with its hardware, completes the functions required by the units included in the face recognition device of this application embodiment, or executes the face recognition method of the method embodiment of this application.

[0199] The communication interface 803 uses transceiver devices, such as, but not limited to, transceivers, to enable communication between the device 800 and other devices or communication networks. For example, an image to be processed can be acquired through the communication interface 803.

[0200] Bus 804 may include a pathway for transmitting information between various components of device 800 (e.g., memory 801, processor 802, communication interface 803).

[0201] It should be noted that although the above-described device 800 only shows a memory, processor, and communication interface, those skilled in the art should understand that in specific implementations, device 800 may also include other devices necessary for normal operation. Furthermore, depending on specific needs, those skilled in the art should understand that device 800 may also include hardware devices for implementing other additional functions. Moreover, those skilled in the art should understand that device 800 may only include the devices necessary for implementing the embodiments of this application, and may not necessarily include... Figure 8 All the devices shown.

[0202] like Figure 9 As shown, this application embodiment also provides an electronic device 900, which may include the device 700 described above. For example, the electronic device 900 is a smart door lock, mobile phone, computer, access control system, or other device that requires facial recognition. The device 700 includes software and hardware devices for facial recognition in the electronic device 900.

[0203] The terminal devices in this application embodiment may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 9 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0204] It should be understood that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0205] It is understood that the face recognition device in this application embodiment may further include a memory, which may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0206] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figures 1-6 The method of the illustrated embodiment.

[0207] It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0208] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0209] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0210] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire at least two Internet Protocol (IP) addresses; send a node evaluation request including the at least two IP addresses to a node evaluation device, wherein the node evaluation device selects an IP address from the at least two IP addresses and returns it; and receive the IP address returned by the node evaluation device; wherein the acquired IP address indicates an edge node in a content delivery network.

[0211] Alternatively, the aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: receive a node evaluation request including at least two Internet Protocol (IP) addresses; select an IP address from the at least two IP addresses; and return the selected IP address; wherein the received IP address indicates an edge node in the content delivery network.

[0212] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0213] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0214] This application also provides a computer program comprising instructions that, when executed by a computer, enable the computer to perform... Figure 1-6 The method of the illustrated embodiment.

[0215] For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart.

[0216] This application also provides a chip, which includes an input / output interface, at least one processor, at least one memory, and a bus. The at least one memory is used to store instructions, and the at least one processor is used to call the instructions in the at least one memory to execute them. Figure 1-6 The method of the illustrated embodiment. For example, the chip may be a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a system-on-chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0217] It should be understood that the processor can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor that reads software code stored in memory. The memory can be integrated with the processor, for example, the memory can be integrated into the processor, or the memory can be located outside the processor and exist independently.

[0218] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0219] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0220] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0221] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0222] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0223] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0224] It should be understood that the phrase "an embodiment" or "one embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in one embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence number of the above-described processes does not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0225] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed across two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored.

[0226] It should also be understood that the first, second, third, fourth, and various numerical designations used herein are merely distinctions for ease of description and are not intended to limit the scope of the embodiments of this application.

[0227] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0228] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0229] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0230] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0231] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0232] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0233] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0234] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0235] In summary, the above description is merely a preferred embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A face recognition method, characterized in that, include: Acquire a first face image and a target face image of the user, wherein the first face image includes at least two face images of the user, and the target face image is used for comparison and reference; The first face image is preliminarily verified based on the target face image to determine whether the user in the first face image and the user in the target face image are the same person. After determining that the user in the first face image and the user in the target face image are the same person, images of facial features are extracted from the first face image and the target face image. The facial features include at least two regions selected from the left eyebrow, right eyebrow, left eye, right eye, nose, and mouth. Based on the image feature information of the facial feature region, the similarity between the image of the facial feature region in the first face image and the corresponding region image of the target face image is determined. The image feature information of the facial feature region includes the facial contour lines within the image of the facial feature region. Determining the similarity between the image of the facial feature region in the first face image and the corresponding region image of the target face image based on the image feature information of the facial feature region includes: Based on the facial feature contour lines within the image of the facial feature region, the similarity between the image of the facial feature region in the first face image and the corresponding region image of the target face image is determined, wherein the facial feature contour line is a closed two-dimensional curve enclosing the facial feature region; Images of the facial features region whose similarity meets the threshold requirement are used as candidate facial features region images. Using the first face image as a base image, the candidate facial feature region image is fused with the base image in the corresponding region of the base image to generate a second face image; The user is identified based on the second facial image.

2. The method according to claim 1, characterized in that, The step of extracting facial feature regions from the first face image and the target face image includes: An adaptive enhancement algorithm is used to construct a facial sensory detector, which is a screening-type cascaded classifier. The first face image and the target face image are input into the facial feature detector, and the facial feature detector sequentially detects different facial feature regions from the face image; Based on the position and size of the facial features, images of different facial features are extracted from the first face image and the target face image.

3. The method according to claim 1, characterized in that, The step of extracting facial feature regions from the first face image and the target face image includes: Obtain facial features, wherein the facial features are features of the first facial image and the target facial image; The facial features are used as input to the convolutional neural network extraction model, and the parameter set of facial features is used as the weight of the convolutional neural network extraction model. The facial features detected by the convolutional neural network extraction model in the facial features are obtained. The parameter set of facial features includes facial feature parameters of different facial features. When detecting features of different facial features, the facial feature parameters of the corresponding facial features are used. The image of the facial features region is extracted based on the facial features.

4. The method according to claim 1, characterized in that, The step of determining the similarity between the image of the facial features in the first face image and the corresponding region image of the target face image based on the facial feature contour lines within the image of the facial feature region includes: The curvature of the facial contour lines is calculated using the facial contour lines. , Where t is an arbitrary parameter, and These are the x-coordinate x(t) of each point on the facial contour line, the y-coordinate y(t) of each point on the facial contour line, and the Gaussian function, respectively. The result of convolution, and They are The first and second derivatives with respect to t, and They are For the first and second derivatives of t, σ is the standard deviation in the Gaussian function, v is a parameter, Δv is the step size, and x(v) and y(v) are functions of v and the horizontal and vertical coordinates on the facial contour line, respectively. The extreme point set c1 on the facial feature contour line in the first face image and the extreme point set c2 on the facial feature contour line in the target face image are determined by curvature detection. The similarity between the image of the facial features region in the first face image and the corresponding region image of the target face image is determined based on the Hausdorff distance between the extreme point set c1 and the extreme point set c2.

5. The method according to any one of claims 1 to 4, characterized in that, Using the first face image as a base image includes: The first face image containing the most candidate facial feature regions is used as the base image.

6. A face recognition device, characterized in that, include: The image acquisition module is used to acquire a first face image of the user and a target face image. The first face image includes at least two face images of the user, and the target face image is used for comparison and reference. The processor is configured to perform preliminary verification on the first face image based on the target face image to determine whether the user in the first face image and the user in the target face image are the same person. After determining that the user in the first face image and the user in the target face image are the same person, the processor extracts images of facial feature regions from the first face image and the target face image. The facial feature regions include at least two regions selected from the left eyebrow, right eyebrow, left eye, right eye, nose, and mouth. The processor is further configured to determine, based on the image feature information of the facial feature region, the similarity between the image of the facial feature region in the first face image and the corresponding region image of the target face image, wherein the image feature information of the facial feature region includes the facial feature contour lines within the image of the facial feature region, and the processor is specifically configured to: Based on the facial feature contour lines within the image of the facial feature region, the similarity between the image of the facial feature region in the first face image and the corresponding region image of the target face image is determined, wherein the facial feature contour line is a closed two-dimensional curve enclosing the facial feature region. The processor is further configured to use images of the facial features region whose similarity meets the threshold requirement as candidate facial features region images; The processor is further configured to use the first face image as a base image, and fuse the candidate facial feature region image with the base image in the corresponding region of the base image to generate a second face image; The processor is also configured to identify the user based on the second facial image.

7. The apparatus according to claim 6, characterized in that, The processor is specifically used to extract facial feature regions from the first face image and the target face image, wherein the processor is configured to: An adaptive enhancement algorithm is used to construct a facial sensory detector, which is a screening-type cascaded classifier. The first face image and the target face image are input into the facial feature detector, and the facial feature detector sequentially detects different facial feature regions from the face image; Based on the position and size of the facial features, images of different facial features are extracted from the first face image and the target face image.

8. The apparatus according to claim 6, characterized in that, The processor is specifically used to extract facial feature regions from the first face image and the target face image, wherein the processor is configured to: Obtain facial features, wherein the facial features are features of the first facial image and the target facial image; The facial features are used as input to the convolutional neural network extraction model, and the parameter set of facial features is used as the weight of the convolutional neural network extraction model. The facial features detected by the convolutional neural network extraction model in the facial features are obtained. The parameter set of facial features includes facial feature parameters of different facial features. When detecting features of different facial features, the facial feature parameters of the corresponding facial features are used. The image of the facial features region is extracted based on the facial features.

9. The apparatus according to claim 6, characterized in that, The processor is specifically used to determine the similarity between the image of the facial features in the first face image and the corresponding region image of the target face image, based on the facial feature contour lines within the image of the facial feature region. The curvature of the facial contour lines is calculated using the facial contour lines. , Where t is an arbitrary parameter, and These are the x-coordinate x(t) of each point on the facial contour line, the y-coordinate y(t) of each point on the facial contour line, and the Gaussian function, respectively. The result of convolution, and They are The first and second derivatives with respect to t, and They are For the first and second derivatives of t, σ is the standard deviation in the Gaussian function, v is a parameter, Δv is the step size, and x(v) and y(v) are functions of v and the horizontal and vertical coordinates on the facial contour line, respectively. The extreme point set c1 on the facial feature contour line in the first face image and the extreme point set c2 on the facial feature contour line in the target face image are determined by curvature detection. The similarity is determined based on the Hausdorff distance between the extreme point set c1 and the extreme point set c2.

10. The apparatus according to any one of claims 6 to 9, characterized in that, The processor is specifically used to: use the first face image as a base image. The first face image containing the most candidate facial feature regions is used as the base image.

11. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by a computer, the computer performs the face recognition method as described in any one of claims 1-5.

12. An electronic device, characterized in that, include: The face recognition apparatus as described in any one of claims 6 to 10.

13. A chip system, characterized in that, The chip system includes a processor and a data interface. The processor reads instructions stored in the memory through the data interface to execute the method as described in any one of claims 1 to 5.

Images