Online car-hailing driver face verification algorithm
By designing a face verification algorithm for online ride-hailing drivers, using feature extraction networks and classification networks, combining convolutional layers and residual connections, the loss function is optimized to enhance the cohesion of feature vectors, and the problem of lack of cohesion of feature vectors in the prior art is solved, and face verification with high accuracy is achieved.
Patent Information
- Application Number
- CN202411838683.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-13
- Publication Date
- 2025-04-25
AI Technical Summary
In the prior art, in the face verification of online car-hailing drivers, the feature vector lacks cohesion, which leads to difficulties in determining whether it passes the verification in the face verification of open sets.
By designing a face verification algorithm for online ride-hailing drivers, using feature extraction networks and classification networks, combining convolutional layers and residual connections, the loss function is optimized to enhance the cohesion of feature vectors. The specific steps include extracting and classifying facial images during the training stage, calculating losses and optimizing network parameters, and calculating the cos distance of the feature vector in the testing stage to determine whether it is the same person.
The stronger cohesion of the eigenvectors is achieved, the training overhead is reduced, and the accuracy of 98.59% is achieved on the test set.
Smart Images
Figure CN119888807A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing and face verification, and in particular to a face verification algorithm for online car-hailing drivers. Background Art
[0002] With the rapid development of mobile Internet and the rise of the online car-hailing industry, driver identity verification has become an important part of ensuring passenger safety. In this context, the present invention constructs a set of online car-hailing driver face verification algorithms based on face-related technologies. The algorithm has been applied to online car-hailing platforms as a necessary process for driver registration and employment. During the driver registration stage, the algorithm can ensure that the identity information submitted by the driver matches the real face to prevent false identity registration. During the driver employment stage, the algorithm can continuously monitor the driver's identity, strengthen the passengers' sense of security and trust, and ensure the passengers' safe travel.
[0003] Based on whether the face set that can be obtained during model training and the face set that needs to be verified after going online are consistent, face verification includes two categories: closed set face verification and open set face verification.
[0004] Closed-set Face Verification is a verification performed on a known limited face database. In closed-set face verification, the system already knows in advance that the face to be verified belongs to a person in the database, and only needs to determine whether the input face matches this known identity. For example, in an employee access control system, it only needs to verify whether the employee's identity matches the registered identity.
[0005] Open-set Face Verification is a verification performed in an unknown face database. In open-set face verification, the system not only needs to verify the authenticity of the face, but also needs to determine whether the input face belongs to a new user outside the known face database. In this case, the system needs to be able to identify known and unknown identities and perform further processing as needed. For example, in a security monitoring system in a public place, it is necessary to verify whether the face belongs to a known suspect or a person on the blacklist. In the online car-hailing scenario, old drivers are constantly leaving and new drivers are joining every day, so it belongs to open-set face verification;
[0006] For closed-set face verification, since the set of users to be verified is fixed, we can ensure that the training set contains all the users to be verified. At this time, the face verification problem is treated as a multi-classification problem, that is, each user is a category, and any face image is passed through the model to obtain a category result, and determine whether the estimated category is consistent with the actual category. If they are consistent, the verification is passed. In multi-classification problems, softmax and cross entropy loss functions (hereinafter collectively referred to as softmax loss) are usually used to optimize model parameters.
[0007] In open-set face verification, the existing technical idea is to perform face detection first and then face verification. In the face detection stage, the system automatically detects whether there is a face in the input image and determines the position and size of the face; in the face recognition stage, the system extracts the facial features in the image and then compares them with the pre-registered facial features to determine whether the person is the pre-registered person. The specific process is as follows:
[0008] (1) The original input image is passed through the face detection model to detect one or more faces, and each face image is marked with a rectangular frame. At the same time, five facial key points are detected, namely the left eye, right eye, nose tip, and both sides of the lips.
[0009] (2) Then, face alignment is performed based on the detected faces and key points.
[0010] (3) The aligned faces are sent to the face verification model for feature extraction and a feature vector of fixed length is output.
[0011] (4) At the same time, feature vectors are also obtained based on the facial photos registered in the database.
[0012] Finally, the similarity of the two feature vectors is calculated. If it is higher than a certain threshold, they are judged to be one person, otherwise the verification fails.
[0013] Current technical solutions usually transfer the softmax loss to the optimization process of the open set face verification model to train the model parameters and obtain the feature vector of the face based on this model. We observed this vector and found that the feature vectors corresponding to the face images of different users are separable, but lack cohesion. Separability means that there is a hyperplane in the vector space that better distinguishes the two categories; cohesion means that the intra-class distance is small and the inter-class distance is large. Ideally, the maximum intra-class distance of each class is also smaller than the minimum inter-class distance.
[0014] like Figure 4As shown in the figure, each point represents a feature vector (assuming that the dimension of each vector is 2), and a color represents a user category. The left side is only separable, and it can be seen that there are many cases where the intra-class distance is greater than the inter-class distance, such as AB belongs to the same user category, but the distance of AB is greater than the distance of AC; while the distribution of the feature vectors on the right is cohesive.
[0015] For closed-set face verification, the lack of cohesive feature vectors is not a problem, but for open-set face verification, the judgment of whether the verification is passed is based on the distance between the feature vectors of two faces, so the feature vector is required to be cohesive.
[0016] In existing technical solutions, the cohesion of feature vectors is improved through various methods, such as using center loss to optimize the loss function, that is, the loss of each feature vector is the distance between the vector and the class center of the true category to which it belongs. Center loss makes the feature vectors of each category closer to each other.
[0017] In addition, cohesion can be enhanced by constructing binary sample data and then optimizing contrastive loss. Specifically, a sample contains two input images. If the two images are of a person, it is a positive sample, otherwise it is a negative sample. If it is a positive sample, the Euclidean distance between the feature vectors corresponding to the two samples is minimized; if it is a negative sample, the greater the distance, the greater the loss, and the loss is 0 only when it reaches a certain threshold.
[0018] There is also a solution based on triplet sample data and triplet loss to enhance cohesion. Randomly select image anchors, positive sample faces of the same user category as the anchor, and negative sample faces of different categories from the anchor. The triplet loss requires that the distance d1 between the anchor and the positive is small, the distance d2 between the anchor and the negative is large, and d2-d1 must be above a certain threshold for the loss to be zero.
[0019] The above solutions all have many problems. Center only minimizes the distance within the class, but does not measure the distance between classes. There may be a situation where the distance between the centers of two classes is very close. Contrastive loss and Triplet loss are based on positive and negative sample pairs for training, which requires secondary construction of the original samples, resulting in an explosive growth in the total number of samples. It is necessary to select appropriate samples from all sample combinations for training, which is easy to fall into the local optimum. Therefore, a face verification algorithm for online car-hailing drivers is proposed. Summary of the invention
[0020] In view of this, an embodiment of the present invention hopes to provide a face verification algorithm for online car-hailing drivers to solve or alleviate the technical problems existing in the prior art and at least provide a beneficial option.
[0021] The technical solution of the embodiment of the present invention is implemented as follows: A face verification algorithm for online car-hailing drivers, comprising the following steps:
[0022] S1. Face images include 100,000 photos of drivers registered on the online car-hailing platform. Face images need to be annotated to form a data set. The database contains 2-10 face images corresponding to each driver during daily login, with a total of about 500,000 images.
[0023] S2. During the training phase, the face images in the data set are extracted through the feature extraction network to obtain feature vectors, which are then sent to the classification network for classification. The loss is then calculated, and the network parameters are optimized by minimizing the loss function.
[0024] S3. During the testing phase, the login face photo and the face photo in the database are sent to the feature extraction network to obtain two feature vectors, the cosine distance of the two feature vectors is calculated, and it is determined whether they are the same person based on the cosine distance.
[0025] In some embodiments, in S1, the specific form of the data set is a tuple (x, y), where x is the driver's facial photo, the facial photo is a registered photo or a photo taken during daily login, and y is the user category, that is, the corresponding driver ID. The data set does not need to undergo secondary construction and can be directly trained using the original sample data.
[0026] In some embodiments, in S2 and S3, a feature extraction model is designed based on convolution in the feature extraction network, and multiple convolution layers and residual connections are superimposed in the convolution design.
[0027] In some embodiments, in S2, the loss function normalizes the feature vector e of the input face image and the standard vector w of each category in the classification network;
[0028] Introduce parameter factor m to act on the angle between e and w;
[0029] Finally, the loss function is:
[0030]
[0031] Among them, N represents the sample size of face images in the training set, and each sample is a binary group (x i ,y i ), x i Indicates the input face image, y iIndicates the corresponding category, that is, which driver the picture is;
[0032] After the input face image is extracted, the feature vector e is obtained; y i The standard category vector corresponding to the category is w; here both e and w are normalized;
[0033] θ represents the angle between vectors e and w;
[0034] m is the introduced hyperparameter;
[0035] After normalization, both the standard vector and the eigenvector are mapped to a hypersphere with a diameter of 1, and the eigenvector of each driver is projected to a small area of the hypersphere.
[0036] In some embodiments, when m=1, there are 2 categories of angle ranges, where
[0037] The standard vectors for driver categories 1 and 2 are w1 and w2, and their angle bisector is p;
[0038] For an input face x, its category is y and its feature vector is e;
[0039] The acute angles between e and w1 and w2 are θ1 and θ2 respectively, and the angle between w1 and w2 is θ 12 ;
[0040] When the model estimates the category of the input face x, it calculates the inner product of the feature vector e corresponding to x and the standard vector of each category, and takes the category corresponding to the maximum value of the inner product as the estimated category;
[0041] Since the present invention normalizes both e and w, the inner product is equivalent to the angle θ between e and w;
[0042] Let the category corresponding to sample x be y, and the value range of y is {1, 2};
[0043] If e is between w1 and p, the predicted category is 1;
[0044] If e is between w2 and p, the predicted category is 2;
[0045] The angle bisector p is the classification boundary of categories 1 and 2. The angle range of each category is The inter-class distance between class 1 and class 2 is 0.
[0046] In some embodiments, when m=2, there are 2 categories of angle ranges, where
[0047] p1 and p2 are the trisection lines of the angle between w1 and w2;
[0048] Assuming y = 1, if cos(2θ1)≥cos(θ2) is obtained, then 2θ1≤θ2 is required;
[0049] At the same time, we know that θ1+θ2=θ 12 ;
[0050] therefore,
[0051] Similarly, if y = 2, then
[0052] That is, the angle range of category 1 is w1 to p1, the angle range of category 2 is w2 to p2, and the angle range of each category is The angular separation between the two classes is
[0053] The embodiment of the present invention has the following advantages due to the adoption of the above technical solution:
[0054] The present invention does not need to construct bigrams and triplets through this algorithm, and only trains on original face samples, which greatly reduces the training overhead and makes the feature vector more cohesive. The face feature extraction model is trained based on the original data set, which completely avoids secondary sample construction and improves the cohesion of the feature vector, achieving an accuracy rate of 98.59% on the test set.
[0055] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0057] Figure 1 A flow chart of training and testing for the present invention;
[0058] Figure 2 This is the angle difference structure diagram of the present invention when m=1;
[0059] Figure 3 This is the angle difference structure diagram of m=2 of the present invention;
[0060] Figure 4 Schematic diagram of the separability and cohesion of the prior art. DETAILED DESCRIPTION
[0061] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.
[0062] It should be noted that the terms "first", "second", "symmetrical", "array", etc. are only used to distinguish between description and position description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the definition of "first", "symmetrical", etc. can explicitly or implicitly include one or more of these features; similarly, when the quantity of certain features is not limited in the form of words such as "two" or "three", it should be noted that this feature also explicitly or implicitly includes one or more feature quantities;
[0063] In the present invention, unless otherwise clearly specified and limited, the terms such as "installation", "connection", "fixation" and the like should be understood in a broad sense; for example, it can be a fixed connection, a detachable connection, or an integral molding; it can be a mechanical connection, a direct connection, welding, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood based on the description and drawings combined with specific circumstances.
[0064] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.
[0065] like Figure 1-Figure 3 As shown, an embodiment of the present invention provides a face verification algorithm for online car-hailing drivers, comprising the following steps:
[0066] S1. Face images include 100,000 photos of drivers registered on the online car-hailing platform. Face images need to be annotated to form a data set. The database contains 2-10 face images corresponding to each driver during daily login, with a total of about 500,000 images.
[0067] S2. During the training phase, the face images in the data set are extracted through the feature extraction network to obtain feature vectors, which are then sent to the classification network for classification. The loss is then calculated, and the network parameters are optimized by minimizing the loss function.
[0068] S3. During the testing phase, the login face photo and the face photo in the database are sent to the feature extraction network to obtain two feature vectors, the cosine distance of the two feature vectors is calculated, and it is determined whether they are the same person based on the cosine distance.
[0069] In this embodiment, specifically, in S1, the specific form of the data set is a tuple (x, y), where x is the driver's facial photo, the facial photo is a registered photo or a photo taken during daily login, and y is the user category, that is, the corresponding driver ID. The data set does not need to undergo secondary construction and can be directly trained using the original sample data.
[0070] In this embodiment, specifically, in S2 and S3, a feature extraction model is designed based on convolution in the feature extraction network, and multiple convolution layers and residual connections are superimposed in the convolution design, as follows:
[0071]
[0072]
[0073] In this embodiment, specifically, in S2, the loss function normalizes the feature vector e of the input face image and the standard vector w of each category in the classification network;
[0074] Introduce parameter factor m to act on the angle between e and w;
[0075] Finally, the loss function is:
[0076]
[0077] Among them, N represents the sample size of face images in the training set, and each sample is a binary group (x i ,y i ), x i Indicates the input face image, y i Indicates the corresponding category, that is, which driver the picture is;
[0078] After the input face image is extracted, the feature vector e is obtained; y i The standard category vector corresponding to the category is w; here both e and w are normalized;
[0079] θ represents the angle between vectors e and w;
[0080] m is the introduced hyperparameter;
[0081] After normalization, both the standard vector and the eigenvector are mapped to a hypersphere with a diameter of 1. The eigenvector of each driver is projected to a small area of the hypersphere. The process of optimizing the model parameters is actually to learn the angular changes of the eigenvectors of different drivers, occupying a certain angle range, and requiring the angle difference between drivers to be as large as possible. As m increases, this angle difference will gradually increase.
[0082] In this embodiment, specifically, Figure 2 As shown, when m=1, the angle range of 2 categories, where
[0083] The standard vectors for driver categories 1 and 2 are w1 and w2, and their angle bisector is p;
[0084] For an input face x, its category is y and its feature vector is e;
[0085] The acute angles between e and w1 and w2 are θ1 and θ2 respectively, and the angle between w1 and w2 is θ 12 ;
[0086] When the model estimates the category of the input face x, it calculates the inner product of the feature vector e corresponding to x and the standard vector of each category, and takes the category corresponding to the maximum value of the inner product as the estimated category;
[0087] Since the present invention normalizes both e and w, the inner product is equivalent to the angle θ between e and w;
[0088] Let the category corresponding to sample x be y, and the value range of y is {1, 2};
[0089] If e is between w1 and p, the predicted category is 1;
[0090] If e is between w2 and p, the predicted category is 2;
[0091] The angle bisector p is the classification boundary of categories 1 and 2. The angle range of each category is The inter-class distance between class 1 and class 2 is 0.
[0092] In this embodiment, specifically, Figure 3 As shown, when m = 2, the angle ranges of the two categories, where p1 and p2 are the trisection lines of the angle between w1 and w2;
[0093] Assuming y = 1, if cos(2θ1)≥cos(θ2) is obtained, then 2θ1≤θ2 is required;
[0094] At the same time, we know that θ1+θ2=θ 12 ;
[0095] therefore,
[0096] Similarly, if y = 2, then
[0097] That is, the angle range of category 1 is w1 to p1, the angle range of category 2 is w2 to p2, and the angle range of each category is The angular separation between the two classes is
[0098] In this embodiment, specifically, by Figure 2 and Figure 3 By comparing the loss function, we can see that the designed loss function makes the feature vectors of different faces of each driver closer, while the distance between classes increases.
[0099] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of various changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A face verification algorithm for online car-hailing drivers, characterized in that: The following steps are involved: S1. Face images include 100,000 photos of drivers registered on the online car-hailing platform. Face images need to be annotated to form a data set. The database contains 2-10 face images corresponding to each driver during daily login, with a total of about 500,000 images. S2. During the training phase, the face images in the data set are extracted through the feature extraction network to obtain feature vectors, which are then sent to the classification network for classification. The loss is then calculated, and the network parameters are optimized by minimizing the loss function. S3. During the testing phase, the login face photo and the face photo in the database are sent to the feature extraction network to obtain two feature vectors, the cosine distance of the two feature vectors is calculated, and it is determined whether they are the same person based on the cosine distance.
2. The face verification algorithm for online car-hailing drivers according to claim 1 is characterized by: In S1, the specific form of the data set is a tuple (x, y), where x is the driver's face photo, which is a registered photo or a photo taken during daily login, and y is the user category, that is, the corresponding driver ID. The data set does not need to undergo secondary construction and can be directly trained using the original sample data.
3. The face verification algorithm for online car-hailing drivers according to claim 1 is characterized by: In S2 and S3, a feature extraction model is designed based on convolution in the feature extraction network, and multiple convolution layers and residual connections are superimposed in the convolution design.
4. The face verification algorithm for online car-hailing drivers according to claim 1 is characterized by: In S2, the loss function normalizes the feature vector e of the input face image and the standard vector w of each category in the classification network; Introduce parameter factor m to act on the angle between e and w; Finally, the loss function is: Among them, N represents the sample size of face images in the training set, and each sample is a binary group (x i ,y i ), x i Indicates the input face image, y i Indicates the corresponding category, that is, which driver the picture is; After the input face image is extracted, the feature vector e is obtained; y i The standard category vector corresponding to the category is w; here both e and w are normalized; θ represents the angle between vectors e and w; m is the introduced hyperparameter; After normalization, both the standard vector and the eigenvector are mapped to a hypersphere with a diameter of 1, and the eigenvector of each driver is projected to a small area of the hypersphere.
5. The face verification algorithm for online car-hailing drivers according to claim 4 is characterized by: When m=1, there are two categories of angle ranges, where The standard vectors for driver categories 1 and 2 are w1 and w2, and their angle bisector is p; For an input face x, its category is y and its feature vector is e; The acute angles between e and w1 and w2 are θ1 and θ2 respectively, and the angle between w1 and w2 is θ 12 ; When the model estimates the category of the input face x, it calculates the inner product of the feature vector e corresponding to x and the standard vector of each category, and takes the category corresponding to the maximum value of the inner product as the estimated category; Since the present invention normalizes both e and w, the inner product is equivalent to the angle θ between e and w; Let the category corresponding to sample x be y, and the value range of y is {1, 2}; If e is between w1 and p, the predicted category is 1; If e is between w2 and p, the predicted category is 2; The angle bisector p is the classification boundary of categories 1 and 2. The angle range of each category is The inter-class distance between class 1 and class 2 is 0.
6. The face verification algorithm for online car-hailing drivers according to claim 4 is characterized by: When m=2, the angle range of 2 categories, where p1 and p2 are the trisection lines of the angle between w1 and w2; Assuming y = 1, if cos(2θ1)≥cos(θ2) is obtained, then 2θ1≤θ2 is required; At the same time, we know that θ1+θ2=θ 12 ; therefore, Similarly, if y = 2, then That is, the angle range of category 1 is w1 to p1, the angle range of category 2 is w2 to p2, and the angle range of each category is The angular separation between the two classes is