A vehicle face image alignment method based on feature matching

Abstract

The application provides a vehicle face image alignment method based on feature matching, and belongs to the technical field of vehicle face image alignment, and comprises the following steps: obtaining a vehicle face image dataset; extracting image feature points of each image in the vehicle face image dataset to obtain a vehicle face image feature point set of each vehicle face image, and performing feature point matching on multiple vehicle face image feature point sets; clustering the vehicle face images by using a DBSCAN algorithm; selecting a sample with the minimum sum of distances from other vehicle face images in the same class as a vehicle face template image from each class; performing feature point matching on a vehicle face image to be aligned and multiple vehicle face template images, and selecting a vehicle face template image with the highest similarity to the vehicle face image to be aligned; calculating a homography matrix between the two sets of image feature points; and performing geometric transformation on the vehicle face image to be aligned by using the homography matrix to obtain an alignment result of the vehicle face image. The method can realize alignment of vehicle face images.

Description

Technical Field

[0001] This invention belongs to the field of vehicle face image alignment technology, specifically relating to a vehicle face image alignment method based on feature matching. Background Technology

[0002] Vehicle face recognition is a technology that uses computer technology to identify different individuals by analyzing the image information of a vehicle's face. The general steps of vehicle face recognition are: image data acquisition, image preprocessing, vehicle face localization, vehicle face image normalization, vehicle face feature extraction, and feature matching.

[0003] In vehicle face recognition, to ensure the consistency between the vehicle face feature library and the features of the vehicle face images used for recognition, it is essential to guarantee that the relative positions of the standardized vehicle face image, including headlights, windows, and license plates, are accurately registered with the standard vehicle face. The vehicle face registration problem refers to whether the positions of key components in the test image and the training image match during vehicle face recognition. Most incorrectly identified samples can be correctly identified by precisely adjusting the vehicle face position; that is, the degradation in vehicle face recognition performance is largely due to the inaccuracy of the automatically located vehicle face position. Therefore, vehicle face alignment is necessary to ensure correct sample recognition.

[0004] The existing vehicle face alignment method is based on MTCNN, a deep learning-based face detection and alignment method. This method analyzes MTCNN, selecting four feature points: the lower left and lower right corners of the windshield, and the lower left and lower right corners of the front of the vehicle. It trains a model on the labeled training set data and performs an affine transformation on the output to achieve vehicle face alignment.

[0005] However, the MTCNN method is supervised, requiring a large number of labeled samples to learn the feature point detection model for vehicle faces. Furthermore, its vehicle feature points are manually defined, making it impossible to make judgments based on more detailed features.

[0006] In summary, existing technologies, such as MTCNN, are supervised and require a large number of labeled samples to learn the feature point detection model for vehicle faces. Furthermore, their vehicle feature points are manually defined, making it impossible to make judgments based on more detailed features. Summary of the Invention

[0007] To overcome the shortcomings of the existing technology, the present invention provides a vehicle face image alignment method based on feature matching.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A vehicle face image alignment method based on feature matching includes:

[0010] Obtain the vehicle face image dataset;

[0011] The feature point extraction network is used to extract the image feature points of each image in the vehicle face image dataset, so as to obtain the feature point set of each vehicle face image, and feature point matching is performed on the feature point sets of multiple vehicle face images.

[0012] The DBSCAN algorithm is used to cluster vehicle face images. The clustering process includes: calculating the distance between vehicle face images based on the matching distance between the feature point sets of vehicle face images, and clustering vehicle face images with close distances into one class.

[0013] Find the sample in each category that has the smallest sum of distances to other car face images in the same category and use it as the car face template image;

[0014] The vehicle face image to be aligned is matched with multiple vehicle face template images by feature point matching, and the vehicle face template image with the highest similarity to the vehicle face image to be aligned is selected.

[0015] Based on the feature point matching relationship between the car face template image with the highest similarity and the car face image to be aligned, the homography matrix H between the two sets of image feature points is calculated;

[0016] Geometric transformations are performed on the vehicle face images to be aligned using the homography matrix to obtain the alignment results.

[0017] Furthermore, the process of obtaining the vehicle face image dataset includes: reading vehicle images collected at traffic checkpoints, detecting and obtaining the vehicle face region, cropping the obtained vehicle face image portion, and obtaining the vehicle face image dataset.

[0018] Furthermore, the step of selecting the car face template image with the highest similarity includes: selecting the match with the lowest distance based on the distance between the feature points, which is the car face template image with the highest similarity.

[0019] Furthermore, the homography matrix H is calculated as follows:

[0020] Given n sets of matching points: The conversion relationship between them is:

[0021]

[0022] Among them, (u i ,v i ) and (x i ,y i ) are the pixel coordinates of the car face template image with the highest similarity and the car face image to be aligned, respectively, and [h1,h2,…,h9] is the homography matrix H;

[0023] Based on the above conversion relationships, the following system of linear equations about h1, h2, ..., h9 can be obtained:

[0024]

[0025] Right now:

[0026]

[0027] Based on this set of 2n linear equations, the following system of linear equations can be constructed:

[0028]

[0029] The coefficient matrix of this linear system of equations is denoted as A. By performing singular value decomposition on A, the eigenvector with the smallest eigenvalues ​​of A is [h1,h2,…,h9], which is the homography matrix H.

[0030] Furthermore, the algorithm for performing geometric transformation on the vehicle face image to be aligned is as follows:

[0031]

[0032] Where (x,y) are the pixel coordinates of the image to be aligned, and (u,v) are the coordinates of (x,y) after alignment with the template.

[0033] Furthermore, the Superglue feature point extraction network is used to extract feature points from the vehicle face image.

[0034] The vehicle face image alignment method based on feature matching provided by this invention has the following beneficial effects:

[0035] This invention uses a feature point extraction network to extract feature points with rich details and obvious features of the vehicle face. It uses the DBSCAN clustering algorithm to cluster the vehicle face images. After clustering, it finds the vehicle face image with the smallest sum of distances to other samples in the same class as the vehicle face template image. It then performs feature point matching between the input vehicle face image and each vehicle face template image, selects the vehicle face template image with the highest similarity, and performs geometric transformation on the vehicle face based on the vehicle face template image with the highest similarity, thereby obtaining the alignment result of the vehicle face image.

[0036] This method is unsupervised and does not require any labeled samples. It only requires unsupervised learning to obtain the vehicle face template image and its feature point set. Compared with the manually defined vehicle feature points in the MTCNN method, the feature points obtained by this method are completely data-driven and can discover feature points that are difficult to determine manually. Furthermore, the number of feature points is not predetermined but is determined based on the characteristics of the vehicle face image itself, making it more accurate than existing technologies. Attached Figure Description

[0037] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart of a vehicle face image alignment method based on feature matching according to an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of feature point extraction according to an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of feature point matching according to an embodiment of the present invention;

[0041] Figure 4 This is a feature point image of the vehicle face template from an embodiment of the present invention;

[0042] Figure 5 This is a diagram illustrating the vehicle face alignment process according to an embodiment of the present invention. Detailed Implementation

[0043] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0044] Example:

[0045] like Figure 1-5 As shown:

[0046] This invention provides a vehicle face image alignment method based on feature matching, specifically as follows: Figure 1 As shown, it includes:

[0047] Read vehicle images collected at traffic checkpoints to obtain vehicle face images, perform modular preprocessing on the vehicle face images to obtain a vehicle face image dataset.

[0048] The Superglue feature point extraction network is used to extract the image feature points of each image in the vehicle face image dataset, and the feature point matching algorithm is used to match the feature points of the vehicle face images to obtain the matching relationship between the feature point sets of the vehicle face images.

[0049] The matching distance between the feature point sets of the vehicle face image is calculated based on the matching relationship between them.

[0050] The DBSCAN algorithm is used to cluster the vehicle face samples to obtain a set of feature points for the vehicle face template image.

[0051] The clustering process calculates the distance between two vehicle face images based on the matching distance between feature point sets of two vehicle face images. Images that are close in distance have high similarity and are clustered into one class.

[0052] Find the best car face image in each category as the cluster center, i.e., the car face template image. Selecting the best car face image includes selecting the sample with the smallest sum of distances to other samples in the same category.

[0053] The vehicle face image to be aligned is matched with multiple vehicle face template images using feature point matching, and the vehicle face template image with the highest similarity is selected. Specifically, selecting the vehicle face template image with the highest similarity involves selecting the match with the lowest distance based on the distance between the feature points, which is the vehicle face template image with the highest similarity.

[0054] Based on the matching relationship between the feature points of the car face template image with the highest similarity and the car face image to be aligned, the homography matrix H between the two is calculated using the least squares method:

[0055]

[0056] Where H is the homography matrix.

[0057] Specifically, this invention uses the direct linear transformation method to estimate the homography matrix, and the calculation method is as follows: Assume there are n sets of matching points: The conversion relationship between them is:

[0058]

[0059] Among them, (u i ,v i ) and (x i ,y i ) are the pixel coordinates of the car face template image with the highest similarity and the car face image to be aligned, respectively, and [h1,h2,…,h9] is the homography matrix H;

[0060] Based on the above conversion relationships, the following system of linear equations about h1, h2, ..., h9 can be obtained:

[0061]

[0062] Right now:

[0063]

[0064] Based on this set of 2n linear equations, the following system of linear equations can be constructed:

[0065]

[0066] The coefficient matrix of this linear system of equations is denoted as A. By performing singular value decomposition on A, the eigenvector with the smallest eigenvalues ​​of A is [h1,h2,…,h9], which is the homography matrix H.

[0067] The algorithm for geometrically transforming the car face using the homography matrix H is as follows:

[0068]

[0069] Where (x,y) are the pixel coordinates of the image to be aligned, and (u,v) are the coordinates of (x,y) after alignment with the template.

[0070] like Figure 2 This is a schematic diagram of image feature point extraction in this embodiment. Depending on actual needs, the Superglue feature extraction network can be used, but is not limited to, to extract feature points of the vehicle face image.

[0071] The above-described embodiments are merely preferred embodiments of the present invention, and the scope of protection of the present invention is not limited thereto. Any simple changes or equivalent substitutions of the technical solutions that can be obviously obtained by those skilled in the art within the scope of the technology disclosed in the present invention shall fall within the scope of protection of the present invention.

Claims

1. A feature matching based vehicle face image alignment method, characterized in that, The method comprises the following steps: acquiring a vehicle face image dataset; extracting image feature points of each image in the vehicle face image dataset using a feature point extraction network to obtain a vehicle face image feature point set of each vehicle face image, and performing feature point matching on a plurality of vehicle face image feature point sets; performing clustering on the vehicle face images using a DBSCAN algorithm, wherein the clustering process comprises: calculating the distance between vehicle face images according to the matching distance between vehicle face image feature point sets, and clustering vehicle face images with similar distances into one category; selecting a sample with the smallest sum of distances to other vehicle face images in the category as a vehicle face template image from each category; performing feature point matching on the vehicle face image to be aligned and a plurality of vehicle face template images, and selecting a vehicle face template image with the highest similarity to the vehicle face image to be aligned; calculating a homography matrix H between the two sets of image feature points based on the feature point matching relationship between the vehicle face template image with the highest similarity and the vehicle face image to be aligned; performing geometric transformation on the vehicle face image to be aligned using the homography matrix to obtain the alignment result of the vehicle face image.

2. The method of claim 1, wherein, The method for acquiring the vehicle face image dataset comprises the following steps: reading vehicle pictures collected by a traffic camera, detecting a vehicle face region, cropping a vehicle face image part to obtain a vehicle face image dataset.

3. The method of claim 1, wherein, The method for selecting a vehicle face template image with the highest similarity to the vehicle face image to be aligned comprises the following steps: selecting a matching with the lowest distance according to the distance of feature point matching, and the matching is the vehicle face template image with the highest similarity.

4. The method of claim 1, wherein, The calculation method of the homography matrix H is as follows: There are n sets of matching points: The conversion relationship between them is: where (u i ,v i ) and (x i ,y i ) are the pixel coordinates of the most similar car face template image and the car face image to be aligned, respectively, and [h1, h2, …, h9] is a homography matrix H. According to the above conversion relationship, the following linear equation group about h1, h2, …, h9 can be obtained: That is: According to the total 2n linear equations, the following linear equation group can be constructed: The coefficient matrix of the linear equation group is denoted as A, and the eigenvector with the smallest eigenvalue of A obtained by singular value decomposition of A is [h1, h2, …, h9], that is, the homography matrix H.

5. The method of claim 1, wherein, The algorithm for performing geometric transformation on the vehicle face image to be aligned using the homography matrix is as follows: Wherein, (x, y) is the pixel coordinate of the image to be aligned, and (u, v) is the coordinate of (x, y) after alignment to the template.

6. The method of claim 1, wherein, The Superglue feature point extraction network is used to extract vehicle face image feature points.

Images