A system and method for face image recognition based on the nine-square principle

Abstract

The present invention provides a system and method for face image recognition based on the Nine Palace Principle. First, the system performs machine learning on a set of face images in a known library. Each face image is sequentially converted to grayscale, normalized, divided into palaces, and pixel feature values ​​of each palace are calculated. The feature values ​​of each palace are embedded into a one-dimensional Nine Palace vector to achieve lightweight storage in a mobile device. During operation, the mobile device takes frames of the video collected by the camera in real time. This frame of the picture is quickly converted into a one-dimensional Nine Palace vector, and the similarity between the Nine Palace vector set stored in the mobile device is calculated. When the similarity is greater than the preset recognition matching degree, the match is successful, the face recognition is completed, and the system obtains the recognition result. The machine learning process of this system for face images is simple, and does not require complex mathematical algorithms. It can achieve lightweight calculation and lightweight data set storage, reduce equipment costs, and improve the working efficiency of the recognition device.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a system and method for face image recognition based on the nine-square principle. Background Art

[0002] With the development of science and technology and the needs of modern life, face recognition technology has been increasingly used, especially in access control systems, customs inspections, criminal investigation monitoring, etc. It has become an indispensable artificial intelligence assistant. The advantage of face recognition lies in its naturalness and the fact that it is not perceived by the individual being tested. Compared with fingerprint recognition and iris recognition, which are not natural, face recognition has many advantages that other recognition cannot match. However, it also has many difficulties. One of the difficulties is that the appearance of the face is very unstable. People can produce many expressions through changes in the face, and the visual image of the face varies greatly from different observation angles. In addition, face recognition is also affected by many factors such as lighting conditions; the second difficulty is that the current face recognition algorithms with high accuracy require powerful processors for machine learning, which restricts the use of the algorithm in harsh environments. For example, in areas without networks or 4G, if border guards need to use face recognition equipment for regional monitoring or on-site face comparison, they cannot connect to and use powerful cloud servers for online face recognition. Although it is also possible to equip expensive offline equipment to conduct offline learning of facial image databases, and then use mobile devices to connect to offline equipment for monitoring and facial comparison, the high cost of the equipment and the complexity of system use have restricted the equipment rate of facial recognition system equipment.

[0003] The popular deep learning-based image processing methods and real-time video processing require powerful CPUs, GPUs or FPGAs, and small microcontrollers cannot process their complex algorithm calculations quickly or at all. Under certain conditions, this not only seriously affects the progress of work, but also reduces the possibility of equipment use, and increases the possibility of encountering dangers in security work.

[0004] In addition to deep learning, Matthew Turk and Alex Pentland proposed a method for recognizing eigenfaces in the Journal of cognitive neuroscience in 1991.

[0005] "Eigenfaces for recognition", this method has a certain anti-noise ability. The image reproduced by PCA looks clearer than the original noisy image, reducing data redundancy and reducing the storage space required for the image. However, it treats all samples as a whole during training to find a projection matrix with the minimum mean square error, while ignoring the category attributes. It is very sensitive to lighting, rotation, and posture, and needs to add new training sets. It is very sensitive to scale and relatively sensitive to background. In 2004, Timo Ahonen, Abdenour Hadid, and Matti Pietikainen proposed "Face Recognition with Local Binary Patterns". Its main idea is to use LBP+k-nearest neighbor classification for face recognition. The advantage is that it is more robust to changes in lighting conditions. The disadvantage is that when the image resolution is high, the feature vector dimension is too high, which slows down the calculation. In some cases, it will miss the local structure because it does not consider the influence of the central pixel and has poor anti-noise ability. In 2006, Aly M. proposed "Face Recognition using SIFT Features", which uses SIFT features to recognize faces. Its advantage is that it remains invariant to rotation, scaling, and brightness changes, but it is quite complex in calculation and generally requires GPU support, and does not support lightweight computing devices. In 2015, Schroff, F., Kalenichenko, D., and Philbin, J. proposed the concept of Facenet at the conference "Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition": "Facenet: a unified embedding for face recognition and clustering", which is also one of the main algorithms for face recognition services currently provided by GOOGLE. Its main idea is to embed the image into a vector and then judge it by Euclidean distance. The advantage is that it proposes a new loss function triplet loss. The input of this function is three pictures, two of which are of the same identity and the third is of a different identity. This loss function makes the feature distance between the same identity as small as possible and the different ones as large as possible. But the disadvantages are also obvious. There are many possible combinations of triplets, the convergence is slow, and it is unstable, and a special method is needed to select triplets. Even so, this new loss function greatly increases the amount of calculation. In addition, the paper uses a large non-public dataset for training, which is 3 orders of magnitude larger than the general public dataset.

[0006] In summary, if the machine learning and real-time comparison test of face recognition can be completed through lightweight image computing, the performance of the face recognition system and its monitoring efficiency will be greatly improved, and the cost of the equipment will be reduced. Summary of the invention

[0007] The purpose of the present invention is to provide a face image processing system and method based on the Nine-square principle, which only requires lightweight computing to achieve machine learning of face images and real-time matching of faces at work, namely face recognition.

[0008] To achieve the above purpose, the present invention provides a system and method for face image recognition based on the Nine-square principle, the system includes a single-chip microcomputer or a lightweight mobile device, a camera and an SD card. The memory of the SD card or the mobile device stores a set of face images to be machine-learned or directly stores a set of one-dimensional vectors that have been processed by the Nine-square principle; when working, the single-chip microcomputer collects face images through the camera, processes the Nine-square principle, and then calculates the similarity with the image that has been machine-learned. If the similarity is higher than a preset threshold, the face image is successfully matched.

[0009] The process of the recognition method includes face image machine learning and real-time matching of face images when the device is working.

[0010] Machine learning for face images includes the following steps:

[0011] Step 1: The system imports the face image learning set;

[0012] Step 2: Convert the image to grayscale, converting the three-primary color or four-primary color images of different formats into a unified grayscale image;

[0013] Step 3: Normalize the image grayscale and normalize the pixel value of the image to the range of 0-255, Xnew=((X-Xmin) / (Xmax-Xmin))*255, X is the original image pixel value, Xmin is the minimum pixel value of the original image, Xmax is the maximum pixel value of the original image, and Xnew is the normalized pixel value. Through normalization, an image that is too bright or too dark can be adjusted to a uniform and normal image;

[0014] Step 4: Image palace division, that is, divide the image into n*n parts, n is the nine palace parameter;

[0015] Step 5: Calculate the eigenvalue in each palace;

[0016] Step 6: embed the eigenvalues ​​of each palace into a one-dimensional vector;

[0017] Step 7: Store the one-dimensional vector of each image in the MCU SD card or the memory of the mobile device.

[0018] Face image matching, or face recognition, includes the following steps:

[0019] Step 8: The single chip microcomputer takes out the face image collected by the camera in real time, i.e. the image to be matched;

[0020] Step nine: Perform nine-square processing on the image to be matched to form a one-dimensional vector with the same length as the image in the single-chip computer database;

[0021] Step 10: Calculate the similarity between the image to be matched and the image in the single-chip computer database;

[0022] Step 11: If the similarity is higher than the preset threshold R1, the match is successful, and the stored information is: the match is successful with the X face, where X is the face code or number in the learning set;

[0023] Step 12: If the similarity is lower than the preset threshold R1, the values ​​lower than the average values ​​of the respective vectors in the nine-square one-dimensional vectors of the image to be tested and the database image are taken out and recombined into a new one-dimensional vector. Because the grayscale value 0 is black and 255 is white in the grayscale image, the values ​​lower than the average values ​​of the respective vectors are the relatively darker parts in the grayscale image.

[0024] Step 13: Calculate the similarity of the two new one-dimensional vectors;

[0025] Step 14: If the similarity is higher than the preset threshold R2, the match is successful, and the stored information is: the match with the face of X is successful, and the final judgment information is pending; because the face images collected in actual work will be offset or deformed from the face images in the database, the system extracts and matches the grayer parts to avoid missed judgments;

[0026] Step 15: If the similarity is lower than the preset threshold R2, the match is unsuccessful, and the system extracts the nine-square one-dimensional vector of the next face image in the database for similarity calculation.

[0027] The present invention realizes lightweight face image matching calculation. Real-time face image matching only needs to calculate the similarity of one-dimensional vectors, and machine learning only needs to convert face images into one-dimensional vectors, which reduces the calculation amount of the single-chip microcomputer. In actual operation, an ordinary single-chip microcomputer with a 16MHz crystal oscillator, an SD card and a camera can realize all the work described in this patent. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a structural schematic diagram of a system and method for face image recognition based on the nine-square principle of the present invention.

[0029] Figure 2The present invention discloses a facial image machine learning flow chart of a facial image recognition system and method based on the nine-square principle.

[0030] Figure 3 The present invention discloses a face image real-time matching flow chart of a face image recognition system and method based on the nine-square principle. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] The specific implementation of the present invention will be described below in conjunction with the accompanying drawings.

[0033] Figure 1 This is a structural diagram of a system and method for face image recognition based on the Nine-square principle, and its structure includes a single-chip microcomputer or a lightweight mobile device, a camera and an SD card. The memory of the SD card or the mobile device stores a set of face images to be machine-learned or directly stores a set of one-dimensional vectors that have been processed by the Nine-square principle. When working, the single-chip microcomputer collects face images through the camera, processes them by the Nine-square principle, and then calculates the similarity with the image that has been machine-learned. If the similarity is higher than the preset threshold, the face image is successfully matched.

[0034] Figure 2 The present invention is a process part of face image machine learning of a system and method for face image recognition based on the Nine Grid Principle, including the following steps:

[0035] Step 1: The system imports the face image learning set;

[0036] Step 2: Convert the image to grayscale, convert the three-primary color or four-primary color images of different formats into a unified grayscale image, for example, convert the RGB three-channel photo into a GRAY single-channel image;

[0037] Step 3: Normalize the image grayscale and normalize the pixel value of the image to the range of 0-255, Xnew=((X-Xmin) / (Xmax-Xmin))*255, X is the original image pixel value, Xmin is the minimum pixel value of the original image, Xmax is the maximum pixel value of the original image, and Xnew is the pixel value after normalization. Through normalization, an image that is too bright or too dark can be adjusted to a unified and normal image. For example, if the original pixel value is 122-255, after normalization, Xnew=((X-122) / (255-122))*255, and the new pixel value range is 0-255;

[0038] Step 4: Image palace division, that is, dividing the image into n*n parts, n is the nine palace parameter, for example, when n=15, the image is divided into 15*15=225 small parts, that is, 255 palaces;

[0039] Step 5: Calculate the eigenvalue in each square, that is, the average value and median of all pixels. For example, for 255 squares, 255 average values ​​and 255 medians are calculated;

[0040] Step 6: embed the mean and median of each palace into a one-dimensional vector. For example, there are 510 mean and median values ​​for 255 palaces, forming a sequence of 1*510, i.e. a one-dimensional vector.

[0041] Step 7: Store the one-dimensional vector of each image into the MCU database.

[0042] Figure 3 The present invention is a face image matching part of a face image recognition system and method based on the Nine Palaces principle, comprising the following steps:

[0043] Step 8: The single chip microcomputer or mobile device takes out the face image collected in real time by the camera, i.e. the image to be matched;

[0044] Step nine: Perform nine-square processing on the image to be matched to form a one-dimensional vector with the same length as the image in the single-chip computer database;

[0045] Step 10: Calculate the similarity between the image to be matched and the image in the MCU database, i.e., the cosine similarity:

[0046]

[0047] Step 11: If the similarity is higher than the preset threshold R1, the match is successful, and the stored information is: the match is successful with the X face, where X is the face code or number in the learning set;

[0048] Step 12: If the similarity is lower than the preset threshold R1, the values ​​lower than the average values ​​of the respective vectors in the nine-square one-dimensional vectors of the image to be tested and the database image are taken out and recombined into a new one-dimensional vector. Because the grayscale value 0 is black and 255 is white in the grayscale image, the values ​​lower than the average values ​​of the respective vectors are the relatively darker parts in the grayscale image.

[0049] Step 13: Calculate the cosine similarity of the two new one-dimensional vectors;

[0050] Step 14: If the cosine similarity is higher than the preset threshold R2, the match is successful, and the stored information is: the match with the face of person X is successful, and the final judgment information is pending; because the images collected in actual work will be offset from the images in the database, the system extracts and matches the higher gray parts to avoid missed judgments;

[0051] Step 15: If the cosine similarity is lower than the preset threshold R2, the match is unsuccessful and the system extracts the next image from the database for matching.

Claims

1. A method for face image recognition based on the Nine Grids principle, characterized in that: The method comprises the following processing steps: Step 1: The face image machine learning part starts, and the system imports the face image learning set; Step 2: Convert the image to grayscale and convert it into a uniform grayscale image; Step 3: Normalize the image grayscale; Step 4: Image palace division, that is, dividing the image into n*n parts, where n is the nine-palace parameter, which is a positive integer; Step 5: Calculate the eigenvalue of each palace; Step 6: embed the eigenvalues ​​of each palace into a one-dimensional vector; Step 7: Store the one-dimensional vector of each image into the MCU database or the memory of the mobile device, and the machine learning part of the nine-square face image is completed; Step 8: The face image matching work starts, and the single chip microcomputer or mobile device takes out the face image collected by the camera in real time, that is, the image to be matched; Step nine: Perform nine-square processing on the image to be matched to form a one-dimensional vector with the same length as the image in the single-chip computer database; Step 10: Calculate the similarity between the image to be matched and the image in the single-chip computer database; Step 11: If the similarity is higher than the preset threshold R1, the match is successful, and the stored information is: the match is successful with the X situation, where the X situation is the situation corresponding to the matching image in the database; Step 12: If the similarity is lower than the preset threshold R1, the values ​​lower than the average values ​​of the respective vectors in the nine-square one-dimensional vectors of the image to be tested and the database image are respectively taken out and recombined into a new one-dimensional vector; Step 13: Calculate the similarity of two new one-dimensional vectors; Step 14: If the similarity is higher than the preset threshold R2, the match is successful, and the stored information is: the match with the X situation is successful, and the final judgment information is pending; Step 15: If the similarity is lower than the preset threshold R2, the match is unsuccessful and the system extracts the next face image in the database for matching.

2. The method for facial image recognition based on the Nine-square principle according to claim 1, characterized in that: In the step 5, the characteristic value of each palace includes the average value and the median value of all pixels in the palace; in the steps 10 to 15, the similarity includes the cosine similarity.

Images