A face recognition method for public figures

Abstract

The present invention discloses a method for facial recognition of public figures, comprising: obtaining an image to be processed; inputting the image to be processed into a facial recognition model, so that the facial recognition model extracts features of the image, performs L2 normalization, and outputs a facial recognition result of the image to be processed based on an embedding vector; the facial recognition model is pre-trained based on a SqueezeNet neural network model. The facial recognition model includes a Fire block, wherein the Expand layer in the Fire block includes: a second convolutional layer with a 1*3 convolution kernel, a third convolutional layer with a 3*1 convolution kernel, a fourth convolutional layer with a 3*1 convolution kernel, and a fifth convolutional layer with a 1*3 convolution kernel. The fourth convolutional layer is connected to the second convolutional layer, and the fifth convolutional layer is connected to the third convolutional layer, replacing the 1*1 convolution and 3*3 convolutions in the Expand layer in the related art. This method can reduce model training time and the size of the model weight file without affecting recognition accuracy, thereby reducing hardware resource requirements and achieving a lightweight network model.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning technology, and specifically relates to a face recognition method for public figures. Background Art

[0002] Facial recognition technology can efficiently and accurately identify multiple faces concurrently without coercion or contact. It is an important means of identity authentication. It has been widely used in many fields such as military, finance, and public security, and is one of the important means of identity authentication.

[0003] There are many classic and powerful neural network structures in related technologies, such as AlexNet, GoogLeNet, VGG, ResNet, DenseNet, Inception, etc. Although these classic large-scale networks have achieved good results in image classification accuracy, their structures are complex and have many parameters, and they have high hardware requirements. When used for face recognition, some mobile devices will not be able to use them because the network model is too large and the training time is too long. Summary of the Invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a method for face recognition of public figures. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0005] The present invention provides a method for face recognition of public figures, comprising:

[0006] Get the image to be processed;

[0007] Inputting the image to be processed into a face recognition model, so that the face recognition model extracts features of the image to be processed, performs L2 normalization, and outputs a face recognition result of the image to be processed according to an embedding vector;

[0008] The face recognition model is pre-trained based on the SqueezeNet neural network model.

[0009] In one embodiment of the present invention, the face recognition model includes a first convolutional layer Conv1, a first pooling layer, Fire blocks: fire2, fire3 and fire4, a second pooling layer, Fire blocks: fire5, fire6, fire7 and fire8, a second pooling layer, a Fire block: fire9, a second convolutional layer Conv10 and a third pooling layer connected in sequence.

[0010] In one embodiment of the present invention, each Fire block includes a Squeeze layer, and the Squeeze layer includes a first convolution layer with a convolution kernel of 1*1.

[0011] In one embodiment of the present invention, each Fire block further includes an Expand layer, wherein the Expand layer includes: a second convolution layer with a convolution kernel of 1*3, a third convolution layer with a convolution kernel of 3*1, a fourth convolution layer with a convolution kernel of 3*1, and a fifth convolution layer with a convolution kernel of 1*3; wherein,

[0012] The fourth convolutional layer is connected to the second convolutional layer, and the fifth convolutional layer is connected to the third convolutional layer.

[0013] In one embodiment of the present invention, each Fire block further includes a concat layer for concatenating the feature maps output by the fourth convolutional layer and the fifth convolutional layer.

[0014] In one embodiment of the present invention, the face recognition model is trained using the following steps:

[0015] Acquire a first category of images and detect the first category of images using a face detection algorithm to obtain training samples; the training samples include face sample images and real categories corresponding to the face sample images;

[0016] After preprocessing the training samples, a preset number of training samples are input into a SqueezeNet neural network model to be trained, where the SqueezeNet neural network model to be trained is a preset initial SqueezeNet neural network model;

[0017] Determining the predicted category of each input training sample using the output result of the SqueezeNet neural network model to be trained;

[0018] Determine a loss value based on a preset loss function, the predicted category and the true category of each input training sample;

[0019] Determining whether the SqueezeNet neural network model to be trained has converged according to the loss value; if converged, the SqueezeNet neural network model to be trained is a trained face recognition model;

[0020] If convergence has not occurred, the network parameters of the SqueezeNet neural network model to be trained are adjusted, and the process returns to the step of inputting a preset number of training samples into the SqueezeNet neural network model to be trained.

[0021] In one embodiment of the present invention, the step of preprocessing the training samples includes:

[0022] Performing affine transformation and data enhancement on the face sample images in the training samples;

[0023] The data enhancement includes performing horizontal mirror transformation, vertical mirror transformation, noise addition and / or size transformation on the face sample image.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] The present invention provides a method for facial recognition of public figures. The facial recognition model used is a pre-trained SqueezeNet neural network model, wherein the Expand layer in the Fire block includes: a second convolutional layer with a 1*3 convolution kernel, a third convolutional layer with a 3*1 convolution kernel, a fourth convolutional layer with a 3*1 convolution kernel, and a fifth convolutional layer with a 1*3 convolution kernel. The fourth convolutional layer is connected to the second convolutional layer, and the fifth convolutional layer is connected to the third convolutional layer. By designing the Expand layer, the present application replaces the 1*1 convolution and 3*3 convolution of the Expand layer in the related art. This can reduce the model training time and the size of the model weight file without affecting the recognition accuracy, thereby reducing the demand for hardware resources and achieving a lightweight network model.

[0026] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of a method for face recognition of public figures provided by an embodiment of the present invention;

[0028] Figure 2 This is a structural diagram of a face recognition model provided by an embodiment of the present invention;

[0029] Figure 3 is a structural diagram of a Fire block provided by an embodiment of the present invention;

[0030] Figure 4 is an example diagram of a face sample image provided by an embodiment of the present invention;

[0031] Figure 5 The embodiment of the present invention provides Figure 4 Schematic diagram of horizontal mirror transformation of the face sample image shown;

[0032] Figure 6 The embodiment of the present invention provides Figure 4 Schematic diagram of vertical mirror transformation of the face sample image shown;

[0033] Figure 7 The embodiment of the present invention provides Figure 4 Schematic diagram of noise addition to the face sample image shown;

[0034] Figure 8This is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0036] Figure 1 This is a flow chart of a method for face recognition of public figures provided by an embodiment of the present invention. Figure 1 As shown, an embodiment of the present invention provides a face recognition method for public figures, comprising:

[0037] S101, obtaining an image to be processed;

[0038] S102, inputting the image to be processed into the face recognition model, so that the face recognition model extracts features of the image to be processed, performs L2 normalization, and outputs a face recognition result of the image to be processed according to the embedding vector;

[0039] Among them, the face recognition model is pre-trained based on the SqueezeNet neural network model.

[0040] Figure 2 is a structural diagram of a face recognition model provided by an embodiment of the present invention. Figure 2 As shown, the face recognition model includes the first convolutional layer Conv1, the first pooling layer, Fire blocks: fire2, fire3 and fire4, the second pooling layer, Fire blocks: fire5, fire6, fire7 and fire8, the second pooling layer, Fire block: fire9, the second convolutional layer Conv10 and the third pooling layer, which are connected in sequence.

[0041] Figure 3 This is a structural diagram of the Fire block provided by an embodiment of the present invention. Figure 3 , each Fire block includes a Squeeze layer, and the Squeeze layer includes a first convolution layer with a convolution kernel of 1*1.

[0042] Optionally, each Fire block further includes an Expand layer, which includes: a second convolution layer with a convolution kernel of 1*3, a third convolution layer with a convolution kernel of 3*1, a fourth convolution layer with a convolution kernel of 3*1, and a fifth convolution layer with a convolution kernel of 1*3; wherein,

[0043] The fourth convolutional layer is connected to the second convolutional layer, and the fifth convolutional layer is connected to the third convolutional layer.

[0044] Optionally, each Fire block further includes a concat layer for concatenating the feature maps output by the fourth convolutional layer and the fifth convolutional layer.

[0045] As you can understand, the SqueezeNet neural network model is a lightweight convolutional neural network. This embodiment uses a trained SqueezeNet neural network model to extract features from the image being processed. Compared to the commonly used Inception-v1 in the prior art, this design not only reduces the computational workload of network training and the hardware resource requirements, but also achieves higher accuracy.

[0046] Optionally, in the face recognition model, the first convolutional layer Conv1 is used to extract features of the input image to be processed and output a feature map of size (111, 111, 96). The first pooling layer Maxpool / 2 is used for dimensionality reduction, reducing the number of parameters, removing redundant information, and compressing the feature map. The output width and height of this layer become smaller and the number of channels remains unchanged. The Fire block is mainly used to reduce the calculation parameters, thereby compressing the model. The second convolutional layer Conv10 replaces the conventional fully connected layer and outputs a 128-dimensional feature map of size (13, 13, 128). The third pooling layer global avgpool is used to reduce the dimension of the feature map, further greatly reducing the network parameters, and outputting a feature vector. The size of the feature map output by this layer is consistent with the feature map output by the second convolutional layer conv10, and the size is (1, 1, 128).

[0047] In this embodiment, the Expand layer of the Fire block includes: a second convolution layer with a convolution kernel of 1*3, a third convolution layer with a convolution kernel of 3*1, a fourth convolution layer with a convolution kernel of 3*1, and a fifth convolution layer with a convolution kernel of 1*3. The fourth convolution layer is connected to the second convolution layer, and the fifth convolution layer is connected to the third convolution layer, replacing the 1*1 convolution and 3*3 convolution of the expand layer in the related art, which can effectively reduce the model size, reduce the model structure parameters, and reduce the training time.

[0048] Furthermore, the concat layer of each Fire block concatenates the feature maps output by the fourth and fifth convolutional layers, and performs L2 normalization after concatenation, also known as feature normalization, which is to divide the concatenated vector by its L2 norm. The specific formula is as follows:

[0049]

[0050] Among them, x i 、y i , i ranges from 1 to 128, and D is 128.

[0051] In the figure, the EMBEDDING feature vector refers to the feature vector generated after L2 normalization.

[0052] Optionally, the face recognition model is trained using the following steps:

[0053] S200, obtaining a first category of images, and detecting the first category of images using a face detection algorithm to obtain training samples; the training samples include face sample images and real categories corresponding to the face sample images;

[0054] S201, after pre-processing the training samples, input a preset number of training samples into the SqueezeNet neural network model to be trained, where the SqueezeNet neural network model to be trained is a preset initial SqueezeNet neural network model;

[0055] S202, using the output results of the SqueezeNet neural network model to be trained, determining the predicted category of each input training sample;

[0056] S203, determining a loss value based on a preset loss function, the predicted category and the true category of each input training sample;

[0057] S204. Determine whether the SqueezeNet neural network model to be trained has converged based on the loss value; if converged, the SqueezeNet neural network model to be trained is a trained face recognition model;

[0058] S205: If convergence has not occurred, the network parameters of the SqueezeNet neural network model to be trained are adjusted, and the process returns to the step of inputting a preset number of training samples into the SqueezeNet neural network model to be trained.

[0059] Specifically, in the above step S200, the first type of image can be a person on the list of China's top 500 people. Each first type of image contains only a single target person, and the face position of the person needs to be detectable. Optionally, a face detection algorithm is used to perform face detection on the first type of image. The code is as follows:

[0060] face_cascade=cv2.CascadeClassifier(r'. / haarcascade_frontalface_default.xml')

[0061] faces=face_cascade.detectMultiScale(image,scaleFactor=1.3,minNeighbors=2)

[0062] After face detection, the retained training samples include 500 target persons, and each target person has approximately 50 face images.

[0063] In the above step S201, the step of pre-processing the training samples includes:

[0064] Perform affine transformation and data enhancement on the face sample images in the training samples;

[0065] The data enhancement includes performing horizontal mirror transformation, vertical mirror transformation, noise addition and / or size transformation on the face sample image.

[0066] It should be understood that since the faces in the face sample images may have different postures and positions, after obtaining the face positions, the key points in the face can be detected and the faces can be standardized based on these key points to eliminate the errors caused by different postures. Specifically, the face area in each face sample image is affine transformed to achieve the correction alignment of the face area based on the coordinates of the eyes and nose points. The code is as follows:

[0067] tranform=cv2.getAffineTransform(source_point,dst_point)

[0068] img_new=cv2.warpAffine(img,tranform,imagesize)

[0069] It should be noted that when training a face recognition model based on the SqueezeNet neural network model, sufficient training samples can increase the model's precision and accuracy while preventing underfitting. In this embodiment, the face recognition model utilizes sufficient training sample data to continuously adjust the weights of the SqueezeNet neural network model to be trained, allowing the model to acquire sufficient learning and obtain good feature expression, thereby obtaining an excellent face recognition model with good recognition results. Therefore, inputting a large amount of appropriate face data sets is a key step in the training process.

[0070] Specifically, in order to reduce the limitations of the samples generated after expansion, this embodiment uses the following four methods to expand the training samples:

[0071] (1) Horizontal mirror transformation

[0072] Figure 4 is an example diagram of a face sample image provided by an embodiment of the present invention. Figure 5 The embodiment of the present invention provides Figure 4 Schematic diagram of horizontal mirror transformation of the face sample image shown in Figure 1. Figure 4-5 In the original face sample image, any pixel point P0(x0,y0) is mirrored horizontally to a new position P(x,y). The transformation formula is:

[0073] x=fw -x0

[0074] y=y0

[0075] Correspondingly, the matrix expression is:

[0076]

[0077] Among them, f w Indicates the width of the original face sample image.

[0078] (2) Vertical mirror transformation

[0079] Figure 6 The embodiment of the present invention provides Figure 4 Schematic diagram of vertical mirror transformation of the face sample image shown in Figure 2. Figure 4 and Figure 6 In the original face sample image, any pixel point P0(x0,y0) is mirrored vertically to a new position P(x,y). The transformation formula is:

[0080]

[0081] Correspondingly, the matrix expression is:

[0082]

[0083] Among them, f H Indicates the height of the original face sample image.

[0084] (3) Noise addition

[0085] Figure 7 The embodiment of the present invention provides Figure 4 The noise adding diagram of the face sample image shown in FIG. Figure 7 As shown in the figure, the addition of noise will affect the image factors. Therefore, it is necessary to ensure that the faces in the face sample images can still be successfully detected after the noise factors are added, so that the network model can overcome the useless features caused by the noise and avoid the added noise no longer affecting the overall recognition accuracy.

[0086] (4) Size conversion

[0087] In order to ensure the consistency of the input face sample image dimensions, the face sample image needs to be resized to adapt to the network input. The code is as follows:

[0088] dst=cv.resize(src,dsize[,dst[,fx[,fy[,interpolation]]]])

[0089] After completing the preprocessing of the training samples, there are about 100 face images of each target task person in the training samples.

[0090] In the above step S203, the preset loss function can be Triplrt Loss. For the face sample image x, TripletLoss can make the mapped vector representation f(x) measurable in the Euclidean space. The goal of TripletLoss is to make the vector Euclidean distances of face sample images containing the same face close in the Euclidean space, and to make the vector Euclidean distances of face sample images containing different faces farther in the Euclidean space. This embodiment uses TriplrtLoss for training, and uses the loss function of LMNN (Large Margin Nearest Neighbor, maximum margin nearest neighbor classification) based on triplets to train the neural network. The network directly outputs a 128-dimensional vector space. The selected triplets contain two matching face thumbnails and one non-matching face thumbnail. The loss function goal is to distinguish between positive and negative classes by distance boundaries. Specifically, it can be expressed as a mathematical formula:

[0091]

[0092] Where, Represents the predicted category of each training sample output, called anchor, the face image of the same person is called positive, and another different face image is It is called negative, and a is the threshold that fully separates positive and negative faces. The anchor, positive, and negative are all feature vectors after L2 regularization.

[0093] It can be seen from the above embodiments that the beneficial effects of the present invention are:

[0094] The present invention provides a method for facial recognition of public figures. The facial recognition model used is a pre-trained SqueezeNet neural network model, wherein the Expand layer in the Fire block includes: a second convolutional layer with a 1*3 convolution kernel, a third convolutional layer with a 3*1 convolution kernel, a fourth convolutional layer with a 3*1 convolution kernel, and a fifth convolutional layer with a 1*3 convolution kernel. The fourth convolutional layer is connected to the second convolutional layer, and the fifth convolutional layer is connected to the third convolutional layer. By designing the Expand layer, the present application replaces the 1*1 convolution and 3*3 convolution of the Expand layer in the related art. This can reduce the model training time and the size of the model weight file without affecting the recognition accuracy, thereby reducing the demand for hardware resources and achieving a lightweight network model.

[0095] The embodiment of the present invention further provides an electronic device, such as Figure 8 As shown, it includes a processor 801, a communication interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804.

[0096] Memory 803, used for storing computer programs;

[0097] The processor 801 is configured to execute the program stored in the memory 803, and implement the following steps:

[0098] Get the image to be processed;

[0099] Inputting the image to be processed into a face recognition model, so that the face recognition model extracts features of the image to be processed, performs L2 normalization, and outputs a face recognition result of the image to be processed according to an embedding vector;

[0100] The face recognition model is pre-trained based on the SqueezeNet neural network model.

[0101] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0102] The communication interface is used for communication between the above electronic device and other devices.

[0103] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0104] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0105] The method provided in the embodiments of the present invention can be applied to electronic devices. Specifically, the electronic devices can be desktop computers, portable computers, smart mobile terminals, servers, etc. This is not limited here; any electronic device that can implement the present invention falls within the scope of protection of the present invention.

[0106] As for the device / electronic device / storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0107] By using the terminal device provided by the embodiment of the present invention, proper nouns and / or fixed phrases can be displayed for user selection, thereby reducing user input time and improving user experience.

[0108] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0109] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification.

[0110] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art can understand and implement other changes to the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0111] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A method for face recognition of public figures, characterized in that: include: Get the image to be processed; Inputting the image to be processed into a face recognition model, so that the face recognition model extracts features of the image to be processed, performs L2 normalization, and outputs a face recognition result of the image to be processed according to an embedding vector; In which, the face recognition model is pre-trained based on the SqueezeNet neural network model; the face recognition model includes a first convolutional layer Conv1, a first pooling layer, Fire blocks: fire2, fire3 and fire4, a second pooling layer, Fire blocks: fire5, fire6, fire7 and fire8, a second pooling layer, a Fire block: fire9, a second convolutional layer Conv10 and a third pooling layer connected in sequence; each Fire block includes an Expand layer, and the Expand layer includes: a second convolutional layer with a convolution kernel of 1*3, a third convolutional layer with a convolution kernel of 3*1, a fourth convolutional layer with a convolution kernel of 3*1 and a fifth convolutional layer with a convolution kernel of 1*3; wherein the fourth convolutional layer is connected to the second convolutional layer to form a first branch, and the fifth convolutional layer is connected to the third convolutional layer to form a second branch parallel to the first branch.

2. The face recognition method for public figures according to claim 1, characterized in that: Each Fire block further includes a Squeeze layer, and the Squeeze layer includes a first convolution layer with a convolution kernel of 1*1.

3. The face recognition method for public figures according to claim 2, characterized in that: Each Fire block further includes a concat layer for concatenating the feature maps output by the fourth convolutional layer and the fifth convolutional layer.

4. The face recognition method for public figures according to claim 1, characterized in that: The face recognition model is trained using the following steps: Acquire a first category of images and detect the first category of images using a face detection algorithm to obtain training samples; the training samples include face sample images and real categories corresponding to the face sample images; After preprocessing the training samples, a preset number of training samples are input into a SqueezeNet neural network model to be trained, where the SqueezeNet neural network model to be trained is a preset initial SqueezeNet neural network model; Determining the predicted category of each input training sample using the output result of the SqueezeNet neural network model to be trained; Determine a loss value based on a preset loss function, the predicted category and the true category of each input training sample; Determining whether the SqueezeNet neural network model to be trained has converged according to the loss value; if converged, the SqueezeNet neural network model to be trained is a trained face recognition model; If convergence has not occurred, the network parameters of the SqueezeNet neural network model to be trained are adjusted, and the process returns to the step of inputting a preset number of training samples into the SqueezeNet neural network model to be trained.

5. The face recognition method for public figures according to claim 4, characterized in that: The step of preprocessing the training samples includes: Performing affine transformation and data enhancement on the face sample images in the training samples; The data enhancement includes performing horizontal mirror transformation, vertical mirror transformation, noise addition and / or size transformation on the face sample image.

Images