Face recognition method, device, equipment and computer-readable storage medium

Abstract

The embodiments of the present invention relate to the field of image processing technology, and specifically to a face recognition method, device, equipment and computer-readable storage medium. The method comprises: image acquisition: acquiring a facial image to be recognized; image processing: removing at least the features of the facial features in the facial image to obtain a target image; model recognition: inputting the target image into a face recognition model to obtain the face type category of the facial image, wherein the face recognition model is obtained by training a convolutional neural network model with multiple sets of training samples, and each set of training samples includes a target image obtained by processing the image processing step. The embodiments of the present invention improve the accuracy of face recognition.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and specifically to a face recognition method, device, equipment and computer-readable storage medium. Background Art

[0002] As people's living standards improve, many people pay more and more attention to their own image. They want to know their face shape, eyebrow shape and other facial features so as to choose the appropriate hairstyle, makeup, etc.

[0003] There are many face recognition methods in the prior art. The more common one is the rule-based method, which calculates the distance or angle between the facial features and determines the face type according to the set rules. This method relies on the positioning of the facial features. If the positioning is inaccurate, it will affect the recognition result. In addition, the distance or angle between the facial features is affected by the shooting angle, facial posture, hairstyle, etc., and the rules cannot cover all situations, resulting in inaccurate recognition results. In short, the recognition accuracy of the rule-based recognition method in the prior art is not high. Summary of the invention

[0004] In view of the above problems, the embodiments of the present invention provide a face recognition method, device, equipment and computer-readable storage medium to improve recognition accuracy.

[0005] According to one aspect of an embodiment of the present invention, a face shape recognition method is provided, the method comprising:

[0006] Image acquisition: obtain the facial image to be recognized;

[0007] Image processing: removing at least the facial features in the facial image to obtain a target image;

[0008] Model recognition: input the target image into a face shape recognition model to obtain the face shape category of the face image, wherein the face shape recognition model is obtained by training a convolutional neural network model with multiple sets of training samples, and each set of training samples includes the target image obtained by the image processing step.

[0009] Optionally, the removing at least the facial features in the facial image further comprises:

[0010] Fill the facial features in the facial image with color.

[0011] Optionally, the removing at least the facial features in the facial image further comprises:

[0012] Obtaining boundary coordinates of the facial features in the facial image;

[0013] connecting the boundary coordinates into a closed figure;

[0014] The closed figure is filled with color.

[0015] Optionally, the image processing further includes:

[0016] Determining coordinates of facial key points in the facial image;

[0017] The obtaining of boundary coordinates of the facial features in the facial image further comprises:

[0018] According to the coordinates of the facial key points, the boundary coordinates of the facial features in the facial image are obtained.

[0019] Optionally, the image processing further includes:

[0020] Determining coordinates of facial key points in the facial image;

[0021] estimating the coordinates of the highest point of the forehead in the facial image;

[0022] Determine the facial region based on the coordinates of the leftmost key point, the rightmost key point, the bottom key point and the highest point of the forehead among the facial key points;

[0023] removing the background image around the facial region to obtain a compact facial image;

[0024] The at least removing the facial features in the facial image is specifically:

[0025] At least the facial features in the compact face image are removed.

[0026] Optionally, the removing at least the facial features in the facial image further comprises:

[0027] According to the coordinates of the facial key points and the coordinates of the highest point of the forehead, the overall boundary coordinates of the facial features and the forehead in the compact facial image are obtained;

[0028] connecting the overall boundary coordinates into a closed figure;

[0029] The closed figure is filled with color.

[0030] Optionally, inputting the target image into a face shape recognition model to obtain the face shape category of the face image further includes:

[0031] Inputting the target image into a face recognition model;

[0032] Obtaining a probability value for each face shape category through inference of the face shape recognition model;

[0033] A face shape category corresponding to a maximum value is selected from the probability values ​​as the face shape category of the facial image.

[0034] Optionally, the convolutional neural network model is a Mobilenet model, a VGG model or an Inception model.

[0035] According to another aspect of an embodiment of the present invention, a face recognition device is provided, the device comprising:

[0036] An acquisition module, used for acquiring a facial image to be recognized;

[0037] A processing module, used for removing at least the features of the facial features in the facial image to obtain a target image;

[0038] The recognition module is used to input the target image into a face shape recognition model to obtain the face shape category of the face image, wherein the face shape recognition model is obtained by training a convolutional neural network model through multiple groups of training samples, and each group of training samples includes the target image obtained by the image processing step.

[0039] According to another aspect of an embodiment of the present invention, a face recognition device is provided, comprising: an image acquisition device, a processor, a memory, a communication interface and a communication bus, wherein the image acquisition device, the processor, the memory and the communication interface communicate with each other via the communication bus;

[0040] The image acquisition device is used to acquire a facial image to be recognized;

[0041] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the steps of the above-mentioned face recognition method.

[0042] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores at least one executable instruction, and the executable instruction enables a processor to execute the steps of the above-mentioned face recognition method.

[0043] The embodiment of the present invention removes the features of the facial features in the facial image to be identified, inputs the target image after the features of the facial features are removed into a face shape recognition model, and identifies the face shape category of the facial image. Since the features of the face shape are retained as much as possible, the interference element of the facial features that have little to do with the face shape but account for a large proportion is removed, and the target image is obtained, thereby improving the accuracy of face shape recognition.

[0044] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0046] Figure 1 A flow chart of a face recognition method provided by an embodiment of the present invention is shown;

[0047] Figure 2 A flowchart showing a specific application example of the face recognition method of the present invention;

[0048] Figure 3 A schematic diagram showing key points of a face in an embodiment of the present invention is shown;

[0049] Figure 4a Shows Figure 2 A facial image to be recognized in ;

[0050] Figure 4b Shows the Figure 4a The image after the boundary coordinates of the facial features are connected into a closed figure;

[0051] Figure 4c Shows the Figure 4b The target image is obtained by filling the closed figures in with white;

[0052] Figure 5 A flowchart showing another specific application example of the face recognition method of the present invention;

[0053] Figure 6a Shows Figure 5 A facial image to be recognized in ;

[0054] Figure 6b Shows the Figure 6a Compact face image after background image removal;

[0055] Figure 6c Shows the Figure 6b The image after the overall boundary coordinates of the facial features and forehead are connected into a closed figure;

[0056] Figure 6d Shows the Figure 6cThe target image is obtained by filling the closed figures in with white;

[0057] Figure 7 A functional block diagram of a face recognition device provided by an embodiment of the present invention is shown;

[0058] Figure 8 A schematic diagram of the structure of a face recognition device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0059] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0060] The application scenarios of the embodiments of the present invention are various application scenarios that require face recognition, such as beauty applications (Application, APP), dressing APP, beauty camera APP, etc., or scenarios that require face recognition of batch images, such as research on face shapes of regions, countries or races. The recognized face shape can be a human face shape or an animal face shape, etc.

[0061] Figure 1 FIG. 4 is a flow chart showing a face recognition method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0062] Step 110: Acquire a facial image to be recognized.

[0063] Among them, the facial image to be recognized includes at least an image of the face, and may also include images of other parts around the face, such as hair, neck, shoulders, etc. In the embodiment of the present invention, the method of obtaining the image to be recognized is different according to different application scenarios. For example, in the application scenario of recognizing a large number of facial images, the facial image to be recognized is an image pre-stored in the image recognition device, or a network image. In some other application scenarios, such as beauty APP, dressing APP, beauty camera APP, etc., the facial image to be recognized can be an image pre-stored in the mobile device where the APP is installed, or an image acquired in real time. When it is an image acquired in real time, the facial image to be recognized can be acquired by the image acquisition device on the mobile device, and the acquired image can be transmitted to the processor of the mobile device for recognition. Among them, the image acquisition device includes but is not limited to a camera.

[0064] In order to ensure the effectiveness of the extraction of the image to be identified, the interference information in the image to be identified is filtered out, and the image to be identified can be preprocessed. In some embodiments, the image to be identified is a three-channel RGB image. In order to reduce the complexity of image processing, the three-channel RGB image is converted into a single-channel grayscale image. In the RGB image, the components of the three channels of R, G, and B are included, and the RGB image is synthesized by the components of these three channels. Each pixel in each component corresponds to a value, and the corresponding pixel values ​​in different components may be the same or different. The value is used to represent the color of the pixel, and the value range is between 0 and 255. Among them, 0 means that the color of the pixel is white, and 255 means that the color of the pixel is black. The value of the corresponding pixel in the grayscale image is determined according to the proportion of the corresponding pixel in the three components. In one embodiment, the proportion of the corresponding pixel in the three components is the same, and the corresponding pixel of the three components is averaged to obtain the value of the pixel in the grayscale image. It should be understood that if the image to be identified obtained is a single-channel grayscale image, the above conversion is not required, and the subsequent steps are directly performed.

[0065] Step 120: At least remove the facial features in the face image to obtain a target image.

[0066] Among them, the removal of facial features can include the following methods:

[0067] 1. Delete the facial features and make them transparent;

[0068] 2. Fill the facial features with color. The color can be a single color, such as white, black, etc.; it can also be a patterned color, such as black and white grid pattern filling, black and white stripe pattern filling, etc., so that the image information after filling is less than the original image information of the facial features; it can also be filled with transparent color to achieve the same effect of transparent format of facial features as in the first method;

[0069] 3. Decolorize the facial features, that is, convert the original facial features color image into a grayscale image through calculation.

[0070] The facial features can include only the individual parts of the facial features, such as individual glasses, eyebrows, nose, mouth, etc.; or they can include the entire facial features area, such as individual facial features and the area between the parts. These areas have little to do with the face shape but account for a large proportion. After removing them, interference factors can be removed and the accuracy of face recognition can be improved. The forehead area is not related to face recognition. In some embodiments, it is also regarded as a facial feature and can be removed together.

[0071] When the facial features only include the various parts of the facial features, the boundary coordinates of the various parts of the facial features in the facial image can be obtained, and the boundary coordinates of each part can be connected into multiple closed figures, and finally each closed figure can be filled with color. When the facial features include the entire facial area, the overall boundary coordinates of the facial features in the facial image can be obtained, and the overall boundary coordinates can be connected into a closed figure, and finally the closed figure can be filled with color. The target image obtained after processing in the above manner retains the facial contour completely because the key part image of face recognition (such as the facial contour line and the in-plane area close to the facial contour line) is not processed, which can improve the recognition accuracy.

[0072] Step 130: Input the target image into the face shape recognition model to obtain the face shape category of the face image.

[0073] Common face shapes include: square face, round face, oval face, heart-shaped face, etc. Determining which face shape the face in the target image belongs to is a classification problem. In the embodiment of the present invention, a deep learning method is used for classification, and the classification result is obtained by model inference after the model is trained.

[0074] The face recognition model in this step is obtained by training a convolutional neural network (CNN) model with multiple sets of training samples, and each set of training samples includes the target image processed in step 120. The convolutional neural network model is a Mobilenet model, a VGG model, or an Inception model. During recognition, the target image is input into the face recognition model, and the probability value of each face category is obtained by inference through the face recognition model. The face category corresponding to the maximum value is selected from the probability values ​​as the face category of the face image.

[0075] CNN includes input layer, convolution layer, pooling layer and fully connected layer. The input layer generally represents the pixel matrix of the image through a three-dimensional matrix. The length and width of the three-dimensional matrix represent the size of the image, and the depth of the three-dimensional matrix represents the color channel of the image. The convolution layer is the most important part of CNN. The input of each node in the convolution layer is a small block in the input layer. The commonly used size of the small block is 3×3 or 5×5. Generally speaking, the node matrix processed by the convolution layer will become deeper. The pooling layer does not change the depth of the three-dimensional matrix, but it can reduce the size of the matrix, and can convert a high-resolution image into a low-resolution image. The number of nodes in the last fully connected layer can be further reduced through the pooling layer, thereby achieving the purpose of reducing the parameters of the entire neural network. The fully connected layer is the last layer of CNN. The activation function can use softmax, and the probability distribution of samples belonging to each category can be obtained. The convolution layer and pooling layer can be designed for multiple rounds. After multiple rounds of convolution and pooling layers, the final classification result is generally given by 1 to 2 fully connected layers at the end of CNN.

[0076] The following is a detailed description of the training and application process of the face recognition model:

[0077] The face shape recognition model divides the target categories of face shape into square face, round face, oval face and heart-shaped face. Each category corresponds to a specific attribute value. For example, the face shape attribute value corresponding to the square face is 0, the face shape attribute value corresponding to the round face is 1, the face shape attribute value corresponding to the oval face is 2, and the face shape attribute value corresponding to the heart-shaped face is 3. The classification result output by the face shape recognition model for the input target image includes the probability distribution for each target category, and the sum of the probability distribution for each category is 1. For example, the probability distribution for the four categories of the aforementioned face shape attributes is (0, 0.01, 0.01, 0.98). Then, according to the probability distribution and the face shape attribute values ​​corresponding to each category, the face shape attribute value of the input target image can be calculated to be 0*0+1*0.01+2*0.01+3*0.98=2.97, thereby determining that the face shape in the input target image belongs to a heart-shaped face.

[0078] Before training, first construct a CNN model as described above and initialize the weights. Then obtain a training sample set, which includes a sample image and a face shape label corresponding to the sample image (i.e., the face shape attribute value described above, which will be used as the target value of the CNN model). Each sample image is an image processed by step 120, and the features of the facial features in the face image are removed from the sample image. The sample image is processed by convolution and pooling of the convolution layer and the pooling layer to extract the feature vector, which is passed into the fully connected layer and classified by the fully connected layer to obtain the output value of the face shape classification recognition, which is the face shape attribute value calculated according to the probability distribution of the face shape category. The above is the forward propagation process.

[0079] When the result of CNN output does not match the expected value, the back propagation process is performed. From the input layer to the output layer, the data passes through the convolution layer, the pooling layer, and the fully connected layer. In the process of data transmission between the layers, it is inevitable that data loss will occur, which will lead to the generation of errors. The error value caused by each layer is different. Therefore, after the total error of CNN is calculated, the error needs to be passed into CNN to find out how much weight each layer should bear for the total error. Therefore, during back propagation, the error between the output value and the target value is first calculated, and then the error is returned layer by layer, the error of each layer is calculated, and the weight of each layer is updated according to the error. Then continue the training process of sample input and output. When the error is equal to or less than the expected value, the training ends.

[0080] Face shape classification based on deep learning can be implemented in various ways in the prior art, and the embodiments of the present invention will not be elaborated in detail.

[0081] The embodiment of the present invention removes the features of the facial features in the facial image to be identified, inputs the target image after the features of the facial features are removed into a face shape recognition model, and identifies the face shape category of the facial image. Since the features of the face shape are retained as much as possible, the interference element of the facial features that have little to do with the face shape but account for a large proportion is removed, and the target image is obtained, thereby improving the accuracy of face shape recognition.

[0082] The face recognition method provided by the embodiment of the present invention can be applied to mobile phone APPs that need to perform face recognition. Users only need to take a photo of their own face to recognize their face shape, which is highly convenient.

[0083] Figure 2 A flowchart of a specific application example of the face recognition method of the present invention is shown as follows: Figure 2 As shown, the method comprises the following steps:

[0084] Step 201: Obtain a facial image to be recognized;

[0085] Step 202: Determine the coordinates of facial key points in the facial image;

[0086] like Figure 3As shown, in this example, the face shape of the face needs to be identified, and the coordinates of the key points of the face are the coordinates of the key points of the face. Among them, the key points of the face refer to one or more key points used to calibrate the key areas of the face (including eyebrows, eyes, nose, mouth, facial contours, etc.), and the positions of these key areas in the face can be located according to the key points of the face. The coordinates of the key points of the face can be determined by a model-based active shape model (Active Shape Model, ACM) or active appearance model (ActiveAppearnce Model, AAM), or a cascaded pose regression (Cascaded poseregression, CPR) based on cascaded shape regression, or a method based on deep learning. In a specific implementation, the coordinates of the key points of the face can be determined by calling tools such as OpenCV, Dlib, and Firebase.

[0087] Step 203: Obtaining the boundary coordinates of the facial features in the facial image according to the coordinates of the facial key points;

[0088] like Figure 3 As shown, according to the boundary coordinate points of eyebrows (numbered 17, 18, 19, 24, 25, 26), the boundary coordinate points of eyes (numbered 36, 45), and the boundary coordinate points of mouth (numbered 48, 59, 58, 57, 56, 55, 54) in the key points, the boundary coordinates of the facial features in the facial image are obtained.

[0089] Step 204: Connect the boundary coordinates into a closed figure;

[0090] Connect the boundary coordinates of the facial features into a closed figure, such as Figure 4b shown.

[0091] Step 205: Fill the closed figure with color to obtain a target image;

[0092] Will Figure 4b Fill the closed figure in white to get Figure 4c The target image is shown.

[0093] Step 206: input the target image into the face recognition model;

[0094] Step 207: Obtain the probability value of each face shape category through face shape recognition model inference;

[0095] Step 208: Select the face shape category corresponding to the maximum value from the probability values ​​as the face shape category of the facial image.

[0096] At this point, the face recognition of this example is completed.

[0097] When the face image to be recognized also includes images of other parts around the face, such as hair, neck, and shoulders, it is necessary to remove the background images of other parts that are not related to face shape recognition as much as possible to remove interference factors and improve recognition accuracy. In addition, most parts of the forehead are also irrelevant to face shape recognition and can be removed together. Figure 5 FIG. 1 is a flowchart showing another specific application example of the face recognition method of the present invention, in which the background image and the image of the forehead are removed. Figure 5 As shown, the method comprises the following steps:

[0098] Step 501: Obtain a facial image to be recognized;

[0099] Step 502: Determine the coordinates of facial key points in the facial image;

[0100] like Figure 3 As shown, in this example, the face shape of a human face needs to be recognized, and the facial key point coordinates are the facial key point coordinates.

[0101] Step 503: estimating the coordinates of the highest point of the forehead in the facial image;

[0102] In this example, the coordinates of the highest point (number 19, 24) are extended upward by the length of the eyes to obtain the coordinates of the highest point of the forehead (number 68).

[0103] Step 504: determining the facial region based on the coordinates of the leftmost key point, the rightmost key point, the bottom key point and the highest point of the forehead among the facial key points;

[0104] like Figure 3 As shown, the facial area is determined based on the leftmost coordinate (number 0), the rightmost coordinate (number 16), the bottommost coordinate (number 8), and the coordinate of the highest point of the forehead (number 68).

[0105] Step 505: removing the background image around the face area to obtain a compact face image;

[0106] Remove irrelevant background images around the face, Figure 6a Get a compact face Figure 6b When removing the background image, a clipping method is used. Based on the coordinates of the boundary points on the left and right sides, the vertical boundary line to which the boundary point coordinates belong is determined, and the external background image is clipped based on the vertical boundary line; based on the coordinates of the boundary points on the upper and lower sides, the horizontal boundary line to which the boundary point coordinates belong is determined, and the external background image is clipped based on the horizontal boundary line.

[0107] Step 506: Obtaining the overall boundary coordinates of the facial features and the forehead in the compact facial image according to the coordinates of the facial key points and the coordinates of the highest point of the forehead;

[0108] like Figure 3 As shown, according to the boundary coordinate points of the eyebrows (numbered 17, 26), the boundary coordinate points of the eyes (numbered 36, 45), the boundary coordinate points of the mouth (numbered 48, 59, 58, 57, 56, 55, 54), and the coordinates of the highest point of the forehead (numbered 68), the boundary coordinates of the facial features and the forehead in the facial image are obtained.

[0109] Step 507: Connecting the overall boundary coordinates into a closed figure;

[0110] Connect the overall boundary coordinates of the facial features and forehead into a closed figure, such as Figure 6c shown.

[0111] Step 508: Fill the closed figure with color;

[0112] Will Figure 6c Fill the closed figure in white to get Figure 6d The target image is shown.

[0113] Step 509: input the target image into the face recognition model;

[0114] Step 510: Obtain the probability value of each face shape category through face shape recognition model inference;

[0115] Step 511: Select the face shape category corresponding to the maximum value from the probability values ​​as the face shape category of the facial image.

[0116] At this point, the face recognition of this example is completed.

[0117] Figure 7 FIG. 2 shows a schematic diagram of the structure of a face recognition device provided by an embodiment of the present invention. Figure 7 As shown, the device 700 includes: an acquisition module 710, a processing module 720 and a recognition module 730. Among them, the acquisition module 710 is used to acquire a facial image to be recognized; the processing module, the processing module 720 is used to remove at least the features of the facial features in the facial image to obtain a target image; the recognition module 730 is used to input the target image into a face shape recognition model to obtain the face shape category of the facial image, and the face shape recognition model is obtained by training a convolutional neural network model with multiple sets of training samples, and each set of training samples includes a target image processed by the image processing module.

[0118] Optionally, the processing module 720 is further configured to:

[0119] Obtaining boundary coordinates of the facial features in the facial image;

[0120] connecting the boundary coordinates into a closed figure;

[0121] The closed figure is filled with color.

[0122] Optionally, the processing module 720 is further configured to:

[0123] Determining coordinates of facial key points in the facial image;

[0124] According to the coordinates of the facial key points, the boundary coordinates of the facial features in the facial image are obtained.

[0125] Optionally, the processing module 720 is further configured to:

[0126] Determining coordinates of facial key points in the facial image;

[0127] estimating the coordinates of the highest point of the forehead in the facial image;

[0128] Determine the facial region based on the coordinates of the leftmost key point, the rightmost key point, the bottom key point and the highest point of the forehead among the facial key points;

[0129] removing the background image around the facial region to obtain a compact facial image;

[0130] At least the facial features in the compact face image are removed.

[0131] Optionally, the processing module 720 is further configured to:

[0132] According to the coordinates of the facial key points and the coordinates of the highest point of the forehead, the overall boundary coordinates of the facial features and the forehead in the compact facial image are obtained;

[0133] connecting the overall boundary coordinates into a closed figure;

[0134] The closed figure is filled with color.

[0135] Optionally, the identification module 730 is further configured to:

[0136] Inputting the target image into a face recognition model;

[0137] Obtaining a probability value for each face shape category through inference of the face shape recognition model;

[0138] A face shape category corresponding to a maximum value is selected from the probability values ​​as the face shape category of the facial image.

[0139] Optionally, the convolutional neural network model is a Mobilenet model, a VGG model or an Inception model.

[0140] The embodiment of the present invention removes the features of the facial features in the facial image to be identified, inputs the target image after the features of the facial features are removed into a face shape recognition model, and identifies the face shape category of the facial image. Since the features of the face shape are retained as much as possible, the interference element of the facial features that have little to do with the face shape but account for a large proportion is removed, and the target image is obtained, thereby improving the accuracy of face shape recognition.

[0141] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one executable instruction, and the computer-executable instruction can execute a face recognition method in any of the above method embodiments.

[0142] An embodiment of the present invention provides a computer program, which can be called by a processor to enable a face recognition device to execute the face recognition method in any of the above method embodiments.

[0143] An embodiment of the present invention provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed on a computer, the computer executes the face recognition method in any of the above method embodiments.

[0144] Figure 8 The schematic diagram of the structure of the face recognition device provided by the embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the face recognition device.

[0145] like Figure 8 As shown, the face recognition device may include: an image acquisition device 801, a processor (processor) 802, a communication interface (Communications Interface) 804, a memory (memory) 806, and a communication bus 808.

[0146] Wherein: the image acquisition device 801 is used to acquire the facial image to be recognized. The image acquisition device 801, the processor 802, the communication interface 804, and the memory 806 communicate with each other through the communication bus 808. The communication interface 804 is used to communicate with other devices such as a server. The processor 802 is used to execute the program 810, which can specifically execute the relevant steps in the above-mentioned embodiment of the face recognition method.

[0147] Specifically, the program 810 may include program codes, which include computer operation instructions.

[0148] The image acquisition device 801 includes but is not limited to a camera.

[0149] The processor 802 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiment of the present invention. The one or more processors included in the face recognition device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0150] The memory 806 is used to store the program 810. The memory 806 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0151] The program 810 may be specifically configured to enable the processor 802 to perform the following operations:

[0152] Image acquisition: obtain the facial image to be recognized;

[0153] Image processing: removing at least the facial features in the facial image to obtain a target image;

[0154] Model recognition: input the target image into a face shape recognition model to obtain the face shape category of the face image, wherein the face shape recognition model is obtained by training a convolutional neural network model with multiple sets of training samples, and each set of training samples includes the target image obtained by the image processing step.

[0155] Optionally, the program 810 may also be specifically configured to enable the processor 802 to perform the following operations:

[0156] Obtaining boundary coordinates of the facial features in the facial image;

[0157] connecting the boundary coordinates into a closed figure;

[0158] The closed figure is filled with color.

[0159] Optionally, the program 810 may also be specifically configured to enable the processor 802 to perform the following operations:

[0160] Determining coordinates of facial key points in the facial image;

[0161] According to the coordinates of the facial key points, the boundary coordinates of the facial features in the facial image are obtained.

[0162] Optionally, the program 810 may also be specifically configured to enable the processor 802 to perform the following operations:

[0163] Determining coordinates of facial key points in the facial image;

[0164] estimating the coordinates of the highest point of the forehead in the facial image;

[0165] Determine the facial region based on the coordinates of the leftmost key point, the rightmost key point, the bottom key point and the highest point of the forehead among the facial key points;

[0166] removing the background image around the facial region to obtain a compact facial image;

[0167] At least the facial features in the compact face image are removed.

[0168] Optionally, the program 810 may also be specifically configured to enable the processor 802 to perform the following operations:

[0169] According to the coordinates of the facial key points and the coordinates of the highest point of the forehead, the overall boundary coordinates of the facial features and the forehead in the compact facial image are obtained;

[0170] connecting the boundary coordinates into a closed figure;

[0171] The closed figure is filled with color.

[0172] Optionally, the program 810 may also be specifically configured to enable the processor 802 to perform the following operations:

[0173] Inputting the target image into a face recognition model;

[0174] Obtaining a probability value for each face shape category through inference of the face shape recognition model;

[0175] A face shape category corresponding to a maximum value is selected from the probability values ​​as the face shape category of the facial image.

[0176] Optionally, the convolutional neural network model is a Mobilenet model, a VGG model or an Inception model.

[0177] The embodiment of the present invention removes the features of the facial features in the facial image to be identified, inputs the target image after the features of the facial features are removed into a face shape recognition model, and identifies the face shape category of the facial image. Since the features of the face shape are retained as much as possible, the interference element of the facial features that have little to do with the face shape but account for a large proportion is removed, and the target image is obtained, thereby improving the accuracy of face shape recognition.

[0178] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0179] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: that the claimed invention requires more features than the features explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0180] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0181] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0182] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A face recognition method, characterized in that: The method comprises: Image acquisition: obtain the facial image to be recognized; Image processing: determining the coordinates of the facial key points in the facial image; estimating the coordinates of the highest point in the forehead in the facial image; determining the facial area based on the coordinates of the leftmost key point, the rightmost key point, the bottommost key point and the highest point in the forehead among the facial key points; removing the background image around the facial area to obtain a compact facial image; removing at least the features of the facial features in the facial image to obtain a target image; the at least removing the features of the facial features in the facial image includes: obtaining the overall boundary coordinates of the facial features and the forehead in the compact facial image according to the coordinates of the facial key points and the coordinates of the highest point in the forehead; connecting the overall boundary coordinates into a closed figure; filling the closed figure with color; Model recognition: input the target image into a face shape recognition model to obtain the face shape category of the face image, wherein the face shape recognition model is obtained by training a convolutional neural network model with multiple sets of training samples, and each set of training samples includes the target image obtained by the image processing step.

2. The method according to claim 1, characterized in that The step of at least removing the facial features in the facial image further comprises: Fill the facial features in the facial image with color.

3. The method according to claim 1, characterized in that The step of at least removing the facial features in the facial image further comprises: Obtaining boundary coordinates of the facial features in the facial image; connecting the boundary coordinates into a closed figure; The closed figure is filled with color.

4. The method according to claim 3, characterized in that The image processing further comprises: Determining coordinates of facial key points in the facial image; The obtaining of boundary coordinates of the facial features in the facial image further comprises: According to the coordinates of the facial key points, the boundary coordinates of the facial features in the facial image are obtained.

5. The method according to claim 1, characterized in that Inputting the target image into a face shape recognition model to obtain the face shape category of the face image further comprises: Inputting the target image into a face recognition model; Obtaining a probability value for each face shape category through inference of the face shape recognition model; A face shape category corresponding to a maximum value is selected from the probability values ​​as the face shape category of the facial image.

6. A face recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring a facial image to be recognized; A processing module, for determining the coordinates of the facial key points in the facial image; estimating the coordinates of the highest point of the forehead in the facial image; determining the facial area based on the coordinates of the leftmost key point, the rightmost key point, the bottommost key point and the highest point of the forehead among the facial key points; removing the background image around the facial area to obtain a compact facial image; removing at least the features of the facial features in the facial image to obtain a target image; the at least removing the features of the facial features in the facial image comprises: obtaining the overall boundary coordinates of the facial features and the forehead in the compact facial image according to the coordinates of the facial key points and the coordinates of the highest point of the forehead; connecting the overall boundary coordinates into a closed figure; and filling the closed figure with color; The recognition module is used to input the target image into a face shape recognition model to obtain the face shape category of the face image, wherein the face shape recognition model is obtained by training a convolutional neural network model through multiple groups of training samples, and each group of training samples includes the target image obtained by the image processing step.

7. A face recognition device, characterized in that: include: An image acquisition device, a processor, a memory, a communication interface and a communication bus, wherein the image acquisition device, the processor, the memory and the communication interface communicate with each other via the communication bus; The image acquisition device is used to acquire a facial image to be recognized; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the steps of a face recognition method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one executable instruction, and the executable instruction enables the processor to execute the steps of a face recognition method according to any one of claims 1-5.

Images