Method and device for recognizing face images
By adding an additive layer to the feature recognition neural network to process feature recognition of face images, the problem of insufficient accuracy of facial expression recognition in the prior art is solved, and higher feature extraction and recognition accuracy is achieved.
Patent Information
- Application Number
- CN202011021820.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-25
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-09-25
AI Technical Summary
The prior art has insufficient accuracy in facial expression recognition, and traditional methods or single-model convolutional neural networks are difficult to effectively extract features of face images, resulting in inaccurate recognition results.
A feature recognition neural network is used to preprocess the face image and add additive layers, such as the central entropy supervision layer and the cross entropy loss layer, to generate feature recognition results to improve feature extraction and recognition accuracy.
Through the use of feature recognition neural network, the feature recognition accuracy of face images is significantly improved, and the accuracy and generalization applicability of recognition results are enhanced.
Smart Images

Figure CN112101280B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence, specifically to the field of computer vision technology and deep learning technology, and in particular to a method, device, electronic device and computer-readable storage medium for recognizing facial images that can be used for face recognition. Background Art
[0002] Currently, in order to better formulate user service strategies, it is often necessary to understand the user's characteristic information in order to understand the user's characteristic status.
[0003] In the prior art, neural networks or models are usually used to recognize users' facial expressions. Facial expression recognition generally uses traditional methods or a single-model convolutional neural network. The user's facial expression image is used as input, and the expression features are extracted through the convolutional neural network or manually, and then the output of the classifier is the expression recognition classification result. Summary of the invention
[0004] The present application provides a facial image recognition method, device, electronic device and storage medium.
[0005] In a first aspect, an embodiment of the present application provides a method for recognizing a facial image, which obtains a preprocessed facial image to be recognized; inputs the facial image to be recognized into a feature recognition neural network to obtain feature recognition information output by the feature recognition neural network; wherein the feature recognition neural network includes an adding layer; and based on the feature recognition information, generates a feature recognition result of the facial image to be recognized.
[0006] In a second aspect, an embodiment of the present application provides a facial image recognition device, comprising: an image acquisition unit, configured to acquire a preprocessed facial image to be recognized; a feature recognition unit, configured to input the facial image to be recognized into a feature recognition neural network, and obtain feature recognition information output by the feature recognition neural network; wherein the feature recognition neural network includes an adding layer; and a recognition result generation unit, configured to generate a feature recognition result of the facial image to be recognized based on the feature recognition information.
[0007] In a third aspect, an embodiment of the present application provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the facial image recognition method described in any implementation method.
[0008] In a fourth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium storing computer instructions, including: the computer instructions are used to enable the computer to execute the facial image recognition method described in any implementation manner.
[0009] After obtaining the preprocessed face image to be identified, the present application inputs the face image to be identified into a feature recognition neural network to obtain feature recognition information output by the feature recognition neural network; wherein the feature recognition neural network includes an additional layer; based on the obtained feature recognition information, a feature recognition result of the face image to be identified is generated, and feature recognition of the face image is performed by the feature recognition neural network including the additional layer, thereby improving the accuracy of the feature recognition result for the face image.
[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.
[0012] Figure 1 is an exemplary system architecture in which embodiments of the present application may be applied;
[0013] Figure 2 is a flow chart of an embodiment of a method for recognizing a facial image according to the present application;
[0014] Figure 3 It is a flowchart of an embodiment of obtaining a pre-processed face image to be recognized in the face image recognition method of the present application;
[0015] Figure 4 is a flow chart of an embodiment of determining a synthetic image recognition neural network in a facial image recognition method according to the present application;
[0016] Figure 5 is a structural schematic diagram of an embodiment of a facial image recognition device according to the present application;
[0017] Figure 6 It is a block diagram of an electronic device suitable for implementing the facial image recognition method of the embodiment of the present application. DETAILED DESCRIPTION
[0018] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0019] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0020] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the facial image recognition method, apparatus, electronic device, and computer-readable storage medium of the present application can be applied.
[0021] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0022] The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to send a face image to be recognized, etc. Various applications related to face feature recognition can be installed on the terminal devices 101, 102, 103, such as emotion analysis applications, social applications, image recognition applications, etc.
[0023] Terminal devices 101, 102, 103 can be hardware or software. When it is hardware, it can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (for example, sending a face image to be recognized, etc.), or it can be implemented as a single software or software module. No specific limitation is made here.
[0024] The server 105 may be a server that provides various services, such as a server that provides facial image recognition for the terminal devices 101, 102, and 103. For example, after obtaining the preprocessed facial image to be recognized, the facial image to be recognized is input into a feature recognition neural network to obtain feature recognition information output by the feature recognition neural network; wherein the feature recognition neural network includes an additional layer; based on the obtained feature recognition information, a feature recognition result of the facial image to be recognized is generated, and feature recognition of the facial image is performed through the feature recognition neural network including the additional layer.
[0025] It should be noted that the facial image recognition method provided in the embodiments of the present application is generally executed by the server 105 , and accordingly, the facial image recognition device is generally disposed in the server 105 .
[0026] It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules for providing distributed services, or it can be implemented as a single software or software module. No specific limitation is made here.
[0027] In addition, the facial image recognition method may also be executed by the terminal devices 101, 102, 103, and accordingly, the facial image recognition device may also be disposed in the terminal devices 101, 102, 103. In this case, the exemplary system architecture 100 may also not include the server 105 and the network 104.
[0028] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0029] Continue to refer Figure 2 , which shows a process 200 of an embodiment of a method for recognizing a face image according to the present application. The method for recognizing a face image comprises the following steps:
[0030] Step 201, obtaining a pre-processed face image to be recognized.
[0031] In this embodiment, the execution subject of the face image recognition method (for example Figure 1 The server 105 shown in the figure can be used from a local or non-local human-computer interaction device (e.g. Figure 1 The terminal devices 101, 102, 103) shown in the figure obtain the face image to be identified, which is not limited in this application.
[0032] It should be understood that after acquiring the image, the above-mentioned execution subject can perform preprocessing to obtain a preprocessed face image to be identified, or the preprocessed face image to be identified can be directly obtained from a local or non-local storage device after other execution subjects complete the preprocessing work.
[0033] In some optional implementations of the present embodiment, the preprocessed face image to be identified is determined through the following steps: obtaining the original image, and using a face detection model to determine the face image area in the original image; using a key point annotation model to add first key point annotation information to the face image area; adjusting the face image in the face image area based on the key point coordinate information to obtain an adjusted face image area; using the key point annotation model to re-add second key point annotation information to the adjusted face image area; removing the non-face image part based on the second key point information to obtain the preprocessed face image to be identified.
[0034] Specifically, refer to Figure 3 , which shows a process 300 of an implementation method of the step of determining a face image to be identified after preprocessing in a face image recognition method, specifically including:
[0035] Step 301, obtaining an original image, and using a face detection model to determine a face image region in the original image.
[0036] Specifically, after obtaining the original image, a face detection model is used to determine the area in the original image that contains the face. The face detection model can adopt face detection models such as a residual network detection model, a semantic segmentation detection model, and a VGG detection model.
[0037] Step 302: Use a key point annotation model to add first key point marking information to the face image region.
[0038] Specifically, first, according to the purpose of feature recognition, a preset number of key points can be analyzed and determined based on the face images in the historical data, and then the first key point marking information is added to the face area in the original image determined in the above step 201 through a key point annotation model, such as an Active Shape Model (ASM), an Active Appearance Model (AAM), or a Cascaded Pose Regression (CPR).
[0039] Exemplary, when it is desired to obtain the emotional features of a facial image to be identified, the facial emotions expressed in the facial image can be classified into seven basic categories: anger, disgust, fear, happiness, sadness, surprise and neutrality, and then based on the facial muscle changes in the facial images in the historical data when the characters in the images show the above expressions, the preset number of key points is determined to be 72, so that the facial expressions in the facial image to be identified can be subsequently analyzed based on the 72 key points.
[0040] Step 303: Adjust the face image in the face image region based on the first key point marking information to obtain an adjusted face image region.
[0041] Specifically, based on the marking information of the first key point determined in the above step 302, a coordinate system is established, corresponding coordinate values are added to each first key point, and then the face image in the face area is adjusted according to the coordinate values of each first key point, and a face alignment operation is performed to make the face image approach a forward state, so as to obtain facial features as accurately as possible.
[0042] It should be understood that after the face alignment operation, the adjusted face area can be extracted separately and enlarged to the size of the original image so that the features in the face area can be identified more clearly later.
[0043] Step 304: Use the key point labeling model to add second key point labeling information to the adjusted facial image area.
[0044] Specifically, the key point labeling model used in the above step 302 is used again to re-label the face region adjusted based on the face image, and add the second key point labeling information.
[0045] It should be understood that after the face area is marked again using the key point annotation model, new coordinates are generated for the second key point information based on the coordinate system determined in step 303.
[0046] Step 305: remove the non-face image portion based on the second key point information to obtain a preprocessed face image to be recognized.
[0047] Specifically, determine the second key point corresponding to the outer contour corresponding to the face image part in the face image area, that is, determine the contour of the face image in the face image area, delete the content outside the face image based on the coordinates of the determined second key point information, and obtain the preprocessed face image to be identified.
[0048] In this implementation, the acquired original image is preprocessed based on the face detection model and the key point labeling model to extract the image of the face area containing key points used for face recognition features in the original image as the subsequent face image to be identified, so as to improve the accuracy of identification using the face image to be identified.
[0049] In addition, in some embodiments, in the above step 305, it can also include: after removing the non-face image part based on the second key point information, the obtained face image area is normalized; and the normalized face image area is subjected to random data enhancement processing to obtain the face image to be identified.
[0050] Specifically, image normalization processing is to normalize each pixel in the image in turn so that each pixel in the image can be successfully identified, avoiding pixel omission due to large differences in pixel values, which affects the final detection effect. Then, random data enhancement processing is performed on the normalized result to obtain multiple training materials based on a single image, thereby improving the quality of model training.
[0051] Exemplarily, the normalization method is to subtract 128 from the pixel value of each pixel and then divide it by 256, so that the pixel value of each pixel is between [-0.5, 0.5], so as to avoid the effect of subsequent detection being affected by excessive differences in pixel value between images.
[0052] Step 202: input the face image to be recognized into a feature recognition neural network to obtain feature recognition information output by the feature recognition neural network.
[0053] In this embodiment, the preprocessed facial image to be identified obtained in the above step 201 is input into the feature recognition neural network, and the synthetic feature recognition neural network includes an additional layer. Based on the additional layer included therein, the facial image to be identified can be further analyzed to obtain feature information of the facial image to be detected, so as to facilitate subsequent judgment of the feature information contained in the facial image to be detected, wherein the specific expression form of the feature information is related to the content of the additional layer.
[0054] For example, feature matrix information, feature vector information, and feature value information of the face image to be detected obtained based on different added layers can facilitate the subsequent recognition of different face images based on the obtained feature information.
[0055] Among them, the added layer may include adding convolutional layers, adding maximum pooling layers, adding fully connected layers, adding central entropy supervision layers, etc. to realize various functions, so as to expand the content in the feature extraction neural network according to the content in the added layer, so as to realize the corresponding functions in the added layer alone, or, jointly use the added layer and the functions in the original feature extraction neural network to optimize and improve the functions in the original feature extraction neural network.
[0056] For example, when one wishes to obtain the emotional features of a facial image to be identified, what is ultimately obtained is an emotional feature vector related to seven basic emotions of a predetermined facial emotion classification category: anger, disgust, fear, happiness, sadness, surprise, and neutrality, so that the emotional features of the facial image can be identified and classified based on the obtained emotional feature vector.
[0057] Among them, the feature recognition neural network is usually a convolutional neural network that can realize feature extraction from an image, for example, a LeNet neural network, an AlexNet neural network, and a VGG (Visual Geometry Group Network) neural network.
[0058] Preferably, a VGG neural network is used, in which small convolution kernels of 1×1 and 3×3 and small pooling kernels of 2x2 are used. Since the convolution kernel focuses on expanding the number of channels and the pooling focuses on reducing the width and height, the model architecture of the VGG neural network is deeper and wider, while the increase in the amount of calculation is slowed down, the number of layers is deeper, and the feature map is wider. In addition, in the network testing phase of the VGG neural network, the three full connections in the training phase are replaced by three convolutions, and the test reuses the parameters during training, so that the fully convolutional network obtained by the test can receive inputs of any width or height because there is no restriction on full connection.
[0059] Step 203: Generate a feature recognition result of the face image to be recognized based on the feature recognition information.
[0060] In this embodiment, the recognition result of the face image to be recognized is determined in combination with the form of the feature recognition information obtained in step 202. For example, the image recognition information obtained in step 202 is a feature vector, which is subsequently used to obtain the recognition result of the face image to be recognized.
[0061] The facial image recognition method provided in the embodiment of the present application first obtains a preprocessed facial image to be recognized, and then inputs the facial image to be recognized into a feature recognition neural network to obtain feature recognition information output by the feature recognition neural network; wherein the feature recognition neural network includes an additional layer; based on the obtained feature recognition information, a feature recognition result of the facial image to be recognized is generated, and feature recognition of the facial image is performed by the feature recognition neural network including the additional layer, thereby improving the accuracy of the feature recognition result of the facial image.
[0062] Continue to refer Figure 4 , which shows an embodiment process 400 of generating a feature recognition neural network in the face image recognition method according to the present application. Specifically, the following steps are included:
[0063] Step 401, obtaining a feature extraction neural network.
[0064] In this embodiment, the above execution entities (such as Figure 1 The server 105 shown in the figure can be used from a local or non-local human-computer interaction device (e.g. Figure 1 The terminal devices 101, 102, 103) shown in the figure obtain a feature extraction neural network, which is not limited in this application.
[0065] Step 402, obtaining structural information of the feature extraction neural network.
[0066] In this embodiment, the structure of the feature extraction neural network is analyzed according to the functions of each part in the feature extraction neural network to obtain the structural information of the feature extraction neural network, which includes at least the position information of the convolutional layer of the feature extraction neural network to indicate the number of convolutional layers included in the feature extraction neural network, the connection relationship between each convolutional layer and the positional relationship of each convolutional layer, so as to facilitate the subsequent generation of a feature recognition neural network based on the position information of the convolutional layer.
[0067] Step 403, in response to receiving a request to generate a feature recognition neural network, obtaining structure information and added layer information in the request to generate the neural network.
[0068] In this embodiment, when a request for generating a feature recognition neural network is received, that is, when a request for generating a feature recognition neural network based on a feature extraction neural network is received, structural information of the feature extraction neural network used to generate the feature recognition neural network and the corresponding added layer in the generation request are obtained, and the added layer includes at least the quantity information and connection structure information of the central entropy supervision layer, so as to facilitate the subsequent addition of the central entropy supervision layer to the feature extraction neural network according to the quantity information and connection structure information of the central entropy supervision layer.
[0069] Among them, the center entropy supervision layer contains the center loss function (center loss):
[0070]
[0071] Among them, c yi Indicates the yth i The feature center of each category, x i represents the features before the fully connected layer, and m represents the number of samples of the data processed in the current batch. Through this center loss function, the feature difference of each feature category can be increased, a category center can be provided for each category, and the distance between each sample in each batch of data and the corresponding category center can be minimized, so as to achieve the purpose of reducing the distance within the class.
[0072] In some optional implementations of this embodiment, the added layer includes: a central entropy supervision layer, which is separately connected after the last convolution layer of the feature extraction neural network.
[0073] Specifically, a central entropy supervision layer is connected to the last convolutional layer of the feature extraction neural network to better cluster the extracted features through the central entropy supervision layer, shorten the distance between each sample and the corresponding sample center during the processing process, and reduce the intra-class distance.
[0074] Step 404: Based on the structural information and the added layer information, the feature extraction neural network is adjusted to generate a feature recognition neural network.
[0075] In this embodiment, after obtaining the structural information and added layer information of the feature extraction neural network, the above-mentioned execution entity adds the added layer to the corresponding structure of the image segmentation neural network based on the instructions in the added layer information and the structural information of the feature extraction neural network to obtain the feature recognition neural network.
[0076] The method for generating a feature recognition neural network provided in this embodiment, after obtaining the feature extraction neural network, adds a central entropy supervision layer according to the structural information of the feature extraction neural network to generate a feature recognition neural network. The feature recognition neural network not only has a high convergence speed during training, but can also complete the training with only a small amount of training materials, thereby improving the training efficiency of the neural network. In addition, in the subsequent process of using the feature recognition neural network to realize feature recognition, it has improved recognition generalization applicability and recognition quality.
[0077] In some optional implementations of this embodiment, the structural information also includes: position information of the fully connected layer of the feature extraction neural network.
[0078] Specifically, the position information of the fully connected layer of the feature extraction neural network can also be obtained, so as to add an additional layer connected to the fully connected layer according to the position information of the fully connected layer, so as to further improve the quality of the obtained feature recognition neural network.
[0079] In some optional implementations of this embodiment, adding layer information also includes: a cross entropy loss layer, and the cross entropy loss layer is separately connected to the fully connected layer of the feature extraction neural network.
[0080] Specifically, after obtaining the fully connected layer of the feature extraction neural network, a cross entropy loss layer can be added after the fully connected layer. The meaning of cross entropy is the difficulty of text recognition using the model, or from the perspective of compression, how many bits are used to encode each word on average. The meaning of complexity is the average number of branches that the model uses to represent the text, and its reciprocal can be regarded as the average probability of each word. Smoothing refers to assigning a probability value to the unobserved N-gram combination to ensure that the word sequence can always obtain a probability value through the language model. Therefore, the cross entropy loss layer can be determined based on a cross entropy loss function to achieve data aggregation and obtain the final feature classification result.
[0081] To deepen the understanding, this application also combines a specific application scenario and provides a specific implementation scheme. In this specific application scenario, in order to identify the emotional features in face images, the VGG11 neural network is used as the feature extraction neural network to generate the feature recognition neural network, and the added layers include the central entropy supervision layer whose quantity information is sequentially connected after the last convolutional layer of the VGG11 neural network, and the cross entropy loss layer connected after the fully connected layer of the VGG11 neural network.
[0082] Get the original image A of size 128x128, use the face detection model to determine the face area in the original image, and divide the facial expressions into 7 basic expressions according to the changes of facial muscles, namely anger, disgust, fear, happiness, sadness, surprise and neutral. And define the face to contain 72 key points, and then use the key point annotation model to add the first key point information to the face area, which are (x 1 ,y 1 )…(x 72 ,y 72 ).
[0083] A coordinate system is determined based on the first key point information, and coordinate information corresponding to the first key point information is generated. Then, face alignment is performed on the face image in the face area according to the coordinate values of each key point in the first key point information. At the same time, the face area is intercepted through affine transformation and adjusted to the same size as the original image size of 128x128.
[0084] Use the above-mentioned key point annotation model to re-add the second key point marking information to the adjusted facial image area, and determine the coordinates of the second key point marking information according to the coordinate system determined based on the first key point information, and based on the coordinate information of the second key point information, determine the outline of the facial image in the facial image area, and delete the content outside the facial image based on the determined coordinates of the second key point information.
[0085] Then, the pixel value of each pixel of the content of the face image part is subtracted by 128 and then divided by 256, so that the pixel value of each pixel is between [-0.5, 0.5]. Then, the obtained face image area is subjected to random data enhancement processing to obtain the face image A1 to be recognized.
[0086] The feature extraction neural network is obtained as a VGG11 neural network, and the structural information of the VGG11 neural network is parsed to obtain the position information of the last convolutional layer and the position information of the fully connected layer of the VGG11 neural network.
[0087] A request for generating a feature recognition neural network is received, and added layer information is obtained, which is a center entropy supervision layer that is separately connected to the last convolutional layer of the VGG1 neural network and a cross entropy loss layer that is separately connected to the fully connected layer of the VGG1 neural network. Based on the added layer information and the structural information of the VGG11 neural network, a center entropy supervision layer is added after the last convolutional layer of the VGG11 neural network, and a cross entropy loss layer is added after the fully connected layer of the VGG11 neural network to obtain a feature recognition neural network B.
[0088] The image A1 to be identified is input into the feature recognition neural network B, and finally the emotion feature recognition result of the face image is obtained.
[0089] It can be seen from this application scenario that the facial image recognition method provided in the embodiment of the present application obtains the preprocessed facial image to be recognized, then inputs the facial image to be recognized into a feature recognition neural network to obtain feature recognition information output by the feature recognition neural network; wherein the feature recognition neural network includes an additional layer; based on the obtained feature recognition information, a feature recognition result of the facial image to be recognized is generated, and feature recognition of the facial image is performed by the feature recognition neural network including the additional layer, thereby improving the accuracy of the feature recognition results for the facial image.
[0090] like Figure 5As shown, the facial image recognition device 500 of this embodiment may include: an image acquisition unit 501, configured to acquire a preprocessed facial image to be recognized; a feature recognition unit 502, configured to input the facial image to be recognized into a feature recognition neural network to obtain feature recognition information output by the feature recognition neural network; wherein the feature recognition neural network includes an adding layer; and a recognition result generation unit 503, configured to generate a feature recognition result of the facial image to be recognized based on the feature recognition information.
[0091] In some optional implementations of the present embodiment, the above-mentioned facial image recognition device also includes: a feature extraction neural network acquisition unit, configured to acquire a feature extraction neural network; a feature extraction neural network structure analysis unit, configured to acquire structural information of the feature extraction neural network; wherein the structural information includes at least convolutional layer position information; an added layer information acquisition unit, configured to, in response to receiving a generation request for a feature recognition neural network, acquire the structural information and the added layer information in the generation request; wherein the added layer information includes at least the number information and connection structure information of the central entropy supervision layer; a feature recognition network generation unit, configured to adjust the feature extraction generation network based on the structural information and the added layer information to generate the feature recognition neural network.
[0092] In some optional implementations of this embodiment, the added layer information in the added layer information acquisition unit includes: a central entropy supervision layer, and the central entropy supervision layer is separately connected after the last convolution layer of the feature extraction neural network.
[0093] In some optional implementations of this embodiment, the structural information in the feature extraction neural network structure analysis unit also includes: position information of the fully connected layer of the feature extraction neural network.
[0094] In some optional implementations of this embodiment, the added layer information in the added layer information acquisition unit also includes: a cross entropy loss layer, and the cross entropy loss layer is separately connected after the fully connected layer of the feature extraction neural network.
[0095] In some optional implementations of the present embodiment, the above-mentioned face image recognition device also includes: an image preprocessing unit 504, which is configured to obtain an original image, and use a face detection model to determine a face image area in the original image; use a key point labeling model to add first key point labeling information to the face image area; adjust the face image in the face image area based on the first key point labeling information to obtain an adjusted face image area; use the key point labeling model to re-add second key point labeling information to the adjusted face image area; remove the non-face image part based on the second key point information to obtain the preprocessed face image to be recognized.
[0096] In some optional implementations of the present embodiment, the image preprocessing unit 504 removes non-face image parts based on the second key point information to obtain the preprocessed face image to be identified, and further includes: normalizing the face image area obtained after removing the non-face image part based on the second key point information; and performing random data enhancement processing on the normalized face image area to obtain the face image to be identified.
[0097] This embodiment exists as a device embodiment corresponding to the above method embodiment. For the same content, refer to the description of the above method embodiment, and no further description is given. The face image recognition device provided by the embodiment of the present application can improve the accuracy of the result of feature recognition for face images by performing feature recognition on face images through a feature recognition neural network including an added layer.
[0098] like Figure 6 , is a block diagram of an electronic device according to a method for recognizing a face image according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.
[0099] like Figure 6 As shown, the electronic device includes: one or more processors 601, a memory 602, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 601 is taken as an example.
[0100] The memory 602 is a non-transitory computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable the at least one processor to perform the facial image recognition method provided in the present application. The non-transitory computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to perform the facial image recognition method provided in the present application.
[0101] The memory 602 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the method for recognizing facial features in the embodiment of the present application (for example, Figure 5 The processor 601 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 602, that is, the face image recognition method in the above method embodiment is implemented.
[0102] The memory 602 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created according to the use of the electronic device for recognizing facial images, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 602 may optionally include a memory remotely arranged relative to the processor 601, and these remote memories may be connected to the electronic device for recognizing facial images via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0103] The electronic device for executing the face image recognition method may further include: an input device 603 and an output device 604. The processor 601, the memory 602, the input device 603 and the output device 604 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.
[0104] The input device 603 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device for recognizing the face image, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 604 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.
[0105] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the above storage system, the above at least one input device, and the above at least one output device.
[0106] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.
[0107] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0108] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0109] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.
[0110] According to the technical solution of the embodiment of the present application, after obtaining the preprocessed face image to be identified, the face image to be identified is input into a feature recognition neural network to obtain feature recognition information output by the feature recognition neural network; wherein the feature recognition neural network includes an additional layer; based on the obtained feature recognition information, a feature recognition result of the face image to be identified is generated, and feature recognition of the face image is performed by the feature recognition neural network including the additional layer, thereby improving the accuracy of the feature recognition results for the face image.
[0111] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.
[0112] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.
Claims
1. A method for recognizing a face image, comprising: Obtain the preprocessed face image to be recognized; Inputting the face image to be identified into a feature recognition neural network to obtain feature recognition information output by the feature recognition neural network; wherein the feature recognition neural network includes an addition layer; Based on the feature recognition information, generating a feature recognition result of the face image to be recognized; The step of generating the feature recognition neural network includes: obtaining a feature extraction neural network; obtaining structural information of the feature extraction neural network; wherein the structural information includes at least convolutional layer position information; in response to receiving a generation request for the feature recognition neural network, obtaining the structural information and added layer information in the generation request; wherein the added layer information includes at least: a central entropy supervision layer, the central entropy supervision layer is separately connected after the last convolutional layer of the feature extraction neural network; based on the structural information and the added layer information, adjusting the feature extraction generation network to generate the feature recognition neural network.
2. The method according to claim 1, wherein: The structural information also includes: The feature extracts position information of a fully connected layer of a neural network.
3. The method according to claim 2, wherein: The added layer information also includes: A cross entropy loss layer, wherein the cross entropy loss layer is connected alone after the fully connected layer of the feature extraction neural network.
4. The method according to claim 1, wherein: The pre-processed face image to be recognized is determined by the following steps: Acquire an original image, and use a face detection model to determine a face image region in the original image; Using a key point annotation model to add first key point marking information to the face image area; Adjust the face image in the face image area based on the first key point marking information to obtain an adjusted face image area; Using the key point annotation model to re-add second key point marking information to the adjusted face image area; The non-face image part is removed based on the second key point information to obtain the preprocessed face image to be recognized.
5. The method according to claim 4, wherein: The step of removing the non-face image portion based on the second key point information to obtain the preprocessed face image to be recognized further comprises: After removing the non-face image part based on the second key point information, the obtained face image area is normalized; The normalized face image region is subjected to random data enhancement processing to obtain the face image to be recognized.
6. A facial image recognition device, comprising: An image acquisition unit, configured to acquire a pre-processed face image to be recognized; A feature recognition unit is configured to input the face image to be recognized into a feature recognition neural network to obtain feature recognition information output by the feature recognition neural network; wherein the feature recognition neural network includes an addition layer; A recognition result generating unit, configured to generate a feature recognition result of the face image to be recognized based on the feature recognition information; A feature extraction neural network acquisition unit is configured to acquire a feature extraction neural network; a feature extraction neural network structure analysis unit is configured to acquire structural information of the feature extraction neural network; wherein the structural information includes at least convolutional layer position information; an added layer information acquisition unit is configured to, in response to receiving a generation request for a feature recognition neural network, acquire the structural information and the added layer information in the generation request; wherein the added layer information includes at least: a central entropy supervision layer, the central entropy supervision layer is separately connected to the last convolutional layer of the feature extraction neural network; a feature recognition network generation unit is configured to adjust the feature extraction generation network based on the structural information and the added layer information to generate the feature recognition neural network.
7. The device according to claim 6, wherein: The structural information in the feature extraction neural network structure analysis unit also includes: The feature extracts position information of a fully connected layer of a neural network.
8. The device according to claim 7, wherein: The added layer information in the added layer information acquisition unit also includes: A cross entropy loss layer, wherein the cross entropy loss layer is connected alone after the fully connected layer of the feature extraction neural network.
9. The apparatus according to claim 6, further comprising: An image preprocessing unit is configured to obtain an original image and determine a face image region in the original image using a face detection model; Using a key point annotation model to add first key point marking information to the face image area; Adjust the face image in the face image area based on the first key point marking information to obtain an adjusted face image area; Using the key point annotation model to re-add second key point marking information to the adjusted face image area; The non-face image part is removed based on the second key point information to obtain the preprocessed face image to be recognized.
10. The device according to claim 9, wherein: The image preprocessing unit removes the non-face image part based on the second key point information to obtain the preprocessed face image to be recognized, and further comprises: After removing the non-face image part based on the second key point information, the obtained face image area is normalized; The normalized face image region is subjected to random data enhancement processing to obtain the face image to be recognized.
11. An electronic device, comprising: at least one processor; as well as A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the facial image recognition method described in any one of claims 1-5.
12. A non-transitory computer-readable storage medium storing computer instructions, comprising: The computer instructions are used to enable the computer to execute the facial image recognition method according to any one of claims 1 to 5.
Citation Information
Patent Citations
Facial image processing method, training method, equipment and storage medium
CN109712144A
Vehicle passing record determination method and device, equipment and storage medium
CN110135318A
Unsupervised representation learning by deep convolution autoencoder for image clustering
IN201821049728A
Blind signal processing system employing information maximization to recover unknown signals through unsupervised minimization of output redundancy
WO1996017309A1
