A face retrieval method, system, computer device and storage medium

Through the combination of pixel mask network and feature extraction network, local interference problems in face retrieval in crowded intersections are solved, the accuracy and speed of face retrieval is improved, and more efficient face recognition effect is achieved.

CN113569626BActive Publication Date: 2024-11-22HUNAN YOUMEI TECH DEV CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110656291.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-06-11
Publication Date
2024-11-22
Estimated Expiration
2041-06-11

AI Technical Summary

Technical Problem

In non-cooperating security scenarios, especially crowded intersections, existing face recognition or retrieval technologies are difficult to effectively deal with local face foreground interference, resulting in low accuracy in face retrieval in large-scale populations.

Method used

The pixel masked face image is obtained by using the pixel mask network, the masked face image is extracted through the face floating point feature extraction network and the deep hash feature extraction network, and the feature comparison module is used for fine comparison, and the loss function is dynamically adjusted to reduce the spacing within the class and increase the class. The spacing between the two reduces interference in key areas of non-effective faces and improves retrieval accuracy.

Benefits of technology

It has achieved the improvement of the accuracy and speed of face retrieval in large-scale populations, reduced interference in key areas of non-effective faces, and enhanced the discrimination and accuracy of searches.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113569626B_ABST
    Figure CN113569626B_ABST
Patent Text Reader

Abstract

The present application relates to a face retrieval method, system, computer device and storage medium. The method comprises: importing a face image into a face image and key point detection network, obtaining a masked face image through a pixel mask network; sending the masked face image into a face feature extraction network, obtaining face floating point features and face hash features; importing the bottom database face floating point features and the bottom database face hash features into a feature comparison module, and comparing the face floating point features with the bottom database face floating point features, and the face hash features with the bottom database face hash features through the feature comparison module, obtaining a detailed comparison result, and completing the face retrieval according to the detailed comparison result. The embodiment of the present invention can realize face retrieval of a large number of people, reduce the interference of non-valid face key areas, and improve the accuracy of face retrieval.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a face retrieval method, system, computer device and storage medium. Background Art

[0002] With the continuous breakthroughs in deep learning technology and the continuous improvement of computing power, the field of computer vision has made breakthrough progress. As a key research direction in the field of computer vision, face retrieval is widely used in security, social networking, daily entertainment and other fields. In recent years, the accuracy of face retrieval has been significantly improved, and the improvement is mainly attributed to two aspects: massive face images and discriminative face feature extraction methods, including traditional face feature extraction methods and deep neural network face feature extraction methods.

[0003] However, many existing face recognition or retrieval technologies use spatial / channel attention mechanisms to enhance the foreground and reduce the impact of background information. They are unable to solve the interference problem of foreground such as some local faces in face images in non-cooperative security scenarios, especially at crowded intersections, resulting in low accuracy of face retrieval in large crowds. Summary of the invention

[0004] Based on this, it is necessary to provide a face retrieval method, system, computer device and storage medium to address the above technical problems.

[0005] In a first aspect, an embodiment of the present invention provides a face retrieval method, comprising the following steps:

[0006] Import the face image into the face image and key point detection network, and obtain the masked face image through the pixel mask network;

[0007] Sending the masked face image to a face feature extraction network to obtain face floating point features and face hash features;

[0008] The base database face floating-point features and the base database face hash features are imported into the feature comparison module, and the face floating-point features are compared with the base database face floating-point features, and the face hash features are compared with the base database face hash features through the feature comparison module to obtain detailed comparison results, and the face retrieval is completed according to the detailed comparison results.

[0009] Further, obtaining the masked face image includes:

[0010] Sending the aligned face images obtained by processing the face detection and key point detection networks into the encoding subnetwork to extract features and obtain a high-dimensional heat map;

[0011] Sending the high-dimensional heat map to a decoding subnetwork to obtain a pixel mask of the original size of the aligned face image, wherein the aligned face image includes one or more foreground areas and one or more background areas;

[0012] The average pixel value of the effective target face key area is calculated, and the pixel value of each pixel in the non-effective target face key area in the aligned face image is replaced with the average pixel value to obtain a masked face image.

[0013] Furthermore, the acquisition of the face floating point feature and the face hash feature includes:

[0014] The face feature extraction network includes a floating point feature extraction subnetwork and a deep hashing subnetwork;

[0015] Sending the masked face image to the floating-point feature extraction subnetwork to obtain the face floating-point feature; the face floating-point feature is a high-dimensional feature vector composed of a series of floating-point numbers;

[0016] The facial floating-point feature is sent to the deep hash subnetwork to obtain the facial hash feature; the facial hash feature is a low-dimensional feature vector composed of a series of 0s and 1s.

[0017] Furthermore, it also includes network training for face retrieval, and the network training for face retrieval includes:

[0018] The aligned face images are divided into a training set and a validation set, and each aligned face image is labeled with a face ID;

[0019] The training set is sent to the pixel mask network, the parameters of the pixel mask network are updated in combination with the pixel mask loss, and then the validation set is used to verify whether the pixel mask network converges, so as to obtain an optimal pixel mask model;

[0020] The training set processed by the pixel mask network is sent to the floating-point feature extraction subnetwork, the parameters of the floating-point feature extraction subnetwork are updated in combination with the improved interval classification loss, and then the verification set is used to verify whether the floating-point feature extraction subnetwork converges, so as to obtain a high-dimensional floating-point feature extraction model of human face;

[0021] The training set processed by the pixel mask network is sent to the face feature extraction network. On the basis of the convergence of the floating-point feature extraction subnetwork, the parameters of the floating-point feature extraction subnetwork are frozen, and the parameters of the deep hash subnetwork are jointly updated in combination with the improved classification loss and the binary quantization loss. The validation set is then used to verify whether the deep hash subnetwork has converged, so as to obtain a low-dimensional face hash feature extraction model.

[0022] Further, obtaining the detailed comparison result includes:

[0023] Using methods including but not limited to Hamming distance or hash bucket, perform a rough search on the query face hash features and the base database face hash features to obtain top N rough search results;

[0024] For the first N rough search results, the similarities between the base database face floating-point features and the query face floating-point features are calculated one by one, the one with the greatest similarity is obtained, and the one with the greatest similarity and its similarity are output.

[0025] Furthermore, it also includes acquiring facial information of a base database, and the acquisition of facial information of a base database includes:

[0026] Send the image to be added to the base face information database and the base face ID to the face detection and key point detection network to obtain the face floating point feature and the face hash feature of the image;

[0027] The base database face information is output according to the base database face identity ID, the face floating point features and the face hash features of the image.

[0028] Furthermore, the construction of the base face information database includes four functional operations on the base face information database, including:

[0029] A new addition operation is performed according to the input facial information of the base database, a modification operation is performed according to the input facial information of the base database, a query operation is performed according to the input facial identity ID, and a deletion operation is performed according to the input facial identity ID.

[0030] On the other hand, an embodiment of the present invention further provides a face retrieval system, comprising:

[0031] The masked face image module is used to import the face image into the face image and key point detection network, and obtain the masked face image through the pixel mask network;

[0032] A feature extraction module, used for sending the masked face image into a face feature extraction network to obtain face floating point features and face hash features;

[0033] The comparison and retrieval module is used to import the base database face floating-point features and the base database face hash features into the feature comparison module, and compare the face floating-point features with the base database face floating-point features, and the face hash features with the base database face hash features through the feature comparison module to obtain detailed comparison results, and complete the face retrieval according to the detailed comparison results.

[0034] An embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0035] Import the face image into the face image and key point detection network, and obtain the masked face image through the pixel mask network;

[0036] Sending the masked face image to a face feature extraction network to obtain face floating point features and face hash features;

[0037] The base database face floating-point features and the base database face hash features are imported into the feature comparison module, and the face floating-point features are compared with the base database face floating-point features, and the face hash features are compared with the base database face hash features through the feature comparison module to obtain a detailed comparison result, and the face retrieval is completed according to the detailed comparison result.

[0038] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0039] Import the face image into the face image and key point detection network, and obtain the masked face image through the pixel mask network;

[0040] Sending the masked face image to a face feature extraction network to obtain face floating point features and face hash features;

[0041] The base database face floating-point features and the base database face hash features are imported into the feature comparison module, and the face floating-point features are compared with the base database face floating-point features, and the face hash features are compared with the base database face hash features through the feature comparison module to obtain a detailed comparison result, and the face retrieval is completed according to the detailed comparison result.

[0042] The beneficial effects of the present application are as follows: the embodiment of the present invention proposes to apply binary quantization loss to the face hash feature layer under the premise of convergence of the face floating-point feature extraction network, so as to ensure that the deep hash network converges quickly and shares weights with the face floating-point feature extraction network, so as to realize that the neural network undergoes one forward reasoning and simultaneously obtains discriminative face floating-point features and face hash features for subsequent rough retrieval and fine comparison, thereby accelerating the retrieval speed and improving the retrieval accuracy. Secondly, an improved loss function of the floating-point feature extraction network and the deep hash network is proposed, which can dynamically adjust the interval penalty factor step by step, gradually reduce the intra-class spacing and increase the inter-class spacing under the premise of ensuring the normal convergence of the network, so as to obtain more discriminative face features. Finally, a pixel mask network is used to obtain the pixel mask of the face image, and the foreground area of ​​the face image with the largest proportion of the pixel mask is selected as the effective face key area, and other foreground areas and background areas are replaced by the average pixels of the face image, so as to reduce the interference of the ineffective face key area and improve the accuracy of face retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 1 is a flow chart of a face retrieval method in one embodiment;

[0044] Figure 2 A schematic diagram of a process for obtaining a masked face image according to an embodiment;

[0045] Figure 3 A schematic diagram of a process for obtaining floating-point features and hash features of a face in one embodiment;

[0046] Figure 4 A schematic diagram of a flow chart of a network training method for face retrieval in one embodiment;

[0047] Figure 5 A schematic diagram of a process for obtaining a detailed comparison result according to an embodiment;

[0048] Figure 6 is a structural block diagram of a face retrieval system in one embodiment;

[0049] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0051] In one embodiment, Figure 1 As shown, a face retrieval method is provided, comprising the following steps:

[0052] Step 101, importing a face image into a face image and key point detection network, and obtaining a masked face image through a pixel mask network;

[0053] Step 102, sending the masked face image to a face feature extraction network to obtain face floating point features and face hash features;

[0054] Step 103, import the base database face floating-point features and the base database face hash features into the feature comparison module, compare the face floating-point features with the base database face floating-point features, and the face hash features with the base database face hash features through the feature comparison module to obtain detailed comparison results, and complete the face retrieval based on the detailed comparison results.

[0055] Specifically, by applying binary quantization loss to the face hash feature layer under the premise of convergence of the face floating-point feature extraction network, the deep hash network is ensured to converge quickly and share weights with the face floating-point feature extraction network. The neural network can obtain discriminative face floating-point features and face hash features at the same time after one forward reasoning, which are used for subsequent rough retrieval and fine comparison, accelerating the retrieval speed and improving the retrieval accuracy. Secondly, an improved loss function of the floating-point feature extraction network and the deep hash network is proposed, which can dynamically adjust the interval penalty factor step by step, gradually reduce the intra-class spacing and increase the inter-class spacing under the premise of ensuring the normal convergence of the network, so as to obtain more discriminative face features. Finally, the pixel mask network is used to obtain the pixel mask of the face image, and the foreground area with the largest proportion of the pixel mask in the entire face image is selected as the effective face key area. The other foreground areas and background areas are replaced by the average pixels of the face image to reduce the interference of non-effective face key areas and improve the accuracy of face retrieval.

[0056] In one embodiment, Figure 2 As shown, obtaining a masked face image includes the following steps:

[0057] Step 201, sending the aligned face image obtained by face detection and key point detection network processing to the encoding sub-network to extract features and obtain a high-dimensional heat map;

[0058] Step 202, sending the high-dimensional heat map to a decoding subnetwork to obtain a pixel mask of the original size of the aligned face image, where the aligned face image includes one or more foreground areas and one or more background areas;

[0059] Step 203, calculate the average pixel value of the effective target face key area, replace the pixel value of each pixel in the non-effective target face key area in the aligned face image with the average pixel value, and obtain a masked face image.

[0060] Preferably, obtaining the face floating point features and the face hash features includes:

[0061] Step 301, sending the masked face image to the floating point feature extraction sub-network to obtain the face floating point feature; the face floating point feature is a high-dimensional feature vector composed of a series of floating point numbers;

[0062] Step 302, sending the facial floating-point features to the deep hash subnetwork to obtain facial hash features; the facial hash features are low-dimensional feature vectors consisting of a series of 0s and 1s.

[0063] Among them, the face detection and key point detection networks use including but not limited to multi-task convolutional neural networks, the loss function of the pixel mask network uses cross entropy loss, and the pixel mask network can obtain pixel masks of the background and different foregrounds, and the area with the largest pixel mask ratio in the foreground is selected as the effective target face key area.

[0064] In one embodiment, Figure 4 As shown, the face network training method includes:

[0065] Step 401, dividing the aligned face images into a training set and a validation set, and labeling each aligned face image with a face identity ID;

[0066] Step 402, sending the training set to the pixel mask network, updating the parameters of the pixel mask network in combination with the pixel mask loss, and then using the validation set to verify whether the pixel mask network converges to obtain the optimal pixel mask model;

[0067] Step 403, sending the training set processed by the pixel mask network to the floating-point feature extraction sub-network, updating the parameters of the floating-point feature extraction sub-network in combination with the improved classification loss with intervals, and then using the validation set to verify whether the floating-point feature extraction sub-network has converged, thereby obtaining a high-dimensional floating-point feature extraction model for faces;

[0068] In step 404, the training set processed by the pixel mask network is sent to the face feature extraction network. On the basis of the convergence of the floating-point feature extraction subnetwork, the parameters of the floating-point feature extraction subnetwork are frozen, and the parameters of the deep hash subnetwork are updated together with the improved classification loss and the binary quantization loss. The validation set is then used to verify whether the deep hash subnetwork has converged, and a low-dimensional face hash feature extraction model is obtained.

[0069] Specifically, the training step of the face retrieval method adopts but is not limited to a multi-task convolutional neural network to perform face detection and key point detection on the training image containing the face to obtain an aligned face image. The aligned face image is divided into a training set and a test set. The training set is sent to the pixel mask network, and the parameters of the pixel mask network are updated in combination with the pixel mask loss function, and then the verification set is used to verify whether the pixel mask network converges to obtain the optimal pixel mask model. The training set processed by the pixel mask network is sent to the floating-point feature extraction network, and the parameters of the floating-point feature extraction network are updated in combination with the floating-point loss function, and then the verification set is used to verify whether the floating-point feature extraction network converges to obtain a high-dimensional floating-point feature extraction model of the face. The training set processed by the pixel mask network is sent to the face feature extraction network, and on the basis of the convergence of the floating-point feature extraction network, the parameters of the floating-point feature extraction network are frozen, and the parameters of the deep hash network are updated in combination with the hash loss function, and then the verification set is used to verify whether the deep hash network converges to obtain a low-dimensional face hash feature extraction model. In summary, the network training of the face retrieval method is completed.

[0070] In one embodiment, Figure 5 As shown, obtaining detailed comparison results includes:

[0071] Step 501, using methods including but not limited to Hamming distance or hash bucket, perform a rough search on the query face hash features and the database face hash features to obtain the top N rough search results;

[0072] Step 502, for the first N rough search results, calculate the similarity between the base database face floating-point features and the query face floating-point features one by one, obtain the one with the greatest similarity, and output the one with the greatest similarity and its similarity.

[0073] Preferably, the construction of the base database face information includes: sending the image to be added to the base database face information library and the base database face identity ID into the face detection and key point detection network; outputting the base database face information according to the face floating point features and face hash features of the base database face information library.

[0074] Furthermore, the construction of the base database facial information includes four functional operations on the base database facial information database. The four functional operations include: performing a new operation based on the input base database facial information, performing a modification operation based on the input base database facial information, performing a query operation based on the input facial identity ID, and performing a deletion operation based on the input facial identity ID.

[0075] Specifically, the base database face information construction includes: all base database images are sent to the face detection and face key point detection network to obtain face key points, and after affine transformation, the aligned base database face images are obtained. In addition, all base database images correspond to unique face identity IDs. The aligned base database face images are sent to the pixel mask network to obtain pixel masks of different foregrounds and backgrounds. According to prior knowledge, the target face often accounts for the largest proportion in the aligned base database face images. Therefore, the effective target face key area is obtained according to the proportion of the pixel mask, and then the non-effective target face key area is replaced with the average pixel of the effective target face key area, and then the masked base database face image is obtained. The masked base database face image is sent to the face feature extraction network, which includes a floating point feature extraction network and a deep hash network. After the two networks, high-dimensional base database face floating point features and low-dimensional base database face hash real number features are obtained respectively. The low-dimensional base database face hash real number features are processed by the sign function to obtain the base database face hash features composed of 0 and 1. The face identity ID, the base database face floating point features and the base database face hash features are stored in the disk as a base database face information.

[0076] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0077] In one embodiment, Figure 6 As shown, a face retrieval system is provided, including: a mask face image module 601, a feature extraction module 602, and a comparison retrieval module 603; wherein:

[0078] The masked face image module 601 is used to import the face image into the face image and key point detection network, and obtain the masked face image through the pixel mask network;

[0079] A feature extraction module 602 is used to obtain facial floating point features and facial hash features according to a facial feature extraction network;

[0080] The comparison and retrieval module 603 is used to import the base database face floating-point features and the base database face hash features into the feature comparison module, compare the face floating-point features and the face hash features through the feature comparison module, obtain the detailed comparison results, and complete the face retrieval according to the detailed comparison results.

[0081] In one embodiment, Figure 6 As shown, the masked face image module 601 includes a pixel mask unit 6011, which is used to: send the aligned face image obtained by the face detection and key point detection network processing to the encoding sub-network to extract features and obtain a high-dimensional heat map;

[0082] The high-dimensional heat map is fed into a decoding subnetwork to obtain a pixel mask of the original size of the aligned face image, where the aligned face image includes one or more foreground regions and one or more background regions;

[0083] The average pixel value of the effective target face key area is calculated, and the pixel value of each pixel in the non-effective target face key area in the aligned face image is replaced by the average pixel value to obtain a masked face image.

[0084] In one embodiment, Figure 6 As shown, the feature extraction module 602 is also used to: send the masked face image into a floating-point feature extraction subnetwork to obtain face floating-point features; face floating-point features, a high-dimensional feature vector composed of a series of floating-point numbers; send the face floating-point features into a deep hash subnetwork to obtain face hash features; face hash features, a low-dimensional feature vector composed of a series of 0s and 1s.

[0085] In one embodiment, the face retrieval system further includes a face training module, which is used to:

[0086] The aligned face images are divided into a training set and a validation set, and each aligned face image is labeled with a face ID;

[0087] The training set is sent to the pixel mask network, and the parameters of the pixel mask network are updated in combination with the pixel mask loss. Then the validation set is used to verify whether the pixel mask network converges to obtain the optimal pixel mask model.

[0088] The training set processed by the pixel mask network is sent to the floating-point feature extraction sub-network. The parameters of the floating-point feature extraction sub-network are updated in combination with the improved classification loss with intervals. The validation set is then used to verify whether the floating-point feature extraction sub-network has converged, and a high-dimensional floating-point feature extraction model for faces is obtained.

[0089] The training set processed by the pixel mask network is sent to the face feature extraction network. On the basis of the convergence of the floating-point feature extraction subnetwork, the parameters of the floating-point feature extraction subnetwork are frozen, and the parameters of the deep hash subnetwork are updated together with the improved classification loss and binary quantization loss. The validation set is then used to verify whether the deep hash subnetwork has converged, and a low-dimensional face hash feature extraction model is obtained.

[0090] In one embodiment, the comparison and retrieval module 603 includes a feature comparison unit 6031. The feature comparison unit 6031 first uses the Hamming distance or hash bucket method to perform a rough search on the query face hash features and the base database face hash features to obtain the top N rough search results.

[0091] Then, for the first N rough search results, calculate the similarity between the base database face floating-point features and the query face floating-point features one by one, obtain the one with the greatest similarity, and output the one with the greatest similarity and its similarity.

[0092] Preferably, the comparison and retrieval module 603 also includes a base database face information management unit 6032, which is used to: send the image to be added to the base database face information library and the base database face identity ID to the face detection and key point detection network; output the base database face information according to the face floating point features and face hash features of the base database face information library. The base database face information management unit 6032 is also used to construct the base database face information, including four functional operations on the base database face information library, and the four functional operations include: performing a new addition operation according to the input base database face information, performing a modification operation according to the input base database face information, performing a query operation according to the input face identity ID, and performing a deletion operation according to the input face identity ID.

[0093] For the specific definition of the face retrieval system, please refer to the definition of the face retrieval method above, which will not be repeated here. Each module in the above face retrieval system can be implemented in whole or in part by software, hardware and their combination. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0094] Figure 7 FIG. 2 shows an internal structure diagram of a computer device in one embodiment. Figure 7As shown, the computer device includes a processor, a memory, a network interface, an input device and a display screen connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the permission anomaly detection method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute the permission anomaly detection method. The display screen of the computer device may be a liquid crystal display or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0095] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0096] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: importing a face image into a face image and key point detection network, and obtaining a masked face image through a pixel mask network;

[0097] Obtain facial floating-point features and facial hash features according to the facial feature extraction network;

[0098] The base database face floating-point features and base database face hash features are imported into the feature comparison module, and the face floating-point features and face hash features are compared through the feature comparison module to obtain detailed comparison results, and the face retrieval is completed according to the detailed comparison results.

[0099] In one embodiment, when the processor executes the computer program, the following steps are further implemented: sending the aligned face image obtained by processing the face detection and key point detection network to the encoding subnetwork to extract features and obtain a high-dimensional heat map;

[0100] The high-dimensional heat map is fed into a decoding subnetwork to obtain a pixel mask of the original size of the aligned face image, where the aligned face image includes one or more foreground regions and one or more background regions;

[0101] The average pixel value of the effective target face key area is calculated, and the pixel value of each pixel in the non-effective target face key area in the aligned face image is replaced by the average pixel value to obtain a masked face image.

[0102] In one embodiment, when the processor executes the computer program, the following steps are further implemented: sending the masked face image to the floating point feature extraction subnetwork to obtain the face floating point feature; the face floating point feature is a high-dimensional feature vector composed of a series of floating point numbers;

[0103] The floating-point features of the face are sent to the deep hash subnetwork to obtain the face hash features; the face hash features are low-dimensional feature vectors composed of a series of 0s and 1s.

[0104] In one embodiment, when the processor executes the computer program, the following steps are further implemented: dividing the aligned face images into a training set and a verification set, and labeling each aligned face image with a face identity ID;

[0105] The training set is sent to the pixel mask network, and the parameters of the pixel mask network are updated in combination with the pixel mask loss. Then, the validation set is used to verify whether the pixel mask network converges to obtain the optimal pixel mask model.

[0106] The training set processed by the pixel mask network is sent to the floating-point feature extraction sub-network. The parameters of the floating-point feature extraction sub-network are updated in combination with the improved classification loss with intervals. The validation set is then used to verify whether the floating-point feature extraction sub-network has converged, and a high-dimensional floating-point feature extraction model for faces is obtained.

[0107] The training set processed by the pixel mask network is sent to the face feature extraction network. On the basis of the convergence of the floating-point feature extraction subnetwork, the parameters of the floating-point feature extraction subnetwork are frozen, and the parameters of the deep hash subnetwork are updated together with the improved classification loss and binary quantization loss. The validation set is then used to verify whether the deep hash subnetwork has converged, and a low-dimensional face hash feature extraction model is obtained.

[0108] In one embodiment, when the processor executes the computer program, the following steps are further implemented: using methods including but not limited to Hamming distance or hash bucket, performing a rough search on the query face hash features and the database face hash features to obtain the top N rough search results;

[0109] For the first N rough search results, calculate the similarity between the floating-point features of the base database face and the floating-point features of the query face one by one, obtain the one with the greatest similarity, and output the one with the greatest similarity and its similarity.

[0110] In one embodiment, when the processor executes the computer program, the following steps are also implemented: sending the image to be added to the base database face information database and the base database face identity ID to the face detection and key point detection network;

[0111] According to the facial floating-point features and facial hash features of the base database face information database, the base database face information is output.

[0112] In one embodiment, when the processor executes the computer program, it also implements the following steps: performing a new operation based on the input base database facial information, performing a modification operation based on the input base database facial information, performing a query operation based on the input facial identity ID, and performing a deletion operation based on the input facial identity ID.

[0113] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: importing a face image into a face image and key point detection network, and obtaining a masked face image through a pixel mask network;

[0114] Obtain facial floating-point features and facial hash features according to the facial feature extraction network;

[0115] The base database face floating-point features and base database face hash features are imported into the feature comparison module, and the face floating-point features and face hash features are compared through the feature comparison module to obtain detailed comparison results, and the face retrieval is completed according to the detailed comparison results.

[0116] In one embodiment, when the processor executes the computer program, the following steps are further implemented: sending the aligned face image obtained by processing the face detection and key point detection network to the encoding subnetwork to extract features and obtain a high-dimensional heat map;

[0117] The high-dimensional heat map is fed into a decoding subnetwork to obtain a pixel mask of the original size of the aligned face image, where the aligned face image includes one or more foreground regions and one or more background regions;

[0118] The average pixel value of the effective target face key area is calculated, and the pixel value of each pixel in the non-effective target face key area in the aligned face image is replaced by the average pixel value to obtain a masked face image.

[0119] In one embodiment, when the processor executes the computer program, the following steps are further implemented: sending the masked face image to the floating point feature extraction subnetwork to obtain the face floating point feature; the face floating point feature is a high-dimensional feature vector composed of a series of floating point numbers;

[0120] The floating-point features of the face are sent to the deep hash subnetwork to obtain the face hash features; the face hash features are low-dimensional feature vectors composed of a series of 0s and 1s.

[0121] In one embodiment, when the processor executes the computer program, the following steps are further implemented: dividing the aligned face images into a training set and a verification set, and marking each aligned face image with a face identity ID;

[0122] The training set is sent to the pixel mask network, and the parameters of the pixel mask network are updated in combination with the pixel mask loss. Then, the validation set is used to verify whether the pixel mask network converges to obtain the optimal pixel mask model.

[0123] The training set processed by the pixel mask network is sent to the floating-point feature extraction sub-network. The parameters of the floating-point feature extraction sub-network are updated in combination with the improved classification loss with intervals. The validation set is then used to verify whether the floating-point feature extraction sub-network has converged, and a high-dimensional floating-point feature extraction model for faces is obtained.

[0124] The training set processed by the pixel mask network is sent to the face feature extraction network. On the basis of the convergence of the floating-point feature extraction subnetwork, the parameters of the floating-point feature extraction subnetwork are frozen, and the parameters of the deep hash subnetwork are updated together with the improved classification loss and binary quantization loss. The validation set is then used to verify whether the deep hash subnetwork has converged, and a low-dimensional face hash feature extraction model is obtained.

[0125] In one embodiment, when the processor executes the computer program, the following steps are further implemented: using methods including but not limited to Hamming distance or hash bucket, performing a rough search on the query face hash features and the database face hash features to obtain the top N rough search results;

[0126] For the first N rough search results, calculate the similarity between the floating-point features of the base database face and the floating-point features of the query face one by one, obtain the one with the greatest similarity, and output the one with the greatest similarity and its similarity.

[0127] In one embodiment, when the processor executes the computer program, the following steps are also implemented: sending the image to be added to the base database face information database and the base database face identity ID to the face detection and key point detection network;

[0128] According to the facial floating-point features and facial hash features of the base database face information database, the base database face information is output.

[0129] In one embodiment, when the processor executes the computer program, it also implements the following steps: performing a new operation based on the input base database facial information, performing a modification operation based on the input base database facial information, performing a query operation based on the input facial identity ID, and performing a deletion operation based on the input facial identity ID.

[0130] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0131] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A face retrieval method, characterized in that: The following steps are involved: Import the face image into the face image and key point detection network to obtain the aligned face image, and obtain the masked face image through the pixel mask network; Send the masked face image to a face feature extraction network to obtain face floating-point features and face hash features, wherein the face floating-point features are high-dimensional feature vectors composed of a series of floating-point numbers, and the face hash features are low-dimensional feature vectors composed of a series of 0s and 1s; Importing the base database face floating point features and the base database face hash features into the feature comparison module, the face floating point features are compared with the base database face floating point features, and the face hash features are compared with the base database face hash features through the feature comparison module to obtain a detailed comparison result, and completing the face retrieval according to the detailed comparison result; The step of obtaining the masked face image through the pixel mask network includes: Sending the aligned face image obtained by processing the face image and the key point detection network to the encoding subnetwork to extract features and obtain a high-dimensional heat map; Sending the high-dimensional heat map to a decoding subnetwork to obtain a pixel mask of the original size of the aligned face image, wherein the aligned face image includes one or more foreground areas and one or more background areas; The average pixel value of the effective target face key area is calculated, and the pixel value of each pixel in the non-effective target face key area in the aligned face image is replaced with the average pixel value to obtain a masked face image.

2. The face retrieval method according to claim 1, characterized in that: The acquisition of the face floating point feature and the face hash feature includes: The face feature extraction network includes a floating point feature extraction subnetwork and a deep hashing subnetwork; Sending the masked face image to the floating-point feature extraction subnetwork to obtain the floating-point features of the face; The facial floating-point features are sent to the deep hash subnetwork to obtain the facial hash features.

3. The face retrieval method according to claim 1, characterized in that: It also includes network training for face retrieval, wherein the network training for face retrieval includes: The aligned face images are divided into a training set and a validation set, and each aligned face image is labeled with a face ID; The training set is sent to the pixel mask network, the parameters of the pixel mask network are updated in combination with the pixel mask loss, and then the validation set is used to verify whether the pixel mask network converges, so as to obtain an optimal pixel mask model; The training set processed by the pixel mask network is sent to the floating-point feature extraction subnetwork, the parameters of the floating-point feature extraction subnetwork are updated in combination with the improved interval classification loss, and then the verification set is used to verify whether the floating-point feature extraction subnetwork converges, so as to obtain a high-dimensional floating-point feature extraction model of human face; The training set processed by the pixel mask network is sent to the face feature extraction network. On the basis of the convergence of the floating-point feature extraction subnetwork, the parameters of the floating-point feature extraction subnetwork are frozen, and the parameters of the deep hash subnetwork are jointly updated in combination with the improved classification loss and the binary quantization loss. The validation set is then used to verify whether the deep hash subnetwork has converged, so as to obtain a low-dimensional face hash feature extraction model.

4. The face retrieval method according to claim 1, characterized in that: Obtaining the detailed comparison result includes: Using methods including but not limited to Hamming distance or hash bucket, perform a rough search on the query face hash features and the base database face hash features to obtain top N rough search results; For the first N rough search results, the similarities between the base database face floating-point features and the query face floating-point features are calculated one by one, the one with the greatest similarity is obtained, and the one with the greatest similarity and its similarity are output.

5. The face retrieval method according to claim 1, characterized in that: The acquisition of facial information from the base database includes: Send the image to be added to the base database face information database and the base database face identity ID into the face image and key point detection network to obtain the face floating point feature and the face hash feature of the image; The base database face information is output according to the base database face identity ID, the face floating point features and the face hash features of the image.

6. The face retrieval method according to claim 5, characterized in that: The construction of the base face information database includes four functional operations on the base face information database, including: A new addition operation is performed according to the input facial information of the base database, a modification operation is performed according to the input facial information of the base database, a query operation is performed according to the input facial identity ID, and a deletion operation is performed according to the input facial identity ID.

7. A face retrieval system based on the face retrieval method according to any one of claims 1 to 6, characterized in that: include: A masked face image module, used to import a face image into the face image and key point detection network, and obtain a masked face image through a pixel mask network; A feature extraction module, used to obtain facial floating-point features and facial hash features based on a facial feature extraction network; The comparison and retrieval module is used to import the base database face floating-point features and the base database face hash features into the feature comparison module, and compare the base database face floating-point features with the face floating-point features, and the base database face hash features with the face hash features through the feature comparison module to obtain detailed comparison results, and complete the face retrieval based on the detailed comparison results.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method of any one of claims 1 to 6 are implemented.