Face retrieval method and device based on local features, equipment and storage medium

Abstract

The application provides a face retrieval method and device based on local features, equipment and storage medium. The method comprises the following steps: extracting a query face feature and an original base library face feature, selecting a target base library picture according to the cosine similarity of the face feature and the original base library face feature; obtaining local feature maps corresponding to the query picture and the target base library picture respectively, calculating the distance between the local feature maps, and determining the weight corresponding to the local feature maps; according to the local feature maps of the query picture, the local feature maps of the target base library picture, the distance and the weight, calculating the distance between the local feature maps of the query picture and the local feature maps of the target base library picture, minimizing the distance, and taking the target base library picture corresponding to the minimum distance as the retrieval result corresponding to the query picture. The application improves the face retrieval accuracy and retrieval effect under difficult example samples.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a face retrieval method and device based on local features, an apparatus, and a storage medium. BACKGROUND

[0002] With the rapid development of face recognition technology, face retrieval algorithms have been widely applied, for example, in personnel clock-in projects. Existing face retrieval algorithms calculate the similarity of extracted face features and face features in a base library, and then obtain a retrieval result. However, when at least one of a query image and a base library image has a problem such as occlusion or illumination, the similarity of the two will be very low, which causes the existing face retrieval algorithm to easily mis-retrieve difficult example samples, and reduces face retrieval accuracy and retrieval effect. SUMMARY

[0003] Therefore, embodiments of the present application provide a face retrieval method and device based on local features, an apparatus, and a storage medium, to solve the problem of existing technology that mis-retrieves difficult example samples, and reduces face retrieval accuracy and retrieval effect.

[0004] In a first aspect, a face retrieval method based on local features is provided, including: extracting a query face feature corresponding to a query image and an original base library face feature corresponding to an original base library image, calculating a cosine similarity between the query face feature and the original base library face feature, and selecting a preset number of target base library images according to the cosine similarity; using a feature extraction network to fuse three-stage output features and four-stage output features corresponding to the query image and the target base library images, respectively, to obtain local feature maps corresponding to the query image and the target base library images, respectively; calculating a ground distance between feature blocks of the local feature map of the query image and feature blocks of the local feature map of the target base library images, and determining weights corresponding to the local feature map of the query image and the local feature map of the target base library images, respectively; calculating a distance between the local feature map of the query image and the local feature map of the target base library images according to the local feature map of the query image, the local feature map of the target base library images, the ground distance, and the weights, minimizing the distance, and taking the target base library image corresponding to the minimum distance as a retrieval result corresponding to the query image.

[0005] In a second aspect, the embodiment of the present application provides a face retrieval device based on local features, comprising: an extraction module configured to extract a query face feature corresponding to a query picture and an original database face feature corresponding to an original database picture, calculate a cosine similarity between the query face feature and the original database face feature, and select a preset number of target database pictures according to the cosine similarity; a fusion module configured to fuse three-stage output features and four-stage output features corresponding to the query picture and the target database picture respectively by using a feature extraction network to obtain local feature maps corresponding to the query picture and the target database picture respectively; a calculation module configured to calculate a ground distance between feature blocks of the local feature map of the query picture and feature blocks of the local feature map of the target database picture, and determine weights corresponding to the local feature map of the query picture and the local feature map of the target database picture respectively; and a retrieval module configured to calculate distances between the local feature map of the query picture and the local feature map of the target database picture according to the local feature map of the query picture, the local feature map of the target database picture, the ground distance and the weights, minimize the distances, and take the target database picture corresponding to the minimum distance as a retrieval result corresponding to the query picture.

[0006] In a third aspect, the embodiment of the present application provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above method when executing the program.

[0007] In a fourth aspect, the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the above method when executed by a processor.

[0008] The above at least one technical scheme adopted by the embodiment of the present application can achieve the following beneficial effects:

[0009] The query face feature corresponding to the query picture and the original base library face feature corresponding to the original base library picture are extracted, the cosine similarity of the query face feature and the original base library face feature is calculated, and a preset number of target base library pictures are selected according to the cosine similarity; the three-stage output feature and the four-stage output feature corresponding to the query picture and the target base library picture are fused respectively by using the feature extraction network, to obtain the local feature map corresponding to the query picture and the target base library picture respectively; the distance between the feature blocks of the local feature map of the query picture and the local feature map of the target base library picture is calculated, and the weights corresponding to the local feature map of the query picture and the local feature map of the target base library picture are determined; according to the local feature map of the query picture, the local feature map of the target base library picture, the distance and the weight, the distance between the local feature map of the query picture and the local feature map of the target base library picture is calculated, the distance is minimized, and the target base library picture corresponding to the minimum distance is taken as the retrieval result corresponding to the query picture. By introducing the feature blocks of the local feature map in the face retrieval task, the local fine-grained feature calculation of the face is introduced, and the face retrieval precision and retrieval effect under the difficult example sample are improved. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0011] Figure 1 is a flowchart of a face retrieval method based on local features provided by the embodiments of the present application;

[0012] Figure 2 is a structural schematic diagram of a face retrieval device based on local features provided by the embodiments of the present application;

[0013] Figure 3 is a structural schematic diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0014] In the following description, specific details such as specific system structures, techniques, etc. are presented in order to thoroughly understand the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits and methods are omitted to avoid unnecessary details that hinder the description of the present application.

[0015] As described in the background, with the development of face recognition technology, face retrieval algorithms have been widely applied, such as personnel clock-in projects, which calculate the similarity of extracted face features and face features in the base library, and then obtain the retrieval result. However, the commonly used face retrieval algorithm often causes false retrieval problems on difficult example samples (such as face images with occlusion, illumination and other problems).

[0016] In addition, the existing face retrieval algorithm only contains one-stage face feature retrieval, that is, by calculating the similarity of the query picture and the base library picture, and then selecting the base library picture with the highest similarity and meeting the similarity threshold. However, when one of the query picture and the base library picture has an occlusion or illumination problem, and the other is a normal face picture, the similarity of the two will be very low, thereby affecting the retrieval accuracy and retrieval effect.

[0017] Therefore, in order to solve the problems of false retrieval and low retrieval accuracy in difficult example sample scenarios, the embodiment of the present application provides a face retrieval method based on local features. The present application calculates the cosine similarity of the query face feature and the original base library face feature, selects a preset number of target base library pictures, respectively fuses the three-stage output features and the four-stage output features corresponding to the query picture and the target base library picture, obtains the local feature maps corresponding to the query picture and the target base library picture respectively, and calculates the distance between the feature blocks of the local feature map of the query picture and the local feature map of the target base library picture. Therefore, the feature blocks of the local feature map are introduced into the face retrieval algorithm, and the fine-grained feature similarity calculation is introduced when calculating the distance between the local feature map of the query picture and the local feature map of the target base library picture. Therefore, the face retrieval accuracy in difficult example sample scenarios is significantly improved, and the face retrieval effect is improved.

[0018] The content of the technical scheme of the present application will be described in detail below in combination with the drawings and specific embodiments.

[0019] Figure 1 is a flowchart of the face retrieval method based on local features provided by the embodiment of the present application. Figure 1 The face retrieval method based on local features of the present application can be executed by a server. As shown in Figure 1 , the face retrieval method based on local features can specifically include:

[0020] S101, extracting the query face feature corresponding to the query picture and the original base library face feature corresponding to the original base library picture, calculating the cosine similarity of the query face feature and the original base library face feature, and selecting a preset number of target base library pictures according to the cosine similarity;

[0021] S102, fuse the three-stage output features and the four-stage output features of the query picture and the target base library picture respectively by using the feature extraction network to obtain the local feature maps corresponding to the query picture and the target base library picture respectively.

[0022] S103, calculate the ground distance between the feature blocks of the local feature map of the query picture and the local feature map of the target base library picture, and determine the weights corresponding to the local feature map of the query picture and the local feature map of the target base library picture respectively.

[0023] S104, according to the local feature map of the query picture, the local feature map of the target base library picture, the ground distance and the weight, calculate the distance between the local feature map of the query picture and the local feature map of the target base library picture, minimize the distance, and take the target base library picture corresponding to the minimum distance as the search result corresponding to the query picture.

[0024] In some embodiments, extracting the query face features corresponding to the query picture and the original base library face features corresponding to the original base library picture comprises: obtaining the query picture, and extracting the query face features corresponding to the query picture and the original base library face features corresponding to the original base library picture by using a preset feature extraction network, wherein the feature extraction network adopts a ResNet50 network.

[0025] Specifically, after obtaining the query picture, the query face features corresponding to the query picture and the original base library face features corresponding to the original base library picture are extracted by using the feature extraction network; in actual application, the feature extraction network Backbone of the present application can adopt a ResNet50 network, that is, the query face features and the original base library face features are first extracted by using the ResNet50 network.

[0026] In the following embodiments of the present application, the query face features can also be referred to as query features, and the original base library face features can also be referred to as gallery features. After the query features and the gallery features are extracted, the cosine distance (i.e. cosine similarity) between the query features and the gallery features is calculated.

[0027] In some embodiments, selecting a preset number of target base library pictures according to the cosine similarity comprises: sorting the cosine similarities in descending order, and selecting the target base library pictures corresponding to the preset number of cosine similarities according to the order.

[0028] Specifically, after calculating the cosine similarity between the query face feature and each original base face feature, the cosine similarities are arranged in descending order, and the original base pictures corresponding to the top K cosine similarities are selected as the target base pictures, for example, the original base pictures corresponding to the top 20 cosine similarities are selected as the target base pictures.

[0029] In some embodiments, the three-stage output features and the four-stage output features corresponding to the query picture and the target base picture are fused by using the feature extraction network, including: inputting the query picture into the feature extraction network, fusing the three-stage output features and the four-stage output features of the feature extraction network corresponding to the query picture to obtain the fusion features corresponding to the query picture; inputting the target base picture into the feature extraction network, fusing the three-stage output features and the four-stage output features of the feature extraction network corresponding to the target base picture to obtain the fusion features corresponding to the target base picture.

[0030] Specifically, after the target base pictures are selected, the query picture and the target base picture are input into the feature extraction network Backbone, and the features output by Stage3 (i.e. the third stage) and Stage4 (i.e. the fourth stage) are fused in the feature extraction network Backbone to obtain the fusion features. Here, the feature extraction network Backbone can also use the ResNet50 network.

[0031] Further, after the query picture is input into the feature extraction network Backbone, the features output by Stage3 and Stage4 are fused in the feature extraction network Backbone to obtain the fusion features corresponding to the query picture; similarly, after the target base picture is input into the feature extraction network Backbone, the features output by Stage3 and Stage4 are fused in the feature extraction network Backbone to obtain the fusion features corresponding to the target base picture.

[0032] In some embodiments, after obtaining the fusion features corresponding to the query picture and the fusion features corresponding to the target base picture, the method further includes: performing global average pooling processing on the fusion features corresponding to the query picture and the fusion features corresponding to the target base picture respectively to obtain local feature maps corresponding to the query picture and the target base picture respectively; wherein each feature block in the local feature map represents a local fine-grained feature.

[0033] Specifically, after obtaining the fusion features corresponding to the query picture and the fusion features corresponding to the target gallery picture respectively, the fusion features are input into a pooling layer for global average pooling processing to obtain a local feature map with a length and width of 8x8, which is also called a patch feature map.

[0034] Further, through the above processing, the patch feature map of the query picture and the patch feature map of all target gallery pictures are obtained, each 1x1 grid in the patch feature map is taken as a feature block, and each feature block corresponds to a respective local fine-grained feature, that is, each feature block in the local feature map patch corresponds to a local fine-grained feature. Therefore, through the above processing, fine-grained face sample features of the query picture and the target gallery picture can be obtained. Compared with the original global similarity calculation method, the similarity calculation method of the local feature adopted by the present application can reduce the influence of occlusion, illumination and the like on the face retrieval task.

[0035] In some embodiments, determining the weights corresponding to the local feature map of the query picture and the local feature map of the target gallery picture respectively includes: performing face detection on the query picture and the target gallery picture to obtain face key points, and assigning weights to the local feature map of the query picture and the local feature map of the target gallery picture according to the density of the face key points.

[0036] Specifically, after obtaining the patch feature map corresponding to the query picture and the patch feature map corresponding to the target gallery picture respectively, the present application calculates the Ground Distance between the feature block (referred to as query feature block) of each query picture patch feature map and the feature block (referred to as gallery feature block) of each target gallery picture patch feature map, that is, calculates the ground distance between the query feature block and the gallery feature block. In actual application, the Ground Distance distance calculation method can be used to calculate the ground distance, wherein Ground Distance is a distance measurement method.

[0037] Further, the present application also detects the query picture and the target gallery picture respectively by using a face detection algorithm to obtain 68 face key points, and assigns patch weights to the patch feature maps corresponding to the query picture and the target gallery picture according to the density of the face key points, that is, the greater the density of the face key points, the greater the weight of the patch feature map.

[0038] In some embodiments, the distance between the local feature map of the query picture and the local feature map of the target base library picture is calculated according to the local feature map of the query picture, the local feature map of the target base library picture, the ground distance, and the weight, including: taking the local feature map of the query picture, the local feature map of the target base library picture, the ground distance, and the weight as the input of a preset EMD distance algorithm, and outputting the EMD distance between the local feature map of the query picture and the local feature map of each target base library picture by using the EMD distance algorithm.

[0039] Specifically, the EMD distance between the query feature map (i.e., the patch feature map of the query picture) and the gallery feature map (i.e., the patch feature map of the target base library picture) is calculated based on the patch feature map of the query picture, the patch feature map of the target base library picture, the ground distance, and the patch weight, where the EMD distance algorithm is a commonly used image similarity measurement method, and the EMD distance algorithm outputs the EMD distance between each query feature map and each gallery feature map.

[0040] Further, after obtaining the EMD distance between each query feature map and each gallery feature map, the distance between the local feature map corresponding to the query picture and the local feature map corresponding to the target base library picture in the pairing can be found by minimizing the distance, i.e., solving the optimization problem, and the target base library picture corresponding to the minimized distance is taken as the face retrieval result of the query picture.

[0041] According to the technical scheme provided in the embodiments of the present application, in order to solve the false retrieval problem caused by occlusion, illumination, etc. in the face retrieval task, a Two-Stage-based fine-grained face retrieval scheme is proposed. In the scheme, the face features of the query picture and the face features of all target base library pictures are divided into 8x8 local feature maps patch, each feature block in the local feature map patch represents a local fine-grained feature, fine-grained retrieval is performed based on the local fine-grained features, and the face retrieval result is obtained, thereby avoiding the influence of occlusion, illumination, etc. on the original full-image similarity calculation. In the scheme, the EMD distance between the patch feature map of the query picture and the patch feature map of all target base library pictures is minimized to calculate the retrieval result, thereby significantly improving the face retrieval accuracy in the difficult example sample scenario and improving the face retrieval effect.

[0042] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the apparatus embodiments of the present application, please refer to the method embodiments of the present application.

[0043] Figure 2Figure 1 is a structural schematic diagram of a face retrieval device based on local features provided by an embodiment of the present application.

[0044] As shown in Figure 2 the face retrieval device based on local features comprises:

[0045] The extraction module 201 is configured to extract query face features corresponding to a query picture and original database face features corresponding to original database pictures, calculate cosine similarity of the query face features and the original database face features, and select a preset number of target database pictures according to the cosine similarity.

[0046] The fusion module 202 is configured to fuse three-stage output features and four-stage output features corresponding to the query picture and the target database picture respectively by using a feature extraction network, to obtain local feature maps corresponding to the query picture and the target database picture respectively.

[0047] The calculation module 203 is configured to calculate the distance between feature blocks of the local feature map of the query picture and feature blocks of the local feature map of the target database picture, and determine weights corresponding to the local feature map of the query picture and the local feature map of the target database picture respectively.

[0048] The retrieval module 204 is configured to calculate the distance between the local feature map of the query picture and the local feature map of the target database picture according to the local feature map of the query picture, the local feature map of the target database picture, the distance, and the weight, minimize the distance, and take the target database picture corresponding to the minimum distance as the retrieval result of the query picture.

[0049] In some embodiments, Figure 2 The extraction module 201 of the face retrieval device based on local features obtains a query picture, extracts query face features corresponding to the query picture and original database face features corresponding to original database pictures by using a preset feature extraction network, and the feature extraction network adopts a ResNet50 network.

[0050] In some embodiments, Figure 2 The extraction module 201 of the face retrieval device based on local features sorts the cosine similarity in descending order, and selects a preset number of target database pictures corresponding to the cosine similarity according to the order.

[0051] In some embodiments, Figure 2 The fusion module 202 of the face retrieval device based on local features inputs the query picture into the feature extraction network, fuses three-stage output features and four-stage output features of the feature extraction network corresponding to the query picture to obtain fusion features corresponding to the query picture, and inputs the target database picture into the feature extraction network, fuses three-stage output features and four-stage output features of the feature extraction network corresponding to the target database picture to obtain fusion features corresponding to the target database picture.

[0052] In some embodiments, Figure 2 The fusion module 202 fuses the local feature maps of the query image and the target base image to obtain a fused feature of the query image and a fused feature of the target base image.

[0053] In some embodiments, Figure 2 The calculation module 203 detects the face of the query image and the target base image to obtain face key points, and assigns weights to the local feature maps of the query image and the target base image according to the density of the face key points.

[0054] In some embodiments, Figure 2 The retrieval module 204 inputs the local feature maps of the query image and the target base image, the distance, and the weights into a preset EMD distance algorithm, and outputs the EMD distance between the local feature map of the query image and the local feature map of each target base image by using the EMD distance algorithm.

[0055] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0056] Figure 3 is a structural schematic diagram of an electronic device 3 provided by the embodiments of the present application. As shown in Figure 3 The electronic device 3 of the embodiment includes a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. The processor 301 implements the steps in each of the above method embodiments when executing the computer program 303. Alternatively, the processor 301 implements the functions of each module / unit in each of the above device embodiments when executing the computer program 303.

[0057] Exemplarily, the computer program 303 can be divided into one or more modules / units, which are stored in the memory 302 and executed by the processor 301 to complete the present application. One or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 303 in the electronic device 3.

[0058] The electronic device 3 can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The electronic device 3 can include, but is not limited to, a processor 301 and a memory 302. Those skilled in the art can understand that Figure 3 The electronic device 3 is merely an example and does not constitute a limitation on the electronic device 3, and can include more or fewer components than shown, or combine certain components, or different components, for example, the electronic device can also include an input / output device, a network access device, a bus, and the like.

[0059] The processor 301 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0060] The memory 302 can be an internal storage unit of the electronic device 3, for example, a hard disk or a memory of the electronic device 3. The memory 302 can also be an external storage device of the electronic device 3, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like. Further, the memory 302 can include both the internal storage unit and the external storage device of the electronic device 3. The memory 302 is used to store computer programs and other programs and data required by the electronic device. The memory 302 can also be used to temporarily store data that has been output or will be output.

[0061] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is taken as an example, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0062] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0063] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the application.

[0064] In the embodiments provided in the present application, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed each other can be indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0065] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0066] In addition, each of the function units in each of the embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0067] If the integrated module / unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can be executed by a processor to implement the steps of each method embodiment. The computer program can include computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0068] The above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A face retrieval method based on local features, characterized in that, include: Extract the query face features corresponding to the query image and the original database face features corresponding to the original database images, calculate the cosine similarity between the query face features and the original database face features, and select a preset number of target database images based on the cosine similarity. The three-stage output features and four-stage output features corresponding to the query image and the target database image are fused using a feature extraction network to obtain local feature maps corresponding to the query image and the target database image respectively. Calculate the distance between the feature blocks of the local feature map of the query image and the feature blocks of the local feature map of the target database image, and determine the weights corresponding to the local feature map of the query image and the local feature map of the target database image, respectively. Based on the local feature map of the query image, the local feature map of the target database image, the distance, and the weight, the distance between the local feature map of the query image and the local feature map of the target database image is calculated. The distance is minimized, and the target database image corresponding to the minimum distance is taken as the retrieval result corresponding to the query image. The step of determining the weights corresponding to the local feature maps of the query image and the target database images includes: Face detection is performed on the query image and the target database image to obtain facial key points. Weights are assigned to the local feature maps of the query image and the target database image based on the density of the facial key points, wherein the greater the density of the facial key points, the greater the weight of the local feature map.

2. The method according to claim 1, characterized in that, The extraction of the query face features corresponding to the query image and the original database face features corresponding to the original database images includes: The query image is obtained, and the query face features corresponding to the query image and the original database face features corresponding to the original database image are extracted using a preset feature extraction network. The feature extraction network adopts the ResNet50 network.

3. The method according to claim 1, characterized in that, The step of selecting a preset number of target base images based on the cosine similarity includes: The cosine similarity scores are sorted in descending order, and a preset number of target base images corresponding to the cosine similarity scores are selected according to the order.

4. The method according to claim 1, characterized in that, The process of fusing the three-stage output features and four-stage output features corresponding to the query image and the target database image using a feature extraction network includes: The query image is input into the feature extraction network, and the three-stage output features and the four-stage output features of the feature extraction network corresponding to the query image are fused to obtain the fused features corresponding to the query image. The target database image is input into the feature extraction network, and the three-stage output features and four-stage output features of the feature extraction network corresponding to the target database image are fused to obtain the fused features corresponding to the target database image.

5. The method according to claim 4, characterized in that, After obtaining the fusion features corresponding to the query image and the fusion features corresponding to the target database image, the method further includes: Global average pooling is performed on the fusion features corresponding to the query image and the fusion features corresponding to the target database image to obtain local feature maps corresponding to the query image and the target database image respectively; wherein, each feature block in the local feature map represents a local fine-grained feature.

6. The method according to claim 1, characterized in that, The step of calculating the distance between the local feature map of the query image and the local feature map of the target database image based on the local feature map of the query image, the local feature map of the target database image, the distance, and the weight includes: The local feature map of the query image, the local feature map of the target database image, the distance, and the weight are used as inputs to a preset EMD distance algorithm. The EMD distance algorithm is then used to output the EMD distance between the local feature map of the query image and the local feature map of each target database image.

7. A face retrieval device based on local features, characterized in that, include: The extraction module is configured to extract the query face features corresponding to the query image and the original database face features corresponding to the original database images, calculate the cosine similarity between the query face features and the original database face features, and select a preset number of target database images based on the cosine similarity. The fusion module is configured to use a feature extraction network to fuse the three-stage output features and the four-stage output features corresponding to the query image and the target database image respectively, to obtain local feature maps corresponding to the query image and the target database image respectively; The calculation module is configured to calculate the distance between the feature blocks of the local feature map of the query image and the feature blocks of the local feature map of the target database image, and to determine the weights corresponding to the local feature map of the query image and the local feature map of the target database image, respectively. The retrieval module is configured to calculate the distance between the local feature map of the query image and the local feature map of the target database image based on the local feature map of the query image, the local feature map of the target database image, the distance, and the weight; minimize the distance; and take the target database image with the smallest distance as the retrieval result corresponding to the query image. The calculation module is used to perform face detection on the query image and the target database image to obtain facial key points, and to assign weights to the local feature maps of the query image and the target database image according to the density of the facial key points, wherein the greater the density of the facial key points, the greater the weight of the local feature map.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images