Adaptive face search method, system, medium, device and terminal

Abstract

The present invention belongs to the field of image recognition technology, and discloses an adaptive face search method, system, medium, device and terminal, which initializes the face search engine, uploads the face picture to be searched or added; uses the face feature extraction module to extract features from the face picture; the face search engine receives the scheduling of the face search module, searches and adds face feature vectors and returns the results to the face search module. The face search method and system of the present invention, which are based on decision trees, face feature recognition and vector search engines, can achieve accurate identification of face feature points. The present invention calculates the feature differences between faces through an algorithm, which can provide assistance for object search and personnel identity authentication. The decision tree of the present invention can determine the optimal index construction method under different data levels, and realize the adaptive search strategy under the condition of continuous growth of data volume; combined with fixed and mobile monitoring systems, it can achieve prevention and control of key personnel in the monitoring area.

Description

Technical Field

[0001] The present invention belongs to the field of image recognition technology, and in particular relates to an adaptive face search method, system, medium, device and terminal. Background Art

[0002] With the widespread adoption of video capture devices and the advancement of deep learning technology, computer vision technology is experiencing a surge in development. Facial recognition technology is particularly widely used in everyday applications, such as mobile payments, access control systems, and video surveillance. Face recognition and search are essentially a fuzzy search process. Currently, common face recognition methods convert facial features into a distribution of feature points, which are then stored as feature vectors or feature codes. This transforms the face search problem into a feature vector distance calculation or code matching problem. In summary, the key to transforming a fuzzy search problem into a numerical classification or regression problem lies in the facial feature extractor and facial feature comparison.

[0003] The facial feature extractor maps facial feature vectors extracted from different facial photos of the same person to similar regions in a high-dimensional space. The distance between these feature vectors serves as the basis for determining whether they represent the same person. Therefore, the accuracy of the facial feature extractor directly impacts the accuracy of face search. Existing facial feature encoding systems are mostly based on 5-, 32-, or 68-point features, resulting in fewer feature points and a coarse granularity in facial feature extraction.

[0004] The increasing prevalence of facial acquisition equipment is generating an ever-increasing amount of facial data. If the traditional one-by-one comparison method is still used for face matching, the search time will increase linearly with the growth of data volume. The current common method is to train a search model before searching, and to build an index for all faces in the database to pre-classify or encode the faces. Faces with similar features belong to the same cluster. During the search, the target face is compared with the most representative face in the database in each cluster to determine whether there is a face similar to the target face in the cluster, thereby accelerating the search process. However, a single search model is often limited by its own index construction method and can only achieve optimal performance within a specific range of data volume, and cannot cope with the continuous growth of data.

[0005] Through the above analysis, the problems and defects of the existing technology are as follows:

[0006] (1) The facial feature extraction granularity of existing face recognition search systems is relatively coarse.

[0007] (2) Existing single search models are often limited by their own index construction methods and can only achieve optimal performance within a specific range of data volume, and cannot cope with the situation of continuous data growth. Summary of the Invention

[0008] In response to the problems of existing technologies such as decreased search accuracy and linear growth of search time as the amount of data increases in scenarios with continuous data growth, the present invention provides an adaptive face search method, system, medium, device and terminal, and in particular relates to an adaptive face search method, system, medium, device and terminal based on a decision tree.

[0009] The present invention is implemented as follows: an adaptive face search method, which includes deep learning and a search engine, uses a face feature extractor to extract features from the face, then uses a decision tree algorithm to select the optimal search engine index construction method based on the amount of data in the library and the amount of available system resources, and finally uses a face search engine to accelerate the search process.

[0010] Furthermore, the adaptive face search method comprises the following steps:

[0011] Step 1: Initialize the face search engine and upload the face image to be searched or added;

[0012] Step 2: Use the face feature extraction module to extract features from the face image;

[0013] Step 3: The face search engine receives the scheduling of the face search module, searches and adds the face feature vectors and returns the results to the face search module.

[0014] Furthermore, the face search engine initialization in step one includes: the face search module inputs the information of the number of face feature vectors, system resource usage and number of CPU cores in the face database into a pre-trained decision tree, and the decision tree outputs the index construction method with the best performance under the current circumstances; the face search module uses the index construction method given by the decision tree to train the data to obtain a search model of the search engine, which is called the initial model. After the face search engine loads the initial model, it waits for a call; wherein, after the face feature extraction module extracts features from the initial data, the face search engine pulls the initial data in the database to initialize the search engine model. The uploading of face pictures to be searched or added includes: the user uploads the face picture to the search interface and the upload interface, and then the interface passes the face picture to the pre-trained face feature extraction module.

[0015] The data size, device memory size, whether a GPU is used, and the number of CPU cores used to build the index are OneHot encoded and then input into a pre-built decision tree. The decision tree uses different index construction methods, including HNSWFFlat, IVFFlat, and IVFPQ. Before inputting the number of facial feature vectors in the face database, system resource usage, and the number of CPU cores into the pre-trained decision tree, the following steps are also included: based on the number of facial feature vectors in the database, whether a GPU is used, the number of CPU cores, and the system memory size data, an index construction method for determining the best performance under the current conditions is constructed and trained. The decision tree uses the GINI coefficient as a feature measurement standard:

[0016]

[0017] The HNSWFlat index is constructed as follows: all feature vectors in the database are mapped to points in a high-dimensional space with dimension D, where the feature vectors are the coordinates of the points; all coordinate points in the database are inserted and connected one by one, and a stepped NSW network is constructed during the insertion process. After the map is constructed, the structure and parameters of the map are saved as the HNSWF index model.

[0018] The IVFFlat index is constructed as follows: all feature vectors in the database are mapped to points in a high-dimensional space with dimension D, with the feature vectors being the coordinates of the points; all feature vectors are clustered using the KMeans clustering algorithm to obtain K clusters and cluster centers; the L2 distance is used to calculate the distance between vectors; an inverted list is established for each cluster center, and all vectors in the cluster are added to the inverted list, and the clustering parameters and the inverted list are saved as the IVFFlat index model.

[0019] The IVFPQ index construction method is as follows: based on the IVFFlat index construction method, N vectors are clustered to obtain K clusters and cluster centers, K inverted lists are established, and the residual vectors of each vector and the cluster center are stored in the inverted lists; all residual vectors in each inverted list are product-quantized to obtain a codebook; the codebook and all clustering parameters are saved as the IVFPQ index model.

[0020] The product quantization process includes: dividing N residual vectors with a dimension of D, each vector is divided into M sub-vectors, and each sub-vector has a dimension of D / M; performing Kmeans clustering on the obtained M groups, each group containing N vectors with a dimension of D / M; each group contains K2 clusters, with a total of MK2 clusters and cluster centers, and the cluster centers are encoded, and the obtained MK2 IDs are used as the codebook.

[0021] After building the index model, the search engine model is loaded into the child process and the search engine is called via a remote procedure call. When the main program calls the search function of the face search engine, the search engine automatically accelerates the search process based on the index building method and returns the matching feature vector ID.

[0022] When the main program passes newly stored facial feature vectors, the search engine builds an index for the newly added vectors and keeps the current number of clusters and search engine model parameters unchanged; users set the search engine initialization interval through task scheduling in the main program to obtain optimal performance under incremental data.

[0023] Furthermore, the step 2 of extracting features from the face image using the face feature extraction module includes: the face feature extraction module uses a deep neural network algorithm to extract features from the acquired face data to obtain a 128-dimensional face feature vector, and passes the face feature vector to the face search module for face search or uploading new data; wherein, the face feature extraction module uses 17 feature points to mark the chin, 5 feature points to mark the left and right eyebrows respectively, 6 feature points to mark the left and right eyes respectively, 4 and 5 feature points to mark the bridge of the nose and the tip of the nose respectively, and 12 feature points to mark the upper and lower lips respectively, for a total of 72 feature points; after the feature point sequence is calculated by the neural network, a one-dimensional vector with a length of 128 is obtained as the identification information of the face.

[0024] Among them, before the pre-trained facial feature extraction module extracts the D-dimensional facial feature vector, it also includes: using a manually labeled facial feature point dataset to train the ResNet50 deep learning model, stopping the training when the feature extraction module recognition reaches the expected value, and fixing the model parameters as the facial feature extraction model.

[0025] Furthermore, the face search module in step three searches and adds face feature vectors, including: when the face search module provides a search function, a remote procedure calls a face search engine to perform a search operation; the face search engine uses different search methods according to the current index construction method to return all search results that meet the similarity threshold to the face search module, and the face search module then merges the results returned by the search engine with the additional information in the database as search results and returns them to the user; when the face search module provides an upload function, a remote procedure calls a face search engine to add a new face feature vector; the face search engine uses different insertion methods according to the current index construction method to build an index and create an ID for the feature vector.

[0026] Another object of the present invention is to provide an adaptive face search system that applies the adaptive face search method, wherein the adaptive face search system includes: an interactive interface, a face feature extractor, a message queue, a face search module, a database, and a face search engine.

[0027] The search interface is used to complete the interaction with the user, including receiving requests and pictures from the user and returning information to the user in an appropriate manner;

[0028] The facial feature extractor is used to identify feature vectors containing facial feature information from facial images;

[0029] The message queue is used to cache pictures that need to be added and uploaded by users through the interface;

[0030] The database is used to store feature vectors and unstructured information related to the image;

[0031] The face search module is the core control and scheduling module of the entire system and the processing center of all data. It is responsible for receiving the scheduling of the face search module, searching and adding face feature vectors and returning the results to the face search engine.

[0032] Furthermore, the adaptive face search system further includes:

[0033] The face picture uploading interface and the search receive face pictures and pass them to the face feature extraction module;

[0034] The facial feature extraction module is used to pre-process the acquired facial image data, identify feature vectors including eyebrows, nose, eyes, mouth and facial contours using a deep convolutional neural network, and then transfer the feature vectors and image-related information to the storage module and the face search module for storage;

[0035] The search module is used to input the number of faces in the database and related device information into a decision tree; the decision tree adaptively selects the index construction method with the best performance, trains and generates a face search engine model, and the face search engine model is maintained in memory to provide search and incremental update storage functions;

[0036] The search interface is used to extract facial features from the uploaded image after receiving it, and input the extracted facial feature vector into the face search engine;

[0037] The face search engine is used to calculate the L2 distance between the current face feature vector and the stored feature vector as the similarity between faces. For all pictures that meet the search similarity provided by the user, the system retrieves the corresponding face picture information from the storage module and returns it to the user for display.

[0038] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the adaptive face search method.

[0039] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the adaptive face search method.

[0040] Another object of the present invention is to provide an information data processing terminal, which is used to implement the adaptive face search system.

[0041] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0042] First, in view of the technical problems existing in the above-mentioned prior art and the difficulty of solving these problems, we closely combine the technical solutions to be protected by the present invention and the results and data during the research and development process, and conduct a detailed and in-depth analysis of how the technical solutions of the present invention solve the technical problems and some creative technical effects brought about by solving the problems. The specific description is as follows:

[0043] The present invention is a face search method and system based on a decision tree, facial feature recognition and a vector search engine, which can realize accurate recognition of facial feature points. The present invention calculates the feature differences between faces through an algorithm, which can provide assistance for object search and personnel identity authentication. The decision tree provided by the present invention can determine the optimal index construction method under different data levels, and realizes the adaptive search strategy under the condition of continuous growth of data volume. The face search engine gives the present invention the ability to quickly search for relevant personnel in massive data. The present invention is combined with fixed and mobile monitoring systems to realize the prevention and control of key personnel in the monitoring area; at the same time, the present invention can provide image search function as an image search search engine.

[0044] Second, considering the technical solution as a whole or from the perspective of the product, the technical effects and advantages of the technical solution to be protected by the present invention are described in detail as follows:

[0045] The present invention provides an adaptive face search method and system based on a decision tree, which realizes high-precision and rapid search of faces in a scene, and particularly involves deep learning and search engines. Feature extraction of face images, selection of index construction methods and face search are realized through feature extraction models, decision trees and search engines.

[0046] This paper selects 72 representative facial feature points as facial features and trains a corresponding ResNet50 model as a facial feature extractor. The method uses different index construction methods based on the size of the initialization data and device memory capacity to adapt to different application scenarios.

[0047] Third, as auxiliary evidence for the inventiveness of the claims of the present invention, it is also reflected in the following important aspects:

[0048] (1) The technical solution of the present invention fills the technical gap in the industry at home and abroad:

[0049] The technical solution of the present invention fills the technical gap of adaptive face search based on resource constraints at home and abroad.

[0050] (2) The technical solution of the present invention solves the technical problems that people have been eager to solve but have never been able to solve successfully:

[0051] The technical solution of the present invention solves the technical problem that the search efficiency and accuracy of a single index construction method gradually decrease as the data volume increases in a scenario where data continues to grow. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0053] Figure 1 is a flow chart of an adaptive face search method provided by an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of the principle of the adaptive face search method provided by an embodiment of the present invention;

[0055] Figure 3 is a structural diagram of an adaptive face search system provided by an embodiment of the present invention;

[0056] Figure 4 Schematic diagram of selected facial feature points provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0058] In response to the problems existing in the prior art, the present invention provides an adaptive face search method, system, medium, device and terminal. The present invention is described in detail below with reference to the accompanying drawings.

[0059] 1. Explanatory Examples In order to enable those skilled in the art to fully understand how to implement the present invention, this section provides an illustrative example that expands upon the technical solutions of the claims.

[0060] like Figure 1 As shown, the adaptive face search method provided by the embodiment of the present invention includes the following steps:

[0061] S101, initialize the face search engine and upload the face image to be searched or added;

[0062] S102, extracting features from the face image using a face feature extraction module;

[0063] S103, the face search engine receives the scheduling of the face search module, searches and adds face feature vectors and returns the results to the face search module.

[0064] As a preferred embodiment, Figure 2 As shown, the adaptive face search method provided by the embodiment of the present invention specifically includes the following steps:

[0065] Step 1, first initialize the face search engine, specifically including: the face search module inputs the information of the number of face feature vectors, system resource usage and number of CPU cores in the face database into a pre-trained decision tree, and the decision tree outputs the index construction method with the best performance under the current circumstances; the face search module uses the index construction method given by the decision tree to train the data to obtain a search engine search model, which is called the initial model. After the face search engine loads the initial model, it waits for a call; wherein, after the face feature extraction module extracts features from the initial data, the face search engine pulls the initial data in the database to initialize the search engine model. The uploading of face pictures to be searched or added includes: the user uploads the face picture to the search interface and the upload interface, and then the interface passes the face picture to the pre-trained face feature extraction module.

[0066] The face search module inputs information such as the number of facial feature vectors in the face database, system resource usage, and the number of CPU cores into a pre-trained decision tree. The decision tree then outputs the optimal index construction method for the current situation. The face search module uses this index construction method to train this data to generate a search engine index model. The face search engine loads the initialized model and waits for calls.

[0067] Step 2: The user uploads the face image to be searched or added, which includes:

[0068] Users upload face images to the search interface and upload interface, and then the interface passes the face images to the pre-trained face feature extraction module.

[0069] Step 3: The facial feature extraction module extracts features from the facial image, specifically including:

[0070] The facial feature extraction module uses a deep neural network algorithm to extract features from the acquired facial data, obtains a 128-dimensional facial feature vector, and passes the facial feature vector to the face search module for face search or uploading new data.

[0071] Step 4: The face search module provides search functions, including:

[0072] When the face search module provides search functionality, a remote procedure call (RPC) is made to the face search engine to perform the search. Based on the current index construction method, the face search engine uses different search methods to return all search results that meet the similarity threshold to the face search module. The face search module then combines the search engine results with additional information in the database and returns them to the user as search results.

[0073] Step 5: The face search module provides additional functions, including:

[0074] When the face search module provides the upload function, the remote procedure call (RPC) adds a new face feature vector to the face search engine. The face search engine uses different insertion methods to build an index and create an ID for the feature vector based on the current index construction method.

[0075] The facial feature extraction module provided by the embodiment of the present invention uses 17 feature points to mark the chin, 5 feature points to mark the left and right eyebrows respectively, 6 feature points to mark the left and right eyes respectively, 4 and 5 feature points to mark the bridge of the nose and the tip of the nose respectively, and 12 feature points to mark the upper and lower lips respectively, for a total of 72 feature points; after the above feature point sequence is calculated by a neural network, a one-dimensional vector of length 128 is obtained as the identification information of the face. The schematic diagram of the selected facial feature points provided by the embodiment of the present invention is as follows Figure 4 shown.

[0076] The face search engine provided by the embodiment of the present invention pulls the initial data from the database to initialize the search engine model after the face feature extraction module extracts features from the initial data.

[0077] In this embodiment of the present invention, the data size, device memory size, whether a GPU is used, the number of CPU cores, and other conditions used to construct the index are OneHot encoded and then input into a pre-built decision tree. The decision tree will use different index construction methods, including HNSWFFlat, IVFFlat, and IVFPQ.

[0078] After constructing the index model, this embodiment of the present invention loads the search engine model into a child process, implementing a remote procedure call to the search engine. When the main program invokes the face search engine's search function, the search engine automatically accelerates the search process based on the index construction method and returns the matching feature vector ID.

[0079] When the main program of this embodiment of the present invention passes newly stored facial feature vectors, the search engine constructs an index for the newly added vectors while maintaining the current number of clusters and search engine model parameters. Users can set the search engine initialization interval through task scheduling in the main program to achieve optimal performance with incremental data.

[0080] As a preferred embodiment, before the pre-trained facial feature extraction module extracts the multi-dimensional facial feature vector, it also includes: using a manually labeled facial feature point dataset to train the ResNet50 deep learning model, stopping the training when the recognition accuracy of the feature extraction module reaches the expected value, and fixing the model parameters as the facial feature extraction model.

[0081] As a preferred embodiment, before inputting information such as the number of facial feature vectors in the face database, system resource usage, and the number of CPU cores into the pre-trained decision tree, the method further includes: constructing and training an index construction method for determining the optimal performance under current conditions based on data such as the number of facial feature vectors in the database, whether a GPU is used, the number of CPU cores, and the size of system memory; wherein the decision tree uses the GINI coefficient as a feature measurement standard:

[0082]

[0083] As a preferred embodiment, the index construction methods include: HNSWFFlat, IVFFlat and IVFPQ three index construction methods.

[0084] As a preferred embodiment, the HNSWFlat index is constructed by mapping all feature vectors in the database to points in a high-dimensional space of dimension D, where the feature vectors are the coordinates of the points. All coordinate points in the database are inserted and connected one by one, constructing a stepped NSW network during the insertion process. After the map is constructed, the structure and parameters of the map are saved as the HNSWF index model.

[0085] As a preferred embodiment, the IVFFlat index is constructed by mapping all feature vectors in the database to points in a high-dimensional space of dimension D, with the feature vectors representing the coordinates of the points. All feature vectors are clustered using the KMeans clustering algorithm to obtain K clusters and cluster centers. The L2 distance is used to calculate the distance between vectors. An inverted list is then created for each cluster center, and all vectors in that cluster are added to the inverted list. The clustering parameters and inverted list are saved as the IVFFlat index model.

[0086] As a preferred embodiment, the IVFPQ index is constructed as follows: First, based on the IVFFlat index construction method, N vectors are clustered to obtain K clusters and cluster centers. K inverted lists are then created, and the residual vector between each vector and its cluster center is stored in the inverted lists. All residual vectors in each inverted list are then multiplied and quantized to obtain a codebook. The codebook and all clustering parameters are saved as the IVFPQ index model.

[0087] As a preferred embodiment, the product quantization process includes: first, dividing N residual vectors of dimension D into M subvectors, each with dimension D / M. Kmeans clustering is performed on the resulting M groups, each containing N vectors of dimension D / M. Each group contains K2 clusters, for a total of MK2 clusters and cluster centers. These cluster centers are encoded, and the resulting MK2 IDs are the codebook.

[0088] After the above five steps, a real-time face search system based on facial feature point recognition can be realized.

[0089] like Figure 3 As shown, the overall structure of the adaptive face search system provided by the embodiment of the present invention includes six parts, namely, an interactive interface, a face feature extractor, a message queue, a face search module, a database, and a face search engine. The main function of the search interface is to complete the interaction with the user, including receiving requests and pictures issued by the user, and returning information to the user in an appropriate manner. The main function of the face feature extractor is to identify feature vectors containing facial feature information from face pictures. The message queue is used to cache pictures that need to be added and uploaded by users through the interface. The database is used to store feature vectors and unstructured information related to the pictures. The face search module is the core control and scheduling module of the entire system and is the processing center of all data. The face search engine is responsible for receiving the scheduling of the face search module, searching and adding face feature vectors and returning the results to the face search engine.

[0090] The embodiment of the present invention illustrates the user's entire system operation process. First, the user needs to import some facial images as the original face library within the database. The user then uses the system's initialization tool to train an original search engine model before restarting the entire system. The system automatically calls the original search engine and performs scheduled training based on the user's settings. Unlike fixed parameters during upload, scheduled training pulls all data from the database to train a new search engine model, thereby optimizing the problem of decreased search accuracy as data grows. Once the system is started, the user can perform real-time searches and upload faces.

[0091] like Figure 2 As shown, the embodiment of the present invention can show the overall process of users adding new pictures. The user first sends an HTTP request to upload the picture to the upload interface. After the system extracts the features of the newly uploaded picture, it will not directly insert it into the search engine. Instead, it will first put the feature vector into the cache message queue, and then insert the data in the message queue into the search engine in batches when the message queue is full or the search module actively polls. An index is built for the newly added vector in the search engine, and the current number of clusters and the center of each cluster are kept unchanged. Since the search engine training process is blocking, this system avoids frequent blocking and waiting of the system by batch insertion and keeping the parameters unchanged, thereby ensuring the real-time performance of the system.

[0092] like Figure 2 As shown in the figure, the overall process of a user searching for a face in an embodiment of the present invention can be seen. The user first sends an HTTP request to upload an image and similarity requirements to the search interface. After extracting features from the image to be searched, the system compares the feature vector with the cluster center within the search engine according to the search engine index construction method. If the cluster center meets the search similarity requirements, the target feature vector is compared with each node within the cluster, and the feature vector and ID that meet the conditions are returned. The face search module then retrieves the image and related information corresponding to each ID from the database as the search result returned to the user.

[0093] 2. Application Examples: In order to demonstrate the creativity and technical value of the technical solution of the present invention, this section provides application examples of the claimed technical solution on specific products or related technologies.

[0094] In one embodiment of a police system, the technical solution of the present invention serves as a backend facial recognition service, providing the system with face search capabilities. By setting up scheduled database synchronization and training, search performance is maintained even in scenarios with increasing data. Furthermore, by configuring a distributed ID generator and message queue, the technical solution of the present invention offers distributed scalability. The use of coroutine technology in the search upload interface further increases system throughput, ultimately achieving over 2000 TPS with 16 processes on a single machine.

[0095] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art will appreciate that the above-mentioned devices and methods can be implemented using computer-executable instructions and / or contained in processor control code, for example, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, or can be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0096] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. An adaptive face search method, characterized in that: The adaptive face search method includes deep learning and a search engine. A face feature extractor is used to extract features from faces. A decision tree algorithm is then used to select the optimal search engine index construction method based on the amount of data in the database and the amount of available system resources. Finally, a face search engine is used to accelerate the search process. The self-adaptive face search method comprises the following steps: Step 1: Initialize the face search engine and upload the face image to be searched or added; Step 2: Use the face feature extraction module to extract features from the face image; Step 3: The face search engine receives the dispatch of the face search module, searches and adds the face feature vectors and returns the results to the face search module; The face search engine initialization in step 1 includes: the face search module inputs the information of the number of face feature vectors, system resource usage and number of CPU cores in the face database into a pre-trained decision tree, and the decision tree outputs the index construction method with the best performance under the current circumstances; the face search module uses the index construction method given by the decision tree to train the data to obtain a search model of the search engine, which is called the initial model; the face search engine loads the initial model and waits for calling; wherein, after the face feature extraction module extracts the features of the initial data, the face search engine pulls the initial data in the database to initialize the search engine model; the uploading of the face picture to be searched or added includes: the user uploads the face picture to the search interface and the upload interface, and then the interface passes the face picture to the pre-trained face feature extraction module The data size, device memory size, GPU usage, and number of CPU cores used to build the index are OneHot encoded and then input into a pre-built decision tree. The decision tree uses different index construction methods, including HNSWFlat, IVFFlat, and IVFPQ. Before inputting the number of facial feature vectors in the face database, system resource usage, and number of CPU cores into the pre-trained decision tree, the following steps are also included: Based on the number of facial feature vectors in the database, GPU usage, number of CPU cores, and system memory size data, an index construction method with optimal performance under the current conditions is constructed and trained. The decision tree uses the GINI coefficient as a feature measurement criterion: The HNSWFlat index is constructed by mapping all feature vectors in the database to points in a high-dimensional space with a dimension of D, where the feature vectors are the coordinates of the points; inserting and connecting all coordinate points in the database one by one, and constructing a stepped NSW network during the insertion process. After the map is constructed, the structure and parameters of the map are saved as the HNSWF index model; The IVFFlat index is constructed by mapping all feature vectors in the database to points in a high-dimensional space of dimension D, with the feature vectors being the coordinates of the points; clustering all feature vectors using the KMeans clustering algorithm to obtain K clusters and cluster centers; wherein the distance between vectors is calculated using the L2 distance; creating an inverted list for each cluster center, adding all vectors in the cluster to the inverted list, and saving the clustering parameters and the inverted list as the IVFFlat index model; The IVFPQ index construction method is as follows: Based on the IVFFlat index construction method, N vectors are clustered to obtain K clusters and cluster centers, K inverted lists are established, and the residual vectors of each vector and the cluster center are stored in the inverted lists; all residual vectors in each inverted list are then multiplied and quantized to obtain a codebook; the codebook and all clustering parameters are saved as the IVFPQ index model; The product quantization process includes: dividing N residual vectors of dimension D, each vector is divided into M sub-vectors, each sub-vector dimension is D / M; performing Kmeans clustering on the resulting M groups, each group containing N vectors of dimension D / M; each group contains K2 clusters, for a total of MK2 clusters and cluster centers, and encoding the cluster centers to obtain MK2 IDs as a codebook; After building the index model, the search engine model is loaded into the child process and the search engine is called via a remote procedure call. When the main program calls the search function of the face search engine, the search engine automatically accelerates the search process based on the index building method and returns the matching feature vector ID. When the main program passes newly stored facial feature vectors, the search engine builds an index for the newly added vectors and keeps the current number of clusters and search engine model parameters unchanged; users set the search engine initialization interval through task scheduling in the main program to obtain optimal performance under incremental data.

2. The adaptive face search method according to claim 1, wherein: The step 2 of extracting features from the face image using the face feature extraction module includes: the face feature extraction module uses a deep neural network algorithm to extract features from the acquired face data to obtain a 128-dimensional face feature vector, and passes the face feature vector to the face search module for face search or uploading new data; wherein the face feature extraction module uses 17 feature points to mark the chin, 5 feature points to mark the left and right eyebrows respectively, 6 feature points to mark the left and right eyes respectively, 4 feature points to mark the bridge of the nose and 5 feature points to mark the tip of the nose respectively, and 12 feature points to mark the upper and lower lips respectively, for a total of 72 feature points; after the feature point sequence is calculated by the neural network, a one-dimensional vector with a length of 128 is obtained as the identification information of the face; Among them, before the pre-trained facial feature extraction module extracts the D-dimensional facial feature vector, it also includes: using a manually labeled facial feature point dataset to train the ResNet50 deep learning model, stopping the training when the feature extraction module recognition reaches the expected value, and fixing the model parameters as the facial feature extraction model.

3. The adaptive face search method according to claim 1, wherein: The face search module in step three searches and adds face feature vectors, including: when the face search module provides a search function, a remote procedure calls a face search engine to perform a search operation; the face search engine uses different search methods according to the current index construction method to return all search results that meet the similarity threshold to the face search module, and the face search module then merges the search engine return results and additional information in the database as search results and returns them to the user; when the face search module provides an upload function, the remote procedure calls the face search engine to add a new face feature vector; the face search engine uses different insertion methods according to the current index construction method to build an index and create an ID for the feature vector.

4. An adaptive face search system using the adaptive face search method according to any one of claims 1 to 3, characterized in that: The adaptive face search system includes: an interactive interface, a face feature extractor, a message queue, a face search module, a database, and a face search engine; The search interface is used to complete the interaction with the user, including receiving requests and pictures from the user and returning information to the user in an appropriate manner; The facial feature extractor is used to identify feature vectors containing facial feature information from facial images; The message queue is used to cache pictures that need to be added and uploaded by users through the interface; The database is used to store feature vectors and unstructured information related to the image; The face search module is the core control and scheduling module of the entire system and the processing center of all data. It is responsible for receiving the scheduling of the face search module, searching and adding face feature vectors and returning the results to the face search engine.

5. The adaptive face search system according to claim 4, wherein: The adaptive face search system further includes: The face picture uploading interface and the search receive face pictures and pass them to the face feature extraction module; The facial feature extraction module is used to pre-process the acquired facial image data, identify feature vectors including eyebrows, nose, eyes, mouth and facial contours using a deep convolutional neural network, and then transfer the feature vectors and image-related information to the storage module and the face search module for storage; The search module is used to input the number of faces in the database and related device information into a decision tree; the decision tree adaptively selects the index construction method with the best performance, trains and generates a face search engine model, and the face search engine model is maintained in memory to provide search and incremental update storage functions; The search interface is used to extract facial features from the uploaded image after receiving it, and input the extracted facial feature vector into the face search engine; The face search engine is used to calculate the L2 distance between the current face feature vector and the stored feature vector as the similarity between faces. For all pictures that meet the search similarity provided by the user, the system retrieves the corresponding face picture information from the storage module and returns it to the user for display.

6. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the adaptive face search method according to any one of claims 1 to 3.

7. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the adaptive face search method according to any one of claims 1 to 3.

8. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the adaptive face search system as described in any one of claims 4 to 5.

Images