Face recognition feature online updating and recognition method and device

Abstract

The invention discloses a face recognition feature online updating and recognition method and device, and relates to the field of face recognition, and the method comprises the steps: matching a first feature of a face image with a first feature library corresponding to a lightweight model; similarity comparison is carried out on the second feature of the face picture and a second feature library corresponding to the high-precision model, and a first feature library corresponding to the updated lightweight model is judged through the similarity; the high-precision model is introduced on the basis of the lightweight model, and the first feature library corresponding to the lightweight model is updated through similarity judgment of the high-precision model, so that the accuracy and reliability of the features in the first feature library are improved, and online updating of the feature library in face recognition is realized. The recognition method based on the face recognition feature online updating method not only can guarantee the recognition rate, but also effectively improves the recognition precision, and achieves the high efficiency and accuracy of face recognition.

Description

Technical Field

[0001] The present invention relates to the field of face recognition, and in particular to a face recognition feature online updating, recognition method and device. Background Art

[0002] Face recognition is a biometric technology that identifies people based on their facial features. Combined with deep learning network models, it is widely used in access control systems, home security, electronic payments, and identity verification. Face recognition tasks have evolved from manually marking features to training deep learning neural network models to extract facial features, and the accuracy and robustness of recognition have been improved. However, deep learning models use convolution to extract features, and different terminal devices, such as identity authentication on mobile phones or access control recognition tasks, are limited by hardware device resources.

[0003] At the same time, due to the lightweight development of existing recognition models, such as reducing model parameters and reducing the resource usage of deep learning models, the face image database is limited, and when multiple tasks need to be run simultaneously, the recognition task cannot be performed in real time and efficiently.

[0004] To solve the above problems, a new solution is urgently needed. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a method and device for online updating and recognition of face recognition features, which introduces a high-precision model on the basis of a lightweight model, and updates the first feature library corresponding to the lightweight model through similarity determination of the high-precision model, thereby improving the accuracy and reliability of the features in the first feature library and realizing online updating of the feature library in face recognition. The recognition method based on the online update method of face recognition features can not only ensure the recognition rate, but also effectively improve the recognition accuracy, thus realizing efficient and accurate face recognition.

[0006] The present invention is achieved through the following technical solutions: The present invention provides a method for online updating of face recognition features, the method comprising the following steps: Acquire a face image, input the face image into the lightweight model to extract a first feature, and match the extracted first feature with a first feature library; If the preset conditions are met for the high-precision model to run, the face image that successfully matches the first feature library is input into the high-precision model to extract the second feature, and the second feature extracted by the high-precision model is compared with the second feature library for similarity; When the similarity meets the set similarity threshold, the first feature of the corresponding face image is input into the first feature library to update the first feature library.

[0007] Preferably, before inputting the face pictures successfully matched with the first feature library into the high-precision model, the face pictures successfully matched with the first feature library are accumulated into the face picture set; Compare the number of face images in the face image set with a set number threshold; When the number of face picture sets is greater than or equal to a set number threshold, if the preset conditions are met for the high-precision model to run, the face pictures in the face picture sets are input into the high-precision model.

[0008] Preferably, when the similarity between the second feature extracted by the high-precision model and the second feature library does not meet a set similarity threshold, the first feature extracted from the lightweight model of the corresponding face image is deleted.

[0009] Preferably, when the number of face pictures in the face picture set is greater than or equal to a set number threshold, but the preset conditions are not met when the high-precision model is running, the current storage space is judged. When the current storage space is sufficient, the face pictures are temporarily stored; when the storage space is insufficient, the face pictures in the face picture set are deleted in chronological order starting from the first picture.

[0010] Preferably, the following steps are included before obtaining the face image: obtaining a face target frame, and cropping the face target frame to obtain the face image.

[0011] A method for performing face recognition using the first feature library obtained by the above-mentioned face feature online update method, the method comprising: Get face pictures; The face image is input into the lightweight model to extract the first feature, and the first feature is matched with the first feature library to generate a face recognition result.

[0012] Preferably, the face target frame corresponding to the successfully matched face image is assigned to the Kalman filter to achieve real-time tracking of the corresponding face image.

[0013] The present invention also provides a face recognition feature online updating device, the online updating device comprising: Lightweight model feature extraction module: obtains a face image and inputs the face image into the lightweight model for first feature extraction; A first feature library matching module: matches the first feature obtained by the lightweight model extraction module with the features in the first feature library; Preset condition determination module: determines whether the preset conditions meet the operation of the high-precision model; High-precision model feature extraction module: When the preset conditions are met for the operation of the high-precision model, the face image that successfully matches the first feature library is input into the high-precision model for second feature extraction; Second feature library comparison module: compares the second feature extracted by the high-precision model with the features in the second feature library for similarity; Determination update module: When the similarity meets the set similarity threshold, the corresponding face image is input into the lightweight model to extract the first feature and input it into the first feature library.

[0014] Preferably, the high-precision model feature extraction module includes: Face image accumulation module: when the preset conditions are met for the high-precision model to run, the face images that successfully match the first feature library are accumulated into the face image set; Face picture quantity determination module: compares the number of face pictures in the face picture set with the set quantity threshold; Face picture set input module: When the number of face picture sets is greater than or equal to the set number threshold, the face pictures in the face picture sets are input into the high-precision model.

[0015] The present invention also provides a device based on the above-mentioned face recognition method, including a first feature library obtained by the face feature online update method, and the device includes: Lightweight model feature extraction module: obtains face images and inputs them into the lightweight model for feature extraction; The first feature library matching module matches the features obtained by the lightweight model extraction module with the features in the first feature library to obtain lightweight feature matching results.

[0016] An electronic device comprises: a processor and a memory for storing a computer program; wherein the processor is configured to implement the steps of the above-mentioned face recognition feature online update method and / or face recognition method when executing the computer program.

[0017] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned face recognition feature online update method and / or face recognition method.

[0018] Compared with the prior art, the present invention has the following beneficial technical effects: The present invention provides a method for online updating of face recognition features, which realizes the extraction of the first feature of a face picture through a lightweight model, and inputs the face picture successfully matched with the first feature library into a high-precision model by matching the extracted first feature with a first feature library, and compares the second feature extracted by the high-precision model with the features in the second feature library; and determines whether the similarity meets a set similarity threshold, and inputs the first feature corresponding to the second feature that meets the set similarity threshold into the first feature library, thereby realizing online updating of the first feature library.

[0019] Furthermore, the number of face images in the successfully matched face image set is accumulated, and by comparing the accumulated number of face images with a set number threshold, the face image set greater than or equal to the set number threshold is input into the high-precision type for second feature extraction.

[0020] Furthermore, by setting a similarity threshold, on the one hand, it can meet the accuracy requirements in different applications, and at the same time, by deleting the first feature extracted from the lightweight model corresponding to the face image that does not meet the similarity threshold, it can effectively ensure the operation of the device in the actual application.

[0021] Furthermore, by determining whether the preset conditions at different times meet the operation judgment of the high-precision model, the problem of insufficient operating space caused by the preset conditions is reduced, ensuring the efficiency of face recognition and the stability of the actual application equipment.

[0022] Furthermore, in the present invention, when the preset conditions are not met for the operation of the high-precision model, the current storage space is judged to achieve temporary storage of face images in the face image set, which can not only ensure stable operation at the current moment; at the same time, when the high-precision model operation conditions are met, the face images in the face image set in the storage space are read, thereby realizing the application of face images at different times.

[0023] The present invention also provides a face recognition method, which not only ensures recognition accuracy but also ensures recognition efficiency by matching the first feature extracted by the lightweight model with the first feature library updated online.

[0024] Furthermore, the present invention assigns the face target frame corresponding to the successfully matched face image to the Kalman filter to achieve real-time tracking of the corresponding face image, thereby effectively improving the application of the successfully matched face image. This solves the problem of low recognition accuracy and poor tracking effect when the target face is moving. In the face tracking process, the matching results of the face image are introduced to improve the accuracy of the tracking results, and when occlusion occurs or cross-lens face re-recognition is required, the target can be quickly located based on the matching features. The target performance of face tracking is improved. When the target is recognized incorrectly for multiple consecutive frames, it means that the target tracking is lost, and the tracker needs to be destroyed or a new tracker needs to be regenerated. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1A system diagram of a facial feature online updating method in a specific application environment provided by a preferred embodiment of the present invention; Figure 2 A flowchart of a method for online updating of face recognition features provided in a preferred embodiment of the present invention; Figure 3 A flowchart of a face recognition method provided in a preferred embodiment of the present invention; Figure 4 A schematic diagram of the structure of a device for online updating of face recognition features provided by a preferred embodiment of the present invention; Figure 5 A schematic diagram of the structure of a high-precision model feature extraction module provided in a preferred embodiment of the present invention; Figure 6 A schematic diagram of the structure of a preset condition determination module provided in a preferred embodiment of the present invention; Figure 7 A flowchart of a method for online updating of face recognition features provided in a preferred embodiment of the present invention when the similarity does not meet a set similarity threshold; Figure 8 The present invention is a flowchart of another method for online updating of face recognition features provided in a preferred embodiment of the present invention.

[0027] In the figure: 102, terminal; 103, network; 104, server; 106, database. DETAILED DESCRIPTION

[0028] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0029] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0030] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision that uses cameras and computers to replace human eyes to identify and measure targets, and further performs graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0031] The solution provided by the embodiment of the present invention designs the image recognition technology of artificial intelligence, and is specifically described through the following embodiments: The image recognition model training method provided in this application can be applied to Figure 1 The system in the illustrated environment includes a terminal 102 , a network 103 , a server 104 and a database 106 .

[0032] Terminal 102: The terminal 102 communicates with the server 104 via the network 103. The management terminal 102 can send a recognition instruction to the server 104, and the server 104 obtains the face image from the database 106 according to the instruction sent by the management terminal 102.

[0033] The user may use the terminal 102 to send a request for face image recognition and / or face image feature update to be processed. The terminal 102 may provide an interface that enables the user of the terminal 102 to interact with the terminal 102. The terminal 102 may also output information to the user via the interface. In addition to the terminal 102 provided in this embodiment, various numbers of terminals 102 may be supported in actual use.

[0034] The terminal 102 may include various categories of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, game systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computer devices may run various categories and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as GOOGLE ChromeOS); or include various mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular phones, smart phones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Game systems may include various handheld game devices, Internet-enabled game devices, etc. Client devices are capable of executing a variety of different applications, such as various Internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and may use various communication protocols.

[0035] The network 103 may be any type of network known to those skilled in the art, which may support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, one or more networks 103 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0036] Server 104: Server 104 obtains the face picture from database 106; server 104 calls the lightweight model and the high-precision model based on the face picture to update the face features and perform face recognition.

[0037] The server 104 may run one or more services or software applications that enable the execution of the facial feature update and / or recognition method. The server 104 may also provide other services or software applications, which may include non-virtual environments and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to the terminal 102 under a software as a service (SaaS) model.

[0038] The server 104 may include one or more components that implement the functions performed by the server 104. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. A user operating the terminal 102 may in turn utilize one or more client applications to interact with the server 104 to utilize the services provided by these components. It should be understood that various different system configurations are possible, which may be different from the system provided in this embodiment.

[0039] The server 104 may include one or more general-purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other appropriate arrangement and / or combination. The server 104 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that can be virtualized to maintain a server's virtual storage device). In different embodiments, the server 104 may run one or more services or software applications that provide the functionality described below. The computing unit in the server 104 may run one or more operating systems including any of the above-mentioned operating systems and any commercially available server operating system. The server 104 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, and the like.

[0040] Database 106: Database 106 provides data services for server 104; database 106 can be used to store face image data. One or more of databases 106 can also be used by applications to store application data. The database used by the application can be a database of different categories, such as a key-value storage library, an object storage library, or a conventional storage library supported by a file system.

[0041] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0042] In a preferred embodiment of the present invention, a method for updating face recognition features online is provided. Figure 2 As shown, the face recognition feature online updating method comprises the following steps: Step 210: Acquire a face image; input the face image into the lightweight model to extract the first feature, match the extracted first feature with the feature in the first feature library; temporarily store the successfully matched first feature; temporarily store the successfully matched first feature in the operating space of the system; Step 220: Determine whether the preset conditions at the current moment meet the high-precision model operation; Step 230: When the preset conditions are met and the high-precision model is running, the face image that successfully matches the first feature library is input into the high-precision model to extract the second feature, and the second feature extracted by the high-precision model is compared with the second feature library for similarity; Step 240: obtaining similarity by comparing the second feature with the second feature library, and comparing the similarity with a set similarity threshold; Step 250: When the similarity meets the set similarity threshold, the first feature extracted during the lightweight input of the corresponding face image is input into the first feature library.

[0043] The above-mentioned facial image feature update method is a feature update method for the same face recognition. In other embodiments of the present invention, simultaneous recognition of multiple faces can be achieved. In this case, step 220 is not affected by step 210. Whether the preset conditions are met and the operation of the high-precision model is a real-time process. According to the preset conditions at different times, it directly proceeds to the next step.

[0044] In a preferred embodiment of the present invention, before obtaining a face image, the following steps are included: obtaining a face target frame, and cropping the face target frame to obtain a face image.

[0045] In a preferred embodiment of the present invention, a face image is input into a lightweight model for feature extraction to obtain a first feature extracted after the face image is input into the lightweight model; and a face image in a face image set is input into a high-precision model for feature extraction to obtain a second feature extracted after the face image is input into the high-precision model.

[0046] Specifically, the first feature vector is matched with the features in the first feature library to obtain the matching result of the face image and the first feature library; the second feature vector is compared with the features in the second feature library for similarity to perform a secondary screening of the face image, thereby improving the matching accuracy of the face image.

[0047] In a preferred embodiment of the present invention, when the similarity does not meet the set similarity threshold, Figure 7 As shown, step 241 is also included: deleting the first feature extracted from the lightweight model of the corresponding face picture whose second feature vector does not meet the similarity threshold.

[0048] The first feature corresponding to the face image that does not meet the similarity requirement is deleted to increase the storage space. In this embodiment, the first feature extracted by the lightweight model is temporarily stored in the running space. By deleting the first feature, the memory running space can be effectively released. At the same time, by matching the second feature vector with the features in the second feature library, a high-precision model matching result can also be achieved, thereby improving the accuracy of face recognition.

[0049] In a preferred embodiment of the present invention, after matching the first feature of the face image with the features in the first feature library, Figure 8 As shown, the following steps are included: Step 221: adding the face images that successfully match the first feature library to the face image set; Step 212: comparing the number of face images in the face image set with a set number threshold, and determining whether the number of face images in the face image set is greater than or equal to the set number threshold of face images; When the number of face images in the face image set is greater than or equal to the set number threshold, step 220 is performed: whether the preset conditions are met for high-precision model operation; When the number of face pictures in the face picture set is less than the set number threshold, return to step 212 and continue to accumulate face pictures in the face picture set until the number threshold requirement of the face picture set is met.

[0050] In this preferred embodiment, there are 25 face pictures accumulated in the face picture set, and the set quantity threshold is 20, which meets the set quantity threshold requirement. In this embodiment, the quantity threshold is the initially set quantity threshold, and in actual use, the quantity threshold can also be changed according to different storage space changes.

[0051] In this preferred embodiment, each picture is set By setting an upper limit on the number of face images in a face image set, the occupancy of the operating space in different application environments can be effectively limited. By storing them in the storage space, smooth operation in actual use can be further guaranteed.

[0052] In a preferred embodiment of the present invention, before determining the number of face images in the face image set, it is determined whether the preset conditions at the current moment meet the operation of the high-precision model. When the high-precision model is not satisfied, the storage space determination at the current moment is entered. When the storage space is sufficient, the face image set is temporarily stored. In this embodiment, the face image set is temporarily stored in the storage space. When the storage space is sufficient, when the preset conditions for the operation of the high-precision model are met, the face image set in the storage space is read; at the same time, when the face image set is temporarily stored, face images continue to be accumulated; by temporarily storing, when the preset conditions meet the operation of the high-precision model, feature extraction is performed on the previously acquired face image set. This step effectively improves the utilization rate of face images that are successfully matched in the first feature library.

[0053] At the same time, when the storage space is insufficient, the face pictures in the face picture set are deleted from the first picture in the queue, thereby ensuring the smooth operation of the face recognition device; by following the first-in-first-out principle according to the different time sequences, the effective use of the pictures is further guaranteed.

[0054] In a preferred embodiment of the present invention, when using Figure 2 In the face recognition feature updating method shown, after the preset space does not meet the requirements for running a high-precision model, when the storage space is determined to be insufficient, the corresponding face image and / or the first feature is deleted.

[0055] In this preferred embodiment, the similarity threshold can be set before updating / matching, and can also be changed according to different requirements during actual operation.

[0056] In this preferred embodiment, the first feature and the second feature of the face image are both embodied in the form of feature vectors. In the similarity calculation of face images based on feature vectors, the following similarity calculation methods are mainly included: 1. Euclidean distance: The difference between two facial features is quantified as a distance value. The smaller the distance, the more similar the two faces are.

[0057] 2. Cosine similarity: quantifies the angle between two facial features into a similarity value. The higher the cosine similarity, the more similar the two faces are.

[0058] 3. Cross-correlation: The correlation between two facial features is quantified into a similarity value.

[0059] In this preferred embodiment, the calculation of the feature vector similarity of the second feature adopts cosine similarity, and the specific calculation method is as follows: A and B represent the feature vector of the second feature and the feature vector of the feature in the second feature library respectively. The formula is as follows: Formula (1); In this embodiment, the length of the facial feature vectors of A and B is a high-dimensional space (usually the feature vector length is , in this embodiment, 256-dimensional space is exemplarily adopted); wherein, A = (1, 1, 2, 1, ..., 1, 1, 0, 0, 0); B = (1, 1, 1, 0,…, 1, 1, 1, 1, 1).

[0060] According to formula (1), .

[0061] In this embodiment, the similarity threshold is set to 0.7, and the similarity obtained is greater than or equal to the set similarity threshold. The facial image corresponding to the feature vector of the second feature represented by A meets the similarity requirement of the present invention, and the first feature of the corresponding facial image is input into the first feature library to realize online update of the first feature library.

[0062] It not only ensures the matching speed of the first feature library, but also enables the high-precision model to update the first feature library in real time for the lightweight model. Under limited preset conditions, the lightweight model can be used to quickly identify the target face, and according to the resource occupancy, the high-precision model with a larger number of parameters can be used to update the first feature library. At the same time, through the more accurate recognition of the high-precision model and the optimization of the matching effect of the second feature library, the online update of the features in the first feature library can be achieved, further improving the efficiency of face recognition.

[0063] In this preferred embodiment, the lightweight model includes EdgeFace, GhostFaceNet, etc. By reducing the number of model parameters, it is possible to have the characteristics of less resource occupation and high recognition efficiency while maintaining a certain accuracy.

[0064] In this preferred embodiment, the high-precision model uses Fine-Tuned ArcFace, which has the advantages of high precision, large amount of calculation, and is suitable for complex and changeable detection environments. When updating the feature library, it can obtain accurate and reliable matching features. The high-precision model can narrow the gap between similar features of input face images of the same category and expand the feature difference between face images of different categories. The solution provided by the present invention can effectively ensure the recognition efficiency and recognition accuracy of the face recognition method.

[0065] In a preferred embodiment of the present invention, a method for performing face recognition based on the first feature library obtained by the above-mentioned face feature online updating method is also provided. Figure 3 As shown, the following steps are included: Step 410: Obtain a face image; Step 420: Input the face image into the lightweight model to extract the first feature, and match the first feature with the first feature library to generate a face recognition result.

[0066] In a preferred embodiment of the present invention, during the face recognition process, after obtaining the matching result of the first feature and the first feature library, it is judged whether the preset conditions meet the high-precision model operation. If the preset conditions meet the high-precision model operation, the face image is input into the high-precision model for second feature extraction, and the second feature extracted by the high-precision model is matched with the second feature library to obtain the high-precision model feature matching result to generate the face recognition result of the high-precision model feature. When the recognition result of the high-precision model feature is different from the recognition result of the lightweight feature, the first feature of the corresponding face image is deleted.

[0067] By directly obtaining high-precision matching results. In this embodiment, by selecting a high-precision model, the problem of difficulty in extracting features due to the small number of parameters in the lightweight model can be effectively solved, while ensuring the recognition effect under different lighting, ages and facial postures. In addition, by comparing the feature recognition results of the high-precision model and the lightweight model, the first feature is screened.

[0068] In a preferred embodiment of the present invention, the face target frame corresponding to the successfully matched face image is input into the Kalman filter, and the face target frame is tracked in real time through the Kalman filter. The Kalman filter predicts the position of the face target frame in the next frame according to the position of the currently detected face target frame; in actual use, when the target cannot be detected for a long time, the Kalman filter is terminated, and when the corresponding face target frame is detected again, the Kalman filter is started again. In this way, real-time tracking of the face image is achieved.

[0069] In a preferred embodiment of the present invention, a color image is obtained by an image sensor, a face target frame is detected by using a face detection model on the color image, face key point information is obtained by cropping the face target frame, and the face key point information of the face target frame is aligned with the model to be matched to obtain a face picture. Specifically, the face key point information of the face target frame is extracted, and the extracted face key point information is aligned with the preset key point information.

[0070] In a preferred embodiment of the present invention, the image is acquired by an image sensor, wherein the image sensor is used to collect various image data, and can be a monocular or multi-camera (such as an RGB camera, a depth camera or a point cloud camera, etc.), a video camera, or a mobile phone, tablet computer, laptop computer, desktop computer or other device including a camera or a video camera.

[0071] In a preferred embodiment of the present invention, face target frame detection adopts a deep convolution model, such as Yolo, Co-DETR, EVA, InternImage-H, MoCaE, Focal-Stable-DINO, etc.

[0072] In a preferred embodiment of the present invention, the preset conditions include hardware resources; whether the hardware conditions meet the operating requirements of the high-precision model, including GPU resource occupancy, memory occupancy, etc.

[0073] Further references Figure 4 , as a response to the above Figure 2 In order to realize the method shown in the figure, a preferred embodiment of the present invention also provides a device for updating face recognition features online. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied in a controller.

[0074] like Figure 4 As shown, a face recognition feature online updating device 300 provided in this preferred embodiment includes: Lightweight model feature extraction module 310: obtains a face image, and inputs the face image into the lightweight model to perform first feature extraction; First feature library matching module 320: matches the first feature obtained by the lightweight model extraction module with the features in the first feature library; Preset condition determination module 331: determines whether the preset condition at the current moment satisfies the operation of the high-precision model; High-precision model feature extraction module 330: inputting the face image successfully matched with the first feature library into the high-precision model to extract the second feature; Second feature library comparison module 340: performs similarity comparison between features extracted by the high-precision model and features in the second feature library; Determination update module 350: When the similarity meets the set similarity threshold, the corresponding face image is input into the lightweight model to extract the first feature and input it into the first feature library.

[0075] In a preferred embodiment of the present invention, Figure 5 As shown, the high-precision model feature extraction module 330 also includes: Face picture accumulation module 332: when the preset conditions are met for the high-precision model to run, the face pictures that successfully match the first feature library are accumulated into the face picture set; The face picture quantity determination module 333 compares the number of face pictures in the face picture set with a set number threshold; Face picture set input module 334: when the number of face picture sets is greater than or equal to a set number threshold, the face pictures in the face picture sets are input into the high-precision model.

[0076] In a preferred embodiment of the present invention, the determination update module 350 also includes: Lightweight model feature deletion module: When the similarity between the second feature extracted by the high-precision model and the second feature library does not meet the set similarity threshold, the first feature extracted from the lightweight model of the corresponding face image is deleted.

[0077] In this embodiment, the preset condition determination module 331 further includes: Storage space determination module: When the preset conditions are not met when the high-precision model is running, the current storage space is determined.

[0078] Face picture set temporary storage module: When the current storage space is sufficient, the face picture set will be temporarily stored.

[0079] Face image deletion module: When the storage space is insufficient, the face images in the face image set are deleted in a queue starting from the first one.

[0080] In this embodiment, the lightweight model feature extraction module 310 further includes: Face image acquisition module: obtain the face target frame and crop the face target frame to obtain the face image.

[0081] Further references Figure 6 , as a response to the above Figure 3 In order to realize the method shown in the figure, a preferred embodiment of the present invention further provides a face recognition device 400, including the first feature library obtained by the online updating method. Figure 3 Corresponding to the method embodiment shown, the device can be specifically applied in a controller.

[0082] like Figure 6 As shown, a face recognition device 400 provided in this preferred embodiment includes: Lightweight model feature extraction module 310: obtains a face image, and inputs the face image into the lightweight model to perform first feature extraction; The first feature library matching module 320 matches the first feature obtained by the lightweight model extraction module with the features in the first feature library to obtain a lightweight feature matching result.

[0083] In a preferred embodiment of the present invention, the first feature library matching module 320 further includes: Preset condition determination module: determines whether the preset conditions at the current moment meet the operation of the high-precision model; High-precision model feature extraction module: input the face image that successfully matches the first feature library into the high-precision model for second feature extraction; The second feature library matching module matches the second feature extracted by the high-precision model with the features in the second feature library.

[0084] In a preferred embodiment of the present invention, the lightweight model feature extraction module 310 further includes: Face image tracking module: assigns the face target frame corresponding to the successfully matched face image to the Kalman filter.

[0085] It can be understood that the face recognition feature online updating method and the face recognition method provided in the embodiments of the present invention can be applied to any electronic device.

[0086] A preferred embodiment of the present invention also provides an electronic device, comprising: a processor and a memory for storing a computer program; wherein the processor is configured to implement the above-mentioned steps of the online update method of facial recognition features and / or the facial recognition method when executing the computer program.

[0087] Among them, the controller can be but is not limited to: a computer terminal (Personal Computer, PC); an industrial control computer terminal (Industrial Personal Computer, IPC); a mobile terminal; a server; a system including a terminal and a server, and realized through the interaction between the terminal and the server; a programmable logic controller (Programmable Logic Controller, PLC); a field programmable gate array (Field-Programmable Gate Array, FPGA); a digital signal processor (Digital Signal Processer, DSP) or a microcontroller unit (Microcontroller unit, MCU).

[0088] The electronic devices include, for example, a robot arm, or a server, workstation, personal computer, mobile terminal (such as a mobile phone, tablet computer, etc.), wearable device (such as glasses, watches, etc.), robot, etc., which is used as a robot arm control device. The robot may be, but is not limited to, a humanoid robot; or a robot connected in series or in parallel (such as a Delta robot, a four-axis robot, or a six-axis robot).

[0089] A preferred embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned face recognition feature online update method and / or face recognition method.

[0090] The electronic device may include one or more of the following components: a processor, a memory, a power component, a multimedia component, an audio component, an input / output (I / O) interface, a sensor component, and a communication component.

[0091] In this embodiment, the processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor is used to run program codes stored in the memory or process data.

[0092] The processor generally controls the overall operation of the electronic device, such as display, phone calls, data communications, camera operations, and operations associated with recording operations. The processor may include one or more processors to execute instructions to complete all or part of the steps of the above-mentioned face recognition feature online update method and / or face recognition method. In addition, the processor may include one or more modules to facilitate the interaction between the processor and other components. For example, the processor may include a multimedia module to facilitate the interaction between the multimedia components and the processor.

[0093] The memory is configured to store various types of data to support the operation of the device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0094] The memory may be implemented by any type of volatile or nonvolatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0095] The power assembly provides power to the various components of the device. The power assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic devices.

[0096] The multimedia component includes a screen that provides an output interface between the electronic device and the user. In this embodiment, the screen may include a liquid crystal display panel and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In this embodiment, the multimedia component includes a front camera and / or a rear camera. When the device is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0097] The audio component is configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in a memory or sent via a communication component. In this embodiment, the audio component also includes a speaker for outputting an audio signal.

[0098] The I / O interface provides an interface between the processor and the peripheral interface module, which may be a keyboard, a click wheel, buttons, etc. These buttons may include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0099] The sensor assembly includes one or more sensors for providing various aspects of status assessment for the device. For example, the sensor assembly can detect the open / closed state of the device and the relative positioning of the components. For example, if the components are display panels and keypads of electronic devices, the sensor assembly can also detect changes in the position of the device or a component of the device, the presence or absence of user contact with the device, the orientation or acceleration / deceleration of the electronic device, and the temperature change of the device. The sensor assembly can also include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In the preferred embodiment, the sensor assembly can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0100] The communication component is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, and the wireless connection method can be but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultrawideband) connection, and other wireless connection methods currently known or developed in the future.

[0101] In this preferred embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In this embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies. In an exemplary embodiment, the electronic device can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned robotic arm plug-in control method.

[0102] The present invention also provides a computer-readable storage medium, including a memory of instructions, the instructions can be executed by a processor of a device to complete the above-mentioned dexterous hand grasping control method. For example, a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0103] Among them, the memory includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc.

[0104] The memory may also be an external storage device of a computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the computer device.

[0105] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0106] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0107] Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can essentially be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, disk, or CD), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present application.

[0108] Among them, computer equipment is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0109] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with the user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0110] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the attached claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any figure mark in the claims should not be regarded as limiting the claims involved.

[0111] In addition, it should be understood that although this specification is described in accordance with the implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation modes that can be understood by those skilled in the art. The above content is only to illustrate the technical idea of ​​the present invention, and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution according to the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A method for online updating of face recognition features, characterized in that: The method comprises the following steps: Acquire a face image, input the face image into the lightweight model to extract a first feature, and match the extracted first feature with a first feature library; If the preset conditions are met for the high-precision model to run, the face image that successfully matches the first feature library is input into the high-precision model to extract the second feature, and the second feature extracted by the high-precision model is compared with the second feature library for similarity; When the similarity meets the set similarity threshold, the first feature of the corresponding face image is input into the first feature library to update the first feature library.

2. The method for online updating of face recognition features according to claim 1, characterized in that: Before inputting the face images successfully matched with the first feature library into the high-precision model, the face images successfully matched with the first feature library are accumulated into the face image set; Compare the number of face images in the face image set with a set number threshold; When the number of face picture sets is greater than or equal to a set number threshold, if the preset conditions are met for the high-precision model to run, the face pictures in the face picture sets are input into the high-precision model.

3. A method for online updating of face recognition features according to claim 1 or 2, characterized in that: When the similarity between the second feature extracted by the high-precision model and the second feature library does not meet the set similarity threshold, the first feature extracted from the lightweight model of the corresponding face image is deleted.

4. The method for online updating of face recognition features according to claim 2, characterized in that: When the number of face images in the face image set is greater than or equal to the set number threshold, but the preset conditions are not met when the high-precision model is running, the current storage space is judged. When the current storage space is sufficient, the face images are temporarily stored; when the storage space is insufficient, the face images in the face image set are deleted in chronological order starting from the first image.

5. A method for online updating of face recognition features according to claim 1 or claim 2, characterized in that: The following steps are included before obtaining the face image: obtaining the face target frame, and cropping the face target frame to obtain the face image.

6. A method for face recognition based on the first feature library obtained by the face feature online updating method according to any one of claims 1 to 5, characterized in that: The method comprises: Get face pictures; The face image is input into the lightweight model to extract the first feature, and the first feature is matched with the first feature library to generate a face recognition result.

7. The method according to claim 6, characterized in that The face target frame corresponding to the successfully matched face image is assigned to the Kalman filter to achieve real-time tracking of the corresponding face image.

8. A face recognition feature online updating device, characterized in that: The online updating device comprises: Lightweight model feature extraction module: obtains a face image and inputs the face image into the lightweight model for first feature extraction; A first feature library matching module: matches the first feature obtained by the lightweight model extraction module with the features in the first feature library; Preset condition determination module: determines whether the preset conditions meet the operation of the high-precision model; High-precision model feature extraction module: When the preset conditions are met for the operation of the high-precision model, the face image that successfully matches the first feature library is input into the high-precision model for second feature extraction; Second feature library comparison module: compares the second feature extracted by the high-precision model with the features in the second feature library for similarity; Determination update module: When the similarity meets the set similarity threshold, the corresponding face image is input into the lightweight model to extract the first feature and input it into the first feature library.

9. The device for online updating of face recognition features according to claim 8, characterized in that: The high-precision model feature extraction module includes: Face image accumulation module: when the preset conditions are met for the high-precision model to run, the face images that successfully match the first feature library are accumulated into the face image set; Face picture quantity determination module: compares the number of face pictures in the face picture set with the set quantity threshold; Face picture set input module: When the number of face picture sets is greater than or equal to the set number threshold, the face pictures in the face picture sets are input into the high-precision model.

10. A device based on the face recognition method according to claims 6 to 7, comprising a first feature library obtained by an online face feature update method, characterized in that: The device comprises: Lightweight model feature extraction module: obtains face images and inputs them into the lightweight model for feature extraction; The first feature library matching module matches the features obtained by the lightweight model extraction module with the features in the first feature library to obtain lightweight feature matching results.

Images