A face recognition system and a face recognition method

Abstract

The embodiment of the present invention discloses a face recognition system and a face recognition method. In a specific implementation, the face recognition system includes a collection device, a first terminal device, a first server, and a second server; the first terminal device is configured to receive image data and / or video data and send it to the second server; the second server is configured to receive image data and / or video data; perform face detection on the image data and / or video data to obtain a face image; recognize the face image to obtain the face attribute and the first face feature corresponding to the face image; classify the face image according to the first face feature and store it in a face library. The face recognition system of this implementation can classify and store the detected face images in real time, so as to facilitate later data statistics and face queries, and can achieve a high level in terms of accuracy and real-time performance, fully meeting the needs of face control analysis.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more specifically, to a face recognition system and a face recognition method. Background Art

[0002] At present, in areas with high traffic volume such as major events, major festivals, concerts, and stations, facial recognition can be used to carry out intelligent monitoring of key personnel.

[0003] The existing facial recognition control technology usually uses a camera to collect facial images in the control area, and then uses network transmission to send the facial images to the system background for recognition and comparison, so as to accurately identify key personnel under control and achieve better security prevention and monitoring effects.

[0004] However, the existing control systems generally send the collected facial images to the system background for identification and comparison, which performs poorly in terms of accuracy and real-time performance, and can hardly meet the needs of facial recognition control. Summary of the invention

[0005] The object of the present invention is to provide a face recognition system and a face recognition method to solve at least one of the problems existing in the prior art.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A first aspect of the present invention provides a face recognition system, comprising a collection device, a first terminal device, a first server and a second server;

[0008] A first terminal device is configured to receive image data and / or video data from the acquisition device and send the image data and / or video data to the second server;

[0009] The second server is configured as:

[0010] Receiving image data and / or video data from a first terminal device;

[0011] Calling a trained model in the first server to perform face detection on the image data and / or video data to obtain a face image;

[0012] Calling the trained model in the first server to recognize the facial image to obtain facial attributes and first facial features corresponding to the facial image;

[0013] The face image is classified according to the first face feature and stored in a face library, which includes the face image, the first face feature and the corresponding unique identification code.

[0014] Optionally, the first server is a cloud server including a model repository for storing trained models.

[0015] Optionally, the first server is configured as:

[0016] Receive the trained model and store it in the model warehouse;

[0017] Importing the trained model in the model warehouse into the second server;

[0018] A face recognition creation request is sent to the second server, and a face recognition interface is received from the second server.

[0019] Optionally, the second server is an AI server.

[0020] Optionally, the first terminal device is also used to obtain an authentication request sent by the acquisition device and parse the authentication request to authenticate the physical coding information of the acquisition device according to a preset device table, the authentication request includes the physical coding information of the acquisition device, and the preset device table includes a preset acquisition device code; if the authentication is successful, a response message is sent to the acquisition device.

[0021] Optionally, the authentication request is ciphertext data generated based on character string encryption, and the ciphertext data is encoded using base64 encoding.

[0022] Optionally, the response message adopts a Json string format;

[0023] The response message includes a response field for adding a status request.

[0024] A second aspect of the present invention provides a face recognition method, comprising:

[0025] The first terminal device acquires image data and / or video data from the acquisition device and sends it to the second server;

[0026] The second server receives the image data and / or video data from the first terminal device;

[0027] The second server calls the trained model in the first server to perform face detection on the image data and / or video data to obtain a face image;

[0028] The second server calls the trained model in the first server to recognize the facial image to obtain the facial attribute and the first facial feature corresponding to the facial image;

[0029] The second server calls the trained model in the first server, classifies the facial image according to the first facial feature and stores it in a facial library, which includes the facial image, its first facial feature and a corresponding unique identification code.

[0030] Optionally, the second server calls the trained model in the first server to perform face detection on the image data and / or video data, and obtaining the face image further includes:

[0031] Detecting a face area in the image data and / or video data;

[0032] Performing facial key point recognition on the detected face area to determine the facial key points of the face area;

[0033] Based on the facial key points, determine whether the quality of the facial image in the facial area is qualified.

[0034] Optionally, detecting a face area in the image data and / or video data further comprises:

[0035] Performing face detection on the image data and / or video data of the current frame to obtain a plurality of face position frames and a confidence level corresponding to each face position frame;

[0036] Remove face location frames with confidence values ​​below a preset threshold;

[0037] Acquire multiple predicted face position frames obtained by predicting the face position frames based on the image data and / or video data of the current frame;

[0038] Performing face detection on the image data and / or video data of the next frame to obtain multiple face position frames;

[0039] A plurality of face position frames detected based on the image data and / or video data of the next frame are matched with the plurality of predicted face position frames, and the position of the face position frame of the next frame is updated.

[0040] Optionally, detecting a face area in the image data and / or video data further comprises:

[0041] Corresponding to each face position frame creation and first storage;

[0042] storing the corresponding face position frames and position information thereof in subsequent continuous multiple frames of image data and / or video data in the first memory;

[0043] Based on a plurality of face position frames and position information thereof in the first memory, determining an interaction ratio between face position frames in adjacent frame image data and / or video data in the first memory;

[0044] It is determined whether the plurality of interaction ratios are all greater than a preset threshold.

[0045] Optionally, the second server calls the trained model in the first server to classify the face image according to the first face feature and stores it in the face library, further comprising:

[0046] Calculating the similarity between the first facial feature of the facial image and the first facial feature of the facial images in the face library;

[0047] Based on the similarity calculation result, determining whether the first facial feature of the facial image is successfully matched in the face database;

[0048] If the match is successful, the facial image and its first facial feature, the corresponding facial position frame, and the corresponding image data and / or video data are stored in the unique identification code corresponding to the successfully matched facial image in the face library;

[0049] If the match is unsuccessful, a unique identification code is created corresponding to the facial image, and the facial image and its first facial feature, the corresponding facial position frame, and the corresponding image data and / or video data are stored in the unique identification code;

[0050] Display the face position frame of the face image and its unique identification code as well as the corresponding image data and / or video data.

[0051] Optionally, the face position frame and its unique identification code for displaying the face image and the corresponding image data and / or video data further include:

[0052] Create a second memory corresponding to each face position frame;

[0053] storing the corresponding face position frames and position information thereof in the subsequent continuous multiple frames of image data and / or video data in the second memory;

[0054] Determine, based on the plurality of face position frames and position information thereof in the first memory, an interaction ratio between face position frames in adjacent frame image data and / or video data in the second memory;

[0055] It is determined whether the plurality of interaction ratios are all greater than a preset threshold.

[0056] Optionally, the method further comprises:

[0057] The second server receives the image to be recognized from the second terminal device;

[0058] The second server calls the trained model in the first server to identify the image to be identified, so as to obtain the facial attribute and the second facial feature corresponding to the image to be identified;

[0059] The second server calls the trained model in the first server to calculate the similarity between the second facial feature of the image to be recognized and the first facial feature of the facial image in the face library;

[0060] Based on the similarity calculation result, determining whether the second facial feature of the image to be identified is successfully matched in the face database;

[0061] If the match is successful, the unique identification code corresponding to the first facial feature of the successful match, as well as the corresponding facial image, facial position frame and corresponding video frame image are displayed;

[0062] If the match is unsuccessful, the second server creates a specific identification code corresponding to the image to be identified, and stores the image to be identified and its second facial feature in the specific identification code;

[0063] The second facial feature of the image to be identified in the specific identification code is compared with the first facial feature in the image data and / or video data subsequently received from the first terminal device.

[0064] The beneficial effects of the present invention are as follows:

[0065] In view of the technical problems existing in the prior art, the embodiments of the present invention provide a face recognition system and a face recognition method. The face recognition system of this embodiment can classify the detected face images and store them in a face library, so as to perform rapid face recognition comparison and classification through its own face library, thereby improving the recognition efficiency of face recognition, and facilitating later data statistics and face query, and can achieve a high level in terms of accuracy and real-time performance, fully meeting the needs of face control analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The specific implementation modes of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0067] Figure 1 A diagram showing an exemplary system architecture in which one embodiment of the present invention may be applied.

[0068] Figure 2 A schematic structural diagram of a face recognition system in one embodiment of the present invention is shown.

[0069] Figure 3 A flowchart showing a face recognition system according to an embodiment of the present invention is shown.

[0070] Figure 4A flow chart showing a face recognition method according to an embodiment of the present invention.

[0071] Figure 5 An algorithm flow chart of a face recognition method according to an embodiment of the present invention is shown.

[0072] Figure 6 A diagram showing an application scenario of a face recognition system according to an embodiment of the present invention.

[0073] Figure 7 A schematic diagram showing the structure of a computer system implementing a server provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0074] In order to more clearly illustrate the present invention, the present invention is further described below in conjunction with the embodiments and the accompanying drawings. Similar components in the accompanying drawings are represented by the same reference numerals. It should be understood by those skilled in the art that the content specifically described below is illustrative rather than restrictive, and the protection scope of the present invention should not be limited thereto.

[0075] It should be noted that, although the terms "first", "second", etc. may be used herein to describe various parts, components, elements, regions, layers and / or parts, these parts, components, elements, regions, layers and / or parts should not be limited by these terms. Instead, these terms are used to distinguish one part, component, element, region, layer and / or part from another. Thus, for example, the first part, first member, first element, first region, first layer and / or first part discussed below may be referred to as the second part, second member, second element, second region, second layer and / or second part without departing from the teachings of the present invention.

[0076] Figure 1 An exemplary system architecture 100 is shown to which the face recognition method and apparatus of the present invention can be applied.

[0077] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, networks 104, 106, and servers 105, 107. The networks 104, 106 are used to provide a medium for communication links between the terminal devices 101, 102, 103 and the servers 105, 107. The networks 104, 106 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0078] Users can use terminal devices 101, 102, 103 to interact with servers 105, 107 through networks 104, 107 to receive or send messages, etc. For example, servers 105 and 107 can be accessed through application clients or web page access provided on terminal devices 101, 102, 103. Various communication client applications can be installed on terminal devices 101, 102, 103, such as face recognition applications, face control applications, web browser applications, shopping applications with face recognition functions, search applications, instant messaging tools, email clients, social platform software, and financial software.

[0079] Terminal devices 101, 102, 103 can be hardware or software. When terminal devices 101, 102, 103 are hardware, they can be various electronic devices with display screens that can obtain image data and / or video data from acquisition devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made here.

[0080] Servers 105 and 107 can be servers that provide various services, such as CPUs, GPUs, edge computing devices, cloud servers, or AI servers. The servers can also include a face library for storing facial images, corresponding first facial features, and corresponding unique identification codes, or a model warehouse for storing trained models.

[0081] In addition, the servers 105 and 107 may also be background servers that support face recognition applications on the terminal devices 101, 102, and 103. The background server may call a trained model in the background server to recognize the face image in the image data and / or video data according to the image data and / or video data provided by the user, so as to obtain the face attribute and the first face feature corresponding to the face image. The background server may also classify the face image according to the first face feature obtained by recognition and store it in the face library.

[0082] It should be noted that the face recognition method provided in the embodiment of the present invention is generally executed by the server 107 , and accordingly, the device for face recognition is generally set in the server 107 .

[0083] It should be noted that the servers 105 and 107 can be hardware or software. When the servers 105 and 107 are hardware, they can be implemented as a distributed server cluster consisting of multiple servers, or as a single server, or as a cloud server; when the server is software, it can be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made here.

[0084] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any suitable number of terminal devices, networks and servers.

[0085] In order to solve the technical problems in the prior art, an embodiment of the present invention provides a face recognition system, such as Figure 2-3 As shown, the face recognition system 400 includes a collection device 401, a first terminal device 402, a first server 403 and a second server 404; wherein the collection device 401 is configured to collect image data and / or video data; the first terminal device 402 is configured to receive image data and / or video data from the collection device 401 and send it to the second server 404; the second server 404 is configured to receive image data and / or video data from the first terminal device; calling the trained model in the first server 403 to perform face detection on the image data and / or video data to obtain a face image; calling the trained model in the first server 403 to identify the face image to obtain the face attribute and the first face feature corresponding to the face image; classifying the face image according to the first face feature and storing it in a face library, and the face library includes the face image and its first face feature and the corresponding unique identification code.

[0086] The face recognition method of this embodiment classifies the detected face images and stores them in a face library, so that fast face recognition comparison and classification can be performed through its own face library, thereby improving the recognition efficiency of face recognition. It is also convenient for subsequent data statistics and face queries, and can achieve a high level in accuracy and real-time performance, fully meeting the needs of face control analysis.

[0087] In a possible implementation, the first server 403 is a cloud server including a model warehouse for storing trained models. In a specific example, the first server 403 includes but is not limited to a CPU, a GPU, and an edge computing device.

[0088] In a specific implementation, the first server 403 may be constructed by the following steps:

[0089] a. Create a virtual device (such as a virtual machine) in the open physical space provided by the cloud disk, that is, a complete computer system with complete hardware system functions running in a completely isolated environment is simulated by software, so as to complete the work that can be completed in the driver's computer device in the virtual device. The virtual device can be, for example, a Linux virtual machine.

[0090] b. Create an image repository in the virtual device to store the trained model, where the model can be, for example, a CNN-based deep neural network model for face position frame detection, face feature extraction, face feature comparison, and face attribute recognition.

[0091] In a possible implementation, the second server 404 is an AI server.

[0092] In a specific example, face algorithm function points such as face detection, face key point detection, face position frame prediction, face quality recognition, face posture recognition, face occlusion recognition, face blur recognition, face glasses recognition, face beard recognition, face age recognition, face gender recognition, face mask recognition, first face feature recognition, face attribute recognition and subsequent face feature comparison are independently encapsulated into the underlying toolkit (Software Development Kit, SDK) for the upper algorithm interface (Application Programming Interface, API) of the second server 404 to call, so that the user can arbitrarily combine the required face recognition algorithm function in the form of function point switches, thereby effectively improving the performance of the algorithm and the layout efficiency of the face recognition algorithm. In another specific example, the models corresponding to the above face algorithm function points all use CNN-based detection and recognition models, and for each face algorithm function point, the best performing deep neural network model is selected for prediction, so as to achieve the best accuracy.

[0093] In one possible implementation, the first server 403 is configured to receive a trained model and store it in a model warehouse; import the trained model in the model warehouse into the second server 404; send a request to create face recognition to the second server 404, and receive a face recognition interface from the second server 404.

[0094] In a specific example, the trained model is deployed in the first server 403, and the model is used to perform online reasoning services, which includes the following steps:

[0095] 1. Prepare model files and perform training. The model files can be based on mainstream deep learning frameworks such as TensorFlow, Pytorch, and Caffe.

[0096] In a specific example, based on the different functions of the model, different loss functions can be used for training. For example, for a model used for face recognition, the loss function may be a pairwise loss function, a triplet loss function, or a quadruplet loss function, thereby achieving one-time face recognition of the model.

[0097] 2. Upload the model file to a model repository of a specified specification in the first server 403;

[0098] 3. Import the model file in the first server 403 into the second server 404 , for example, import it into the face recognition service model space in the second server 404 .

[0099] In a specific example, the model file in the first server 403 is output to the second server 404 by mounting the network disk, thereby reducing the difficulty of data processing on the cloud server, ensuring that the model file is quickly transmitted between the first server 403 and the second server 404, and enabling the second server 404 to call the model in the first server 403 in real time for face recognition. In another specific example, in a specific example, the model in the face recognition service model space can correspond to the openVINO optimization acceleration and TensorRT optimization acceleration of the CPU, forming different versions of model files for different face recognition scenarios.

[0100] 4. The first server 403 sends a request to create face recognition to the second server 404, for example, by sending a request to the second server 404 via an http request. The face recognition request may be, for example, to create a CPU / GPU model inference service. The second server 404 receives the request to create face recognition and feeds back a face recognition interface, such as a face recognition API, to the first server 403, indicating that the face recognition service status of the model is "available".

[0101] 5. Through the face recognition API fed back to the first server 403 by the second server 404, the second server 404 can call multiple trained models in the first server 403 to form the required face recognition algorithm for face recognition through any combination.

[0102] This implementation method sets up a first server 403 and a second server 404, wherein the first server 403 is used to store trained models of multiple different functions, and the second server 404 uses the face recognition API to call the trained model in the first server 403 through HTTPgRPC or Rgb / yuv / nv12 to implement elastic reasoning services. By decoupling the deep neural network model reasoning and algorithm business, compared with the strong correlation of traditional deep neural network algorithm deployment models, this implementation method has the advantages of ease of use, high availability, easy management, and health management, so that users do not need to consider the infrastructure, but only need to pay attention to the model and reasoning service. At the same time, it has its own high availability guarantee and fault tolerance mechanism, and can complete the upgrade of the deep neural network with one click. It supports CPU / GPU scalability and distributed load balancing, which can effectively improve the deployment efficiency of AI and lower the deployment threshold of AI.

[0103] In a possible implementation, the first terminal device 402 is also used to obtain the authentication request sent by the collection device 401 and parse the authentication request to authenticate the physical coding information of the collection device 401 according to the preset device table, the authentication request includes the physical coding information of the collection device 401, and the preset device table includes the preset collection device code; if the authentication is successful, a response message is sent to the collection device 401.

[0104] In a specific example, the first terminal device 402 obtains an authentication request sent by at least one collection device 401, parses the authentication request to authenticate the physical coding information of the collection device 401 according to the preset device table; when the physical coding information of the collection device 401 matches the preset collection device coding, it is determined that the collection device 401 is authenticated, and a response message is sent to the collection device 401.

[0105] When the physical coding information of the collection device 401 does not match all the preset collection device codes in the preset device table and the total number of preset collection device codes in the preset device table does not reach the threshold, in response to the operation of adding the device table, the physical coding information of the collection device 401 is added to the preset device table and it is determined that the collection device 401 has passed the authentication.

[0106] When the physical coding information of the collection device does not match all the preset collection device codes in the preset device table and the total number of preset collection device codes in the preset device table reaches a threshold, it is determined that the collection device 401 authentication fails.

[0107] This implementation method receives and parses the authentication request sent by the acquisition device 401, and authenticates the acquisition device 401 according to the preset device table, thereby ensuring the security and reliability of communication between the acquisition device 401 and the first terminal device 402, while protecting the key modules of the face recognition system (such as face images and face features in the face library).

[0108] In a specific example, the physical coding information of the acquisition device 401 may be a hardware-specific number, an international mobile equipment identity code, a media access control address, etc. of the acquisition device 401, which is used to identify the device characteristics or uniqueness of the acquisition device 401. The acquisition device 401 may be a smart phone, a tablet computer, a personal computer, a teller machine, an entrance gate, a camera, etc.

[0109] In a specific embodiment, the preset acquisition device code in the preset device table can be set according to the physical coding information of the acquisition device. It should be noted that the number of preset acquisition device codes included in the preset device table can be fixed or not. In a specific example, the number of preset acquisition device codes included in the preset device table is fixed, that is, only a preset number of preset acquisition device codes can be added to the preset device table, and the preset number can be set according to parameters such as the application scenario of face recognition, the performance of the first terminal device, and user needs. For example, the number of preset acquisition device codes in the preset device table can be set to 100, 200, 300 or 500, etc., so as to keep the number of preset acquisition device codes relatively stable and maintain the security and stability of the first terminal device.

[0110] In a specific embodiment, when the first terminal device obtains authentication requests sent by multiple collection devices, distributed concurrent authentication is adopted. For example, 200 collection devices simultaneously send authentication requests to the first terminal device, and the first terminal device can simultaneously receive and process the authentication requests. Correspondingly, the first terminal device in concurrent mode can simultaneously respond to data processing requests of multiple authenticated collection devices, thereby improving the efficiency of collection device authentication, shortening the waiting time for authentication results, and optimizing the user experience.

[0111] In a possible implementation, the authentication request is ciphertext data generated based on character string encryption, and the ciphertext data is encoded using base64 encoding.

[0112] In a specific example, the data of the authentication request sent by the collection device 401 to the first terminal device 402 needs to be encrypted and encoded as a string and converted to base64 for transmission, thereby further ensuring the communication security between the collection device 401 and the first terminal device 402.

[0113] In a possible implementation, the response message adopts a Json string format, and the response message includes a response field for adding a status request.

[0114] In a specific example, the response message can be sent in the form of JavaScript Object Notation (JSON). In the Json string, the response field of the status request can be flexibly added to call different data to adapt to the multi-occasion application of device authentication, and the specific application method is not limited. For example, an instruction to request the call of data representing the gender of a person can be added, and the data representing the gender of a person can be called for service recommendations and other operations. This implementation method allows the authentication of the first terminal device 402 to take into account more application scenarios and expand the application scenarios of the authentication of the first terminal device 402.

[0115] In a possible implementation, the authentication request is sent via an Http Post request.

[0116] In a specific implementation, since Http Post will not be cached or saved in the log of the collection device 401, using Http Post to send the authentication request can effectively limit the number of access collection devices 401, and can ensure the communication security between the first terminal device 402 and the collection device 401. In addition, since Http Post can send a large amount of data and a large number of data types, sending the authentication request by means of Http Post request can not only ensure the communication security between the first terminal device 402 and the collection device 401, but also can transmit larger data and take into account more usage scenarios.

[0117] Continue to refer Figure 4-6 , Figure 4-6 A face recognition method according to an embodiment of the present invention is shown. The face recognition method can be applied to scenarios such as face control and person search. The face recognition method includes the following steps:

[0118] S201, a first terminal device obtains image data and / or video data from a collection device and sends the data to a second server 404;

[0119] S202, the second server 404 receives image data and / or video data from the first terminal device;

[0120] In this embodiment, the second server 404 can directly receive the image data and / or video data from the first terminal device through a local or remote network, or can receive it through the first server 403. The acquisition device can be, for example, a camera, and the image data and / or video data can be, for example, an image or video taken by the camera. The first terminal device can select from multiple acquisition devices to obtain image data and / or video data of a single or multiple acquisition devices. The second server 404 can remotely retrieve the image data and / or video data obtained by the first terminal device.

[0121] S203: The second server 404 calls the trained model in the first server 403 to perform face detection on the image data and / or video data to obtain a face image. In a specific example, the face image includes a face position frame, position information, and category.

[0122] In a possible implementation, in step S203, the second server 404 calls the trained model in the first server 403 to perform face detection on the image data and / or video data to obtain the face image further includes:

[0123] S2031, detecting a face area in the image data and / or video data;

[0124] In a possible implementation, step S2031 of detecting a face region in the image data and / or video data further includes:

[0125] S20311. Perform face detection on the image data and / or video data of the current frame to obtain multiple face position frames and the confidence level corresponding to each face position frame.

[0126] In a specific embodiment, the second server 404 may set a plurality of sliding windows and step sizes of different preset sizes in advance, and slide the image data and / or video data of the current frame from left to right and from top to bottom to obtain a plurality of local image frames, and respectively input the obtained multiple local image frames into the trained deep neural network for prediction, so as to obtain a plurality of face position frames in the plurality of local image frames and the corresponding confidence levels.

[0127] In a specific implementation, before the image data and / or video data of the current frame is subjected to face detection, the second server 404 first pre-processes the image data and / or video data of the current frame, for example, the image data and / or video data of the current frame are divided into positive direction squares of a certain size, and then the pre-processed image data and / or video data are directly placed in the trained deep neural network for prediction, that is, a single convolution is performed simultaneously in multiple squares, and multiple face position frames and their corresponding confidences are directly output in the output layer. In the single convolution process, there are many shared calculation steps, and in the prediction process, many steps are shared, so that the prediction speed is very fast and real-time recognition can be achieved. This implementation only requires one convolution to achieve the output of multiple face position frames, effectively improving the search efficiency of the face position frame, without repeated attempts, and without considering the aspect ratio of the face position frame, and is suitable for situations with high real-time requirements.

[0128] S20312, removing the face position frame with a confidence level lower than a preset threshold;

[0129] In a specific implementation, the second server 404 may gradually increase the confidence threshold of the detected face position frame, for example, the confidence may be gradually increased from 0.7 to 0.9. For example, after removing the face position frame with a confidence less than 0.7, the face position frame with a confidence less than 0.8 is removed, and finally the face position frame with a confidence less than 0.9 is removed until the recall rate of face angle detection reaches a higher level (for example, greater than 0.99).

[0130] In a specific implementation, the second server 404 first removes the face location frames that are lower than a preset confidence threshold (such as 0.7) for each face location frame, performs a loop operation on the remaining face location frames, selects the face location frame corresponding to the maximum confidence for output, removes the face location frame whose interaction ratio (IoU, Interstction Over Union) with the face location frame is greater than a preset threshold (such as 0.5), and repeats the operation until all face location frames have been output or removed.

[0131] S20313, obtaining a plurality of predicted face position frames obtained by predicting the face position frames based on the image data and / or video data of the current frame;

[0132] In a specific implementation, the second server 404 calls a lightweight and fast model (such as Faster-R-CNN) in the first server 403 to perform face position frame tracking prediction. Wherein, corresponding to each different face position frame prediction, a corresponding predicted face position frame is obtained. In a specific example, the face position frame can be predicted by combining a linear velocity model and a Kalman filter.

[0133] S20314, performing face detection on the image data and / or video data of the next frame to obtain multiple face position frames;

[0134] In a specific example, the second server 404 may, for example, obtain multiple face position frames of the image data and / or video data of the next frame by sliding in advance using multiple sliding windows of different preset sizes and step sizes as in the above embodiment; in another specific example, the second server 404 may also obtain multiple face position frames by dividing the image data and / or video data of the current frame into square grids of the same size and directly putting them into a trained deep neural network for prediction as in the above embodiment.

[0135] S20315. Match multiple face position frames detected based on the image data and / or video data of the next frame with multiple predicted face position frames, and update the position of the face position frame of the next frame.

[0136] In a specific implementation, the second server 404 uses the Hungarian assignment algorithm to match the face position frame of the next frame with the predicted face position frame, thereby updating the position of the face position frame of the next frame. The cost matrix in the Hungarian assignment algorithm is the Interaction Over Union (IoU) between the predicted face position frame and the face position frame of the next frame, that is, the Interaction Over Union is used as the weight of the Hungarian assignment algorithm.

[0137] Compared with the two-stage matching technical solution in the prior art, this implementation can achieve matching through only a one-stage method, and can also realize online real-time tracking of the face position frame to ensure the detection accuracy and speed of the face position frame. At the same time, by setting the interaction ratio as the weight of the algorithm, the problem of short-sighted occlusion of the face position frame can be effectively solved.

[0138] In a possible implementation, step S2031 of detecting a face region in the image data and / or video data further includes:

[0139] S20316, creating a first memory corresponding to each face position frame;

[0140] S20317, storing the corresponding face position frames and position information in the subsequent multiple consecutive frames of image data and / or video data in the first memory;

[0141] S20318, determining an interaction ratio between face position frames in adjacent frame image data and / or video data in the first memory based on a plurality of face position frames and position information thereof in the first memory;

[0142] S20319: Determine whether multiple interaction ratios are all greater than a preset threshold.

[0143] In a specific embodiment, the second server 404 creates a corresponding first memory corresponding to each face position frame, and the first memory is used to store face position frames and their position information of multiple consecutive frames, that is, while performing face detection on multiple consecutive frames of image data and / or video data, the corresponding face position frames and their position information obtained by detection are stored in the same first memory. When a certain amount of face position frames (for example, 3 frames, 5 frames) are stored, the interaction ratio between the face position frames of adjacent frames is calculated, for example, the interaction ratio between the face position frames of the first frame and the second frame, the interaction ratio between the face position frames of the second frame and the third frame, the interaction ratio between the face position frames of the third frame and the fourth frame, and the interaction ratio between the face position frames of the fourth frame and the fifth frame are calculated, and the multiple interaction ratios are compared with preset thresholds respectively. If the multiple interaction ratios are all greater than the preset threshold, that is, the overlap degree of the face position frames of adjacent frames meets the preset requirement, it indicates that the tracking detection of the face position frame is accurate, and the first memory is removed; if the multiple interaction ratios are less than the preset threshold, the first memory is removed, and a new first memory is created corresponding to the face position frame of the latest frame, and the above operation is repeated.

[0144] This implementation method continuously tracks and detects the face position frame, which can effectively ensure the continuous and effective tracking of the face position frame, prevent the face position frame from being lost, further improve the detection efficiency, detection performance and detection accuracy of the face position frame, and achieve an excellent balance between the recall rate and detection accuracy of the same face position frame. In addition, by setting the first memory to store the face position frames of different frames, there is no need to repeatedly identify the face position frames of multiple consecutive frames for interactive ratio calculation, thereby achieving real-time detection and real-time correction.

[0145] S2032, performing facial key point recognition on the detected face area to determine the facial key points of the face area;

[0146] In a specific embodiment, the second server 404 may, for example, perform facial key point recognition on the detected face area, where the facial key points may be, for example, the center of the two eyes, the left and right corners of the mouth, and the tip of the nose, etc. In a specific example, the facial key points may be recognized by outputting the (x, y) coordinates of the facial key points through a deep neural network model.

[0147] S2033. Determine whether the quality of the facial image in the facial area is qualified based on the facial key points.

[0148] In a specific embodiment, the second server 404 uses a reflection transformation algorithm to perform face alignment based on the extracted facial key points (centers of two eyes, left and right corners of the mouth, and nose tip), and normalizes to the front face to calculate a standard front face facial image. Then, it is determined whether the quality of the obtained standard front face facial image is qualified. If the quality threshold is met, the facial image is recognized to extract facial attributes and the first facial feature.

[0149] In a possible implementation, after determining whether the quality of the face image in the face area is qualified based on the face key points in step S2033, the method further includes:

[0150] S2034, performing posture detection, occlusion recognition, and blur recognition on the face image in the face area.

[0151] In a specific embodiment, the second server 404 performs posture detection, occlusion recognition, and blur recognition on the facial image of the facial area through the corresponding deep neural network model, thereby ensuring that the face in the facial image is clear and unobstructed, and ensuring the accuracy of the facial attributes and the first facial features extracted subsequently.

[0152] S204, the second server 404 calls the trained model in the first server 403 to recognize the face image, so as to obtain the face attribute and the first face feature corresponding to the face image;

[0153] In a specific embodiment, the second server 404 can identify the face image based on CNN (Convolutional Neural Networks) to obtain the face attributes corresponding to the face image. In a specific example, the face attributes are face structured information, such as whether to wear glasses, whether to have a beard, age, gender, etc. At the same time, the second server 404 visualizes the extracted face attributes.

[0154] In a specific embodiment, the second server 404 calculates a standard frontal face image based on the CNN model according to the facial key points (the centers of the two eyes, the left and right corners of the mouth, and the tip of the nose), and extracts 512-dimensional data from the facial image as the first facial feature.

[0155] S205. The second server 404 calls the trained model in the first server 403, classifies the facial image according to the first facial feature and stores it in a facial library, which includes the facial image, its first facial feature and the corresponding unique identification code.

[0156] In a possible implementation, step S205 in which the second server 404 calls the trained model in the first server 403 to classify the face image according to the first face feature and stores it in the face library further includes:

[0157] S2051, calculating the similarity between the first facial feature of the facial image and the first facial feature of the facial image in the face library;

[0158] S2052. Based on the similarity calculation result, determine whether the first facial feature of the facial image is successfully matched in the face library; if the match is successful, store the facial image and its first facial feature, the corresponding face position frame, and the corresponding image data and / or video data in the unique identification code corresponding to the successfully matched facial image in the face library; if the match is unsuccessful, create a unique identification code for the corresponding facial image, and store the facial image and its first facial feature, the corresponding face position frame, and the corresponding image data and / or video data in the unique identification code.

[0159] In a specific embodiment, the second server 404 pre-adopts a loss function such as a pairwise loss function, a triplet loss function, or a quadruplet loss function to train a model for facial feature comparison, and inputs the first facial feature of the facial image and the first facial feature of the candidate facial image in the face library into the trained model, thereby outputting the difference value between the two, that is, the similarity calculation result.

[0160] If the difference between the two is small enough, it can be determined that the facial image matches the candidate facial images in the face library successfully. The facial image and its first facial feature extracted from the image data and / or video data, the corresponding face position frame and the corresponding image data and / or video data are uniformly stored in the unique identification code corresponding to the successfully matched facial image to expand the facial image corresponding to the unique identification code. If the difference between the two is large, it is determined that the facial image matches the candidate facial images in the face library unsuccessfully. When the facial image and the facial images in the face library both fail to match successfully, the second server 404 registers the corresponding unique identification code corresponding to the facial image in the face library, and stores the facial image and its first facial feature extracted from the image data and / or video data, the corresponding face position frame and the corresponding image data and / or video data in the newly created unique identification code, so that it can be used as a face database to facilitate subsequent face queries or trajectory queries.

[0161] S2053, displaying the face position frame of the face image and its unique identification code and the corresponding image data and / or video data.

[0162] In a specific embodiment, the second server 404 sends the face position frame, its position information, and the corresponding unique identification code to the third terminal device for storage for subsequent big data storage, statistics, query or alarm. Then the third terminal device sends the face position frame, its position information, and the corresponding unique identification code to the visual display device for display. At the same time, the second server 404 transmits the corresponding image data and / or video data to the fourth terminal device for storage and encoding. The fourth terminal device transmits the corresponding image data and / or video data to the visual display device for display.

[0163] In a possible implementation, step S2053 of displaying the face position frame of the face image and its unique identification code and the corresponding image data and / or video data further includes:

[0164] S20531, creating a second memory corresponding to each face position frame;

[0165] S20532, storing the corresponding face position frame and position information in the subsequent continuous multiple frames of image data and / or video data in the second memory;

[0166] S20533, determining an interaction ratio between face position frames in adjacent frame image data and / or video data in the second memory based on a plurality of face position frames and position information thereof in the second memory;

[0167] S20534. Determine whether multiple interaction ratios are all greater than a preset threshold.

[0168] In a specific embodiment, the second server 404 creates a corresponding second memory corresponding to each visually displayed face position frame, and the second memory is used to store the position information of the face position frame in the next multiple consecutive frames, that is, when the second server 404 performs face detection on the next multiple consecutive frames of image data and / or video data, the corresponding face position frame and its position information obtained by the detection are stored in the same second memory. When a certain number of face position frames (for example, 3 frames, 5 frames) are stored, the interaction ratio between the face position frames of adjacent frames is calculated, for example, the interaction ratio between the face position frames of the first frame and the second frame is calculated. The interaction ratios are compared with each other, the interaction ratio between the face position frames of the second frame and the third frame, the interaction ratio between the face position frames of the third frame and the fourth frame, and the interaction ratio between the face position frames of the fourth frame and the fifth frame, and the multiple interaction ratios are compared with the preset thresholds respectively. If the multiple interaction ratios are all greater than the preset threshold, that is, the overlap degree of the face position frames of adjacent frames meets the preset requirements, it indicates that the tracking detection of the face position frame is accurate and there is no missing tracking, then the second memory is removed; if the multiple interaction ratios are less than the preset threshold, the first memory is removed, and a new second memory is created corresponding to the face position frame of the latest frame, and the above operation is repeated.

[0169] This implementation method continuously tracks and detects the face position frame, which can effectively ensure the continuous and effective tracking of the face position frame, prevent the face position frame from being missed, further improve the detection efficiency, detection performance and detection accuracy of the face position frame, and achieve an excellent balance between the recall rate and detection accuracy of the same face position frame. In addition, by setting the second memory to store the face position frames of different frames, there is no need to repeatedly identify the face position frames of multiple consecutive frames for interactive ratio calculation, thereby achieving real-time detection and real-time correction.

[0170] In a possible implementation, the method further includes:

[0171] S206, the second server 404 receives the image to be recognized from the second terminal device;

[0172] S207, the second server 404 calls the trained model in the first server 403 to recognize the image to be recognized, so as to obtain the facial attributes and second facial features corresponding to the image to be recognized;

[0173] In a specific embodiment, the second server 404 can identify the image to be identified based on CNN (Convolutional Neural Networks) to obtain the facial attributes corresponding to the image to be identified. In a specific example, the facial attributes are structured facial information, such as whether the person wears glasses, whether he has a beard, age, gender, etc. At the same time, the second server 404 visualizes the extracted facial attributes.

[0174] In a specific embodiment, the second server 404 extracts 512-dimensional data from the image to be identified as the second facial feature based on a CNN model.

[0175] S208, the second server 404 calls the trained model in the first server 403 to calculate the similarity between the second facial feature of the image to be recognized and the first facial feature of the facial image in the face database;

[0176] S209, based on the similarity calculation result, determine whether the second facial feature of the image to be identified is successfully matched in the face database; if the match is successful, display the unique identification code corresponding to the first facial feature that successfully matched, as well as the corresponding face image, face position frame, and corresponding video frame image; if the match is unsuccessful, create a specific identification code corresponding to the image to be identified, and store the image to be identified and its second facial feature in the specific identification code;

[0177] In a specific implementation, the second server 404 may, as described in the above embodiments, pre-train a model for facial feature comparison using a loss function such as a pairwise loss function, a triplet loss function, or a quadruplet loss function, and input the second facial features of the image to be identified and the first facial features of the candidate facial images in the face library into the trained model, thereby outputting the difference between the two, i.e., the similarity calculation result.

[0178] If the difference between the two is small enough, it can be determined that the image to be identified is successfully matched with the candidate face images in the face library, and all face images under the unique identification code corresponding to the first face feature that successfully matches, the corresponding image data and / or video data and the face position frame are displayed in a certain order.

[0179] If the difference between the two is large, it is determined that the facial image fails to match the candidate facial images in the face library. When the facial image fails to match the facial images in the face library, the second server 404 registers the corresponding specific identification code for the facial image in the face library, and stores the facial image and its second facial feature extracted from the image to be identified, the corresponding face position box and the corresponding image to be identified in the newly created specific identification code, so as to facilitate comparison with the facial image and its first facial feature obtained later.

[0180] S210: Compare the second facial feature of the image to be identified in the specific identification code with the first facial feature in the image data and / or video data subsequently received from the first terminal device.

[0181] In a specific implementation, in an application scenario of searching for a person, the second server 404 receives the image to be identified of the person, and compares the second facial feature of the image to be identified with the first facial feature of the candidate facial image in the face library.

[0182] If the confidence after comparison is greater than the preset confidence threshold (for example, 0.9), the comparison is considered successful, and a unique identification code for successful identification is output. All face images corresponding to the person in the face library, the corresponding image data and / or video data, and the face position frame are sorted and displayed according to the acquisition time of the image data and / or video data, so as to facilitate understanding of the person's movement trajectory and facilitate rapid finding of the person.

[0183] If the comparison is unsuccessful, that is, the person is not retrieved in the current face database, the second server 404 registers the face of the image to be identified in the face database, creates a specific identification code, and stores the face image extracted from the image to be identified and its second face feature, the corresponding face position frame and the corresponding image to be identified in the newly created specific identification code. After the second server 404 subsequently receives the image data and / or video data from the first terminal device and detects the face image and the first face feature, it compares the subsequently identified face image and the first face feature with the image to be identified and its second face feature in the specific identification code one by one.

[0184] This implementation saves the process of identifying the image to be identified multiple times and only requires one identification of the image to be identified, which greatly saves the time of identifying the image to be identified and effectively improves the efficiency and speed of face recognition.

[0185] In a possible implementation, the method further includes:

[0186] S206, the second server 404 collects an image of the identity document and an image to be identified of the person holding the identity document;

[0187] S207, the second server 404 calls the trained model in the first server 403 to obtain the third facial feature on the identity document from the collected image of the identity document;

[0188] S208: The second server 404 calls the trained model in the first server 403 to recognize the image to be recognized, so as to obtain the facial attributes and the second facial features corresponding to the image to be recognized.

[0189] S209, the second server 404 calls the trained model in the first server 403 to calculate the similarity between the second facial feature of the image to be recognized and the third facial feature on the identity document;

[0190] S210: Based on the similarity calculation result, the identity of the holder of the identity document is authenticated.

[0191] See also Figure 6 , Figure 6 FIG. 1 is a schematic diagram of an application scenario of a face recognition method according to an embodiment of the present invention. Figure 6In the application scenario shown, the acquisition device 501 acquires image data and / or video data, and sends an authentication request to the first terminal device 502. The first terminal device 502 encodes the image data and / or video data transmitted by the acquisition device and confirms its authentication request. After successful authentication, the acquisition device 501 sends the acquired image data and / or video data to the first terminal device 502. The first terminal device 502 starts the AI ​​cloud inference service and runs the face recognition interface, thereby accessing the cloud-based AI inference server 503 through HTTP gRPC or Rgb / yuv / nv12, etc. The cloud-based AI inference server 503 calls the trained model for face recognition to extract the corresponding face image and its face attributes and the first face feature, outputs the face attributes to the visualization display device 505 for display, and classifies the face image according to the first face feature and stores it in the face library of the cloud-based AI inference server 503. The corresponding face position frame, its position information and the corresponding unique identification code are then sent to the third terminal device 504 for storage for subsequent big data storage, statistics, query or alarm. After that, the third terminal device 504 sends the face position frame, its position information and the corresponding unique identification code to the visual display device 505 for display. At the same time, the corresponding image data and / or video data are transmitted to the fourth terminal device 506 for storage and encoding. The fourth terminal device 506 transmits the corresponding image data and / or video data to the visual display device 505 for display.

[0192] In addition, in Figure 6 In the application scenario shown, the second terminal device 507 collects the image to be identified and sends an authentication request to the first terminal device 502. The first terminal device 502 encodes the image to be identified transmitted by the second terminal device 507 and confirms its authentication request. After successful authentication, the first terminal device 502 sends the acquired image to be identified to the cloud AI reasoning server 503 (including the first server and the second server). The cloud AI reasoning server 503 calls the trained model to perform face recognition to extract the corresponding face image and its face attributes and second face features, outputs the face attributes to the visual display device 505 for display, and compares the second face feature with the first face feature of the candidate face image in the face library. If the match is successful, the unique identification code corresponding to the first face feature that successfully matches, as well as the corresponding face image, face position frame and corresponding video frame image are output to the visual display device 505 for display. The second terminal device 507 can be, for example, a smart phone, a tablet computer, a personal computer, a teller machine, an entrance gate, a camera and other devices.

[0193] This implementation method builds an efficient elastic cloud inference service and designs a face recognition algorithm process to perform face detection, feature calculation and feature comparison, thereby effectively performing distributed authentication on multiple acquisition devices. Regardless of whether it is publicly or privately deployed, the face recognition system can ultimately achieve a high level of accuracy and real-time performance to meet the needs of face control and analysis.

[0194] like Figure 7 As shown, a computer device suitable for implementing the second server in the face recognition system provided by the above embodiment includes a central processing module (CPU), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage portion into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer system are also stored. The CPU, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0195] The following components are connected to the I / O interface: an input part including a keyboard, a mouse, etc.; an output part including a liquid crystal display (LCD), etc. and a speaker, etc.; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a LAN card, a modem, etc. The communication part performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. Removable media, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., are installed on the drive as needed so that the computer program read therefrom is installed into the storage part as needed.

[0196] In particular, according to this embodiment, the process described in the flowchart above can be implemented as a computer software program. For example, this embodiment includes a computer program product, which includes a computer program tangibly contained on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium.

[0197] The flow chart and schematic diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product of the present embodiment. In this regard, each square box in the flow chart or schematic diagram can represent a part of a module, program segment or code, and a part of the above-mentioned module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the schematic diagram and / or flow chart, and the combination of the square boxes in the schematic diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0198] The modules involved in the description of this embodiment may be implemented by software or hardware. The modules described may also be arranged in a processor. The names of these modules do not constitute a limitation on the modules themselves in some cases.

[0199] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the embodiments here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. A face recognition system, characterized in that: include: A collection device, a first terminal device, a first server and a second server; A first terminal device is configured to receive image data and / or video data from the acquisition device and send the image data and / or video data to the second server; The second server is configured as: Receiving image data and / or video data from a first terminal device; Calling a trained model in the first server to perform face detection on the image data and / or video data to obtain a face image; Calling the trained model in the first server to recognize the facial image to obtain facial attributes and first facial features corresponding to the facial image; The face image is classified according to the first face feature and stored in a face library, which includes the face image, the first face feature and the corresponding unique identification code.

2. The system according to claim 1, characterized in that The first server is a cloud server including a model repository for storing trained models.

3. The system according to claim 2, characterized in that The first server is configured as: Receive the trained model and store it in the model warehouse; Importing the trained model in the model warehouse into the second server; A face recognition creation request is sent to the second server, and a face recognition interface is received from the second server.

4. The system according to claim 1, characterized in that The second server is an AI server.

5. The system according to claim 1, characterized in that The first terminal device is further used to obtain an authentication request sent by the acquisition device and parse the authentication request to authenticate the physical coding information of the acquisition device according to a preset device table, wherein the authentication request includes the physical coding information of the acquisition device, and the preset device table includes a preset acquisition device code; If the authentication is successful, a response message is sent to the acquisition device.

6. The system according to claim 5, characterized in that The authentication request is ciphertext data generated based on character string encryption, and the ciphertext data is encoded using base64 encoding.

7. The system according to claim 5, characterized in that The response message adopts Json string format; The response message includes a response field for adding a status request.

8. A face recognition method, characterized in that: include: The first terminal device acquires image data and / or video data from the acquisition device and sends it to the second server; The second server receives the image data and / or video data from the first terminal device; The second server calls the trained model in the first server to perform face detection on the image data and / or video data to obtain a face image; The second server calls the trained model in the first server to recognize the facial image to obtain the facial attribute and the first facial feature corresponding to the facial image; The second server calls the trained model in the first server, classifies the facial image according to the first facial feature and stores it in a facial library, which includes the facial image, its first facial feature and a corresponding unique identification code.

9. The method according to claim 8, characterized in that The second server calls the trained model in the first server to perform face detection on the image data and / or video data to obtain the face image further comprising: Detecting a face area in the image data and / or video data; Performing facial key point recognition on the detected face area to determine the facial key points of the face area; Based on the facial key points, determine whether the quality of the facial image in the facial area is qualified.

10. The method according to claim 9, characterized in that The detecting of a face area in the image data and / or video data further comprises: Performing face detection on the image data and / or video data of the current frame to obtain a plurality of face position frames and a confidence level corresponding to each face position frame; Remove face location frames with confidence values ​​below a preset threshold; Acquire multiple predicted face position frames obtained by predicting the face position frames based on the image data and / or video data of the current frame; Performing face detection on the image data and / or video data of the next frame to obtain multiple face position frames; A plurality of face position frames detected based on the image data and / or video data of the next frame are matched with the plurality of predicted face position frames, and the position of the face position frame of the next frame is updated.

11. The method according to claim 10, characterized in that The detecting of the face area in the image data and / or video data further comprises: Corresponding to each face position frame creation and first storage; storing the corresponding face position frames and position information thereof in subsequent continuous multiple frames of image data and / or video data in the first memory; Based on a plurality of face position frames and position information thereof in the first memory, determining an interaction ratio between face position frames in adjacent frame image data and / or video data in the first memory; It is determined whether the plurality of interaction ratios are all greater than a preset threshold.

12. The method according to claim 10, characterized in that The second server calls the trained model in the first server, and classifies the face image according to the first face feature and stores it in the face library, further comprising: Calculating the similarity between the first facial feature of the facial image and the first facial feature of the facial images in the face library; Based on the similarity calculation result, determining whether the first facial feature of the facial image is successfully matched in the face database; If the match is successful, the facial image and its first facial feature, the corresponding facial position frame, and the corresponding image data and / or video data are stored in the unique identification code corresponding to the successfully matched facial image in the face library; If the match is unsuccessful, a unique identification code is created corresponding to the facial image, and the facial image and its first facial feature, the corresponding facial position frame, and the corresponding image data and / or video data are stored in the unique identification code; Display the face position frame of the face image and its unique identification code as well as the corresponding image data and / or video data.

13. The method according to claim 12, characterized in that The face position frame and its unique identification code showing the face image and the corresponding image data and / or video data further include: Create a second memory corresponding to each face position frame; storing the corresponding face position frames and position information thereof in the subsequent continuous multiple frames of image data and / or video data in the second memory; Based on a plurality of face position frames and position information thereof in the second memory, determining an interaction ratio between face position frames in adjacent frame image data and / or video data in the second memory; It is determined whether the plurality of interaction ratios are all greater than a preset threshold.

14. The method according to claim 10, characterized in that The method further comprises: The second server receives the image to be recognized from the second terminal device; The second server calls the trained model in the first server to identify the image to be identified, so as to obtain the facial attribute and the second facial feature corresponding to the image to be identified; The second server calls the trained model in the first server to calculate the similarity between the second facial feature of the image to be recognized and the first facial feature of the facial image in the face library; Based on the similarity calculation result, determining whether the second facial feature of the image to be identified is successfully matched in the face database; If the match is successful, the unique identification code corresponding to the first facial feature of the successful match, as well as the corresponding facial image, facial position frame and corresponding video frame image are displayed; If the match is unsuccessful, the second server creates a specific identification code corresponding to the image to be identified, and stores the image to be identified and its second facial feature in the specific identification code; The second facial feature of the image to be identified in the specific identification code is compared with the first facial feature in the image data and / or video data subsequently received from the first terminal device.

Images