Identity identification method, device and system, electronic device and storage medium

Abstract

The present application relates to the technical field of computer vision, and provides an identity recognition method, device, system, electronic equipment and storage medium, wherein the method comprises: acquiring a video comprising a to-be-recognized object; performing face detection on each frame image in the video to obtain face patches in the each frame image; performing face authenticity detection on the video based on a time sequence relationship between each face patch and a position relationship between key parts in the each face patch to obtain a detection result; and in a case where the detection result is a real face, recognizing the identity of the to-be-recognized object based on the each face patch. The identity recognition method, device, system, electronic equipment and storage medium provided by the present application can effectively recognize the authenticity of faces in a video, can avoid misrecognition caused by the use of fake images, and can improve the accuracy of identity recognition.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to an identity recognition method, apparatus, system, electronic device, and storage medium. Background Technology

[0002] Pig identification is crucial in pig farming insurance claims. Related technologies primarily rely on ear tag information or pig facial recognition for identification. However, due to pigs' tendency to bite and fight, ear tags are easily detached, and they can also be replaced intentionally. While pig facial recognition solves the problem of detached or replaced ear tags, it is prone to misidentification, leading to duplicate or fraudulent claims.

[0003] Therefore, there is an urgent need for an identity verification method with higher accuracy to help the insurance industry accurately identify livestock. Summary of the Invention

[0004] This invention provides an identity recognition method, device, system, electronic device, and storage medium to address the shortcomings of existing technologies where pig face recognition is prone to misidentification, leading to duplicate and fraudulent claims.

[0005] This invention provides an identity recognition method, comprising:

[0006] Acquire video containing the object to be identified;

[0007] Perform face detection on each frame of the video to obtain facial patches in each frame;

[0008] Based on the temporal relationship between each facial image patch and the positional relationship between key parts in each facial image patch, the video is subjected to facial authenticity detection to obtain the detection result.

[0009] If the detection result is a real face, the identity of the object to be identified is determined based on each facial image patch.

[0010] According to the identity recognition method provided by the present invention, the step of performing facial authenticity detection on the video based on the temporal relationship between each facial image patch and the positional relationship between key parts in each facial image patch to obtain the detection result includes:

[0011] Feature extraction is performed on each key part of each facial image to obtain the location features of each key part;

[0012] Based on the temporal relationship between the facial image patches, temporal features are extracted from the location features of each key part to obtain temporal features;

[0013] Based on the location features of each key part, positional features between each key part are extracted from each facial image block, and the positional features characterize the positional relationship;

[0014] Based on the temporal and positional features, facial authenticity detection is performed on the video to obtain the detection results.

[0015] According to the identity recognition method provided by the present invention, the step of extracting positional features between key facial features from each facial image based on the feature characteristics of each key feature includes:

[0016] Feature fusion is performed on the location features of each key part to obtain weighted features, which represent the importance of each key part in each facial image patch;

[0017] Based on the weighted features, the feature features of each key part are extracted from each facial patch;

[0018] Based on the location patch features of each key part, the positional features between each key part are extracted.

[0019] According to the identity recognition method provided by the present invention, the step of extracting the feature features of key parts from each facial patch based on the weighted features includes:

[0020] Based on the weighted features, candidate patch features for each key part are extracted from each facial patch;

[0021] Feature scoring is performed on the candidate image features of each key part, and the location image features of each key part are selected from the candidate image features of each key part based on the scoring results.

[0022] According to the identity recognition method provided by the present invention, the step of performing face detection on each frame of the video to obtain facial patches in each frame includes:

[0023] Based on the face detection model, face detection is performed on each frame of the video to obtain face patches in each frame.

[0024] The face detection model is trained based on face detection tasks and facial landmark detection tasks.

[0025] According to the identity recognition method provided by the present invention, the step of recognizing the identity of the object to be identified based on the facial image patches includes:

[0026] Given the ID of the object to be identified, the identity of the object to be identified is determined based on the similarity between each facial image patch and the feature template corresponding to the ID.

[0027] In the absence of the ID of the object to be identified, the facial image is matched with the feature template based on the similarity between each facial image and each feature template, and the identity of the object to be identified is determined based on the matching result.

[0028] According to the identity recognition method provided by the present invention, the step of determining the feature template includes:

[0029] Perform facial detection on the video of the recorded object to obtain the recorded facial image block of the recorded object;

[0030] Based on the temporal relationship between each recorded facial image patch and the positional relationship between key parts in each recorded facial image patch, the recording temporal features and the recording positional features are extracted, and the recording temporal features and the recording positional features are fused to obtain the recording fusion features.

[0031] Based on the input fusion features, facial authenticity detection is performed on the video of the input object to obtain the input detection result. If the input detection result is a real face, the input fusion features are used as the feature template of the input object.

[0032] The present invention also provides an identity recognition device, comprising:

[0033] The video acquisition unit is used to acquire video including the object to be identified.

[0034] A face detection unit is used to perform face detection on each frame of the video to obtain face patches in each frame of the video.

[0035] The authenticity detection unit is used to perform facial authenticity detection on the video based on the temporal relationship between each facial image and the positional relationship between key parts in each facial image, and to obtain the detection result.

[0036] An identity recognition unit is used to identify the identity of the object to be identified based on each facial image block when the detection result is a real face.

[0037] The present invention also provides an identity recognition system, comprising: an image acquisition device and an identity recognition device;

[0038] The image acquisition device is used to acquire a video including the object to be identified, and send the video to the identity recognition device;

[0039] The identity recognition device is used to perform face detection on each frame of the video to obtain facial patches in each frame. Based on the temporal relationship between each facial patch and the positional relationship between key parts in each facial patch, the device performs face authenticity detection on the video to obtain detection results. If the detection results indicate a real face, the device identifies the identity of the object to be identified based on each facial patch.

[0040] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the identity recognition method as described above.

[0041] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the identity recognition method as described above.

[0042] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the identity recognition method as described above.

[0043] The identity recognition method, apparatus, system, electronic device, and storage medium provided by this invention perform facial detection on each frame of a video to obtain facial patches in each frame. Based on the temporal relationship between each facial patch and the positional relationship between key parts in each facial patch, the authenticity of the face in the video is detected. When the detection result is a real face, the identity of the object to be identified is identified based on each facial patch. This can effectively identify the authenticity of faces in the video and avoid misidentification caused by the use of forged images, thereby improving the accuracy of identity recognition. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0045] Figure 1 This is one of the flowcharts illustrating the identity recognition method provided by the present invention;

[0046] Figure 2 This is a flowchart illustrating step 130 in the identity recognition method provided by the present invention;

[0047] Figure 3 This is the second flowchart illustrating the identity recognition method provided by the present invention;

[0048] Figure 4 This is the third flowchart illustrating the identity recognition method provided by the present invention;

[0049] Figure 5 This is the fourth flowchart illustrating the identity recognition method provided by the present invention;

[0050] Figure 6 This is the fifth flowchart illustrating the identity recognition method provided by the present invention;

[0051] Figure 7 This is the sixth flowchart illustrating the identity recognition method provided by the present invention;

[0052] Figure 8 This is a schematic diagram of the structure of the identity recognition device provided by the present invention;

[0053] Figure 9 This is a schematic diagram of the structure of the identity recognition system provided by the present invention;

[0054] Figure 10 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0056] In related technologies, pig identification is mainly achieved through ear tag information or pig face recognition. Because pigs are prone to biting and fighting, ear tags are easily detached, and they can also be replaced manually. While pig face recognition solves the problem of ear tag detachment or replacement, it is prone to misidentification, leading to duplicate and fraudulent claims.

[0057] Based on the above considerations, this invention provides an identity recognition method that can be applied to identity recognition scenarios in artificial intelligence technology, such as facial recognition and livestock insurance claims, to improve the recognition accuracy in complex scenarios.

[0058] Figure 1This is one of the flowcharts of the identity recognition method provided by the present invention. The execution subject of each step in the method can be an identity recognition device, which can be implemented by software and / or hardware. The device can be integrated into an electronic device, which can be a terminal device (such as a smartphone, personal computer, etc.), a server (such as a local server or cloud server, or a server cluster, etc.), a processor, or a chip, etc.

[0059] like Figure 1 As shown, the method may include the following steps:

[0060] Step 110: Obtain a video including the object to be identified.

[0061] Specifically, the object to be identified is the object that needs to be identified. The object to be identified can be a person or a livestock such as pigs, cattle, or sheep. For example, it can be a dead pig, a dead cow, or a live pig. This embodiment of the invention does not make specific limitations on this.

[0062] Video of the object to be identified can be acquired using an image acquisition device, which can be a mobile camera or a fixed camera. Taking pigs as an example, a mobile phone, smart tablet, or similar device can be used to record video of the pig's face inside the pigpen, and then crop segments of the pig's face to obtain a video of the pig.

[0063] In an optional embodiment, a fixed camera can be installed at the location where the pigs drink water and eat. When the pigs drink water or eat, the acquisition mechanism is triggered, and the camera begins to record and capture the pig's face, thus obtaining a video of the pig.

[0064] Here, the video of the object to be identified can include consecutive image frames from different perspectives of the object. Simultaneously, the ID of the object to be identified can also be obtained. The ID of the object to be identified can uniquely identify the object's identity information, such as ear tag information for pigs, or personnel numbers or names.

[0065] Step 120: Perform face detection on each frame of the video to obtain facial patches in each frame.

[0066] Specifically, considering the complex video shooting environment and the presence of various background noises in the video, face detection and image segmentation can be performed on each frame of the video to obtain facial patches in each frame.

[0067] To obtain facial patches in each frame of the image, face detection algorithms or pre-trained face detection models can be used to perform face detection on each frame of the video. Based on the face detection results, i.e., the position of the bounding box of the face in each frame, image segmentation is performed on each frame to obtain the facial patches in each frame.

[0068] In an optional embodiment, to further improve the accuracy of face detection, the training task of the face detection model may include a face detection task and a key point detection task, where key points refer to key facial features, such as the nose, eyes, and mouth. When training the face detection model, the detected key points can be constrained to be within the face bounding box.

[0069] Step 130: Based on the temporal relationship between each facial image patch and the positional relationship between key parts in each facial image patch, perform facial authenticity detection on the video to obtain the detection result;

[0070] Step 140: If the detection result is a real face, identify the identity of the object to be identified based on each facial patch.

[0071] Specifically, existing technologies typically perform facial recognition directly on the acquired facial images, without being able to determine whether the input image is a real face or a fake one. Taking pig insurance claims as an example, using fake images for duplicate or fraudulent claims results in significant waste of claims funds and greatly increases the operating costs of livestock insurance business.

[0072] In step 130 of this embodiment, after obtaining each facial image patch, a authenticity detection can be further performed on the face of the object to be identified contained in the video to obtain the detection result. Here, facial authenticity detection refers to detecting whether the face in the video originates from a real human or animal body, and is not a fake or forged image, such as a counterfeit photo or mask. Facial authenticity detection can be achieved through the temporal relationship between each facial image patch and the positional relationship between key parts within each facial image patch.

[0073] Each facial patch can have multiple pieces of information related to the time dimension, that is, the temporal relationship between the facial patches. The temporal relationship can specifically be a time feature, such as the frame time of each facial patch in an image sequence composed of multiple frames, or the acquisition time corresponding to each frame in a video, or the time difference between the acquisition time corresponding to each frame and a labeled time, etc. The embodiments of the present invention do not impose specific limitations on this.

[0074] Key features in each facial patch can include different types of key facial features such as ears, left and right eyes, nose, and mouth. Each frame of the video contains multiple pieces of information related to spatial dimensions, and the positional relationships between key features represent the spatial information of these key features within each facial patch.

[0075] In some embodiments, a convolutional neural network can be used to determine the temporal relationships between facial patches and the positional relationships between key parts within each facial patch. For example, multiple frames of images from a video can be input into the temporal and positional branches of a convolutional neural network, respectively, to obtain multiple feature maps output by each branch. The feature maps output by the temporal branch represent the temporal relationships between facial patches, and the feature maps output by the positional branch represent the positional relationships between key parts within each facial patch.

[0076] Based on the temporal relationship between each facial patch and the positional relationship between key parts in each facial patch, the feature maps representing the temporal relationship between each facial patch and the feature maps representing the positional relationship between key parts in each facial patch can be input into the trained facial authenticity detection model to obtain the detection results output by the trained facial authenticity detection model.

[0077] Furthermore, the face authenticity detection model can be a binary classification model. During the training process, the model continuously learns the temporal and positional relationships of each facial patch corresponding to real facial sample videos, as well as the temporal and positional relationships of each facial patch corresponding to fake facial sample videos, so that the trained face authenticity detection model can accurately classify faces as real or fake.

[0078] If the detection result shows a real face, it indicates that the video obtained in step 110 was captured based on the real face of the subject to be identified, and is not forged or impersonated. The identity of the subject is then identified based on each facial image patch. For example, each facial image patch can be searched and matched with template images stored in a pre-set database, and identity recognition is performed based on the matching results. The search and matching process can be divided into two categories: one is confirmation, which is a one-to-one image comparison process; the other is identification, which is a one-to-many image matching and comparison process.

[0079] Understandably, in cases where the detection result indicates a forged face, an alert can be issued without further identity verification.

[0080] The method provided in this invention performs face detection on each frame of a video to obtain facial patches in each frame. Based on the temporal relationship between the facial patches and the positional relationship between key parts in each facial patch, the method performs face authenticity detection on the video. If the detection result is a real face, the method identifies the identity of the object to be identified based on each facial patch. This method can effectively identify the authenticity of faces in the video and avoid misidentification caused by using fake images, thereby improving the accuracy of identity recognition.

[0081] Based on the above embodiments, Figure 2 This is a flowchart illustrating step 130 of the identity recognition method provided by the present invention, as follows: Figure 2 As shown, step 130 specifically includes:

[0082] Step 131: Extract features from each key part of each facial image to obtain the feature characteristics of each key part;

[0083] Step 132: Based on the temporal relationship between each facial patch, extract the temporal features of each key part to obtain the temporal features;

[0084] Step 133: Based on the location features of each key part, extract the positional features between each key part from each facial image patch. The positional features represent the positional relationship.

[0085] Step 134: Based on temporal and location features, perform facial authenticity detection on the video to obtain the detection results.

[0086] Specifically, the features used to describe key parts of the face of the object to be identified can be called the location features of each key part. The number of location features can be preset, for example, it can be K, where K is a positive integer greater than 1. Location features can be obtained by parsing each key part in each facial patch to determine the image representation vector corresponding to each key part, and using it as the location feature of each key part.

[0087] After obtaining the features of each key facial region, temporal features can be extracted from these features. For example, a recurrent neural network, such as an LSTM (Long Short-Term Memory) module, can be used. The features of each key facial region are input into the LSTM module, which learns the temporal relationships between facial patches, thus obtaining the temporal features output by the LSTM module.

[0088] Furthermore, based on the location features of each key part, positional features between key parts can be extracted from each facial patch. These positional features can include the position of any key part within each facial patch, or the position of each key part within any facial patch. In other words, positional features characterize the positional relationships between key parts in each facial patch. For example, the relative positions between the nose and mouth, or between the nose and ears, can be extracted, as can the positional changes of the nose across different frames.

[0089] After obtaining the temporal features representing temporal relationships and the positional features representing positional relationships, facial authenticity detection can be performed on the video based on these features, yielding the detection results. The temporal and positional features can be fused, for example, by concatenating them. The concatenated features are then input into a trained facial authenticity detection model, which classifies faces as real or fake, obtaining the detection results. These results can include either real or fake faces.

[0090] The method provided in this invention extracts temporal and positional features for the features of each key part, and performs facial authenticity detection on the video based on the extracted temporal and positional features, which can further improve the reliability and accuracy of facial authenticity detection.

[0091] Based on any of the above embodiments Figure 3 This is the second flowchart illustrating the identity recognition method provided by the present invention, as shown below. Figure 3 As shown, step 133 specifically includes:

[0092] Step 133-1: Perform feature fusion on the feature characteristics of each key part to obtain weighted features. The weighted features represent the importance of each key part in each facial patch.

[0093] Step 133-2: Based on weighted features, extract the feature features of each key part from each facial patch;

[0094] Step 133-3: Based on the location patch features of each key part, extract the positional features between each key part.

[0095] Specifically, to obtain the positional features between key parts, feature extraction can be performed based on the part patch features of each key part. Here, the part patch features of each key part are the feature representations of each key part in each facial patch. The scale of the part patch features of each key part can be consistent with the scale of each facial patch, and the number of part patch features can be the same as the number of parts in each key part.

[0096] Extracting the feature values ​​of key facial features from each facial patch can be achieved using weighted features. Weighted features characterize the importance of each key feature within each facial patch and can be obtained by fusing features from the features of each key feature. For example, the feature values ​​of each key feature can be input into the convolutional layer of a convolutional neural network to obtain the weighted features.

[0097] Understandably, the parameters in the weighted features correspond to the importance of any key part of any facial patch. For any facial patch, the parameters of a certain key part in the weighted features can reflect the importance of that key part in the facial patch.

[0098] Based on any of the above embodiments Figure 4 This is the third flowchart of the identity recognition method provided by the present invention, as shown below. Figure 4 As shown, step 133-2 extracts the feature maps of key parts from each facial patch, including:

[0099] Steps 133-21: Based on weighted features, extract candidate patch features for each key part from each facial patch;

[0100] Steps 133-22: Feature scoring is performed on the candidate map features of each key part, and the location map features of each key part are selected from the candidate map features of each key part based on the scoring results.

[0101] Specifically, to extract the feature maps of key parts from each facial patch, candidate feature maps of key parts can be extracted from each facial patch based on weighted features. The number of candidate feature maps can be greater than the number of feature maps of each key part.

[0102] Based on this, feature scoring is performed on the candidate image features of each key part. Feature scoring can be performed from various dimensions such as the degree of distortion, blurriness, and color classification of each key part to obtain the scoring results. The candidate image features can be sorted in descending order of the scoring results, and a preset number of part image features with the highest scores can be selected from the sorted candidate image features. Alternatively, a scoring threshold can be preset, and candidate image features with scores greater than the scoring threshold can be used as part image features of each key part.

[0103] Based on any of the above embodiments Figure 5 This is the fourth flowchart of the identity recognition method provided by the present invention, as shown below. Figure 5As shown, a facial authenticity detection model is used to detect fake faces. The model employs a two-branch approach: one branch learns temporal relationships, and the other branch learns positional relationships. The model uses M consecutive frames of facial images as input, first passing them through a CNN (fully convolutional neural network), and then through a max pooling layer to obtain the feature characteristics of K key areas (K is the preset number of anchors).

[0104] The location features of K key locations are input into the LSTM (Long Short-Term Memory) module to obtain temporal features.

[0105] Simultaneously, the region features of the K key regions obtained above are processed through a CNN and a 1*1 convolution to obtain weighted features with M*K*2 parameters. These parameters are multiplied by the input M facial patches, and bilinear interpolation is used to obtain M*K candidate patch features of the same size as the facial patches. Maxpooling is then performed to obtain a 5*5 score map, and the Top K region patch features are selected from the candidate patch features of each key region based on the score results. Each region patch feature is processed through a CNN feature extraction layer with the same structure and shared weights to obtain location features.

[0106] Finally, the temporal and positional features are concatenated, and after passing through softmax, facial authenticity is classified to obtain the detection result.

[0107] Based on any of the above embodiments, step 120 specifically includes:

[0108] Based on the face detection model, face detection is performed on each frame of the video to obtain face patches in each frame.

[0109] The face detection model is trained based on face detection tasks and facial landmark detection tasks.

[0110] Specifically, face detection can be performed on each frame of the video using a face detection model. Each frame is input into a pre-trained face detection model, which then performs face detection, producing bounding boxes for facial patches in each frame.

[0111] Prior to this, a face detection model can be pre-trained. To make the face detection model more accurate, it is trained based on face detection tasks and facial key point detection tasks.

[0112] The face detection model can include two branches: a face detection task branch and a facial landmark detection task branch. The face detection task is used to detect face bounding boxes in each frame of the image, and the facial landmark detection task is used to detect facial landmarks in each frame of the image. Facial landmarks may include landmarks such as eyes, nose, and mouth.

[0113] Understandably, for the same face to be identified, the coordinates of any facial key point must lie within the face detection bounding box. The loss function of the face detection model can improve the detection reliability by constraining the coordinates of facial key points to lie within the face detection bounding box. The face detection model trained in this way can output the face detection bounding box and the positions of facial key points for any given input frame.

[0114] In some embodiments, taking pig face detection as an example, for sample images of pigs, key point annotation methods can be used to annotate the outline of the pig face, the pig's eyes, the pig's nose, and the pig's mouth.

[0115] Pig face detection boxes are generated from the labeled pig face contours, and all labels are divided into training and testing sets.

[0116] The initial pig face detection model is trained using the training set data. The model structure of the initial pig face detection model can use a RetinaFace model similar to that used for human face detection. Targeted improvements are made to this network; after the network outputs pyramid features, in addition to the normal pig face detection branch, a pig face keypoint supervision branch is added. The loss function of the pig face detection model can include keypoint detection loss and face detection loss.

[0117] The keypoint supervision branch is used to regress five keypoints of the pig's face (two eye keypoints, one nose keypoint, and one snout keypoint). The detected keypoint coordinates are converted into anchor-based normalized parameters, and then the keypoint detection loss is calculated based on the difference between the predicted keypoints and the labeled keypoints.

[0118] The method provided in this invention trains a face detection model based on face detection tasks and facial key point detection tasks, which can further improve the accuracy and reliability of face detection.

[0119] Based on any of the above embodiments Figure 6 This is the fifth flowchart illustrating the identity recognition method provided by the present invention, as shown below. Figure 6 As shown, step 140, which identifies the identity of the object to be identified based on each facial image patch, specifically includes:

[0120] Step 141: Given the ID of the object to be identified, identify the identity of the object based on the similarity between each facial patch and the feature template corresponding to the ID.

[0121] Step 142: In the absence of an unknown ID of the object to be identified, each facial patch is matched with each feature template based on the similarity between each facial patch and each feature template, and the identity of the object to be identified is determined based on the matching results.

[0122] Specifically, identifying the identity of the object to be identified based on facial patches can be divided into two cases. In this case, the face has been determined to be a real face, and facial patches from each frame image have been obtained. The obtained facial patches can be matched with feature templates in a pre-stored database. There are two matching methods:

[0123] The first method is one-to-one facial matching. Given the ID of the object to be identified, the identity is determined based on the similarity between each facial patch and the feature template corresponding to that ID. This similarity can be calculated using conventional similarity calculation methods, such as cosine distance or Euclidean distance. A successful match is considered successful when the similarity score is within a preset threshold, indicating that the object is already entered into the database. Furthermore, for pig insurance claims, this matching record can be simultaneously stored in the database, avoiding duplicate comparisons and insurance claims.

[0124] The second method is one-to-many facial matching. This involves matching each facial image patch with a feature template in the database one by one based on the similarity between each patch and the feature template, without knowing the ID of the object to be identified. The feature template with the highest similarity is then selected as the original data for the object to be identified. Simultaneously, it is necessary to ensure that this similarity remains within a certain threshold to guarantee that the object to be identified is the same object already entered into the database.

[0125] The method provided in this invention can further improve the accuracy and reliability of identity recognition by performing identity recognition based on the similarity between facial images and preset feature templates.

[0126] Based on any of the above embodiments Figure 7 This is the sixth flowchart of the identity recognition method provided by the present invention, as shown below. Figure 7 As shown, the steps for determining the feature template include:

[0127] Step 710: Perform facial detection on the video of the subject to obtain the facial image of the subject.

[0128] Step 720: Based on the temporal relationship between each recorded facial image patch and the positional relationship between key parts in each recorded facial image patch, record temporal features and record positional features are extracted, and the record temporal features and record positional features are fused to obtain the record fusion features.

[0129] Step 730: Based on the input fusion features, perform facial authenticity detection on the video of the input object to obtain the input detection result. If the input detection result is a real face, use the input fusion features as the feature template of the input object.

[0130] Specifically, in the database, each entered object has a unique feature template and a unique ID. The feature template can be determined through the following steps:

[0131] Face detection is performed on the video of the subject to obtain facial patches. Then, based on the temporal relationship between the facial patches and the positional relationship between key parts in each facial patch, the temporal and positional features of the recording are extracted.

[0132] The extraction process for the input time sequence features and input location features can be found in the description of the aforementioned embodiments, and will not be repeated here. Then, the input time sequence features and input location features are fused, specifically, feature concatenation, to obtain the input fused features.

[0133] Based on this, facial authenticity detection is performed on the video of the input object using the input fusion features to obtain the input detection results. If the input detection results show a real face, the input fusion features are used as the feature template of the input object.

[0134] Based on any of the above embodiments, taking pigs as an example, a method for identifying pigs is provided, comprising:

[0135] 1) Sample data collection can be divided into artificially created samples and real samples. Artificially created samples include artificially created attack samples and samples that have been augmented based on the artificial data.

[0136] Real-world data collection: Mobile devices such as smartphones and tablets are used to film the pigs' faces inside the pigpen, and video clips of the pigs' faces are extracted. Simultaneously, the pig's ear tag and other identification information are recorded and added to the dataset. Fixed cameras are installed at the pigs' drinking and eating locations. When a pig drinks or eats, the data collection mechanism is triggered, and the camera begins recording and filming the pig's face, uploading information such as ear tags. Footage is then extracted and matched with the pig's identification information.

[0137] Data fabrication was carried out by playing the aforementioned real data on different devices and then recording these videos using different devices, including various models of mobile phones, tablets, computers, and cameras. To avoid any traces of forgery, screen reflections, moiré patterns, and other glare were avoided during the recording process.

[0138] The aforementioned real data was augmented, a portion of the data was blurred, and real pig face samples were processed using a Gaussian kernel of random size with perspective distortion correction. The real pig samples were then cropped to screen size, resulting in a 1x sample. This sample was weighted with a random image to obtain a weighted image, which was then weighted and fused with the background to obtain artificially fabricated data.

[0139] 2) Using the above sample data, a face detection model is trained based on the face detection task and the face key point detection task. The face detection model is used to perform face detection on each frame of the input video. Using the above sample data, a face authenticity detection model is trained. The face authenticity detection model is used to perform face authenticity detection on each frame of the input video.

[0140] 3) Perform facial detection on the videos of the pigs to obtain facial patches of the pigs; based on the temporal relationship between each facial patch and the positional relationship between key parts in each facial patch, extract the temporal and positional features of the recording, and fuse the temporal and positional features to obtain the fused features of the recording; based on the fused features of the recording, perform facial authenticity detection on the videos of the pigs to obtain the detection results; if the detection results show a real face, use the fused features of the recording as the feature template for the pigs.

[0141] 4) Acquire a video containing the pig to be identified, perform face detection on each frame of the video to obtain face patches in each frame, and perform face authenticity detection on the video based on the temporal relationship between each face patch and the positional relationship between key parts in each face patch to obtain the detection results.

[0142] When the detection result is a real face, the identity of the pig to be identified is determined based on each facial patch. If the ID of the pig to be identified is known, the identity is determined based on the similarity between each facial patch and the feature template corresponding to the ID. If the ID of the pig to be identified is unknown, each facial patch is matched with each feature template based on the similarity between each facial patch and each feature template, and the identity of the pig is determined based on the matching results.

[0143] The identity recognition device provided by the present invention is described below. The identity recognition device described below and the identity recognition method described above can be referred to in correspondence.

[0144] Based on any of the above embodiments Figure 8 This is a schematic diagram of the identity recognition device provided by the present invention, as shown below. Figure 8 As shown, the identity recognition device includes a video acquisition unit 810, a face detection unit 820, a authenticity detection unit 830, and an identity recognition unit 840, wherein...

[0145] The video acquisition unit 810 is used to acquire a video including the object to be identified.

[0146] The face detection unit 820 is used to perform face detection on each frame of the video to obtain face patches in each frame.

[0147] The authenticity detection unit 830 is used to perform facial authenticity detection on the video based on the temporal relationship between each facial patch and the positional relationship between key parts in each facial patch, and to obtain the detection result.

[0148] The identity recognition unit 840 is used to identify the identity of the object to be identified based on each facial patch when the detection result is a real face.

[0149] The identity recognition device provided in this invention performs facial detection on each frame of a video to obtain facial patches in each frame. Based on the temporal relationship between the facial patches and the positional relationship between key parts in each facial patch, it performs facial authenticity detection on the video. If the detection result is a real face, it identifies the identity of the object to be identified based on each facial patch. This can effectively identify the authenticity of faces in the video and avoid misidentification caused by using forged images, thereby improving the accuracy of identity recognition.

[0150] Based on the above embodiments, the authenticity detection unit is specifically used for:

[0151] Feature extraction is performed on each key part of each facial image to obtain the location features of each key part;

[0152] Based on the temporal relationship between the facial image patches, temporal features are extracted from the location features of each key part to obtain temporal features;

[0153] Based on the location features of each key part, positional features between each key part are extracted from each facial image block, and the positional features characterize the positional relationship;

[0154] Based on the temporal and positional features, facial authenticity detection is performed on the video to obtain the detection results.

[0155] Based on the above embodiments, the authenticity detection unit is specifically used for:

[0156] Feature fusion is performed on the location features of each key part to obtain weighted features, which represent the importance of each key part in each facial image patch;

[0157] Based on the weighted features, the feature features of each key part are extracted from each facial patch;

[0158] Based on the location patch features of each key part, the positional features between each key part are extracted.

[0159] Based on the above embodiments, the authenticity detection unit is specifically used for:

[0160] Based on the weighted features, candidate patch features for each key part are extracted from each facial patch;

[0161] Feature scoring is performed on the candidate image features of each key part, and the location image features of each key part are selected from the candidate image features of each key part based on the scoring results.

[0162] Based on the above embodiments, the face detection unit is specifically used for:

[0163] Based on the face detection model, face detection is performed on each frame of the video to obtain face patches in each frame.

[0164] The face detection model is trained based on face detection tasks and facial landmark detection tasks.

[0165] Based on the above embodiments, the identity recognition unit is specifically used for:

[0166] Given the ID of the object to be identified, the identity of the object to be identified is determined based on the similarity between each facial image patch and the feature template corresponding to the ID.

[0167] In the absence of the ID of the object to be identified, the facial image is matched with the feature template based on the similarity between each facial image and each feature template, and the identity of the object to be identified is determined based on the matching result.

[0168] Based on the above embodiments, the identity recognition device further includes a template determination unit, used for:

[0169] Perform facial detection on the video of the recorded object to obtain the recorded facial image block of the recorded object;

[0170] Based on the temporal relationship between each recorded facial image patch and the positional relationship between key parts in each recorded facial image patch, the recording temporal features and the recording positional features are extracted, and the recording temporal features and the recording positional features are fused to obtain the recording fusion features.

[0171] Based on the input fusion features, facial authenticity detection is performed on the video of the input object to obtain the input detection result. If the input detection result is a real face, the input fusion features are used as the feature template of the input object.

[0172] Based on any of the above embodiments Figure 9 This is a schematic diagram of the identity recognition system provided by the present invention, as shown below. Figure 9 As shown, the identity recognition system includes: an image acquisition device 910 and an identity recognition device 920, wherein,

[0173] Image acquisition equipment is used to acquire video including the object to be identified and send the video to the identity recognition equipment;

[0174] The identity recognition device is used to perform face detection on each frame of a video to obtain facial patches in each frame. Based on the temporal relationship between each facial patch and the positional relationship between key parts in each facial patch, the device performs face authenticity detection on the video to obtain the detection result. If the detection result is a real face, the device identifies the identity of the object to be identified based on each facial patch.

[0175] Specifically, the identity recognition system provided in this embodiment of the invention can automatically and intelligently recognize the identity of the object to be identified. The image acquisition device is used to acquire video including the object to be identified and send the video to the identity recognition device. The image acquisition device can be a smartphone, tablet computer, fixed camera, etc., equipped with a camera function.

[0176] The identity recognition device is used to identify the identity of the object to be identified. The specific recognition process has been described in detail in the foregoing embodiments and will not be repeated here.

[0177] Identity recognition devices and image acquisition devices can be integrated into the same electronic device, such as a smartphone. After the smartphone captures a video of the object to be identified, it sends the video to the identity recognition device for identity verification.

[0178] The identity recognition device can also be set up in the cloud. The image acquisition device sends the captured video to the identity recognition device via the network, and the identity recognition device completes the identity recognition in the cloud.

[0179] The identity recognition system provided in this invention can automatically and intelligently recognize the identity of the object to be identified, effectively identify the authenticity of faces in videos, avoid misidentification caused by the use of forged images, and improve the accuracy of identity recognition.

[0180] Figure 10 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 10 As shown, the electronic device may include a processor 1010, a communications interface 1020, a memory 1030, and a communication bus 1040, wherein the processor 1010, the communications interface 1020, and the memory 1030 communicate with each other via the communication bus 1040. The processor 1010 can call logical instructions in the memory 1030 to execute an identity recognition method, which includes: acquiring a video including an object to be identified; performing face detection on each frame of the video to obtain facial patches in each frame; performing face authenticity detection on the video based on the temporal relationship between each facial patch and the positional relationship between key parts in each facial patch to obtain a detection result; and, if the detection result indicates a real face, identifying the identity of the object to be identified based on the facial patches.

[0181] Furthermore, the logical instructions in the aforementioned memory 1030 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0182] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the identity recognition method provided by the above methods. The method includes: acquiring a video including an object to be identified; performing face detection on each frame of the video to obtain facial patches in each frame; performing face authenticity detection on the video based on the temporal relationship between each facial patch and the positional relationship between key parts in each facial patch to obtain a detection result; and identifying the identity of the object to be identified based on the facial patches when the detection result is a real face.

[0183] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the identity recognition method provided by the above methods. The method includes: acquiring a video including an object to be identified; performing face detection on each frame of the video to obtain facial patches in each frame; performing face authenticity detection on the video based on the temporal relationship between each facial patch and the positional relationship between key parts in each facial patch to obtain a detection result; and, if the detection result indicates a real face, identifying the identity of the object to be identified based on the facial patches.

[0184] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0185] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0186] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An identity recognition method, characterized in that, include: Acquire video containing the object to be identified; Perform face detection on each frame of the video to obtain facial patches in each frame; Based on the temporal relationship between each facial image patch and the positional relationship between key parts in each facial image patch, the video is subjected to facial authenticity detection to obtain the detection result. If the detection result is a real face, the identity of the object to be identified is determined based on each facial image patch; The positional relationship is characterized by the positional features between key parts in each facial image patch, and these positional features are extracted through the following steps: The location features of each key part in each facial image are obtained and the features are fused to obtain weighted features; Based on the weighted features, candidate patch features for each key part are extracted from each facial patch; Feature scoring is performed on the candidate patch features of each key part, and the part patch features of each key part are selected from the candidate patch features of each key part based on the scoring results. Based on the location patch features of each key part, the positional features between each key part are extracted.

2. The identity recognition method according to claim 1, characterized in that, The video is subjected to facial authenticity detection based on the temporal relationship between facial patches and the positional relationship between key parts in each facial patch, and the detection results include: Feature extraction is performed on each key part of each facial image to obtain the location features of each key part; Based on the temporal relationship between the facial image patches, temporal features are extracted from the location features of each key part to obtain temporal features; Based on the temporal and positional features, the video is subjected to facial authenticity detection to obtain the detection results.

3. The identity recognition method according to claim 2, characterized in that, The weighted features characterize the importance of each key part in each facial patch.

4. The identity recognition method according to claim 1, characterized in that, The step of performing face detection on each frame of the video to obtain facial patches in each frame includes: Based on the face detection model, face detection is performed on each frame of the video to obtain face patches in each frame. The face detection model is trained based on face detection tasks and facial landmark detection tasks.

5. The identity recognition method according to any one of claims 1-4, characterized in that, The process of identifying the identity of the object to be identified based on each facial image patch includes: Given the ID of the object to be identified, the identity of the object to be identified is determined based on the similarity between each facial image patch and the feature template corresponding to the ID. In the absence of the ID of the object to be identified, the facial image is matched with the feature template based on the similarity between each facial image and each feature template, and the identity of the object to be identified is determined based on the matching result.

6. The identity recognition method according to claim 5, characterized in that, The steps for determining the feature template include: Perform facial detection on the video of the recorded object to obtain the recorded facial image block of the recorded object; Based on the temporal relationship between each recorded facial image patch and the positional relationship between key parts in each recorded facial image patch, the recording temporal features and the recording positional features are extracted, and the recording temporal features and the recording positional features are fused to obtain the recording fusion features. Based on the input fusion features, facial authenticity detection is performed on the video of the input object to obtain the input detection result. If the input detection result is a real face, the input fusion features are used as the feature template of the input object.

7. An identity recognition device, characterized in that, include: The video acquisition unit is used to acquire a video including the object to be identified. A face detection unit is used to perform face detection on each frame of the video to obtain face patches in each frame of the video. The authenticity detection unit is used to perform facial authenticity detection on the video based on the temporal relationship between each facial image and the positional relationship between key parts in each facial image, and to obtain the detection result. An identity recognition unit is used to identify the identity of the object to be identified based on each facial image when the detection result is a real face. The positional relationship is characterized by the positional features between key parts in each facial image patch, and these positional features are extracted through the following steps: The location features of each key part in each facial image are obtained and the features are fused to obtain weighted features; Based on the weighted features, candidate patch features for each key part are extracted from each facial patch; Feature scoring is performed on the candidate patch features of each key part, and the part patch features of each key part are selected from the candidate patch features of each key part based on the scoring results. Based on the location patch features of each key part, the positional features between each key part are extracted.

8. An identity recognition system, characterized in that, include: Image acquisition equipment and identity recognition equipment; The image acquisition device is used to acquire a video including the object to be identified, and send the video to the identity recognition device; The identity recognition device is used to perform face detection on each frame of the video to obtain face patches in each frame of the video. Based on the temporal relationship between each face patch and the positional relationship between key parts in each face patch, the device performs face authenticity detection on the video to obtain the detection result. If the detection result is a real face, the device identifies the identity of the object to be identified based on each face patch. The positional relationship is characterized by the positional features between key parts in each facial image patch, and these positional features are extracted through the following steps: The location features of each key part in each facial image are obtained and the features are fused to obtain weighted features; Based on the weighted features, candidate patch features for each key part are extracted from each facial patch; Feature scoring is performed on the candidate patch features of each key part, and the part patch features of each key part are selected from the candidate patch features of each key part based on the scoring results. Based on the location patch features of each key part, the positional features between each key part are extracted.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the identity recognition method as described in any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the identity recognition method as described in any one of claims 1 to 6.

Images