Part recognition method and related equipment and device
Through multi-frame image processing and optical flow map generation network, the identification accuracy and real-time problems caused by traditional optical flow acquisition methods are solved, and accurate and real-time part recognition is achieved, especially the ability to distinguish live objects from prostheses.
Patent Information
- Application Number
- CN202010561461.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-06-18
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2040-06-18
AI Technical Summary
In the prior art, in part recognition, traditional optical flow acquisition methods lead to problems of identification accuracy and real-time, and it is difficult to accurately and in real-time differentiate between living bodies and prostheses.
By acquiring the original images of multiple frames of the camera device, object detection is performed and image groups are formed, and network processing is used to generate a dense optical flow map, and then a dense optical flow map is determined whether the part to be identified is a living part.
Accurate and real-time part recognition is achieved, complex calculations are avoided, and the recognition is robust and real-time.
Smart Images

Figure CN111814589B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of information technology, and in particular to a part recognition method and related equipment and devices. Background Art
[0002] With the development of information technology, the recognition of parts such as face has been widely used in mobile payment, security inspection, office attendance and other scenarios. Part recognition generally includes part detection, liveness detection, part comparison, etc., and since prostheses (such as printed portraits, electronic portraits, three-dimensional masks, etc.) will affect system security, liveness detection is the most critical among them.
[0003] Currently, one of the methods of liveness detection is to capture moving targets based on optical flow to distinguish between live objects and prostheses. However, using traditional optical flow acquisition methods such as the Lucas-Kanad method to obtain sparse optical flow maps of feature points will affect recognition accuracy, while using traditional optical flow acquisition to obtain dense optical flow maps will affect recognition real-time performance due to high computational complexity. In view of this, how to accurately and real-timely realize part recognition has become an urgent problem to be solved. Summary of the invention
[0004] The main technical problem solved by the present application is to provide a part recognition method and related equipment and devices, which can realize part recognition accurately and in real time.
[0005] In order to solve the above problems, the first aspect of the present application provides a part recognition method, including: acquiring multiple frames of original images taken by a camera device; performing target detection on the multiple frames of original images to obtain a target area of a part to be recognized in each frame of the original image; using the multiple frames of original images to form at least one group of original images, and using the image data of the target area in each group of original images as a group of data to be processed, wherein each group of original images includes at least two frames of original images; using a trained optical flow map generation network to process at least one group of data to be processed to obtain a corresponding part optical flow map; using the part optical flow map to determine whether the part to be recognized is a living part.
[0006] In order to solve the above problem, the second aspect of the present application provides an electronic device, including a memory and a processor coupled to each other, the memory stores program instructions, and the processor is used to execute the program instructions to implement the part recognition method in the above first aspect.
[0007] In order to solve the above problem, the third aspect of the present application provides a storage device storing program instructions that can be executed by a processor, and the program instructions are used to implement the part recognition method in the above first aspect.
[0008] The above scheme obtains multiple frames of original images taken by the camera device and performs target detection on the multiple frames of original images to obtain the target area of the part to be identified in each frame of the original image, thereby using the multiple frames of original images to form at least one group of original images, and the image data of the target area in each group of original images is used as a group of images to be processed, and each group of original images includes at least two frames of original images, and then uses the trained optical flow map generation network to process at least one group of data to be processed to obtain the corresponding part optical flow map, so that it is possible to avoid complex calculations and quickly obtain a dense optical flow map, and then use the part optical flow map to accurately and in real time determine whether the part to be identified is a living part. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] Figure 1 It is a flow chart of an embodiment of the part identification method of the present application;
[0010] Figure 2 yes Figure 1 A flowchart of an embodiment of a training method for an optical flow map generation network;
[0011] Figure 3 yes Figure 1 A schematic diagram of the framework of an embodiment of an optical flow graph generation network;
[0012] Figure 4 yes Figure 1 A schematic diagram of a flow chart of an embodiment of step S14;
[0013] Figure 5 It is a schematic diagram of the framework of an embodiment of a feature fusion sub-network;
[0014] Figure 6 yes Figure 1 A schematic diagram of a flow chart of an embodiment of step S15;
[0015] Figure 7 is a schematic diagram of an embodiment of a key point in a target area;
[0016] Figure 8 It is a schematic diagram of the framework of an embodiment of the part recognition device of the present application;
[0017] Fig. 9 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application;
[0018] Fig.10 It is a schematic diagram of a framework of an embodiment of a storage device. DETAILED DESCRIPTION
[0019] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.
[0020] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.
[0021] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.
[0022] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of the part recognition method of the present application. Specifically, it may include the following steps:
[0023] Step S11: Acquire multiple frames of original images captured by the imaging device.
[0024] The camera device can be set according to the actual application scenario. For example, in the mobile payment scenario, the camera device can be a mobile terminal (such as a mobile phone, tablet computer, etc.) that can run a payment program (APP), a self-service checkout machine, etc.; in the security and inspection scenario, the camera device can be a surveillance camera, etc.; in the office attendance scenario, the camera device can be an attendance machine, an access control machine, etc.; in the building visitor scenario, the camera device can be a surveillance camera, an access control machine, etc., which can be specifically set according to the application scenario, and examples will not be given one by one here.
[0025] The multiple frames of original images obtained may be two frames of original images, three frames of original images, four frames of original images, etc., which are not limited here. In one implementation scenario, the number of frames of original images to be obtained may be preset, and when the original images of the set number of frames are not obtained, a prompt message may be output to prompt the user to be within the shooting range of the camera device so that the camera device can obtain the original images of the set number of frames.
[0026] Step S12: performing target detection on multiple frames of original images to obtain a target area of a part to be identified in each frame of the original image.
[0027] In an implementation scenario, a pre-trained network model can be used to perform target detection on multiple frames of original images to obtain the target area of the part to be identified in each frame of the original image. Taking the part to be identified as a face as an example, SSH (Single stage Headless Face Detector) can be used to perform target detection on multiple frames of original images to obtain the target area of the face in each frame of the original image. Since SSH is scale-invariant, faces of different scales can be detected with only one original image, and SSH can be used to ensure detection accuracy while also realizing real-time detection of faces in the original image. In addition, network models such as face R-CNN, MTCNN, and faceness-net can also be used to perform target detection on the original image to obtain the target area of the face in each frame of the original image. When the part to be identified is other parts, it can be deduced by analogy, and examples are not given one by one here.
[0028] Step S13: using multiple frames of original images to form at least one group of original images, and using the image data of the target area in each group of original images as a group of data to be processed.
[0029] Each group of original images includes at least two frames of original images, for example, two frames of original images, three frames of original images, etc. Specifically, each group of original images may include two adjacent frames of original images, that is, two frames of original images captured by the imaging device at adjacent times.
[0030] The image data of the target area in each group of original images can form a group of data to be processed for subsequent processing.
[0031] Step S14: using the trained optical flow map generation network to process at least one set of data to be processed to obtain a corresponding part optical flow map.
[0032] In one implementation scenario, the number of groups of data to be processed may be one group, so the trained optical flow map generation network may be used to process one group of data to be processed to obtain a part optical flow map corresponding to the group of data to be processed. In another implementation scenario, the number of groups of data to be processed may also be multiple groups (for example, two groups, three groups, etc.), so the trained optical flow map generation network may be used to process multiple groups of data to be processed to obtain a part optical flow map corresponding to each group of data to be processed, which is not limited here.
[0033] Still taking the case where the part to be identified is a face as an example, a trained optical flow map generation network can be used to process at least one set of data to be processed to obtain a corresponding face optical flow map.
[0034] In an implementation scenario, before processing at least one group of data to be processed, at least one group of sample data and its corresponding sample optical flow map can also be obtained, and each group of sample data and its corresponding sample optical flow map are obtained using the image data of the target area in a pair of adjacent sample images. The sample optical flow map is a dense optical flow map. Unlike sparse optical flow, which only targets a number of feature points on the image, dense optical flow calculates the optical flow of all points on the image, so that each group of sample data is processed using an optical flow map generation network to obtain a predicted optical flow map, and it is beneficial to predict the optical flow map and the sample optical flow map, obtain the loss value of the optical flow map generation network, and then use the loss value to adjust the parameters of the optical flow map generation network. In a specific implementation scenario, when the preset training end condition is met, the training of the optical flow map generation network can be terminated. The preset training end condition can specifically include at least one of the following: the loss value is less than a preset loss threshold, and the number of training times reaches a preset number threshold. In another specific implementation scenario, the parameters of the optical flow map generation network can be adjusted by at least one of the following methods: Stochastic Gradient Descent (SGD), Batch Gradient Descent (BGD), Mini-Batch Gradient Descent (MBGD), where Stochastic Gradient Descent means using one sample to update the parameters at each iteration; Mini-Batch Gradient Descent means using a batch of samples to update the parameters at each iteration, which will not be repeated here.
[0035] Step S15: using the part optical flow map, determine whether the part to be identified is a living part.
[0036] In one implementation scenario, when a corresponding part optical flow map is obtained using a set of data to be processed, the part optical flow map can be analyzed to determine whether the part to be identified is a living part. In another implementation scenario, when multiple sets of data to be processed are used to obtain multiple corresponding part optical flow maps, the multiple part optical flow maps can be analyzed separately, and according to the analysis results corresponding to each part optical flow map, it is determined whether the part to be identified is a living part. In a specific implementation scenario, the proportion of the number of analysis results of living parts in the total number of all analysis results can be counted. If the proportion is greater than a preset proportion, the part to be identified can be considered to be a living part, thereby avoiding the recognition error caused by a false detection, and thus improving the robustness of part recognition.
[0037] In one implementation scenario, still taking the case where the part to be identified is a face as an example, the part optical flow map can be used to count the optical flow information near the eyes of the face, and determine whether it is a live face based on the optical flow information near the eyes. In a specific implementation scenario, the optical flow amplitude near the eyes can be counted. If the optical flow amplitude obtained by counting satisfies the preset amplitude condition (for example, the sum of the optical flow amplitudes is greater than a preset amplitude threshold), it can be determined as a live face, otherwise it can be determined as a prosthetic face. In another specific implementation scenario, the optical flow direction near the eyes can also be counted. If the optical flow direction obtained by counting satisfies the preset direction condition (for example, the distribution of the optical flow direction conforms to a preset direction rule), it can be determined as a live face, which is not limited here. When the part to be identified is other parts, it can be deduced by analogy, and examples are not given one by one here.
[0038] The above scheme obtains multiple frames of original images taken by the camera device and performs target detection on the multiple frames of original images to obtain the target area of the part to be identified in each frame of the original image, thereby using the multiple frames of original images to form at least one group of original images, and the image data of the target area in each group of original images is used as a group of images to be processed, and each group of original images includes at least two frames of original images, and then uses the trained optical flow map generation network to process at least one group of data to be processed to obtain the corresponding part optical flow map, so that it is possible to avoid complex calculations and quickly obtain a dense optical flow map, and then use the part optical flow map to accurately and in real time determine whether the part to be identified is a living part.
[0039] See also Figure 2 , Figure 2 yes Figure 1 A flowchart of an embodiment of a training method for generating an optical flow map network is provided, which may specifically include the following steps:
[0040] Step S21: Obtain at least one set of sample data and its corresponding sample optical flow map.
[0041] Each set of sample data and its corresponding sample optical flow map are obtained using the image data of the target area in a pair of adjacent sample images. In a specific implementation scenario, target detection can be performed on adjacent sample images to obtain the target area of the target part in each frame of the sample image, and the image data of the target area is used as a set of sample data; or, the sample image is annotated with a target area corresponding to the target part, so that the image data of the target area is directly used as a set of sample data. In another specific implementation scenario, the sample data can be optically extracted using the Gunnar Farneback algorithm and other methods to obtain a dense sample optical flow map. The Gunnar Farneback dense optical flow algorithm is a motion estimation algorithm based on all pixels in the previous and next two frames of images, and optical flow tracking is achieved through the displacement vectors of all pixels in the previous and next two frames of images.
[0042] Step S22: using the optical flow map generation network to process each group of sample data, and obtain multiple predicted optical flow maps corresponding to different network depths of the optical flow map generation network.
[0043] Please refer to Figure 3 , Figure 3 yes Figure 1 Schematic diagram of the framework of an embodiment of an optical flow graph generation network. Figure 3 As shown, the optical flow map generation network may include an encoder and a decoder, the encoder is mainly used to extract features from the sample data to encode the sample data, and the decoder is mainly used to reversely reconstruct the extracted data to decode the data obtained by the encoding process, thereby obtaining a corresponding predicted optical flow map. In one implementation scenario, the size of the image data of the target area of a sample image is w*h, and the channel of each pixel in the image data is 3 (e.g., red, green, and blue). The size of the image data of the target area of the adjacent sample image is w*h, and the channel of each pixel in the image data is 3 (e.g., red, green, and blue). Therefore, each group of sample data is spliced to obtain spliced data with a size of w*h and a channel of 6 for each pixel, so that the optical flow map generation network encodes, decodes, and other processes on the spliced data to obtain a corresponding predicted optical flow map. In another implementation scenario, the decoder may include deconvolution layers at different network depths for decoding to obtain predicted optical flow maps of different scales. Deconvolution is divided into transposed convolution and fractionally strided convolution. Transposed convolution expands the feature image size by padding with zeros in a certain proportion, rotates the convolution kernel, and performs forward convolution; fractionally strided convolution inserts several zeros between the units of the input feature image, uses the inserted zeros as new feature input, and extracts features with a step size of one.
[0044] Step S23: using the plurality of predicted optical flow maps and the sample optical flow maps respectively to obtain a plurality of loss values.
[0045] By utilizing the differences between multiple predicted optical flow maps and sample optical flow maps, the loss values of the optical flow map generation network at different network depths can be obtained.
[0046] Step S24: using multiple loss values, adjusting the parameters of the optical flow map generation network.
[0047] In an implementation scenario, in order to speed up the convergence of the network, the preset weights corresponding to the multiple loss values can be used to perform weighted processing on the multiple loss values to obtain the final loss value of the optical flow map generation network, so as to use the final loss value to adjust the parameters of the optical flow map generation network, and the numerical values of the preset weights corresponding to the multiple loss values and the network depths corresponding to the multiple loss values are positively correlated or negatively correlated. Specifically, the preset weights corresponding to the multiple loss values can be set to be positively correlated or negatively correlated with the network depth in accordance with different training stages. For example, in the early stage of training, the preset weights corresponding to the multiple loss values can be set to be positively correlated with the network depth, and in the later stage of training, the preset weights corresponding to the multiple loss values can be set to be negatively correlated with the network depth. This is not limited here.
[0048] Different from the aforementioned embodiments, at least one group of sample data and its corresponding sample optical flow map are obtained, and each group of sample data and its corresponding sample optical flow map are obtained by using the image data of the target area in a pair of adjacent sample images, and each group of sample data is processed by using the optical flow map generation network to obtain a plurality of predicted optical flow maps corresponding to different network depths of the optical flow map generation network, thereby respectively using the plurality of predicted optical flow maps and the sample optical flow maps to obtain a plurality of loss values, and using the plurality of loss values to adjust the parameters of the optical flow map generation network, thereby accelerating network training.
[0049] See also Figure 4 , Figure 4 yes Figure 1 The flowchart of step S14 in the embodiment of FIG. 1 may specifically include the following steps:
[0050] Step S141: performing splicing processing on each group of data to be processed to obtain spliced data, and performing feature extraction on the spliced data to obtain a first feature map.
[0051] Specifically, the corresponding pixel points in the image data of the target areas of the two original images in each group of data to be processed can be spliced, so that the splicing can be performed in the channel dimension to obtain spliced data. For example, the size of the image data of the target area of a certain original image is w*h, and the channel of each pixel in the image data is 3 (e.g., red, green, and blue). The size of the image data of the target area of the original image adjacent to it is w*h, and the channel of each pixel in the image data is 3 (e.g., red, green, and blue). Therefore, each group of data to be processed is spliced to obtain spliced data with a size of w*h and 6 channels for each pixel. Other scenarios can be deduced by analogy, and no examples are given here one by one.
[0052] In an implementation scenario, such as Figure 3As shown, the optical flow map generation network may include a feature extraction subnetwork, so that the feature extraction subnetwork can be used to extract features from the spliced data to obtain a first feature map. In a specific implementation scenario, the feature extraction subnetwork may include multiple sequentially connected convolutional layers. In another specific implementation scenario, in order to deepen the network depth and effectively improve the depth features, the feature extraction subnetwork may include at least one sequentially connected residual block (resblock).
[0053] Step S142: Perform dimensionality reduction processing on the first feature map to obtain at least one second feature map.
[0054] In an implementation scenario, the first feature map may be subjected to dimensionality reduction processing at different dimensionality reduction scales to obtain second feature maps at different scales.
[0055] In another implementation scenario, in order to further expand the global information in the third feature map used for decoding processing, the optical flow map generation network may also include multiple sequentially connected feature fusion subnetworks, and for each feature fusion subnetwork, the feature map processed by the previous and adjacent subnetwork is used as the first feature map. For example, for the first feature fusion subnetwork, the feature map processed by the previous and adjacent feature extraction subnetwork may be used as the first feature map; or, for the i-th feature fusion subnetwork, the feature map processed by the i-1-th feature fusion subnetwork may be used as the first feature map, and i is an integer not less than 1. After obtaining the first feature map, the first feature map may be pooled using at least one pooling layer in the feature fusion subnetwork to obtain at least one second feature map. Please refer to Figure 5 , Figure 5 Schematic diagram of the framework of an embodiment of a feature fusion sub-network. Figure 5 As shown, the feature fusion subnetwork includes 4 pooling layers, specifically, they can be 64×64 average pooling blocks, 32×32 average pooling blocks, 16×16 average pooling blocks and 8×8 average pooling blocks. In addition, each average pooling block can be connected to a 1×1 convolution and each is connected to an upsampling processing module.
[0056] Step S143: Fusing the first feature map and at least one second feature map to obtain a third feature map.
[0057] Specifically, the first feature map and at least one second feature map may be concatenated to obtain a third feature map, so that the third feature map can integrate global context information.
[0058] In an implementation scenario, in order to further integrate global context information, for each of the multiple feature fusion subnetworks, a third feature map processed by the second feature fusion subnetwork before it is obtained, and the first feature map, at least one second feature map and the obtained third feature map are concatenated to obtain the third feature map. Figure 5 ,like Figure 5 As shown, the third feature map processed by the first feature map and other feature fusion subnetworks (such as the second feature fusion subnetwork before it) is downsampled and concatenated with at least one second feature map after upsampling to obtain the third feature map processed by the current feature fusion subnetwork, so that the third feature map processed by the last feature fusion subnetwork can integrate the global context information as much as possible.
[0059] Step S144: Decode the third feature map to obtain a part optical flow map.
[0060] In an implementation scenario, after obtaining the third feature map, the third feature map can be extracted using multiple convolutional layers in the encoder to obtain encoded data, and the encoded data can be decoded using a decoder in the optical flow map generation network to obtain a part optical flow map. Figure 3 ,The optical flow map generation network can include the above-mentioned feature extraction subnetwork, feature fusion subnetwork, and multiple convolutional layers.
[0061] Different from the aforementioned embodiment, each group of data to be processed is spliced to obtain spliced data, and features are extracted from the spliced data to obtain a first feature map, and the first feature map is subjected to dimensionality reduction processing to obtain at least one second feature map, thereby the first feature map and the at least one second feature map are fused to obtain a third feature map, so that the third feature map can integrate global context information as much as possible, thereby decoding the third feature map, which can improve the accuracy of the obtained part optical flow map, and thus can help improve the accuracy of part recognition.
[0062] See also Figure 6 , Figure 6 yes Figure 1 Schematic diagram of the process of step S15 in an embodiment. Specifically, Figure 6 A flowchart of an embodiment of using a part optical flow map to determine whether the part to be identified is a living part when the part to be identified is a human face. Specifically, the following steps may be included:
[0063] Step S151: Compare the optical flow amplitudes of the eye area and the peri-eye area in the face optical flow map to obtain a first comparison score.
[0064] The eye area can be obtained by performing eye area detection on the target area, and the periocular area can be obtained by performing periocular area detection on the target area, and the coverage range of the eye area is larger than the coverage range of the periocular area. For example, the eye area can include the eyebrow area, some skin areas above and below the eyes, and around the left and right eyes.
[0065] In an implementation scenario, the target region can be detected using an Adaboost stacked eye classifier to obtain the eye region. Adaboost is an iterative algorithm, the core idea of which is to train different classifiers (weak classifiers) for the same training set, and then combine these weak classifiers to form a stronger final classifier (strong classifier).
[0066] In another implementation scenario, key point detection can be performed on the target area to obtain key points in the target area, and the eye area in the target area can be determined by using the key points corresponding to the eyes. Figure 7 , Figure 7 : is a schematic diagram of an embodiment of key points in the target area. The key points may include key points corresponding to the facial contour (i.e., key points 1 to 17), key points corresponding to the eyebrows (i.e., key points 18 to 27), key points corresponding to the nose (i.e., key points 28 to 36), key points corresponding to the eyes (i.e., key points 37 to 48), and key points corresponding to the mouth (i.e., key points 49 to 68). In another specific implementation scenario, the key points in the target area may be detected using methods such as dlib and firebase. In yet another specific implementation scenario, the area enclosed by the key points corresponding to the eyes may be used as the peri-ocular area.
[0067] Specifically, the target pixels whose optical flow amplitudes in the eye area meet the preset amplitude conditions can be counted, and the proportion of the target pixels in the eye area is used as the first comparison score. In an implementation scenario, the optical flow value of each pixel in the local optical flow map is a vector containing the optical flow amplitude and the optical flow direction, so the horizontal component u and the vertical component v of the vector can be used to obtain the optical flow amplitude s and the optical flow direction θ respectively through the following formulas:
[0068]
[0069] θ=tan -1 (v / u)
[0070] In another implementation scenario, the preset amplitude condition may include: if the optical flow amplitude is greater than a preset amplitude threshold, then the target pixels with larger optical flow amplitudes in the periocular area may be counted, and the proportion of the target pixels in the eye area may be calculated as the first comparison score.
[0071] Step S152: Compare the optical flow directions of the eye area and the peri-eye area in the face optical flow map to obtain a second comparison score.
[0072] Specifically, the first distribution of the optical flow direction of the key points in the target area can be counted, and the second distribution of the optical flow direction of the key points in the periocular area corresponding to the eyes can be counted, so as to determine the similarity between the first distribution and the second distribution, and use the similarity as the second comparison score. In one implementation scenario, the first distribution and the second distribution can be an optical flow direction histogram. Specifically, 0 to 2π can be divided into 12 intervals, and the optical flow direction of the key points in the target area can be counted to belong to one of the 12 intervals, so as to obtain the first distribution, and the optical flow direction of the key points in the periocular area corresponding to the eyes can be counted to belong to one of the 12 intervals, so as to obtain the second distribution. In another implementation scenario, the similarity between the first distribution and the second distribution can be determined by using chi-square test, Bhattacharyya coefficient, etc., as the second comparison score.
[0073] The above-mentioned step S151 and step S152 can be executed in sequence, for example, step S151 is executed first and then step S152, or step S152 is executed first and then step S151. In addition, the above-mentioned step S151 and step S152 can also be executed simultaneously, which is not limited here.
[0074] Step S153: using the first comparison score and the second comparison score, determining whether the face to be identified is a living face.
[0075] Specifically, the sum of the first comparison score and the second comparison score can be used as the final score of the face to be identified. If the final score meets the preset score condition, the face to be identified is determined to be a live face. In one implementation scenario, the preset score condition includes being greater than a preset score threshold. In addition, if the final score does not meet the preset score condition, the face to be identified is determined to be a prosthetic face (for example, a printed face image, an electronic face image, a three-dimensional mask, etc.).
[0076] Different from the aforementioned embodiment, a first comparison score is obtained by comparing the optical flow amplitudes of the eye area and the area around the eyes in the face optical flow map, and a second comparison score is obtained by comparing the optical flow directions of the eye area and the area around the eyes in the face optical flow map, thereby using the first comparison score and the second comparison score to determine whether the face to be identified is a living face. The optical flow amplitude and the optical flow direction can be used simultaneously to determine whether it is a living face, thereby improving the accuracy of recognition.
[0077] See also Figure 8 , Figure 8It is a schematic diagram of the framework of an embodiment of a part recognition device 80 of the present application. The part recognition device 80 includes an image acquisition module 81, a target detection module 82, a data acquisition module 83, an optical flow generation module 84 and a live body determination module 85. The image acquisition module 81 is used to acquire multiple frames of original images captured by the camera device; the target detection module 82 is used to perform target detection on multiple frames of original images to obtain the target area of the part to be recognized in each frame of original image; the data acquisition module 83 is used to use multiple frames of original images to form at least one group of original images, and use the image data of the target area in each group of original images as a group of data to be processed, wherein each group of original images includes at least two frames of original images; the optical flow generation module 84 is used to process at least one group of data to be processed using a trained optical flow map generation network to obtain a corresponding part optical flow map; the live body determination module 85 is used to use the part optical flow map to determine whether the part to be recognized is a live body part.
[0078] The above scheme obtains multiple frames of original images taken by the camera device and performs target detection on the multiple frames of original images to obtain the target area of the part to be identified in each frame of the original image, thereby using the multiple frames of original images to form at least one group of original images, and the image data of the target area in each group of original images is used as a group of images to be processed, and each group of original images includes at least two frames of original images, and then uses the trained optical flow map generation network to process at least one group of data to be processed to obtain the corresponding part optical flow map, so that it is possible to avoid complex calculations and quickly obtain a dense optical flow map, and then use the part optical flow map to accurately and in real time determine whether the part to be identified is a living part.
[0079] In some embodiments, each group of original images includes two adjacent frames of original images, and the part identification device 80 includes a sample acquisition module for acquiring at least one group of sample data and its corresponding sample optical flow map, wherein each group of sample data and its corresponding sample optical flow map are obtained using image data of a target area in a pair of adjacent sample images. The part identification device 80 includes an optical flow prediction module for processing each group of sample data using an optical flow map generation network to obtain multiple predicted optical flow maps corresponding to different network depths of the optical flow map generation network. The part identification device 80 includes a loss determination module for respectively using multiple predicted optical flow maps and sample optical flow maps to obtain multiple loss values. The part identification device 80 includes a parameter adjustment module for adjusting the parameters of the optical flow map generation network using multiple loss values.
[0080] Different from the aforementioned embodiments, at least one group of sample data and its corresponding sample optical flow map are obtained, and each group of sample data and its corresponding sample optical flow map are obtained by using the image data of the target area in a pair of adjacent sample images, and each group of sample data is processed by using the optical flow map generation network to obtain a plurality of predicted optical flow maps corresponding to different network depths of the optical flow map generation network, thereby respectively using the plurality of predicted optical flow maps and the sample optical flow maps to obtain a plurality of loss values, and using the plurality of loss values to adjust the parameters of the optical flow map generation network, thereby accelerating network training.
[0081] In some embodiments, the parameter adjustment module includes a weighted processing submodule, which is used to perform weighted processing on multiple loss values using preset weights corresponding to the multiple loss values to obtain a final loss value of the optical flow map generation network. The parameter adjustment module includes a parameter adjustment submodule, which is used to adjust the parameters of the optical flow map generation network using the final loss value; wherein the numerical values of the preset weights corresponding to the multiple loss values and the network depths corresponding to the multiple loss values are positively correlated or negatively correlated.
[0082] Different from the aforementioned embodiment, multiple loss values are weighted by using preset weights corresponding to the multiple loss values to obtain the final loss value of the optical flow map generation network, and the final loss value is used to adjust the parameters of the optical flow map generation network. The numerical values of the preset weights corresponding to the multiple loss values and the network depths corresponding to the multiple loss values are positively correlated or negatively correlated, which can help speed up the convergence of the network.
[0083] In some embodiments, the optical flow generation module 84 includes a first feature map acquisition submodule, which is used to splice each group of data to be processed to obtain spliced data, and perform feature extraction on the spliced data to obtain a first feature map. The optical flow generation module 84 includes a second feature map acquisition submodule, which is used to perform dimensionality reduction processing on the first feature map to obtain at least one second feature map. The optical flow generation module 84 includes a third feature map acquisition submodule, which is used to fuse the first feature map and at least one second feature map to obtain a third feature map. The optical flow generation module 84 includes a decoding processing submodule, which is used to decode the third feature map to obtain a part optical flow map.
[0084] Different from the aforementioned embodiment, each group of data to be processed is spliced to obtain spliced data, and features are extracted from the spliced data to obtain a first feature map, and the first feature map is subjected to dimensionality reduction processing to obtain at least one second feature map, thereby the first feature map and the at least one second feature map are fused to obtain a third feature map, so that the third feature map can integrate global context information as much as possible, thereby decoding the third feature map, which can improve the accuracy of the obtained part optical flow map, and thus can help improve the accuracy of part recognition.
[0085] In some embodiments, the first feature map acquisition submodule is specifically used to use the feature extraction subnetwork in the optical flow map generation network to extract features from the spliced data to obtain a first feature map; wherein the feature extraction subnetwork includes at least one sequentially connected residual block.
[0086] Different from the aforementioned embodiment, using at least one sequentially connected residual block for feature extraction can help deepen the network depth, thereby effectively improving the depth features.
[0087] In some embodiments, the optical flow map generation network also includes multiple sequentially connected feature fusion sub-networks, and the second feature map acquisition sub-module is specifically used to, for each feature fusion sub-network, use the feature map obtained by processing the previous and adjacent sub-network as the first feature map, and use at least one pooling layer in the feature fusion sub-network to pool the first feature map to obtain at least one second feature map. The third feature map acquisition sub-module includes a feature map acquisition unit, which is used to obtain, for each feature fusion sub-network, a third feature map obtained by processing the second feature fusion sub-network before it, and the third feature map acquisition sub-module includes a feature map splicing unit, which is used to splice the first feature map, at least one second feature map and the acquired third feature map to obtain the third feature map.
[0088] Different from the aforementioned embodiments, the optical flow map generation network also includes multiple sequentially connected feature fusion sub-networks, and for each feature fusion sub-network, the feature map obtained by processing the previous and adjacent sub-network is used as the first feature map, and the first feature map is pooled using at least one pooling layer in the feature fusion sub-network to obtain at least one second feature map, and for each feature fusion sub-network, a third feature map obtained by processing the second previous feature fusion sub-network is obtained, and the first feature map, at least one second feature map and the obtained third feature map are spliced to obtain the third feature map, which can help make the third feature map contain global context information as much as possible, thereby helping to improve the accuracy of the part optical flow map, and further can help to improve the accuracy of subsequent part recognition.
[0089] In some embodiments, the part to be identified is a face to be identified, the part optical flow map is a face optical flow map, the part identification device 80 also includes an eye detection module, which is used to perform eye area detection on the target area to obtain the eye area in the target area, the part identification device 80 also includes an eye periphery detection module, which is used to perform eye periphery detection on the target area to obtain the eye periphery in the target area, and the coverage range of the eye area is larger than the coverage range of the eye periphery, the liveness determination module 85 includes a first comparison score acquisition submodule, which is used to compare the optical flow amplitude of the eye area and the eye periphery in the face optical flow map to obtain a first comparison score, the liveness determination module 85 includes a second comparison score acquisition submodule, which is used to compare the optical flow direction of the eye area and the eye periphery in the face optical flow map to obtain a second comparison score, and the liveness determination module 85 includes a liveness determination submodule, which is used to use the first comparison score and the second comparison score to determine whether the face to be identified is a live face.
[0090] Different from the aforementioned embodiment, a first comparison score is obtained by comparing the optical flow amplitudes of the eye area and the area around the eyes in the face optical flow map, and a second comparison score is obtained by comparing the optical flow directions of the eye area and the area around the eyes in the face optical flow map, thereby using the first comparison score and the second comparison score to determine whether the face to be identified is a living face. The optical flow amplitude and the optical flow direction can be used simultaneously to determine whether it is a living face, thereby improving the accuracy of recognition.
[0091] In some embodiments, the first comparison score acquisition submodule is used to count the target pixels in the periocular area whose optical flow amplitudes meet the preset amplitude conditions, and take the proportion of the target pixels in the eye area as the first comparison score.
[0092] Different from the aforementioned embodiment, by counting the target pixels in the periocular area whose optical flow amplitude meets the preset amplitude condition, and taking the proportion of the target pixels in the eye area as the first comparison score, the first comparison score related to the optical flow amplitude can be accurately obtained.
[0093] In some embodiments, the periocular detection module includes a key point detection submodule, which is used to perform key point detection on the target area to obtain key points in the target area. The periocular detection module includes an area determination submodule, which is used to determine the periocular area in the target area using the key points corresponding to the eyes. The second comparison score acquisition submodule includes a distribution statistics unit, which is used to count the first distribution of the optical flow direction of the key points in the target area, and count the second distribution of the optical flow direction of at least the key points corresponding to the eyes in the periocular area. The second comparison score acquisition submodule includes a similarity determination unit, which is used to determine the similarity between the first distribution and the second distribution, and use the similarity as the second comparison score.
[0094] Different from the aforementioned embodiment, by performing key point detection on the target area, the key points in the target area are obtained, and the key points corresponding to the eyes are used to determine the periocular area in the target area, so as to count the first distribution of the optical flow direction of the key points in the target area, and count the second distribution of the optical flow direction of at least the key points corresponding to the eyes in the periocular area, and determine the similarity between the first distribution and the second distribution, and use the similarity as the second comparison score, so as to accurately obtain the second comparison score related to the optical flow direction.
[0095] In some embodiments, the liveness determination submodule includes a final score determination unit, which is used to use the sum of the first comparison score and the second comparison score as the final score of the face to be identified. The liveness determination submodule includes a score judgment unit, which is used to determine that the face to be identified is a live face when the final score meets a preset score condition.
[0096] Different from the aforementioned embodiment, the sum of the first comparison score and the second comparison score is used as the final score of the face to be identified, and when the final score meets the preset score condition, the face to be identified is determined to be a living face. The optical flow amplitude and optical flow direction can be integrated, thereby improving the accuracy of liveness recognition.
[0097] See also Fig. 9 , Fig. 9 1 is a schematic diagram of a framework of an embodiment of an electronic device 90 of the present application. The electronic device 90 comprises a memory 91 and a processor 92 coupled to each other, the memory 91 stores program instructions, and the processor 92 is used to execute the program instructions to implement the steps in any of the above-mentioned part recognition method embodiments.
[0098] Specifically, the processor 92 is used to control itself and the memory 91 to implement the steps in any of the above-mentioned part identification method embodiments. The processor 92 can also be called a CPU (Central Processing Unit). The processor 92 may be an integrated circuit chip with signal processing capabilities. The processor 92 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 92 can be implemented by multiple integrated circuit chips.
[0099] In this embodiment, the processor 92 is used to obtain multiple frames of original images taken by the camera device; the processor 92 is used to perform target detection on the multiple frames of original images to obtain the target area of the part to be identified in each frame of the original image; the processor 92 is used to use the multiple frames of original images to form at least one group of original images, and use the image data of the target area in each group of original images as a group of data to be processed, wherein each group of original images includes at least two frames of original images; the processor 92 is used to use the trained optical flow map generation network to process at least one group of data to be processed to obtain the corresponding part optical flow map; the processor 92 is used to use the part optical flow map to determine whether the part to be identified is a living part.
[0100] The above scheme obtains multiple frames of original images taken by the camera device and performs target detection on the multiple frames of original images to obtain the target area of the part to be identified in each frame of the original image, thereby using the multiple frames of original images to form at least one group of original images, and the image data of the target area in each group of original images is used as a group of images to be processed, and each group of original images includes at least two frames of original images, and then uses the trained optical flow map generation network to process at least one group of data to be processed to obtain the corresponding part optical flow map, so that it is possible to avoid complex calculations and quickly obtain a dense optical flow map, and then use the part optical flow map to accurately and in real time determine whether the part to be identified is a living part.
[0101] In some embodiments, each group of original images includes two adjacent frames of original images, and the processor 92 is used to obtain at least one group of sample data and its corresponding sample optical flow map, wherein each group of sample data and its corresponding sample optical flow map are obtained using image data of a target area in a pair of adjacent sample images; the processor 92 is used to process each group of sample data using an optical flow map generation network to obtain a plurality of predicted optical flow maps corresponding to different network depths of the optical flow map generation network; the processor 92 is used to respectively use a plurality of predicted optical flow maps and sample optical flow maps to obtain a plurality of loss values; the processor 92 is used to adjust the parameters of the optical flow map generation network using a plurality of loss values.
[0102] Different from the aforementioned embodiments, at least one group of sample data and its corresponding sample optical flow map are obtained, and each group of sample data and its corresponding sample optical flow map are obtained by using the image data of the target area in a pair of adjacent sample images, and each group of sample data is processed by using the optical flow map generation network to obtain a plurality of predicted optical flow maps corresponding to different network depths of the optical flow map generation network, thereby respectively using the plurality of predicted optical flow maps and the sample optical flow maps to obtain a plurality of loss values, and using the plurality of loss values to adjust the parameters of the optical flow map generation network, thereby accelerating network training.
[0103] In some embodiments, the processor 92 is used to perform weighted processing on multiple loss values using preset weights corresponding to the multiple loss values to obtain a final loss value of the optical flow map generation network; the processor 92 is used to adjust the parameters of the optical flow map generation network using the final loss value; wherein the numerical values of the preset weights corresponding to the multiple loss values and the network depths corresponding to the multiple loss values are positively correlated or negatively correlated.
[0104] Different from the aforementioned embodiment, multiple loss values are weighted by using preset weights corresponding to the multiple loss values to obtain the final loss value of the optical flow map generation network, and the final loss value is used to adjust the parameters of the optical flow map generation network. The numerical values of the preset weights corresponding to the multiple loss values and the network depths corresponding to the multiple loss values are positively correlated or negatively correlated, which can help speed up the convergence of the network.
[0105] In some embodiments, the processor 92 is used to perform splicing processing on each group of data to be processed to obtain spliced data, and perform feature extraction on the spliced data to obtain a first feature map; the processor 92 is used to perform dimensionality reduction processing on the first feature map to obtain at least one second feature map; the processor 92 is used to fuse the first feature map and at least one second feature map to obtain a third feature map; the processor 92 is used to decode the third feature map to obtain a part optical flow map.
[0106] Different from the aforementioned embodiment, each group of data to be processed is spliced to obtain spliced data, and features are extracted from the spliced data to obtain a first feature map, and the first feature map is subjected to dimensionality reduction processing to obtain at least one second feature map, thereby the first feature map and the at least one second feature map are fused to obtain a third feature map, so that the third feature map can integrate global context information as much as possible, thereby decoding the third feature map, which can improve the accuracy of the obtained part optical flow map, and thus can help improve the accuracy of part recognition.
[0107] In some embodiments, the processor 92 is used to extract features from the spliced data using a feature extraction subnetwork in the optical flow map generation network to obtain a first feature map; wherein the feature extraction subnetwork includes at least one sequentially connected residual block.
[0108] Different from the aforementioned embodiment, using at least one sequentially connected residual block for feature extraction can help deepen the network depth, thereby effectively improving the depth features.
[0109] In some embodiments, the optical flow map generation network further includes a plurality of sequentially connected feature fusion subnetworks, and the processor 92 is used to
[0110] In some embodiments, the processor 92 is used to, for each feature fusion subnetwork, use the feature map obtained by processing the previous and adjacent subnetwork as the first feature map, and use at least one pooling layer in the feature fusion subnetwork to pool the first feature map to obtain at least one second feature map. The processor 92 is used to, for each feature fusion subnetwork, obtain the third feature map obtained by processing the second previous feature fusion subnetwork; the processor 92 is used to splice the first feature map, at least one second feature map and the obtained third feature map to obtain the third feature map.
[0111] Different from the aforementioned embodiments, the optical flow map generation network also includes multiple sequentially connected feature fusion sub-networks, and for each feature fusion sub-network, the feature map obtained by processing the previous and adjacent sub-network is used as the first feature map, and the first feature map is pooled using at least one pooling layer in the feature fusion sub-network to obtain at least one second feature map, and for each feature fusion sub-network, a third feature map obtained by processing the second previous feature fusion sub-network is obtained, and the first feature map, at least one second feature map and the obtained third feature map are spliced to obtain the third feature map, which can help make the third feature map contain global context information as much as possible, thereby helping to improve the accuracy of the part optical flow map, and further can help to improve the accuracy of subsequent part recognition.
[0112] In some embodiments, the part to be identified is a face to be identified, the part optical flow map is a face optical flow map, the processor 92 is used to perform eye area detection on the target area to obtain the eye area in the target area, and perform periocular area detection on the target area to obtain the periocular area in the target area, wherein the coverage of the eye area is larger than the coverage of the periocular area, the processor 92 is used to compare the optical flow amplitudes of the eye area and the periocular area in the face optical flow map to obtain a first comparison score; and the processor 92 is used to compare the optical flow directions of the eye area and the periocular area in the face optical flow map to obtain a second comparison score; the processor 92 is used to determine whether the face to be identified is a living face using the first comparison score and the second comparison score.
[0113] Different from the aforementioned embodiment, a first comparison score is obtained by comparing the optical flow amplitudes of the eye area and the area around the eyes in the face optical flow map, and a second comparison score is obtained by comparing the optical flow directions of the eye area and the area around the eyes in the face optical flow map, thereby using the first comparison score and the second comparison score to determine whether the face to be identified is a living face. The optical flow amplitude and the optical flow direction can be used simultaneously to determine whether it is a living face, thereby improving the accuracy of recognition.
[0114] In some embodiments, the processor 92 is used to count the target pixels in the periocular area whose optical flow amplitudes meet a preset amplitude condition, and use the proportion of the target pixels in the eye area as the first comparison score.
[0115] Different from the aforementioned embodiment, by counting the target pixels in the periocular area whose optical flow amplitude meets the preset amplitude condition, and taking the proportion of the target pixels in the eye area as the first comparison score, the first comparison score related to the optical flow amplitude can be accurately obtained.
[0116] In some embodiments, processor 92 is used to perform key point detection on the target area to obtain the key points in the target area; processor 92 is used to determine the periocular area in the target area using the key points corresponding to the eyes; processor 92 is used to count a first distribution of optical flow directions of the key points in the target area, and to count a second distribution of optical flow directions of at least the key points corresponding to the eyes in the periocular area; processor 92 is used to determine the similarity between the first distribution and the second distribution, and use the similarity as a second comparison score.
[0117] Different from the aforementioned embodiment, by performing key point detection on the target area, the key points in the target area are obtained, and the key points corresponding to the eyes are used to determine the periocular area in the target area, so as to count the first distribution of the optical flow direction of the key points in the target area, and count the second distribution of the optical flow direction of at least the key points corresponding to the eyes in the periocular area, and determine the similarity between the first distribution and the second distribution, and use the similarity as the second comparison score, so as to accurately obtain the second comparison score related to the optical flow direction.
[0118] In some embodiments, the processor 92 is used to use the sum of the first comparison score and the second comparison score as the final score of the face to be identified; the processor 92 is used to determine that the face to be identified is a living face when it is determined that the final score meets a preset score condition.
[0119] Different from the aforementioned embodiment, the sum of the first comparison score and the second comparison score is used as the final score of the face to be identified, and when the final score meets the preset score condition, the face to be identified is determined to be a living face. The optical flow amplitude and optical flow direction can be integrated, thereby improving the accuracy of liveness recognition.
[0120] See also Fig.10 , Fig.10 The schematic diagram of the framework of an embodiment of the storage device 100 of the present application. The storage device 100 stores program instructions 101 that can be executed by a processor, and the program instructions 101 are used to implement the steps in any of the above-mentioned embodiments of the part recognition method.
[0121] The above scheme can accurately and in real time determine whether the part to be identified is a living part.
[0122] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0123] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0124] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0125] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.
Claims
1. A part recognition method, characterized in that: include: Acquire multiple frames of original images captured by the camera device; Performing target detection on the multiple frames of original images to obtain a target area of a part to be identified in each frame of the original image; Using the multiple frames of original images to form at least one group of original images, and using the image data of the target area in each group of the original images as a group of data to be processed, wherein each group of the original images includes at least two frames of original images; Using a trained optical flow map generation network to process at least one set of the data to be processed to obtain a corresponding part optical flow map; Determining whether the part to be identified is a living part by using the part optical flow map; Wherein, each group of the original images includes two adjacent frames of the original images; Before the at least one group of the to-be-processed data is processed using the trained optical flow map generation network to obtain the corresponding part optical flow map, the method further includes: Acquire at least one set of sample data and its corresponding sample optical flow map, wherein each set of sample data and its corresponding sample optical flow map is obtained by using image data of a target area in a pair of adjacent sample images; Using the optical flow map generation network to process each group of the sample data, to obtain a plurality of predicted optical flow maps corresponding to different network depths of the optical flow map generation network for each group of the sample data; Using the multiple predicted optical flow maps and the sample optical flow map respectively, to obtain multiple loss values; Performing weighted processing on the multiple loss values using preset weights corresponding to the multiple loss values to obtain a final loss value of the optical flow map generation network; Using the final loss value, adjusting the parameters of the optical flow map generation network; Among them, in the early stage of training, the preset weights corresponding to the multiple loss values are set to be positively correlated with the network depth, and in the late stage of training, the preset weights corresponding to the multiple loss values are set to be negatively correlated with the network depth.
2. The part recognition method according to claim 1, characterized in that: The step of processing at least one set of the data to be processed by using the trained optical flow map generation network to obtain a corresponding part optical flow map comprises: Performing splicing processing on each group of the data to be processed to obtain spliced data, and performing feature extraction on the spliced data to obtain a first feature map; Performing dimensionality reduction processing on the first feature map to obtain at least one second feature map; Fusing the first feature map and the at least one second feature map to obtain a third feature map; The third feature map is decoded to obtain the part optical flow map.
3. The part recognition method according to claim 2, characterized in that: The performing feature extraction on the spliced data to obtain a first feature map comprises: Using the feature extraction subnetwork in the optical flow map generation network to extract features from the spliced data, to obtain a first feature map; Wherein, the feature extraction subnetwork includes at least one sequentially connected residual block.
4. The part recognition method according to claim 2, characterized in that: The optical flow map generation network also includes a plurality of sequentially connected feature fusion sub-networks; The performing dimensionality reduction processing on the first feature map to obtain at least one second feature map comprises: For each of the feature fusion sub-networks, a feature map obtained by processing a previous and adjacent sub-network is used as the first feature map, and the first feature map is pooled using at least one pooling layer in the feature fusion sub-network to obtain the at least one second feature map; The fusing the first feature map and the at least one second feature map to obtain a third feature map comprises: For each of the feature fusion sub-networks, obtaining a third feature map processed by the second feature fusion sub-network located before the feature fusion sub-network; The first feature map, the at least one second feature map and the acquired third feature map are concatenated to obtain the third feature map.
5. The part recognition method according to claim 1, characterized in that: The part to be identified is a face to be identified, and the part optical flow map is a face optical flow map; before using the part optical flow map to determine whether the part to be identified is a living part, the method further includes: Performing eye region detection on the target area to obtain an eye region in the target area, and performing eye periorbital region detection on the target area to obtain an eye periorbital region in the target area, wherein a coverage range of the eye region is greater than a coverage range of the eye periorbital region; The determining whether the part to be identified is a living part by using the part optical flow map comprises: Compare the optical flow amplitudes of the eye area and the eye periphery area in the face optical flow map to obtain a first comparison score; and, Comparing the optical flow directions of the eye area and the eye periphery area in the face optical flow map to obtain a second comparison score; Using the first comparison score and the second comparison score, determine whether the face to be identified is a living face.
6. The part recognition method according to claim 5, characterized in that: The comparing the optical flow amplitudes of the eye area and the eye periphery area in the face optical flow map to obtain a first comparison score includes: Count the target pixels in the periocular area where the optical flow amplitude meets a preset amplitude condition, and use the proportion of the target pixels in the eye area as the first comparison score.
7. The part recognition method according to claim 5, characterized in that: The detecting of the eye region around the target area to obtain the eye region around the target area includes: Performing key point detection on the target area to obtain key points in the target area; Determine the eye area in the target area by using key points corresponding to the eyes; The comparing the optical flow directions of the eye area and the eye periphery area in the face optical flow map to obtain a second comparison score includes: Counting a first distribution of optical flow directions of key points in the target area, and counting a second distribution of optical flow directions of key points in the periocular area at least corresponding to the eyes; The similarity between the first distribution and the second distribution is determined, and the similarity is used as the second comparison score.
8. The part recognition method according to claim 5, characterized in that: The determining whether the to-be-recognized face is a live face by using the first comparison score and the second comparison score includes: Taking the sum of the first comparison score and the second comparison score as the final score of the face to be recognized; If the final score meets the preset score condition, it is determined that the face to be identified is a living face.
9. An electronic device, characterized in that: The invention comprises a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the part recognition method according to any one of claims 1 to 8.
10. A storage device, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the part recognition method described in any one of claims 1 to 8.
