Face information detection method and terminal
By obtaining the face detection frame and size, matching different distance features, and combining with MobileNet and FPN networks, the problem of low long-distance facial recognition accuracy is solved, high-accuracy recognition at different distances is achieved, and training time and space is reduced Occupancy.
Patent Information
- Application Number
- CN202411057949.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-12-10
AI Technical Summary
The existing facial recognition algorithm has low recognition accuracy at long distances, which affects the recognition effect, and requires additional training to increase operation time for long-distance scenarios.
By obtaining the face detection frame and size in the image to be tested, the face characteristics to be tested match the stored data face characteristics, and the feature information is stored according to the target distance that matches successfully, and the features are detected in combination with MobileNet and FPN networks. Reduce processing time and improve accuracy.
There is no need to retrain the model, improve the accuracy of face recognition at different distances, especially long distances, and take into account both close and long distance characteristics, reducing the impact of distance factors on the recognition results.
Smart Images
Figure CN119107677A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of face verification, and in particular to a face information detection method and terminal. Background Art
[0002] Face recognition algorithms are now widely used in various scenarios, such as station security checks and access control. For these applications, face acquisition requires ensuring that the machine and the target person are within a certain distance (usually no more than 2 meters). For long-distance face recognition, such as face recognition in surveillance, directly using the algorithms used in security checks and access control will greatly reduce the accuracy and affect the recognition effect. Special training for long-distance scenarios will increase the operation time. Summary of the invention
[0003] The technical problem to be solved by the present invention is to provide a face information detection method and a terminal to improve the recognition accuracy of face images at different distances.
[0004] In order to solve the above technical problems, a technical solution adopted by the present invention is:
[0005] A method for detecting face information, comprising the steps of:
[0006] Acquire an image to be tested, and acquire a face detection frame and a size of the face detection frame in the image to be tested;
[0007] Acquire the face features to be tested according to the face detection frame, and match the face features to be tested with the stored data face features, obtain the identity information corresponding to the successfully matched target data face features and return it;
[0008] The target distance corresponding to the size is obtained, and the target distance, the facial features to be measured, and the identity information are associated and stored.
[0009] In order to solve the above technical problems, another technical solution adopted by the present invention is:
[0010] A face information detection terminal comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:
[0011] Acquire an image to be tested, and acquire a face detection frame and a size of the face detection frame in the image to be tested;
[0012] Acquire the face features to be tested according to the face detection frame, and match the face features to be tested with the stored data face features, obtain the identity information corresponding to the successfully matched target data face features and return it;
[0013] The target distance corresponding to the size is obtained, and the target distance, the facial features to be measured, and the identity information are associated and stored.
[0014] The beneficial effects of the present invention are as follows: a face detection frame and its size are obtained in an image to be detected, face features to be detected are obtained according to the face detection frame and matched in stored data face features, if the match is successful, a target distance corresponding to the size is obtained, the target distance, face features to be detected and matched identity information are stored in association, the stored data face features include features of different target distances, so that different features of close distances and long distances can be taken into account when matching faces, and there is no need to retrain the model, thereby improving the accuracy of face recognition at different distances, especially at long distances. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 A flowchart of a method for detecting facial information according to an embodiment of the present invention;
[0016] Figure 2 Another flow chart of a method for detecting face information according to an embodiment of the present invention;
[0017] Figure 3 This is a schematic diagram of the structure of a face information detection terminal according to an embodiment of the present invention;
[0018] Description of labels:
[0019] 1. A face information detection terminal; 2. A processor; 3. A memory. DETAILED DESCRIPTION
[0020] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.
[0021] Please refer to Figure 1 , a face information detection method, comprising the steps of:
[0022] Acquire an image to be tested, and acquire a face detection frame and a size of the face detection frame in the image to be tested;
[0023] Acquire the face features to be tested according to the face detection frame, and match the face features to be tested with the stored data face features, obtain the identity information corresponding to the successfully matched target data face features and return it;
[0024] The target distance corresponding to the size is obtained, and the target distance, the facial features to be measured, and the identity information are associated and stored.
[0025] From the above description, it can be seen that the beneficial effects of the present invention are: obtaining a face detection frame and its size in the image to be detected, obtaining the face features to be detected based on the face detection frame and matching them in the stored data face features, if the match is successful, obtaining the target distance corresponding to the size, and storing the target distance, the face features to be detected and the matched identity information in association, then the stored data face features include features of different target distances, so that different features of close distances and long distances can be taken into account when matching faces, and there is no need to retrain the model, thereby improving the accuracy of face recognition at different distances, especially at long distances.
[0026] Further, the acquiring of a face detection frame and a size of the face detection frame in the image to be detected includes:
[0027] Input the image to be tested into a face detection and calibration model to obtain an initial face detection frame and face key points;
[0028] Verify the initial face detection frame according to the face key points, and use the verified initial face detection frame as the face detection frame;
[0029] Get the size of the face detection frame.
[0030] From the above description, it can be seen that when obtaining the face detection frame of the image to be tested, the face key points are also obtained, so the position of the face detection frame can be verified and adjusted according to the face key points, further improving the accuracy of the face detection frame marking.
[0031] Further, the checking the initial face detection frame according to the face key points includes:
[0032] Determine whether the initial face detection frame includes the face key points, if so, pass the verification, otherwise, fail the verification.
[0033] From the above description, it can be seen that it is necessary to determine whether the face detection frame includes the face key points. If so, it means that the position of the face detection frame is correct and the subsequent steps can be continued. Otherwise, it means that the mark of the face detection frame is offset or incorrect, and the corresponding face detection frame needs to be deleted.
[0034] Furthermore, matching the facial features to be tested with the stored data facial features, obtaining the identity information corresponding to the successfully matched target data facial features and returning the information includes:
[0035] It is determined whether there is a target data face feature in the data face feature whose matching degree with the face feature to be tested is greater than a matching threshold. If so, the match is successful.
[0036] From the above description, it can be seen that by setting the corresponding matching threshold, the target data facial features and the target data facial features whose matching degree is greater than the matching threshold are considered to be successfully matched. This can be flexibly set based on the balance between accuracy and successful matching rate, further improving the user experience when performing face verification.
[0037] Further, the acquiring of a face detection frame and a size of the face detection frame in the image to be detected includes:
[0038] Performing convolution and pooling operations on the image to be tested through a MobileNet network to obtain feature maps of different scales;
[0039] Using the feature maps of different scales as the first feature map of the FPN network;
[0040] Obtain a maximum feature map with the largest scale in the MobileNet network, and upsample the maximum feature map to obtain a sampled feature map;
[0041] Performing feature fusion on the sampled feature map and the part with the same scale as the first feature map to obtain a second feature map;
[0042] Transversely connecting the first feature map and the second feature map to obtain a third feature map;
[0043] An initial face detection frame and the face key points are obtained in the image to be tested according to the first feature map, the second feature map and the third feature map.
[0044] From the above description, it can be seen that combining the MobileNet network and the FPN network to obtain feature maps of different scales can better detect facial features of different sizes and improve the accuracy of face detection frames and face key points; and face detection and verification can be completed in one network model, which reduces the processing time of face data and improves the calculation efficiency compared to the algorithm that implements face detection and verification separately, and verifies the face detection frame through face key points to improve the accuracy of face recognition; and the three key feature layers of FPN can be fused to detect targets of different scales and better distinguish feature matches at different distances, and the MobileNet network is a lightweight network that can speed up detection and recognition time.
[0045] Further, the obtaining of the target distance corresponding to the size and the storage of the target distance, the facial features to be measured and the identity information in association with each other include:
[0046] Obtaining a target distance corresponding to the size, and determining whether there is a stored face feature corresponding to the target distance in the face feature data corresponding to the identity information;
[0047] If not, the target distance, the facial features to be measured and the identity information are stored in association.
[0048] From the above description, it can be seen that when storing the facial features to be tested, it is determined whether there is data for the target distance corresponding to the facial features to be tested. If no data exists, it is stored. This avoids the problem of occupying space when storing multiple facial features to be tested at the same target distance. If the corresponding target distance already exists, it does not need to be stored, saving interaction time.
[0049] Further, the determining whether there is a stored facial feature corresponding to the target distance in the facial feature data corresponding to the identity information includes:
[0050] If there is a stored facial feature corresponding to the target distance, the stored facial feature is updated according to the facial feature to be measured.
[0051] From the above description, it can be seen that if there are stored facial features corresponding to the target distance, the stored facial features will be updated. In this way, even if the facial features to be tested continue to change compared to the facial features of the data at the time of entry, the features in the model can be updated automatically to follow up on the continuous changes, thereby further improving the accuracy of the model's judgment results.
[0052] Further, the data facial features include infrared facial features;
[0053] Before matching the facial features to be tested with the facial features of stored data, the method includes:
[0054] Adjust the grayscale value feature of the registered face image to the infrared mode to obtain an infrared face image;
[0055] Acquire infrared facial features in the infrared facial image and store them in association with identity information;
[0056] The matching of the face feature to be tested with the face feature of the stored data includes:
[0057] It is determined whether the image to be tested is an image acquired in the infrared mode. If so, the facial features to be tested are matched with the stored infrared facial features.
[0058] From the above description, it can be seen that when performing face registration, that is, binding the initial identity information with the data facial features, if the infrared image under the infrared mode is obtained specifically, the registration efficiency will be reduced. Therefore, the method of adjusting the grayscale value to simulate the infrared image is adopted here. Not only the facial features under normal lighting but also the infrared facial features under the infrared mode are stored in the database. When judging the image to be tested obtained under the infrared mode, it is only matched with the infrared facial features, which improves the matching speed and accuracy.
[0059] Furthermore, the storing the target distance, the facial features to be measured and the identity information in association with each other includes:
[0060] The target distance, the facial features to be measured and the identity information are associated and stored in an XML file.
[0061] From the above description, it can be seen that storage through XML files has good scalability, and the content and form are separated from each other, which is convenient for subsequent retrieval and other operations.
[0062] Please refer to Figure 3 A face information detection terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, each step in the above-mentioned face information detection method is implemented.
[0063] The above-mentioned facial information detection method and terminal of the present invention can be applied to scenarios where faces at different distances need to be detected, such as monitoring, business robots, and gates, etc., and are described below through specific implementation methods. In the present invention, the collection of facial images has obtained personal consent and is only used for legal purposes, or a prominent reminder sign for facial image collection is set, and the collection of facial images is only used for legal purposes.
[0064] Please refer to Figure 1-2 , Embodiment 1 of the present invention is:
[0065] A method for detecting face information, comprising the steps of:
[0066] S0. Face registration: obtaining a registration image and identity information associated with the registration image, obtaining data face features according to the registration image, and storing the identity information and the data face features in an associated manner;
[0067] In an optional implementation, the identity information and the data face features are stored in an XML file;
[0068] In an optional implementation, the method further includes: adjusting the grayscale value feature of the registered face image to an infrared mode to obtain an infrared face image; obtaining infrared face features in the infrared face image, and storing them in association with the identity information; in this way, there is no need to obtain face data twice during the registration process;
[0069] In an optional implementation, during the face registration process, the face features of the data may be obtained by the method in step S1, that is, feature maps of three different scales may be obtained as features for face registration;
[0070] S1, obtaining an image to be tested, and obtaining a face detection frame and a size of the face detection frame in the image to be tested, including:
[0071] S11, inputting the image to be tested into a face detection and calibration model to obtain an initial face detection frame and face key points, including:
[0072] S111, performing convolution and pooling operations on the image to be tested through a MobileNet network to obtain feature maps of different scales;
[0073] S112, using the feature maps of different scales as the first feature map of the FPN network;
[0074] S113, obtaining a maximum feature map with the largest scale in the MobileNet network, and upsampling the maximum feature map to obtain a sampled feature map;
[0075] S114, performing feature fusion on the sampled feature map and the part of the first feature map having the same scale as each other to obtain a second feature map;
[0076] S115, horizontally connecting the first feature map and the second feature map to obtain a third feature map;
[0077] S116, obtaining an initial face detection frame and face key points in the image to be tested according to the first feature map, the second feature map and the third feature map; wherein the first feature map is directly extracted from the image to be tested, corresponding to the medium-distance feature; the second feature map is a target with a larger scale highlighted after upsampling, corresponding to the close-range feature; the third layer splices the first feature map and the second feature map to enrich the included features, corresponding to the long-distance feature for very large or very blurred targets;
[0078] In an optional implementation, the method further includes: setting the loss function to Multi-task Loss, which includes:
[0079] Loss Function
[0080]
[0081]
[0082]
[0083] (1)L cls Classification loss Face classification loss:
[0084] L cls (p i , p i *) Among them, p i Indicates the probability that the predicted i-th anchor (face detection box) is a face.i * represents the true label. When the anchor is a positive sample (i.e., it contains a face), the value is 1, and when it is a negative sample, the value is 0;
[0085] (2)L box Face box regression loss:
[0086]
[0087] λ1 represents the balancing coefficient to reduce the impact of bounding box regression loss on the total loss;
[0088] represents the jth coordinate representation of the predicted i-th face detection box (for example, the bounding box coordinates include the center point coordinates, width and height, then j can be the center point coordinates, width or height); Represents the jth coordinate representation of the true i-th face detection box; Smooth L1 loss is used to calculate the bounding box regression loss. Smooth L1 loss is a combination of L1 loss and L2 loss. It is expressed as L2 loss (square loss) when the error is small, and as L1 loss (absolute value loss) when the error is large;
[0089] (3)L pts LossFacial landmark regression loss
[0090] Represents the lth predicted coordinate representation of the kth facial key point of the i-th face detection frame (for example, the coordinate representation of facial key points includes eye center, nose tip, mouth corner, etc.; then l can correspond to eye center, nose tip or mouth corner, etc.);
[0091] Represents the lth true coordinate representation of the kth facial key point of the i-th face detection frame;
[0092] λ2: balance coefficient, set to a value slightly larger than λ1, for example, set to a value within 0.1 larger than λ1; to increase the importance of facial key point regression loss; in an optional implementation, λ1 is set to 0.1 and λ2 is set to 0.2;
[0093] In an optional implementation manner, before S11, S10 is further included, preprocessing the image to be tested, such as modifying the image size, converting the image format, etc., so that it can be used as an input of the model; then S11 is performed based on the preprocessed image to be tested;
[0094] S12, verifying the initial face detection frame according to the face key points, and using the verified initial face detection frame as the face detection frame, including: determining whether the initial face detection frame includes the face key points, if so, passing the verification, otherwise, failing the verification;
[0095] In an optional implementation, the method further includes: rotating the image to be tested to a preset angle according to the key points of the face, so that the angle of the face in the face detection frame is close to the angle of the face in the registered image; for example, five key points of the face may be obtained to correct the angle;
[0096] S13, obtaining the size of the face detection frame;
[0097] In an optional implementation, the picture is acquired by a camera;
[0098] S2, obtaining the face features to be tested according to the face detection frame, and matching the face features to be tested with the face features of the stored data;
[0099] In an optional implementation, the image to be tested is cropped according to the face detection frame, and the cropped image to be tested is input into the face recognition model to obtain the face features to be tested; the face features to be tested are vectors of length 512; if the networks are the same, the length of the vector remains unchanged;
[0100] S3, obtaining the identity information corresponding to the successfully matched target data facial features and returning it, including: judging whether there is a target data facial feature in the data facial features whose matching degree of the tested facial features is greater than the matching threshold, if so, the match is successful;
[0101] In an optional implementation, the matching threshold is 0.6; if the accuracy requirement is high, a value greater than 0.6, such as 0.65, 0.8, etc., may also be set;
[0102] The matching will output a matching degree between 0 and 1. The matching degree indicates the similarity between the facial features of the tested person and the facial features of the data. The greater the matching degree, the greater the similarity between the two, and the more likely they are to correspond to the same identity information.
[0103] S4, obtaining a target distance corresponding to the size, and storing the target distance, the facial features to be measured, and the identity information in association, including:
[0104] S41, obtaining the target distance corresponding to the size, and determining whether there is a stored face feature corresponding to the target distance in the face feature data corresponding to the identity information; if so, executing S42 or S43; otherwise, executing S44;
[0105] S42, jump to execute S1;
[0106] S43, updating the stored facial features according to the facial features to be tested;
[0107] S44, storing the target distance, the facial features to be measured, and the identity information in association, including: storing the target distance, the facial features to be measured, and the identity information in association in an XML file;
[0108] When the facial features to be tested have successfully matched the facial features of the target data, two steps are performed: one is to directly output the matching identity information; the other is to determine whether the current facial features to be tested need to be stored. Whether to store the current facial features to be tested requires checking the size of the verification face, that is, the face detection frame. If the size of the face detection frame falls within a certain range, check whether the database already has facial feature information in this range. If not, insert it into the database to fill in feature representations at different distances. Otherwise, the features are not stored or the existing features are updated. This ensures that facial information at different distances is basically saved in the database. When matching faces at different distances, features at different distances can be obtained in the database and matched with the input image features. This ensures the matching rate of faces at different distances, thereby improving the recognition accuracy. If the matching degree is less than the matching threshold, it means that no facial information is matched, that is, the input face cannot be identified;
[0109] In an optional implementation, since the size of each face detection frame cannot be exactly the same, if all are stored, it will take up too much space. Therefore, the corresponding facial feature information is classified into close-range features, medium-range features and long-range features according to the size range of the face detection frame, and each type of facial feature information is stored in one copy, thereby further saving storage space under the premise of ensuring the matching effect; further, a matching score is set, and if during the detection process, the matching score is higher than the matching score of the stored facial feature information, the facial features of all data corresponding to the distance in the database are updated, thereby ensuring the accuracy of recognition under different lighting and changes in face angles;
[0110] Among them, the matching score is obtained according to the output result of the model. After the test picture is input into the model, the model first determines whether there is a face in the picture. At this time, a probability of 0.9 that there is a face in the face detection frame of the test picture will be obtained, and the probability will be recorded as the matching score; if the test face feature in the face detection frame is finally matched to the corresponding identity information, the matching score is compared with the matching score of the corresponding data face feature. If it is larger, for example, the matching score of the data face feature is 0.8, then all data face features stored under the corresponding identity information are updated according to the face detection frame in the test picture, for example, the face detection frame is used as the registered image to re-extract the data face feature.
[0111] Please refer to Figure 3, Embodiment 2 of the present invention is:
[0112] A face information detection terminal 1 includes a processor 2, a memory 3, and a computer program stored in the memory 3 and executable on the processor 2. When the processor 2 executes the computer program, each step in the first embodiment is implemented.
[0113] In summary, the present invention provides a face information detection method and terminal, which simultaneously obtain an initial face detection frame and face key points through a model, correct the initial face detection frame through the face key points, obtain face features through the corrected face detection frame, match the acquired face features to be tested with the stored data face features, output matched identity information, and obtain the corresponding distance according to the size of the face detection frame, store face features at different distances, and then when comparing with the data face features, directly match with features at different target distances, thereby improving the matching accuracy, and well solving the problem of low accuracy of face recognition at different distances, especially long-distance face recognition, and reducing the influence of distance factors on face recognition results during face recognition.
[0114] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A method for detecting face information, characterized in that: Includes steps: Acquire an image to be tested, and acquire a face detection frame and a size of the face detection frame in the image to be tested; Acquire the face features to be tested according to the face detection frame, and match the face features to be tested with the stored data face features, obtain the identity information corresponding to the successfully matched target data face features and return it; The target distance corresponding to the size is obtained, and the target distance, the facial features to be measured, and the identity information are associated and stored.
2. A facial information detection method according to claim 1, characterized in that: The step of obtaining a face detection frame and a size of the face detection frame in the image to be detected comprises: Input the image to be tested into a face detection and calibration model to obtain an initial face detection frame and face key points; Verify the initial face detection frame according to the face key points, and use the verified initial face detection frame as the face detection frame; Get the size of the face detection frame.
3. A facial information detection method according to claim 2, characterized in that: The checking the initial face detection frame according to the face key points comprises: Determine whether the initial face detection frame includes the face key points, if so, pass the verification, otherwise, fail the verification.
4. A facial information detection method according to claim 1, characterized in that: The step of matching the facial features to be tested with the facial features of the stored data, obtaining the identity information corresponding to the facial features of the successfully matched target data and returning the information includes: It is determined whether there is a target data face feature in the data face feature whose matching degree with the face feature to be tested is greater than a matching threshold. If so, the match is successful.
5. The method for detecting face information according to claim 1, characterized in that: The step of obtaining a face detection frame and a size of the face detection frame in the image to be detected comprises: Performing convolution and pooling operations on the image to be tested through a MobileNet network to obtain feature maps of different scales; Using the feature maps of different scales as the first feature map of the FPN network; Obtain a maximum feature map with the largest scale in the MobileNet network, and upsample the maximum feature map to obtain a sampled feature map; Performing feature fusion on the sampled feature map and the part with the same scale as the first feature map to obtain a second feature map; Transversely connecting the first feature map and the second feature map to obtain a third feature map; The sizes of an initial face detection frame and face key points are obtained in the image to be tested according to the first feature map, the second feature map and the third feature map.
6. A facial information detection method according to claim 1, characterized in that: The obtaining of the target distance corresponding to the size and the storage of the target distance, the facial features to be measured and the identity information in association with each other include: Obtaining a target distance corresponding to the size, and determining whether there is a stored face feature corresponding to the target distance in the face feature data corresponding to the identity information; If not, the target distance, the facial features to be measured and the identity information are stored in association.
7. A facial information detection method according to claim 6, characterized in that: The step of determining whether there is a stored facial feature corresponding to the target distance in the facial feature data corresponding to the identity information comprises: If there is a stored facial feature corresponding to the target distance, the stored facial feature is updated according to the facial feature to be measured.
8. A facial information detection method according to claim 1, characterized in that: The data facial features include infrared facial features; Before matching the facial features to be tested with the facial features of stored data, the method includes: Adjust the grayscale value feature of the registered face image to the infrared mode to obtain an infrared face image; Acquire infrared facial features in the infrared facial image and store them in association with identity information; The matching of the face feature to be tested with the face feature of the stored data includes: It is determined whether the image to be tested is an image acquired in the infrared mode. If so, the facial features to be tested are matched with the stored infrared facial features.
9. A facial information detection method according to claim 1, characterized in that: The storing of the target distance, the facial features to be measured and the identity information in association with each other includes: The target distance, the facial features to be measured and the identity information are associated and stored in an XML file.
10. A face information detection terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, each step of the face information detection method described in any one of claims 1-9 is implemented.