Face position matching method, system and computer equipment based on dual-camera system

Through adaptive online learning and homography matrix correction mapping, the problem of face position matching error in dual-camera systems is solved, accurate matching at different distances and resolutions is achieved, and the security and convenience of face recognition terminals are improved. sex.

CN114140840BActive Publication Date: 2024-11-19RECONOVA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111092743.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-17
Publication Date
2024-11-19
Estimated Expiration
2041-09-17

AI Technical Summary

Technical Problem

There is an error in the matching of face position in the dual-camera system. Especially at different distances, the offset compensation method of manual calibration cannot adapt to various situations, resulting in a reduction in the security of face recognition equipment and a decrease in user convenience.

Method used

Adaptive online learning method is adopted to generate anchor frame groups under different face sizes, obtain images of the dual-camera system and calculate the homography matrix, so as to achieve correction mapping and IOU matching of face frames to ensure that at any resolution and Accurate match at distance.

Benefits of technology

It improves the accurate matching of faces under dual camera lenses, enhances the security of face recognition terminals, avoids manual calibration errors, simplifies the use process, and improves production efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114140840B_ABST
    Figure CN114140840B_ABST
Patent Text Reader

Abstract

The present invention provides a face position matching method, system and computer equipment based on a dual-camera system, the method comprising: generating an anchor frame group for storing homography matrices at different face size scales based on the resolution size of a first imaging system; simultaneously acquiring a first image generated by the first imaging system and a second image generated by a second imaging system, identifying the number of faces detected by the first image and the second image, and if both include a face, finding the optimal matching anchor frame based on the face position in the first image, and performing adaptive online learning calibration and recording on the homography matrix of the corresponding position of the optimal matching anchor frame; if more than one is included, mapping and matching association of the face position based on the homography matrix. The present invention can accurately match and associate dual-camera imaging faces at any resolution and any distance, thereby improving the security of face recognition terminals and solving the problem that faces are easily mismatched when the dual-camera system detects multiple faces.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of smart security terminal technology, and in particular to a face position matching method, system and computer equipment based on a dual-camera system. Background Art

[0002] With the rise of face recognition technology, face recognition terminal devices are being introduced into more and more life and work scenarios. Face recognition technology can provide convenience for life, but the security risks during use are also rising quietly. Criminals will use the owner's face pictures stolen from the Internet to attack face recognition terminals. In order to more effectively deal with various fake face attacks, the device development end often adds multiple cameras for face liveness detection, such as binocular infrared modules, binocular visible light modules, and dual-camera visible light plus infrared modules.

[0003] Although multi-camera face recognition equipment can improve the security of face recognition, it will also introduce a new problem: the problem of matching the face position between the two cameras. Taking the dual-camera visible light plus infrared as an example, the face image under the infrared camera is used for face liveness detection, and the face image under the visible light camera is clear for face recognition comparison. Because the internal parameters of the dual-camera lenses are different, and there will be a certain angle between the optical axis directions of the two lenses during the installation process. This may cause: the overlap between the infrared face position and the visible light face position changes with the distance, and even at certain distances, the infrared face and the visible light face have no overlap at all. In order to make face matching easier and more accurate during use, developers often measure the x-direction offset and y-direction offset of the face frame of the same face under the left and right lens imaging at a suitable distance. The face of a certain picture actually needs to be detected with the offset in the x and y directions before the face frame calculation overlap is matched. However, this simple calibration offset compensation matching method has huge limitations. The face position relationship between the two images at different distances in the dual-camera system changes, that is, the x and y offsets of the face at different distances change. Therefore, it is not possible to calibrate the offset of only one position to compensate for all face positions.

[0004] like Figure 1As shown, the first row of pictures are pictures with manual calibration offset. The face position of the left picture is (191, 47, 185, 185), and the face position of the right picture is (238, 26, 175, 175). The offset calibration is based on the upper left corner of the left picture. That is, the face x of the current left picture needs to be offset to the right by 47 pixels, and the face y needs to be offset upward by 21 pixels so that the two faces can coincide and match. The second row of pictures is obtained by directly correcting the position of the infrared frame based on the offset result of the first row of pictures. The left picture detects a real blue frame (188, 88, 68, 68) , after the previously calibrated x and y direction offset correction, the position of the green frame in the right picture is obtained (235,67,68,68), the mobile phone face frame in the right picture is (226,89,58,58) and the real person frame is (290,86,63,63); at this time, it is obvious that the infrared face frame detected in the left picture after offset calibration and correction matches the mobile phone face frame in the visible light image, indicating that the practice of manually measuring the face offset of the dual-camera device at a certain distance as the face offset correction at all distances has obvious loopholes. If the infrared plus visible light dual-camera device only uses infrared images for face liveness detection and uses visible light images for face comparison and recognition, then the face recognition device will be attacked because of the wrong face matching of the dual-camera lens. From another perspective, if there are multiple real people in both the left and right images, the real face in the infrared registered database on the left image matches the real face in the visible light unregistered database on the right image. In this case, the liveness judgment is passed, but the recognition is not passed because the face match is inaccurate, resulting in the person who should have passed being unable to pass. Moreover, this method of manually calibrating the offset parameters of each device reduces the convenience of user use. Therefore, the face position matching of the two lenses in the dual-camera system is very critical. In addition, it is also very important that the matching process can adapt to various devices without manual calibration. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a face position matching method, system and computer device based on a dual-camera system with adaptive online learning, which can accurately match and associate dual-camera imaged faces at any resolution and any distance, thereby improving the security of face recognition terminals, solving the problem that faces are easily mismatched when the dual-camera system detects multiple faces, and avoiding the matching error problem caused by manually preset fixed offsets.

[0006] The present invention adopts the following technical solution:

[0007] On the one hand, a face position matching method based on a dual-camera system comprises the following steps:

[0008] S101, generating a plurality of anchor frames at different face size scales based on the resolution size of a first imaging system of the dual-camera system, wherein the plurality of anchor frames form an anchor frame group, and each anchor frame is used to store a homography matrix;

[0009] S102, simultaneously acquiring a first image generated by a first imaging system and a second image generated by a second imaging system of a dual-camera system, identifying the number of faces detected by the first image and the second image, and if both images include only one face, proceeding to S103; otherwise, proceeding to S104;

[0010] S103, finding the best matching anchor frame based on the face position in the first image, determining whether a homography matrix of the position corresponding to the best matching anchor frame exists, and if not, obtaining the homography matrix based on the correspondence between the feature points of the face in the first image and the feature points of the face in the second image, and storing the matrix in the position corresponding to the best matching anchor frame;

[0011] S104, taking an original face detected in the first image as a reference, finding the best matching anchor frame under the anchor frame group, and determining whether a homography matrix of a position corresponding to the best matching anchor frame exists; if so, based on the homography matrix, correcting and mapping the face frame corresponding to the original face to an imaging surface of a second imaging system to obtain a corrected face frame, and performing IOU matching between the corrected face frame and the face frame in the second image in turn, and obtaining a face in the second image corresponding to the maximum IOU value as a matching face.

[0012] Preferably, the S104 further includes:

[0013] If the homography matrix of the position corresponding to the optimal matching anchor frame does not exist, the original face frame and the face frame in the second image are sequentially subjected to IOU matching, and the face in the second image corresponding to the maximum IOU value is obtained as the matching face.

[0014] Preferably, after S104, the step further includes:

[0015] If the original face frames detected in the first image include multiple ones, S104 is repeated until all the original face frames detected in the first image are matched.

[0016] Preferably, in S101, based on the resolution size of the first imaging system of the dual-camera system, generating multiple anchor frames at different face size scales specifically includes:

[0017] Based on the resolution size of the first imaging system of the dual-camera system, multiple anchor frames of different face sizes are generated through a pyramid scaling factor.

[0018] Preferably, the size S of each anchor box is as follows:

[0019] S = min_size / (factor n )

[0020] Among them, min_size represents the minimum scale face; max_size represents the minimum scale face; factor represents the pyramid scaling factor; n = 0, 1, 2...; when min_size / (factor n )>max_size, no anchor box is generated;

[0021] The number of anchor boxes M at a certain scale is as follows:

[0022] M=(W / (min_size / (factor n ))+1)*(H / (min_size / (factor n ))+1)

[0023] Wherein, W represents that the resolution of the first imaging system is wide; H represents that the resolution of the first imaging system is high.

[0024] Preferably, in S103, finding the best matching anchor frame based on the face position in the first image specifically includes:

[0025] The face position in the first image is obtained, and the IOU is calculated based on the width and height of the face and the size of the anchor frame of each layer of the pyramid. The face in the first image falls on the pyramid with the largest IOU calculation result; the anchor frame index on the specific pyramid corresponding to the face is calculated based on the upper left corner position of the face in the first image, that is, the optimal matching anchor frame is found.

[0026] Preferably, in S103, obtaining a homography matrix based on the correspondence between feature points of the face in the first image and feature points of the face in the second image specifically includes:

[0027] 106 feature points are located for the single faces in the first image and the second image respectively, and the 106 feature points are located using the PFLD algorithm; then the 106 feature points are used to solve the homography matrix using the findHomography function in OpenCV and saved to the corresponding anchor frame index.

[0028] Preferably, taking an original face detected in the first image as a reference, finding the best matching anchor frame under the anchor frame group specifically includes:

[0029] The position of an original face detected in the first image is obtained, and the IOU is calculated based on the width and height of the face and the size of the anchor frame of each layer of the pyramid. The face in the first image falls on the pyramid with the largest IOU calculation result; the anchor frame index on the specific pyramid corresponding to the face is calculated based on the upper left corner position of the face in the first image, that is, the optimal matching anchor frame is found.

[0030] Preferably, in S104, the facial coordinate points in the corrected face frame are obtained as follows:

[0031] x_correct=(x_ir*h11+y_ir*h12+h13) / (x_ir*h31+y_ir*h32+h33)

[0032] y_correct=(x_ir*h21+y_ir*h22+h23) / (x_ir*h31+y_ir*h32+h33)

[0033] Among them, x_correct represents the corrected x coordinate; y_correct represents the corrected y coordinate; x_ir represents the x coordinate on the face frame corresponding to the original face; y_ir represents the y coordinate on the face frame corresponding to the original face, and h11, h12, h13, h21, h22, h23, h31, h32 and h33 are the values ​​of the homography matrix H.

[0034] On the other hand, a face position matching system based on a dual-camera system includes:

[0035] An anchor frame group generation module, used to generate multiple anchor frames at different face size scales based on the resolution size of the first imaging system of the dual-camera system, wherein the multiple anchor frames form an anchor frame group, and each anchor frame is used to store a homography matrix;

[0036] A face number determination module, used to simultaneously acquire a first image generated by a first imaging system of a dual-camera system and a second image generated by a second imaging system, and identify the number of faces detected by the first image and the second image;

[0037] A homography matrix calibration module, if the number of faces detected in the first image and the second image is only one, then find the best matching anchor frame based on the face position in the first image, determine whether the homography matrix of the corresponding position of the best matching anchor frame exists, if not, obtain the homography matrix based on the correspondence between the feature points of the face in the first image and the feature points of the face in the second image, store it in the corresponding position of the best matching anchor frame; and match the face in the first image with the face detected in the second image;

[0038] A multi-face matching module, if the number of faces detected in the first image and the second image includes more than one, then based on an original face detected in the first image, find the best matching anchor frame under the anchor frame group, and determine whether the homography matrix of the corresponding position of the best matching anchor frame exists; if so, based on the homography matrix, correct the face frame corresponding to the original face and map it to the imaging surface of the second imaging system to obtain a corrected face frame, and perform IOU matching on the corrected face frame and the face frame in the second image in turn, and obtain the face in the second image corresponding to the maximum IOU value as the matching face.

[0039] On the other hand, a computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements a face position matching method based on a dual-camera system when executing the computer program.

[0040] The present invention has the following beneficial effects:

[0041] (1) The present invention regularly designs a series of small-area homography matrix groups in accordance with anchor frames according to different face sizes; the homography matrix groups are adaptively learned and calibrated during the use of the dual-camera device; the face frame (such as an infrared face frame) in the first image is mapped to the imaging surface of the second imaging system (such as the visible light imaging surface) through the homography matrix, and then matched with the face frame (such as the visible light face frame) in the second image in sequence, so that accurate matching of multiple faces at various distances under the dual-camera lens can be achieved;

[0042] (2) The present invention is based on a dual-camera system. By designing an adaptive learning face position matching method, the dual-camera imaged faces at any resolution and any distance can be matched and associated. The accurate matching of dual-camera faces can improve the security of face recognition terminals and solve the vulnerability of inaccurate matching of infrared faces and visible light faces. The entire face matching process is adaptive online learning, and the use process is convenient and fast, without the need for manual calibration process, thereby improving production efficiency.

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of matching errors using an existing simple calibration offset compensation matching method;

[0045] Figure 2 The process of the face position matching method based on the dual-camera system of the embodiment of the present invention is as follows Figure 1 ;

[0046] Figure 3 The process of the face position matching method based on the dual-camera system of the embodiment of the present invention is as follows Figure 2 ;

[0047] Figure 4 A schematic diagram of generating multiple anchor frames according to an embodiment of the present invention; wherein Figures (a), (b) and (c) respectively represent anchor frames of different sizes;

[0048] Figure 5 106 feature points of a dual-camera face according to an embodiment of the present invention; (a) is a schematic diagram of 106 feature points of an infrared single face; (b) is a schematic diagram of 106 feature points of a visible light single face;

[0049] Figure 6 Schematic diagram of face correction using a homography matrix according to the present invention; wherein (a) represents an original infrared face; (b) represents a corrected face on a visible light imaging system;

[0050] Figure 7 4 is a structural block diagram of a face position matching method system based on a dual-camera system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meanings that can be generally understood by those skilled in the art when reading the contents of the specification and the drawings. It should also be understood that, for example, terms defined in commonly used dictionaries should be understood to have meanings consistent with their meanings in the context of the relevant field, and should not be understood as idealized or excessive formal meanings unless explicitly defined herein.

[0052] See also Figure 2 As shown, the present invention provides a face position matching method based on a dual-camera system, comprising the following steps:

[0053] S101, generating a plurality of anchor frames at different face size scales based on the resolution size of a first imaging system of the dual-camera system, wherein the plurality of anchor frames form an anchor frame group, and each anchor frame is used to store a homography matrix;

[0054] S102, simultaneously acquiring a first image generated by a first imaging system and a second image generated by a second imaging system of a dual-camera system, identifying the number of faces detected by the first image and the second image, and if both images include only one face, proceeding to S103; otherwise, proceeding to S104;

[0055] S103, finding the best matching anchor frame based on the face position in the first image, determining whether a homography matrix of the position corresponding to the best matching anchor frame exists, and if not, obtaining the homography matrix based on the correspondence between the feature points of the face in the first image and the feature points of the face in the second image, and storing the matrix in the position corresponding to the best matching anchor frame;

[0056] S104, taking an original face detected in the first image as a reference, finding the best matching anchor frame under the anchor frame group, and determining whether a homography matrix of a position corresponding to the best matching anchor frame exists; if so, based on the homography matrix, correcting and mapping the face frame corresponding to the original face to an imaging surface of a second imaging system to obtain a corrected face frame, and performing IOU matching between the corrected face frame and the face frame in the second image in turn, and obtaining a face in the second image corresponding to the maximum IOU value as a matching face.

[0057] Furthermore, the S104 further includes:

[0058] If the homography matrix of the position corresponding to the optimal matching anchor frame does not exist, the original face frame and the face frame in the second image are sequentially subjected to IOU matching, and the face in the second image corresponding to the maximum IOU value is obtained as the matching face.

[0059] Furthermore, after S104, the step further includes:

[0060] If the original face frames detected in the first image include multiple ones, S104 is repeated until all the original face frames detected in the first image are matched.

[0061] Specifically, the dual-camera system includes a first imaging system and a second imaging system. The first imaging system can be an infrared imaging system or a visible light imaging system; when the first imaging system is an infrared imaging system, the second imaging system can be an infrared imaging system or a visible light imaging system; when the first imaging system is a visible light imaging system, the second imaging system can be an infrared imaging system or a visible light imaging system. That is, the dual-camera system of the present invention can cover any dual-camera system.

[0062] In this embodiment, the first imaging system is an infrared imaging system and the second imaging system is a visible light imaging system. Correspondingly, the first image is an infrared image, and the face detected in the first image is an infrared face; the second image is a visible light image, and the face detected in the second image is a visible light face.

[0063] For details, see Figure 3As shown in FIG, a face position matching method based on adaptive learning of a dual-camera system mainly includes the following three steps: 1. Initialization of the homography matrix group of the dual-camera lens; 2. Adaptive online learning calibration and recording of the homography matrix group; 3. Mapping and matching association of the face position of the dual-camera lens. These three steps will be introduced in detail below.

[0064] 1. Initialization of the dual-camera homography matrix group:

[0065] For a dual-camera system, the face position relationship of the two lenses taken at different positions of the same person is different. For example, in an infrared plus visible light dual-camera system, the infrared face position may be to the left of the visible light face position at a close distance, and to the right of the visible light face position at a long distance. In addition, the offset angle of the optical axis direction of the two lenses in different dual-camera systems is uncertain. Therefore, it is not possible to directly add a unified empirical offset value to the detected face frame to correct the frame, nor can it be mapped by calibrating a spatial mapping matrix.

[0066] This embodiment maps the face frame position of a certain lens to the coordinates of another lens by designing a regular homography matrix group (the function of the homography matrix is ​​to map a point on a projective plane to another projective plane), thereby matching the face position under two cameras. The homography matrix group is composed of a series of homography matrices of small areas, and the homography matrix of the small area can make the position space mapping near this small area more accurate. At the same time, considering the adaptation to various distances, the homography matrix group of this embodiment sets a multi-scale homography matrix according to the change of face size. The rules for initializing the homography matrix group are as follows: Assume that the resolution width of the infrared imaging of the dual-camera system is W and the height is H. The minimum face to be detected in actual use is min_size and the maximum face is max_size. Then in order to cover faces at different positions, the anchor frame group at different face size scales is generated by the pyramid scaling factor factor, and each anchor frame stores the homography matrix of the corresponding position, where factor is selected as 0.709. The number of anchor frames for the minimum face size min_size is: (W / min_size+1)*(H / min_size+1), where 1 is added to take into account the small part of the face at the edge when the width and height are not divisible. The number of anchor frames for a certain scale is: (W / (min_size / (factor n ))+1)*(H / (min_size / (factor n ))+1), until the anchor box size min_size / (factor n ) is larger than max_size, the generation of homography matrix groups will stop. The generation diagram of anchor frame groups for faces at different pyramid scales is shown in the figure below. Figure 4If the homography matrix group model file does not exist in the corresponding directory, then the anchor frame is initialized according to the above method, where the homography matrix in the anchor frame is empty; otherwise, the homography matrix model file in the corresponding directory is loaded. It should be noted that the homography matrix model file can also be generated according to the above generation method.

[0067] 2. Adaptive online learning calibration and recording of homography matrix groups.

[0068] Because the values ​​of the homography matrices in the anchor frame group reinitialized in the first step are all empty, or most of the homography matrices in the anchor frame group loaded from the model file are also empty, the device needs to adaptively learn calibration and record during use. The calibration and recording of this solution are adaptive and imperceptible. The dual-camera system simultaneously acquires infrared images and visible light images, and the dual-camera images use the face detection algorithm S3FD (Single Shot Scale-invariant FaceDetector) to detect the faces of the images. When multiple faces are detected in the infrared image and the visible light image at the same time, it is difficult to accurately match the corresponding relationship of the faces 100%; only when both the infrared image and the visible light image detect a face can it be determined that the face of the corresponding person is matched, and then the process of online learning calibration is entered. The homography matrix group of this embodiment is designed based on the resolution size of the infrared image, so the optimal matching anchor frame can be found according to the detected infrared face position, and the optimal homography matrix is ​​also found. The process of finding the best matching anchor frame needs to introduce the concept of IOU, that is, the intersection ratio of the two frames.

[0069] (1) Calculation of the index of the anchor frame corresponding to the face frame: The detection result of the infrared face is known to be (ir_face_x, ir_face_y, ir_face_w, ir_face_h). The IOU is calculated based on the width and height of the face and the size of the anchor frame of each layer of the pyramid. The infrared face falls on the pyramid with the largest IOU calculation result. Then, considering the upper left corner position of the infrared face, the index of the anchor frame corresponding to the infrared face on the specific pyramid can be calculated; after knowing the index of the anchor frame, it is possible to obtain whether the homography matrix here exists;

[0070] (2) Online learning of homography matrix: In the previous step, the position of the anchor frame is found through the position of the face. Determine whether the homography matrix in the anchor frame is empty. If it is not empty, it means that it has been calibrated through online learning; otherwise, the homography matrix in the current anchor frame needs to be calibrated through online learning. The homography matrix H = [h11, h12, h13; h21, h22, h23; h31, h32, h33], the degree of freedom of H is 8, h33 = 1, and theoretically only the correspondence between 4 points between the two views is needed to calculate the value of H. In order to calculate the homography matrix more accurately, this patent application uses 106 feature point pairs of the face to solve H, and the PFLD (A Practical Facial Landmark Detector) algorithm is used to locate the 106 feature points of the single face detected above. Figure 5 Then use the 106 feature point pairs to use the findHomography function in OpenCV to solve the homography matrix H and save it to the corresponding anchor box index.

[0071] 3. Mapping and matching association of dual-camera face positions.

[0072] When only one face is detected by the dual-camera system in both infrared and visible light, the pair of faces is considered to be automatically matched and associated, and no correction mapping of the face frame is required. For the dual-camera system, if a screen face attack is placed next to a real face, because the content on the screen cannot be imaged under infrared, there is only one real face under infrared imaging, but two faces will be imaged under visible light. In order to prevent matching errors, face position mapping and matching association are required; if it is a paper attack, multiple faces are imaged under infrared and multiple faces are also imaged under visible light, and face frame mapping and matching association are also required.

[0073] Face mapping and matching association: Taking the infrared detected face frame as the benchmark, the detected infrared face frame is used to find the anchor frame index according to the method in step 2. (1) If the homography matrix in the current anchor frame is empty, the original detected infrared face coordinates are returned, and the original infrared coordinates are used to perform IOU matching with the visible light face coordinate group. The visible light face with the maximum IOU value is considered to be the paired face. Of course, this face matching association may be wrong. This situation is inevitable during the initial operation of the terminal device equipped with a dual-camera system. However, after the terminal device has run adaptive online learning for a period of time, the homography matrix in the anchor frame will exist (that is, the scene and adaptive online learning calibration method in step 2); (2) If the homography matrix in the current infrared face anchor frame exists, then the current infrared face is corrected and mapped to the visible light imaging plane, and the mapping takes the points in the upper left and lower right corners of the infrared face. The formula is as follows: x_correct = (x_ir*h11+y_ir*h12+h13) / (x_ir*h31+y_ir*h32+h 33),y_correct=(x_ir*h21+y_ir*h22+h23) / (x_ir*h31+y_ir*h32+h33), where (x_ir, y_ir) are the coordinates on the original infrared image detected, (x_correct, y_correct) are the face coordinates corrected and mapped to the visible light image coordinates, h11, h12, h13, h21, h22, h23, h31, h32 and h33 are the values ​​of the homography matrix H. At this time, the infrared face frame after correction mapping is matched with the visible light face frame group for IOU matching. The visible light face at the maximum IOU value is considered to be the paired face. According to the above method, the position matching of multiple faces in the dual-camera system can be completed. The schematic diagram of homography matrix correction face is as follows: Figure 6 , the face position after correction and compensation is the face position after the infrared face position detected on the left image plus the offset correction, the face detected in the visible light image on the right image is detected directly by the algorithm on the visible light image, and the face position after homography matrix correction is the infrared face detected on the left image corrected by the homography matrix. As can be seen from the figure, the face position after correction and compensation has a very low overlap with the real person's visible light face frame, while the face position after correction and compensation has a high overlap with the mobile phone's visible light face frame, which will cause the dual-camera face frame to fail to match accurately; while the face position after homography matrix correction has a high overlap with the real person's visible light face frame, which shows that the face frame corrected by homography matrix mapping can more accurately match the corresponding face through overlap calculation.

[0074] See also Figure 7 As shown, the present invention provides a face position matching system based on a dual-camera system, comprising:

[0075] An anchor frame group generating module 701, used for generating multiple anchor frames at different face size scales based on the resolution size of the first imaging system of the dual-camera system, wherein the multiple anchor frames form an anchor frame group, and each anchor frame is used for storing a homography matrix;

[0076] A face number determination module 702 is used to simultaneously acquire a first image generated by a first imaging system and a second image generated by a second imaging system of a dual-camera system, and identify the number of faces detected in the first image and the second image;

[0077] The homography matrix calibration module 703, if the number of faces detected in the first image and the second image is only one, finds the best matching anchor frame based on the face position in the first image, determines whether the homography matrix of the corresponding position of the best matching anchor frame exists, and if not, obtains the homography matrix based on the correspondence between the feature points of the face in the first image and the feature points of the face in the second image, and stores it in the corresponding position of the best matching anchor frame;

[0078] The multi-face matching module 704, if the number of faces detected in the first image and the second image includes more than one, then based on the original face detected in the first image, find the best matching anchor frame under the anchor frame group, and determine whether the homography matrix of the corresponding position of the best matching anchor frame exists. If it exists, based on the homography matrix, the face frame corresponding to the original face is corrected and mapped to the imaging surface of the second imaging system to obtain a corrected face frame, and the corrected face frame is matched with the face frame in the second image in turn by IOU, and the face in the second image corresponding to the maximum IOU value is obtained as the matching face.

[0079] The specific implementation of each module of a face position matching system based on a dual-camera system is consistent with the specific implementation of each step of a fatigue detection method based on deep learning, and will not be repeated in this embodiment.

[0080] On the other hand, a computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements a face position matching method based on a dual-camera system when executing the computer program.

[0081] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish an entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements that are not clearly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the statement "comprise one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. In addition, the statements about steps such as "S101", "S102" are only for more convenient description and do not represent the order of actual execution.

[0082] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0083] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A face position matching method based on a dual-camera system, It is characterized in that The steps include: S101, generating a plurality of anchor frames at different face size scales based on the resolution size of a first imaging system of the dual-camera system, wherein the plurality of anchor frames form an anchor frame group, and each anchor frame is used to store a homography matrix; S102, simultaneously acquiring a first image generated by a first imaging system and a second image generated by a second imaging system of a dual-camera system, identifying the number of faces detected by the first image and the second image, and if both images include only one face, proceeding to S103; otherwise, proceeding to S104; S103, finding the best matching anchor frame based on the face position in the first image, determining whether a homography matrix of the position corresponding to the best matching anchor frame exists, and if not, obtaining the homography matrix based on the correspondence between the feature points of the face in the first image and the feature points of the face in the second image, and storing the matrix in the position corresponding to the best matching anchor frame; S104, taking an original face detected in the first image as a reference, finding the best matching anchor frame under the anchor frame group, and determining whether a homography matrix of a position corresponding to the best matching anchor frame exists; if so, based on the homography matrix, correcting and mapping the face frame corresponding to the original face to an imaging surface of a second imaging system to obtain a corrected face frame, and performing IOU matching between the corrected face frame and the face frame in the second image in turn, and obtaining a face in the second image corresponding to the maximum IOU value as a matching face.

2. The face position matching method based on the dual-camera system according to claim 1, It is characterized in that The S104 further includes: If the homography matrix of the position corresponding to the optimal matching anchor frame does not exist, the original face frame and the face frame in the second image are sequentially subjected to IOU matching, and the face in the second image corresponding to the maximum IOU value is obtained as the matching face.

3. The face position matching method based on the dual-camera system according to claim 1, It is characterized in that After S104, the method further includes: If the original face frames detected in the first image include multiple ones, S104 is repeated until all the original face frames detected in the first image are matched.

4. The face position matching method based on the dual-camera system according to claim 1, It is characterized in that In S101, based on the resolution size of the first imaging system of the dual-camera system, generating multiple anchor frames at different face size scales specifically includes: Based on the resolution size of the first imaging system of the dual-camera system, multiple anchor frames of different face sizes are generated through a pyramid scaling factor.

5. The face position matching method based on the dual-camera system according to claim 4, It is characterized in that The size S of each anchor box is as follows: S=min_size / (factor n ) Where min_size represents the minimum scale face; max_size represents the minimum scale face; factor represents the pyramid scaling factor; n = 0, 1, 2...; when min_size / (factor n )>max_size, no anchor box is generated; The number of anchor boxes M at a certain scale is as follows: M=(W / (min_size / (factor n ))+1)*(H / (min_size / (factor n ))+1) Wherein, W represents that the resolution of the first imaging system is wide; H represents that the resolution of the first imaging system is high.

6. The face position matching method based on the dual-camera system according to claim 1, It is characterized in that In S103, finding the best matching anchor frame based on the face position in the first image specifically includes: The face position in the first image is obtained, and the IOU is calculated based on the width and height of the face and the size of the anchor frame of each layer of the pyramid. The face in the first image falls on the pyramid with the largest IOU calculation result; the anchor frame index on the specific pyramid corresponding to the face is calculated based on the upper left corner position of the face in the first image, that is, the optimal matching anchor frame is found.

7. The face position matching method based on the dual-camera system according to claim 1, It is characterized in that In S103, based on the correspondence between the feature points of the face in the first image and the feature points of the face in the second image, obtaining a homography matrix specifically includes: 106 feature points are located for the single faces in the first image and the second image respectively, and the 106 feature points are located using the PFLD algorithm; then the 106 feature points are used to solve the homography matrix using the findHomography function in OpenCV and saved to the corresponding anchor frame index.

8. The face position matching method based on the dual-camera system according to claim 1, It is characterized in that Taking an original face detected in the first image as a reference, finding the best matching anchor frame under the anchor frame group specifically includes: The position of an original face detected in the first image is obtained, and the IOU is calculated based on the width and height of the face and the size of the anchor frame of each layer of the pyramid. The face in the first image falls on the pyramid with the largest IOU calculation result; the anchor frame index on the specific pyramid corresponding to the face is calculated based on the upper left corner position of the face in the first image, that is, the optimal matching anchor frame is found.

9. The face position matching method based on the dual-camera system according to claim 1, It is characterized in that In S104, the facial coordinate points in the corrected face frame are obtained as follows: x_correct=(x_ir*hll+y_ir*h12+h13) / (x_ir*h31+y_ir*h32+h33) y_correct=(x_ir*h21+y_ir*h22+h23) / (x_ir*h31+y_ir*h32+h33) Among them, x_correct represents the corrected x coordinate; y_correct represents the corrected y coordinate; x_ir represents the x coordinate on the face frame corresponding to the original face; y_ir represents the y coordinate on the face frame corresponding to the original face. h11, h12, h13, h21, h22, h23, h31, h32 and h33 are the values ​​of the homography matrix H.

10. A face position matching system based on a dual-camera system, It is characterized in that include: An anchor frame group generation module, used to generate multiple anchor frames at different face size scales based on the resolution size of the first imaging system of the dual-camera system, wherein the multiple anchor frames form an anchor frame group, and each anchor frame is used to store a homography matrix; A face number determination module, used to simultaneously acquire a first image generated by a first imaging system of a dual-camera system and a second image generated by a second imaging system, and identify the number of faces detected by the first image and the second image; A homography matrix calibration module, if the number of faces detected in the first image and the second image is only one, then find the best matching anchor frame based on the face position in the first image, determine whether the homography matrix of the corresponding position of the best matching anchor frame exists, and if not, obtain the homography matrix based on the correspondence between the feature points of the face in the first image and the feature points of the face in the second image, and store it in the corresponding position of the best matching anchor frame; A multi-face matching module, if the number of faces detected in the first image and the second image includes more than one, then based on an original face detected in the first image, find the best matching anchor frame under the anchor frame group, and determine whether the homography matrix of the corresponding position of the best matching anchor frame exists; if so, based on the homography matrix, correct the face frame corresponding to the original face and map it to the imaging surface of the second imaging system to obtain a corrected face frame, and perform IOU matching on the corrected face frame and the face frame in the second image in turn, and obtain the face in the second image corresponding to the maximum IOU value as the matching face.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the method according to any one of claims 1 to 9 is implemented.