Long-range binocular 3D face recognition method and system

Abstract

This invention provides a long-range binocular 3D face recognition method and system. The method includes: determining the offset angle between the face photos captured by the first camera C1 and the second camera C2 and the cameras; a processor predicting the future three-dimensional position coordinates of each face at the next moment based on pre-stored historical correct data using a Kalman filter algorithm; the processor calculating the predicted offset angle of each face based on the future three-dimensional position coordinates of each face; the processor comparing the predicted a1' and b1' with a1 and b1 respectively, and simultaneously comparing the predicted a2' and b2' with a2 and b2 respectively. If all comparisons are successful, the identity ID of the face in the face photo can be determined. This method, through coordinate calculation, Kalman filtering, and angle inversion, can perform more accurate 3D face recognition at a long distance even in poor lighting conditions.

Description

Technical Field

[0001] This invention relates to the field of facial recognition technology, and in particular to a long-range binocular 3D facial recognition method and system. Background Technology

[0002] Existing binocular 3D face recognition methods have a baseline length (i.e., the distance between the two cameras) of less than 20 centimeters, and the farthest distance at which 3D face recognition can be achieved is about 2 meters. This distance is too short to meet the needs of long-distance recognition applications.

[0003] Furthermore, existing facial recognition technologies have significant errors in recognition results under poor lighting conditions. Summary of the Invention

[0004] Based on this, the purpose of this invention is to provide a long-distance binocular 3D face recognition method and system for performing 3D face recognition in environments with poor lighting conditions at a distance.

[0005] This embodiment provides a long-range binocular 3D face recognition method. The method is applied to a long-range binocular 3D face recognition system, which includes: a first camera, a second camera, a first TOF sensor, a second TOF sensor, a processor, and a control system. The method includes: the first camera C1 and the second camera C2 determining the offset angle between their respective captured face photos and the camera, wherein the face photo captured by C1 is the first face photo f1, and the face photo captured by C2 is the second face photo f2. The first offset angle of f1 relative to C1 is a1 and b1, and the second offset angle of f2 relative to C2 is a2 and b2. The processor is based on a Kalman filter algorithm, according to... The pre-stored historical correct data predicts the future three-dimensional position coordinates of each face at the next moment. The historical correct data includes the identity ID of each face and the three-dimensional position coordinates at the current moment. Based on the future three-dimensional position coordinates of each face, the processor calculates the predicted offset angle of each face. The predicted offset angles include: predicted offset angles a1' and b1' relative to C1 and predicted offset angles a2' and b2' relative to C2. The processor compares the predicted a1' and b1' with a1 and b1 respectively, and at the same time, compares the predicted a2' and b2' with a2 and b2 respectively. If all comparisons are successful, the identity ID of the face in the face photo can be determined.

[0006] Further, after the first camera C1 and the second camera C2 determine the offset angle between their respective captured face photos and the cameras, the method further includes: the first TOF sensor S1 and the second TOF sensor S2 determining the distances d1 and d2 between the face photos and the cameras, wherein S1 and C1 are in the same position, and S2 and C2 are in the same position; the processor determines the three-dimensional position coordinates of the face photos based on the distances between the face photos and the cameras and the offset angle, and matches f1 and f2 with the same three-dimensional position coordinates as the same face; the processor confirms the 3D face information based on the parallax between f1 and f2 of the same face, and sends it to the control system; the control system compares the 3D face information with the information in the pre-stored 3D face database to determine the face's identity ID.

[0007] Furthermore, after the control system compares the 3D face information with the information in the pre-stored 3D face database to determine the face's identity ID, the method further includes: the control system sending the face's identity ID to the processor; the processor packaging the face's identity ID, the shooting time, the offset angles of the successfully matched f1 and f2 relative to the camera, and the current three-dimensional position coordinates as historical correct data.

[0008] Furthermore, the processor, based on the Kalman filter algorithm, predicts the future three-dimensional position coordinates of each face at the next moment according to the pre-stored historical correct data. This includes: if multiple f1 and multiple f2 images captured by C1 and C2 respectively cannot be matched successfully; the processor inputs a preset number of pre-stored historical correct data into the Kalman filter; the Kalman filter predicts the future three-dimensional position coordinates based on the three-dimensional position coordinates in the historical correct data.

[0009] Furthermore, the processor determines the three-dimensional position coordinates of the face photo based on the distance and offset angle between the face photo and the camera, and the step of matching f1 and f2 with the same three-dimensional position coordinates as the same face includes: calculating the three-dimensional position coordinates of f1 based on a1, b1, and d1.

[0010] Based on a2, b2, and d2, the three-dimensional position coordinates of f2 can be calculated. The three-dimensional position coordinates of f1 and f2 are compared respectively. If the three-dimensional position coordinates are the same, f1 and f2 are matched as the same face, thus achieving face matching.

[0011] Furthermore, the first offset angle of f1 relative to C1 includes: the angle a1 formed by the baseline, C1, and f1, and the pitch angle b1 of f1 relative to C1, wherein the baseline is the line connecting C1 and C2; the second offset angle of f2 relative to C2 includes: the angle a2 formed by the baseline, C2, and f2, and the pitch angle b2 of f2 relative to C2.

[0012] Secondly, this embodiment provides a long-range binocular 3D face recognition system, which is used to execute the above-described long-range binocular 3D face recognition method; a first camera and a first TOF sensor are located at the same position; a second camera and a second TOF sensor are located at the same position; the first camera and the first TOF sensor are respectively connected to a processor; the second camera and the second TOF sensor are respectively connected to a processor; the processor is also connected to a control system; a 3D face database is pre-stored in the control system.

[0013] The beneficial effects of the embodiments of the present invention are as follows:

[0014] The technical solution of this invention can perform more accurate 3D face recognition in environments with long distances and poor lighting.

[0015] Other features and advantages of the invention will be set forth in the following description, or some features and advantages may be inferred from the description or determined without doubt, or may be learned by practicing the techniques described above.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram illustrating the positional relationship between a binocular camera and a target, provided in an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of a long-range binocular 3D face recognition system provided in an embodiment of the present invention. Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention. Example

[0022] This embodiment provides a long-range binocular 3D face recognition method, which is applied to a long-range binocular 3D face recognition system. The system includes: a first camera C1, a second camera C2, a first TOF (Time-of-Flight) sensor S1, a second TOF sensor S2, a processor, and a control system.

[0023] The long-range binocular 3D face recognition method of this application includes:

[0024] Step S102: The first camera, the second camera, the first TOF sensor, and the second TOF sensor determine the distance and offset angle between the face photo and the camera, wherein the face photo taken by the first camera is the first face photo, and the face photo taken by the second camera is the second face photo.

[0025] In step S1022, the first camera takes multiple first face photos and the second camera takes multiple second face photos.

[0026] Specifically, the first face photo is denoted by f1, and the second face photo is denoted by f2. The first offset angle of f1 relative to C1 is a1 and b1, and the second offset angle of f2 relative to C2 is a2 and b2.

[0027] Specifically, the photos taken by the camera may contain faces or not, so preprocessing is required to remove non-face photos.

[0028] In step S1024, the first TOF sensor acquires the first distance between each first face photo and the first camera, and the second TOF sensor acquires the second distance between each second face photo and the second camera.

[0029] Specifically, the first distance is denoted by d1, and the second distance is denoted by d2.

[0030] Step S1026: The first camera acquires the first offset angle between each first face photo and the first camera, and the second camera acquires the second offset angle between each second face photo and the second camera.

[0031] Specifically, such as Figure 1As shown, the first offset angle includes: the angle a1 formed by the baseline and the first camera -f1, and the pitch angle of f1 relative to the first camera, referred to as the first pitch angle b1 (not shown in the figure). The baseline is the line connecting the first camera and the second camera. The second offset angle includes: the angle a2 formed by the baseline and the second camera -f2, and the pitch angle of f2 relative to the second camera, referred to as the second pitch angle b2 (not shown in the figure).

[0032] In step S104, the processor determines the three-dimensional position coordinates of the face photo based on the distance and offset angle between the face photo and the camera, and matches the first photo and the second photo with the same three-dimensional position coordinates as the same face.

[0033] Step S1042: Based on the first offset angle and d1, the three-dimensional position coordinates of f1 can be calculated; based on the second offset angle and d2, the three-dimensional position coordinates of f2 can be calculated.

[0034] Specifically, the coordinates of the two cameras and the two TOF sensors are known.

[0035] Step S1044: Compare the three-dimensional position coordinates of f1 and f2 respectively. If the three-dimensional position coordinates are the same, then match f1 and f2 as the same face to achieve face matching.

[0036] Specifically, in crowded environments, the camera will detect many faces, and a position is calculated for each face in each camera image. This step involves pairwise matching of face positions between two cameras.

[0037] Due to environmental factors such as changes in light, the distance measured by the TOF sensor may contain invalid data, making it impossible to match f1 and f2 as the same face. In this case, steps S110-S114 need to be executed. If step S104 can be performed normally, then step S106 is executed.

[0038] In step S106, the processor uses the parallax between the first and second face photos of the same face to obtain 3D face information and sends it to the control system.

[0039] Specifically, 3D facial information is the three-dimensional information of a face, such as the height of the nose and the depth of the eye sockets, which reflect the degree of concavity and convexity of the face.

[0040] In step S108, the control system compares the 3D face information with the information in the pre-stored 3D face database to determine the identity of the face in the photo.

[0041] Specifically, after the control system determines the identity of the face in the photo, it sends the face's identity document to the processor. The processor then packages the identity document, the shooting time, the offset angles a1 and b1 of the first successfully matched face photo relative to the two cameras (i.e., the offset angles of the second face photo relative to the two cameras), and the current three-dimensional position coordinates. The packaged data is called historical correct data and is stored locally.

[0042] In step S110, the processor uses the Kalman filter algorithm to predict the future three-dimensional position coordinates of each face at the next moment based on the pre-stored historical correct data.

[0043] Specifically, this step assumes that each of the two cameras has captured N face photos, resulting in 2N face photos. Each face photo also has corresponding angle information (a1, a2, b1, b2), but these 2N photos cannot be matched. At this point, the 3D position coordinates of the previously matched M faces from the previous second, two seconds ago, etc., need to be retrieved (i.e., information from historical correct data; the specific data collected from the previous few seconds can be pre-set). These correct 3D position coordinates from the previous second and two seconds ago are then input into a Kalman filter. The Kalman filter will predict and output the future 3D position coordinates of each of the M faces. Here, since the input is historical correct data, the identity IDs of the faces in the photos are also known.

[0044] Specifically, "one second ago, two seconds ago" is actually a metaphor. In reality, cameras typically operate at 30 frames per second, so there may be 30 processing steps per second.

[0045] Step S112: Based on the future three-dimensional position coordinates of each face, calculate the predicted offset angle of each face. The predicted offset angle includes: the predicted offset angles a1' and b1' relative to the first camera and the predicted offset angles a2' and b2' relative to the second camera.

[0046] In step S114, a1' and b1' are compared with a1 and b1 obtained in step S1026, and a2' and b2' are compared with a2 and b2 obtained in step S1026. If the comparison is successful, the identity ID of the face in the face photo can be determined.

[0047] Regarding steps S110-S114, there is no need to match the first photo with the second photo. Simply compare each of the 2N unmatched face photos with the historical correct data processed by Kalman filtering to confirm the identity ID of the face in the face photo.

[0048] This method, through coordinate calculation, Kalman filtering, and angle inverse calculation, can perform more accurate 3D face recognition from a distance even in poor lighting conditions. Example

[0049] This embodiment provides a long-range binocular 3D face recognition system, which includes: a first camera, a second camera, a first TOF sensor, a second TOF sensor, a processor, and a control system, such as... Figure 2 As shown.

[0050] Each camera and TOF sensor is connected to a processor via a data bus. The processor can be an ARM or DSP. The processor is connected to the control system via a data bus (which can be an Ethernet, WiFi, 4G, 5G network, etc.). The control system can be a PC or a server. The control system includes a pre-acquired 3D face database.

[0051] Each camera and its corresponding TOF sensor are installed as close as possible in space. By adjusting the angle and focal length of the camera and the TOF sensor, their perspectives are made to overlap, so as to facilitate the measurement of angle and distance.

[0052] The long-range binocular 3D face recognition system provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned long-range binocular 3D face recognition method embodiment. For the sake of brevity, any parts not mentioned in the system embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A long-range binocular 3D face recognition method, characterized in that, The method is applied to a long-range binocular 3D face recognition system. The system includes: a first camera, a second camera, a first TOF sensor, a second TOF sensor, a processor, and a control system; The method includes: The first camera C1 and the second camera C2 determine the offset angle between the face photos they capture and the camera. The face photo captured by C1 is the first face photo f1, and the face photo captured by C2 is the second face photo f2. The first offset angle of f1 relative to C1 is a1 and b1, and the second offset angle of f2 relative to C2 is a2 and b2. The line connecting the first camera C1 and the second camera C2 is the baseline. The angle formed by the baseline with C1 and f1 is a1, and the pitch angle of f1 relative to C1 is b1. The angle formed by the baseline with C2 and f2 is a2, and the pitch angle of f2 relative to C2 is b2. The first TOF sensor S1 and the second TOF sensor S2 determine the distances d1 and d2 between the face photo and the camera, where S1 and C1 are in the same position, and S2 and C2 are in the same position. The processor determines the three-dimensional position coordinates of the face photo based on the distance and offset angle between the face photo and the camera, and matches f1 and f2 with the same three-dimensional position coordinates as the same face; The processor uses a Kalman filter algorithm to predict the future three-dimensional position coordinates of each face at the next moment based on pre-stored historical correct data. The historical correct data includes the identity ID of each face and the three-dimensional position coordinates at the current moment. Based on the future three-dimensional position coordinates of each face, the processor calculates the predicted offset angle of each face. The predicted offset angles include: the predicted offset angles a1' and b1' relative to C1 and the predicted offset angles a2' and b2' relative to C2. The processor compares the predicted a1' and b1' with a1 and b1 respectively, and at the same time, it compares the predicted a2' and b2' with a2 and b2 respectively. If all comparisons are successful, the identity ID of the face in the face photo can be determined.

2. The long-range binocular 3D face recognition method according to claim 1, characterized in that, After the steps of determining the offset angle between the face photos captured by the first camera C1 and the second camera C2 and the camera itself, the method further includes: The processor confirms the 3D face information based on the parallax between f1 and f2 of the same face and sends it to the control system. The control system compares the 3D facial information with information in a pre-stored 3D facial database to determine the facial identity ID.

3. The long-range binocular 3D face recognition method according to claim 2, characterized in that, After the control system compares the 3D face information with information in a pre-stored 3D face database to determine the face's identity ID, the method further includes: The control system sends the facial identification ID to the processor; The processor packages the face's identity ID, shooting time, the offset angles of the successfully matched f1 and f2 relative to the camera, and the current 3D position coordinates as historical correct data.

4. The long-range binocular 3D face recognition method according to claim 1, characterized in that, The processor, based on the Kalman filter algorithm, predicts the future 3D coordinates of each face at the next moment using pre-stored historical correct data. The steps include: If multiple f1 and multiple f2 images taken by C1 and C2 respectively cannot be matched successfully; The processor inputs a preset number of pre-stored historical correct data into the Kalman filter; The Kalman filter predicts the future three-dimensional position coordinates based on the three-dimensional position coordinates in the historical correct data.

5. The long-range binocular 3D face recognition method according to claim 2, characterized in that, The processor determines the three-dimensional position coordinates of the face photo based on the distance and offset angle between the face photo and the camera, and matches f1 and f2 with the same three-dimensional position coordinates as the same face. The steps include: Determine the three-dimensional position coordinates of f1 based on a1, b1, and d1; Determine the three-dimensional position coordinates of f2 based on a2, b2, and d2; Compare the three-dimensional position coordinates of f1 and f2 respectively. If the three-dimensional position coordinates are the same, then f1 and f2 are matched as the same face, thus achieving face matching.

6. The long-range binocular 3D face recognition method according to claim 1, characterized in that, The first offset angle of f1 relative to C1 includes: the angle a1 formed by the baseline, C1, and f1, and the pitch angle b1 of f1 relative to C1, wherein the baseline is the line connecting C1 and C2; The second offset angle of f2 relative to C2 includes: the angle a2 formed by the baseline, C2, and f2, and the pitch angle b2 of f2 relative to C2.

7. A long-range binocular 3D face recognition system, characterized in that, The system is used to perform the long-range binocular 3D face recognition method according to any one of claims 1-6; The first camera and the first TOF sensor are located in the same position; The second camera and the second TOF sensor are located in the same position; The first camera and the first TOF sensor are respectively connected to the processor; The second camera and the second TOF sensor are respectively connected to the processor; The processor is also connected to the control system, which contains a pre-stored 3D face database.

Images