Face pose angle detection device, method, apparatus and storage medium

Abstract

This disclosure relates to a face pose angle detection device, method, apparatus, and storage medium, relating to the field of face recognition. The device includes: a positioning component, multiple supporting components, a fixing component, a data acquisition component, and a computing module. The positioning component provides support for the multiple supporting and fixing components, which are fixedly connected to the positioning component to maintain a fixed relative position between the positioning component and the face. A data acquisition component is provided on the positioning component and / or each supporting component to acquire the face's pose information and send it to the computing module. The computing module determines the face pose angle based on the pose information. This disclosure can be used to verify the face pose angle detection results of a face recognition device.

Description

Technical Field

[0001] This application relates to the field of detection technology, and in particular to a face pose angle detection device, method, apparatus and storage medium. Background Technology

[0002] "Pose angle" refers to the angle of a face relative to a face recognition device in three-dimensional space. It is the rotation angle relative to the three coordinate axes in a right-handed Cartesian coordinate system with the tip of the nose as the origin: pitch, yaw, and roll. "Pose angle" is one of the important indicators for measuring the performance of face recognition image data, and some current industry standards in my country have put forward corresponding requirements for "pose angle" during face recognition. For example, the "Technical Requirements for Face Recognition Applications in Public Security" (GB / T 35678), "Security Face Recognition Application Systems Part 2: Face Image Data" (GA / T922.2), and "Technical Requirements for Biometric Face Recognition Systems in Information Technology" (GB / T 41772-2022) all stipulate that "the horizontal rotation angle of the face should be within ±10°, the pitch angle should be within ±10°, and the tilt angle should be within ±10°." The "Technical Specifications for Face Verification Systems for Security at Civil Transport Airports (Trial)" (Civil Aviation Administration of China Document No. 17

[2021] ) stipulates that "the face posture: the horizontal rotation angle should not exceed ±35°, the pitch angle should not exceed ±20°, and the tilt angle should not exceed ±45°."

[0003] How to scientifically, objectively, and accurately detect the pose angle of a person's face relative to a facial recognition device has always been a technical challenge for the industry. Summary of the Invention

[0004] Embodiments of this disclosure provide a face pose angle detection device, method, apparatus, and storage medium.

[0005] In a first aspect, embodiments of this disclosure provide a face pose angle detection device, comprising: a positioning component, multiple supporting components, a fixing component, a data acquisition component, and a calculation module; the positioning component provides support for the multiple supporting components and the fixing component, the multiple supporting components and the fixing component are fixedly connected to the positioning component to maintain a fixed relative position between the positioning component and the face; the positioning component and / or each supporting component are provided with a data acquisition component, the data acquisition component is used to acquire the pose information of the face and send the pose information to the calculation module; the calculation module is used to determine the pose angle of the face based on the pose information.

[0006] Secondly, embodiments of this disclosure provide a face pose angle detection method, applied to a face pose angle detection device as described in the first aspect, wherein the face pose angle detection device is worn on a person's head, and the method includes: acquiring real-time pose information collected by a data acquisition component, wherein the real-time pose information includes real-time pose information and real-time distance information; determining a face reference plane based on the real-time distance information; determining a real-time face pose angle based on the real-time pose information; determining a reference pose angle based on the face reference plane; and determining the true value of the face pose angle based on the real-time face pose angle and the reference pose angle.

[0007] Thirdly, embodiments of this disclosure provide a face pose angle detection device, comprising: a data acquisition unit configured to acquire real-time pose information collected by a data acquisition component, wherein the real-time pose information includes real-time pose information and real-time distance information; a reference plane determination unit configured to determine a face reference plane based on the real-time distance information; a pose angle determination unit configured to determine a real-time face pose angle based on the real-time pose information; a reference pose angle determination unit configured to determine a reference pose angle based on the face reference plane; and a pose angle truth value determination unit configured to determine a true face pose angle based on the real-time face pose angle and the reference pose angle.

[0008] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the face pose angle detection method as described in the first aspect.

[0009] The technical solution disclosed herein is easy for people to wear and can detect the pose angle of a face. It can be used to verify the face pose angle recognition results of face recognition devices, thereby determining whether the face recognition device meets relevant standards.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0012] Figure 1a , Figure 1b The images show a front view and a side view of an embodiment of the face pose angle detection device disclosed herein.

[0013] Figure 2a , Figure 2b These are front and side views of another embodiment of the face pose angle detection device of this disclosure;

[0014] Figure 3 This is a schematic diagram of the clip structure of the fixing component of the face pose angle detection device disclosed herein;

[0015] Figure 4a , Figure 4b This is a schematic diagram of two possible structures for the adjustment component of the face pose angle detection device disclosed herein;

[0016] Figure 5 This is a schematic diagram illustrating the application scenarios of the face pose angle detection device disclosed herein;

[0017] Figure 6 This is a flowchart illustrating one embodiment of the face pose angle detection disclosed herein.

[0018] Figure 7 This is a flowchart illustrating another embodiment of the face pose angle detection disclosed herein.

[0019] Figure 8 This is a schematic diagram of one embodiment of the face pose angle detection device disclosed herein. Detailed Implementation

[0020] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0021] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0022] Where there is no conflict, the embodiments and features described herein can be combined with each other.

[0023] To make the technical solutions and advantages of this disclosure clearer, the following description, in conjunction with the accompanying drawings and specific embodiments, will provide a more detailed account of this disclosure.

[0024] Figure 1a and Figure 1b A front view and a side view of an embodiment of a face pose angle detection device according to the present disclosure are shown. Figure 1a and Figure 1bAs shown, the face pose angle detection device 100 of this embodiment may include a positioning component 101, multiple support components 102, and a fixing component 103. The positioning component 101 provides positioning support for the support components 102 and the fixing component 103. The support components 102 and the fixing component 103 are fixedly connected to the positioning component 101 to maintain a fixed relative position between the face pose angle detection device 100 and the face, preventing wobbling. In some specific implementations, the positioning component 101 can be a ring-shaped structure, which facilitates the fixing of multiple support components 102. The size of the ring-shaped structure can be slightly larger than the size of the human head to avoid obscuring facial features. The fixing component 103 can be fixed to the back of the human head, allowing the face pose angle detection device to be more stably fixed to the head.

[0025] Multiple support components 102 can be fixed to the aforementioned positioning component 101. Specifically, each support component 102 can be fixed at a different position on the positioning component 101. For example, it can be positioned at the top, bottom, left, or right of the positioning component 101. The support component 102 can be a strip-shaped structure, with one end fixed to the positioning component 101 and the other end located inside the annular structure formed by the positioning component 101. A data acquisition component 104 can be provided at this end. In some specific practices, to avoid the support components 102 obstructing the face, the support component 102 can be a long and thin strip-shaped structure, and it can be made of transparent material.

[0026] The data acquisition component 104 can include various types of sensors, such as distance sensors, gyroscope sensors, accelerometers, and magnetometers. In some specific applications, the distance sensor in the data acquisition component 104 can be located at one end of the support component 102. During use, the lower surface of the end of the support component 102 where the distance sensor is located can be placed against the surface of the face. Specifically, the data acquisition component 104 can include multiple distance sensors for measuring the distance between different positions of the face and the face recognition device or face pose angle recognition device. To determine the accuracy of the face plane, each distance sensor can be positioned at a location that characterizes the face plane. This allows for more accurate acquisition of face position data, thereby enabling more accurate calculation of the face pose angle. The distance sensors can be used to measure the distance between the face and the screen of the face recognition device; the gyroscope sensor can measure the head rotation angular velocity along the three axes of the Cartesian coordinate system; the accelerometer can measure the head rotation acceleration along the three axes of the Cartesian coordinate system; and the magnetometer can measure the head magnetic heading angle along the three axes of the Cartesian coordinate system. Understandably, to avoid the data acquisition component 104 obstructing the face, some components (such as the gyroscope sensor, accelerometer, and magnetometer) can be mounted on the positioning component 101. Only the distance sensor is mounted on the support component 102, and a smaller distance sensor can be used to minimize the obstruction of the face by the distance sensor.

[0027] The data acquisition unit 104 can send the acquired data to the calculation module 105, which can determine the facial pose angle based on the data acquired by the data acquisition unit 104. In some specific practices, the calculation module 105 can be implemented using a chip, such as an FPGA chip or a microcontroller. The calculation module 105 can also output the calculated facial pose angle for technicians to view.

[0028] The face pose angle detection device provided in the above embodiments of this disclosure can be fixed on a person's head. The face pose information can be collected by the data acquisition component set on it, and the face pose angle can be determined based on the above pose information, thereby realizing the detection of face pose angle.

[0029] Figure 2a and Figure 2b Figure 2 shows a front view and a side view of another embodiment of the face pose angle detection device of this disclosure. As shown in Figure 2, the face pose angle detection device 200 in this embodiment may include: a positioning component 201, a plurality of support components 202, a fixing component 203, and an adjustment component 204 disposed on the support components 202.

[0030] The positioning component 201 provides positioning support for the support component 202 and the fixing component 203. In some specific practices, the positioning component 201 can be a ring-shaped structure, specifically an elliptical structure. The area of ​​the ellipse is larger than the area of ​​the human face, thus avoiding obstruction of the face. To improve the rigidity of the positioning component 201, it may include a positioning ring 2011 and a reinforcing rib 2012. The positioning ring 2011 can be perpendicular to the reinforcing rib 2012 to improve the strength of the positioning ring 2011, making it less prone to bending and preventing deformation of the positioning ring 2011 during wear.

[0031] In some specific practices, to ensure that the positioning component 201 does not obscure the face, both the positioning ring 2011 and the reinforcing rib 2012 can be made of lightweight materials such as transparent acrylic. The advantage of using transparent materials is that they do not obscure facial details and do not reduce the similarity value of face recognition. The positioning ring 2011 can be elliptical, approximately 20mm wide and 2-5mm thick, with the inner ring area slightly larger than the face. In practice, the inner ring area can be adjusted to create positioning rings 2011 of various sizes to accommodate faces of different sizes. The reinforcing rib 2012 can be perpendicular to the positioning ring 2011, located at the center of the positioning ring 2011, with a thickness of approximately 2mm and a height of approximately 5mm.

[0032] Multiple support components 202 can be respectively disposed at different positions of the positioning component 201. In some specific practices, the support component 202 may include at least four strip-shaped structures with dimensions of 5mm width, 2mm thickness, and 100mm length. These four strip-shaped structures can be respectively disposed at the top, bottom, left, and right positions of the positioning component 201. Specifically, when the positioning component 201 is an elliptical structure, two of the four strip-shaped structures can be disposed on the major axis of the ellipse, and the other two can be disposed on a horizontal line perpendicular to the major axis. To more accurately locate the face plane, distance sensors disposed on the four strip-shaped components can be respectively disposed on the forehead, jawline, and below the cheekbone.

[0033] The fixing component 203 may include multiple fixing straps 2031. These fixing straps 2031 can be connected to the positioning component 201 in various ways. For example, the fixing straps 2031 can be fixed to the positioning component 201 via buckles or through holes. In some specific practices, the fixing straps 2031 are detachably connected to the positioning component 201. The fixing straps 2031 can also be connected to each other in various ways. For example, the fixing straps can be fixedly connected via buckles 2033. The buckles 2033 are movably connected to each fixing strap 2031, facilitating adjustment of each fixing strap 2031 to fit different head sizes. Alternatively, the fixing straps 2031 may have through holes through which fixing bolts 2034 can connect to each fixing strap. In this way, each fixing strap 2031 can be adjusted according to different head sizes, thereby making the space formed by the fixing straps 2031 more suitable for the head.

[0034] In some specific practices, the fixing component 203 may include three fixing straps 2031, one each on the left, top, and right, which can be made of soft plastic or linen. Each fixing strap 2031 is approximately 300mm long, 30mm wide, and 2mm thick. Different sizes of fixing straps can also be manufactured as needed. One end of the fixing strap 2031 passes through the connecting hole 2032 and is fixed to the positioning component 201. Figure 3 As shown, the buckle 2033 has a cuboid structure with dimensions of 40mm × 40mm × 25mm and is made of plastic. Each of the 40mm × 25mm surfaces has a square through hole with a length of 35mm and a width of 10mm for the fastening strap 2031 to pass through. One of the 40mm × 40mm surfaces has an M15 diameter internal threaded hole for installing the fastening bolt 2034 of the fastening strap 2031. The fastening bolt 2034 can be a 20mm long, M15 diameter bolt made of plastic.

[0035] In some optional embodiments of this example, the support member 202 is provided with multiple through holes 2021, and the positioning member 201 may be provided with protrusions 2013 adapted to the through holes for fixing the support member 202 to the positioning member 201. In some specific practices, the through holes may have a diameter of 2 mm and a center distance of 8 mm. The protrusions 2013 may also be fixed by bolts to enhance the stability of the support member 202.

[0036] Adjustment component 204 can be disposed between support component 202 and positioning component 201, and is used to adjust the angle of data acquisition component 205 disposed on support component 202. Adjustment component 204 can be fixed to positioning component 201 by bolt 2041.

[0037] The aforementioned data acquisition component 205 may include a distance sensor, a gyroscope sensor, an accelerometer sensor, a magnetometer sensor, etc. The data acquisition component 205 can send the acquired data to the computing module 206. In some specific practices, the distance sensor in the data acquisition component 205 is located at the end of the support component 202, while the gyroscope sensor, accelerometer sensor, and magnetometer sensor can be located at other positions besides the face, such as on the positioning component 201. The aforementioned distance sensor can be a laser rangefinder sensor, and the adjustment component 204 can adjust the angle of the laser rangefinder sensor so that the lasers emitted by each laser rangefinder sensor can be simultaneously projected onto the screen of the face recognition device.

[0038] When the laser rangefinder sensor is positioned at one end of the support component 202, the large size of the wireless laser rangefinder sensor would obscure a significant portion of the face. Therefore, in this embodiment, a wired laser rangefinder sensor can be used, with the electronic components of the laser rangefinder sensor placed only at one end of the support component 202. The electronic components for power supply, data processing, and other circuits are placed independently in the data acquisition component 205. In this case, the connecting cable 2051 of the laser rangefinder sensor can be hidden within the positioning component 201 and the support component 202 for ease of use and to ensure the communication stability of the data acquisition component 205.

[0039] Laser rangefinders are used to measure the distance between a face and the screen of a face recognition device; gyroscopes are used to measure the angular velocity of head rotation; accelerometers are used to measure the acceleration of head rotation; and magnetometers are used to measure the magnetic heading angle of the head.

[0040] Miniature laser rangefinders can be selected, such as the VL53L0X or VL53L1X laser rangefinders that are well-known in the market. These laser rangefinders are only 4.4mm × 2.4mm × 1mm in size, which can minimize the obstruction of the face.

[0041] Gyroscope sensors can be used to measure the angular velocity of head rotation along the three axes of a Cartesian coordinate system and output the measurement results to the main controller. Gyroscope sensors can be standalone three-axis gyroscopes, such as the ADIS16265 from EVICES or the TAC-450 fiber optic gyroscope from EMCORE, or integrated gyroscope sensors, such as the ADIS16495 or ADIS16445 from ANALOG DEVICES or the STIM300 inertial measurement unit from Sensonor AS (Norway).

[0042] Furthermore, the accelerometer 403 is used to measure the head rotation acceleration in the three coordinate axes of the Cartesian coordinate system and outputs the measurement results to the main controller 401. The accelerometer 403 can be a standalone triaxial accelerometer, such as the ADXL700 or ADXL703 manufactured by ANALOG DEVICES, or the Q-FLEX QA3000, QA2000, and QA750 manufactured by Honeywell, or an integrated triaxial accelerometer, such as the ADIS16495 or ADIS16445 manufactured by ANALOG DEVICES, or the STIM300 inertial measurement unit manufactured by Sensonor AS of Norway.

[0043] Furthermore, the magnetic sensor 404 is used to measure the head magnetic heading angle in the three coordinate axes of the Cartesian coordinate system and outputs the measurement results to the main controller 401. The magnetic sensor 404 can be a standalone magnetic sensor, such as the HMC2003 or HMC5883L manufactured by Honeywell, or an integrated magnetic sensor, such as the ADIS16495 or ADIS16445 manufactured by Analog Devices, or the STIM300 inertial measurement unit manufactured by Sensonor AS of Norway.

[0044] The calculation module 206 can receive data collected by the data acquisition component 205 and perform calculations based on the received data to determine the pose angle of the face. The calculation module 206 can be implemented using an FPGA, a microcontroller, or the like.

[0045] In some optional embodiments of this example, the positioning component 201 can be a symmetrical annular structure, with the axis of symmetry coinciding with the midline of the nose of the face. The aforementioned plurality of support components 202 may further include horizontal sub-support components and vertical sub-support components. Correspondingly, the adjustment component may include a first sub-adjustment component and a second sub-adjustment component. The vertical sub-support component may be disposed on the aforementioned axis of symmetry, and the horizontal sub-support component may be disposed on a horizontal line perpendicular to the aforementioned axis of symmetry. The first sub-adjustment component is disposed between the horizontal sub-support component and the fixing component 201, and the second sub-adjustment component is disposed between the vertical sub-support component and the fixing component 201.

[0046] The first and second sub-adjustment components can be right-angled triangular or right-angled trapezoidal shims, such as... Figure 4a and Figure 4bAs shown. Its width can be the same as the width of the strip structure of the support member 202. In order to adjust the angle of the distance sensor provided at one end of the support member 202 and keep the above angle fixed, through holes can be provided on the adjusting member 204 and the support member 202, and the adjusting member and the support member can be fixed together by bolts. In some specific practices, the size of the above through holes can be a diameter of 2mm and a center distance of 8mm.

[0047] In some optional implementations of this embodiment, the vertical sub-support component includes an upper sub-support component and a lower sub-support component, both of which are positioned on the axis of symmetry of the positioning component 201 that coincides with the midline of the nose of the face. The horizontal sub-support component includes a left sub-support component and a right sub-support component, both positioned on the same horizontal line perpendicular to the aforementioned axis of symmetry, and symmetrically arranged about the axis of symmetry. It should be noted that the height of the shim used to adjust the left sub-support component should be the same as the height of the shim used to adjust the right sub-support component. Similarly, the height of the shim used to adjust the upper sub-support component should be the same as the height of the shim used to adjust the lower sub-support component. This ensures that the angle of the distance sensor located on the left sub-support component is consistent with the angle of the distance sensor located on the right sub-support component, and that the angle of the distance sensor located on the upper sub-support component is consistent with the angle of the distance sensor located on the lower sub-support component.

[0048] In some optional implementations of this embodiment, the face pose angle detection device may further include an audio output component (not shown in the figure) for outputting corresponding prompt information based on the distances measured by each distance sensor. Specifically, the audio output component may output a first prompt message when the distances measured by the upper and lower distance sensors are equal, and a second prompt message when the distances measured by the left and right distance sensors are equal. Alternatively, voice information may be continuously output while the face is wearing the face pose angle detection device. The face pose angle detection device provided in the above embodiments of this disclosure can be fixed to a person's head. A data acquisition component disposed on the device can collect facial pose information and determine the face pose angle based on the pose information, thereby realizing the detection of the face pose angle.

[0049] See also Figure 5 This diagram illustrates an application scenario of the face pose angle detection device of this disclosure. Figure 5In the scenario shown, the tester wears the disclosed facial pose angle detection device and faces the screen of the facial recognition device. The tester slightly rotates their head until they hear the audio player on the facial pose angle detection device indicating that the vertical and horizontal distances are the same. The face is then held still for 3 seconds to allow the calculation module in the facial pose angle detection device to record the current pose information, which is the reference pose angle. Then, the head can be slightly rotated to determine the real-time facial pose angle. The calculation module determines the true value of the facial pose angle based on the real-time pose angle and the reference pose angle. The true value of the facial pose angle is compared with the pose angle output by the facial recognition device to verify whether the pose angle calculation result of the facial recognition device meets the requirements.

[0050] See Figure 6 This illustrates a flowchart 600 of an embodiment of a face pose angle detection method according to the present disclosure. For example... Figure 6 As shown, in this embodiment, the face pose angle detection device is worn on a person's head, and each distance sensor is in contact with the face surface. The device can be stably worn on the head by adjusting the fixing components. The method in this embodiment can be executed by the calculation module of the face pose angle detection device. The method in this embodiment may include the following steps:

[0051] Step 601: Obtain the real-time pose information acquired by the data acquisition component.

[0052] In this embodiment, the computing module can acquire real-time pose information collected by the data acquisition components. These data acquisition components may include a gyroscope sensor, an accelerometer sensor, a magnetometer sensor, and multiple distance sensors. The real-time pose information includes real-time attitude information and real-time distance information. The real-time attitude information can be information collected by the gyroscope sensor, accelerometer sensor, and magnetometer sensor, and may include the head rotation angular velocity collected by the gyroscope sensor, the head rotation acceleration collected by the accelerometer sensor, and the head magnetic heading angle collected by the magnetometer sensor. The real-time distance information may include the distances transmitted by each distance sensor. Each distance sensor, gyroscope sensor, accelerometer sensor, and magnetometer sensor can communicate with the computing module through a pre-set communication port, thereby transmitting the collected data to the computing module in real time.

[0053] Step 602: Determine the facial reference plane based on real-time distance information.

[0054] In this embodiment, the facial reference plane can be determined based on real-time distance information. Specifically, it can be determined whether the distance measured by the distance sensor on the upper sub-support component is equal to the distance measured by the distance sensor on the lower sub-support component. Simultaneously, it can be determined whether the line connecting the projection points of the beams from these two distance sensors passes through the center of the facial recognition camera and is perpendicular to the top and bottom edges of the screen. It can also be determined whether the distance measured by the distance sensor on the left sub-support component is equal to the distance measured by the distance sensor on the right sub-support component. When both pairs of distances are detected to be equal, the plane determined by each distance sensor can be considered the facial reference plane.

[0055] Step 603: Determine the real-time face pose angle based on the real-time pose information.

[0056] The calculation module can also determine the real-time facial pose angles based on real-time pose information. Specifically, it can determine the predicted facial pose angles using the transformation matrix between the coordinate system of the facial recognition device and the head coordinate system, combined with data measured by the gyroscope sensor. Furthermore, it can use the measurement results from the accelerometer and magnetometer to correct the predicted pose angles, obtaining the corrected pose angles. These can then be used as the real-time pose angles.

[0057] Step 604: Determine the reference pose angle based on the face reference plane.

[0058] After determining the facial reference plane, the reference pose angle can be determined based on the pose information corresponding to the facial reference plane. Specifically, the reference pose angle can be determined using the same principle as in step 603.

[0059] Step 605: Determine the true value of the face pose angle based on the real-time face pose angle and the reference pose angle.

[0060] After determining the real-time and reference pose angles of the face, the true value of the face pose angle can be determined based on these two values. Specifically, the real-time face pose angle at any given time can be subtracted from the reference pose angle, and the resulting value can be used as the true value of the face pose angle. Alternatively, the real-time face pose angle can be multiplied by a preset coefficient, and then subtracted from the reference pose angle, and the resulting value can be used as the true value of the face pose angle.

[0061] The face pose angle detection method provided in the above embodiments of this disclosure can determine the face reference plane by measuring the distances from two pairs of distance sensors. Based on this face reference plane, the true value of the face pose angle is determined, which is not only convenient and efficient in calculation, but also improves the accuracy of the calculation results.

[0062] Figure 7 The flowchart 700 of another embodiment of the face pose angle detection method of this disclosure is shown.

[0063] like Figure 7 As shown, the face pose angle detection method in this embodiment may include the following steps:

[0064] Step 701: Obtain the real-time head rotation angular velocity collected by the gyroscope sensor, the real-time head rotation acceleration collected by the accelerometer sensor, the real-time head magnetic heading angle collected by the magnetometer sensor, and the real-time distance information collected by the distance sensor.

[0065] The real-time pose information includes real-time attitude information and real-time distance information. The real-time distance information includes the first distance, second distance, third distance, and fourth distance measured by the first distance sensor, the second distance sensor, the third distance sensor, and the fourth distance sensor, respectively.

[0066] Step 702: When it is determined that the first distance is equal to the second distance and the third distance is equal to the fourth distance, the plane located by the first distance sensor, the second distance sensor, the third distance sensor and the fourth distance sensor is used as the face reference plane.

[0067] In this embodiment, the first and second distance sensors can be respectively disposed at the ends of the upper and lower sub-support components. The third and fourth distance sensors can be respectively disposed at the ends of the left and right sub-support components. Specifically, the upper sub-support component can be fixed to the upper part of the positioning component, ensuring that the end of the upper sub-support component with the first distance sensor is close to the upper part of the face, such as the forehead. The lower sub-support component can be fixed to the lower part of the positioning component, ensuring that the end of the lower sub-support component with the second distance sensor is close to the lower part of the face, such as the chin. The left sub-support component can be fixed to the left side of the positioning component, ensuring that the end of the left sub-support component with the third distance sensor is close to the left side of the face, such as below the left cheekbone. The right sub-support component can be fixed to the right side of the positioning component, ensuring that the end of the right sub-support component with the fourth distance sensor is close to the right side of the face, such as below the right cheekbone. When the first and second distances are equal, the nasal midline of the face is considered to be parallel to the plane of the face recognition device. When the third distance and the fourth distance are equal, the horizontal line perpendicular to the midline of the nose on the face is considered parallel to the plane of the face recognition device. When the first distance and the second distance are equal, and the third distance and the fourth distance are also equal, the face plane is considered parallel to the plane of the face recognition device. This face plane is then used as the face reference plane.

[0068] Step 703: Determine the first attitude information based on the real-time head rotation angular velocity; determine the second attitude information based on the real-time head rotation acceleration and the real-time head magnetic heading angle; fuse the first attitude information and the second attitude information to determine the real-time face attitude angle.

[0069] In this embodiment, the first posture information can be determined based on the real-time head rotation angular velocity. Specifically, the transformation matrix between the face recognition device coordinate system and the head coordinate system can be determined first. Then, based on the transformation matrix and the head rotation angular velocity output by the gyroscope sensor, the horizontal rotation angle, roll angle, and pitch angle are determined as the first posture information.

[0070] Simultaneously, the second attitude information is determined based on the real-time head rotation acceleration and the real-time head magnetic heading angle. Specifically, a matrix of acceleration measurements along the three axes of the head coordinate system can be determined first. Combining this with the real-time head rotation acceleration measured by the accelerometer, the pitch and roll angles can be obtained. Similarly, a matrix of acceleration measurements along the three axes of the head coordinate system can be determined. Combining this with the real-time head magnetic heading angle, the rotation angle can be obtained, serving as the second attitude information.

[0071] Finally, the first pose information and the second pose information are fused to determine the real-time facial pose angle. Various methods can be used for fusion. For example, the first pose information and the second pose information can be weighted. Alternatively, depending on the specific application scenario, either the first pose information or the second pose information can be selected as the final pose information.

[0072] In some optional implementations of this embodiment, the fusion of the first attitude information and the second attitude information can be achieved through the following sub-steps:

[0073] Sub-step 7031: Establish the state prediction equation, the covariance equation of the prediction error, and the state update equation.

[0074] Sub-step 7032: Determine the process noise covariance based on the error of the gyroscope sensor.

[0075] Sub-step 7033: Determine the fusion error based on the errors of the gyroscope sensor, accelerometer sensor, and magnetometer sensor.

[0076] Sub-step 7034: Determine the measurement noise covariance based on the fusion error.

[0077] Sub-step 7035: Substitute the first pose information, the second pose information, the process noise covariance, and the measurement noise covariance into the state prediction equation, the prediction error covariance equation, and the state update equation to determine the real-time face pose angle.

[0078] This implementation method may specifically include the following solutions:

[0079] First, establish the face recognition device coordinate system O-XYZ and the head coordinate system o-xyz. The face recognition device coordinate system is formed by rotating the head coordinate system around the z-axis, x-axis, and y-axis, respectively. The conversion relationship between the results of degrees (horizontal rotation angle), θ degrees (roll angle), and ψ degrees (pitch angle) can be expressed as follows:

[0080]

[0081] Where, x s y s z s These are the coordinates in the facial recognition device's coordinate system, x b y b z b These are coordinates in the head coordinate system. It is a rotation matrix, represented by Euler angles as follows:

[0082]

[0083] Due to limitations in human motor function, the pitch angle of a person's head relative to a face recognition device cannot reach ±90 degrees. Furthermore, national and industry standards do not require a maximum facial pose angle of 90 degrees. Therefore, there is no pose angle singularity problem; that is, the face recognition pose angle can be calculated using Euler angles. When the head moves, the head pose angle can be calculated using equation (2) and the output data from the gyroscope sensor as follows:

[0084] Horizontal rotation angle:

[0085] Roll angle: θ = arctanT 32 (4)

[0086] Pitch angle:

[0087] Because gyroscope sensors have high dynamic accuracy, but accumulate errors after long-term operation, while accelerometers and magnetometers have poor dynamic accuracy but high static angle measurement accuracy, the measurement results of accelerometers and magnetometers are used to calibrate gyroscope sensors to improve calculation accuracy.

[0088] The matrix of acceleration measurements on the three axes of the head coordinate system can be represented as A:

[0089]

[0090] In the above formula, g is the acceleration due to gravity.

[0091] The calculated pitch and roll angles obtained from the accelerometer are as follows:

[0092] Pitch angle:

[0093] Roll angle:

[0094] The matrix of magnetic field strength measurements on the three axes of the head coordinate system can be represented as M:

[0095]

[0096] Assume the local magnetic declination (the angle between true north and magnetic north) is known. Then the magnetic heading angle of the head relative to true north It can be solved as follows:

[0097]

[0098] Based on the Kalman filter principle, the calculated angle from the gyroscope sensor is used as the predicted angle, and the calculated angles from the accelerometer and magnetometer are used as the real-time measured angles. A state prediction equation for the head attitude angle and a covariance equation for the prediction error are then established.

[0099]

[0100] In the formula: The optimal predicted value for the head pose angle at time k-1 is... Using the state at time k-1 for time k Predicted head pose angle, A k Let W be the state transition matrix. k For the process noise of the gyroscope sensor; P k,k-1 It is using the state P at time k-1 at time k. k-1,k-1 The covariance matrix of the prior prediction error, P k-1,k-1 for The corresponding covariance matrix of the prediction error, Q k This is the process noise covariance matrix of the gyroscope sensor during attitude angle prediction.

[0101] Based on the measured head attitude angle at time k and the optimal predicted head attitude angle at time k-1, the state update equation for the head attitude angle can be derived as follows:

[0102]

[0103] P k =(1-G k H k )P k,k-1 (14)

[0104]

[0105] In the formula: Z is the optimal predicted value for the updated head pose angle. kFor the angle calculated by the accelerometer and magnetometer at time k, G k P is the Kalman filter gain at head pose angle k. k,k-1 H is the covariance of the prior prediction error at head pose angle k. k For the measurement matrix, P k R is the covariance of the posterior prediction error at head pose angle k. k Let be the measurement noise covariance of the accelerometer and magnetometer at time k.

[0106] In the Kalman filter recursive process, the noise covariance matrix Q needs to be updated at each step. k and measurement noise covariance R k Q k The error of the gyroscope sensor can be obtained through experimental measurement. R k The distance can be obtained by calculating the second-order center distance of the errors of the gyroscope sensor, accelerometer sensor, and magnetometer sensor, as follows:

[0107] Assume the true attitude angle at a certain moment is The attitude angle obtained after Kalman filtering the gyroscope sensor measurement can be expressed as the sum of the true attitude angle and the gyroscope sensor error:

[0108]

[0109] The difference in attitude angle between two consecutive moments of the gyroscope sensor is:

[0110]

[0111] In the formula: Let be the error in the gyroscope sensor's attitude angle prediction at time k. The error in the gyroscope sensor attitude angle prediction at time k-1 is denoted as .

[0112] The attitude angle obtained after Kalman filtering the measurements from the accelerometer and magnetometer can also be expressed as the sum of the true attitude angle and the errors from the accelerometer and magnetometer:

[0113]

[0114] The difference in attitude angle between the accelerometer and magnetometer at two consecutive moments is:

[0115]

[0116] In the formula: Let k be the error in the attitude angle prediction by the accelerometer and magnetometer at time k. This represents the error in the attitude angle prediction by the accelerometer and magnetometer at time k-1.

[0117] Therefore, the fusion error f of the angles calculated by the gyroscope sensor, accelerometer sensor, and magnetometer at any two adjacent moments is:

[0118]

[0119] Solve for the second-order center distance R of the fusion error f. k as follows:

[0120]

[0121] Due to δ g and δ a Since they are mutually independent zero-mean white noise sequences, they satisfy E[δ] g ·δ a ] = 0. The errors of a single sensor at two adjacent moments are independent of each other, that is... Gyroscope sensors have high stability and measurement accuracy; their errors are much smaller than those of accelerometers and magnetometers, and can be ignored. The error changes of accelerometers and magnetometers are small between any two adjacent moments, and can be approximated. Therefore, substituting the above conclusions into formula (21), we obtain the simplified measurement noise covariance of the accelerometer and magnetometer at time k as follows:

[0122]

[0123] Step 704: Determine the reference pose angle based on the face reference plane.

[0124] In this step, the principle for determining the reference attitude angle is similar to that in step 703, and will not be repeated here.

[0125] Step 705: Determine the true value of the face pose angle based on the difference between the real-time face pose angle and the reference pose angle.

[0126] After determining the real-time facial pose angle at any given time, subtract it from the reference pose angle and use the difference as the true value of the facial pose angle.

[0127] After explaining the calculation principle, the attitude angles can be calculated using the following steps:

[0128] ① Calculate the initial attitude angle and the initial process noise covariance.

[0129] After the calculation module is powered on, at time k=0, the head posture angle (horizontal rotation angle) is calculated using the output data of the gyroscope sensor and formulas (3), (4), and (5). Roll angle θ, pitch angle ψ). The head posture angle measurements are obtained using an accelerometer and a magnetometer and formulas (7), (8), and (10). The difference between the head posture angle measurements and the head posture angle is taken as the measurement error of the gyroscope, and the initial process noise covariance Q of the gyroscope sensor is calculated. k And assume that the initial value of the covariance of the prediction error is P0 = 1.

[0130] ② Calculate the predicted attitude angle at time k=1

[0131] At k=1, the head attitude angle is calculated using the output data of the gyroscope sensor and formulas (3), (4), and (5), and the current attitude angle is predicted using formula (11). Using the initial process noise covariance Q k The initial value of the covariance of the prediction error P0 and the calculation of the covariance P of the prior prediction error using formula (12) are used. k,k-1 .

[0132] ③ Calculate the Kalman filter gain

[0133] The measurement noise covariance R is calculated using the output data from the accelerometer and magnetometer and formula (22). k Then, the Kalman filter gain G is calculated using formula (15). k .

[0134] ④ Update the covariance of the optimal predicted head pose angle and the posterior prediction error.

[0135] The head posture angle measurement value Z is calculated using the output data of the accelerometer and magnetometer and formulas (7), (8), and (10). k Then, the optimal predicted value of the head posture angle is calculated using formula (13). (Horizontal rotation angle) The roll angle θ and pitch angle ψ are used to calculate the covariance P of the posterior prediction error of the head attitude angle using formula (14). k .

[0136] ⑤ Calculate the predicted attitude angle at time k > 1

[0137] The head attitude angle is calculated using the output data of the gyroscope sensor and formulas (3), (4), and (5), and the new attitude angle is predicted using formula (11). Using process noise covariance Q k The covariance P of the posterior prediction error obtained in step ④ k The covariance P of the new prior prediction error is calculated using formula (12). k,k-1 .

[0138] ⑥ Recursive calculation

[0139] Repeat steps ③ through ⑤.

[0140] The optimal predicted head posture angle calculated in step ④ above (Horizontal rotation angle) The roll angle θ and pitch angle ψ are the calculated real-time face pose angles. Using the same method, the reference pose angles when the face is on the face reference plane are calculated and denoted as... Then, subtracting the calculated real-time facial pose angle from the reference pose angle at the aforementioned facial reference plane yields the true value of the facial pose angle after rotation relative to the facial recognition device, i.e.:

[0141]

[0142] The true value of the face roll angle is θ - θ0.

[0143] The true value of the face pitch angle is ψ - ψ0.

[0144] The face pose angle detection method provided in the above embodiments of this disclosure measures the distance between four laser rangefinders and the screen of the face recognition device, making the face parallel to the screen of the face recognition device. This allows for the rapid and accurate measurement of the reference pose angle of the face reference plane, making the head coordinate system and the face recognition device coordinate system parallel coordinate systems at this time. Subsequently, any rotation angle of the head can be regarded as a rotation angle relative to this reference pose angle, simplifying the subsequent calculation process of the face pose angle.

[0145] Further reference Figure 8 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a face pose angle detection device, which corresponds to the method embodiment shown in Figure 1, and the device can be applied to various electronic devices.

[0146] like Figure 8 As shown, the face pose angle detection device 800 of this embodiment includes: a data acquisition unit 801, a reference plane determination unit 802, a pose angle determination unit 803, a reference pose angle determination unit 804, and a pose angle true value determination unit 805.

[0147] The data acquisition unit 801 is configured to acquire real-time pose information collected by the data acquisition component, wherein the real-time pose information includes real-time attitude information and real-time distance information.

[0148] The reference plane determination unit 802 is configured to determine the face reference plane based on real-time distance information.

[0149] The pose angle determination unit 803 is configured to determine the real-time pose angle of the face based on the real-time pose information.

[0150] The reference attitude angle determination unit 804 is configured to determine the reference attitude angle based on the face reference plane.

[0151] The pose angle truth determination unit 805 is configured to determine the true value of the face pose angle based on the real-time face pose angle and the reference pose angle.

[0152] In addition, one embodiment of this disclosure also provides a non-transitory computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the above-described method, including, for example,: acquiring real-time pose information collected by a data acquisition component, wherein the real-time pose information includes real-time posture information and real-time distance information; determining a face reference plane based on the real-time distance information; determining a real-time face pose angle based on the real-time posture information; determining a reference pose angle based on the face reference plane; and determining the true value of the face pose angle based on the real-time face pose angle and the reference pose angle.

[0153] In summary, the technical solution disclosed herein utilizes multiple laser rangefinders to measure the distance between the face and the screen of the face recognition device, ensuring that the face is parallel to the screen. This allows for the rapid and accurate acquisition of the reference pose angle of the face reference plane. Furthermore, the true value of the face pose angle is obtained by subtracting the reference pose angle from the real-time face pose angle at any given time, simplifying the calculation process for the face pose angle.

[0154] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A face pose angle detection device, comprising: The system includes a positioning component, multiple supporting components, a fixing component, an adjustment component disposed between the supporting component and the fixing component, a data acquisition component, and a computing module. The positioning component is used to provide support for the plurality of supporting components and the fixing component. The plurality of supporting components, the fixing component and the positioning component are fixedly connected to the positioning component to keep the relative position between the positioning component and the face fixed. The adjustment component is used to adjust the angle of the data acquisition component; The positioning component and / or each of the supporting components are provided with a data acquisition component, which is used to acquire the pose information of the face and send the pose information to the calculation module. The calculation module is used to determine the pose angle of the face based on the pose information; The positioning component is a ring-shaped structure, and the size of the ring-shaped structure is larger than the size of a human head; The fixing component is fixed to the back of the human head; The supporting component includes a horizontal sub-support component and a vertical sub-support component; the adjusting component includes a first sub-adjusting component and a second sub-adjusting component; The first sub-adjustment component is disposed between the horizontal sub-support component and the positioning component, and the second sub-adjustment component is disposed between the vertical sub-support component and the positioning component; The positioning component has a symmetrical structure, and the axis of symmetry of the positioning component coincides with the midline of the nose of the human face; The vertical sub-support component includes an upper sub-support component and a lower sub-support component, which are arranged on the axis of symmetry. The lateral sub-support component includes a left sub-support component and a right sub-support component, which are arranged on the same horizontal line perpendicular to the axis of symmetry, and are symmetrical about the axis of symmetry. The supporting component is a strip-shaped structure. One end of the strip-shaped structure is fixed to the positioning component, and the other end is located inside the annular structure formed by the positioning component. The data acquisition component is provided on this end. The positioning component and the supporting component are made of transparent material. The data acquisition component includes four distance sensors. The angle of the distance sensor on the left sub-support component is the same as the angle of the distance sensor on the right sub-support component, and the angle of the distance sensor on the upper sub-support component is the same as the angle of the distance sensor on the lower sub-support component. The four distance sensors are respectively positioned on the forehead, chin, and below the cheekbone of a person to measure the distance between different positions of the face and the screen of the face recognition device; The distance sensor is a wired laser rangefinder. The adjustment component adjusts the angle of the laser rangefinder so that the lasers emitted by each laser rangefinder can be projected onto the screen simultaneously. The connecting wires of the laser rangefinders are hidden in the positioning component and the support component.

2. The face pose angle detection device according to claim 1, wherein, The adjusting component is a shim; The number of shims used to adjust the upper sub-support component is the same as the number of shims used to adjust the lower sub-support component; The number of shims used to adjust the left sub-support component is the same as the number of shims used to adjust the right sub-support component.

3. The face pose angle detection device according to claim 2, wherein, The gasket has a triangular or trapezoidal cross-section.

4. The face pose angle detection device according to claim 1, wherein, The supporting component has multiple through holes, and the positioning component has protrusions that fit the through holes, for fixing the supporting component to the positioning component; The adjustment component is disposed between the positioning component and the support component, and is used to adjust the angle between the plane in which the support component and the positioning component are located.

5. The face pose angle detection device according to claim 4, wherein, The adjusting component is provided with a through hole of the same size as the through hole on the supporting component, and the adjusting component and the supporting component are fixed through the through hole.

6. The face pose angle detection device according to claim 1, wherein, The fixing component includes multiple fixing straps; The multiple fixing straps are connected through connecting holes and secured with buckles.

7. The face pose angle detection device according to claim 1, wherein, The data acquisition component includes multiple distance sensors, gyroscope sensors, accelerometer sensors, and magnetometers; The plurality of distance sensors are used to measure the distance between the face and the screen of the face recognition device, and the distance sensors are laser rangefinders; The gyroscope sensor is used to measure the angular velocity of head rotation. The accelerometer is used to measure the head rotation acceleration of a person's head; The magnetic sensor is used to measure the magnetic heading angle of a human head.

8. The face pose angle detection device according to claim 7, wherein, The supporting component is a strip-shaped structure, one end of which is fixed to the positioning component, and the other end of which is equipped with a distance sensor; The gyroscope sensor, accelerometer, and magnetometer are disposed at the connection point between the strip structure and the positioning component, or are disposed on the positioning component.

9. The face pose angle detection device according to claim 8, wherein, The face pose angle detection device further includes: an audio output component connected to the calculation module; The calculation module is also used to determine the prompt information based on the distances measured by the various distance sensors. The audio output component plays the prompt message aloud.

10. A method for detecting facial pose angles, applied to a facial pose angle detection device as described in any one of claims 1 to 9, wherein the facial pose angle detection device is worn on a person's head, and the method comprises: The system acquires real-time pose information collected by a data acquisition component. This real-time pose information includes real-time posture information and real-time distance information. The real-time distance information includes a first distance, a second distance, a third distance, and a fourth distance measured by a first distance sensor, a second distance sensor, a third distance sensor, and a fourth distance sensor, respectively. The first and second distance sensors are respectively located at the ends of the upper and lower sub-support components, respectively. The third and fourth distance sensors are respectively located at the ends of the left and right sub-support components, respectively. The first distance sensor is located at the forehead, the second distance sensor at the chin, and the third and fourth distance sensors are located below the cheekbone. These sensors are used to measure the distance between different positions of the face and the screen of the face recognition device. The first, second, third, and fourth distance sensors are laser rangefinders, and the lasers emitted by each laser rangefinder can be simultaneously projected onto the screen. The facial reference plane is determined based on real-time distance information; Based on the real-time pose information, determine the real-time pose angle of the face; Based on the aforementioned facial reference plane, determine the reference pose angle; The true value of the face pose angle is determined based on the real-time face pose angle and the reference pose angle. The step of determining the facial reference plane based on real-time distance information includes: When it is determined that the first distance is equal to the second distance and the third distance is equal to the fourth distance, the plane located by the first distance sensor, the second distance sensor, the third distance sensor and the fourth distance sensor is used as the face reference plane.

11. The method according to claim 10, wherein, The acquisition of real-time pose information collected by the data acquisition component includes: The system acquires real-time head rotation angular velocity from the gyroscope sensor, real-time head rotation acceleration from the accelerometer sensor, real-time head magnetic heading angle from the magnetometer sensor, and real-time distance information from the distance sensor.

12. The method according to claim 11, wherein, Determining the real-time facial pose angle based on the real-time pose information includes: The first posture information is determined based on the real-time head rotation angular velocity; The second attitude information is determined based on the real-time head rotation acceleration and the real-time head magnetic heading angle. The first pose information and the second pose information are fused to determine the real-time pose angle of the face.

13. The method according to claim 12, wherein, The step of fusing the first pose information and the second pose information to determine the real-time facial pose angle includes: Establish the state prediction equation, the covariance equation of the prediction error, and the state update equation; The process noise covariance is determined based on the error of the gyroscope sensor. The fusion error is determined based on the errors of the gyroscope sensor, the accelerometer sensor, and the magnetometer sensor. Based on the fusion error, determine the measurement noise covariance; Substituting the first pose information, the second pose information, the process noise covariance, and the measurement noise covariance into the state prediction equation, the prediction error covariance equation, and the state update equation, the real-time face pose angle is determined.

14. The method of claim 10, wherein, The step of determining the true value of the face pose angle based on the real-time face pose angle and the reference pose angle includes: The true value of the face pose angle is determined based on the difference between the real-time face pose angle and the reference pose angle.

15. A face pose angle detection device, comprising: The data acquisition unit is configured to acquire real-time pose information collected by the data acquisition component. The real-time pose information includes real-time posture information and real-time distance information. The real-time distance information includes a first distance, a second distance, a third distance, and a fourth distance measured by a first distance sensor, a second distance sensor, a third distance sensor, and a fourth distance sensor, respectively. The first distance sensor and the second distance sensor are respectively located at the ends of the upper and lower sub-support components. The third distance sensor and the fourth distance sensor are respectively located at the ends of the left and right sub-support components. The first distance sensor is located at the forehead, the second distance sensor at the chin, and the third and fourth distance sensors are located below the cheekbone. These sensors are used to measure the distance between different positions of the face and the screen of the face recognition device. The first distance sensor, the second distance sensor, the third distance sensor, and the fourth distance sensor are laser rangefinders, and the lasers emitted by each laser rangefinder can be simultaneously projected onto the screen. The reference plane determination unit is configured to determine the face reference plane based on real-time distance information; The pose angle determination unit is configured to determine the real-time pose angle of the face based on the real-time pose information. The reference pose angle determination unit is configured to determine the reference pose angle based on the face reference plane; The pose angle truth determination unit is configured to determine the true value of the face pose angle based on the real-time face pose angle and the reference pose angle. The reference plane determination unit is further configured to: When it is determined that the first distance is equal to the second distance and the third distance is equal to the fourth distance, the plane located by the first distance sensor, the second distance sensor, the third distance sensor and the fourth distance sensor is used as the face reference plane.

16. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the face pose angle detection method as described in any one of claims 10 to 14.

Images