Face orientation detection method and device, equipment and medium
Through face key point detection and pose estimation, combined with the probability calculation of rotation angle threshold, the problem of large consumption and delay of face orientation detection in the prior art is solved, and efficient and real-time face orientation detection is achieved.
Patent Information
- Application Number
- CN202410211489.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-26
- Publication Date
- 2025-08-26
AI Technical Summary
The existing face orientation detection methods rely on deep learning and machine learning models, resulting in high data acquisition and labeling costs, high computational complexity, and prone to delays in practical applications.
By detecting the key point of the face in the image to be tested, pose estimation is performed using the first face key point and the second face key point of the preset rotation angle, the rotation angle of the face is calculated, and probability calculation is performed based on the rotation angle threshold of the preset orientation, and the face orientation is determined, replacing the traditional classification detection model.
It reduces the cost of data acquisition and labeling, reduces computing resource consumption, improves the real-timeness of face orientation detection, reduces the computational complexity, and adjusts the detection orientation without retraining the model.
Smart Images
Figure CN120544243A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a face orientation detection method, apparatus, device, and medium. Background Art
[0002] Machine learning and deep learning are hot research areas in artificial intelligence. Face orientation detection typically uses deep learning classification and machine learning methods to process images and output face orientation detection results. For example, a single-channel image is captured by a monocular camera and fed into a pre-trained classification detection model to output the face orientation detection results.
[0003] Existing technologies use deep learning classification methods, such as convolutional neural networks, trained on large amounts of labeled data to achieve face orientation recognition. Machine learning methods rely on feature engineering and classification algorithms, extracting features from facial images and building models to predict face orientation. Both approaches rely on training models with large amounts of labeled data, resulting in high data collection and annotation costs, high computational complexity, and significant computing resources, which can easily lead to delays in practical applications. Summary of the Invention
[0004] In order to solve the above technical problems, the present disclosure provides a face orientation detection method, apparatus, device and storage medium.
[0005] In a first aspect, an embodiment of the present disclosure provides a method for detecting a face orientation, comprising:
[0006] Perform key point detection on the face in the image to be tested to obtain the first face key point;
[0007] Estimating the face pose based on the first and second facial key points to obtain a rotation angle of the face; wherein the second facial key point is a pre-set facial key point with a known rotation angle;
[0008] Calculating the probability of the face facing each preset orientation based on the rotation angle of the face and the rotation angle threshold corresponding to each preset orientation; wherein the greater the difference between the rotation angle of the face and the rotation angle threshold, the smaller the probability;
[0009] A face orientation detection result of the face is determined based on the probability that the face is facing each preset orientation.
[0010] In a second aspect, an embodiment of the present disclosure provides a face orientation detection device, comprising:
[0011] A detection module is used to detect key points of the face in the image to be tested and obtain the first face key points;
[0012] a generating module, configured to estimate the pose of the face based on the first facial key points and the second facial key points to obtain a rotation angle of the face; wherein the second facial key points are pre-set facial key points with a known rotation angle;
[0013] a prediction module, configured to calculate, based on the rotation angle of the face and the rotation angle threshold corresponding to each preset orientation, a probability that the face is facing each preset orientation; wherein the greater the difference between the rotation angle of the face and the rotation angle threshold, the smaller the probability;
[0014] The determination module is used to determine the face orientation detection result of the face according to the probability that the face is facing each preset orientation.
[0015] In a third aspect, an embodiment of the present disclosure provides an electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor for reading the executable instructions from the memory and executing the instructions to implement the face orientation detection method described in the first aspect above.
[0016] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the face orientation detection method described in the first aspect is implemented.
[0017] In a fifth aspect, an embodiment of the present disclosure provides a vehicle, comprising the face orientation detection device as described in the second aspect above, or the electronic device as described in the third aspect above.
[0018] Compared with the prior art, the technical solution provided by the embodiment of the present disclosure has the following advantages: by performing key point detection on the face in the image to be tested, a first face key point is obtained, and based on the first face key point and a preset second face key point with a known rotation angle, the rotation angle of the face is obtained, and then, according to the rotation angle of the face and the rotation angle threshold corresponding to each preset orientation, the probability of the face facing each preset orientation is calculated, and the face orientation detection result of the face is determined according to the probability of the face facing each preset orientation. Therefore, there is no need to classify the detection model when detecting the face orientation, but instead the second face key point and the rotation angle threshold of each preset orientation are set, and the rotation angle threshold of each preset orientation is calculated by the first face key point. The rotation angle of the face is obtained by combining the first face key point and the second face key point, and the probability calculation is performed in combination with the rotation angle threshold to obtain the face orientation detection result. Mathematical calculation is used to replace the classification detection model based on deep learning / machine learning, which reduces the data collection and annotation costs of the classification detection model. In addition, only the face key point detection model needs to be introduced to perform key point detection to obtain the face key points. The face key point detection model consumes less computing resources than the classification detection model, which solves the problem of large computing resource consumption and easy delay in actual application in the existing face orientation detection method, reduces the computational complexity, reduces computing resource consumption, and improves the real-time performance of face orientation detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0020] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0021] Figure 1 A schematic flow chart of a face orientation detection method provided by an embodiment of the present disclosure;
[0022] Figure 2 A schematic flow chart of another face orientation detection method provided by an embodiment of the present disclosure;
[0023] Figure 3 A schematic diagram of a flow chart for calculating the probability of a face facing a target provided by an embodiment of the present disclosure;
[0024] Figure 4 A schematic structural diagram of a face orientation detection device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0025] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.
[0026] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0027] Figure 1 This is a flow chart of a face orientation detection method provided in an embodiment of the present disclosure. The method provided in an embodiment of the present disclosure can be executed by a face orientation detection device, which can be implemented using software and / or hardware and can be integrated into any electronic device with computing capabilities, such as a vehicle-mounted terminal.
[0028] like Figure 1 As shown, the face orientation detection method provided by the embodiment of the present disclosure may include:
[0029] Step 101: perform key point detection on a face in a test image to obtain a first face key point.
[0030] In this embodiment, an image to be tested is obtained and facial key point detection is performed on the image to obtain a first facial key point of the face in the image to be tested. The image to be tested may be an image captured during face orientation detection. For example, a camera is provided in a vehicle to capture images within the vehicle, and the image captured by the camera serves as the image to be tested.
[0031] As an example, the image to be tested is input into a pre-trained facial key point detection model for key point detection to obtain the first facial key point of the face in the image to be tested. The facial key point detection model can be obtained based on machine learning or deep learning technology. The input of the facial key point detection model is an image, and the output is the facial key point in the image.
[0032] Step 102: Estimate the face pose based on the first facial key point and the second facial key point to obtain a rotation angle of the face.
[0033] In this embodiment, the second facial key point is a pre-set facial key point with a known rotation angle. Optionally, the face represented by the second facial key point can be a face facing the camera or a face at other rotation angles, that is, the position of the second facial key point relative to the camera is known, and no specific restrictions are made here.
[0034] There are many ways to determine the rotation angle of the face in the image to be tested.
[0035] As a possible implementation method, the face pose is estimated using a camera pose estimation function. The camera pose estimation function is used to estimate the camera pose. The camera pose includes the camera's rotation and translation matrices. The camera pose estimation function is, for example, the solvePnP function. The solvePnP function estimates the camera's rotation and translation matrices through the positions of known three-dimensional points in the world coordinate system and the projection points on the image. In this implementation method, the fitting method of the camera pose estimation function is adopted. The calibrated camera parameters are obtained in advance through camera calibration. The camera parameters include intrinsic parameters, extrinsic parameters and distortion parameters. The calibrated camera parameters, the first face key point and the second face key point are used as inputs to the camera pose estimation function. The output rotation vector and translation vector are used to represent the rotation vector and translation vector of the face relative to the camera, so as to express the relative pose of the face in space. Optionally, the camera parameters are iterated using the calibrated camera parameters as initial values. During each iteration, the first facial key point and the second facial key point are converted to the same coordinate system using the current camera parameters, and the first facial key point and the second facial key point in the same coordinate system are compared until the error between the first facial key point and the second facial key point in the same coordinate system is minimized. Then, the camera parameters corresponding to the minimum error are converted into the rotation angle of the face.
[0036] As another possible implementation method, the first facial key point is compared with the second facial key point of the standard face to obtain the angle between the face represented by the first facial key point and the face represented by the second facial key point. Based on the angle and the known rotation angle of the second facial key point, the rotation angle of the face in the test image is generated.
[0037] Step 103 , calculating based on the rotation angle of the face and the rotation angle thresholds corresponding to the respective preset orientations to obtain the probability that the face is facing the respective preset orientations.
[0038] In this embodiment, at least one orientation is preset, and a corresponding rotation angle threshold is set for each orientation. This threshold can be predetermined based on experiments and is used to represent the orientation. After the rotation angle of the face is obtained, a calculation is performed based on the rotation angle of the face and the rotation angle threshold corresponding to each preset orientation to obtain the probability of the face facing each preset orientation. The greater the difference between the rotation angle of the face and the rotation angle threshold, the lower the probability.
[0039] As an example, the rotation angles include yaw angles, pitch angles, and roll angles. The rotation angle thresholds corresponding to each preset orientation include yaw thresholds, pitch thresholds, and roll thresholds. Taking one of the preset orientations as an example, a first sub-probability is calculated based on the yaw angle of the face and the yaw threshold of that orientation. A second sub-probability is calculated based on the pitch angle of the face and the pitch threshold of that orientation. A third sub-probability is calculated based on the roll angle of the face and the roll threshold of that orientation. Based on the first, second, and third sub-probabilities, the probability that the face is facing the preset orientation is determined. Methods for comprehensively determining the final probability based on multiple sub-probabilities include, but are not limited to, square root averaging, weighted averaging, and the like.
[0040] Step 104 : determining a face orientation detection result of the face according to the probability that the face is facing each preset orientation.
[0041] In this embodiment, the greater the probability, the higher the possibility that the face is facing the preset direction. Optionally, if the probability of the face facing each preset direction is less than or equal to a threshold, it is determined that the face is not facing any preset direction. If there is a target probability greater than the threshold among the probabilities of the face facing each preset direction, the preset direction corresponding to the maximum value among the target probabilities is taken as the direction of the face. The threshold can be determined based on experiments or set according to the needs of the actual application scenario, and is not limited here.
[0042] As an example, for the voice wake-up scenario in a vehicle, the pre-set orientation may be the orientation of the device to be awakened, and a corresponding rotation angle threshold is set for the orientation of the device to be awakened. When the user says the specified wake-up word through voice collection and voice recognition, the aforementioned steps 101 to 103 are performed on the test image captured by the camera in the vehicle to obtain the probability that the user's face is facing the direction of the device to be awakened. If the probability is greater than or equal to the threshold, it is determined that the face is facing the device to be awakened, and the wake-up operation is performed at this time. If the probability is less than the threshold, it is determined that the face is not facing the device to be awakened. In some voice wake-up scenarios, face orientation detection is used to assist in voice wake-up to reduce the number of false wake-ups and improve the accuracy of voice wake-up.
[0043] According to the technical solution of the embodiment of the present disclosure, by performing key point detection on the face in the image to be tested, a first face key point is obtained, and based on the first face key point and a second face key point with a preset known rotation angle, the rotation angle of the face is obtained. Then, according to the rotation angle of the face and the rotation angle threshold corresponding to each preset orientation, calculation is performed to obtain the probability that the face is facing each preset orientation. According to the probability that the face is facing each preset orientation, the face orientation detection result of the face is determined. Therefore, there is no need to classify the detection model when detecting the face orientation. Instead, the second face key point and the rotation angle threshold of each preset orientation are set, and the face orientation is obtained through the first face key point and the second face key point. The rotation angle is combined with the rotation angle threshold to perform probability calculation to obtain the face orientation detection result. Mathematical calculation is used to replace the classification detection model based on deep learning / machine learning, and only the face key point detection model needs to be introduced. The face key point detection model consumes less computing resources than the classification detection model, which solves the problem of large computing resource consumption and easy delay in practical application in the existing face orientation detection method. It reduces the computational complexity, reduces the computing resource consumption, improves the real-time performance of face orientation detection, and reduces the data collection and labeling cost of the classification detection model. When adding / reducing the orientation that needs to be paid attention to, it is only necessary to increase / decrease the rotation angle threshold without retraining the model.
[0044] Based on the above embodiments, Figure 2 A flow chart of another face orientation detection method provided by an embodiment of the present disclosure is shown as follows: Figure 2 As shown, the method includes:
[0045] Step 201: perform key point detection on the face in the image to be tested to obtain the first face key point.
[0046] Step 202: Using the calibrated camera parameters, the first facial key points, and the second facial key points as inputs to a camera pose estimation function to obtain a rotation vector and a translation vector.
[0047] The second facial key point is a pre-set facial key point with a known rotation angle. The first facial key point is a two-dimensional facial key point in the image coordinate system, and the second facial key point is a three-dimensional facial key point in the world coordinate system.
[0048] In this embodiment, a camera pose estimation function is used to determine the rotation angle of the face. The camera pose estimation function uses a camera projection model to establish the relationship between the world coordinate system and the image coordinate system. Using the input three-dimensional points and the corresponding projection points on the two-dimensional image, the camera pose estimation function estimates the initial pose of the camera. It then iteratively adjusts the camera's rotation and translation to minimize the error between the projection points of the three-dimensional points on the image and the true two-dimensional points. When the optimization process converges, the camera pose estimation function outputs the camera's rotation vector and translation vector to describe the camera's pose in space. In this step, the fitting method in the camera pose estimation function is used. The camera's intrinsic parameters, extrinsic parameters, and distortion parameters are pre-calibrated. The calibrated camera parameters, the first facial key point, and the second facial key point are used as input to the camera pose estimation function. The output rotation vector and translation vector are used to represent the rotation vector and translation vector of the face relative to the camera, thereby expressing the face's relative pose in space.
[0049] As an example, the camera parameters are iterated using the calibrated camera parameters as the initial values. During each iteration, the first facial key point is transformed from the image coordinate system to the world coordinate system using the current camera parameters, and the first facial key point and the second facial key point in the world coordinate system are compared until the error between the first and second facial key points in the world coordinate system is minimized. The camera parameters corresponding to the minimum error are converted into the rotation angle of the face. In this example, during each iteration, the first facial key point is transformed from the image coordinate system to the world coordinate system based on the calibrated camera intrinsic parameters, distortion parameters, and the currently adjusted camera rotation and translation, and compared with the second facial key point to obtain the error in this iteration. Multiple iterations are performed to minimize the error between the first and second facial key points in the world coordinate system, and the rotation vector and translation vector output by the camera pose estimation function when the error is minimized are converted into the rotation angle of the face.
[0050] As another example, the camera parameters are iterated using the calibrated camera parameters as the initial values. During each iteration, the second facial key point is transformed from the world coordinate system to the image coordinate system using the current camera parameters, and the first and second facial key points in the image coordinate system are compared until the error between the first and second facial key points in the image coordinate system is minimized. The camera parameters corresponding to the minimum error are converted into the rotation angle of the face. In this example, during each iteration, the second facial key point is transformed from the world coordinate system to the image coordinate system based on the calibrated camera intrinsic parameters, distortion parameters, and the currently adjusted camera rotation and translation, and compared with the first facial key point to obtain the error in this iteration. Multiple iterations are performed to minimize the error between the first and second facial key points in the image coordinate system, and the rotation vector and translation vector output by the camera pose estimation function when the error is minimized are converted into the rotation angle of the face.
[0051] Step 203: Convert the rotation vector and translation vector output by the camera pose estimation function into the rotation angle of the face.
[0052] In this embodiment, taking the rotation angle of the second facial key point being set to zero as an example, that is, the rotation angles yaw, pitch and roll of the second facial key point are all 0, the camera parameters corresponding to the minimum error represent the rotation vector and translation vector of the face relative to the camera. Optionally, the conversion process is as follows: the rotation vector is converted into a rotation matrix, and the rotation matrix and the translation vector are spliced to obtain a posture matrix, and the posture matrix is decomposed by Euler angles to obtain the rotation angle of the face, wherein the implementation method of converting the rotation vector into a rotation matrix includes but is not limited to the Rodriguez function, and the implementation method of performing Euler angle decomposition on the posture matrix includes but is not limited to the projection matrix decomposition algorithm, and the obtained rotation angles of the face include yaw angle, pitch angle and roll angle.
[0053] Step 204 : Using a preset probability scoring function, calculate the rotation angle of the face and the rotation angle thresholds of each orientation to obtain the probability that the face is facing each preset orientation.
[0054] In this embodiment, a probability scoring function is pre-constructed. When calculating the probability that a face is facing a preset orientation, the rotation angle of the face and the rotation angle threshold of the orientation are substituted into the probability scoring function for calculation to obtain the probability that the face is facing the orientation.
[0055] As an example, the rotation angle of a face includes a yaw angle, a pitch angle, and a roll angle, which respectively represent the heading angle, the pitch angle, and the roll angle. The rotation angle threshold corresponding to each orientation includes a yaw threshold, a pitch threshold, and a roll threshold. When calculating the probability that the face is facing each preset orientation, for each preset orientation, the yaw angle and the yaw threshold, the pitch angle and the pitch threshold, and the roll angle and the roll threshold are respectively input into the preset probability scoring function for calculation, and the first sub-probability corresponding to the yaw angle, the second sub-probability corresponding to the pitch angle, and the third sub-probability corresponding to the roll angle are output. Then, based on at least one of the first sub-probability, the second sub-probability, and the third sub-probability, the probability of the face facing the preset orientation is determined by using the square root average method.
[0056] In this example, the square root mean is calculated as follows:
[0057]
[0058] Where p1 is the first sub-probability, p2 is the second sub-probability, and p3 is the third sub-probability, and p is the probability that the face is facing the specified orientation. The above is a calculation formula for determining the probability that the face is facing the specified orientation based on the first, second, and third sub-probabilities. In actual applications, if the specified orientation has no special requirements for one or two of yaw, pitch, and roll, you can also calculate only the remaining one or two sub-probabilities. When calculating the final probability, the square root of the digit can be changed accordingly. For example, etc., no specific limitation is given here.
[0059] The probability scoring function is explained below.
[0060] In one embodiment of the present disclosure, the rotation angle threshold includes a positive value and an offset range, and the probability scoring function is set according to the following rules: when the rotation angle is equal to the positive value, the probability is determined to be the maximum value; when the rotation angle exceeds the boundary value of the offset range, the probability is determined to be zero; when the rotation angle changes from the positive value to the boundary value of the offset range, the probability decreases from the maximum value to zero.
[0061] As an example, taking the yaw threshold as an example, for the yaw threshold of a face facing the upper right side of the screen, the yaw threshold is {positive value: -20, offset range: [-10, 10]}, which means that when the yaw angle of the face is -20, the face is considered to be facing the right direction in the yaw dimension. When the yaw angle of the face is in the range of [-30, -10], there is also a certain probability that the face is facing the right direction. The probability scoring function is as follows:
[0062] p(x)=max(100-(100÷T 2×(xZ) 2 ),0)
[0063] Wherein, x represents the rotation angle of the face, Z represents the facing value, and T represents the boundary value of the offset range. When x>Z, T takes the upper boundary, otherwise, T takes the lower boundary. For this formula, when x is equal to Z, the probability score value p(x)=100 is the highest score. When x is on the boundary of the offset range or outside the offset range, p(x)=0. When x is within the offset range, the closer to the facing value Z, the higher the probability score value p(x). It should be noted that the above formula is only an example and can be adjusted as needed. As long as the setting rules of the probability scoring function are met, no specific restrictions are made here.
[0064] Step 205 : determining a face orientation detection result of the face according to the probability that the face is facing each preset orientation.
[0065] In this embodiment, refer to Figure 3 After obtaining the yaw angle, pitch angle, and roll angle of the face, when judging whether the face is facing a preset direction, the yaw angle and yaw threshold, the pitch angle and pitch threshold, and the roll angle and roll threshold are respectively input into the probability scoring function to obtain the probability score value of the yaw angle, the probability score value of the pitch angle, and the probability score value of the roll angle, and then comprehensively determine the probability of the face facing that direction.
[0066] In the disclosed embodiment, image acquisition based on a monocular camera is used in combination with facial key point detection, and only a small model for facial key point detection is additionally introduced. Compared with the method of deploying a classification detection model, computing resources are saved and computing time is reduced. Moreover, when adding / reducing orientations that need to be paid attention to, only the rotation angle threshold needs to be increased / reduced. This method can be quickly deployed and debugged without the need to retrain the model. This solves the problem of large computing resource consumption and easy delay in actual application in existing face orientation detection methods, reduces computational complexity, reduces computing resource consumption, and improves the real-time performance of face orientation detection.
[0067] The disclosed embodiment also provides a face orientation detection device.
[0068] Figure 4 This is a structural diagram of a face orientation detection device provided by an embodiment of the present disclosure, such as Figure 4 As shown, the face orientation detection device includes: a detection module 41, a generation module 42, a prediction module 43, and a determination module 44.
[0069] The detection module 41 is used to detect key points of the face in the image to be tested and obtain the first key points of the face;
[0070] A generating module 42 is configured to estimate the face pose based on the first facial key points and the second facial key points to obtain a rotation angle of the face; wherein the second facial key points are pre-set facial key points with a known rotation angle;
[0071] a prediction module 43 configured to calculate, based on the rotation angle of the face and the rotation angle threshold corresponding to each preset orientation, a probability that the face is facing each preset orientation; wherein the greater the difference between the rotation angle of the face and the rotation angle threshold, the smaller the probability;
[0072] The determination module 44 is configured to determine a face orientation detection result of the face according to the probability that the face is facing each preset orientation.
[0073] In one embodiment of the present disclosure, the generating module 42 is specifically configured to:
[0074] Iterate the camera parameters using the calibrated camera parameters as initial values. During each iteration, convert the first facial key point from the image coordinate system to the world coordinate system using the current camera parameters, and compare the first facial key point with the second facial key point in the world coordinate system until the error between the first facial key point and the second facial key point in the world coordinate system is minimized.
[0075] The camera parameters corresponding to the minimum error are converted into the rotation angle of the face.
[0076] In one embodiment of the present disclosure, the generating module 42 is specifically configured to:
[0077] Iterating the camera parameters using the calibrated camera parameters as initial values, and in each iteration, transforming the second facial key point from the world coordinate system to the image coordinate system using the current camera parameters, and comparing the first facial key point and the second facial key point in the image coordinate system until the error between the first facial key point and the second facial key point in the image coordinate system is minimized;
[0078] The camera parameters corresponding to the minimum error are converted into the rotation angle of the face.
[0079] In one embodiment of the present disclosure, the camera parameters corresponding to the minimum error represent the rotation vector and translation vector of the face relative to the camera, and the generation module 42 is specifically configured to:
[0080] Convert the rotation vector into a rotation matrix, and concatenate the rotation matrix and the translation vector to obtain a posture matrix;
[0081] The projection matrix decomposition algorithm is used to perform Euler angle decomposition on the posture matrix to obtain the rotation angle of the face.
[0082] In one embodiment of the present disclosure, the rotation angle of the face includes a yaw angle, a pitch angle, and a roll angle. The rotation angle threshold corresponding to each orientation includes a yaw threshold, a pitch threshold, and a roll threshold. The prediction module 43 is specifically configured to:
[0083] For each preset orientation, the yaw angle and the yaw threshold, the pitch angle and the pitch threshold, and the roll angle and the roll threshold are respectively input into a preset probability scoring function for calculation, and a first sub-probability corresponding to the yaw angle, a second sub-probability corresponding to the pitch angle, and a third sub-probability corresponding to the roll angle are output;
[0084] The probability that the face is facing the preset orientation is determined by adopting a square root average method according to at least one of the first sub-probability, the second sub-probability, and the third sub-probability.
[0085] In one embodiment of the present disclosure, the rotation angle threshold includes a facing value and an offset range, and the greater the difference between the rotation angle of the face and the rotation angle threshold, the smaller the probability, including:
[0086] When the rotation angle of the face is equal to the facing value, determining that the probability is maximum;
[0087] When the rotation angle of the face exceeds the boundary value of the offset range, determining the probability to be zero;
[0088] When the rotation angle of the face changes from the facing value to the boundary value of the offset range, the probability decreases from the maximum value to zero.
[0089] In one embodiment of the present disclosure, the determination module 44 is specifically configured to:
[0090] If the probability that the face is facing each preset direction is less than or equal to a threshold, it is determined that the face is not facing any preset direction;
[0091] If there is a target probability greater than the threshold among the probabilities that the face is facing each preset direction, the preset direction corresponding to the maximum value among the target probabilities is used as the direction of the face.
[0092] The face orientation detection device provided in the embodiments of the present disclosure can execute any face orientation detection method provided in the embodiments of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method. For any content not fully described in the embodiments of the present disclosure, please refer to the description of any method embodiment of the present disclosure.
[0093] An embodiment of the present disclosure further provides an electronic device, which includes one or more processors and a memory.
[0094] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.
[0095] The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may execute the program instructions to implement the methods of the embodiments of the present disclosure above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.
[0096] In one example, the electronic device may further include an input device and an output device, these components being interconnected via a bus system and / or other forms of connection mechanisms. Furthermore, the input device may include, for example, a keyboard, a mouse, etc. The output device may output various information to the outside, including determined distance information, direction information, etc. The output device may include, for example, a display, a speaker, a printer, a communication network, and remote output devices connected thereto.
[0097] Of course, for simplicity, only some of the components in the electronic device related to the present disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may further include any other appropriate components according to specific application scenarios.
[0098] In addition to the above methods and devices, the embodiments of the present disclosure may also be a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the processor executes any method provided by the embodiments of the present disclosure.
[0099] The computer program product may be written in any combination of one or more programming languages to implement the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, and conventional procedural programming languages such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0100] In addition, the embodiments of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor is enabled to perform any method provided by the embodiments of the present disclosure.
[0101] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.
[0102] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.
[0103] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein.
Claims
1. A face orientation detection method, characterized in that: The method comprises: Perform key point detection on the face in the image to be tested to obtain the first face key point; Estimating the face pose based on the first and second facial key points to obtain a rotation angle of the face; wherein the second facial key point is a pre-set facial key point with a known rotation angle; Calculating the probability of the face facing each preset orientation based on the rotation angle of the face and the rotation angle threshold corresponding to each preset orientation; wherein the greater the difference between the rotation angle of the face and the rotation angle threshold, the smaller the probability; A face orientation detection result of the face is determined based on the probability that the face is facing each preset orientation.
2. The method according to claim 1, wherein The step of estimating the pose of the face based on the first facial key points and the second facial key points to obtain a rotation angle of the face includes: Iterate the camera parameters using the calibrated camera parameters as initial values; in each iteration, transform the first facial key point from the image coordinate system to the world coordinate system using the current camera parameters, and compare the first facial key point and the second facial key point in the world coordinate system until the error between the first facial key point and the second facial key point in the world coordinate system is minimized; The camera parameters corresponding to the minimum error are converted into the rotation angle of the face.
3. The method according to claim 1, wherein The step of estimating the pose of the face based on the first facial key points and the second facial key points to obtain a rotation angle of the face includes: Iterating the camera parameters using the calibrated camera parameters as initial values; in each iteration, transforming the second facial key point from the world coordinate system to the image coordinate system using the current camera parameters, and comparing the first facial key point and the second facial key point in the image coordinate system until the error between the first facial key point and the second facial key point in the image coordinate system is minimized; The camera parameters corresponding to the minimum error are converted into the rotation angle of the face.
4. The method according to claim 2 or 3, wherein: The camera parameters corresponding to the minimum error represent a rotation vector and a translation vector of the face relative to the camera, and converting the camera parameters corresponding to the minimum error into a rotation angle of the face includes: Convert the rotation vector into a rotation matrix, and concatenate the rotation matrix and the translation vector to obtain a posture matrix; The projection matrix decomposition algorithm is used to perform Euler angle decomposition on the posture matrix to obtain the rotation angle of the face.
5. The method according to claim 1, wherein The rotation angle of the face includes a yaw angle, a pitch angle, and a roll angle. The rotation angle threshold corresponding to each orientation includes a yaw threshold, a pitch threshold, and a roll threshold. The calculation based on the rotation angle of the face and the rotation angle threshold corresponding to each preset orientation to obtain the probability of the face facing each preset orientation includes: For each preset orientation, the yaw angle and the yaw threshold, the pitch angle and the pitch threshold, and the roll angle and the roll threshold are respectively input into a preset probability scoring function for calculation, and a first sub-probability corresponding to the yaw angle, a second sub-probability corresponding to the pitch angle, and a third sub-probability corresponding to the roll angle are output; The probability that the face is facing the preset orientation is determined by adopting a square root average method according to at least one of the first sub-probability, the second sub-probability, and the third sub-probability.
6. The method according to claim 1, wherein The rotation angle threshold includes a facing value and an offset range. The greater the difference between the rotation angle of the face and the rotation angle threshold, the smaller the probability, including: When the rotation angle of the face is equal to the facing value, determining that the probability is maximum; When the rotation angle of the face exceeds the boundary value of the offset range, determining the probability to be zero; When the rotation angle of the face changes from the facing value to the boundary value of the offset range, the probability decreases from the maximum value to zero.
7. The method according to claim 1, wherein The determining of the face orientation detection result of the face according to the probability that the face is facing each preset orientation includes: If the probability that the face is facing each preset direction is less than or equal to a threshold, it is determined that the face is not facing any preset direction; If there is a target probability greater than the threshold among the probabilities that the face is facing each preset direction, the preset direction corresponding to the maximum value among the target probabilities is used as the direction of the face.
8. A face orientation detection device, characterized in that: include: A detection module is used to detect key points of the face in the image to be tested and obtain the first face key points; a generating module, configured to estimate the pose of the face based on the first facial key points and the second facial key points to obtain a rotation angle of the face; wherein the second facial key points are pre-set facial key points with a known rotation angle; a prediction module, configured to calculate, based on the rotation angle of the face and the rotation angle threshold corresponding to each preset orientation, a probability that the face is facing each preset orientation; wherein the greater the difference between the rotation angle of the face and the rotation angle threshold, the smaller the probability; The determination module is used to determine the face orientation detection result of the face according to the probability that the face is facing each preset orientation.
9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the face orientation detection method described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by the processor, the face orientation detection method according to any one of claims 1 to 7 is implemented.
11. A vehicle, characterized in that: include: The apparatus according to claim 8, or the electronic device according to claim 9.