Face pose estimation method and apparatus

Abstract

The present application provides a kind of face posture estimation method and device, method includes: obtaining face image and its corresponding key point information;Based on key point information, the projection distance of nose relative to two eye midpoints is obtained, and the distance between two eyes is obtained;Based on projection distance and the distance between two eyes, using the mapping relationship obtained by fitting based on experience parameter in advance, the face posture is obtained.The present application obtains face image and its corresponding key point information, so as to estimate face posture according to key point information, and utilize the mapping relationship obtained by fitting in advance, to utilize relatively accurate mapping relationship to predict face posture, improve the accuracy and reliability of posture estimation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a face pose estimation method and device. BACKGROUND

[0002] With the continuous development of computer vision, face recognition related algorithms emerge in endlessly, and the face recognition algorithm based on deep learning has reached a very high recognition accuracy in the ideal experimental environment, but in the real scene, different face poses, such as the left and right of the face, the change of the pitch and the in-plane rotation angle, will cause the face information to be missing, thereby making the face recognition effect face a great challenge. Because the face pose change will cause the face information to be lost and different, the similarity of the side face of different people is higher than that between the side face and the front face of the same person, so in order to improve the face recognition effect, the face pose needs to be accurately estimated first.

[0003] At present, face pose estimation generally includes two methods: the first method is to project the face image to each principal component analysis pose space, and the pose of the closest projection coefficient space is taken as the face pose of the image; the second method is to define the geometric structure of the face key points in advance, and directly regress the three angle values of the face pose: pitch, yaw and roll through the key point detection and the mapping relationship from three-dimension (3D) to two-dimension (2D) of the model.

[0004] However, the first method directly depends on the overall pixel information of the image, so the calculation dimension is high, and the pose space is discontinuous, which requires a large number of face image samples of different poses; the second method mainly depends on the key points and the 3D face model, but at present, it is still unable to detect the key points at a large angle, so it has certain limitations, and if the key point prediction is wrong, the pose value estimated by fitting with the 3D deformable face model will have a very large error. SUMMARY

[0005] The present application provides a face pose estimation method and device to solve the defect that the pose estimation precision is poor due to the influence of the shooting angle in the prior art, and effectively improves the precision and efficiency of face pose estimation.

[0006] The present application provides a face pose estimation method, comprising: obtaining a face image and its corresponding key point information; based on the key point information, obtaining the projection distance of the nose relative to the midpoint of the two eyes, and obtaining the distance between the two eyes; based on the projection distance and the distance between the two eyes, using the mapping relationship obtained by fitting the experience parameters in advance, obtaining the face pose.

[0007] According to the face posture estimation method provided by the present application, the projection distance of the nose relative to the midpoint of the two eyes is obtained based on the key point information, which comprises: obtaining the coordinates of the nose and the coordinates of the two eyes based on the key point information; obtaining the midpoint coordinates based on the coordinates of the two eyes; and obtaining the projection distance of the nose relative to the midpoint according to the midpoint coordinates and the nose coordinates.

[0008] According to the face posture estimation method provided by the present application, the projection distance of the nose relative to the midpoint is obtained according to the midpoint coordinates and the nose coordinates, which comprises: obtaining a first line segment according to the midpoint coordinates and the nose coordinates; obtaining a second line segment according to the coordinates of the two eyes; obtaining the included angle between the first line segment and the second line segment according to the first line segment and the second line segment; and projecting the first line segment onto the second line segment based on the included angle to obtain the projection distance.

[0009] According to the face posture estimation method provided by the present application, the face posture is obtained based on the projection distance and the distance between the two eyes by using the mapping relationship obtained by fitting the empirical parameters in advance, which comprises: obtaining the proportion of the nose relative to the two eyes based on the projection distance and the distance between the two eyes; obtaining the angle information according to the proportion by using the mapping relationship obtained by fitting the empirical parameters in advance; and obtaining the face posture according to the angle information.

[0010] According to the face posture estimation method provided by the present application, the proportion is expressed as:

[0011]

[0012] Wherein, r represents the proportion, d p represents the projection distance, and cosm represents the cosine value of the included angle between the first line segment formed by the nose and the midpoint of the two eyes and the second line segment formed by the two eyes; d e is the distance between the two eyes.

[0013] According to the face posture estimation method provided by the present application, before obtaining the face posture, it further comprises: obtaining the empirical parameters in a preset angle range, wherein the empirical parameters comprise the angle empirical parameters corresponding to the historical face images and the key point empirical parameters corresponding to the historical face images; obtaining the proportion empirical parameters of the nose relative to the two eyes by using the key point empirical parameters; and obtaining the mapping relationship based on the proportion empirical parameters and the corresponding angle empirical parameters.

[0014] The application further provides a face posture estimation device, comprising: a data acquisition module, which acquires a face image and corresponding key point information thereof; a parameter acquisition module, which obtains a projection distance of a nose relative to the midpoint of two eyes and an interocular distance based on the key point information; and a posture estimation module, which obtains a face posture by using a mapping relationship obtained by fitting an empirical parameter in advance based on the projection distance and the interocular distance.

[0015] The application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the face posture estimation method according to any one of the above when executing the program.

[0016] The application further provides a non-transitory computer readable storage medium, which stores a computer program, wherein the computer program implements the steps of the face posture estimation method according to any one of the above when executed by a processor.

[0017] The application further provides a computer program product, comprising a computer program, wherein the computer program implements the steps of the face posture estimation method according to any one of the above when executed by a processor.

[0018] The face posture estimation method and device provided by the application obtain a face image and corresponding key point information thereof, so as to estimate a face posture according to the key point information and by using a mapping relationship obtained by fitting in advance, to correspondingly predict the face posture by using a more accurate mapping relationship, and to improve the accuracy and reliability of posture estimation. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the present application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0020] Figure 1 is a flowchart of the face posture estimation method provided by the application;

[0021] Figure 2 is a flowchart of the method for fitting a mapping relationship based on an empirical parameter provided by the application;

[0022] Figure 3 is a structural diagram of the face posture estimation device provided by the application;

[0023] Figure 4 is a structural diagram of the mapping relationship acquisition module provided by the application;

[0024] Figure 5 is a structural schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0025] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0026] Figure 1 A flowchart of a face posture estimation method according to the present application is shown. The method comprises the following steps:

[0027] S11, obtaining a face image and corresponding key point information thereof;

[0028] S12, obtaining a projection distance of a nose relative to a midpoint of two eyes and an interocular distance based on the key point information;

[0029] S13, obtaining a face posture based on the projection distance and the interocular distance by using a mapping relationship obtained by pre-fitting based on experience parameters.

[0030] It should be noted that S1N in the present specification does not represent the order of the face posture estimation method, and the face posture estimation method of the present application will be described in detail below.

[0031] Step S11, obtaining a face image and corresponding key point information thereof.

[0032] In the present embodiment, obtaining a face image and corresponding key point information thereof comprises: obtaining a face image; inputting the face image into a face key point detection model to obtain key point information output by the face key point detection model, wherein the face key point detection model is trained based on a face sample image and corresponding face key point true value. It should be noted that the key point information comprises coordinate information of each face key point.

[0033] In an optional embodiment, obtaining a face image and corresponding key point information thereof comprises: obtaining a face image; performing key point labeling on the face image to obtain key point information. It should be noted that in order to improve the accuracy of face posture estimation, thereby facilitating to provide a large number of training samples for the identification of fatigue driving behavior, the obtained face image can be pre-detected to obtain key point information.

[0034] In an optional embodiment, the face image is obtained by: based on an electronic device or an application platform applying the face key point detection method, obtaining a face image to be detected by the face key point detection method; or based on a terminal device connected to the electronic device or the application platform applying the face key point detection method, obtaining the face image. It should be noted that the terminal device can obtain the face image of the person in the recognition area through the visual sensor connected thereto. It should be noted that the face image can be a single frame of picture or a sequence of picture frames obtained by shooting, or an image frame or a sequence of image frames related to the face to be detected obtained by lens segmentation of a video.

[0035] In addition, the visual sensor includes at least one of a millimeter wave radar, a laser radar, a detector, a camera, and other camera devices, and the specific type of the visual sensor is not limited further herein.

[0036] In step S12, the projection distance of the nose relative to the midpoint of the two eyes is obtained based on the key point information, and the distance between the two eyes is obtained.

[0037] Specifically, the projection distance of the nose relative to the midpoint of the two eyes is obtained based on the key point information, including: obtaining the nose coordinates and the two eye coordinates based on the key point information; obtaining the midpoint coordinates based on the two eye coordinates; and obtaining the projection distance of the nose relative to the midpoint according to the midpoint coordinates and the nose coordinates.

[0038] First, the nose coordinates and the two eye coordinates are obtained based on the key point information. It should be noted that the obtained key point information includes face key point coordinate information, which includes nose coordinates (x1, y1, z1) and corresponding coordinates of the two eyes (x2, y2, z2) and (x3, y3, z3).

[0039] Second, the midpoint coordinates are obtained based on the two eye coordinates. It should be noted that the midpoint is the midpoint of the line segment connecting the two eyes, and the midpoint coordinates can be obtained according to the two eye coordinates. For example, the two eye coordinates are (x2, y2, z2) and (x3, y3, z3), and the midpoint coordinates are

[0040] Finally, the projection distance of the nose relative to the midpoint is obtained according to the midpoint coordinates and the nose coordinates. In this embodiment, the projection distance of the nose relative to the midpoint is obtained according to the midpoint coordinates and the nose coordinates, including: obtaining a first line segment according to the midpoint coordinates and the nose coordinates; obtaining a second line segment according to the two eye coordinates; obtaining an included angle between the first line segment and the second line segment according to the first line segment and the second line segment; and projecting the first line segment onto the second line segment based on the included angle to obtain the projection distance.

[0041] For example, the midpoint coordinates are The nose coordinate is (x1, y1, z1), and the first line segment L1 and the length of the first line segment can be obtained. The two eye coordinates are (x2, y2, z2) and (x3, y3, z3), and the second line segment L2 can be obtained. The included angle m between the first line segment L1 and the second line segment L2 is obtained according to the first line segment L1 and the second line segment L2. The first line segment is projected to the second line segment based on the included angle m, and the projection distance d is obtained according to the included angle m and the length of the first line segment p .

[0042] In step S13, the face pose is obtained based on the projection distance and the interocular distance by using a mapping relationship obtained by fitting an empirical parameter in advance.

[0043] In this embodiment, the face pose is obtained based on the projection distance and the interocular distance by using a mapping relationship obtained by fitting an empirical parameter in advance, including: obtaining the proportion of the nose relative to the eyes based on the projection distance and the interocular distance; obtaining angle information by using a mapping relationship obtained by fitting an empirical parameter in advance according to the proportion; and obtaining the face pose according to the angle information.

[0044] It should be noted that the proportion is represented as:

[0045]

[0046] wherein r represents the proportion, d p represents the projection distance, and cosm represents the cosine value of the included angle between the first line segment formed by the nose and the midpoint of the two eyes and the second line segment formed by the two eyes; d e is the interocular distance.

[0047] In this embodiment, the angle information includes at least one of a pitch angle (Pitch), a yaw angle (Yaw), and a roll angle (Roll). In actual application, the face pose type needs to be divided according to the size of the angle information in advance, so that the angle information within a certain angle range corresponds to a specific face pose type. For example, the face pose type is divided into six categories according to the angle value of left and right rotation, and each face pose type includes angle values of [-90, -60), [-60, -30), [-30, 0), [0, 30), [30, 60), and [60, 90]. For another example, the face pose type can also be divided according to the angle value of the pitch rotation or the in-plane rotation of the face pose, and the face pose type includes at least two categories. For another example, a specific face pose corresponding to a certain angle range can also be set based on the specific angle included in the angle information.

[0048] In an optional embodiment, the reference Figure 2 Before obtaining the face pose, the mapping relationship obtained by fitting the empirical parameter further includes:

[0049] S21, obtaining experience parameters in a preset angle range, the experience parameters including angle experience parameters corresponding to historical face images and key point experience parameters corresponding to the historical face images;

[0050] S22, obtaining a proportion experience parameter of a nose relative to eyes by using the key point experience parameters;

[0051] S23, obtaining a mapping relationship based on the proportion experience parameter and corresponding angle experience parameters.

[0052] It should be noted that S2N in the present specification does not represent the order of the mapping relationship based on the experience parameters, and the process of the mapping relationship based on the experience parameters of the present application will be described in detail below.

[0053] Step S21, obtaining experience parameters in a preset angle range, the experience parameters including angle experience parameters corresponding to historical face images and key point experience parameters corresponding to the historical face images.

[0054] It should be noted that obtaining experience parameters in a preset angle range includes: obtaining historical face images in a preset angle range; and labeling the historical face images to obtain the experience parameters. It should be noted that the historical face images can be obtained based on different attitude angles, occlusions, light, and other external factors.

[0055] In an optional embodiment, the preset angle range includes a yaw angle range, and the yaw angle range is selected as [-30°, 30°], so as to accurately estimate the mapping relationship between the yaw angle and the face key points based on the small angle range, and then to accurately estimate the face posture in a large angle range [-90°, -30°) U (30°, 90°] based on the key point information, thereby improving the reliability of face posture estimation.

[0056] Step S22, obtaining a proportion experience parameter of a nose relative to eyes by using the key point experience parameters;

[0057] In the embodiment, the proportion empirical parameter of the nose relative to the two eyes is obtained by using the key point empirical parameter, including: obtaining nose reference coordinates and two eye reference coordinates based on the key point empirical parameter; obtaining midpoint reference coordinates, a third line segment formed by the two eyes and a length of the third line segment according to the two eye reference coordinates; obtaining a fourth line segment and a length of the fourth line segment according to the midpoint reference coordinates and the nose reference coordinates; obtaining an included angle empirical parameter between the third line segment and the fourth line segment according to the third line segment and the fourth line segment; projecting the fourth line segment to the third line segment according to the included angle empirical parameter between the third line segment and the fourth line segment, and obtaining a projection distance parameter of the fourth line segment projected to the third line segment according to the included angle empirical parameter between the third line segment and the fourth line segment and the length of the fourth line segment; and obtaining the proportion empirical parameter of the nose relative to the two eyes according to the projection distance parameter and the length of the third line segment.

[0058] In the embodiment, the mapping relationship is obtained based on the proportion empirical parameter and the corresponding angle empirical parameter.

[0059] In the embodiment, the mapping relationship is obtained based on the proportion empirical parameter and the corresponding angle empirical parameter.

[0060] In summary, the embodiment of the present application obtains a face image and corresponding key point information thereof, so as to estimate a face posture according to the key point information and by using a pre-fitted mapping relationship, to predict a face posture by using a relatively accurate mapping relationship, and to improve the accuracy and reliability of posture estimation.

[0061] The face posture estimation device provided by the present application is described below, and the face posture estimation device described below can be correspondingly referred to the face posture estimation method described above.

[0062] Figure 3 The structure of a face posture estimation device according to the present application is shown in the structural schematic diagram of the device, which comprises:

[0063] The data acquisition module 31 acquires a face image and corresponding key point information thereof;

[0064] The parameter acquisition module 32 obtains a projection distance of a nose relative to a midpoint of two eyes and an interocular distance based on the key point information.

[0065] The pose estimation module 33 obtains the face pose based on the projection distance and the interocular distance, and a mapping relationship obtained in advance based on experience parameters.

[0066] In this embodiment, the data acquisition module 31 includes an image acquisition unit configured to acquire the face image, and a key point detection unit configured to input the face image into a face key point detection model to obtain key point information output by the face key point detection model, wherein the face key point detection model is trained based on face sample images and corresponding face key point true values. It should be noted that the key point information includes coordinate information of each face key point.

[0067] In an optional embodiment, the data acquisition module 31 includes an image acquisition unit configured to acquire the face image, and a labeling unit configured to label the face image with key points to obtain the key point information. It should be noted that, in order to improve the accuracy of face pose estimation and thus provide a large number of training samples for the identification of fatigue driving behavior, the acquired face image can be pre-detected to obtain the key point information.

[0068] In an optional embodiment, the image acquisition unit configured to acquire the face image includes an electronic device or application platform applying a face key point detection method, and configured to obtain a face image to be detected for face key points; or a terminal device connected to the electronic device or application platform applying the face key point detection method, and configured to obtain the face image. It should be noted that the terminal device can obtain the face image of a person in a recognition area through a visual sensor connected thereto. It should be noted that the face image can be a single frame of picture or a sequence of picture frames, or an image frame or a sequence of image frames related to the face to be detected obtained by lens segmentation of a video.

[0069] In addition, the visual sensor includes at least one of a millimeter wave radar, a laser radar, a detector, a camera, and other camera devices, and the specific type of the visual sensor is not limited further herein.

[0070] The parameter acquisition module 32 includes a key point acquisition unit configured to obtain nose coordinates and eye coordinates based on the key point information, a midpoint acquisition unit configured to obtain midpoint coordinates based on the eye coordinates, and a projection distance acquisition unit configured to obtain a projection distance of the nose relative to the midpoint based on the midpoint coordinates and the nose coordinates.

[0071] Specifically, the projection distance acquisition unit includes a first line segment acquisition subunit configured to obtain a first line segment based on the midpoint coordinates and the nose coordinates, a second line segment acquisition subunit configured to obtain a second line segment based on the eye coordinates, an intersection angle acquisition subunit configured to obtain an intersection angle between the first line segment and the second line segment based on the first line segment and the second line segment, and a projection subunit configured to project the first line segment onto the second line segment based on the intersection angle to obtain the projection distance.

[0072] The pose estimation module 33 comprises: a proportion obtaining unit, configured to obtain a proportion of the nose relative to the eyes based on the projection distance and the interocular distance; an angle obtaining unit, configured to obtain angle information according to the proportion by using a mapping relationship obtained by fitting experience parameters in advance; and a pose estimation unit, configured to obtain the face pose according to the angle information.

[0073] It should be noted that the angle information comprises at least one of a pitch angle, a yaw angle and a roll angle. In actual application, the face pose type is divided according to the size of the angle information in advance, so that the angle information within a certain angle range corresponds to a specific face pose type. For example, the face pose type is divided into six categories according to the angle value of left and right rotation, and each face pose type contains angle values of [-90, -60), [-60, -30), [-30, 0), [0, 30), [30, 60) and [60, 90] respectively. For another example, the face pose type can also be divided according to the angle value of the pitch rotation or the in-plane rotation of the face pose, and the face pose type comprises at least two categories. For another example, a specific face pose corresponding to a certain angle range can also be set based on the specific angle contained in the angle information.

[0074] In an optional embodiment, the reference Figure 4 The device further comprises a mapping relationship obtaining module based on a mapping relationship obtained by fitting experience parameters. Specifically, the mapping relationship obtaining module comprises:

[0075] The data obtaining unit 41 obtains experience parameters in a preset angle range, and the experience parameters comprise angle experience parameters corresponding to historical face images and key point experience parameters corresponding to the historical face images.

[0076] The intermediate parameter obtaining unit 42 obtains proportion experience parameters of the nose relative to the eyes by using the key point experience parameters.

[0077] The mapping relationship obtaining unit 43 obtains a mapping relationship based on the proportion experience parameters and corresponding angle experience parameters.

[0078] In this embodiment, the data obtaining unit 41 comprises: an image obtaining subunit, configured to obtain historical face images in a preset angle range; and a labeling subunit, configured to label the historical face images to obtain the experience parameters. It should be noted that the historical face images can be obtained based on different pose angles, occlusions, illuminations and other external factors when the image obtaining subunit obtains the historical face images.

[0079] In an optional embodiment, the preset angle range includes a yaw angle range, and the yaw angle range is selected as [-30°, 30°], so as to subsequently accurately estimate the mapping relationship between the yaw angle and the face key points according to the small angle range, and then facilitate subsequent accurate estimation of the face pose in a large angle range [-90°, -30°) ∪ (30°, 90°] according to the key point information, thereby improving the reliability of face pose estimation.

[0080] The intermediate parameter acquisition unit 42 includes: a coordinate acquisition subunit that obtains a nose reference coordinate and two eye reference coordinates based on the key point experience parameter; a first intermediate parameter acquisition subunit that obtains a midpoint reference coordinate, a third line segment formed by the two eyes, and a length of the third line segment according to the two eye reference coordinates; a second intermediate parameter acquisition subunit that obtains a fourth line segment and a length of the fourth line segment according to the midpoint reference coordinate and the nose reference coordinate; an angle experience parameter between the third line segment and the fourth line segment is obtained according to the third line segment and the fourth line segment; a projection distance parameter of the projection of the fourth line segment to the third line segment is obtained according to the angle experience parameter between the third line segment and the fourth line segment and the length of the fourth line segment; and a proportion experience parameter of the nose relative to the two eyes is obtained according to the projection distance parameter and the length of the third line segment.

[0081] The mapping relationship acquisition unit 43 includes: a function construction subunit that constructs a function based on the proportion experience parameter and the corresponding angle experience parameter; and a mapping relationship is obtained based on the function. It should be noted that the function is y = f(x), where y represents the angle experience parameter, f represents the mapping relationship, and x represents the proportion experience parameter. In addition, the experience parameters obtained in the preset angle range are used to fit a more accurate mapping relationship, thereby facilitating subsequent estimation of the face pose in a large angle range according to the mapping relationship, and thereby improving the accuracy of the face pose estimation in the large angle range.

[0082] Figure 5 An example of an electronic device is shown in the physical structure diagram, as shown in the figure, the electronic device can include: processor 51, communications interface (Communications Interface) 52, memory 53 and communication bus 54, wherein the processor 51, communications interface 52, memory 53 through the communication bus 54 complete mutual communication. The processor 51 can call the logic instructions in the memory 53 to execute the face pose estimation method, which includes: obtaining a face image and its corresponding key point information; based on the key point information, obtaining the projection distance of the nose relative to the midpoint of the two eyes, and obtaining the distance between the two eyes; based on the projection distance and the distance between the two eyes, a mapping relationship is obtained by using a mapping relationship fitted in advance based on experience parameters. Figure 5 The processor 51 can call the logic instructions in the memory 53 to execute the face pose estimation method, which includes: obtaining a face image and its corresponding key point information; based on the key point information, obtaining the projection distance of the nose relative to the midpoint of the two eyes, and obtaining the distance between the two eyes; based on the projection distance and the distance between the two eyes, a mapping relationship is obtained by using a mapping relationship fitted in advance based on experience parameters.

[0083] Moreover, the logic instructions in the memory 53 described above can be realized in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0084] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program is executed by a processor, so that the computer can execute the face pose estimation method provided by the above-mentioned method. The method comprises: obtaining a face image and its corresponding key point information; based on the key point information, obtaining the projection distance of the nose relative to the midpoint of the two eyes, and obtaining the distance between the two eyes; based on the projection distance and the distance between the two eyes, using a mapping relationship obtained by fitting based on experience parameters in advance, obtaining the face pose.

[0085] In another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the face pose estimation method provided by the above-mentioned method. The method comprises: obtaining a face image and its corresponding key point information; based on the key point information, obtaining the projection distance of the nose relative to the midpoint of the two eyes, and obtaining the distance between the two eyes; based on the projection distance and the distance between the two eyes, using a mapping relationship obtained by fitting based on experience parameters in advance, obtaining the face pose.

[0086] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0087] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0088] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A face pose estimation method, characterized in that, include: Obtain facial images and their corresponding key point information; Based on the key point information, the projected distance of the nose relative to the midpoint of the two eyes is obtained, as well as the distance between the two eyes; Based on the projection distance and the interocular distance, the face pose is obtained using a mapping relationship that has been fitted in advance based on empirical parameters. The step of obtaining the projected distance of the nose relative to the midpoints of the two eyes based on the key point information includes: Based on the key point information, the coordinates of the nose and the coordinates of both eyes are obtained; Based on the coordinates of the two eyes, the coordinates of the midpoint are obtained; Based on the midpoint coordinates and the nose coordinates, the projected distance of the nose relative to the midpoint is obtained; The step of obtaining the projected distance of the nose relative to the midpoint based on the midpoint coordinates and the nose coordinates includes: The first line segment is obtained based on the coordinates of the midpoint and the coordinates of the nose; Based on the coordinates of the two eyes, the second line segment is obtained; Based on the first line segment and the second line segment, the included angle between the first line segment and the second line segment is obtained; Based on the included angle, the first line segment is projected onto the second line segment to obtain the projection distance; The process of obtaining the face pose based on the projection distance and the interocular distance using a pre-fitted mapping relationship based on empirical parameters includes: Based on the projection distance and the interocular distance, the relative weight of the nose to the eyes is obtained; Based on the stated weight, angle information is obtained using a mapping relationship pre-fitted based on empirical parameters; The facial pose is obtained based on the angle information.

2. The face pose estimation method according to claim 1, characterized in that, The specific gravity is expressed as: Where r represents specific gravity, d p d represents the projected distance; cosm represents the cosine of the angle between the first line segment formed by the nose and the midpoints of the two eyes and the second line segment formed by the two eyes; e The distance between the eyes.

3. The face pose estimation method according to claim 1, characterized in that, Before obtaining the face pose, the process also includes: Obtain empirical parameters within a preset angle range, including angle empirical parameters corresponding to historical face images and key point empirical parameters corresponding to historical face images; Using the aforementioned key point empirical parameters, the relative weight of the nose to the eyes is obtained as empirical parameters. Based on the specific gravity empirical parameter and its corresponding angle empirical parameter, a mapping relationship is obtained.

4. A face pose estimation device, characterized in that, include: The data acquisition module acquires facial images and their corresponding key point information; The parameter acquisition module obtains the projection distance of the nose relative to the midpoint of the two eyes, and the distance between the two eyes, based on the key point information; The pose prediction module obtains the face pose based on the projection distance and the interocular distance using a mapping relationship pre-fitted based on empirical parameters. The parameter acquisition module includes: The key point acquisition unit obtains the nose coordinates and the coordinates of both eyes based on the key point information; The midpoint acquisition unit obtains the midpoint coordinates based on the coordinates of the two eyes; The projection distance acquisition unit obtains the projection distance of the nose relative to the midpoint based on the midpoint coordinates and the nose coordinates; The projection distance acquisition unit includes: The first line segment is obtained by acquiring the sub-unit based on the midpoint coordinates and the nose coordinates; The second line segment is obtained by acquiring the sub-unit based on the coordinates of the two eyes; The angle acquisition sub-unit obtains the included angle between the first line segment and the second line segment based on the first line segment and the second line segment; The projection subunit projects the first line segment onto the second line segment based on the included angle to obtain the projection distance; The attitude prediction module includes: The specific gravity acquisition unit obtains the specific gravity of the nose relative to the eyes based on the projection distance and the interocular distance; Angle acquisition unit obtains angle information based on the weight using a mapping relationship pre-fitted based on empirical parameters; The pose estimation unit obtains the face pose based on the angle information.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the face pose estimation method as described in any one of claims 1 to 3.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the face pose estimation method as described in any one of claims 1 to 3.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the face pose estimation method as described in any one of claims 1 to 3.

Images