A method and device for determining face deflection angle

By determining the characteristic points of the left eye, right eye and nose in the face image, and building a triangle to calculate the deflection angle, the problem of large error in determining the deflection angle of the face in the prior art is solved, and a high accuracy and simple process is achieved .

CN113011230BActive Publication Date: 2024-12-03HANGZHOU EZVIZ SOFTWARE CO LTD
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN201911326091.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-12-20
Publication Date
2024-12-03
Estimated Expiration
2039-12-20

AI Technical Summary

Technical Problem

The prior art has large errors when determining the face deflection angle, requiring multiple feature points, and complex parameter settings, which affect the accuracy of the result.

Method used

By obtaining the target face image, determine the position information of the feature points of the left eye, right eye and nose, use these feature points to build an isosceles triangle or right triangle, calculate the angle of the triangle to determine the horizontal and vertical deflection information of the face, only Three feature points are required to achieve it.

Benefits of technology

It reduces the difficulty of feature point acquisition, improves the accuracy of face deflection information, reduces calculation errors and calculation amounts, and simplifies the implementation process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113011230B_ABST
    Figure CN113011230B_ABST
Patent Text Reader

Abstract

One or more embodiments of this specification disclose a method and device for determining the deflection angle of a face. The method includes: obtaining a target face image; determining target feature points in the target face image, and determining first position information of each target feature point; the target feature points include left eye feature points, right eye feature points and nose feature points; the left eye feature points and the right eye feature points are located on the same horizontal plane; according to the first position information, determining the face deflection information in the target face image; the face deflection information includes horizontal deflection information and / or vertical deflection information. This technical solution can determine the face deflection information through only three feature points, which reduces the difficulty of collecting feature points and makes the implementation process simple and convenient; and, since the collected target feature points are all feature points on the face, the face deflection information finally determined conforms to the objective situation and has a high accuracy rate.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of face recognition technology, and in particular to a method and device for determining a face deflection angle. Background Art

[0002] In the prior art, when determining the face deflection angle, the face deflection angle is usually fitted into a linear model, and 5 feature points are required to determine the angle. This method of determining the face deflection angle requires many feature points, which places high demands on the front-end equipment and increases the difficulty of implementation; in addition, the face deflection angle is a standard triangular relationship model, and the error of fitting it into a linear model is very large; at the same time, many parameters need to be pre-set during calculation, and these parameters are strongly correlated with the final result, resulting in a great impact on the final result. Therefore, providing a more reliable method for determining the face deflection angle has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The purpose of one or more embodiments of the present specification is to provide a method and device for determining a face deflection angle, so as to overcome the problem of large errors in determining a face deflection angle in the prior art.

[0004] To solve the above technical problems, one or more embodiments of this specification are implemented as follows:

[0005] On the one hand, one or more embodiments of this specification provide a method for determining a face deflection angle, including:

[0006] Get the target face image;

[0007] Determine target feature points in the target face image, and determine first position information of each target feature point; the target feature points include left eye feature points, right eye feature points and nose feature points; the left eye feature points and the right eye feature points are located on the same horizontal plane;

[0008] According to the first position information, face deflection information in the target face image is determined; the face deflection information includes horizontal deflection information and / or vertical deflection information.

[0009] On the other hand, one or more embodiments of the present specification provide a device for determining a face deflection angle, including:

[0010] An acquisition module is used to acquire a target face image;

[0011] A first determination module is used to determine target feature points in the target face image, and to determine first position information of each of the target feature points; the target feature points include left eye feature points, right eye feature points and nose feature points; the left eye feature points and the right eye feature points are located on the same horizontal plane;

[0012] The second determination module is used to determine the face deflection information in the target face image according to the first position information; the face deflection information includes horizontal deflection information and / or vertical deflection information.

[0013] On another aspect, one or more embodiments of the present specification provide a device for determining a face deflection angle, comprising: a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory communicate with each other via the bus; the memory is used to store computer programs; and the processor is used to execute the programs stored in the memory to implement the steps of a method for determining a face deflection angle.

[0014] On the other hand, an embodiment of the present application provides a storage medium for storing computer-executable instructions, which, when executed, implement the steps of a method for determining a face deflection angle.

[0015] The technical solution of one or more embodiments of the present specification is adopted, by determining the target feature points in the target face image and the first position information of each target feature point, and then determining the face deflection information in the target face image according to the first position information of each target feature point; the face deflection information includes horizontal deflection information and / or vertical deflection information. Since the target feature points include three feature points, namely, the left eye feature point, the right eye feature point and the nose feature point, the technical solution can determine the face deflection information only through three feature points, thereby reducing the difficulty of collecting feature points and making the implementation process simple and convenient; and since the collected target feature points are all feature points on the face, the face deflection information finally determined conforms to the objective situation and has a high accuracy rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in one or more embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0017] Figure 1 This is a schematic flowchart of a method for determining a face deflection angle according to an embodiment of the present specification.

[0018] Figure 2 It is a schematic diagram of a horizontal face deflection situation of an embodiment of this specification.

[0019] Figure 3It is a schematic diagram of a face deflection situation under a vertical viewing angle according to an embodiment of this specification.

[0020] Figure 4 It is a schematic diagram of a vertical face deflection situation of an embodiment of this specification.

[0021] Figure 5 It is a schematic diagram of a face deflection situation at a right viewing angle according to an embodiment of this specification.

[0022] Figure 6 It is a structural schematic diagram of a device for determining a face deflection angle according to an embodiment of the present specification.

[0023] Figure 7 It is a structural schematic diagram of a device for determining a face deflection angle according to an embodiment of the present specification. DETAILED DESCRIPTION

[0024] One or more embodiments of the present specification provide a method and device for determining a face deflection angle.

[0025] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of one or more embodiments of this specification.

[0026] Figure 1 is a schematic flow chart of a method for determining a face deflection angle according to an embodiment of the present specification, such as Figure 1 As shown, the method includes:

[0027] Step 102, obtaining a target face image.

[0028] Herein, a single-frame image containing a face captured from a dynamic video may be used as the target face image, or a single-frame face image input by a user may be used as the target face image.

[0029] Step 104, determining target feature points in the target face image, and determining first position information of each target feature point.

[0030] The target feature points include left eye feature points, right eye feature points and nose feature points; the left eye feature points and the right eye feature points are located on the same horizontal plane.

[0031] In this step, if the left eye feature points and the right eye feature points are not located on the same horizontal plane, the left eye feature points and the right eye feature points need to be corrected in advance so that the corrected left eye feature points and the right eye feature points are located on the same horizontal plane.

[0032] The target feature points in the target face image can be determined by face detection algorithm or manual calibration. For example, after obtaining the target face image, the face detection algorithm - CascadeCNN can be used to process the target face image (such as removing the window) to make the remaining detection window closer to the potential face image, and finally lock the output detection window at the target feature point. At present, the face detection algorithm is very popular and will not be described here.

[0033] Step 106: determine face deflection information in the target face image based on the first position information; the face deflection information includes horizontal deflection information and / or vertical deflection information.

[0034] In this embodiment, the target feature points in the target face image and the first position information of each target feature point can be determined, and then the face deflection information in the target face image can be determined according to the first position information of each target feature point; the face deflection information includes horizontal deflection information and / or vertical deflection information. Since the target feature points include three feature points, namely, the left eye feature point, the right eye feature point and the nose feature point, the technical solution can determine the face deflection information only through three feature points, thereby reducing the difficulty of collecting feature points and making the implementation process simple and convenient; and since the collected target feature points are all feature points on the face, the face deflection information finally determined conforms to the objective situation and has a high accuracy rate.

[0035] In one embodiment, before determining the first position information of each target feature point, the second position information of each target feature point may be first collected; secondly, the second position information of each target feature point may be corrected; and then, based on the result of the correction, the first position information of each target feature point may be determined.

[0036] The second position information is the initial position information of the target feature point without correction. The correction process includes adjusting the left eye feature point and the right eye feature point to the same horizontal plane. The face correction algorithm can be used to perform an affine transformation on the image according to the second position information of the target feature point, so that the image is rotated so that the left eye feature point and the right eye feature point are located on the same horizontal plane.

[0037] In this embodiment, the first position information of each target feature point is determined by correcting the second position information of each target feature point, and then the face deflection information is determined based on the first position information, thereby avoiding the error in the face deflection information caused by the deviation of the position information of the target feature point and improving the reliability of the scheme.

[0038] In one embodiment, the horizontal deflection information includes a horizontal deflection angle. When determining the face deflection information in the target face image according to the first position information, the following method can be used:

[0039] First, under a vertical viewing angle, an isosceles triangle formed by the left eye feature point, the right eye feature point and the nose feature point is constructed; wherein the left eye feature point, the right eye feature point and the nose feature point are the vertices of the isosceles triangle respectively.

[0040] At the same time, under the vertical viewing angle, the first reference coordinate axis when the face is not deflected in the target face image is constructed with the nose feature point as the coordinate origin; the horizontal axis of the first reference coordinate axis is parallel to the line connecting the two eyes when the face is not deflected.

[0041] Thereby, a first deflection angle of the isosceles triangle relative to the first reference coordinate axis is determined as a horizontal deflection angle.

[0042] The vertical viewing angle is a perspective that simulates looking down from the top of the head, and the effect of the Y-axis coordinate on the horizontal deflection angle is not taken into account in this perspective.

[0043] In this embodiment, the construction order of the isosceles triangle and the first reference coordinate axis is not limited. In addition to the method of constructing the first reference coordinate axis while constructing the isosceles triangle listed in this embodiment, the isosceles triangle can be constructed first and then the first reference coordinate axis, or the first reference coordinate axis can be constructed first and then the isosceles triangle. Any construction order will not affect the final result of this embodiment.

[0044] In one embodiment, when determining the first deflection angle of an isosceles triangle relative to a first reference coordinate axis, the midline of the base of the isosceles triangle can be determined; thereby, based on the projection lengths of the vertex angle and each waist of the isosceles triangle on the horizontal axis of the first reference coordinate axis, the angle between the midline and the vertical axis of the first reference coordinate axis is calculated as the first deflection angle.

[0045] For example, in the normal viewing angle, the deflection of the face in the horizontal direction is as follows: Figure 2 As shown, feature point A is the feature point of the left eye, feature point B is the feature point of the right eye, and feature point C is the feature point of the nose. From the figure, we can know the X-axis and Y-axis coordinates of the three feature points: feature point A (Xa, Ya), feature point B (Xb, Yb), and feature point C (Xc, Yc).

[0046] Under the vertical viewing angle, determine the horizontal deflection angle when the face is deflected as follows Figure 3As shown. The coordinates of each feature point on the XCZ coordinate axis are as follows: the coordinates of feature point A are (Xa, Za), the coordinates of feature point B are (Xb, Zb), the coordinates of feature point C are (Xc, Zc), the vertical point of feature point A on the X-axis is vertical point D (Xd, Zd), the vertical point of feature point B on the X-axis is vertical point E (Xe, Ze), angle θ is the angle between AC and the X-axis, angle α is the angle between CA and CB, angle γ is the angle between BC and the X-axis, distance d1 is the length of DC, distance d2 is the length of CE, and angle β is the angle between the midline of the base AB and the Z-axis.

[0047] First, an isosceles triangle formed by feature points A, B, and C is constructed; at the same time, under a vertical viewing angle, with feature point C as the coordinate origin, the first reference coordinate axis XCZ is constructed when the face in the target face image is not deflected; wherein the horizontal axis (X axis) of the first reference coordinate axis is parallel to the line connecting the eyes when the face is not deflected. When the isosceles triangle ABC is not deflected, the midline of its base AB is on the Z axis. When the isosceles triangle ABC is deflected, the midline of the base AB deviates from the Z axis. From the figure, it can be determined that the first deflection angle β of the isosceles triangle ABC relative to the first reference coordinate axis XCZ (i.e., the angle between the midline of the base AB and the Z axis) is the horizontal deflection angle.

[0048] In this embodiment, Figure 3 As shown, the angle β between the midline of the base of the isosceles triangle ABC and the longitudinal axis (Z axis) of the first reference coordinate axis XCZ can be determined as the first deflection angle. When determining the deflection angle β, trigonometric functions can be used for calculation.

[0049] Among them, we can first determine the vertex angle of the isosceles triangle ABC and the projection length of each waist on the X-axis of the first reference coordinate axis XCZ, that is, determine Figure 3 The vertical point D, vertical point E, distance d1, and distance d2 in the figure are given. Since the X-axis coordinates of feature points A, B, and C are known, it can be determined that the horizontal coordinate of vertical point D is equal to that of feature point A, that is, Xd=Xa, and the horizontal coordinate of vertical point E is equal to that of feature point B, that is, Xe=Xb. Therefore, it can be determined that the distance d1=abs(Xc-Xd) and d2=abs(Xe-Xc), where abs means taking the absolute value.

[0050] Secondly, the isosceles triangle ABC can be constructed with the two sides being equal to each other to determine the equation relationship: AC = BC. Based on the distance d1 and distance d2 determined in the previous step, and using trigonometric functions, it can be determined that d1 / cosθ = AC, d2 / cosγ = BC. Therefore, d1 / cosθ = d2 / cosγ.

[0051] Then, from Figure 3 The angle can be obtained from Angle θ=π-α-γ. Substituting angle γ and angle θ into d1 / cosθ=d2 / cosγ, we can get:

[0052]

[0053] From (1), we can determine the variables as α and β, and from Figure 3 It can be seen that α is the angle between AC and BC, which represents the angle from the nose to the eyes. After a large sample test, it can be estimated that the range of α is 100°~200°.

[0054] Assuming α = 110°, substituting the value of α into (1), we can obtain:

[0055]

[0056] At this time, there is only a single variable β in (2), that is, only the first deflection angle β is unknown. By transforming the formula, the first deflection angle can be obtained

[0057] In one embodiment, the horizontal deflection information also includes the horizontal deflection direction; using the above example, Figure 3 When establishing the coordinate axis and the isosceles triangle in the manner shown, if the first deflection angle is greater than zero, the horizontal deflection direction is determined to be deflected to the left; if the first deflection angle is less than zero, the horizontal deflection direction is determined to be deflected to the right; if the first deflection angle is equal to zero, it is determined that there is no deflection in the horizontal direction.

[0058] In the above embodiment, when determining the horizontal deflection information, the deflection angle of the target face image in the horizontal direction can be determined by only three feature points on the target image, and then using trigonometric functions and four arithmetic operations. Since fewer feature points are required, the implementation of this solution is simple, and the use of trigonometric functions for calculation is more in line with the actual face deflection situation, which can reduce calculation errors and reduce the amount of calculation. In addition, only a large sample test data is used in the calculation process, which reduces the deviation caused to the final result.

[0059] In one embodiment, the vertical deflection information includes a vertical deflection angle; when determining the face deflection information in the target face image according to the first position information, the following method may be used:

[0060] First, from a side angle of view, with the first vertical point of the horizontal plane from the eye feature point to the nose initial point as the coordinate origin and the straight line connecting the nose initial point and the first vertical point as the horizontal axis, a second reference coordinate axis when the face is not deflected in the target face image is constructed; wherein the nose initial point is the position of the nose when the face is not deflected, and the eye feature point is the left eye feature point or the right eye feature point.

[0061] In this embodiment, the side viewing angle is a viewing angle that simulates observation from the right or left side of the head, and the influence of the X-axis coordinate on the vertical deflection angle is not taken into account in this viewing angle. Since the second reference coordinate axis is constructed under the side viewing angle, and the left eye feature points and the right eye feature points are coincident under the side viewing angle, the second reference coordinate axis determined by the left eye feature points or the right eye feature points is the same. Among them, when constructing the second reference coordinate axis under the right viewing angle, the eye feature points can be the right eye feature points; when constructing the second reference coordinate axis under the left viewing angle, the eye feature points can be the left eye feature points.

[0062] At the same time, from a side viewing angle, a right triangle formed by the eye feature points, the nose initial point and the coordinate origin is constructed; wherein the eye feature points, the nose initial point and the coordinate origin are the vertices of the right triangle respectively.

[0063] Thus, a second deflection angle of the hypotenuse of the right triangle corresponding to the line between the eye feature point and the nose feature point is determined as the vertical deflection angle.

[0064] In one embodiment, when determining the second deflection angle of the hypotenuse of the right triangle corresponding to the line between the eye feature point and the nose feature point, the second perpendicular point from the nose feature point to the longitudinal axis of the second reference coordinate axis can be determined first; secondly, based on the distance from the eye feature point to the second perpendicular point, the distance from the eye feature point to the origin of the coordinate system, and the angle between the hypotenuse and the longitudinal axis of the right triangle, the angle between the line between the eye feature point and the nose feature point and the hypotenuse is determined as the second deflection angle.

[0065] For example, in the normal viewing angle, the deflection of the face in the vertical direction is as follows: Figure 4 As shown, feature point A is the feature point of the left eye, feature point B is the feature point of the right eye, and feature point C' is the feature point of the nose. From the figure, we can know the X-axis and Y-axis coordinates of the three feature points: feature point A (Xa, Ya), feature point B (Xb, Yb), and feature point C' (Xc', Yc').

[0066] Under the right viewing angle, determine the vertical deflection angle when the face is deflected as follows Figure 5 As shown. Among them, the eye feature point is the right eye feature point. The coordinates of each feature point on the ZEY coordinate axis are as follows: the coordinates of feature point B are (Zb, Yb), the coordinates of feature point C are (Zc, Yc), the coordinates of feature point C' are (Zc', Yc'), the vertical point of feature point C' on the Y axis is vertical point D (Zd, Yd), the vertical point of feature point B on the Z axis is vertical point E (Ze, Ye), angle α is the angle between BC' and BC, and angle β is the angle between BC and BE.

[0067] First, the first vertical point (vertical point E) on the horizontal plane from the right eye feature point (feature point B) to the initial point of the nose (feature point C) is the origin of the coordinate system, and the straight line where the line connecting the feature point C and the vertical point E is located is the horizontal axis (Z axis), so as to construct the second reference coordinate axis ZEY when the face in the target face image is not deflected. When the face is not deflected, the angle between BC and the Y axis is angle β, and when the face is deflected, BC' produces an angle α relative to BC. At the same time, a right triangle BEC formed by the feature point B, the feature point C and the origin of the coordinate system is constructed. Among them, the feature point B, the feature point C and the origin of the coordinate system are the vertices of the right triangle BEC respectively. From the figure, it can be determined that the second deflection angle α of the hypotenuse BC of the right triangle BEC corresponding to the line between the feature point B and the feature point C' is the vertical deflection angle.

[0068] In this embodiment, Figure 5 As shown, the angle α of the hypotenuse BC of the right triangle BEC corresponding to the line between the feature point B and the feature point C′ can be determined as the second deflection angle. When determining the deflection angle α, trigonometric functions can be used for calculation.

[0069] Among them, the eye feature points change together with the nose feature points when the face image is deflected. Therefore, the length from the eye feature points to the nose is unchanged before and after the deflection, that is, BC = BC'. First, the second perpendicular point from the feature point C' to the longitudinal axis (Y axis) of the second reference coordinate axis ZEY can be determined, that is, Figure 5 The vertical point D in the figure. Since the Y-axis coordinate of the feature point C' is known, and the Y-axis coordinate of the vertical point D is equal to the Y-axis coordinate of the feature point C', that is, Yd=Yc', the value of Yd can be obtained.

[0070] Secondly, the distance BD from the feature point B to the vertical point D and the distance BE from the feature point B to the origin of the coordinate system can be determined. Since the Y-axis coordinates of the feature point B, the vertical point D, and the origin of the coordinate system are known, it can be determined that BD = abs (Yb-Yd) and BE = abs (Yb-Ye), where abs means taking the absolute value.

[0071] Then, the length from the eye feature point to the nose remains unchanged before and after the deflection, that is, BC = BC', and trigonometric functions can be used to determine BE / cosβ = BC, BD / cos(α+β) = BC'. Therefore, BE / cosβ = BD / cos(α+β). (3)

[0072] And by Figure 5 It can be seen that β is the angle between BC and the Y axis. After a large sample test, it can be estimated that the range of β is 15° to 25°. Assuming β = 20°, substituting the value of β into (3), we can get BE / cos20° = BD / cos(α+20°). By transforming this formula, we can get α = arccos(BC*cosβ / BE)-β.

[0073] Among them, BE is the Y-axis distance from the eye feature point to the nose feature point when the face is not deflected. After a large sample test, it can be estimated that BE=AB*0.65, where AB is the distance between the eyes.

[0074] In one embodiment, the vertical deflection information also includes a vertical deflection direction; using the above example, Figure 5 When the coordinate axis and the right triangle are established in the manner shown, if the second deflection angle is greater than zero, the vertical deflection direction is determined to be upward deflection; if the second deflection angle is less than zero, the vertical deflection direction is determined to be downward deflection; if the second deflection angle is equal to zero, it is determined that there is no deflection in the vertical direction.

[0075] In the above embodiment, when determining the vertical deflection information, the deflection angle of the target face image in the vertical direction can be determined by only three feature points on the target image, and then using trigonometric functions and four arithmetic operations. Since fewer feature points are required, the implementation of this solution is simple, and the use of trigonometric functions for calculation is more in line with the actual face deflection situation, which can reduce calculation errors and reduce the amount of calculation. In addition, only two large sample test data are used in the calculation process, which reduces the deviation caused to the final result.

[0076] In summary, specific embodiments of the present subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recorded in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing can be advantageous.

[0077] The above is a method for determining a face deflection angle provided in one or more embodiments of this specification. Based on the same idea, one or more embodiments of this specification also provide a device for determining a face deflection angle.

[0078] Figure 6 This is a schematic diagram of the structure of a device for determining a face deflection angle according to an embodiment of this specification. Figure 6 , a device 600 for determining a face deflection angle may include:

[0079] An acquisition module 610 is used to acquire a target face image;

[0080] The first determination module 620 is used to determine target feature points in the target face image, and to determine first position information of each target feature point; the target feature points include left eye feature points, right eye feature points and nose feature points; the left eye feature points and the right eye feature points are located on the same horizontal plane;

[0081] The second determination module 630 is used to determine face deflection information in the target face image according to the first position information; the face deflection information includes horizontal deflection information and / or vertical deflection information.

[0082] In one embodiment, the apparatus 600 further includes:

[0083] A collection module, used for collecting second position information of each target feature point;

[0084] A processing module, used for performing correction processing on the second position information of each target feature point; the correction processing includes adjusting the left eye feature point and the right eye feature point to the same horizontal plane;

[0085] The third determination module is used to determine the first position information of each target feature point according to the result of the correction processing.

[0086] In one embodiment, the horizontal deflection information includes a horizontal deflection angle; the second determination module 630 includes:

[0087] A first construction unit is used to construct an isosceles triangle formed by a left eye feature point, a right eye feature point and a nose feature point under a vertical viewing angle; wherein the left eye feature point, the right eye feature point and the nose feature point are vertices of the isosceles triangle respectively;

[0088] The second construction unit is used to construct a first reference coordinate axis when the face is not deflected in the target face image under a vertical viewing angle, taking the nose feature point as the coordinate origin; the horizontal axis of the first reference coordinate axis is parallel to the eye line when the face is not deflected;

[0089] The first determining unit is used to determine a first deflection angle of the isosceles triangle relative to a first reference coordinate axis as a horizontal deflection angle.

[0090] In one embodiment, the first determining unit includes:

[0091] Determine the median of the base of an isosceles triangle;

[0092] According to the vertex angle of the isosceles triangle and the projection length of each waist on the horizontal axis of the first reference coordinate axis, the angle between the midline and the vertical axis of the first reference coordinate axis is calculated as the first deflection angle.

[0093] In one embodiment, the horizontal deflection information further includes a horizontal deflection direction;

[0094] If the first deflection angle is greater than zero, the horizontal deflection direction is determined to be a left deflection;

[0095] If the first deflection angle is less than zero, the horizontal deflection direction is determined to be a rightward deflection;

[0096] If the first deflection angle is equal to zero, it is determined that there is no deflection in the horizontal direction.

[0097] In one embodiment, the vertical deflection information includes a vertical deflection angle; the second determination module 630 includes:

[0098] The third construction unit is used to construct a second reference coordinate axis when the face is not deflected in the target face image under a side view, with the first vertical point of the horizontal plane from the eye feature point to the nose initial point as the coordinate origin and the straight line where the nose initial point and the first vertical point are connected as the horizontal axis; the nose initial point is the nose position when the face is not deflected; the eye feature point is the left eye feature point or the right eye feature point;

[0099] The fourth construction unit is used to construct a right triangle formed by the eye feature point, the nose initial point and the coordinate origin in a side view; wherein the eye feature point, the nose initial point and the coordinate origin are vertices of the right triangle respectively;

[0100] The second determination unit is used to determine a second deflection angle of the hypotenuse of the right triangle corresponding to the line between the eye feature point and the nose feature point as the vertical deflection angle.

[0101] In one embodiment, the second determining unit includes:

[0102] Determine a second perpendicular point from the nose feature point to the longitudinal axis of the second reference coordinate axis;

[0103] According to the distance from the eye feature point to the second vertical point, the distance from the eye feature point to the coordinate origin and the angle between the hypotenuse and the longitudinal axis of the right triangle, the angle between the line between the eye feature point and the nose feature point and the hypotenuse is determined as the second deflection angle.

[0104] In one embodiment, the vertical deflection information further includes a vertical deflection direction;

[0105] If the second deflection angle is greater than zero, the vertical deflection direction is determined to be an upward deflection;

[0106] If the second deflection angle is less than zero, the vertical deflection direction is determined to be a downward deflection;

[0107] If the second deflection angle is equal to zero, it is determined that there is no deflection in the vertical direction.

[0108] Those skilled in the art should understand that the above-mentioned device for determining the face deflection angle can be used to implement the method for determining the face deflection angle described above, and the detailed description thereof should be similar to that described in the above method part, and will not be further described here to avoid tediousness.

[0109] Based on the same idea, one or more embodiments of this specification also provide a device for determining a face deflection angle, such as Figure 7 As shown. The device for determining the deflection angle of a human face may have relatively large differences due to different configurations or performances, and may include one or more processors 701 and a memory 702, and the memory 702 may store one or more storage applications or data. Among them, the memory 702 may be a short-term storage or a permanent storage. The application stored in the memory 702 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the device for determining the deflection angle of a human face. Furthermore, the processor 701 may be configured to communicate with the memory 702, and execute a series of computer executable instructions in the memory 702 on the device for determining the deflection angle of a human face. The device for determining the deflection angle of a human face may also include one or more power supplies 703, one or more wired or wireless network interfaces 704, one or more input and output interfaces 705, and one or more keyboards 706.

[0110] Specifically in this embodiment, the device for determining the face deflection angle includes a memory, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions for determining the face deflection angle in the device, and is configured to be executed by one or more processors. The one or more programs include the following computer executable instructions: obtaining a target face image; determining target feature points in the target face image, and determining first position information of each target feature point; the target feature points include left eye feature points, right eye feature points and nose feature points; the left eye feature points and the right eye feature points are located on the same horizontal plane; according to the first position information, determining the face deflection information in the target face image; the face deflection information includes horizontal deflection information and / or vertical deflection information.

[0111] Optionally, when the computer executable instructions are executed, they may also cause the processor to execute the steps of the above-mentioned method for determining the deflection angle of the face.

[0112] One or more embodiments of the present specification also propose a computer-readable storage medium, which stores one or more programs, and the one or more programs include instructions, which, when executed by an electronic device including multiple applications, can enable the electronic device to perform the steps of the above-mentioned method for determining the face deflection angle.

[0113] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0114] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0115] Those skilled in the art will appreciate that one or more embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, one or more embodiments of this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] One or more embodiments of this specification are described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0117] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0119] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0120] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0121] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0122] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0123] One or more embodiments of the present specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0124] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0125] The above description is only one or more embodiments of this specification and is not intended to limit this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of one or more embodiments of this specification.

Claims

1. A method for determining a face deflection angle, characterized in that: include: Get the target face image; Determine target feature points in the target face image, and determine first position information of each target feature point; The target feature points include a left eye feature point, a right eye feature point and a nose feature point; the left eye feature point and the right eye feature point are located on the same horizontal plane; Determining face deflection information in the target face image according to the first position information; The face deflection information includes horizontal deflection information and / or vertical deflection information; The step of determining face deflection information in the target face image according to the first position information includes: In a vertical viewing angle, the horizontal deflection information is determined by using a trigonometric function according to the horizontal position information in the first position information, and the vertical viewing angle is a viewing angle simulating observation from the top of the head downward; The method further comprises: In the case that the left eye feature points and the right eye feature points are not located on the same horizontal plane, the left eye feature points and the right eye feature points are adjusted to be on the same horizontal plane in advance.

2. The method according to claim 1, characterized in that Before determining the first position information of each target feature point, the method further includes: Collecting second position information of each target feature point; Performing correction processing on the second position information of each of the target feature points; the correction processing includes adjusting the left eye feature point and the right eye feature point to the same horizontal plane; The first position information of each of the target feature points is determined according to a result of the correction process.

3. The method according to claim 1, characterized in that The horizontal deflection information includes a horizontal deflection angle; and determining the face deflection information in the target face image according to the first position information includes: Under a vertical viewing angle, construct an isosceles triangle formed by the left eye feature point, the right eye feature point and the nose feature point; wherein the left eye feature point, the right eye feature point and the nose feature point are vertices of the isosceles triangle respectively; Under the vertical viewing angle, taking the nose feature point as the coordinate origin, constructing a first reference coordinate axis when the face in the target face image is not deflected; the horizontal axis of the first reference coordinate axis is parallel to the line connecting the two eyes when the face is not deflected; A first deflection angle of the isosceles triangle relative to the first reference coordinate axis is determined as the horizontal deflection angle.

4. The method according to claim 3, characterized in that: Determining a first deflection angle of the isosceles triangle relative to the first reference coordinate axis includes: Determine the midline of the base of the isosceles triangle; According to the projection lengths of the vertex angles and waists of the isosceles triangle on the horizontal axis of the first reference coordinate axis, the angle between the midline and the vertical axis of the first reference coordinate axis is calculated as the first deflection angle.

5. The method according to claim 4, characterized in that The horizontal deflection information also includes a horizontal deflection direction; If the first deflection angle is greater than zero, determining that the horizontal deflection direction is a left deflection; If the first deflection angle is less than zero, determining that the horizontal deflection direction is a rightward deflection; If the first deflection angle is equal to zero, it is determined that there is no deflection in the horizontal direction.

6. The method according to claim 1, characterized in that The vertical deflection information includes a vertical deflection angle; and determining the face deflection information in the target face image according to the first position information includes: In a side view, a second reference coordinate axis when the face is not deflected in the target face image is constructed with the first vertical point of the horizontal plane from the eye feature point to the nose initial point as the coordinate origin and the straight line where the line connecting the nose initial point and the first vertical point is located as the horizontal axis; the nose initial point is the nose position when the face is not deflected; the eye feature point is the left eye feature point or the right eye feature point; Under the side viewing angle, a right triangle formed by the eye feature point, the nose initial point and the coordinate origin is constructed; wherein the eye feature point, the nose initial point and the coordinate origin are vertices of the right triangle respectively; A second deflection angle of the hypotenuse of the right triangle corresponding to the line between the eye feature point and the nose feature point is determined as the vertical deflection angle.

7. The method according to claim 6, characterized in that The determining a second deflection angle of the hypotenuse of the right triangle corresponding to a line connecting the eye feature point and the nose feature point comprises: Determine a second perpendicular point from the nose feature point to the longitudinal axis of the second reference coordinate axis; According to the distance from the eye feature point to the second vertical point, the distance from the eye feature point to the coordinate origin, and the angle between the hypotenuse of the right triangle and the longitudinal axis, the angle between the line between the eye feature point and the nose feature point and the hypotenuse is determined to be the second deflection angle.

8. The method according to claim 7, characterized in that The vertical deflection information also includes a vertical deflection direction; If the second deflection angle is greater than zero, determining that the vertical deflection direction is an upward deflection; If the second deflection angle is less than zero, determining that the vertical deflection direction is a downward deflection; If the second deflection angle is equal to zero, it is determined that there is no deflection in the vertical direction.

9. A device for determining a face deflection angle, characterized in that: include: An acquisition module is used to acquire a target face image; A first determination module, used to determine target feature points in the target face image, and to determine first position information of each target feature point; The target feature points include a left eye feature point, a right eye feature point and a nose feature point; the left eye feature point and the right eye feature point are located on the same horizontal plane; A second determining module, configured to determine face deflection information in the target face image according to the first position information; The face deflection information includes horizontal deflection information and / or vertical deflection information; a processing module, configured to pre-adjust the left eye feature points and the right eye feature points to be on the same horizontal plane if the left eye feature points and the right eye feature points are not on the same horizontal plane; The second determining module is further used to: Under the vertical viewing angle, the horizontal deflection information is determined by using a trigonometric function according to the horizontal position information in the first position information. Said The vertical viewing angle simulates looking down from the top of the head.

10. A device for determining a face deflection angle, characterized in that: It comprises a processor, a communication interface, a memory and a communication bus; wherein the processor, the communication interface and the memory communicate with each other through the bus; the memory is used to store computer programs; the processor is used to execute the programs stored in the memory to implement the method steps described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 8 are implemented.

Citation Information

Patent Citations

  • Comic exaggeration method, device and system for human face digital image supporting position deflection

    CN102103756A

  • Face recognition method and device

    CN106203400A

  • Method and device for determining face rotating angle

    CN106991367A

  • Student sitting posture detection and correction system based on image recognition

    CN107103309A

  • Method and device for acquiring face image

    CN108875479A