Method for classifying a shape of a face

Abstract

Computer-implemented method for classifying a shape of a face comprising the steps: a. receiving a picture of the face; b. identifying at least two homogeneous characteristic quantities of the face on the basis of the picture; c. calculating at least a first ratio dividing the two characteristic quantities; d. performing a comparison between the first ratio and a first threshold; e. classifying the shape among a plurality of classes on the basis of the comparison.

Description

[0001] METHOD FOR CLASSIFYING A SHAPE OF A FACE

[0002] The present invention relates to a computer-implemented method for classifying a shape of a face on the basis of a picture of that face.

[0003] The classification of the shape of a face is particularly important in many sectors of the art, even very different from each other. These include the possibility of diagnosing a facial pathology on the basis of a change in the shape of the face, the possibility of reducing the number of face comparisons to be carried out in order to recognise a face captured by a camera, and the possibility of identifying the most suitable spectacle frame for a user's face.

[0004] With regard to the latter application, it should be emphasised that the choice of a spectacle frame appropriate for the user's face is crucial not only for aesthetic reasons, but also to ensure the correct positioning of the lenses in front of the eyes and the optimal distribution of the weight of the spectacles between the nose and the ears. A wrong positioning of the spectacles, e.g. due to a frame that does not fit the shape of the user's face, could in fact cause eye diseases and nose and ear pain to the user.

[0005] Currently, there is a consensus across the various technical sectors involved to define a limited number of classes of face shapes, namely the triangle shape (first class) , the oval shape (second class) , the round shape (third class) and the square shape (fourth class) . The second class, i.e. the one of the oval shape, is further subdivided into two sub-classes: the oval-angled shape (first sub-class) and the oval-rounded shape (second sub-class) .

[0006] Generally, the classification of the shape of the face, i.e. the inclusion of the shape of the face in one of the above-mentioned classes, is performed by persons particularly skilled in the art of physiognomy, on the basis of their own perception and experience. This "manual" classification is extremely precise, but it is time-consuming and therefore difficult to apply when there are large numbers of faces to be classified; it also obviously requires the involvement of an expert, who may not always be available and who in any case involves non-negligible economic costs.

[0007] Machine learning algorithms classifying the shape of the face automatically have also been developed. These algorithms make it possible to quickly classify even large quantities of faces at a low cost, but are generally highly dependent on the data used for training and not very adaptable once trained. In order to obtain acceptable results, it is in fact necessary to train these algorithms with many pictures of faces preferably having similar somatic features and taken by the same camera under substantially the same light exposure conditions; consequently, the need to classify faces having different somatic features, or whose pictures were taken with a different camera or under different light exposure conditions, requires re-training the algorithm with a new and different set of many pictures: these pictures are not always immediately available, and in any case the re-training operation is usually timeconsuming and burdening.

[0008] The object of the present invention is to overcome the above-mentioned drawbacks, and in particular to carry out a computer-implemented method for classi fying the shape of a face that is accurate , fast , cheap and easi ly adaptable .

[0009] This and other results are obtained according to the present invention by carrying out a method according to claim 1 .

[0010] Further characteristics of the method are the obj ect o f the dependent claims .

[0011] The present invention will now be described, by way of an il lustrative , though non-limiting example , according to preferred embodiments thereof , with reference to the enclosed figures , wherein :

[0012] - Figure 1 is a simpli fied representation of a frontal picture of a subj ect ' s face , wherein quantities useful for carrying out a method according to the invention are highlighted;

[0013] - Figure 2 is a schematic representation of a first embodiment of a method according to the invention;

[0014] - Figure 3 is a schematic representation of a second embodiment of a method according to the invention .

[0015] With reference to Figure 1 , the first step of the method according to the invention is to receive a picture I o f the face whose shape is to be classified . The received picture I can be previously taken, or more generally generated, according to known processes using a camera . At least two homogeneous characteristic quantities of the face are then identi fied on the basis of picture I , and a first ratio R1 is calculated by dividing them . The first ratio R1 thus obtained is then compared with a first threshold S I , and on the basis of this comparison the shape of the faces is classi fied among the known classes . The fact that two homogeneous quantities are used in the method according to the invention makes the first ratio R1 dimensionless, and thus invariant for enlargements and shrinkages of the picture I. Therefore, the distance of the face from the lens with which the picture I is generated is irrelevant.

[0016] More specifically, the following quantities are advantageously identified on the basis of picture I:

[0017] - face length 10, defined as the distance between the trichion 11 (i.e. the point belonging to the vertical median plane of the face and located at the height of the hairline) and the chin 12 (i.e. the point belonging to the vertical median plane of the face and defining the lower end of the face) ;

[0018] - the forehead width 20, defined as the maximum width of the face at the height of the forehead, delimited by a first forehead end 21 and a second forehead end 22;

[0019] - the zygomatic width 30, defined as the width of the face at the height of the cheekbones, comprised between a first zygomatic end 31 and a second zygomatic end 32;

[0020] - the jaw width 40, defined as the width of the face at the height of the jaw, comprised between a first jaw end 41 and a second jaw end 42;

[0021] - the face area 50, defined as the surface contained within the contour line 51 of the face, this contour line 51 being defined as the closed curve passing through the chin 12, the jaw ends 41, 42, the zygomatic ends 31, 32, the forehead ends 21, 22 and a mid-forehead point 80 defined at the centre of the forehead of the face, below the trichion 11;

[0022] - the rectangular area 60, defined as the surface of the minimum area rectangle that entirely contains the contour line 51 of the face ;

[0023] - the chin- j aw angle a, defined as the angle between a hori zontal line passing through the chin 12 and a line passing through the chin 12 and the first aw end 41 ;

[0024] - the forehead- zygoma- j aw angle p , defined as the angle comprised between a line passing through the first forehead end 21 and the first zygomatic end 31 and a line passing through the first zygomatic end 31 and the first j aw end 41 ; the chin-mandible angle y , defined as the angle comprised between a hori zontal l ine passing through the chin 12 and a line passing through the chin 12 and the lateral end 70 of the face at the mandible height . Preferably, the height of the mandible can be defined as hal f the height of the j aw .

[0025] These quantities are preferably identified by means of machine learning algorithms applied to the received face picture I , these algorithms being configured to identi fy landmarks on the picture I according to known methodologies and to identi fy the necessary quantities accordingly . The contour line 51 of the face is also identi fied in the same way using the landmarks identi fied on picture I .

[0026] As far as the detection of the trichion 11 is concerned, a known machine learning algorithm can be used for thi s purpose , which among the various landmarks also identi fies the trichion 11 on picture I .

[0027] Alternatively, it is possible to identi fy the trichion 11 indirectly, using the position of the mid- forehead point 80 and of the chin 12 in combination with the face segmentation line 52 . The segmentation line 52 is identi fied on the picture I by means of segmentation models according to known methodologies, and is defined as the closed curve passing through the chin 12, the jaw ends 41, 42, the zygomatic ends 31, 32, the forehead ends 21, 22 and the top of the face, separating the face from any hair. In particular, in order to identify the trichion 11 indirectly, a known machine learning algorithm is first used to identify the mid-forehead point 80 and the chin 12 on the picture I. The trichion 11 is then identified as the upper intersection point between the segmentation line 52 and the line passing through the chin 12 and the mid-forehead point 80.

[0028] Preferably, after the reference points have been identified on the picture I and the various quantities have been identified, an angle of rotation is calculated as the angle that the forehead width 20 forms relative to a horizontal line. At this point, the chin-jaw angle a and the chin-mandible angle y are recalculated by subtracting the value of the angle of rotation from the values initially identified: in this way, the chin-jaw angle a and the chin-mandible angle y are not affected by any rotations or inclinations of the face at the time the picture I is generated.

[0029] Preferably, the face length 10, forehead width 20, zygomatic width 30 and jaw width 40 are then "normalised"; in detail, each of these quantities is divided by the total sum of these quantities, so as to make each of these quantities invariant for enlargements and shrinkages of the picture I.

[0030] The following ratios are then calculated, which are dimensionless as they are ratios between homogeneous quantities : the first ratio R1 defined as the ratio of the forehead width 20 to the jaw width 40; a second ratio R2 defined as the ratio of the rectangular area 60 to the face area 50;

[0031] - a third ratio R3 defined as the ratio of the face length 10 to the zygomatic width 30.

[0032] We then proceed to check whether the face belongs to the first class Cl, i.e. whether it has a triangle shape. It is checked if the first ratio R1 is greater than the first threshold SI and if the second ratio R2 is greater than a second threshold S2 : if at least one of the two conditions occurs, the shape of the face is classified in the first class Cl, i.e. as a triangle shape.

[0033] The first threshold SI is comprised between 1.10 and 1.15. In particular, the first threshold SI is defined as 1.14 if picture I was generated using the camera of a mobile phone, while otherwise the first threshold SI is defined as 1.12.

[0034] The second threshold S2 is comprised between 1.21 and 1.22. In particular, the second threshold S2 is defined as 1.215.

[0035] If the shape of the face is not classified in the first class Cl, one proceeds to check whether the face belongs to the second class C2, i.e. whether it has an oval shape. It is checked if the third ratio R3 is greater than a third threshold S3: if this condition occurs, the shape of the face is classified in the second class C2, i.e. as an oval shape.

[0036] The third threshold S3 is comprised between 1.35 and 1.45. In particular, the third threshold S3 is defined as 1.44 if the picture I was generated using the camera of a mobile phone, while otherwise the third threshold

[0037] S3 is defined as 1.38. If the shape of the face is classified in the second class C2, one proceeds to check whether the face belongs to the first sub-class C2-1 or the second sub-class C2- 2, i.e. whether it has an oval-angled shape or an oval- rounded shape. It is checked if the chin-mandible angle Y is greater than a third angular threshold SA3 : if this condition occurs, the shape of the face is classified in the first sub-class C2-1, i.e. as an oval-angular shape, while otherwise the face shape is classified in the second sub-class C2-2, i.e. as an oval-rounded shape. The third angular threshold SA3 is comprised between 17 degrees and 22 degrees. In particular, the third angular threshold SA3 is defined as 20 degrees if picture I was generated using a mobile phone camera, while otherwise the third angular threshold SA3 is defined as 19 degrees. If the shape of the face is not classified in the second class C2 either, one proceeds to check whether the face belongs to the third class C3, i.e. if it has a round shape. It is checked if the third ratio R3 is less than a fourth threshold S4 and if the forehead-zygoma- j aw angle p is less than a first angular threshold SAI: if at least one of the two conditions occurs, the shape of the face is classified in the third class C3, i.e. as a round shape .

[0038] The fourth threshold S4 is comprised between 1.15 and 1.35. In particular, the fourth threshold S4 is defined as 1.30 if picture I was generated using the camera of a mobile phone, while otherwise the fourth threshold S4 is defined as 1.20.

[0039] The first angular threshold SAI is comprised between 155 degrees and 160 degrees. Specifically, the first angular threshold SAI is defined as 158 degrees if picture I was generated using a mobile phone camera, while otherwise the first angular threshold SAI is defined as 156 degrees .

[0040] I f the shape of the face is also not classified in class C3 , a final check is carried out . It is checked i f the chin- j aw angle a is less than a second angular threshold SA2 : i f this condition occurs the shape of the face is classi fied in any case in the third class C3 , i . e . as a round shape , while otherwise the face shape is classi fied in the fourth class C4 , i . e . as a square shape .

[0041] The second angular threshold SA2 is comprised between 25 degrees and 30 degrees . Speci fically, the second angular threshold SA2 is defined as 28 degrees i f picture I was generated using a mobile phone camera, while otherwise the second angular threshold SA2 is defined as 26 degrees .

[0042] It is clear that in the method according to the invention only dimensionless ratios and angles are used to classi fy the shape o f a face . This choice , as already explained, makes the distance of the subj ect from the lens with which picture I is generated irrelevant , as the ratios of homogeneous quantities and angles are quantities invariant for enlargements and shrinkages of picture I . The steps described so far consist of a first embodiment of the method according to the invention, shown in Figure 2 .

[0043] In a second embodiment of the method according to the invention, shown in Figure 3 , two further steps are added before the steps of comparing quantities and ratios with the corresponding thresholds . These further steps consist in identi fying whether the picture I was generated by means of a mobile phone camera, and in modifying accordingly the first threshold SI, the third threshold S3, the fourth threshold S4, the first angular threshold SAI, the second angular threshold SA2 and the third angular threshold SAS according to the preferred values listed above. This makes it possible to automatically adapt the method to the case where the picture I was generated using a mobile phone camera, as the thresholds are automatically modified on the basis of the fact that picture I was generated by a mobile phone camera.

[0044] In fact, it was observed that the threshold values for achieving the most accurate classification are different depending on whether the picture I was generated by a mobile phone camera (such as a smartphone) or by a different camera (such as a laptop camera) . This is why, in the first embodiment of the method, the optimal thresholds are different depending on whether the picture I was generated using a mobile phone camera or a different camera.

[0045] This is a consequence of the difference between the typical focal length of mobile phone cameras and that of other cameras, which results in a slight difference in the deformation of the picture I generated.

[0046] In use, the method according to the invention has proven to be more accurate in classifying the shape of faces than generally used machine learning algorithms.

[0047] In particular, a first test was conducted by classifying the same 144 pictures using both the method according to the invention and a known machine learning algorithm. The 144 test pictures were previously classified by a person skilled in the art, whose classification, as described above, is extremely precise and reliable. In this first test, the method according to the invention correctly classified 59.0% of the pictures, while the machine learning algorithm correctly classified only 45.1% of the pictures. It should be specified that the classification of a picture is considered correct if it corresponds to that performed by the person skilled in the art .

[0048] A second test was also conducted by classifying 48 groups of pictures, each consisting of 3 pictures of the same face previously classified by a person skilled in the art, using both the method according to the invention and the machine learning algorithm. This second test was intended to assess the "accuracy" of the method when it is required to classify a face on the basis of several pictures .

[0049] In this second test, the method according to the invention correctly classified 66.7% of the groups of pictures, while the machine learning algorithm correctly classified only 45.8% of the groups of pictures. It should be specified that the classification of a group of pictures is considered correct if at least two of the three pictures making up the group have been classified correctly with respect to the classification performed by the person skilled in the art.

[0050] In both tests, the time taken by the method according to the invention and the machine learning algorithm to complete the classification were substantially identical .

[0051] It is therefore clear that the method according to the invention allows to achieve the intended objects. In particular, the method makes it possible to classify the shape of faces more quickly and cost-effectively than could be achieved by a person skil led in the art , and more accurately than with a machine learning algorithm of the known type .

[0052] In addition, the method according to the invention is particularly adaptable , as it is suf ficient to modi fy the thresholds to adapt the clas si fication to di f ferent camera types ( consider in this regard the simplicity with which the method can be adapted to classi fy pictures generated by a mobile phone camera ) , di f ferent light exposures or di f ferent somatic features of the subj ects . The method according to the invention is implemented by means of a computer . Therefore , they are also included in the scope of protection of the invention an apparatus compri sing a data processing unit configured to carry out the method according to the invention;

[0053] - a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method according to the invention;

[0054] - a computer-readable data storage device comprising instructions which, when executed by the computer, cause the computer to carry out the method according to the invention .

[0055] An apparatus comprising a data processing unit and a camera is also included within the scope of protection of the invention, wherein the camera is configured to generate a picture I and to transmit it to the data processing unit , which is configured to carry out the method according to the invention on the basis of the received picture I .

[0056] The present invention has been described by way of a non-limiting illustrative example according to preferred embodiments thereof , however, it is understood that variations and / or modi fications may be made by a person skilled in the art , without thereby departing from the relative scope of protection, as defined in the attached claims .

Claims

CLAIMS1) Computer implemented method for classifying a shape of a face comprising the steps: a. receiving a picture (I) of said face; b. identifying at least two homogeneous characteristic quantities of said face on the basis of said picture (I) ; c. calculating at least a first ratio (Rl) dividing two of said at least two characteristic quantities; d. performing a comparison between said first ratio (Rl) and a first threshold (SI) ; e. classifying said shape among a plurality of classes (Cl, C2, C3, C4) on the basis of said comparison.2) Method according to claim 1 wherein:- said step b comprises identifying a face length (10) , a forehead width (20) , a zygomatic width (30) , a jaw width (40) , a face area (50) , a rectangle area (60) , a chin- aw angle (a) and a forehead-zygoma- j aw angle (p) of said face on the basis of said picture (I) ;- said step c comprises calculating said first ratio (Rl) dividing said forehead width (20) by said jaw width (40) , calculating a second ratio (R2) dividing said rectangle area (60) by said face area (50) and calculating a third ratio (R3) dividing said face length (10) by said zygomatic width (30) ;- said steps d and e comprise the following further steps :- if said first ratio (Rl) is greater than said first threshold (SI) or said second ratio (R2) is greater than a second threshold (S2) , classifying said shape in a first class (Cl ) ;- else, if said third ratio (R3) is greater than a thirdthreshold (S3) , classifying said shape in a second class (C2) ;- else, if said third ratio (R3) is less than a fourth threshold (S4) or said forehead-zygoma- j aw angle (p) is less than a first angle threshold (SAI) , classifying said shape in a third class (C3) ;- else, if said chin-jaw angle (a) is less than a second angle threshold (SA2) , classifying said shape in a third class (C3 ) ; - else classifying said shape in a fourth class (C4) .3) Method according to claim 2, characterized by comprising the following further steps:- identifying if said picture (I) was generated by means of a mobile phone camera; - if said picture (I) was generated by means of a mobile phone camera, modifying at least one of said first threshold (SI) , second threshold (S2) , third threshold (S3) , fourth threshold (S4) , first angle threshold (SAI) and second angle threshold (SA2) . 4) Method according to any of claims 2 or 3, characterized in that said first threshold (SI) is comprised between 1,10 and 1,15, in particular 1,14 if said picture (I) was generated by means of a mobile phone camera and 1,12 otherwise. 5) Method according to any of claims 2 to 4, characterized in that said second threshold (S2) is comprised between 1,21 and 1,22, in particular 1,215.6) Method according to any of claims 2 to 5, characterized in that said third threshold (S3) is comprised between 1,35 and 1,45, in particular 1,44 if said picture (I) was generated by means of a mobile phone camera and 1,38 otherwise.7) Method according to any of claims 2 to 6, characterized in that said fourth threshold (S4) is comprised between 1,15 and 1,35, in particular 1,30 if said picture (I) was generated by means of a mobile phone camera and 1,20 otherwise.8) Method according to any of claims 2 to 7, characterized in that said first angle threshold (SAI) is comprised between 155 degrees and 160 degrees, in particular 158 degrees if said picture (I) was generated by means of a mobile phone camera and 156 degrees otherwise .9) Method according to any of claims 2 to 8, characterized in that said second angle threshold (SA2) is comprised between 25 degrees and 30 degrees, in particular 28 degrees if said picture (I) was generated by means of a mobile phone camera and 26 degrees otherwise .10) Method according to any of claims 2 to 9, characterized in that, if said shape is classified in said second class (C2) , it comprises the following further steps:- identifying a chin-mandible angle (y) of said face on the basis of said picture (I) ;- if said chin-mandible angle (y) is greater than a third angle threshold (SA3) , classifying said shape in a first sub-class (C2-1) of said second class (C2) ;- else classifying said shape in a second sub-class (C2- 2) of said second class (C2) .11) Method according to claim 10, characterized in that said third angle threshold (SA3) is comprised between 17 degrees and 22 degrees, in particular 20 degrees if said picture (I) was generated by means of a mobile phonecamera and 19 degrees otherwise.12) Method according to any of claims 2 to 11, characterized in that said face length (10) , forehead width (20) , zygomatic width (30) , jaw width (40) , face area (50) , rectangle area (60) , chin-jaw angle (a) , forehead-zygoma- j aw (p) and a chin-mandible angle (y) of said face are identified by means of landmarks defined on the basis of said picture (I) , said landmarks being preferably defined by means of a machine learning algorithm.13) Apparatus comprising a data processing unit designed to carry out a method for classifying a shape of a face comprising the steps: a. receiving a picture (I) of said face; b. identifying at least two homogeneous characteristic quantities of said face on the basis of said picture (I) ; c. calculating at least a first ratio (Rl) dividing two of said at least two characteristic quantities; d. performing a comparison between said first ratio (Rl) and a first threshold (SI) ; e. classifying said shape among a plurality of classes (Cl, C2, C3, C4) on the basis of said comparison.14) Apparatus according to claim 13 characterized in that it further comprises a camera designed to generate said picture (I) and to send said picture (I) to said data processing unit.15) Computer program comprising instructions that, when said program is executed by a computer, cause the computer to carry out a method for classifying a shape of a face comprising the steps: a. receiving a picture (I) of said face;b. identifying at least two homogeneous characteristic quantities of said face on the basis of said picture (I) ; c. calculating at least a first ratio (Rl) dividing two of said at least two characteristic quantities; d. performing a comparison between said first ratio (Rl) and a first threshold (SI) ; e. classifying said shape among a plurality of classes (Cl, C2, C3, C4) on the basis of said comparison.

Images