Device and method for determining information concerning quantities relating to a face

Abstract

The present invention relates to a method for determining information concerning quantities relating to a face captured by at least one camera, comprising - detecting at least one face in at least one image captured by the at least one camera - adjusting a model of a reference face on which key points are located, based on an estimate of a morphological typing of the detected face; - on the detected face, localizing key points corresponding to at least a first subset of the key points of the reference face, and obtaining the location, relative to the camera, of the at least one detected face relative to the at least one camera in a length measurement system, said location being obtained based on a focal length of the camera, pixel coordinates of at least a second subset of the key points located on the at least one detected face, and coordinates in the length measurement system of the key points of the first subset located on the adjusted modelled reference face.

Description

Description Title of the invention: Device and method for determining size information relating to a face Technical Field

[0001] The present invention relates to the field of computer vision and more specifically to the processing of images captured by conventional type cameras. Prior art

[0002] Many techniques for detecting people in images or videos exist. Indeed, more and more applications, such as surveillance and security applications, are using these methods to identify people. Several of these approaches are based on artificial intelligence and in particular on "deep learning". One of the limitations of existing methods is that they allow people to be located in the image frame of reference (for example, a two-dimensional space whose distances are expressed in pixels), and not to locate them relative to the camera, in a three-dimensional space, whose distances are expressed in a spatial coordinate system, such as a metric system.

[0003] There are also methods for refining face detection, by proposing to detect the orientation of the gaze in a two-dimensional frame of reference of the image. However, the known methods do not allow both to locate people relative to the camera in a three-dimensional frame of reference equipped with a length measurement system, and to detect the orientation of the person's gaze in this same measurement system.

[0004] Thus, there is a need to improve the localization of person(s) in a repository. Statement of the invention

[0005] The invention proposes to overcome at least one drawback of the prior art by proposing a method for determining magnitude information relating to a face captured by at least one camera.

[0006] According to at least one embodiment, the method comprises: - a location of key points on at least one face detected in an image captured by a camera, said key points corresponding to key points of a modeling of at least one reference face and, - obtaining a location relative to the camera, in a real-world reference frame, of said at least one detected face, from a focal distance of said camera, the coordinates of at least one of said key points located on said at least one detected face in said image, the coordinates in said reference frame of the key points located on said modeling.

[0007] According to at least one embodiment, the method comprises: - an adjustment of said modeling of at least one reference face, from an estimation of a morphology of said detected face.

[0008] According to at least one embodiment, the method comprises - obtaining an orientation of said at least one detected face from the focal length of said camera, the pixel coordinates of at least one of said key points located on said at least one detected face, the coordinates in said length measurement system of the key points located on said modeling.

[0009] According to at least one embodiment, obtaining a location and obtaining an orientation are obtained by minimizing the same error function.

[0010] According to at least one embodiment, the adjustment of said modeling of at least one reference face comprises a selection of a modeling of a reference face corresponding to said morphology.

[0011] According to at least one embodiment, adjusting a model of at least one face comprises: - a selection of a reference face from a plurality of reference faces, the selection taking into account at least one characteristic of said morphology, - an adjustment of the selected reference face based on one or more characteristics of said morphology not taken into account when selecting the reference face.

[0012] According to at least one embodiment, the adjustment of a modeling of at least one reference face comprises an adjustment of the coordinates of the key points of said reference face according to said morphology.

[0013] According to at least one embodiment, adjusting a model of a reference face comprises: - obtaining the coordinates of said key points on said modeling of the reference face by multiplying the coordinates of said key points on said reference face by a coefficient relating to an estimated age of said at least one detected face and by a coefficient relating to an estimated gender of said at least one detected face.

[0014] According to at least one embodiment, said key points of said at least one detected face are located on salient points of said at least one detected face, at least some of which are located on one or the other or more of the eyes, the mouth, the nose, or the contours of said at least one detected face.

[0015] According to at least one embodiment, said reference system is a three-dimensional length measurement system, the origin of which is centered on a position of the camera.

[0016] Thus, according to at least one embodiment, the method comprises: - a detection of at least one face in at least one image captured by the at least one camera; - an adjustment of a modeling of at least one reference face, on which a plurality of key points are located, from an estimation of a morphological typing of said at least one detected face; - a location, on said at least one detected face, of key points corresponding to at least a first subset of said key points of said modeling of the reference face and, - obtaining a location relative to the camera, in a length measurement system, of said at least one detected face relative to said at least one camera, said location being obtained from a focal length of said camera, coordinates in pixels of at least a second subset of said key points located on said at least one detected face, coordinates in said length measurement system of the key points of said first subset located on said adjusted reference face model.

[0017] According to at least one embodiment, the method comprises: - obtaining an orientation of said at least one detected face from the focal length of said camera, the pixel coordinates of at least said second subset of said key points on said at least one detected face, the coordinates in said length measurement system of the key points of said first subset located on said adjusted reference face model.

[0018] According to at least one embodiment, obtaining a location and obtaining an orientation are obtained by minimizing the same error function.

[0019] According to at least one embodiment, the adjustment of a modeling of at least one reference face comprises a selection of a modeling of a reference face corresponding to said morphological typing.

[0020] According to at least one embodiment, adjusting a model of at least one reference face comprises: - a selection of a reference face from a plurality of reference faces, the selection taking into account at least one characteristic of said morphological typing, - an adjustment of the selected reference face based on one or more characteristics of said morphological typing not taken into account when selecting the reference face.

[0021] According to at least one embodiment, the adjustment of a modeling of at least one reference face comprises an adjustment of the coordinates of the key points of said modeling of the reference face according to said morphological typing.

[0022] According to at least one embodiment, said distance and said orientation are obtained by minimizing an error function from the RANSAC method.

[0023] According to at least one embodiment, adjusting a reference face model comprises: - obtaining the coordinates of said key points on said adjusted reference face model by multiplying the coordinates of said key points on the reference model by a coefficient relating to an estimated age of said at least one detected face and by a coefficient relating to an estimated gender of said at least one detected face.

[0024] According to at least one embodiment, said first subset comprises at least 5 key points.

[0025] According to at least one embodiment, said key points of said at least one detected face are located on salient points of said at least one detected face, at least some of which are located on one or the other or more of the eyes, the mouth, the nose, or the contours of said at least one detected face.

[0026] According to at least one embodiment, detecting a face comprises determining a bounding box around the detected face.

[0027] According to at least one embodiment, the length measuring system is a three-dimensional system whose origin of the reference frame is centered on a position of the camera.

[0028] The features presented in isolation in the present application in connection with certain embodiments of the method of the present application can be combined with each other according to other embodiments of the present method.

[0029] The present invention also relates to a computer program comprising instructions for executing the steps of the method according to the invention, according to one any of its embodiments, when said program is executed by a computer.

[0030] The present invention also relates to a computer-readable recording medium on which is recorded a computer program comprising instructions for executing the steps of the method according to the invention, according to any one of its embodiments.

[0031] The present invention also relates to a device suitable for implementing the method of the present application in any of its embodiments.

[0032] Thus, the present invention relates to a device for determining information on quantities relating to a face captured by at least one camera.

[0033] According to at least one embodiment, the device comprises one or more processors configured together or separately to: - locating key points on at least one face detected in an image captured by a camera, said key points corresponding to key points of a model of at least one reference face and, - obtaining a location relative to the camera, in a real-world reference frame, of said at least one detected face, from a focal distance of said camera, the coordinates of at least one of said key points located on said at least one detected face in said image, and / or the coordinates in said reference frame of the key points located on said modeling.

[0034] According to at least one embodiment, said device comprises means for detecting said at least one face in said at least one image.

[0035] According to at least one embodiment, said camera is a camera of said device.

[0036] According to at least one embodiment, the device comprises one or more processors configured together or separately to - detect at least one face in at least one image captured by said at least one camera, - adjusting a model of at least one reference face, on which a plurality of key points are located, from an estimation of a morphological typing of said at least one detected face - locating, on said at least one detected face, key points corresponding to at least a first subset of said key points of said modeling of the reference face, - obtaining a location, in a length measurement system, of said at least one detected face relative to said at least one camera, said location being obtained from a focal length of said camera, pixel coordinates of at least a second subset of said key points located on said at least one detected face, coordinates in said length measurement system of the key points of said first subset located on said adjusted reference face model.

[0037] Other characteristics and advantages of the present invention will emerge from the description given below, with reference to the accompanying drawings which illustrate an exemplary embodiment thereof without any limiting character. Brief description of the drawings

[0038] [Fig. 1] Figure 1 shows a method according to certain embodiments of the present invention,

[0039] [Fig. 2] Figure 2 represents an example of adaptation of a reference model of a face to the gender and age of the person,

[0040] [Fig. 3] Figure 3 shows an example of the localization of keypoints on the face of the captured person,

[0041] [Fig. 4] Figure 4 shows a device according to certain embodiments of the present invention. Description of the embodiments

[0042] The present invention mainly relates to the determination of magnitude information relating to a face. It can in particular allow the localization of people in a three-dimensional frame of reference, for example in the form of a vector of x, y and z coordinates expressed in the metric system of units (for example in meters); Thus, it is a localization in the real world. Such a three-dimensional frame of reference differs from an image frame of reference in which the dimensions can for example be expressed in pixels. Such a three-dimensional frame of reference can for example have as its origin the position of the camera.

[0043] The present application relates to may also allow in certain embodiments to also detect the direction of the gaze of the person, the direction also being determined in the three-dimensional reference frame. By reference frame, we can understand, in other words, a reference frame in a length measurement system making it possible to determine a distance between the camera and the detected person. In the same way, the orientation of the face of the detected person is determined in a length measuring system such as a three-dimensional reference frame in which the camera is located and more particularly according to certain embodiments, a reference frame in which the camera is located at the origin of the three axes of the reference frame, therefore with coordinates the vector (0,0,0).

[0044] In a step E1, one or more faces are detected in an image captured by a camera. The detected face is also called the target face in the remainder of the description. Several people, therefore faces, can be detected in the same image or in the same video, representing a scene acquired by one or more cameras. Depending on the context of the present disclosure, this may for example be an indoor or outdoor scene. This scene may include one or more moving or static people.

[0045] For example, in the case of a panography, consisting of reconstituting an image representation of a real scene by assembling a plurality of shots, taken for example by a plurality of cameras, it is possible to reconstitute a virtual shooting camera to which a focal length and a viewing angle are associated. The steps of the method can therefore be applied from this reconstituted image and the virtual camera, the center of the three-dimensional reference then being the position of the virtual camera.

[0046] According to certain embodiments, the detection of a face may comprise a step of tracing (or determining or delimiting) an area around the face. This area may for example be a rectangle, known in English as a "bounding box" or a bounding box in French.

[0047] Thus, at the input of step E1, one or more images or videos are received and at the output one or more images or videos are obtained (or alternatively one or more portions of at least one of these still images or videos) on which detected faces or certain detected faces are identified (or marked, or located or identified, or determined).

[0048] According to certain embodiments, images can be selected based on criteria such as: -the image includes at least N faces, N being greater than or equal to 1, -the image includes at least one face whose size, in pixels, is greater than a first value (for example a first value determined beforehand, for example during a configuration). The closer the faces are to the camera, the larger their size - the image includes a type of face that we wish to detect, man, woman, child, - criteria relating to facial recognition: we can enter a certain number of visual or morphological criteria or a robot portrait of an individual and only select images based on these criteria or meeting one or more of these criteria.

[0049] According to certain embodiments, it is possible to select from the images selected as indicated above, and comprising one or more faces, one or more faces according to criteria such as those defined above and to position a bounding box for example, only on the face(s) selected in the selected images.

[0050] We can therefore, for example, determine a zone per face or a zone only for certain faces, by configuring step E1.

[0051] According to certain embodiments, if several faces are detected, the following steps E2 to E5 of the method can be implemented simultaneously for each of the detected faces or iteratively, face by face. At least some of the detected faces may have been selected, following their detection, according to criteria as mentioned above and the following steps can be applied only to these selected faces. It is thus possible to detect the distance of at least some of the faces (for example of each of the faces) relative to the camera and their orientation relative to the camera, or a frame of reference of the camera.

[0052] According to certain embodiments, the face detection of step E1 can be carried out using methods based on a deep neural network. Such methods are for example methods based on networks of the convolutional neural networks (CNN), Transformer, or support vector machines (SVM) type trained on labeled databases.

[0053] The following steps E2 and E3 can be performed simultaneously or sequentially depending on the embodiments. These steps can be performed from the contents of the bounding box(es) determined or selected.

[0054] The still or video image(s), or portions of still or video images, in which the faces are identified or possibly selected, following their identification, are analyzed, step E2, so as to estimate a morphological typing (or morphological type) of this face or of the person to whom this face belongs. By morphological typing, we can mean for example the shape or size of the face, the endomorphic, mesomorphic and ectomorphic traits, the age, the gender of the person.

[0055] Thus, depending on the morphological typing criteria that one wishes to estimate, step E2 may include one or more estimates. This or these estimates may, for example, determine one or more coefficients representative of the morphological typing. This or these coefficients obtained may be combined or used during the subsequent step E4.

[0056] So, for example, we may want to determine the age and / or gender of the person or face during this step.

[0057] For example, according to a first embodiment, the still or video image(s), or portions of still or video images, in which the faces are identified or possibly selected following their identification, can be analyzed, step E21, so as to estimate or determine an age of this face or of the person to whom this face belongs.

[0058] At the end of step E21, an integer representative of the estimated age of the person can be obtained. Alternatively, a range of values ​​can be obtained in which the age of the person whose face is identified is found.

[0059] According to certain embodiments, the age detection of step E21 can be carried out using methods based on a deep neural network. Such methods are for example methods based on networks of the convolutional neural network (CNN) type, or transform or making it possible to solve a regression problem. Such neural networks can receive as input for example the bounding box determined during step E1, or the selected face, and give as output an estimate of the age of the individual or an age range (for example an integer relating to an age range rather than an age). According to certain variants, the output of the neural network can also be a probability density and the age associated with the greatest probability can be retained as the estimated age.

[0060] For example, according to a second embodiment, the still or video image(s), or portions of still or video images, in which the faces are identified or possibly selected following their identification, can be analyzed, step E22, so as to estimate or determine a gender of this face or of the person to whom this face belongs. At the end of step E22, a variable representative of the gender of the person to whom the face belongs, whether male or female, can be obtained. For example, a “0” if it is a man, a “1” if it is a woman (or vice versa) and a “2” if the face is androgynous (non-gendered).

[0061] For example, according to another embodiment, during step E22, the gender of the person whose face has been detected is estimated or determined. At the end of step E22, we can obtain a variable representing the gender of the person to whom the face belongs, whether male, female, or androgynous. For example, a "0" if it is a man, a "1" if it is a woman (or vice versa) and a "2" if the face is androgynous (gender-neutral).

[0062] According to certain embodiments, the gender detection of step E22 can be carried out using methods based on a deep neural network. Such methods are for example methods based on networks of the convolutional neural network (CNN) type or for solving a regression problem. Such neural networks can receive as input for example the bounding box determined during step E1, or the selected face, and give as output an estimate of the gender of the individual. According to certain variants, the output of the neural network can also be a probability density and the gender associated with the greatest probability can be retained as the estimated age.

[0063] According to certain embodiments, not shown in Figure 1, step E2 could include other determinations, such as the shape of the face, long, thin, narrow, wide, etc., and thus provide varied morphological typings.

[0064] At the output of step E2, we obtain a morphological typing Typ.

[0065] In step E3, key points of a reference model (or face) (as defined later with respect to step E4) are determined or located on the detected face(s). The key points are characterized by their coordinates (ui, Vj) in pixels in the image. Here, key points located on a reference model are compared with key points on the detected or selected face(s). In other words, this involves positioning key points of a reference model on the same areas on the detected face. For example, the key points, or part of the key points, located on the eyes of the reference model are placed on the eyes of the detected face. It is thus possible to match a key point i of the reference model with a point] of the detected face.

[0066] According to some embodiments, the number of key points is between 5 and 100.

[0067] According to some embodiments, the minimum number of keypoints is 5. The minimum number of keypoints may for example be in a range between 5 and 15 according to the embodiments.

[0068] According to some embodiments, the optimal number of keypoints is 68. The optimal number of keypoints may for example be in a range between 55 and 75 according to the embodiments.

[0069] According to certain embodiments, one or more key points, preferably 5 key points, are distributed or located: - on the nose and / or, - on the mouth and / or - on the eyes, or eyebrows and / or, - on the contours of the face.

[0070] In some embodiments, the keypoints may represent all keypoints of a reference face model.

[0071] According to certain embodiments, the key points may represent a subset of the key points of a reference face model. This subset preferably comprises key points located on salient areas of the face. By salient area of ​​the face, one may in particular understand one or more areas comprising a prominent element when establishing a saliency map of the face. It may thus be estimated that areas having contrasts or different colors may constitute salient areas. Such areas may for example include the nose, the mouth, the eyes, the eyebrows, the cheekbones, the contours of the face. Such areas may indeed offer the advantage of being easily detectable automatically in a face (for example by artificial intelligence techniques)

[0072] According to some embodiments, the localization of the points of the reference model on the target face, performed during step E3, can be carried out using methods based on a deep neural network.

[0073] In some embodiments, the coordinates are two-dimensional coordinates, expressed in pixels. The two-dimensional reference frame or coordinate system having its origin in the top left corner of the image.

[0074] At the end of step E3, we obtain a set of key points of the detected face defined by their coordinates in pixels in a two-dimensional space, the axes of which are defined relative to a reference point defined in the image.

[0075] In a step E4, one or more facial reference models, MR, are adapted or modified from the morphological typing estimated in step E2. More generally, step E4 consists of obtaining an adjusted reference model, based on morphological typing. In other words, step E4 consists of adjusting a modeling of at least one reference face, on which a plurality of key points, of coordinates (xi, yi, Zj), are located, from an estimate of a morphological typing of the detected face;

[0076] According to a first embodiment, a single reference model is obtained and adapted to the morphological typing as determined during step E2 to obtain an adjusted reference model, also called adjusted reference face or modeling of the adjusted reference face. This adaptation may comprise a determination of one or more coefficients representative of the typing and a modification of the coordinates of the key points on the face by applying the coefficient(s), or a combination of these coefficients, to the coordinates of the key points of the reference face.

[0077] For example, the reference model can be adapted from the age estimated in step E21 and the gender estimated in step E22. As mentioned previously, the reference face is a face on which a set of keypoints is positioned. The location of the keypoints is determined on salient areas of the image, such as, for example, one or more of: - the nose, - the mouth - the eyes, and / or the eyebrows, - the contours of the face.

[0078] The coordinates of the key points of the reference face model are expressed in a length measurement system, in a 3-dimensional frame of reference. It is possible to estimate, from cohorts of faces, a reference model of a human face. On the reference model, we can for example arbitrarily place the origin of the 3-dimensional frame of reference by choosing the point with coordinates (0,0,0) at the center of the face, for example on the nose. We can thus deduce the position of the other points from this point, in a length measurement system, such as the metric system, expressed for example in meters or centimeters, or an imperial system, expressed in feet.

[0079] Coefficients relating to the age determined during step E21 and to the gender determined during step E22 are applied to this reference model.

[0080] Ka coefficients ge and kg enre can thus be applied to the coordinates of the key points of the reference model to obtain a new reference model adapted to the age and gender of the person whose face was identified during step E1. The coefficients correspond to a weighting applied to the coordinates of the reference model to adjust the reference model. According to certain embodiments, step E4 may comprise applying weighting coefficients to the coordinates of the key points or a portion of the key points of the reference model (or of a modeling of the reference face), to correspond to the morphological typing or to adapt a selected reference model to a morphological typing.

[0081] According to certain embodiments, the coefficient Kg enr e can represent a ratio between the size of a man's face and that of a woman.

[0082] Since the reference model can be a reference model corresponding to a man's face, a woman's face or an androgynous face (i.e. "non-gendered"), we can for example use the following values ​​for the coefficient Kg enr e : - if the reference model corresponds to a female face and the detected gender of the target face is female, then the coefficient Kg enr e is equal to 1, - if the reference model corresponds to a female face and the detected gender of the target face is male, then the coefficient Kg enr e is equal to 1.025, - if the reference model corresponds to a female face and the detected gender of the target face is androgynous, then the coefficient Kg enr e is equal to 1.0125, - if the reference model corresponds to a male face and the detected gender of the target face is female, then the coefficient Kg enr e is equal to 0.975, - if the reference model corresponds to a male face and the detected gender of the target face is male, then the coefficient Kg enr e is equal to 1, - if the reference model corresponds to a male face and the detected gender of the target face is androgynous, then the coefficient Kg enr e is equal to 0.9875, - if the reference model corresponds to an androgynous face, and the detected gender of the target face is female, then the coefficient Kg enr e is equal to 0.9875. - if the reference model corresponds to an androgynous face, and the detected gender of the target face is male, then the coefficient Kg enr e is equal to 1.0125. - if the reference model corresponds to an androgynous face, and the detected gender of the target face is androgynous, then the coefficient Kg enr e is equal to 1.

[0083] These values ​​can be summarized in the table below:

[0084] These values ​​are values ​​from physiological statistics studies and constitute a simple example of values ​​that can be used. Of course, other values ​​can be used depending on the implementation methods. For example, the coefficient Kâ ge can be determined according to the following formula:

[0086] With "age" representing the age of the person whose face was detected and obtained during step E21. This formula can be obtained, for example, by regression of a curve expressing the evolution of head size as a function of age. It is derived from charts of physiological observations of the size of an individual's skull as a function of their age. Of course, other values ​​can be used depending on the implementation methods.

[0087] The application of the Kâ coefficients ge and K genre to the reference model can consist of multiplying the coordinates of the points of the reference face by the product (Kâ ge * Kgenre).

[0088] As mentioned previously with regard to step E2, when this comprises the determination of the shape or a type of shape of the face, for example, face, thin, long, wide, narrow, a coefficient relating to this typing can be determined. When the face is narrow, a coefficient can be applied to the coordinates of the reference face, the effect of which makes it possible to bring the key points closer together, in other words to reduce the distance between the key points. In the same way, if the typology of the face corresponds to a wide face, a coefficient can be applied to the coordinates of the reference face, the effect of which makes it possible to move the key points apart, in other words to increase the distance between the key points.

[0089] In other words, step E4 may comprise the application of weighting coefficients to the coordinates of the key points positioned on a reference model in order to obtain a model of a reference face adjusted to the morphological typing of the detected face. The reference model may have been previously selected from a set of reference models.

[0090] According to certain embodiments, several reference models may be envisaged. For example, but not limited to, one may envisage a model by gender, or a model by age or age group, or a model by facial morphology, such as for example a long, wide, thin face, or a model combining at least some of these characteristics. A reference model may for example be predefined by morphological typing. The morphological typing obtained during step E2 may then make it possible to select one from among several reference models or to match a reference model to a morphological typing. In such an embodiment, step E4 may comprise a selection of the reference model corresponding to the determined morphological typing.

[0091] According to certain embodiments, it may be envisaged to combine the two approaches previously described for adjusting a model of the reference face of step E4. In other words, step E4 may comprise: - the selection of a reference face from a plurality of reference faces, the selection taking into account at least one characteristic of a morphological typing, - the adjustment of the selected reference face based on one or more characteristics of the morphological typing not taken into account when selecting the reference face.

[0092] According to a first example, when the determination of the morphological typing, step E2, estimates the age and gender of the detected face, a reference model relating to gender can be selected and a coefficient as mentioned above can be applied to adjust this reference model relating to gender, to the estimated age. Conversely, in this example, an age reference model could be selected and a coefficient as mentioned above can be applied to adjust this reference model relating to age, to the estimated gender.

[0093] According to a second example, when the determination of the morphological typing, step E2, estimates the age and the shape of the detected face, an age-related reference model can be selected and a coefficient as mentioned above can be applied to adjust this age-related reference model to the estimated shape. Conversely, in this example, a shape-related reference model (wide, narrow, thin...) could be selected and a coefficient as mentioned above can be applied to adjust this shape-related reference model to the estimated age.

[0094] Figure 2 illustrates an example of adapting a reference model corresponding to an adult male to that of an 8-year-old boy. This produces a reference model adapted to morphological typing and, in this example, adapted to age and gender.

[0095] The coordinates shown in Figure 2 are coordinates expressed in a unit of the metric system (for example in meters or millimeters), from an axis centered on the face.

[0096] At the end of step E4, we thus obtain an adjusted reference model, MRA, to the morphological typing and in this example, adapted to the gender and age of the target face.

[0097] In step E5, the coordinates of the key points of the face are determined, for example in a three-dimensional length measurement system, from - of the adjusted reference model, - the focal length of the camera, and - two-dimensional pixel coordinates, keypoints positioned on the target face.

[0098] More precisely, a location and therefore a distance are obtained, in a length measurement system, of the detected face(s) relative to the at least one camera. The location is obtained from a focal length of the camera, the pixel coordinates of at least a second subset of the key points located on the detected face(s), the coordinates in the length measurement system of the key points located on the adjusted modeled reference face.

[0099] It can also be noted that the direction of gaze can be obtained simultaneously, for example in the form of a vector expressing the direction of gaze, which can be expressed by the rotation of the face.

[0100] For a key point of index i, we can note: - (ui, Vj,) its pixel coordinates in the captured image of the target face, - (xi, yi, Zi) its coordinates in a length measurement system in the fitted model.

[0101] The relationship between these quantities can be expressed as follows:

[0103] With - f the focal length in pixels of the camera, - w and h representing respectively the horizontal and vertical resolution in pixels of the captured image, - x, y and z are the coordinates in the length measurement system, in a frame of the camera expressing the position of the face in said frame, and the coefficients n,j correspond to a rotation matrix expressing the rotation of the face relative to the axis of the camera.

[0104] The present disclosure aims in particular to determine the location of the detected face in the three-dimensional reference frame, by determining the coordinates (x,y,z) making it possible to locate the face, as well as to determine the coefficients n,j making it possible to obtain the direction of gaze. One of the advantages of the present invention is to help locate the face and determine the direction of gaze from the minimization of the same error function and can therefore help to simplify the calculations and reduce the calculation time necessary to obtain these quantities.

[0105] The previous equation can be solved, i.e. the coordinates (x,y,z) as well as the coefficients n,j can be obtained by minimizing the following error function:

[0107] According to certain embodiments, this function can be minimized using a RANSAC (RANdom SAmple Consensus) method. This method advantageously makes it possible to consider only a subset of key points of the adjusted reference model. Certain associations between (ui,Vj) and points of the adjusted model (Xj, y j,Zj) can be very imprecise and can therefore be discarded by the RANSAC method.

[0108] It can also allow not to take into account very specific key points on the face, for example expression lines, non-standard faces, presenting for example certain disproportions on certain organs, linked for example to one or more malformations of the target face. As such, it takes into account a second subset of key points, the second subset being obtained from the set of key points in which the key points i whose coordinates (ui,Vj) on the detected face and the coordinates (xi, yi, zi) on the adjusted modeled reference face are too far apart, are discarded.

[0109] In some embodiments, this function may be minimized using a least median method.

[0110] The coordinates (x, y, z) representing the location of the detected face relative to the camera can therefore be obtained when minimizing this function. The coordinates are expressed in the distance measurement system of the reference model coordinates, not in pixels. Thus, the coordinates (x, y, z) are also expressed in the length measurement system. The distance between the face and the camera is then obtained by calculating the Euclidean distance between the camera and the point with coordinates (x,y,z).

[0111] We can also, from the minimization of the same error function, obtain the direction of a vector expressing the direction of gaze of the target face in the form of the following vector:

[0113] According to some embodiments, the length measurement system may be the metric system in which distances may be expressed in meters.

[0114] In some embodiments, the length measurement system may be the imperial system in which distances may be expressed in feet.

[0115] Other examples of length measurement systems can be used. We can understand that one of the goals is to measure (or more precisely estimate) real distances between the camera and the face and therefore not to determine a distance in the image frame of reference which could be expressed in pixels.

[0116] Figure 4 represents a device configured to implement at least one of the embodiments of the invention. The device 10 comprises in particular a processor 1, a random access memory 2, a read-only memory 3 and a non-volatile memory 4. They may further comprise communication means 5, and / or a user interface 6. More generally, the device 10 has the hardware architecture of a computer.

[0117] The read-only memory 3 constitutes a recording medium in accordance with at least one embodiment of the present disclosure, readable by the processor 1 and on which is recorded a computer program PROG in accordance with at least one embodiment of the present disclosure, comprising instructions for executing steps of the method according to at least one embodiment of the present disclosure. The program PROG defines functional modules of the device.

[0118] The communication means 5 allow in particular the device 10 to exchange data with capture means, such as a camera configured to capture a scene and in particular one or more people present in this scene in accordance with the present disclosure or one of its embodiments. For this purpose, the communication means 5 may for example comprise a computer data bus capable of transmitting data. Alternatively, said data can be transmitted via a communication interface, wired or wireless, capable of implementing any suitable protocol known to those skilled in the art.

Claims

Claims

1. A determination method comprising - a location (E3) of key points on at least one face detected in an image captured by a camera, said key points corresponding to key points of a modeling of at least one reference face (MR) and, - obtaining (E5) a location (x,y,z) relative to the camera, in a real-world frame of reference, of said at least one detected face, from a focal distance (f) of said camera, the coordinates (ui,Vj) of at least one of said key points located on said at least one face detected in said image, the coordinates (xi,yi,Zj) in said frame of reference of the key points located on said modeling.

2. A method according to claim 1 comprising - obtaining (E5) an orientation of said at least one detected face from the focal distance (f) of said camera, the coordinates (ui, ) in pixels of at least one of said key points located on said at least one detected face, the coordinates (xi, yi, Zj) in said length measurement system of the key points located on said modeling.

3. Method according to one of the preceding claims in which obtaining (E5) a location and obtaining (E5) an orientation are obtained by minimizing the same error function.

4. Method according to at least one of the preceding claims wherein the method comprises: - an adjustment (E4) of said modeling of at least one reference face (MR), from an estimation (E2) of a morphology of said detected face.

5. Method according to at least one of the preceding claims in which the adjustment (E4) of said modeling of at least one reference face (MR) comprises a selection of a modeling of a reference face (MR) corresponding to said morphology.

6. Method according to claim 4 or claim 5 in which the adjustment (E4) of a modeling of at least one face (MR) comprises: - a selection of a reference face (MR) from a plurality of reference faces, the selection taking into account at least one characteristic of said morphology, - an adjustment of the reference face (MR) selected according to one or several characteristics of said morphology not having been taken into account during the selection of the reference face (MR).

7. Method according to at least one of claims 4 to 6 in which the adjustment (E4) of a modeling of at least one reference face (MR) comprises an adjustment of the coordinates of the key points of said reference face (MR) as a function of said morphology (typ).

8. Method according to at least one of claims 4 to 7 in which the adjustment of a model of a reference face (MR) comprises: - obtaining the coordinates of said key points on said modeling of the reference face by multiplying the coordinates of said key points on said reference face (MR) by a coefficient (Kâ ge ) relating to an estimated age of said at least one detected face and by a coefficient (Kg enr e) relating to an estimated gender of said at least one detected face.

9. Method according to at least one of the preceding claims in which said key points of said at least one detected face are located on salient points of said at least one detected face, at least some of which are located on one or the other or more of the eyes, the mouth, the nose, or the contours of said at least one detected face.

10. Method according to one of the preceding claims in which said reference system is a three-dimensional length measurement system, the origin of which is centered on a position of the camera.

11. Device for determining information on quantities relating to a face captured by at least one camera, the device comprising one or more processors configured together or separately to: - locating (E3) key points on at least one face detected in an image captured by a camera, said key points corresponding to key points of a modeling of at least one reference face (MR) and, - obtaining (E5) a location (x,y,z) relative to the camera, in a real-world frame of reference, of said at least one detected face, from a focal distance (f) of said camera, the coordinates (ui,Vj) of at least one of said key points located on said at least one face detected in said image, the coordinates (xi,yi,Zj) in said frame of reference of the key points located on said modeling.

12. Device according to claim 11 wherein said device comprises means for detecting said at least one face in said at least one image.

13. A device according to claim 11 or 12 wherein said camera is a camera of said device.

14. Computer program comprising instructions for executing the method according to at least one of claims 1 to 10, when said program is executed by a computer.

15. A computer-readable recording medium on which is recorded a computer program comprising instructions for executing the method according to at least one of claims 1 to 10.

Images