Method for defining a three-dimensional model of the face of a motor vehicle user.

The method employs two cameras in a motor vehicle to capture and fuse images of a driver's face, creating a reliable three-dimensional model for generating an accurate avatar, addressing the limitations of existing technologies.

FR3157628A1Active Publication Date: 2025-06-27AMPERE SAS
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
FR2023015036
Authority / Receiving Office
FR · FR
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-12-22
Publication Date
2025-06-27
Estimated Expiration
2043-12-22

AI Technical Summary

Technical Problem

Current methods for creating a three-dimensional model of a driver's face in a motor vehicle are not simple or reliable, lacking an efficient way to generate an avatar representing the driver's face.

Method used

A method utilizing two cameras placed at distinct positions in the passenger compartment of a motor vehicle, capturing simultaneous two-dimensional images of the driver's face, and using image fusion algorithms to determine the three-dimensional position of facial points, thereby creating a reliable three-dimensional model.

Benefits of technology

This method allows for the simple and reliable creation of a three-dimensional model of a driver's face, enabling the generation of an accurate avatar that can be used for various applications, such as driver monitoring and communication.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

Method for defining a three-dimensional model of a face of a driver of a motor vehicle. Method for defining a three-dimensional model of a face of a driver of a motor vehicle equipped with a first and a second camera placed at distinct positions in the passenger compartment of the motor vehicle, and capable of simultaneously capturing images of a driver's face, the method comprising an iteration on the following steps: a step of capturing a first two-dimensional image of the driver's face by the first camera and a second two-dimensional image of the driver's face by the second camera, a step of associating a first pixel of the first image and a second pixel of the second image corresponding to the same given point of the driver's face, a step of determining a three-dimensional position of the given point of the driver's face. Figure for the abstract: 3
Need to check novelty before this filing date? Find Prior Art

Description

Title of the invention: Method for defining a three-dimensional model of the face of a user of a motor vehicle.

[0001] The invention relates to a method for defining a three-dimensional model of a face of a user of a motor vehicle. The invention also relates to a device for defining a three-dimensional model of a face of a user of a motor vehicle. The invention also relates to a computer program implementing the mentioned method. The invention finally relates to a recording medium on which such a program is recorded.

[0002] In the automotive industry, new features for welcoming and supporting a driver are being developed. In particular, such features can be based on the staging of an avatar.

[0003] The aim of the invention is to provide a method for defining a three-dimensional model of a face of a user of a motor vehicle which is simple and reliable and which makes it possible to create an avatar representing a face of the driver.

[0004] To this end, the invention relates to a method for defining a three-dimensional model of a face of a user of a motor vehicle equipped with a first and a second camera placed at distinct positions in the passenger compartment of the motor vehicle, and capable of simultaneously capturing images of a face of a driver of the motor vehicle, the method comprising • a step of determining a position, relative to a fixed reference point of the motor vehicle, of a center of a lens of the first camera and of a center of a lens of the second camera, then, an iteration on the following steps: • a step of capturing, at the same given instant, a first two-dimensional image of the driver's face by the first camera and a second two-dimensional image of the driver's face by the second camera, the first image being decomposed into a first grid of pixels and the second image being decomposed into a second grid of pixels, • an association step, between a first pixel of the first grid and a second pixel of the second grid corresponding to the same given point of the driver's face, • a step of determining a three-dimensional position of the given point of the driver's face as a function of the respective positions of a center of a lens of the first camera and a center of a lens of the second camera.

[0005] In one embodiment, the centers of the respective lenses of the first and second cameras are close to each other, in particular they are separated by a distance of less than 100 millimeters.

[0006] In one embodiment, the associating step comprises implementing an image fusion algorithm.

[0007] In one embodiment, the step of determining a three-dimensional position of the given point comprises calculating a point of intersection between • a first straight line passing through the first pixel and the center of the lens of the first camera, and • a second line passing through the second pixel and the center of the lens of the second camera.

[0008] In one embodiment, a first color is associated with the first pixel and a second color is associated with the second pixel, and the color of the given point is determined to be an average between the first and second colors.

[0009] In one embodiment, an angle between a sagittal plane of a driver's head and a longitudinal axis of the vehicle is calculated by an algorithm for monitoring driver vigilance, and the association step comprises taking into account said angle between the sagittal plane and the longitudinal axis.

[0010] In one embodiment, the vehicle is equipped with at least one ultrasonic sensor making it possible to measure a distance between the ultrasonic sensor and a point on the face closest to the ultrasonic sensor, and the association step takes this distance into account.

[0011] In one embodiment, the determining step comprises: • a creation, in particular by simulation, of a database comprising a plurality of associations between, on the one hand, a given image of a headrest of a driver's seat of the motor vehicle and, on the other hand, a position of a center of a lens and an orientation of a camera having captured the given image, and optionally an angle measured between an axis of said camera and a longitudinal axis of the motor vehicle, • a capture by the first and second cameras of the motor vehicle, of a first and a second image of the headrest of the driver's seat, • a comparison respectively of the first image and the second image of the headrest with images of the plurality of associations recorded in the database to respectively determine the position, relative to a fixed reference point, of a center of the lens of the first camera and of a center of the lens of the second camera.

[0012] The invention also relates to a device for defining a three-dimensional model of a face of a user of a motor vehicle equipped with a first and of a second camera placed at distinct positions in the passenger compartment of the motor vehicle, the device comprising hardware and / or software elements implementing the method according to the invention, in particular hardware and / or software elements designed to implement the method according to the invention.

[0013] The invention also relates to a computer program product comprising program code instructions recorded on a computer-readable medium for implementing the steps of the method according to the invention when said program operates on a computer.

[0014] The invention also relates to a computer-readable data recording medium on which is recorded a computer program comprising program code instructions for implementing the above-mentioned method.

[0015] The attached drawing represents, by way of example, an embodiment of a device for defining a three-dimensional model of a user's face according to the invention and an embodiment of a method for defining a three-dimensional model of a user's face according to the invention.

[0016] [Fig.l] represents a motor vehicle equipped with a device for defining a three-dimensional model of a user's face.

[0017] [Fig.2] defines a direct orthonormal reference frame of the motor vehicle according to the invention.

[0018] [Fig. 3] defines a first and a second direct orthonormal reference frame respectively associated with a first and a second camera of a definition device according to the invention.

[0019] [Fig.4] is a flowchart of an embodiment of a definition method according to the invention.

[0020] [Fig.5] is a first illustration of a step of determining a position, relative to the direct orthonormal reference frame of the motor vehicle, of a center of a lens of a first camera and of a second camera of the definition device.

[0021] [Fig.6] is a second illustration of a step of determining a position, relative to the direct orthonormal reference frame of the motor vehicle, of a center of a lens of a first camera and of a second camera of the definition device.

[0022] [Fig.7] is a view of a passenger compartment of the motor vehicle equipped with the invention.

[0023] [Fig.8] is a first image captured by the first camera at a time given.

[0024] [Fig.9] is a second image captured by the second camera at the given time.

[0025] [Fig. 10] is a first illustration of a step of association between a first pixel of the first image and a second pixel of the second image.

[0026] [Fig. 11] is a second illustration of a step of association between a first pixel from the first image and a second pixel from the second image.

[0027] An example of a motor vehicle 100 equipped with a device 10, according to the invention, for defining a three-dimensional model of a face of a user of the vehicle is described below with reference to [Fig. 1].

[0028] The motor vehicle 100 may be a vehicle of any type, for example a passenger vehicle or a utility vehicle.

[0029] The motor vehicle 100 mainly comprises: - a passenger compartment 1 defining the interior space of the motor vehicle 100, - a first and a second camera 21, 22 arranged in the passenger compartment 1, the first and second cameras 21, 22 forming part of the device 10.

[0030] The two cameras are part of a set of sensors 2 of the motor vehicle 100. The set of sensors 2 may further comprise an ultrasonic sensor 24.

[0031] The two cameras 21, 22 have identical optical characteristics. For example, each of the first and second cameras has the following characteristics: - a focal length of the camera is equal to 2 mm, - a width of a photographic sensor of the camera is 1.9 mm, - a height of the photographic sensor of the camera is equal to 1.5 mm, - the horizontal and vertical resolutions are defined, in particular according to a horizontal resolution of 1360 pixels and a vertical resolution of 1180 pixels.

[0032] With reference to [Fig.2], a first direct orthonormal reference frame R0 (X0, Y0, Z0) is defined, represented by [Fig.2], the axis X0 being parallel to the longitudinal axis of the motor vehicle 100 and oriented towards the rear of the motor vehicle 100, the axis Y0 being transverse and directed towards the right of the vehicle, and the axis Z0 being vertical and oriented towards the top of the vehicle.

[0033] With reference to [Fig.3], a second and a third direct orthonormal reference frame R21 and R22 are further defined, respectively associated with the first and second cameras.

[0034] The center of the reference frame R21 is determined by a position 2111, defined in the reference frame R0, of a center 211 of a lens of the first camera 21. In the embodiment illustrated by [Fig.3], a first axis X21 of the reference frame R21 is located in a plane parallel to the axes X0 and Z0 of the reference frame R0. The first axis X21 forms an angle of inclination 212 with the axis X0 of the reference frame R0. A second axis Y21 of the reference frame R21 is substantially directed along the axis Y0 of the reference frame R0. A third axis Z21 is perpendicular to the axes X21 and Y21.

[0035] Similarly, the center of the reference frame R22 is determined by a position 2211, defined in the reference frame R0, of a center 221 of a lens of the second camera 22. In the embodiment illustrated by [Fig.3], a first axis X22 of the reference frame R22 is located in a plane parallel to the axes X0 and Z0 of the reference frame R0. The first axis X22 forms an angle of inclination 222 with the axis XO of the reference RO. A second axis Y22 of the reference R22 is substantially directed along the axis YO of the reference RO. A third axis Z22 is perpendicular to the axes X22 and Y22.

[0036] In one embodiment, the two cameras 21, 22 are located close to each other. For example - the center 2111 of the lens 211 of the first camera 21 and the center 2211 of the lens 221 of the second camera 22 are located on the same straight line perpendicular to a plane (XO, Z0), and - a distance 23, measured along the axis YO of the reference RO, between the center 211 of the first camera 21 and the center 221 of the second camera 22, is substantially equal to 100 millimeters.

[0037] Advantageously, the distance 23 and the values ​​of the inclination angles 212 and 222 are defined so that a field 214, 224 of each camera 21, 22 is capable of containing a face 51 of a driver seated on a driver's seat 11 of the motor vehicle 100.

[0038] The cameras 21, 22 are used to create stereovision images, that is to say the two-dimensional images, coming from each of the cameras 21, 22, are used together to construct three-dimensional images, as explained in the remainder of the document.

[0039] In the remainder of the document, an orientation of the first camera 21 is defined by the angles measured respectively between the axes X21, Y21, Z21 of the reference frame R21 and the axes X0, Y0, Z0 of the reference frame R0. Similarly, an orientation of the second camera 22 is defined by the angles measured respectively between the axes X22, Y22, Z22 of the reference frame R22 and the axes X0, Y0, Z0 of the reference frame R0.

[0040] In the embodiment illustrated by figures 3 to 11, the cameras 21, 22 are fixed on a dashboard 12 of the passenger compartment 1, in particular in an area 121 located behind a steering wheel 13. Other positions of the cameras 21, 22 are conceivable, for example on a side pillar 14 of the passenger compartment 1.

[0041] In an advantageous embodiment, the motor vehicle 100 is furthermore equipped with a system 3 for monitoring the driver's vigilance and the system 3 comprises means for determining a current angle 52 measured between a sagittal plane 53 of a driver's head and the longitudinal axis X0 of the motor vehicle 100.

[0042] In one embodiment, the device 10 comprises simulation means 300. The simulation means 300 make it possible to construct a database 301. Advantageously, the simulation data are specific to a given vehicle model. The data recorded in the database 301 comprise a plurality of associations between, on the one hand, a given image of a headrest of a seat driver of the motor vehicle and, on the other hand, a position of a center of a lens of a camera having captured the given image.

[0043] The device 10 further comprises a microprocessor 4. The device 10, and particularly the microprocessor 4 enabling the implementation of a method for defining a three-dimensional model of a face of a user of a motor vehicle, mainly comprises the following modules: - a module 41 for determining the position 2111, 2211, relative to a fixed reference RO of the motor vehicle 100, of the center 211 of the lens of the first camera 21 and of the center 221 of the lens of the second camera 22, the module 40 collaborating with the database 301, - a module 42 for capturing, at the same given instant T_CAPT, a first two-dimensional image IMG1 of the driver's face by the first camera 21 and a second two-dimensional image IMG2 of the driver's face by the second camera 22, the module 41 collaborating with the first and second cameras 21, 22 - an association module 43, between a first pixel PI 1 and a second pixel P12 corresponding to the same given point PI of the driver's face, the first pixel Pli belonging to a first grid of pixels G1 representing the first image IMG1, and the second pixel P12 belonging to a second grid of pixels G2 representing the second image IMG2, - a module 44 for determining a three-dimensional position 3D_pos of the given point PI of the driver's face as a function of the respective positions 2111, 2212 of a center of a lens 211 of the first camera 21 and of a center 221 of a lens of the second camera 22.

[0044] The motor vehicle 100, in particular the definition device 10, preferably comprises all the hardware and / or software elements configured so as to implement the method defined in the subject of the invention or the method described below.

[0045] An embodiment of the parking method is described below with reference to Figures 4 to 11. In the embodiment represented by Figures 4 to 11, the method comprises four steps E1 to E4.

[0046] In the initial step E1, a position 2111, 2211 is determined, relative to the fixed reference point R0 of the motor vehicle 100, of a center 211 of a lens of the first camera 21 and of a center 221 of a lens of the second camera 22.

[0047] Indeed, during the development of a new vehicle, the position of the cameras is studied in simulation, which makes it possible to define ideal values ​​of the coordinates and orientations of each camera.

[0048] When real cameras are mounted on a vehicle in the factory, the actual positions and orientations differ by a few millimeters and / or degrees from the ideal data. However, to ensure the accuracy of the model created by the cameras, measuring the true position and orientation of the cameras is necessary.

[0049] In an advantageous embodiment, the two cameras are delivered in a single piece. As a result, a measured distance between the two respective centers of the camera lenses is fixed and known.

[0050] In order to determine the actual positions and orientation of the two cameras 21, 22, step E1 comprises: • a first sub-step El 1 of creation, in particular by simulation, of the database 301 comprising a plurality of associations between, on the one hand, a given image of a headrest 111 of a driver's seat 11 of the motor vehicle and, on the other hand, a position of a center of an objective and an orientation of a camera having captured the given image, and optionally an orientation of said camera, • a second sub-step E12 of capturing by the first and second cameras of the motor vehicle a first and a second image ATI, AT2 of the headrest 111 of the driver's seat, • a third sub-step El3 of comparing respectively the first image ATI and the second image AT2 of the headrest 111 with the images recorded in the database 301 to determine respectively the position of the center 2111 of the lens 211 of the first camera 21 and optionally its orientation, and the position of the center 2211 of the lens 221 of the second camera 22 and optionally its orientation.

[0051] In the first sub-step El 1, a multitude of image captures of a driver's headrest 11 are computer-simulated, by varying the position of a center of a lens and the orientation of a simulated camera capturing the images.

[0052] The data from the simulation are stored in a format allowing them to be used later by querying the database.

[0053] For example, the storage format may be a data pair associating - on the one hand, an image file, and - on the other hand (i) the three-dimensional coordinates of a point corresponding to the center of the lens of the simulated camera that produced the image file, and (ii) the orientation, relative to the R0 reference frame, of the simulated camera which produced the image file.

[0054] Thus, the simulation consists of varying a sextuple of variable parameters relating to the position and orientation of the simulated camera and simulating an image capture by the simulated camera.

[0055] Variable parameters include: - a first coordinate X21, measured along the axis X0, of a center of a lens of the camera 21, the first coordinate being able to vary between a minimum threshold xxl and a maximum threshold xx2, - a second coordinate Y21, measured along the axis YO, of the center of the lens of the camera 21, the second coordinate being able to vary between a minimum threshold yyl and a maximum threshold yy2, - a third coordinate Z21, measured along the ZO axis, of the center of the lens of the camera 21, the third coordinate being able to vary between a minimum threshold zzl and a maximum threshold zz2, - a first angle A21 measured between an axis of the camera 21 and the longitudinal axis XO of the motor vehicle, the first angle being able to vary between a minimum threshold aal and a maximum threshold aa2, - a second angle B21 measured between the axis of the camera 21 and the transverse axis YO of the motor vehicle, the second angle being able to vary between a minimum threshold bbl and a maximum threshold bb2 - a third angle C21 measured between the axis of the camera 21 and the vertical axis ZO of the motor vehicle, the second angle being able to vary between a minimum threshold ccl and a maximum threshold cc2.

[0056] The thresholds xxl, yyl, zzl, xx2, yy2, zz2 can be expressed in millimeters, and the thresholds aal, bbl, ccl, aa2, bb2, cc2 can be expressed in degrees.

[0057] The minimum thresholds xxl, yyl, zzl, aal, bbl, ccl are advantageously negative values. The maximum thresholds xx2, yy2, zz2, aa2, bb2, cc2 are advantageously positive values.

[0058] For each value of the sextuple of coordinates (X21, Y21, Z21, A21, B21, C21), an image capture by the first camera is simulated, consisting of generating an image which would be captured by a camera whose position and orientation is defined by said sextuple of coordinates.

[0059] In the described embodiment, the image files contain an image of a headrest 111 of the driver's seat 11. Alternatively, the image files could contain another given element of the passenger compartment whose position and geometry are known at the time of calibration of the motor vehicle. For example, any fixed element of the passenger compartment can be used for calibration purposes as soon as it appears in the central zone of the field of vision of the cameras.

[0060] Whatever the given element on which the simulation step relates, in the database, the image file can be supplemented or replaced by a set of positions of characteristic points of the given element.

[0061] An example of characteristic points PI, P2, P3 of the given element (here a headrest of the driver's seat) is illustrated in Figures 5 and 6. Depending on the simulated position of the center of the camera lens, the position of these points in the captured image varies. For example, in [Fig.5], the point PI is located at a first position Pl_posl of the ATI image, and in [Fig.6], the point PI is located at a second position Pl_pos2 of the AT_2 image.

[0062] Then we move on to the second sub-step E12, illustrated by figures 5 and 6, in which we capture - with the first camera 21 of the motor vehicle 100, a first ATI image of the headrest 111 of the driver's seat, and - with the second camera 22 of the motor vehicle 100, a second image AT2 of the headrest 111 of the driver's seat.

[0063] Sub-step E12 advantageously takes place during the calibration of the motor vehicle 100 in the factory.

[0064] Then in sub-step E13, each of the images ATI, AT2 is compared to the images recorded in the database 301.

[0065] The image comparison can relate to the position of a set of characteristic points PI, P2, P3.

[0066] The data recorded in the database 301 are recorded in a format which makes it possible to determine - from an ATI, AT2 shot of the passenger compartment 1 -, - a position 2111, 2211 of a camera 21, 22 having captured the shot, that is to say a position of a center 2111, 2211 of a lens 211, 221 of said camera 21, 22, and optionally - an orientation of cameras 21, 22.

[0067] Then we move on to capture step E2, illustrated by figures 7 to 9, - [Fig.7] representing a driver of the motor vehicle 100 whose face is substantially oriented towards the front of the vehicle, - [Fig.8] is a first image IMG1 captured by the first camera 21 at a time T_CAPT, - [Fig.9] is a second image IMG2 captured by the second camera 22 at time T_CAPT.

[0068] In step E2, a first two-dimensional image IMG1 of the driver's face is captured at the same given instant T_CAPT by the first camera 21 and a second two-dimensional image IMG1 of the driver's face by the second camera 22, the first image IMG1 being decomposed into a first grid G1 of pixels and the second image IMG2 being decomposed into a second grid G2 of pixels.

[0069] Then we continue with the association step E3, between a first pixel PI 1 of the first grid G1 and a second pixel P12 of the second grid G2 corresponding to the same given point PI of the driver's face. The association step is illustrated by figures 10 and 11.

[0070] For this purpose, the images IMG1, IMG2 are processed by an image fusion algorithm, in particular an artificial intelligence algorithm, which determines a set of given points P_l, P_2, P_3 of the driver's face, each given point P_l, P_2, P_3 being represented by a first pixel P11, P21, P31 of the first image IMG1, and a second pixel P12, P22, P32 of the second image IMG2.

[0071] The term “image fusion algorithm” designates a computer algorithm capable of creating an image from a combination of several images of different origins, in particular images from several sensors.

[0072] From the set of given points P_l, P_2.. ,P_i of the driver's face, we generate: - a first table Table 1 of pixels from the first image IMG1, and - a second table Table 2 of pixels from the second image IMG2.

[0073] An example of tables Table 1, Table 2 are provided below:

[0074] [Tables 1] Driver's face point Pixel of the first image IMG1 Coordinate in pixels according to Y21 Coordinate in pixels according to Z21 P_1 Fold 182 32 P_2 P12 197 32 P_3 P13 201 32 P_4 P14 209 32 P_5 P15 280 32 P_6 P16 284 32 P_7 P17 286 33

[0075] [Tables2] Driver's face point Pixel of the second image IMG2 Coordinate in pixels along Y22 Coordinate in pixels along Z22 P_1 P21 239 32 P_2 P22 247 32 P_3 P23 250 32 P_4 P24 253 32 P_5 P25 259 32 P_6 P26 271 32 P_7 P27 277 32

[0076] A first line of the table Table 1 contains the coordinates of a pixel representing the first point P_1 on the first image IMG1.

[0077] An ith line of the table Table 1 contains the coordinates of a pixel representing the point P_i on the first image IMG1.

[0078] The first column of a given line of the table Table 1 contains a coordinate of the given pixel in a first direction, in particular in the direction Y21 of the reference frame R21 associated with the first camera 21. In addition, the second column of the given line contains a coordinate of the given pixel in a second direction, in particular in the direction Z21 of the reference frame R21 associated with the first camera 21.

[0079] Similarly, a first line of the table Table 2 contains the coordinates of a pixel representing the first point PI on the second image IMG2.

[0080] An ith line of the table Table 2 contains the coordinates of a pixel representing the point P_i on the second image IMG2.

[0081] The first column of a given line of the table Table 2 contains a coordinate of the given pixel in a first direction, in particular in the direction Y22 of the reference frame R22 associated with the second camera 22. In addition, the second column of the given line contains a coordinate of the given pixel in a second direction, in particular in the direction Z22 of the reference frame R22 associated with the first camera 22.

[0082] For example, pixel Pli (182, 32) and pixel P21 (239, 32) correspond to the same point of the conductive face P_1 in three dimensions.

[0083] Then we continue with step E4 of determining a three-dimensional position 3D_pos_i of each given point P_i of the driver's face.

[0084] To do this, we refer to the ith line of table 1, which provides the coordinates of the pixel Pli of the image IMG1 corresponding to the given point P_i, and we determine a first straight line Dl_i passing through a center 211 of the lens of the first camera 21 and the pixel Pli.

[0085] Similarly, we refer to the ith line of table 2, which provides the coordinates of the pixel P2i of the image IMG2 corresponding to the given point P_i, and we determine a first straight line D2_i passing through a center 221 of the lens of the second camera 22 and the pixel P2i.

[0086] The 3D_pos_i coordinates of the given point P_i are then defined as being a point of intersection between the first and second lines Dl_i, D2_i.

[0087] In addition, a color associated with the given point P_i is determined based on a first color associated with the pixel Pli of the image IMG1 and a second color associated with the pixel P2i of the image IMG2. In one embodiment, the color of the given point P_i is the average between the first and second colors. In one embodiment, the colors are expressed according to an RGB code, for red-green-blue.

[0088] Advantageously, the angle 52 measured between a sagittal plane 53 of the driver's head and the longitudinal axis X0 of the vehicle is data provided by the driver's vigilance monitoring system 3. Knowledge of the angle 52 makes it possible to use images captured by the cameras 21, 22 while the driver turns his face to the right or left.

[0089] In one embodiment, the motor vehicle 100 is equipped with at least one ultrasonic sensor 24 making it possible to measure a given distance between the sensor 24 and a point on the driver's face which is closest to the sensor 24. Advantageously, several detectors of the ultrasonic sensor type or a radar are used. The detectors make it possible to measure a distance along the X0 axis (distance also called depth) between the face and the sensors. Thus, a single camera is sufficient to constitute a point cloud.

[0090] A three-dimensional point cloud P_i modeling the driver's face is thus obtained. From this point cloud, surface reconstruction software can reconstruct the surface of the driver's face.

[0091] The three-dimensional face thus obtained can be used as a digital representation of the driver, i.e. an avatar of the driver, in tools intended to communicate with the driver.

[0092] The avatar can be used in particular in animations intended to transmit information to the driver, for example to communicate indicators to him about his driving style. The user of an avatar resembling the driver is intended to capture the driver's attention more effectively and to encourage his adherence to the messages transmitted.

[0093] The three-dimensional model of the driver could also be used by vehicle functions, such as driver alertness monitoring software.

[0094] The implementation of the invention is simple, and it requires only two digital cameras.

Claims

Claims

1. Method for defining a three-dimensional model of a face of a driver of a motor vehicle (100) equipped with a first and a second camera (21, 22) placed at distinct positions (2111, 2211) of a passenger compartment (1) of the motor vehicle (100), and capable of simultaneously capturing images (IMG1, IMG2) of a face of a driver of the motor vehicle (100), the method comprising • a step (El) of determining a position (2111, 2211), relative to a fixed reference point (R0) of the motor vehicle (100), of a center (211) of a lens of the first camera (21) and of a center (221) of a lens of the second camera (22), then, an iteration on the following steps: • a step (E2) of capturing, at the same given instant, a first image (IMG1) in two dimensions of the driver's face by the first camera (21) and a second image (IMG2) in two dimensions of the driver's face by the second camera (22),the first image being decomposed into a first grid of pixels and the second image being decomposed into a second grid of pixels, • a step (E3) of association, between a first pixel of the first grid and a second pixel of the second grid corresponding to the same given point (PI) of the driver's face, • a step (E4) of determining a three-dimensional position of the given point (PI) of the driver's face as a function of the respective positions (2111, 2211) of a center of a lens (211) of the first camera (21) and of a center (221) of a lens of the second camera (22).,

2. Definition method according to the preceding claim, characterized in that the centers (211, 221) of the respective objectives of the first and second cameras (21, 22) are close to each other, in particular they are separated by a distance of less than 100 millimeters.

3. Definition method according to one of the preceding claims, characterized in that the association step (E3) comprises the implementation of an image fusion algorithm.

4. Definition method according to one of the preceding claims, characterized in that the step of determining a three-dimensional position of the given point comprises a calculation of a point of intersection between • a first straight line passing through the first pixel and the center (211) of the lens of the first camera (21), and • a second straight line passing through the second pixel and the center (221) of the lens of the second camera (22).

5. Definition method according to one of the preceding claims, characterized in that a first color is associated with the first pixel and a second color is associated with the second pixel, and the color of the given point (PI) is determined as being an average between the first and the second color.

6. Definition method according to one of the preceding claims, characterized in that an angle (52) between a sagittal plane (53) of a driver's head and a longitudinal axis (XO) of the vehicle is calculated by a monitoring algorithm (3) of driver vigilance, and in that the association step comprises taking into account said angle (52) between the sagittal plane (53) and the longitudinal axis (XO).

7. Definition method according to one of the preceding claims, characterized in that the vehicle is equipped with at least one ultrasonic sensor (24) making it possible to measure a distance between the ultrasonic sensor (24) and a point on the face closest to the ultrasonic sensor, and in that the association step (E3) takes this distance into account.

8. Definition method according to one of the preceding claims, characterized in that the determination step (El) comprises: • a creation, in particular by simulation, of a database (301) comprising a plurality of associations between, on the one hand, a given image of a headrest of a driver's seat of the motor vehicle and, on the other hand, a position of a center of an objective and an orientation of a camera having captured the given image, and optionally a angle measured between an axis of said camera and a longitudinal axis of the motor vehicle (100), • a capture by the first and second cameras (21, 22) of the motor vehicle (100), of a first and a second image (ATI, AT2) of the headrest (111) of the driver's seat (H), • a comparison respectively of the first image (ATI) and the second image (AT2) of the headrest (111) with images of the plurality of associations recorded in the database (301) to determine respectively the position (2111, 2211), relative to a fixed reference (R0), of a center (211) of the lens of the first camera (21) and of a center (221) of the lens of the second camera (22).

9. Device (10) for defining a three-dimensional model of a face of a user of a motor vehicle (100) equipped with a first and a second camera (21, 22) placed at distinct positions (2111, 2211) of a passenger compartment (1) of the motor vehicle (100), the device comprising hardware and / or software elements (1, 2, 3, 4, 11, 12, 13, 14, 21, 22, 23, 24, 31, 41, 42, 43, 44, 111, 211, 212, 213, 214, 221, 222, 223, 224, 300, 301, 2111, 2211) implementing the method according to one of claims 1 to 8, in particular hardware elements (1, 2, 3, 4, 11, 12, 13, 14, 21, 22, 23, 24, 31, 111, 211, 212, 213, 214, 221, 222, 223, 224, 300, 301, 2111, 2211) and / or software designed to implement the method according to one of claims 1 to 8.

10. A computer program product comprising program code instructions recorded on a computer-readable medium for implementing the steps of the method according to any one of claims 1 to 8 when said program is running on a computer.

Citation Information

Patent Citations

  • Automobile driving posture detection method based on multi-view vision

    CN111832373A

  • Vehicle occupant head positioning system

    US20180096475A1