SYSTEM AND METHOD FOR MONITORING A VEHICLE OCCUPANT

Description

technical field

[0001] The present invention relates to an interior rearview mirror comprising a vehicle occupant monitoring system and a method implemented by such a system. The invention also relates to a device and a method for acquiring images and depth. The present invention also relates to a vehicle interior rearview mirror. Technological background

[0002] Driver monitoring systems and occupant monitoring systems are now widely implemented in vehicles, particularly for safety purposes such as checking the proper use of seat belts by vehicle occupants or detecting a state of driver drowsiness.

[0003] Monitoring systems also make it possible to improve driver comfort, for example by automatically adapting the position of a driver's seat according to the driver's height, the position of an interior or exterior rearview mirror according to the position of the driver's eyes, or by sending information representative of the presence of occupants in the vehicle to a vehicle's air conditioning system.

[0004] Such a surveillance system must be positioned outside the driver's field of vision so as not to obstruct their view of the environment in which the vehicle is traveling, while still allowing data to be acquired relating to both the driver's face and the bodies of all the vehicle's occupants.

[0005] Such a system, including a camera, can produce high-resolution images, but it has limitations in terms of depth determination. Indeed, a monoscopic vision system can estimate relative depths but cannot determine the absolute depth or distance of an object.

[0006] Such a system, incorporating a depth sensor, can obtain a very precise depth or distance value to an object, but its resolution is limited. US patent document 2015 / 085116 A1 discloses an example of such a rearview mirror. Summary of the present invention

[0007] One object of the present invention is to solve at least one of the problems of the technological background described above.

[0008] Another object of the present invention is to improve the quality of data acquired by a vehicle occupant monitoring system.

[0009] Another object of the invention is to improve the integration of a vehicle occupant monitoring system in a confined space environment.

[0010] According to a first aspect, the present invention relates to an interior rearview mirror comprising a vehicle occupant monitoring system, the system comprising a set of electronic modules, the set comprising: a depth sensor, a camera configured to capture visible and infrared waves, and an infrared wave emitter, characterized in that the system comprises a single printed circuit board configured to receive and connect the set of electronic modules, the system being positioned in an interior rearview mirror of the vehicle.

[0011] Such a rearview mirror has the advantage of being compact thanks to the use of a single printed circuit board. Its position within an interior rearview mirror ensures that it does not obstruct the driver's view while still providing a vantage point from which to see all the vehicle's occupants.

[0012] According to one variant of the rearview mirror, the depth sensor is of the time-of-flight (ToF) sensor type.

[0013] Such a sensor requires a small footprint and assesses depths or distances with high accuracy.

[0014] According to another variant of the rearview mirror, the transmitter is controlled by the depth sensor and by the camera.

[0015] The infrared transmitter is thus shared, which allows for a reduction in the system's size and a decrease in the financial cost of such a system.

[0016] According to yet another variant, the rearview mirror includes a chip configured for data transmission to an on-board system of the vehicle, the chip being connected in communication to the set of electronic modules to receive data from the set of electronic modules.

[0017] This data is then used by various onboard vehicle systems, for example, a system responsible for the ventilation of the vehicle's cabin. The chip is also shared by the camera and the depth sensor.

[0018] According to an additional variant of the rearview mirror, the emitter further includes a set of light-emitting diodes configured to emit infrared radiation.

[0019] Light-emitting diodes are efficient and compact emitters of infrared waves, easily placed on the printed circuit board.

[0020] According to yet another variant, the rearview mirror also includes a power management module configured to power the entire set of electronic modules.

[0021] This power module is shared by all the electronic modules in the system and ensures that the electronic modules receive the appropriate power. This limits the number of connections on the electronic board.

[0022] According to yet another variant, the rearview mirror includes a microcontroller configured to control the entire set of electronic modules.

[0023] The electronic modules are thus synchronized and communicate with each other via this controller.

[0024] According to an additional variant of the rearview mirror, the entire set of electronic modules is oriented towards the interior of the vehicle's passenger compartment.

[0025] According to a second aspect, the present invention relates to a method for monitoring a vehicle occupant implemented by a rearview mirror as described above according to the first aspect of the present invention, the method comprising the steps of: emission of infrared waves by the transmitter, reception of first data representative of an image acquired by the camera, reception of second data representative of a depth map acquired by the depth sensor, association of depth values ​​from the map to a set of pixels in the image.

[0026] According to a third aspect, the present invention relates to a vehicle, for example of the automobile type, comprising a rearview mirror as described above according to the first aspect of the present invention. Brief description of the figures

[0027] Other features and advantages of the present invention will become apparent from the description of the specific and non-limiting embodiments of the present invention below, with reference to figures 1 to 3 attached, on which: [ Fig. 1 ] schematically illustrates a vehicle passenger compartment, according to a particular and non-limiting embodiment of the present invention; [ Fig. 2 [This schematically illustrates an interior rearview mirror installed in the vehicle of the] figure 1 and comprising a vehicle occupant monitoring system, according to a particular and non-limiting embodiment of the present invention; [ Fig. 3 [This schematically illustrates a flowchart of the different stages of a vehicle occupant monitoring process] figure 1 , according to a particular and non-limiting example of the present invention. Description of examples of achievements

[0028] A system and method for monitoring a vehicle occupant will now be described in what follows, with joint reference to figures 1 to 3 The same elements are identified with the same reference symbols throughout the description that follows.

[0029] The terms "first," "second" (or "firsts," "seconds"), etc., are used in this document by arbitrary convention to identify and distinguish different elements (such as operations, means, etc.) implemented in the embodiments described below. Such elements may be distinct or correspond to a single element, depending on the embodiment.

[0030] According to a particular and non-limiting embodiment of the present invention, the occupant monitoring system of a vehicle comprises a set of electronic modules, the set comprising: a depth sensor, a camera configured to capture visible and infrared waves, and an infrared wave emitter, characterized in that the system comprises a single printed circuit board configured to receive and connect the set of modules, the system being positioned in an interior rearview mirror of the vehicle.

[0031] There figure 1 illustrates a passenger compartment of a vehicle 10, for example a motor vehicle. According to other examples, vehicle 10 corresponds to a coach, a bus, a truck or a utility vehicle, that is to say a vehicle of the motorized land vehicle type.

[0032] The passenger compartment of the vehicle 10 includes, for example, in the first row, a driver's seat 12 and a passenger seat 13. The driver's seat 12 is located, for example, on the left side of the vehicle, the passenger seat 13 being located on the right side.

[0033] The concepts of right and left are defined according to the direction of travel of the vehicle. 10. The example of the figure 1 This corresponds to an example where vehicles travel on the right, as in France. However, the invention is not limited to such an example and extends to all road configurations, including those where vehicles travel on the left. In which case, the driver's seat 12 and passenger's seat 13 are reversed.

[0034] According to one particular embodiment, other seats are located on other rows behind the first row occupied by the driver's seat 12 and passenger's seat 13. The vehicle 10 has, for example, 1, 2 or 3 rows of seats, each of the seats being able to accommodate one occupant.

[0035] The passenger compartment includes, for example, a seat belt (not shown) for each seat, thus allowing a seat occupant to put on their seat belt in order to be protected by it in the event of a collision with the vehicle 10.

[0036] Vehicle 10 is thus configured to accommodate one or more occupants, including a driver for example.

[0037] The vehicle's interior 10 also includes an interior rearview mirror 11. This is, for example, placed high up, a few centimeters from a headliner or the upper edge of a windshield 14. The rearview mirror is, for example, placed in the middle of the passenger compartment, that is, halfway between the driver's seat 12 and the passenger seat 13. In this position, the rearview mirror does not obstruct the view of a driver of the vehicle 10 located in the driver's seat 13; the driver can thus look at the vehicle's surroundings through the windshield 14 without obstruction.

[0038] Vehicle 10 also includes an occupant monitoring system 2. Such a system makes it possible, as stated in the preamble, to determine, for example, the presence of occupant(s) in the different seats of vehicle 10 or to monitor the face or attitudes of the driver of vehicle 10 in order to detect a state of fatigue and / or drowsiness, for example.

[0039] The occupant monitoring system 2 is positioned in the interior rearview mirror 11 of the vehicle 10. In this position, it has a situation allowing it to monitor all the occupants of the vehicle 10, with no obstacle placed between the interior rearview mirror 11 and each occupant of the vehicle 10.

[0040] The vehicle occupant monitoring system 2 10 comprises a set of electronic modules, said set comprising: a depth sensor 20, a camera 21 configured to capture visible and infrared waves, and an infrared wave emitter 22.

[0041] System 2 also includes a single printed circuit board 23 configured to receive and connect the set of modules.

[0042] According to one particular embodiment, the entire set of electronic modules is oriented towards the interior of the vehicle 10's passenger compartment, thus enabling the depth sensor 20 and the camera 21 to benefit from a field of vision integrating a large part of the passenger compartment, including the seats and any occupants of the vehicle 10. Similarly, the transmitter 22 sends the infrared waves towards the areas included in the previously defined field of vision.

[0043] A depth sensor is for example of the "Time of Flight" (ToF) type, stereoscopic camera or LIDAR (acronym for "light detection and ranging" or "laser imaging detection and ranging", or in French "detection and estimation of distance by light" or "by laser").

[0044] In one particular embodiment, the depth sensor 20 is a time-of-flight (ToF) type sensor. Such a sensor offers numerous advantages, such as: the absence of moving parts, thus guaranteeing good calibration of the sensor and no wear, a very high compactness, its size is on the order of a few mm 3< or even tens of mm 3< , a range allowing to cover the depth of a passenger compartment, on the order of a few meters, a refresh time allowing fast or even real-time processing of data, on the order of 60 images per second (60Hz).

[0045] The depth sensor 20 has a resolution, for example, of around 300,000 pixels or 500,000 pixels.

[0046] System 2 also includes a camera 21 configured to capture visible and infrared waves. This camera thus makes it possible to acquire images of the passenger compartment and its occupants.

[0047] This camera 21 allows us to obtain images, the data being presented in the form of data representing pixels characterized by: coordinates in each image; and data relating to the colors and brightness of objects in the observed scene, here the passenger compartment of vehicle 10 and its occupants, in the form, for example, of RGB (Red Green Blue) or HSL (Hint, Saturation, Luminosity) colorimetric data. The camera has, for example, a resolution between 2 and 15 Megapixels (MPx).

[0048] System 2 also includes an infrared wave emitter 22, the wavelength of which is for example 870, 940 or 950nm.

[0049] The waves emitted by this transmitter 22 are then captured by the camera 21 and the depth sensor 20 after reflection from an object 100 present in the passenger compartment. An object 100 is, for example, a seat 22, 23, a seat belt, an occupant, or any other object present in the field of vision of the camera 21 or the depth sensor 20.

[0050] The time Δt elapsed between the emission of an infrared wave by the emitter 22 and the reception of this wave by the depth sensor 20 after reflection on an object 100 allows us to determine the depth or distance d of the object 100 relative to the sensor 20. This distance d is estimated from this time Δt and as a function of the speed c of light, thus d = cx Δt / 2.

[0051] System 2, positioned in the interior rearview mirror 11 of vehicle 10, thus provides data representative of the vehicle's interior, the presence and position of its occupants, and even data representative of facial expressions, for example, that of the driver. This data is presented in various formats: a point cloud with three-dimensional coordinates acquired by the depth sensor 20, an RGB or HSL image acquired by the camera 21, an infrared image acquired by the camera 21 with the help of the infrared emitter 22.

[0052] The use of a single electronic circuit 23 allows for the sharing of some of its components and saves space, thus miniaturizing system 2. This miniaturization of system 2 allows it to be integrated into an interior rearview mirror 11 of the vehicle 10 without significantly increasing its size. Therefore, the integration of system 2 is easier and more discreet.

[0053] There figure 2 schematically illustrates an interior rearview mirror installed in the vehicle, for example vehicle 10 of the figure 1 and comprising a vehicle occupant monitoring system 2, according to a particular and non-limiting embodiment of the present invention.

[0054] The electronic circuit 23 receives and connects a depth sensor 20, a camera 21 configured to capture visible and infrared waves, and an infrared wave emitter 22 as described in the figure 1 .

[0055] In one particular embodiment, the emitter 22 further comprises an array of light-emitting diodes (LEDs) 22a emitting infrared radiation. This array of LEDs 22a is grouped together to form a single module in this example. In other examples, the LEDs 22a are distributed at various locations on the electronic circuit 23.

[0056] In one variant, the electronic circuit 23 further includes a power management module 25 that supplies power to all the electronic modules. This module consists, for example, of power distribution circuits and / or cables that regulate a voltage to maintain a stable power supply reference for the various electronic modules. Such a power management module performs, for example, a conversion of an electrical voltage received from the vehicle's onboard system 10, for example 12 or 24 V, into an electrical voltage suitable for the electronic modules, for example 2, 5, or 6 V. In particular, it allows, if necessary, the conversion of a direct current voltage into an alternating current or pulsed voltage.

[0057] System 2 includes one (or more) processor(s) or microcontroller(s) 26 configured to execute instructions for carrying out the steps of the process and / or for executing instructions from the software embedded in system 2. The microcontroller 26 may include integrated memory, an input / output interface, and various circuits known to those skilled in the art.

[0058] The microcontroller 26 controls all the electronic modules; for example, it ensures: the synchronization between the sending of an infrared wave by the transmitter 22 and its reception by the depth sensor 20, and / or the simultaneous acquisition of data by the camera 21 and the depth sensor 20, and / or the management of the switching on of the transmitter 22 according to the needs of the camera 21 and / or the depth sensor 20.

[0059] Thus, the transmitter 22 is controlled by the depth sensor 20 and by the camera 21, notably via the microcontroller 26.

[0060] The system 2 further includes at least one memory 27 corresponding for example to volatile and / or non-volatile memory and / or includes a memory storage device which may include volatile and / or non-volatile memory, such as EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic or optical disk.

[0061] The computer code of the embedded software(s) including the instructions to be loaded and executed by the processor is, for example, stored on memory 27.

[0062] According to various specific and non-limiting embodiment examples, system 2 is coupled in communication with other similar devices or systems and / or with communication devices, for example a TCU (Telematic Control Unit), for example via a communication bus or through dedicated input / output ports.

[0063] According to a particular and non-limiting embodiment example, system 2 includes a data transmission chip 24 or a block of interface elements for communicating with various vehicle on-board systems 10.

[0064] According to another specific and non-limiting embodiment, system 2 includes a data transmission chip 24, for example connected to a communication interface that enables communication with other devices (such as other computers in the embedded system or embedded sensors) via a communication channel. The data transmission chip 24 and / or the communication interface correspond, for example, to a transmitter configured to transmit and receive information and / or data via the communication channel.The communication interface corresponds for example to a wired network of type CAN (from the English "Controller Area Network" or in French "Réseau de contrôlers"), CAN FD (from the English "Controller Area Network Flexible Data-Rate" or in French "Réseau de contrôlers à débit de données flexible"), FlexRay (standardized by the ISO 17458 standard), Ethernet (standardized by the ISO / IEC 802-3 standard) or LIN (from the English "Local Interconnect Network", or in French "Réseau interconnecté local").

[0065] In one particular, non-limiting embodiment, system 2 can provide output signals to one or more external devices, such as a display screen, touchscreen or otherwise, and / or other peripherals via respective output interfaces. In one variant, one or more of the external devices is integrated into system 2.

[0066] The data acquired by the depth sensor 20 and / or by the camera 21 feeds, for example, one or more driver assistance systems, known as ADAS (Advanced Driver-Assistance System) embedded in the vehicle 10.

[0067] System 2 thus has many shared elements directly connected to each other via the electronic circuit 23. The electronic circuit 23 thus makes it possible to present only one interface to connect all the electronic modules to the vehicle's on-board system 10, allowing for easier installation.

[0068] There figure 3This illustrates a flowchart of the different steps of a method 3 for monitoring the occupant of a vehicle, for example vehicle 10, according to a particular and non-limiting embodiment of the present invention. The method 3 is, for example, implemented by the system 2 installed in vehicle 10.

[0069] In a first step 31, infrared waves are emitted by the transmitter 22.

[0070] In a second step 32, initial data representing an image acquired by camera 21 are received.

[0071] In a third step 33, second data representative of a depth map acquired by the depth sensor 20 are received.

[0072] In a fourth step 34, depth values ​​from the mapping are associated with a set of pixels from the image.

[0073] According to one variant, the variants and examples of the operations described in relation to the figures 1 to 2 apply to steps 3 of process of the figure 3 .

[0074] The present invention also relates to a vehicle, for example a motor vehicle or more generally a land motor vehicle, comprising system 2 of the figure 1 or 2 .

Claims

1. Interior rearview mirror (11) comprising a system (2) for monitoring an occupant of a vehicle (10), said system comprising a set of electronic units, said set comprising: - a depth sensor (20), - a camera (21) configured to pick up visible and infrared waves, and - an infrared wave sender (22), wherein said system (2) comprises a single printed circuit (23) configured to receive and connect said set of electronic units, said system (2) being positioned in an interior rear-view mirror (11) of said vehicle (10).

2. System (2) according to claim 1, for which the depth sensor (20) is of the flight time sensor type.

3. Rearview mirror according to claim 1 or 2, for which said sender (22) is on dock by said depth sensor (20) and by said camera (21).

4. Rearview mirror according to one of claims 1 to 3, which further comprises a chip (24) configured for a data transmission to an on-board system of said vehicle (10), said chip being connected in communication to said set of electronic modules to receive said data from said set of electronic modules.

5. Rearview mirror according to one of claims 1 to 4, for which said sender 22 further comprises a set of light-emitting diodes (22 a) configured to emit infrared radiation.

6. Rearview mirror according to one of claims 1 to 5, which further comprises a power management module (25) configured to power said set of electronic modules.

7. Mirror according to one of claims 1 to 6, which comprises a microcontroller (26) configured to control said set of electronic modules.

8. Mirror according to one of claims 1 to 7, for which the set of electronic modules is orientated towards the interior of a passenger compartment of said vehicle (10).

9. Method (3) for monitoring an occupant of a vehicle implemented by a rear-view mirror according to one of claims 1 to 8, said method comprising the steps of: - emission (31) of infrared waves by said sender (22), - receipt (32) of first data representative of an image acquired by said camera (21), - receipt (33) of second data representative of a depth map acquired by said depth sensor (20), - association (34) of depth values of said map with a set of pixels of said image.

10. Vehicle (10) comprising the rearview mirror according to one of claims 1 to 8.