vehicle occupant monitoring system and method

Abstract

The present invention relates to a system (2) and a method for monitoring an occupant of a vehicle (10) implemented by such a system comprising a set of electronic modules, the set comprising a depth sensor (20), a camera (21) configured to capture visible and infrared waves and an infrared wave transmitter (22), the system comprising a single printed circuit (23) configured to receive and connect the set of electronic modules, the system being positioned in an interior rearview mirror of the vehicle. Figure for abstract: Figure 1

Description

Title of the invention: system and method for monitoring the occupant of a vehicle Technical field

[0001] The present invention relates to a vehicle occupant monitoring system and a method implemented by such a system. The invention also relates to a device and a method for acquiring images and depths. The present invention also relates to a vehicle interior rearview mirror. Technological background

[0002] Driver monitoring systems and occupant monitoring systems are now widely implemented in vehicles, particularly for safety purposes such as verifying the proper use of seat belts by the occupants of a vehicle or detecting a state of drowsiness of a driver.

[0003] The monitoring systems also make it possible to improve the comfort of a driver, for example by automatically adapting a position of a driver's seat according to the size of the driver, a position of an interior or exterior rearview mirror according to the position of the driver's eyes or even sending information representative of the presence of occupants in the vehicle to an air conditioning system of the vehicle.

[0004] Such a monitoring system must be positioned outside the field of vision of the driver of the vehicle so as not to obstruct his vision of the environment in which the vehicle is traveling, while making it possible to acquire data relating to both the face of the driver and the bodies of all the occupants of the vehicle.

[0005] Such a system comprising a camera makes it possible to obtain images of good resolution, but nevertheless has limits in terms of determining depths. Indeed, a monoscopic vision system also makes it possible to estimate relative depths but does not make it possible to determine an absolute depth or distance of an object.

[0006] Such a system comprising a depth sensor makes it possible to obtain a very precise depth or distance value of an object but its resolution is limited. Summary of the present invention

[0007] An object of the present invention is to solve at least one of the problems of the technological background described above.

[0008] Another object of the present invention is to improve the quality of data acquired by a vehicle occupant monitoring system.

[0009] Another object of the invention is to improve the integration of a vehicle occupant monitoring system in a small environment.

[0010] According to a first aspect, the present invention relates to a vehicle occupant monitoring system, the system comprising a set of electronic modules, the set comprising: - a depth sensor, - a camera configured to capture visible and infrared waves, and - an infrared wave transmitter, characterized in that the system comprises a single printed circuit configured to receive and connect the set of electronic modules, the system being positioned in an interior rearview mirror of the vehicle.

[0011] Such a system has the advantage of being compact thanks to the sharing of a single printed circuit. Its position in an interior rearview mirror means that the driver's vision is not obstructed while still benefiting from a viewpoint allowing all the vehicle's occupants to be seen.

[0012] According to a variant of the system, the depth sensor is of the time-of-flight (ToF) sensor type.

[0013] Such a sensor requires a small footprint and evaluates depths or distances with great precision.

[0014] According to another variant of the system, the transmitter is controlled by the depth sensor and by the camera.

[0015] The infrared transmitter is thus shared, which allows a reduction in the size of the system and a reduction in the financial cost of such a system.

[0016] According to another variant, the system comprises a chip configured for transmitting data to an on-board system of the vehicle, the chip being connected in communication to the set of electronic modules to receive the data from the set of electronic modules.

[0017] This data is then used by various on-board systems in the vehicle, for example a system responsible for ventilation of a vehicle passenger compartment. The chip is also shared for the camera and the depth sensor.

[0018] According to an additional variant of the system, the transmitter further comprises a set of light-emitting diodes configured to emit infrared radiation.

[0019] Light-emitting diodes are efficient and compact infrared wave emitters, easily arranged on the printed circuit.

[0020] According to another variant, the system further comprises a power management module configured to power the set of electronic modules.

[0021] This power module is thus shared for all the electronic modules- Ironic of the system and allows to supply the electronic modules with the adequate power. The number of connections of the electronic card is thus limited.

[0022] According to yet another variant, the system comprises a microcontroller configured to control the set of electronic modules.

[0023] The electronic modules are thus synchronized and communicate with each other via this controller.

[0024] According to an additional variant of the system, all of the electronic modules are oriented towards the interior of a passenger compartment of the vehicle.

[0025] According to a second aspect, the present invention relates to a method for monitoring an occupant of a vehicle implemented by a system as described above according to the first aspect of the present invention, the method comprising the steps of: - emission of infrared waves by the transmitter, - reception of first data representative of an image acquired by the camera, - reception of second data representative of a depth map acquired by the depth sensor, - association of depth values ​​from the mapping to a set of pixels in the image.

[0026] According to a third aspect, the present invention relates to a vehicle, for example of the automobile type, comprising a system as described above according to the first aspect of the present invention.

[0027] According to a fourth aspect, the present invention relates to a computer program which comprises instructions adapted for executing the steps of the method according to the second aspect of the present invention, in particular when the computer program is executed by at least one processor.

[0028] Such a computer program may use any programming language, and be in the form of source code, object code, or intermediate code between source code and object code, such as in a partially compiled form, or in any other desirable form.

[0029] According to a fifth aspect, the present invention relates to a computer-readable recording medium on which is recorded a computer program comprising instructions for carrying out the steps of the method according to the second aspect of the present invention.

[0030] On the one hand, the recording medium may be any entity or device capable of storing the program. For example, the medium may comprise a storage means, such as a ROM memory, a CD-ROM or a microelectronic circuit type ROM memory, or a magnetic recording means or a hard disk.

[0031] On the other hand, this recording medium can also be a trans medium miscible such as an electrical or optical signal, such a signal being able to be conveyed via an electrical or optical cable, by conventional or terrestrial radio or by self-directed laser beam or by other means. The computer program according to the present invention can in particular be downloaded from an Internet-type network.

[0032] Alternatively, the recording medium may be an integrated circuit in which the computer program is incorporated, the integrated circuit being adapted to perform or to be used in performing the method in question. Brief description of the figures

[0033] Other characteristics and advantages of the present invention will emerge from the description of the particular and non-limiting exemplary embodiments of the present invention below, with reference to the appended figures 1 to 3, in which:

[0034] [Fig-1] schematically illustrates a passenger compartment of a vehicle, according to an example of particular and non-limiting embodiment of the present invention;

[0035] [Fig.2] schematically illustrates an interior rearview mirror installed in the vehicle of [Fig.l] and comprising a vehicle occupant monitoring system, according to a particular and non-limiting exemplary embodiment of the present invention;

[0036] [Fig.3] schematically illustrates a flowchart of the different steps of a method for monitoring the occupant of the vehicle of [Fig.l], according to a particular and non-limiting exemplary embodiment of the present invention. Description of examples of implementation

[0037] A system and method for monitoring an occupant of a vehicle will now be described in the following with reference to Figures 1 to 3. The same elements are identified with the same reference signs throughout the description which follows.

[0038] The terms "first(s)", "second(s)" (or "first(s)", "second(s)"), etc. are used in this document by arbitrary convention to enable different elements (such as operations, means, etc.) implemented in the embodiments described below to be identified and distinguished. Such elements may be distinct or correspond to a single element, depending on the embodiment.

[0039] According to a particular and non-limiting example of embodiment of the present invention, the vehicle occupant monitoring system comprises a set of electronic modules, the set comprising: - a depth sensor, - a camera configured to capture visible and infrared waves and - an infrared wave transmitter, characterized in that the system comprises a single printed circuit configured to receive and connect the set of modules, the system being positioned in a re- interior rearview mirror of the vehicle.

[0040] [Fig.l] illustrates a passenger compartment of a vehicle 10, for example a motor vehicle. According to other examples, the vehicle 10 corresponds to a coach, a bus, a truck or a utility vehicle, that is to say a vehicle of the motorized land vehicle type.

[0041] The passenger compartment of the vehicle 10 comprises, for example, in a first row, a driver's seat 12 and a passenger seat 13. The driver's seat 12 is located, for example, on the left side of the vehicle, the passenger seat 13 being located on the right side.

[0042] The concepts of right and left are defined according to the direction of travel of the vehicle 10. The example in [Fig.l] corresponds to an example in which the vehicles travel on the right, as in France. The invention is however not limited to such an example and extends to all road configurations, including those in which the vehicles travel on the left. In which case the driver 12 and passenger 13 seats are reversed.

[0043] According to a particular embodiment, other seats are located in other rows behind the first row occupied by the driver 12 and passenger 13 seats. The vehicle 10 has, for example, 1, 2 or 3 rows of seats, each of the seats being able to accommodate an occupant.

[0044] The passenger compartment comprises for example a seat belt (not shown) for each seat, thus allowing an occupant of a seat to put on his seat belt in order to be protected by the latter in the event of an impact of the vehicle 10.

[0045] The vehicle 10 is thus configured to accommodate one or more occupants, including a driver for example.

[0046] The passenger compartment of the vehicle 10 also comprises an interior rearview mirror 11. This is for example placed high up, a few centimeters from a roof lining or a high end of a windshield 14. The rearview mirror is for example placed in the middle of the passenger compartment, that is to say halfway between the driver's seat 12 and the passenger seat 13. Thus placed, the rearview mirror does not obstruct the view of a driver of the vehicle 10 placed in the driver's seat 13, the driver can thus look at the environment of the vehicle through the windshield 14 without discomfort.

[0047] The vehicle 10 also comprises an occupant monitoring system 2. Such a system makes it possible, as expressed in the preamble, to determine, for example, the presence of occupant(s) in the different seats of the vehicle 10 or to monitor the face or attitudes of the driver of the vehicle 10 in order to detect a state of fatigue and / or drowsiness, for example.

[0048] The occupant monitoring system 2 is positioned in the interior rearview mirror 11 of the vehicle 10. Thus positioned, it has a situation allowing all of the occupants of the vehicle 10 to be monitored, with no obstacle being placed between the re- interior mirror 11 and each occupant of the vehicle 10.

[0049] The occupant monitoring system 2 of a vehicle 10 comprises a set of electronic modules, said set comprising: - a 20 depth sensor, - a camera 21 configured to capture visible and infrared waves, and - an infrared wave transmitter 22.

[0050] The system 2 also comprises a single printed circuit 23 configured to receive and connect the set of modules.

[0051] According to a particular embodiment, all of the electronic modules are oriented towards the interior of the passenger compartment of the vehicle 10, thus allowing the depth sensor 20 and the camera 21 to benefit from a field of vision integrating a large part of the passenger compartment, including the seats and any occupants of the vehicle 10. Similarly, the transmitter 22 thus sends the infrared waves towards the areas included in the previously defined field of vision.

[0052] A depth sensor is for example of the “Time of Flight” (ToF) type, stereoscopic camera or LIDAR (acronym for “light detection and ranging” or “laser imaging detection and ranging”).

[0053] According to a particular exemplary embodiment, the depth sensor 20 is of the time-of-flight (ToF) sensor type. Such a sensor has numerous advantages such as: - the absence of moving parts, thus guaranteeing good calibration of the sensor and an absence of wear, - very high compactness, its size is of the order of a few mm3 or even tens of mm3, - a range allowing the depth of a passenger compartment to be covered, of the order of a few meters, - a refresh time allowing rapid or even real-time processing of data, of the order of 60 images per second (60Hz).

[0054] The depth sensor 20 has a resolution for example of the order of 300,000 pixels or 500,000 pixels.

[0055] The system 2 also comprises a camera 21 configured to capture visible and infrared waves. This camera thus makes it possible to acquire images of the passenger compartment and its occupants.

[0056] This camera 21 makes it possible to obtain images, the data being presented in the form of data representing pixels characterized by: - coordinates in each image; and - data relating to the colors and brightness of the objects in the observed scene, here the passenger compartment of the vehicle 10 and its occupants, in the form, for example, of RGB colorimetric data (from the English “Red Green Blue”) or TSL (Tone, Saturation, Brightness). The camera has, for example, a resolution of between 2 and 15 Mega Pixels (MPx).

[0057] The system 2 also comprises an infrared wave emitter 22, the wavelength being for example 870, 940 or 950nm.

[0058] The waves emitted by this transmitter 22 are then captured by the camera 21 and the depth sensor 20 after reflection on an object 100 present in the passenger compartment. An object 100 is for example a seat 22, 23, a seat belt, an occupant or any other object present in the field of vision of the camera 21 or the depth sensor 20.

[0059] The duration At elapsed between the emission of an infrared wave by the transmitter 22 and the reception of this wave by the depth sensor 20 after reflection on an object 100 makes it possible to determine the depth or distance d of the object 100 relative to the sensor 20. This distance d is estimated from this duration At and as a function of the speed c of the light, thus d = cx At / 2.

[0060] The system 2 positioned in the interior rearview mirror 11 of the vehicle 10 thus makes it possible to obtain data representative of the passenger compartment of the vehicle 10, of the presence and position of its occupants or even data representative of expression on a face, for example that of the driver. This data is presented in different forms: - a point cloud with three-dimensional coordinates acquired by the depth sensor 20, - an RGB or TSL image acquired by camera 21, - an infrared image acquired by the camera 21 with the help of the infrared transmitter 22.

[0061] The use of a single electronic circuit 23 makes it possible to share some of its components and allows space saving, and therefore miniaturization of the system 2. Miniaturization of the system 2 thus makes it possible to integrate it into an interior rearview mirror 11 of the vehicle 10 without having to considerably increase its volume. Thus, the integration of the system 2 is easier and more discreet.

[0062] [Fig. 2] schematically illustrates an interior rearview mirror installed in the vehicle, for example the vehicle 10 of [Fig. 1] and comprising a system 2 for monitoring the occupant of the vehicle, according to a particular and non-limiting exemplary embodiment of the present invention.

[0063] The electronic circuit 23 receives and connects a depth sensor 20, a camera 21 configured to capture visible and infrared waves and a wave transmitter 22 infrared as described in [Fig.l].

[0064] According to a particular embodiment, the transmitter 22 further comprises a set of light-emitting diodes (LEDs) 22a emitting infrared radiation. This set of light-emitting diodes 22a is grouped together to constitute a single module according to this example. According to other examples, the light-emitting diodes 22a are distributed at several locations on the electronic circuit 23.

[0065] According to a variant, the electronic circuit 23 further comprises a power management module 25 supplying power to the set of electronic modules. This module is composed, for example, of power distribution circuits and / or cables, regulating a voltage to maintain a stable electrical supply reference for the various electronic modules. Such a power management module, for example, converts an electrical voltage received from the on-board system of the vehicle 10, for example 12 or 24V, into an electrical voltage suitable for the electronic modules, for example 2, 5 or 6V. It makes it possible, in particular, if necessary, to transform a direct voltage into an alternating or pulsed voltage.

[0066] The system 2 comprises one (or more) processor(s) or microcontroller(s) 26 configured to execute instructions for carrying out the steps of the method and / or for executing the instructions of the software(s) embedded in the system 2. The microcontroller 26 may include integrated memory, an input / output interface, and various circuits known to those skilled in the art.

[0067] The microcontroller 26 controls the set of electronic modules, it ensures for example: - the synchronization between the sending of an infrared wave by the transmitter 22 and its reception by the depth sensor 20, and / or - the simultaneous acquisition of data by the camera 21 and the depth sensor 20, and / or - management of the ignition of the transmitter 22 according to the needs of the camera 21 and / or the depth sensor 20.

[0068] Thus, the transmitter 22 is controlled by the depth sensor 20 and by the camera 21, in particular via the microcontroller 26.

[0069] The system 2 further comprises at least one memory 27 corresponding for example to a volatile and / or non-volatile memory and / or comprises a memory storage device which may comprise volatile and / or non-volatile memory, such as EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic or optical disk.

[0070] The computer code of the embedded software(s) comprising the instructions to be loaded and executed by the processor is for example stored in the memory 27.

[0071] According to various particular and non-limiting embodiments, the system 2 is coupled in communication with other similar devices or systems and / or with communication devices, for example a TCU (from the English “Telematic Control Unit” or in French “Telematic Control Unit”), for example via a communication bus or through dedicated input / output ports.

[0072] According to a particular and non-limiting exemplary embodiment, the system 2 comprises a data transmission chip 24 or a block of interface elements for communicating with various on-board systems of the vehicle 10.

[0073] According to another particular and non-limiting exemplary embodiment, the system 2 comprises a data transmission chip 24, for example connected to a communication interface which makes it possible to establish communication with other devices (such as other computers of the on-board system or on-board sensors) via a communication channel. The data transmission chip 24 and / or the communication interface corresponds for example to a transmitter configured to transmit and receive information and / or data via the communication channel. The communication interface corresponds, for example, to a wired network of the type CAN (from the English "Controller Area Network" or in French "Réseau de contrôles"), CAN FD (from the English "Controller Area Network Flexible Data-Rate" or in French "Réseau de contrôles à débit de données flexible"), FlexRay (standardized by the ISO 17458 standard), Ethernet (standardized by the ISO / IEC 802-3 standard) or LIN (from the English "Local Interconnect Network").

[0074] According to a particular and non-limiting exemplary embodiment, the system 2 can provide output signals to one or more external devices, such as a display screen, touch-sensitive or not, and / or other peripherals via respective output interfaces. According to a variant, one or other of the external devices is integrated into the system 2.

[0075] The data acquired by the depth sensor 20 and / or by the camera 21 feeds, for example, one or more driving assistance systems, called AD AS (from the English “Advanced Driver-Assistance System” or in French “Advanced Driving Assistance System”) on board the vehicle 10.

[0076] The system 2 thus has numerous shared elements directly connected to each other via the electronic circuit 23. The electronic circuit 23 thus makes it possible to have only one interface for connecting all of the electronic modules to the on-board system of the vehicle 10, allowing easier installation.

[0077] [Fig.3] illustrates a flowchart of the different stages of a method 3 of sur occupant monitoring of a vehicle, for example of the vehicle 10, according to a particular and non-limiting exemplary embodiment of the present invention. The method 3 is for example implemented by the system 2 embedded in the vehicle 10.

[0078] In a first step 31, infrared waves are emitted by the transmitter 22.

[0079] In a second step 32, first data representative of an image acquired by the camera 21 are received.

[0080] In a third step 33, second data representative of a depth map acquired by the depth sensor 20 are received.

[0081] In a fourth step 34, depth values ​​of the mapping are associated with a set of pixels of the image.

[0082] According to a variant, the variants and examples of the operations described in relation to figures 1 to 2 apply to the steps of method 3 of [Fig.3].

[0083] The present invention also relates to a vehicle, for example an automobile or more generally a land motor vehicle, comprising the system 2 of [Fig.l] or 2.

Claims

Claims

1. System (2) for monitoring the occupant of a vehicle (10), said system comprising a set of electronic modules, said set comprising: - a depth sensor (20), - a camera (21) configured to capture visible and infrared waves, and - an infrared wave transmitter (22), characterized in that said system (2) comprises a single printed circuit (23) configured to receive and connect said set of electronic modules, said system (2) being positioned in an interior rearview mirror (11) of said vehicle (10).

2. System (2) according to claim 1, wherein the depth sensor (20) is of the time-of-flight sensor type.

3. System (2) according to claim 1 or 2, wherein said transmitter (22) is controlled by said depth sensor (20) and by said camera (21).

4. System (2) according to one of claims 1 to 3, which further comprises a chip (24) configured for transmission of data to an on-board system of said vehicle (10), said chip being communicatively connected to said set of electronic modules to receive said data from said set of electronic modules.

5. System (2) according to one of claims 1 to 4, for which said transmitter 22 further comprises a set of light-emitting diodes (22a) configured to emit infrared radiation.

6. System (2) according to one of claims 1 to 5, which further comprises a power management module (25) configured to power said set of electronic modules.

7. System (2) according to one of claims 1 to 6, which comprises a microcontroller (26) configured to control said set of electronic modules.

8. System (2) according to one of claims 1 to 7, for which the set of electronic modules is oriented towards the interior of a passenger compartment of said vehicle (10).

9. Method (3) for monitoring an occupant of a vehicle implemented by a system (2) according to one of claims 1 to 8, said method comprising the steps of: - emission (31) of infrared waves by said transmitter (22), - reception (32) of first data representative of an image acquired by said camera (21), - reception (33) of second data representative of a depth map acquired by said depth sensor (20), - association (34) of depth values ​​of said map with a set of pixels of said image.

10. Vehicle (10) comprising the system (2) according to one of claims 1 to 8.

Images