Electronic control unit for a vehicle interior monitoring system and associated interior monitoring system

Abstract

The present invention provides an electronic control unit (20) for a vehicle interior monitoring system, the vehicle interior monitoring system including a first camera module (41) including a first camera control unit (121); and at least a second camera module (42, 43) including a second camera control unit (122, 123) connected to a second serialization module (142, 143), the electronic control unit (20) including a first deserialization module (181) connected to the first serialization module (141) and at least a second deserialization module (182, 183), each second deserialization module being connected to each of the corresponding at least one second serialization module (142, 143), the electronic control unit also including a master control device (22) and configured to manage the activation of each camera module with a time lag relative to another camera module.

Description

Technical Field

[0001] The present invention relates to an electronic control unit for a vehicle interior monitoring system and an associated vehicle interior monitoring system.

[0002] This invention belongs to the field of vehicle monitoring systems that use multiple camera modules for general monitoring (such as monitoring the passenger compartment, detecting vital signs, detecting the position of the driver and occupants, or locating them).

[0003] Each camera module used in this type of vehicle-mounted monitoring system typically includes a light source and an image sensor.

[0004] Such systems, such as Driver Monitoring System (DMS) or Occupant Monitoring System (OMS), can be used in, for example, Advanced Driver Assistance Systems (ADAS). Background Technology

[0005] For effective monitoring inside a vehicle, under all conditions including nighttime driving, it is useful to sensitively activate a light source while activating an image sensor to capture one or more images of the vehicle's interior, thereby forming a video. For example, in some applications, light sources in a spectrum other than visible light, such as infrared sources, are used to avoid interfering with the driver. However, with infrared or visible light sources, activation of a light source from a given camera module (even for a short period) may disrupt image acquisition by another camera module by introducing unwanted reflections or excessive illumination into the image captured by that other camera module. Summary of the Invention

[0006] The purpose of this invention is to solve the above-mentioned problems.

[0007] Therefore, the present invention proposes an electronic control unit for a vehicle interior monitoring system, the interior monitoring system including multiple camera modules, the multiple camera modules including:

[0008] - A first camera module, comprising a first light source and a first image sensor, and a first camera control unit connected to a first serialization module.

[0009] - At least a second camera module, the second camera module including a second light source and a second image sensor, the second camera module including a second camera control unit connected to the second serialization module.

[0010] The electronic control unit includes a first deserialization module connected to the first serialization module and at least a second deserialization module, each second deserialization module being connected to each of the corresponding at least one second serialization module. The electronic control unit also includes a master control device connected to the first deserialization module and the second deserialization module. The electronic control unit is configured to manage the activation of each camera module with a time lag relative to another camera module via at least one of the first deserialization module and the second deserialization module.

[0011] Advantageously, the proposed electronic control unit enables control of the activation sequence of camera modules among multiple camera modules to avoid simultaneous activation of the light sources of two separate camera modules, and thus ensures satisfactory image acquisition quality under all conditions.

[0012] In some embodiments of the invention, the electronic control unit includes one or more of the following features, which may be considered individually or in any technically feasible combination.

[0013] The main control device is configured to activate the first camera module via a first trigger signal transmitted through the connection between the first deserialization module and the first serialization module, thereby triggering the first camera control unit to activate the first light source and the first image sensor.

[0014] The first deserialization module is further configured to receive a synchronization signal from the first serialization module and send the synchronization signal to the second deserialization module. The second deserialization module is configured to send the synchronization signal to the corresponding second serialization module of the second camera module. The second camera module is configured to obtain a second trigger signal of the time lag delay when receiving the synchronization signal, and the second trigger signal is applied to activate the second camera module.

[0015] Each second deserialization module is configured to receive a synchronization signal from the connected second serialization module and send the synchronization signal to a subsequent second deserialization module connected to a subsequent second camera module, the subsequent second camera module being configured to receive the synchronization signal and obtain a subsequent second trigger signal that delays the time lag, the subsequent second trigger signal being applied to activate the subsequent second camera module.

[0016] The first deserialization module and at least the second deserialization module are connected via an LVDS (Low Voltage Differential Signaling) link.

[0017] The internal monitoring system includes multiple second camera modules, and the main control device is configured as follows:

[0018] - Generate a first trigger signal for activating the first camera module, and transmit the first trigger signal for activating the first camera module via the first deserialization module, and

[0019] - Generate at least a second trigger signal for activating the or each of the second camera modules, each second trigger signal being delayed by a total hysteresis relative to the first trigger signal, the total hysteresis depending on the level of the second camera module among the plurality of second camera modules, and transmit each second trigger signal via a corresponding second deserialization module to activate the second camera module.

[0020] The main controller transmits each of the first trigger signal and the second trigger signal to the GPIO port of the corresponding first camera module and the second camera module via the general purpose input / output port (GPIO) of the corresponding first deserialization module and the second deserialization module.

[0021] The main control device transmits each of the first trigger signal and the second trigger signal as an I2C (integrated integrated circuit) command.

[0022] The present invention also relates to a vehicle interior monitoring system comprising multiple camera modules, the vehicle interior monitoring system comprising:

[0023] - A first camera module, comprising a first light source and a first image sensor, and a first camera control unit connected to a first serialization module.

[0024] - At least a second camera module, the second camera module including a second light source and a second image sensor, the second camera module including a second camera control unit connected to the second serialization module.

[0025] The internal monitoring system also includes an electronic control unit, as briefly described above.

[0026] Depending on the optional features, the first light source and / or each of the second light sources is an infrared light source. Attached Figure Description

[0027] Other features and advantages of the invention will become apparent from the following description, provided by way of non-limiting example only, with reference to the accompanying drawings, in which:

[0028] Figure 1 The main functional modules of the vehicle interior monitoring system according to the present invention are schematically shown.

[0029] Figure 2 A vehicle interior monitoring system according to a first embodiment is schematically shown;

[0030] Figure 3A vehicle interior monitoring system according to the second embodiment is schematically illustrated, and

[0031] Figure 4 The vehicle interior monitoring system according to the third embodiment is illustrated schematically. Detailed Implementation

[0032] Figure 1 This is a schematic diagram of the internal monitoring system 2 of a vehicle (not shown), referred to as "System 2" in the following description.

[0033] System 2 is suitable for all types of vehicles, especially automobiles.

[0034] System 2 includes multiple vehicle-mounted camera modules, including a first camera module 41 and at least a second camera module 42.

[0035] exist Figure 1 In the example shown, system 2 includes a first camera module 41 and two second camera modules 42 and 43.

[0036] In general, multiple camera modules can include any number N camera modules 41 to 4 N In the example shown, N=3.

[0037] Each camera module has an associated level among multiple camera modules; for example, the first camera module 41 has level 1, the second camera module 42 has level 2, the second camera module 43 has level 3, and more generally, the second camera module 4... N It has a level N.

[0038] Each camera module 41 to 4 N Including light source 61…6 N Image sensor 81…8 N Camera control unit 101…10 N and serialization module 121…12 N They are suitable for communication via an internal communication bus.

[0039] According to one implementation, each camera module also includes serialization modules 121…12 N At least one associated connection port 141…14 N For example, general purpose input / output (GPIO) connection ports and / or integrated circuit (I2C) connection ports.

[0040] For the first camera module, light source 61 can also be called the first light source, image sensor 81 can also be called the first image sensor, camera control unit 101 can also be called the first camera control unit, serialization module 121 can also be called the first serialization control module, and connection port 141 can also be called the first connection port.

[0041] For each second camera module, the light source module can also be called the second light source module, the image sensor can also be called the second image sensor, the camera control unit can also be called the second camera control unit, the serialization module can also be called the second serialization control module, and the connection port can also be called the second connection port.

[0042] Based on one example, light source 61...6 N Some or all of them include one or more light-emitting diodes (LEDs). Depending on a specific example, each light source has 61...6... N Configured to emit light in a spectrum other than the visible spectrum, such as the infrared spectrum. In other words, according to one example, each light source has 61...6... N It is an infrared light source.

[0043] 61...6 per light source N And each image sensor 81...8 N Configured to be controlled by the corresponding camera control unit 101...10 N Activate (i.e., connect).

[0044] Each serialization module 121...12 N It is configured to receive image / video data on a video input interface (not shown), the image / video data being represented as a series of pixel matrices, and to transform the image / video data into a series of bits.

[0045] Multiple camera modules 41...4 N Each camera module in the system is configured to be connected via serial link 151...15 N Use the corresponding deserialization module 181...18 N Communication, each deserialization module 181...18 N It has I / O connection ports 161...16 N .

[0046] According to one implementation scheme, each serial link has 151...15 N It is a physical LVDS (Low Voltage Differential Signaling) or coaxial cable link typically used in cameras.

[0047] According to one implementation scheme, each serial link has 151...15 NIt is a gigabit multimedia serial link (GMSL) or a flat panel display link (FPD-Link), such as GMSL1, GMSL2, FPD-Link3, and FPD-Link4.

[0048] For example, each serialization module is a standard component, such as a CSI-2 (Camera Serial Interface 2) serializer to GMSL or FPD-Link.

[0049] System 2 also includes an electronic control unit 20, which includes a main control device 22. The main control device 22 is preferably implemented as a system-on-a-chip (SoC) and configured to communicate with each deserialization module 181...18 via a bidirectional communication bus. N communication.

[0050] Preferably, the electronic control unit 20 includes a deserialization module 181...18 N and main control unit 22.

[0051] The deserialization module 181 connected to the serialization module 121 of the first camera module is also called the first deserialization module.

[0052] Each deserialization module has 182...18 N ; Connect to the second camera module 42 to 4 N The corresponding serialization module 122...12 N It can also be called the second deserialization module.

[0053] Advantageously, the electronic control unit 20 is configured to communicate via a first deserialization module 181 or via a plurality of deserialization modules 181...18 N This is used to manage the activation of each camera module that has a time lag relative to another camera module.

[0054] Activation of the camera module includes activating the light source, i.e., turning on the light source, and activating the image sensor of the camera module.

[0055] Before activating any other camera module among a plurality of camera modules, activation of a given camera module is followed by its deactivation (specifically, its light source is cut off).

[0056] The time lag is chosen so that only one light source is activated at a time. In other words, when the light source of one camera module is turned on, all other light sources of other camera modules are turned off.

[0057] For example, the time lag D between the activation of two consecutive cameras in a multi-camera setup depends on the FPS (frames per second) parameter of each camera, for example, less than or equal to 10 ms.

[0058] Advantageously, the electronic control unit 20 manages the sequential activation of the camera modules, which avoids any interference between the light sources of individual camera modules.

[0059] The time lag D ensures that when a subsequent camera is activated (i.e., its light source is turned on), the previous camera is deactivated (i.e., its light source is turned off).

[0060] For example, the camera modules are activated in a sorted order.

[0061] For example, when the first (level 1) camera module is activated first, the total time lag between the activation of the first camera module 41 and the activation of the second camera module 42 (level 2 camera module) is D, the total time lag between the activation of the first camera module 41 and the activation of the second camera module 43 (level 3 camera module) is 2xD, and so on, so that the activation of the first camera module 41 and the activation of the second camera module 42 are related to the activation of the second camera module 43. N The total time lag between activations of (N-1) camera modules is (N-1)xD.

[0062] Therefore, the total time lag between the activation of the first camera module and the activation of a given second camera module depends on the rank of the second camera module among the multiple camera modules.

[0063] The following is for reference. Figures 2 to 4 Several implementation schemes for managing the sequence activation of the camera module have been disclosed.

[0064] according to Figure 2 In the first embodiment illustrated schematically, the main control device 22 is configured to activate the first camera module 41 via a first trigger signal S1 transmitted to the first deserializer 181. The deserializer 181 transmits the first trigger signal S1 to the corresponding serialization module 121 via, for example, its GPIO port and then via a serial link 151.

[0065] In this first embodiment, the first camera module 41 acts as the main synchronization module, and then performs the synchronization of the activation of the second camera module in a cascaded manner.

[0066] Upon receiving the first trigger signal S1, the first serialization module 121 sends the first trigger signal S1 to the first camera control unit 101, which then activates the first light source 61 and the first sensor 81. After the first sensor 81 acquires an image, the first light source 61 is turned off, and the first camera control unit 101 is also configured to generate a return synchronization signal S1-o, which is sent to the first deserialization unit 181 via the first serialization module 121 and the serial link 151.

[0067] The first deserialization unit 181 is configured to receive a synchronization signal and send the synchronization signal via, for example, its GPIO port to a second deserialization module of a selected second camera module among a plurality of camera modules, such as the second camera module 42 of level 2.

[0068] The second camera module 42 is configured to receive a synchronization signal S1-o, and to add, for example, a time lag D to the received synchronization signal via the second camera control unit 102 to obtain a second trigger signal S2, which is applied to activate the second camera module 42.

[0069] After the image is acquired by the second sensor 82, the second camera control unit 102 is also configured to generate a return synchronization signal S2-o, which is sent to the second deserialization unit 182 via the second serialization module 122 and the serial link 152.

[0070] When the system includes multiple second camera modules, the operation is repeated sequentially according to the predetermined activation order of the second camera modules, such as according to their corresponding levels.

[0071] According to one implementation scheme, consider any two consecutive camera modules 4 k and 4 k+1 Second deserialization module 18 k Configured to be used via serial link 15 k From the second serialization module 12 k Receive synchronization signal S k -o.

[0072] Second deserialization module 18 k Configured to send a synchronization signal to the second deserialization module 18 k+1 The second deserialization module 18 k+1 It is then configured to send a synchronization signal to camera module 4. k+1 The corresponding serialization module 12 k+1 .

[0073] Second camera module 4 k+1 Configured to receive synchronization signal S k -o, to add, for example, a time lag D to the received synchronization signal in order to obtain a second trigger signal S. k+1 A second trigger signal is applied to activate the second camera module 4. k+1 When the last camera module completes image acquisition, the synchronization signal S3-o is transmitted back to the main control device 22.

[0074] Advantageously, the first implementation scheme only includes hardware communication using a serial communication link.

[0075] according to Figure 3In the second embodiment shown, the main control device 22 is configured to generate continuous trigger signals S1 to S2 according to a predetermined sequence. N To continuously activate camera modules 41 to 4 N Each of these is a first trigger signal that activates the first camera module and a plurality of consecutive second trigger signals that activate multiple second camera modules. Each camera module is activated for a given time period equal to at most a time lag D.

[0076] More generally, two consecutive trigger signals S are generated by applying a time lag D. k and S k+1 And transmit it to the corresponding deserialization module 18 k and 18 k+1 .

[0077] Each deserialization module is configured to send a trigger signal to the corresponding serialization module in order to activate the corresponding camera module.

[0078] According to the second implementation scheme, trigger signals S1...S N Transmitted via a general purpose input / output (GPIO) port, which is connected to the deserialization module 181...18 N and serialization module 121...12 N This serialization module corresponds to each camera module 41...4 N Each serialization module 121...12 N Upon receiving, the trigger signal is transmitted to the camera control unit 101...10 N To activate the camera module.

[0079] Advantageously, in this second embodiment, the delay of the trigger signal is managed by the master control device, thus eliminating the need for a management synchronization signal in the camera module. Therefore, even if an error occurs in one of the multiple camera modules, the management of the activated camera module is ensured.

[0080] according to Figure 4 In the third embodiment shown, the main control device 22 is configured to generate continuous trigger signals C1 to C2. N To activate camera modules 41 to 4 in a predetermined sequence. N Each camera module in the system is activated for a given time lag of D at most.

[0081] In this implementation, the generated trigger signal is in the form of an I2C (“integrated integrated circuit”) command, which is transmitted via the deserialization module 181...18 N The I2C connection port sends data to camera module 41...4 NSerialization module 121...12 N The corresponding I2C connection port. Each serialization module has 121...12 2N Configured to adjust data signals S1...S upon reception N Transmitted to camera control unit 101...10 N This is to activate the camera module.

[0082] In the second implementation scheme, two consecutive trigger signals C are generated by applying a time lag D. k and C k+1 And transmit it to the corresponding deserialization module 18 k and 18 k+1 .

[0083] Advantageously, in this third embodiment, the delay of the trigger signal is managed by the main control device, eliminating the need to manage time lags in the camera modules. Therefore, even if an error occurs in one of the multiple camera modules, the management of the activated camera module is ensured.

Claims

1. An electronic control unit (20) for a vehicle interior monitoring system, the interior monitoring system comprising a plurality of camera modules, the plurality of camera modules comprising: - First camera module (41), the first camera module includes a first light source (61) and a first image sensor (81), the first camera module (41) includes a first camera control unit (121) connected to a first serialization module (141). - At least a second camera module (42, 43), the at least second camera module including a second light source (62, 63) and a second image sensor (82, 83), the second camera module (42, 43) including a second camera control unit (122, 123) connected to a second serialization module (142, 143). The electronic control unit (20) includes a first deserialization module (181) connected to the first serialization module (141) and at least a second deserialization module (182, 183), each second deserialization module being connected to each of the corresponding at least one second serialization module (142, 143). The electronic control unit (20) also includes a master control device (22) connected to the first deserialization module (181) and the second deserialization modules (182, 183). The electronic control unit (20) is configured to manage the activation of each camera module with a time lag relative to the other camera module via at least one of the first deserialization module (181) and the second deserialization modules (182, 183).

2. The electronic control unit according to claim 1, wherein the main control device (22) is configured to activate the first camera module (41) via a first trigger signal (S1) transmitted through the connection between the first deserialization module (181) and the first serialization module (141) to trigger the first camera control unit to activate the first light source (61) and the first image sensor (81).

3. The electronic control unit according to claim 2, wherein the first deserialization module (181) is further configured to receive a synchronization signal from the first serialization module (141) and send the synchronization signal to a second deserialization module (182), the second deserialization module (182) being configured to send the synchronization signal to a corresponding second serialization module (142) of the second camera module (42), the second camera module (42) being configured to obtain a second trigger signal (S2) delayed by the time lag upon receiving the synchronization signal, the second trigger signal being applied to activate the second camera module (42).

4. The electronic control unit of claim 3, wherein the internal monitoring system includes a plurality of second camera modules, and each second deserialization module is configured to receive a synchronization signal from a connected second serialization module and send the synchronization signal to a subsequent second deserialization module connected to a subsequent second camera module, the subsequent second camera module being configured to obtain a subsequent second trigger signal delayed by the time lag upon receiving the synchronization signal, the subsequent second trigger signal being applied to activate the subsequent second camera module.

5. The electronic control unit according to claim 2, wherein the first deserialization module (181) and the at least second deserialization module (182, 183) are connected via a low-voltage differential signaling link (LVDS).

6. The electronic control unit according to claim 1, wherein the main control device (22) is configured to: - Generate a first trigger signal (S1) for activating the first camera module (41), and transmit the first trigger signal for activating the first camera module (41) via the first deserialization module (181), and - Generate at least a second trigger signal (S2, S3) for activating the or each of the second camera modules (42, 43), each second trigger signal being delayed by a total time lag relative to the first trigger signal, the total time lag depending on the level of the second camera module among the plurality of second camera modules, and transmit each second trigger signal via a corresponding second deserialization module (182, 183) to activate the second camera module (42, 43).

7. The electronic control unit according to claim 6, wherein the main control device (22) transmits each of the first trigger signal and the second trigger signal to the GPIO port corresponding to the first camera module and the second camera module via the general purpose input / output port (GPIO) of the corresponding first deserialization module and the second deserialization module.

8. The electronic control unit according to claim 6, wherein the main control device (22) transmits each of the first trigger signal and the second trigger signal in the form of an I2C (integrated integrated circuit) command.

9. A vehicle interior monitoring system, the vehicle interior monitoring system comprising a plurality of camera modules, the plurality of camera modules comprising: - A first camera module (41), the first camera module including a first light source and a first image sensor, the first camera module including a first camera control unit connected to a first serialization module. - At least a second camera module (42, 43), the at least second camera module including a second light source and a second image sensor, the second camera module including a second camera control unit connected to the second serialization module, The internal monitoring system also includes the electronic control unit (20) according to claims 1 to 8.

10. The internal monitoring system according to claim 9, wherein the first light source (61) and / or each second light source (6 2, 63) is an infrared light source.

Images