Image transmission device, image reception device, and image transmission system

Abstract

Provided is an image transmission device capable of serially transferring image data without increasing the number of cables even when the raw data amount of the image data is increased. The image transmission device comprises a signal processing unit, a control unit, a data transmission unit, and a compression unit. The signal processing unit has an image sensor connected thereto, and performs signal processing on image data which is output from the image sensor and includes an object in an imaging field of view in an automobile. The control unit controls the signal processing in the signal processing unit. The data transmission unit transmits, as transmission data to an external device: the image data that has been subjected to the signal processing in the signal processing unit; and control data which is obtained by control processing by the control unit and is necessary for reproducing the image data. The compression unit compresses at least one of the image data and the control data into compressed data having a smaller amount of data than the transmission data, and includes the compressed data in the transmission data.

Description

Image transmitting device, image receiving device, and image transmission system

[0001] The present disclosure (the present technology) relates to an image transmitting device, an image receiving device, and an image transmission system.

[0002] Various systems have been proposed to assist drivers in driving automobiles, motorcycles, and the like (hereinafter referred to as "vehicles") (see, for example, Patent Document 1). In these systems, an RGB camera, radar, or the like mounted on the vehicle captures images of the vehicle's surroundings, a server collects the captured images, and analyzes the collected information to generate traffic-related information (such as accidents and congestion). The server provides this information to the vehicle as dynamic driving assistance information.

[0003] A vehicle is essentially an organic collection of mechanical elements, such as an engine, transmission, airbags, brakes, and steering. Traditionally, these elements have been controlled by mechanical mechanisms to reflect the driver's operations. However, in recent years, vehicles have become increasingly electronic, and various ECUs (Electronic Control Units) that electronically control each part of the vehicle are installed. Examples include an engine control ECU, a stop-start control ECU, a transmission control ECU, an airbag control ECU, a power steering control ECU, and a hybrid control ECU.

[0004] Japanese Patent Application Laid-Open No. 2022-10903

[0005] However, Patent Document 1 discloses a configuration in which the surroundings outside the vehicle are captured and the captured images are collected by a server, but does not disclose a configuration in which the interior of the vehicle is captured and the captured images are transmitted to an ECU.

[0006] The following problems arise when acquiring 3D data inside a vehicle: - Increase in the amount of raw data due to increased pixel size for TOF (Time of Flight) + RGBIR - Increase in the amount of raw data due to the need for real-time processing, which requires 60 frames per second or more As a result, many cables are required for serial transmission of raw data to the ECU, and the increased number of parts leads to higher prices and larger size.

[0007] The present disclosure has been made in consideration of these circumstances, and aims to provide an image sending device, an image receiving device, and an image transmission system that are capable of serially transferring image data without increasing the number of cables, even if the amount of raw image data increases.

[0008] One aspect of the present disclosure is an image transmission device comprising: a signal processing unit that connects an image sensor and performs signal processing on image data output from the image sensor and including an object within an imaging field of view within a vehicle; a control unit that controls the signal processing of the signal processing unit; a data transmission unit that transmits the image data signal-processed by the signal processing unit and control data obtained by control processing of the control unit and necessary for reproducing the image data as transmission data to an external device; and a compression unit that compresses at least one of the image data and the control data into compressed data with a data volume smaller than the transmission data and includes the compressed data in the transmission data.

[0009] Another aspect of the present disclosure is an image receiving device that includes a data receiving unit that receives transmission data sent from an external device, a decoding unit that converts compressed data included in the transmission data back into image data including an object within an imaging field of view inside the automobile or control data required to reproduce the image data, an image reproduction unit that reproduces the image data, and a control unit that controls the reproduction process by the image reproduction unit based on the control data.

[0010] Another aspect of the present disclosure is an image transmission system comprising an image transmitting device, an image receiving device, and a serial transmission path connecting the image transmitting device and the image receiving device, wherein the image transmitting device comprises a signal processing unit to which an image sensor is connected and which performs signal processing on image data output from the image sensor and including an object within an imaging field of view inside the automobile, a transmitting-side control unit which controls the signal processing of the signal processing unit, a data transmitting unit which transmits the image data generated by the signal processing unit and control data obtained by control processing of the control unit and necessary for reproducing the image data to the image receiving device as transmission data, and a compression unit which compresses at least one of the image data and the control data into compressed data having a data amount smaller than the transmission data and includes the compressed data in the transmission data, and the image receiving device comprises a data receiving unit which receives the transmission data transmitted from the image transmitting device, a decoding unit which converts the compressed data included in the transmission data into image data including an object within an imaging field of view inside the automobile or control data necessary for reproducing the image data, an image playback unit which plays back the image data, and a receiving-side control unit which controls the playback processing by the image playback unit based on the control data.

[0011] 1 is a schematic configuration diagram showing the entire image transmission system according to a first embodiment of the present disclosure. FIG. 2 is a block diagram showing the functions of each of a camera module and an ECU of the image transmission system shown in FIG. 1. FIG. 3 is a block diagram showing the functions of each of a camera module and an ECU of an image transmission system according to a first modified example of the first embodiment of the present disclosure. FIG. 4 is a block diagram showing the functions of each of a camera module and an ECU of an image transmission system according to a second modified example of the first embodiment of the present disclosure. FIG. 5 is a block diagram showing the functions of each of a camera module and an ECU of an image transmission system according to a third modified example of the first embodiment of the present disclosure. FIG. 6 is a block diagram showing the functions of each of a camera module and an ECU of an image transmission system according to a fourth modified example of the first embodiment of the present disclosure. FIG. 7 is a block diagram showing the functions of each of a camera module and an ECU of an image transmission system according to a sixth modified example of the first embodiment of the present disclosure. FIG. 8 is a block diagram showing the functions of each of a camera module and an ECU of an image transmission system according to a seventh modified example of the first embodiment of the present disclosure. FIG. 9 is a block diagram showing the functions of each of a camera module and an ECU of an image transmission system according to an eighth modified example of the first embodiment of the present disclosure. FIG. 11 is a block diagram showing the functions of each of a camera module and an ECU of an image transmission system according to a ninth modified example of the first embodiment of the present disclosure. FIG. 12 is a schematic configuration diagram showing the entire image transmission system according to a second embodiment of the present disclosure. FIG. 1 is a block diagram showing a camera module and each function of an image transmission system according to a first modified example of the second embodiment of the present disclosure; FIG. 2 is a block diagram showing a camera module and each function of an image transmission system according to a second modified example of the second embodiment of the present disclosure; FIG. 3 is a block diagram showing a camera module and each function of an image transmission system according to a third modified example of the second embodiment of the present disclosure; FIG. 4 is a block diagram showing a camera module and each function of an image transmission system according to a fourth modified example of the second embodiment of the present disclosure; FIG. 5 is a block diagram showing a camera module and each function of an image transmission system according to a fifth modified example of the second embodiment of the present disclosure;19 is a flowchart showing a control procedure for executing image compression control processing by an ECU according to a third embodiment of the present disclosure. FIG. 19 is a flowchart showing a detailed control procedure of step ST19g shown in FIG. 18. FIG. 19 is a screen diagram (part 1) showing an image compression method according to an operating state of a vehicle according to a third embodiment of the present disclosure. FIG. 20 is a screen diagram (part 2) showing an image compression method according to an operating state of a vehicle according to a third embodiment of the present disclosure. FIG. 21 is a screen diagram (part 3) showing an image compression method according to an operating state of a vehicle according to a third embodiment of the present disclosure. FIG. 22 is a screen diagram (part 4) showing an image compression method according to an operating state of a vehicle according to a third embodiment of the present disclosure. FIG. 23 is a screen diagram (part 5) showing an image compression method according to an operating state of a vehicle according to a third embodiment of the present disclosure. FIG. 24 is a schematic configuration diagram showing a partial example of an image transmission system according to a fourth embodiment of the present disclosure. FIG. 25 is a schematic configuration diagram (part 1) showing a partial example of an image transmission system according to a comparative example of the fourth embodiment of the present disclosure. FIG. 26 is a schematic configuration diagram (part 2) showing a partial example of an image transmission system according to a comparative example of the fourth embodiment of the present disclosure. FIG. 27 is a schematic configuration diagram showing a partial example of an image transmission system according to a first modified example of the fourth embodiment of the present disclosure. FIG. 1 is a schematic configuration diagram showing a partial example of an image transmission system according to a second modified example of the fourth embodiment of the present disclosure. FIG. 2 is a schematic configuration diagram showing a partial example of an image transmission system according to a third modified example of the fourth embodiment of the present disclosure. FIG. 3 is a schematic configuration diagram showing a partial example of an image transmission system according to a fourth modified example of the fourth embodiment of the present disclosure. FIG. 4 is a schematic configuration diagram showing a partial example of an image transmission system according to a fifth modified example of the fourth embodiment of the present disclosure. FIG. 5 is a schematic configuration diagram showing a partial example of an image transmission system according to a fifth modified example of the fourth embodiment of the present disclosure. FIG. 6 is a schematic configuration diagram showing a partial example of an image transmission system according to a seventh modified example of the fourth embodiment of the present disclosure. FIG. 7 is a schematic configuration diagram showing a partial example of an image transmission system according to a fifth embodiment of the present disclosure. FIG. 8 is a schematic configuration diagram showing a partial example of an image transmission system according to a first modified example of the fifth embodiment of the present disclosure. FIG. 9 is a schematic configuration diagram showing a partial example of an image transmission system according to a second modified example of the fifth embodiment of the present disclosure. FIG. 10 is a schematic configuration diagram showing a partial example of an image transmission system according to a third modified example of the fifth embodiment of the present disclosure.FIG. 43 is a schematic configuration diagram showing a partial example of an image transmission system according to a fourth modified example of the fifth embodiment of the present disclosure. FIG. 44 is a schematic configuration diagram showing a partial example of an image transmission system according to a fifth modified example of the fifth embodiment of the present disclosure. FIG. 45 is a schematic configuration diagram showing a partial example of an image transmission system according to a sixth embodiment of the present disclosure. FIG. 46 is a schematic configuration diagram showing a partial example of an image transmission system according to a modified example of the sixth embodiment of the present disclosure. FIG. 47 is a flowchart showing a control procedure for executing image compression control processing by an ECU according to a seventh embodiment of the present disclosure. FIG. 48 is a flowchart showing a detailed control procedure of step ST43f shown in FIG. 43. FIG. 49 is a screen diagram (part 1) showing an image compression method according to the operating state of a vehicle in the eighth embodiment of the present disclosure. FIG. 50 is a screen diagram (part 5) showing an image compression method according to the operating state of a vehicle in the eighth embodiment of the present disclosure. FIG. 23 is a screen diagram (part 6) illustrating an image compression method according to the operating state of a vehicle in the eighth embodiment of the present disclosure.

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following description of the drawings, identical or similar parts will be designated by identical or similar reference numerals, and redundant description will be omitted. However, it should be noted that the drawings are schematic, and the relationship between thickness and planar dimensions, the thickness ratio of each device or component, etc., may differ from the actual ones. Therefore, specific thicknesses and dimensions should be determined with reference to the following description. Furthermore, it goes without saying that there are parts with different dimensional relationships and ratios between the drawings. Note that the effects described in this specification are merely examples and are not limited thereto, and other effects may also be present.

[0013] <First embodiment> Fig. 1 is a schematic configuration diagram showing the entire image transmission system 1A according to a first embodiment of the present disclosure. The image transmission system 1A in Fig. 1 includes a camera module 2 and an ECU 3. The camera module 2 and the ECU 3 are connected by a serial cable 4. The camera module 2 is mounted inside a vehicle 5. The camera module 2 is connected to an indirect TOF ranging sensor 21 (hereinafter referred to as an iTOF sensor 21) and an RGB sensor 22, which will be described later.

[0014] In the first embodiment of the present disclosure, an example will be described in which two types of image sensors, an iTOF sensor 21 and an RGB sensor 22, having different detection characteristics are used to integrate (fuse) information about an object recognized based on the output of each image sensor to recognize the object with high accuracy.

[0015] The iTOF sensor 21 has an imaging range that covers the area inside the vehicle 5 from the front to the rear. The RGB sensor 22 has an imaging range that covers the area inside the vehicle 5 from the front to the rear. The iTOF sensor 21 irradiates infrared light, receives infrared light reflected from an object, generates and accumulates electric charges according to the amount of infrared light received, and generates and outputs a three-dimensional image signal of an imaging field of view at, for example, 30 frames per second according to the amount of accumulated electric charges. The RGB sensor 22 generates and accumulates electric charges according to the amount of visible light received from the imaging field of view, and generates and outputs a two-dimensional image signal of an imaging field of view at, for example, 30 frames per second according to the amount of accumulated electric charges. The RGB sensor 22 may be an RGBIR sensor that can detect infrared light in addition to visible light.

[0016] 2 is a block diagram showing the functions of the camera module 2 and ECU 3 of the image transmission system 1A. The iTOF sensor 21 of the camera module 2 includes a logic circuit 211 as a signal processing unit and an encoding circuit 212 as a compression unit. The logic circuit 211 processes the image signal output from the iTOF sensor 21 and determines a feature value for identifying an object based on the image signal. This feature value may be, for example, a distance coordinate value to the object. The encoding circuit 212 compresses the image data output from the logic circuit 211 (referred to as iTOF image data) using lossless or lossy compression, and outputs the compressed data (referred to as iTOF compressed image data) to the serializer circuit 25.

[0017] The RGB sensor 22 includes a logic circuit 221 as a signal processing unit and an encoding circuit 222 as a compression unit. The logic circuit 221 processes the image signal output from the RGB sensor 22 and obtains feature quantities for identifying an object based on the image signal. These feature quantities include, for example, coordinate values ​​indicating the position of the object and color information of the image of the object. The encoding circuit 222 compresses the image data (referred to as RGB image data) output from the logic circuit 221 using lossless or lossy compression, and outputs the compressed data (referred to as RGB compressed image data) to the serializer circuit 25.

[0018] The camera module 2 includes control circuits 23 and 24. The iTOF sensor 21, the RGB sensor 22, the serializer circuit 25, and the control circuits 23 and 24 are interconnected by a control bus 29 conforming to the I2C (Inter-Integrated Circuit) standard. The control circuit 23 controls the signal processing of the logic circuit 211. The control circuit 23 outputs control data (referred to as iTOF control data) necessary for image reproduction of the iTOF image data to the serializer circuit 25 via the control bus 29. This iTOF control data includes distance data to the target object. The control circuit 24 controls the signal processing of the logic circuit 221. The control circuit 24 outputs control data (referred to as RGB control data) necessary for image reproduction of the RGB image data to the serializer circuit 25 via the control bus 29. This RGB control data includes coordinate position data of the target object and color information of the image of the target object.

[0019] The serializer circuit 25 converts the parallel data of the iTOF compressed image data, iTOF control data, RGB compressed image data, and RGB control data into serial data and transmits the serial data to the ECU 3 via the serial cable 4. The serial cable 4 includes a bidirectional control interface compatible with the I2C (Inter-Integrated Circuit) standard.

[0020] The ECU 3 includes a deserializer circuit 31, a decode circuit 32, an image reproduction circuit 33, and a control circuit 34. The deserializer circuit 31, the decode circuit 32, the image reproduction circuit 33, and the control circuit 34 are interconnected by an I2C control bus 36. The deserializer circuit 31 converts serial data (iTOF compressed image data, iTOF control data, RGB compressed image data, RGB control data) transmitted from the camera module 2 via the serial cable 4 into parallel data. The decode circuit 32 decodes the iTOF compressed image data and RGB compressed image data to restore them to iTOF image data and RGB image data. The iTOF control data and RGB control data are output to the control circuit 34 via the control bus 36.

[0021] The image reproduction circuit 33 reproduces the iTOF image data and the RGB image data, and recognizes the object by fusing information on the object recognized based on each iTOF image and the RGB image. The control circuit 34 controls the reproduction process by the image reproduction circuit 33 based on the iTOF control data and the RGB control data.

[0022] (Example of Operation of Image Transmission System 1A) For example, when the control circuit 34 of the ECU 3 detects that the engine of the vehicle 5 has started or the ignition has been turned on, the control circuit 34 transmits startup control data to the iTOF control circuit 23 of the camera module 2 for starting the iTOF sensor 21, and transmits startup control data to the RGB control circuit 24 of the camera module 2 for starting the RGB sensor 22. The startup control data is input to the iTOF control circuit 23 and the RGB control circuit 24 via the control bus 36, the deserializer circuit 31, the serial cable 4, the serializer circuit 25, and the control bus 29. The iTOF control circuit 23 then activates the iTOF sensor 21 based on the startup control data sent from the ECU 3. When the iTOF sensor 21 is activated, the iTOF control circuit 23 transmits Ack (response) data indicating that the activation was successful to the control circuit 34 of the ECU 3. The Ack data is transmitted to the control circuit 34 of the ECU 3 via the control bus 29, the serializer circuit 25, the serial cable 4, the deserializer circuit 31, and the control bus 36. The RGB control circuit 24 also activates the RGB sensor 22 based on the activation control data transmitted from the ECU 3. When the RGB sensor 22 is activated, the RGB control circuit 24 transmits Ack data indicating that the RGB sensor 22 has been activated correctly to the control circuit 34 of the ECU 3. The Ack data is transmitted to the control circuit 34 of the ECU 3 via the control bus 29, the serializer circuit 25, the serial cable 4, the deserializer circuit 31, and the control bus 36.

[0023] Next, the control circuit 34 of the ECU 3 transmits control data including, for example, lossless compression to the iTOF control circuit 23 of the camera module 2, and transmits control data including, for example, lossless compression to the RGB control circuit 24 of the camera module 2, depending on the state of the vehicle 5. The control data including lossless compression is input to the iTOF control circuit 23 and the RGB control circuit 24 via the control bus 36, the deserializer circuit 31, the serial cable 4, the serializer circuit 25, and the control bus 29. The iTOF control circuit 23 then sets the image compression method of the encoding circuit 212 to lossless compression based on the control data including lossless compression sent from the ECU 3. When the encoding circuit 212 is set to lossless compression, the iTOF control circuit 23 transmits Ack data indicating that the setting has been successful to the control circuit 34 of the ECU 3. The RGB control circuit 24 also sets the image compression method of the encoding circuit 222 to lossless compression based on the control data including lossless compression sent from the ECU 3. When the encoding circuit 222 is set to lossless compression, the RGB control circuit 24 transmits Ack data indicating that the setting is correct to the control circuit 34 of the ECU 3. Note that the control data including lossless compression may be input directly to the encoding circuits 212, 222 without passing through the iTOF control circuit 23 and the RGB control circuit 24.

[0024] Note that the above camera module 2 has been described as an example in which iTOF image data is compressed into iTOF compressed image data by lossless or lossy compression, and RGB image data is compressed into RGB compressed image data by lossless or lossy compression, but it may also be an example in which only RGB image data is compressed into RGB compressed image data by lossless or lossy compression.

[0025] <Effects of First Embodiment> As described above, according to the first embodiment, in the camera module 2, iTOF image data is compressed into iTOF compressed image data by lossless compression or lossy compression, and RGB image data is compressed into RGB compressed image data by lossless compression or lossy compression, and these are transmitted to the ECU 3 via the serial cable 4 in a small amount of data. This makes it possible to significantly reduce the amount of data transmitted to the ECU 3 compared to transmitting the iTOF image data and RGB image data directly via the serial cable 4. As a result, even if the amount of raw data of each of the iTOF image data, iTOF control data, RGB image data, and RGB control data increases, it becomes possible to serially transfer the iTOF image data, iTOF control data, RGB image data, and RGB control data without increasing the number of serial cables 4.

[0026] Furthermore, according to the first embodiment, the ECU 3 can acquire iTOF image data and RGB image data from the camera module 2 and play them back in real time.

[0027] <First Modification of First Embodiment> Fig. 3 is a block diagram showing the functions of the camera module 2 and the ECU 3 of an image transmission system 1B according to a first modification of the first embodiment. In Fig. 3, the same parts as those in Fig. 2 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0028] The iTOF sensor 21 is provided with a serializer circuit 213. The RGB sensor 22 is provided with a serializer circuit 223. The serializer circuit 213 converts parallel data, iTOF compressed image data and iTOF control data, into serial data and transmits the serial data to the ECU 3 via a serial cable 41. The serializer circuit 223 converts parallel data, RGB compressed image data and RGB control data, into serial data and transmits the serial data to the ECU 3 via a serial cable 42. The logic circuit 211, the encoding circuit 212, the serializer circuit 213, and the control circuit 23 are interconnected by a control bus 291 conforming to the I2C standard. The logic circuit 221, the encoding circuit 222, the serializer circuit 223, and the control circuit 24 are interconnected by a control bus 292 conforming to the I2C standard.

[0029] The ECU 3 has a deserializer circuit 31. The deserializer circuit 31 may be composed of two deserializer circuits: a deserializer circuit 31-1 that converts serial data transmitted from the serializer circuit 213 into parallel data, and a deserializer circuit 31-2 that converts serial data transmitted from the serializer circuit 223 into parallel data.

[0030] (Example of Operation of Image Transmission System 1B) For example, when the control circuit 34 of the ECU 3 detects that the engine of the vehicle 5 has started or that the ignition has been turned on, the control circuit 34 transmits startup control data to the iTOF control circuit 23 of the camera module 2 for starting the iTOF sensor 21, and transmits startup control data to the RGB control circuit 24 of the camera module 2 for starting the RGB sensor 22. The startup control data for the iTOF sensor 21 is input to the iTOF control circuit 23 via the control bus 36, the deserializer circuit 31, the serial cable 41, the serializer circuit 213, and the control bus 291. The iTOF control circuit 23 then activates the iTOF sensor 21 based on the startup control data transmitted from the ECU 3. When the iTOF sensor 21 is activated, the iTOF control circuit 23 transmits Ack (response) data indicating that the activation has been successful to the control circuit 34 of the ECU 3. The ACK data is transmitted to the control circuit 34 of the ECU 3 via the control bus 291, the serializer circuit 213, the serial cable 41, the deserializer circuit 31, and the control bus 36. Furthermore, activation control data for the RGB sensor 22 is input to the RGB control circuit 24 via the control bus 36, the deserializer circuit 31, the serial cable 42, the serializer circuit 223, and the control bus 292. The RGB control circuit 24 then activates the RGB sensor 22 based on the activation control data transmitted from the ECU 3. When the RGB sensor 22 is activated, the RGB control circuit 24 transmits ACK (response) data indicating that the activation has been successful to the control circuit 34 of the ECU 3. The ACK data is transmitted to the control circuit 34 of the ECU 3 via the control bus 292, the serializer circuit 223, the serial cable 42, the deserializer circuit 31, and the control bus 36.

[0031] Meanwhile, the control circuit 34 of the ECU 3 transmits control data including, for example, lossless compression to the iTOF control circuit 23 of the camera module 2 in accordance with the state of the vehicle 5. This control data including lossless compression is input to the iTOF control circuit 23 via the control bus 36, the deserializer circuit 31, the serial cable 41, the serializer circuit 213, and the control bus 291. The iTOF control circuit 23 then sets the image compression method of the encoding circuit 212 to lossless compression based on the control data including lossless compression sent from the ECU 3. When the encoding circuit 212 is set to lossless compression, the iTOF control circuit 23 transmits Ack data indicating that the setting has been successful to the control circuit 34 of the ECU 3. Note that the control data including lossless compression may be input directly to the encoding circuit 212 without passing through the iTOF control circuit 23.

[0032] On the other hand, the control circuit 34 of the ECU 3 transmits control data including, for example, lossy compression to the RGB control circuit 24 of the camera module 2 in accordance with the state of the vehicle 5. This control data including lossy compression is input to the RGB control circuit 24 via the control bus 36, the deserializer circuit 31, the serial cable 42, the serializer circuit 223, and the control bus 292. The RGB control circuit 24 then sets the image compression method of the encoding circuit 222 to lossy compression based on the control data including lossy compression sent from the ECU 3. When the encoding circuit 222 is set to lossy compression, the RGB control circuit 24 transmits Ack data indicating that the setting has been successful to the control circuit 34 of the ECU 3. Note that the control data including lossy compression may be input directly to the encoding circuit 222 without passing through the RGB control circuit 24.

[0033] <Operational Effects of the First Modification of the First Embodiment> The first modification of the first embodiment also provides the same operational effects as the first embodiment.

[0034] <Second Modification of First Embodiment> Fig. 4 is a block diagram showing the functions of the camera module 2 and the ECU 3 of an image transmission system 1C according to a second modification of the first embodiment. In Fig. 4, the same parts as those in Fig. 2 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0035] The camera module 2 of the image transmission system 1C includes an encoding circuit 26. The encoding circuit 26 compresses the RGB control data output from the control circuit 24 by lossless compression or lossy compression, and outputs the compressed RGB control data to the serializer circuit 25.

[0036] The serializer circuit 25 converts the iTOF compressed image data, iTOF control data, RGB image data, and RGB compressed control data from parallel data into serial data, and transmits the serial data to the ECU 3 via the serial cable 4 .

[0037] The deserializer circuit 31 converts the serial data transmitted from the camera module 2 via the serial cable 4 into parallel data, iTOF compressed image data, iTOF control data, RGB image data, and RGB compression control data. The decode circuit 32 decodes the iTOF compressed image data and RGB compression control data to restore them to iTOF image data and RGB image data.

[0038] The iTOF sensor 21, the RGB sensor 22, the serializer circuit 25, the control circuits 23 and 24, and the encoding circuit 26 are connected to one another by a control bus 29 conforming to the I2C standard.

[0039] (Example of Operation of Image Transmission System 1C) For example, when the control circuit 34 of the ECU 3 detects that the engine of the vehicle 5 has started or the ignition has been turned on, the control circuit 34 transmits startup control data to the iTOF control circuit 23 of the camera module 2 for starting the iTOF sensor 21, and transmits startup control data to the RGB control circuit 24 of the camera module 2 for starting the RGB sensor 22. The startup control data is input to the iTOF control circuit 23 and the RGB control circuit 24 via the control bus 36, the deserializer circuit 31, the serial cable 4, the serializer circuit 25, and the control bus 29. The iTOF control circuit 23 then activates the iTOF sensor 21 based on the startup control data sent from the ECU 3. When the iTOF sensor 21 is activated, the iTOF control circuit 23 transmits Ack (response) data indicating that the activation was successful to the control circuit 34 of the ECU 3. The ACK data is transmitted to the control circuit 34 of the ECU 3 via the control bus 29, the serializer circuit 25, the serial cable 4, the deserializer circuit 31, and the control bus 36. The RGB control circuit 24 also activates the RGB sensor 22 based on the activation control data transmitted from the ECU 3. At this time, the RGB control circuit 24 sets the power supply of the encode circuit 26 to the on state. When the RGB sensor 22 is activated, the RGB control circuit 24 transmits ACK data indicating that the activation has been successful to the control circuit 34 of the ECU 3. The ACK data is transmitted to the control circuit 34 of the ECU 3 via the control bus 29, the serializer circuit 25, the serial cable 4, the deserializer circuit 31, and the control bus 36.

[0040] Next, the control circuit 34 of the ECU 3 transmits control data including, for example, lossless compression to the iTOF control circuit 23 of the camera module 2, and transmits control data including, for example, lossless compression to the RGB control circuit 24 of the camera module 2, depending on the state of the vehicle 5. The control data including lossless compression is input to the iTOF control circuit 23 and the RGB control circuit 24 via the control bus 36, the deserializer circuit 31, the serial cable 4, the serializer circuit 25, and the control bus 29. The iTOF control circuit 23 then sets the image compression method of the encoding circuit 212 to lossless compression based on the control data including lossless compression sent from the ECU 3. When the encoding circuit 212 is set to lossless compression, the iTOF control circuit 23 transmits Ack data indicating that the setting has been successful to the control circuit 34 of the ECU 3. The RGB control circuit 24 also sets the image compression method of the encoding circuit 26 to lossless compression based on the control data including lossless compression sent from the ECU 3. When the encoding circuit 26 is set to lossless compression, the RGB control circuit 24 transmits Ack data indicating that the setting is correct to the control circuit 34 of the ECU 3. Note that the control data including lossless compression may be input directly to the encoding circuits 212 and 26 without passing through the iTOF control circuit 23 and the RGB control circuit 24.

[0041] Note that the above camera module 2 has been described as an example in which iTOF image data is compressed into iTOF compressed image data by lossless compression or lossy compression, and RGB control data is compressed into RGB compressed control data by lossless compression or lossy compression, but it may also be an example in which only iTOF image data is compressed into iTOF compressed image data by lossless compression or lossy compression.

[0042] <Operational Effects of the Second Modification of the First Embodiment> The second modification of the first embodiment also provides the same operational effects as the first embodiment.

[0043] <Third Modification of First Embodiment> Fig. 5 is a block diagram showing the functions of the camera module 2 and the ECU 3 of an image transmission system 1D according to a third modification of the first embodiment. In Fig. 5, the same parts as those in Fig. 2 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0044] The camera module 2 of the image transmission system 1D includes an encoding circuit 27. The encoding circuit 27 compresses the iTOF control data output from the control circuit 23 using lossless compression or lossy compression, and outputs the iTOF compressed control data to the serializer circuit 25.

[0045] The serializer circuit 25 converts the iTOF image data, iTOF compressed control data, RGB compressed image data, and RGB control data from parallel data into serial data, and transmits the serial data to the ECU 3 via the serial cable 4 .

[0046] The deserializer circuit 31 converts the serial data transmitted from the camera module 2 via the serial cable 4 into parallel data, iTOF image data, iTOF compression control data, RGB compressed image data, and RGB control data. The decode circuit 32 decodes the iTOF compression control data and RGB compressed image data back into iTOF control data and RGB image data.

[0047] The iTOF sensor 21, the RGB sensor 22, the serializer circuit 25, the control circuits 23 and 24, and the encoding circuit 27 are connected to one another by a control bus 29 conforming to the I2C standard.

[0048] (Example of Operation of Image Transmission System 1D) For example, when the control circuit 34 of the ECU 3 detects engine start or ignition on of the vehicle 5, it transmits startup control data to the iTOF control circuit 23 of the camera module 2 for activating the iTOF sensor 21, and transmits startup control data to the RGB control circuit 24 of the camera module 2 for activating the RGB sensor 22. The startup control data is input to the iTOF control circuit 23 and the RGB control circuit 24 via the control bus 36, the deserializer circuit 31, the serial cable 4, the serializer circuit 25, and the control bus 29. The iTOF control circuit 23 then activates the iTOF sensor 21 based on the startup control data sent from the ECU 3. At this time, the iTOF control circuit 23 sets the power supply of the encoding circuit 27 to an ON state. When the iTOF sensor 21 is activated, the iTOF control circuit 23 transmits Ack (response) data indicating that the activation was successful to the control circuit 34 of the ECU 3. The Ack data is transmitted to the control circuit 34 of the ECU 3 via the control bus 29, the serializer circuit 25, the serial cable 4, the deserializer circuit 31, and the control bus 36. The RGB control circuit 24 also activates the RGB sensor 22 based on the activation control data transmitted from the ECU 3. When the RGB sensor 22 is activated, the RGB control circuit 24 transmits Ack data indicating that the RGB sensor 22 has been activated correctly to the control circuit 34 of the ECU 3. The Ack data is transmitted to the control circuit 34 of the ECU 3 via the control bus 29, the serializer circuit 25, the serial cable 4, the deserializer circuit 31, and the control bus 36.

[0049] Next, the control circuit 34 of the ECU 3 transmits control data including, for example, lossless compression to the iTOF control circuit 23 of the camera module 2, and transmits control data including, for example, lossless compression to the RGB control circuit 24 of the camera module 2, depending on the state of the vehicle 5. The control data including lossless compression is input to the iTOF control circuit 23 and the RGB control circuit 24 via the control bus 36, the deserializer circuit 31, the serial cable 4, the serializer circuit 25, and the control bus 29. The iTOF control circuit 23 then sets the image compression method of the encoding circuit 27 to lossless compression based on the control data including lossless compression sent from the ECU 3. When the encoding circuit 27 is set to lossless compression, the iTOF control circuit 23 transmits Ack data indicating that the setting has been successful to the control circuit 34 of the ECU 3. The RGB control circuit 24 also sets the image compression method of the encoding circuit 222 to lossless compression based on the control data including lossless compression sent from the ECU 3. When the encoding circuit 222 is set to lossless compression, the RGB control circuit 24 transmits Ack data indicating that the setting is correct to the control circuit 34 of the ECU 3. Note that the control data including lossless compression may be input directly to the encoding circuits 222 and 27 without passing through the iTOF control circuit 23 and the RGB control circuit 24.

[0050] Note that the above camera module 2 has been described as an example in which iTOF control data is compressed into iTOF compressed control data by lossless compression or lossy compression, and RGB image data is compressed into RGB compressed image data by lossless compression or lossy compression, but it may also be an example in which only RGB image data is compressed into RGB compressed image data by lossless compression or lossy compression.

[0051] <Operational Effects of the Third Modification of the First Embodiment> The third modification of the first embodiment also provides the same operational effects as the first embodiment.

[0052] <Fourth Modification of First Embodiment> Fig. 6 is a block diagram showing the functions of the camera module 2 and the ECU 3 of an image transmission system 1E according to a fourth modification of the first embodiment. In Fig. 6, the same parts as those in Fig. 2 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0053] The camera module 2 of the image transmission system 1E includes an encoding circuit 28. The encoding circuit 28 compresses the iTOF control data output from the control circuit 23 by lossless compression or lossy compression, and outputs the iTOF compressed control data to the serializer circuit 25. The encoding circuit 28 also compresses the RGB control data output from the control circuit 24 by lossless compression or lossy compression, and outputs the RGB compressed control data to the serializer circuit 25.

[0054] The serializer circuit 25 converts the iTOF image data, iTOF compression control data, RGB image data, and RGB compression control data from parallel data into serial data, and transmits the serial data to the ECU 3 via the serial cable 4 .

[0055] The deserializer circuit 31 converts the serial data transmitted from the camera module 2 via the serial cable 4 into parallel data, such as iTOF image data, iTOF compression control data, RGB image data, and RGB compression control data. The decode circuit 32 decodes the iTOF compression control data and RGB compression control data to restore them to iTOF control data and RGB control data.

[0056] The iTOF sensor 21, the RGB sensor 22, the serializer circuit 25, the control circuits 23 and 24, and the encoding circuit 28 are connected to one another by a control bus 29 conforming to the I2C standard.

[0057] (Example of Operation of Image Transmission System 1E) For example, when the control circuit 34 of the ECU 3 detects engine start or ignition on of the vehicle 5, it transmits startup control data to the iTOF control circuit 23 of the camera module 2 for activating the iTOF sensor 21, and transmits startup control data to the RGB control circuit 24 of the camera module 2 for activating the RGB sensor 22. The startup control data is input to the iTOF control circuit 23 and the RGB control circuit 24 via the control bus 36, the deserializer circuit 31, the serial cable 4, the serializer circuit 25, and the control bus 29. The iTOF control circuit 23 then activates the iTOF sensor 21 based on the startup control data sent from the ECU 3. At this time, the iTOF control circuit 23 sets the power supply of the encoding circuit 28 to an ON state. When the iTOF sensor 21 is activated, the iTOF control circuit 23 transmits Ack (response) data indicating that the activation was successful to the control circuit 34 of the ECU 3. The Ack data is transmitted to the control circuit 34 of the ECU 3 via the control bus 29, the serializer circuit 25, the serial cable 4, the deserializer circuit 31, and the control bus 36. The RGB control circuit 24 also activates the RGB sensor 22 based on the activation control data transmitted from the ECU 3. When the RGB sensor 22 is activated, the RGB control circuit 24 transmits Ack data indicating that the RGB sensor 22 has been activated correctly to the control circuit 34 of the ECU 3. The Ack data is transmitted to the control circuit 34 of the ECU 3 via the control bus 29, the serializer circuit 25, the serial cable 4, the deserializer circuit 31, and the control bus 36.

[0058] Meanwhile, the control circuit 34 of the ECU 3 transmits control data including, for example, lossless compression to the iTOF control circuit 23 of the camera module 2 in accordance with the state of the vehicle 5. This control data including lossless compression is input to the iTOF control circuit 23 via the control bus 36, the deserializer circuit 31, the serial cable 4, the serializer circuit 25, and the control bus 29. The iTOF control circuit 23 then sets the image compression method of the encoding circuit 28 to lossless compression based on the control data including lossless compression sent from the ECU 3. When the encoding circuit 28 is set to lossless compression, the iTOF control circuit 23 transmits Ack data indicating that the setting has been successful to the control circuit 34 of the ECU 3. Note that the control data including lossless compression may be input directly to the encoding circuit 28 without passing through the iTOF control circuit 23.

[0059] On the other hand, the control circuit 34 of the ECU 3 transmits control data including, for example, lossy compression to the RGB control circuit 24 of the camera module 2 in accordance with the state of the vehicle 5. This control data including lossy compression is input to the RGB control circuit 24 via the control bus 36, the deserializer circuit 31, the serial cable 4, the serializer circuit 25, and the control bus 29. The RGB control circuit 24 then sets the image compression method of the encoding circuit 28 to lossy compression based on the control data including lossy compression sent from the ECU 3. When the encoding circuit 28 is set to lossy compression, the RGB control circuit 24 transmits Ack data indicating that the setting has been successful to the control circuit 34 of the ECU 3. Note that the control data including lossy compression may be input directly to the encoding circuit 28 without passing through the RGB control circuit 24.

[0060] Note that the above camera module 2 has been described as an example in which iTOF control data is compressed into iTOF compression control data by lossless compression or lossy compression, and RGB control data is compressed into RGB compression control data by lossless compression or lossy compression, but it may also be an example in which only iTOF control data is compressed into iTOF compression control data by lossless compression or lossy compression.

[0061] <Operational Effects of the Fourth Modification of the First Embodiment> The fourth modification of the first embodiment also provides the same operational effects as the first embodiment.

[0062] <Fifth Modification of First Embodiment> Fig. 7 is a block diagram showing the functions of the camera module 2 and the ECU 3 of an image transmission system 1F according to a fifth modification of the first embodiment. In Fig. 7, the same parts as those in Fig. 2 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0063] In the camera module 2 of the image transmission system 1F, the iTOF sensor 21 is provided with an encoding / encryption circuit 214. The RGB sensor 22 is provided with an encoding / encryption circuit 224. The encoding / encryption circuit 214 compresses the iTOF image data output from the logic circuit 211 using lossless or lossy compression, encrypts the data with encryption key information, and outputs the encrypted compressed data (referred to as iTOF encrypted compressed image data) to the serializer circuit 25. The encryption key information is included in the iTOF control data. Furthermore, the encryption key information may be, for example, information determined by the user.

[0064] The encoding / encryption circuit 224 compresses the RGB image data output from the logic circuit 221 using lossless or lossy compression, encrypts the data using encryption key information, and outputs the encrypted compressed data (referred to as RGB encrypted compressed image data) to the serializer circuit 25. The encryption key information is included in the RGB control data.

[0065] The serializer circuit 25 converts the iTOF encrypted compressed image data, the iTOF control data, the RGB encrypted compressed image data, and the RGB control data from parallel data into serial data, and transmits the serial data to the ECU 3 via the serial cable 4 .

[0066] The ECU 3 is equipped with a decode / decryption circuit 35. The deserializer circuit 31 converts serial data transmitted from the camera module 2 via the serial cable 4 into parallel data: iTOF encrypted compressed image data, iTOF control data, RGB encrypted compressed image data, and RGB control data. The decode / decryption circuit 35 decodes the iTOF encrypted compressed image data and RGB encrypted compressed image data based on encryption key information included in the iTOF control data and RGB control data, and decodes the encrypted data based on the encryption key information included in the iTOF control data and RGB control data to restore the iTOF image data and RGB image data.

[0067] In the camera module 2, the iTOF sensor 21, the RGB sensor 22, the serializer circuit 25, and the control circuits 23 and 24 are interconnected by an I2C control bus 29. In the ECU 3, the deserializer circuit 31, the image reproduction circuit 33, the control circuit 34, and the decode / decryption circuit 35 are interconnected by an I2C control bus 36.

[0068] (Example of Operation of Image Transmission System 1F) When the image transmission system 1F is started, it operates in the same manner as the image transmission system 1A. The control circuit 34 of the ECU 3 transmits control data including, for example, lossless compression to the iTOF control circuit 23 of the camera module 2, and transmits control data including, for example, lossless compression to the RGB control circuit 24 of the camera module 2, depending on the state of the vehicle 5. This control data including lossless compression is input to the iTOF control circuit 23 and the RGB control circuit 24 via the control bus 36, the deserializer circuit 31, the serial cable 4, the serializer circuit 25, and the control bus 29. The iTOF control circuit 23 then sets the image compression method of the encoding / encryption circuit 214 to lossless compression based on the control data including lossless compression sent from the ECU 3. When the encoding / encryption circuit 214 is set to lossless compression, the iTOF control circuit 23 transmits Ack data indicating that the setting is correct to the control circuit 34 of the ECU 3. Furthermore, the RGB control circuit 24 sets the image compression method of the encoding / encryption circuit 224 to lossless compression based on control data including lossless compression sent from the ECU 3. When the encoding / encryption circuit 224 is set to lossless compression, the RGB control circuit 24 transmits Ack data indicating that the setting is correct to the control circuit 34 of the ECU 3. Note that the control data including lossless compression may be input directly to the encoding / encryption circuits 214, 224 without passing through the iTOF control circuit 23 and the RGB control circuit 24.

[0069] Note that the above camera module 2 has been described as an example in which iTOF image data is converted into iTOF encrypted compressed image data and RGB image data is converted into RGB encrypted compressed image data, but it may also be an example in which only RGB image data is compressed into RGB encrypted compressed image data.

[0070] <Effects of the fifth variant of the first embodiment> According to the fifth variant of the first embodiment, the same effects as those of the first embodiment can be obtained, and in addition, in order to protect against external hacking, by encrypting the iTOF image data and RGB image data, even if the amount of data increases, it becomes possible to serially transfer the iTOF encrypted compressed image data and RGB encrypted compressed image data without increasing the number of serial cables 4, thereby strengthening security.

[0071] <Sixth Modification of First Embodiment> Fig. 8 is a block diagram showing the functions of the camera module 2 and the ECU 3 of an image transmission system 1G according to a sixth modification of the first embodiment. In Fig. 8, the same parts as those in Figs. 3 and 7 are designated by the same reference numerals and detailed description thereof will be omitted.

[0072] The iTOF sensor 21 is provided with a serializer circuit 213 on the same board as the board on which the logic circuit 211 and the encoding / encryption circuit 214 are mounted. The RGB sensor 22 is provided with a serializer circuit 223 on the same board as the board on which the logic circuit 221 and the encoding / encryption circuit 224 are mounted. The serializer circuit 213 converts the iTOF encrypted compressed image data and the iTOF control data from parallel data to serial data, and transmits the serial data to the ECU 3 via the serial cable 41. The serializer circuit 223 converts the RGB encrypted compressed image data and the RGB control data from parallel data to serial data, and transmits the serial data to the ECU 3 via the serial cable 42.

[0073] The logic circuit 211, the encoding / encryption circuit 214, the serializer circuit 213, and the control circuit 23 are interconnected by an I2C control bus 291. The logic circuit 221, the encoding / encryption circuit 224, the serializer circuit 223, and the control circuit 24 are interconnected by an I2C control bus 292.

[0074] (Example of Operation of Image Transmission System 1G) When the image transmission system 1G is started up, it operates in the same manner as the image transmission system 1B. The control circuit 34 of the ECU 3 transmits control data, for example, including lossless compression, to the iTOF control circuit 23 of the camera module 2 in accordance with the state of the vehicle 5. This control data including lossless compression is input to the iTOF control circuit 23 via the control bus 36, the deserializer circuit 31, the serial cable 41, the serializer circuit 213, and the control bus 291. The iTOF control circuit 23 then sets the image compression method of the encoding / encryption circuit 214 to lossless compression based on the control data including lossless compression sent from the ECU 3. When the encoding / encryption circuit 214 is set to lossless compression, the iTOF control circuit 23 transmits Ack data indicating that the setting has been successful to the control circuit 34 of the ECU 3. Note that the control data including lossless compression may be input directly to the encoding / encryption circuit 214 without passing through the iTOF control circuit 23.

[0075] Meanwhile, the control circuit 34 of the ECU 3 transmits control data including, for example, lossy compression to the RGB control circuit 24 of the camera module 2 in accordance with the state of the vehicle 5. This control data including lossy compression is input to the RGB control circuit 24 via the control bus 36, the deserializer circuit 31, the serial cable 42, the serializer circuit 223, and the control bus 292. The RGB control circuit 24 then sets the image compression method of the encoding / encryption circuit 224 to lossy compression based on the control data including lossy compression sent from the ECU 3. When the encoding / encryption circuit 224 is set to lossy compression, the RGB control circuit 24 transmits Ack data indicating that the setting has been successful to the control circuit 34 of the ECU 3. Note that the control data including lossy compression may be input directly to the encoding / encryption circuit 224 without passing through the RGB control circuit 24.

[0076] <Effects of the Sixth Modification of the First Embodiment> The sixth modification of the first embodiment also provides the same effects as the fifth modification of the first embodiment. Furthermore, since the serializer circuit 213 is disposed on the same substrate as the substrate on which the logic circuit 211 and the encoding / encryption circuit 214 in the iTOF sensor 21 are mounted, and the serializer circuit 223 is disposed on the same substrate as the substrate on which the logic circuit 221 and the encoding / encryption circuit 224 in the RGB sensor 22 are mounted, it is possible to reduce the size of the camera module 2.

[0077] <Seventh Modification of First Embodiment> Fig. 9 is a block diagram showing the functions of the camera module 2 and the ECU 3 of an image transmission system 1H according to a seventh modification of the first embodiment. In Fig. 9, the same parts as those in Figs. 4 and 7 are designated by the same reference numerals, and detailed description thereof will be omitted.

[0078] The camera module 2 of the image transmission system 1H includes an encryption / encoding circuit 51. The encryption / encoding circuit 51 compresses the RGB control data output from the control circuit 24 by lossless compression or lossy compression, and further encrypts the compressed RGB control data to output it to the serializer circuit 25.

[0079] The serializer circuit 25 converts the iTOF image data, iTOF control data, RGB image data, and RGB encrypted compressed control data from parallel data to serial data, and transmits the serial data to the ECU 3 via the serial cable 4 .

[0080] The deserializer circuit 31 converts serial data transmitted from the camera module 2 via the serial cable 4 into parallel data, iTOF image data, iTOF control data, RGB image data, and RGB encrypted compression control data. The decode / decrypt circuit 35 decodes the iTOF image data and RGB encrypted compression control data, and decrypts the encrypted data to restore the iTOF image data and RGB control data. In the camera module 2, the iTOF sensor 21, RGB sensor 22, control circuits 23 and 24, serializer circuit 25, and encode / encrypt circuit 51 are connected to each other by an I2C standard control bus 29.

[0081] (Example of Operation of Image Transmission System 1H) When the image transmission system 1H is started, it operates in the same manner as the image transmission system 1C. The control circuit 34 of the ECU 3 transmits control data, for example, including lossless compression, to the RGB control circuit 24 of the camera module 2 in accordance with the state of the vehicle 5. This control data including lossless compression is input to the RGB control circuit 24 via the control bus 36, the deserializer circuit 31, the serial cable 4, the serializer circuit 25, and the control bus 29. The RGB control circuit 24 then sets the image compression method of the encryption / encoding circuit 51 to lossless compression based on the control data including lossless compression sent from the ECU 3. When the encryption / encoding circuit 51 is set to lossless compression, the RGB control circuit 24 transmits Ack data indicating that the setting has been successful to the control circuit 34 of the ECU 3. Note that the control data including lossless compression may be input directly to the encryption / encoding circuit 51 without passing through the RGB control circuit 24.

[0082] Note that, although the above camera module 2 has been described as an example in which RGB control data is converted into RGB encrypted compressed control data, it may also be an example in which only iTOF image data is compressed into iTOF encrypted compressed image data, or an example in which iTOF image data is converted into iTOF encrypted compressed image data and RGB control data is converted into RGB encrypted compressed control data.

[0083] <Operational Effects of the Seventh Modification of the First Embodiment> The seventh modification of the first embodiment also provides the same operational effects as the fifth modification of the first embodiment.

[0084] <Eighth Modification of First Embodiment> Fig. 10 is a block diagram showing the functions of the camera module 2 and the ECU 3 of an image transmission system 1I according to an eighth modification of the first embodiment. In Fig. 10, the same parts as those in Figs. 5 and 7 are designated by the same reference numerals, and detailed description thereof will be omitted.

[0085] The camera module 2 of the image transmission system 1I includes an encryption / encoding circuit 52. The encryption / encoding circuit 52 compresses and encrypts the iTOF control data output from the control circuit 23 using lossless or lossy compression, and outputs the iTOF encrypted and compressed control data to the serializer circuit 25.

[0086] The serializer circuit 25 converts the iTOF image data, the iTOF encrypted compressed control data, the RGB encrypted compressed image data, and the RGB control data from parallel data into serial data, and transmits the serial data to the ECU 3 via the serial cable 4 .

[0087] The deserializer circuit 31 converts the serial data transmitted from the camera module 2 via the serial cable 4 into parallel data, such as iTOF image data, iTOF encrypted compressed control data, RGB encrypted compressed image data, and RGB control data. The decode / decryption circuit 35 decodes the iTOF encrypted compressed control data and RGB encrypted compressed image data, and decrypts the encrypted data to restore it to iTOF control data and RGB image data.

[0088] The iTOF sensor 21, the RGB sensor 22, the serializer circuit 25, the control circuits 23 and 24, and the encryption / encoding circuit 52 are interconnected by a control bus 29 conforming to the I2C standard.

[0089] (Example of Operation of Image Transmission System 1I) When the image transmission system 1I is started, it operates in the same manner as the image transmission system 1D. The control circuit 34 of the ECU 3 transmits control data including, for example, lossless compression to the iTOF control circuit 23 of the camera module 2, and transmits control data including, for example, lossless compression to the RGB control circuit 24 of the camera module 2, depending on the state of the vehicle 5. This control data including lossless compression is input to the iTOF control circuit 23 and the RGB control circuit 24 via the control bus 36, the deserializer circuit 31, the serial cable 4, the serializer circuit 25, and the control bus 29. The iTOF control circuit 23 then sets the image compression method of the encryption / encoding circuit 52 to lossless compression based on the control data including lossless compression sent from the ECU 3. When the encryption / encoding circuit 52 has been set to lossless compression, the iTOF control circuit 23 transmits Ack data indicating that the setting has been successful to the control circuit 34 of the ECU 3. Furthermore, the RGB control circuit 24 sets the image compression method of the encryption / encoding circuit 224 to lossless compression based on control data including lossless compression sent from the ECU 3. When the encryption / encoding circuit 224 is set to lossless compression, the RGB control circuit 24 transmits Ack data indicating that the setting is correct to the control circuit 34 of the ECU 3. Note that the control data including lossless compression may be input directly to the encryption / encoding circuits 52, 224 without passing through the iTOF control circuit 23 and the RGB control circuit 24.

[0090] Note that, although the above-described camera module 2 converts iTOF control data into iTOF encrypted compressed control data and converts RGB image data into RGB encrypted compressed image data, it may also be an example in which only RGB image data is converted into RGB encrypted compressed image data.

[0091] <Operational Effects of Eighth Modification of First Embodiment> Even in the eighth modification of the first embodiment, the same operational effects as those of the fifth modification of the first embodiment can be obtained.

[0092] <Ninth Modification of First Embodiment> Fig. 11 is a block diagram showing the functions of the camera module 2 and the ECU 3 of an image transmission system 1J according to a ninth modification of the first embodiment. In Fig. 11, the same parts as those in Figs. 6 and 7 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0093] The camera module 2 of the image transmission system 1J includes an encryption / encoding circuit 53. The encryption / encoding circuit 53 compresses and encrypts the iTOF control data output from the control circuit 23 using lossless or lossy compression, and outputs the iTOF encrypted compressed control data to the serializer circuit 25. The encryption / encoding circuit 53 also compresses and encrypts the RGB control data output from the control circuit 24 using lossless or lossy compression, and outputs the RGB encrypted compressed control data to the serializer circuit 25.

[0094] The serializer circuit 25 converts the iTOF image data, the iTOF cryptographic compression control data, the RGB image data, and the RGB cryptographic compression control data from parallel data to serial data, and transmits the serial data to the ECU 3 via the serial cable 4 .

[0095] The deserializer circuit 31 converts the serial data transmitted from the camera module 2 via the serial cable 4 into parallel data, such as iTOF image data, iTOF encrypted compression control data, RGB image data, and RGB encrypted compression control data. The decode / decryption circuit 35 decodes the iTOF encrypted compression control data and RGB encrypted compression control data to decrypt the encrypted data and restore it to iTOF control data and RGB control data.

[0096] The iTOF sensor 21, the RGB sensor 22, the serializer circuit 25, the control circuits 23 and 24, and the encryption / encoding circuit 53 are interconnected by an I2C standard control bus 29.

[0097] (Example of Operation of Image Transmission System 1J) When the image transmission system 1J is started, it operates in the same manner as the image transmission system 1E. The control circuit 34 of the ECU 3 transmits control data, for example, including lossless compression, to the iTOF control circuit 23 of the camera module 2 in accordance with the state of the vehicle 5. This control data including lossless compression is input to the iTOF control circuit 23 via the control bus 36, the deserializer circuit 31, the serial cable 4, the serializer circuit 25, and the control bus 29. The iTOF control circuit 23 then sets the image compression method of the encryption / encoding circuit 53 to lossless compression based on the control data including lossless compression sent from the ECU 3. When the encryption / encoding circuit 53 is set to lossless compression, the iTOF control circuit 23 transmits Ack data indicating that the setting has been successful to the control circuit 34 of the ECU 3. Note that the control data including lossless compression may be input directly to the encryption / encoding circuit 53 without passing through the iTOF control circuit 23.

[0098] Meanwhile, the control circuit 34 of the ECU 3 transmits control data including, for example, lossy compression to the RGB control circuit 24 of the camera module 2 in accordance with the state of the vehicle 5. This control data including lossy compression is input to the RGB control circuit 24 via the control bus 36, the deserializer circuit 31, the serial cable 4, the serializer circuit 25, and the control bus 29. The RGB control circuit 24 then sets the image compression method of the encryption / encoding circuit 53 to lossy compression based on the control data including lossy compression sent from the ECU 3. When the encryption / encoding circuit 53 is set to lossy compression, the RGB control circuit 24 transmits Ack data indicating that the setting has been successful to the control circuit 34 of the ECU 3. Note that the control data including lossy compression may be input directly to the encryption / encoding circuit 53 without passing through the RGB control circuit 24.

[0099] Note that the above camera module 2 has been described as an example in which it converts iTOF control data into iTOF encryption compression control data and RGB control data into RGB encryption compression control data, but it may also be an example in which it converts only iTOF control data into iTOF encryption compression control data.

[0100] <Operational Effects of Ninth Modification of First Embodiment> Even with the ninth modification of the first embodiment, the same operational effects as those of the fifth modification of the first embodiment can be obtained.

[0101] Second Embodiment Fig. 12 is a schematic configuration diagram showing the entire image transmission system 100A according to a second embodiment of the present disclosure. The image transmission system 100A in Fig. 12 includes a camera module 200 and an ECU 300. The camera module 200 and the ECU 300 are connected by a serial cable 400. The camera module 200 is attached to the interior of a vehicle 5. The camera module 200 is connected to an RGB(x) sensor 201. The RGB(x) sensor 201 is an image sensor that has both an iTOF sensor function and an RGB sensor function.

[0102] In a second embodiment of the present disclosure, an example of recognizing an object using an RGB(x) sensor 201 of a single-lens reflex camera will be described. The RGB(x) sensor 201 of the camera module 200 includes a logic circuit 2011 as a signal processing unit and an encoding circuit 2012 as a compression unit. The logic circuit 2011 processes the image signal output from the RGB(x) sensor 201 and obtains feature quantities for identifying the object based on the image signal. These feature quantities include, for example, distance information to the object and position information in two-dimensional coordinates. The encoding circuit 2012 compresses image data including distance information output from the logic circuit 2011 (referred to as distance image data) using lossless or lossy compression, and outputs the compressed data (referred to as compressed distance image data) to the serializer circuit 202.

[0103] The camera module 200 includes a control circuit 203. The RGB(x) sensor 201, serializer circuit 202, and control circuit 203 are interconnected by an I2C control bus 209. The control circuit 203 controls the signal processing of the logic circuit 2011. The control circuit 203 outputs control data (referred to as distance image control data) required for image reproduction of the depth image data to the serializer circuit 202 via the control bus 209. This distance image control data includes distance data to the target object.

[0104] The serializer circuit 202 converts the compressed range image data and the range image control data from parallel data to serial data, and transmits the serial data to the ECU 300 via the serial cable 400. The serial cable 400 includes a bidirectional control interface that is compatible with the I2C standard.

[0105] The ECU 300 includes a deserializer circuit 301, a decode circuit 302, an image reproduction circuit 303, and a control circuit 304. The deserializer circuit 301, the decode circuit 302, the image reproduction circuit 303, and the control circuit 304 are interconnected by an I2C control bus 306. The deserializer circuit 301 converts serial data transmitted from the camera module 200 via a serial cable 400 into parallel data, such as compressed depth image data and depth image control data. The decode circuit 302 decodes the compressed depth image data and converts it back into depth image data. The depth image control data is output to the control circuit 304.

[0106] The image reproduction circuit 303 reproduces the distance image data and recognizes the object based on the distance image. The control circuit 304 controls the reproduction process by the image reproduction circuit 303 via the control bus 306 based on the distance image control data.

[0107] (Example of Operation of Image Transmission System 100A) For example, when control circuit 304 of ECU 300 detects that the engine of vehicle 5 has started or that the ignition has been turned on, it transmits startup control data to control circuit 203 of camera module 200 for activating RGB(x) sensor 201. The startup control data is input to control circuit 203 via control bus 306, deserializer circuit 301, serial cable 400, serializer circuit 202, and control bus 209. Then, control circuit 203 activates RGB(x) sensor 201 based on the startup control data sent from ECU 300. When RGB(x) sensor 201 has been activated, control circuit 203 transmits Ack (response) data indicating that the RGB(x) sensor 201 has been activated to control circuit 304 of ECU 300. The Ack data is transmitted to the control circuit 304 of the ECU 300 via the control bus 209 , the serializer circuit 202 , the serial cable 400 , the deserializer circuit 301 , and the control bus 306 .

[0108] Next, the control circuit 304 of the ECU 300 transmits control data including, for example, lossless compression to the control circuit 203 of the camera module 200 in accordance with the state of the vehicle 5. The control data including lossless compression is input to the control circuit 203 via the control bus 306, the deserializer circuit 301, the serial cable 400, the serializer circuit 202, and the control bus 209. The control circuit 203 then sets the image compression method of the encoding circuit 2012 to lossless compression based on the control data including lossless compression sent from the ECU 300. When the encoding circuit 2012 has been set to lossless compression, the control circuit 203 transmits ACK data indicating that the setting has been successful to the control circuit 304 of the ECU 300. Note that the control data including lossless compression may be input directly to the encoding circuit 2012 without passing through the control circuit 203.

[0109] <Operational Effects of Second Embodiment> As described above, the second embodiment also provides the same operational effects as the first embodiment.

[0110] <First Modification of Second Embodiment> Fig. 13 is a block diagram showing the functions of the camera module 200 and the ECU 300 of an image transmission system 100B according to a first modification of the second embodiment. In Fig. 13, the same parts as those in Fig. 12 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0111] The RGB(x) sensor 201 includes a serializer circuit 2013 on the same board as the logic circuit 2011 and the encoding circuit 2012. The serializer circuit 2013 converts the compressed range image data and range image control data from parallel data to serial data, and transmits the serial data to the ECU 300 via a serial cable 400. The logic circuit 2011, the encoding circuit 2012, the serializer circuit 2013, and the control circuit 203 are interconnected by an I2C control bus 209.

[0112] (Example of Operation of Image Transmission System 100B) When control circuit 304 of ECU 300 detects, for example, engine start or ignition on of vehicle 5, it transmits start-up control data to control circuit 203 of camera module 200 for starting RGB(x) sensor 201. The start-up control data for RGB(x) sensor 201 is input to control circuit 203 via control bus 306, deserializer circuit 301, serial cable 400, serializer circuit 2013, and control bus 209. Then, control circuit 203 starts RGB(x) sensor 201 based on the start-up control data sent from ECU 300. When RGB(x) sensor 201 is started, control circuit 203 transmits Ack (response) data indicating that the start-up has been successful to control circuit 304 of ECU 300. The Ack data is transmitted to the control circuit 304 of the ECU 300 via the control bus 209 , the serializer circuit 2013 , the serial cable 400 , the deserializer circuit 301 , and the control bus 306 .

[0113] The control circuit 304 of the ECU 300 transmits control data including, for example, lossless compression to the control circuit 203 of the camera module 200 in accordance with the state of the vehicle 5. This control data including lossless compression is input to the control circuit 203 via the control bus 306, the deserializer circuit 301, the serial cable 400, the serializer circuit 2013, and the control bus 209. The control circuit 203 then sets the image compression method of the encoding circuit 2012 to lossless compression based on the control data including lossless compression sent from the ECU 300. When the encoding circuit 2012 has been set to lossless compression, the control circuit 203 transmits ACK data indicating that the setting has been successful to the control circuit 304 of the ECU 300. Note that the control data including lossless compression may be input directly to the encoding circuit 2012 without passing through the control circuit 203.

[0114] <Operational Effects of the First Modification of the Second Embodiment> The first modification of the second embodiment also provides the same operational effects as the second embodiment, and also allows the camera module 200 to be made smaller.

[0115] <Second Modification of Second Embodiment> Fig. 14 is a block diagram showing the functions of the camera module 200 and the ECU 300 of an image transmission system 100C according to a second modification of the second embodiment. In Fig. 14, the same parts as those in Fig. 12 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0116] The camera module 200 of the image transmission system 100C includes an encoding circuit 204. The encoding circuit 204 compresses the distance image control data output from the control circuit 203 using lossless compression or lossy compression, and outputs the compressed distance image control data to the serializer circuit 202.

[0117] The serializer circuit 202 converts the range image data and the range image compression control data from parallel data to serial data, and transmits the serial data to the ECU 300 via the serial cable 400 .

[0118] The deserializer circuit 301 converts the serial data transmitted from the camera module 200 via the serial cable 400 into parallel data, that is, depth image data and depth image compression control data. The decode circuit 302 decodes the depth image compression control data back into depth image control data.

[0119] The RGB(x) sensor 201, the serializer circuit 202, the control circuit 203, and the encoding circuit 204 are connected to one another by a control bus 209 conforming to the I2C standard.

[0120] (Example of Operation of Image Transmission System 100C) For example, when the control circuit 304 of the ECU 300 detects that the engine of the vehicle 5 has started or that the ignition has been turned on, the control circuit 304 transmits activation control data for activating the RGB(x) sensor 201 to the control circuit 203 of the camera module 200. The activation control data is input to the control circuit 203 via the control bus 306, the deserializer circuit 301, the serial cable 400, the serializer circuit 202, and the control bus 209. The control circuit 203 then activates the RGB(x) sensor 201 based on the activation control data transmitted from the ECU 300. At this time, the control circuit 203 sets the power supply of the encoding circuit 204 to an ON state. When the RGB(x) sensor 201 has been activated, the control circuit 203 transmits Ack (response) data indicating that the RGB(x) sensor 201 has been activated to the control circuit 304 of the ECU 300. The Ack data is transmitted to the control circuit 304 of the ECU 300 via the control bus 209 , the serializer circuit 202 , the serial cable 400 , the deserializer circuit 301 , and the control bus 306 .

[0121] Next, the control circuit 304 of the ECU 300 transmits control data including, for example, lossless compression to the control circuit 203 of the camera module 200 in accordance with the state of the vehicle 5. The control data including lossless compression is input to the control circuit 203 via the control bus 306, the deserializer circuit 301, the serial cable 400, the serializer circuit 202, and the control bus 209. The control circuit 203 then sets the image compression method of the encoding circuit 204 to lossless compression based on the control data including lossless compression sent from the ECU 300. When the encoding circuit 204 has been set to lossless compression, the control circuit 203 transmits ACK data indicating that the setting has been successful to the control circuit 304 of the ECU 300. Note that the control data including lossless compression may be input directly to the encoding circuit 204 without passing through the control circuit 203.

[0122] <Operational Effects of the Second Modification of the Second Embodiment> The second modification of the second embodiment also provides the same operational effects as the second embodiment.

[0123] <Third Modification of Second Embodiment> Fig. 15 is a block diagram showing the functions of the camera module 200 and the ECU 300 of an image transmission system 100D according to a third modification of the second embodiment. In Fig. 15, the same parts as those in Fig. 12 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0124] In the camera module 200 of the image transmission system 100D, the RGB(x) sensor 201 is provided with an encoding / encryption circuit 2014. The encoding / encryption circuit 2014 compresses the depth image data output from the logic circuit 2011 using lossless or lossy compression, encrypts the data with encryption key information, and outputs the encrypted compressed data (referred to as encrypted compressed depth image data) to the serializer circuit 202. The encryption key information is included in the depth image control data.

[0125] The serializer circuit 202 converts the encrypted compressed distance image data and the distance image control data from parallel data into serial data, and transmits the serial data to the ECU 300 via the serial cable 400 .

[0126] ECU 300 is equipped with a decode / decryption circuit 305. Deserializer circuit 301 converts serial data transmitted from camera module 200 via serial cable 400 into parallel data, namely encrypted compressed distance image data and distance image control data. Decode / decryption circuit 305 decodes the encrypted compressed distance image data based on encryption key information included in the distance image control data, and decrypts the encrypted data to restore it to distance image data.

[0127] In the camera module 200, the RGB(x) sensor 201, the serializer circuit 202, and the control circuit 203 are interconnected by an I2C control bus 209. In the ECU 300, the deserializer circuit 301, the image reproduction circuit 303, the control circuit 304, and the decode / decryption circuit 305 are interconnected by an I2C control bus 306.

[0128] (Example of Operation of Image Transmission System 100D) When the image transmission system 100D is started up, it operates in the same manner as the image transmission system 100A. The control circuit 304 of the ECU 300 transmits control data, for example, including lossless compression, to the control circuit 203 of the camera module 200 in accordance with the state of the vehicle 5. The control data including lossless compression is input to the control circuit 203 via the control bus 306, the deserializer circuit 301, the serial cable 400, the serializer circuit 202, and the control bus 209. The control circuit 203 then sets the image compression method of the encoding / encryption circuit 2014 to lossless compression based on the control data including lossless compression sent from the ECU 300. When the encoding / encryption circuit 2014 is set to lossless compression, the control circuit 203 transmits ACK data indicating that the setting has been successful to the control circuit 304 of the ECU 300. Note that the control data including lossless compression may be input directly to the encoding / encryption circuit 2014 without passing through the control circuit 203.

[0129] <Effects of the third variant of the second embodiment> According to the third variant of the second embodiment, the same effects as those of the second embodiment can be obtained, and even if the amount of data increases by encrypting the distance image data to protect against external hacking, it becomes possible to serially transfer the encrypted compressed distance image data without increasing the number of serial cables 400, thereby strengthening security.

[0130] <Fourth Modification of Second Embodiment> Fig. 16 is a block diagram showing the functions of the camera module 200 and the ECU 300 of an image transmission system 100E according to a fourth modification of the second embodiment. In Fig. 16, the same parts as those in Figs. 13 and 157 are designated by the same reference numerals and detailed description thereof will be omitted.

[0131] RGB(x) sensor 201 is provided with a serializer circuit 2013. Serializer circuit 2013 converts the encrypted compressed distance image data and distance image control data from parallel data to serial data, and transmits the serial data to ECU 300 via serial cable 400.

[0132] The logic circuit 2011, the encoding / encryption circuit 2014, the serializer circuit 2013, and the control circuit 203 are interconnected by a control bus 209 conforming to the I2C standard.

[0133] (Example of Operation of Image Transmission System 100E) When the image transmission system 100E is started up, it operates in the same manner as the image transmission system 100B. The control circuit 304 of the ECU 300 transmits control data, for example, including lossless compression, to the control circuit 203 of the camera module 200 in accordance with the state of the vehicle 5. This control data including lossless compression is input to the control circuit 203 via the control bus 306, the deserializer circuit 301, the serial cable 400, the serializer circuit 2013, and the control bus 209. The control circuit 203 then sets the image compression method of the encoding / encryption circuit 2014 to lossless compression based on the control data including lossless compression sent from the ECU 300. When the encoding / encryption circuit 2014 is set to lossless compression, the control circuit 203 transmits ACK data indicating that the setting has been successful to the control circuit 304 of the ECU 3. Note that the control data including lossless compression may be input directly to the encoding / encryption circuit 2014 without passing through the control circuit 203.

[0134] <Operational Effects of the Fourth Modification of the Second Embodiment> The fourth modification of the second embodiment also provides the same operational effects as the third modification of the second embodiment.

[0135] <Fifth Modification of Second Embodiment> Fig. 17 is a block diagram showing the functions of the camera module 200 and the ECU 300 of an image transmission system 100F according to a fifth modification of the second embodiment. In Fig. 17, the same parts as those in Fig. 15 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0136] The camera module 200 of the image transmission system 100F includes an encryption / encoding circuit 205. The encryption / encoding circuit 205 compresses the distance image control data output from the control circuit 203 using lossless or lossy compression, and further encrypts the compressed distance image control data, thereby outputting the encrypted compressed distance image control data to the serializer circuit 202.

[0137] The serializer circuit 202 converts the range image data and the encrypted range image compression control data from parallel data to serial data, and transmits the serial data to the ECU 300 via the serial cable 400 .

[0138] Deserializer circuit 301 converts serial data transmitted from camera module 200 via serial cable 400 into parallel data, that is, distance image data and distance image encrypted compression control data. Decode / decryption circuit 305 decodes the distance image encrypted compression control data and decrypts the encrypted data to restore it to distance image control data.

[0139] The RGB(x) sensor 201, the serializer circuit 202, the control circuit 203, and the encryption / encoding circuit 205 are connected to one another by a control bus 209 conforming to the I2C standard.

[0140] (Example of Operation of Image Transmission System 100F) When the image transmission system 100F is started up, it operates in the same manner as the image transmission system 100A. The control circuit 304 of the ECU 300 transmits control data, for example, including lossless compression, to the control circuit 203 of the camera module 200 in accordance with the state of the vehicle 5. The control data including lossless compression is input to the control circuit 203 via the control bus 306, the deserializer circuit 301, the serial cable 400, the serializer circuit 202, and the control bus 209. The control circuit 203 then sets the image compression method of the encryption / encoding circuit 205 to lossless compression based on the control data including lossless compression sent from the ECU 300. When the encryption / encoding circuit 205 is set to lossless compression, the control circuit 203 transmits ACK data indicating that the setting has been successful to the control circuit 304 of the ECU 300. Note that the control data including lossless compression may be input directly to the encryption / encoding circuit 205 without passing through the control circuit 203.

[0141] <Operational Effects of Fifth Modification of Second Embodiment> Even in the fifth modification of the second embodiment, the same operational effects as those of the third modification of the second embodiment can be obtained.

[0142] Third Embodiment At least one of the image transmission system 1A according to the first embodiment, the image transmission system 1B according to a first modified example of the first embodiment, the image transmission system 1C according to a second modified example of the first embodiment, the image transmission system 1D according to a third modified example of the first embodiment, the image transmission system 1E according to a fourth modified example of the first embodiment, the image transmission system 1F according to a fifth modified example of the first embodiment, the image transmission system 1G according to a sixth modified example of the first embodiment, the image transmission system 1H according to a seventh modified example of the first embodiment, the image transmission system 1I according to an eighth modified example of the first embodiment, and the image transmission system 1J according to a ninth modified example of the first embodiment can be applied to the image transmission system according to the third embodiment. An example of an image transmission system according to the third embodiment to which the image transmission systems 1A, 1B, 1C, 1D, 1E, 1F, 1G, 1H, 1I, and 1J are applied will be described below.

[0143] In a third embodiment of the present disclosure, an example will be described in which two types of image sensors, an iTOF sensor 21 and an RGB sensor 22, having different detection characteristics are used to integrate (fuse) information about an object recognized based on the output of each image sensor to recognize the object with high accuracy.

[0144] The control circuit 23 for the iTOF sensor 21 changes the image compression method of the encoding circuit 212 in accordance with the operating state of the vehicle 5 notified by the ECU 3 .

[0145] The control circuit 24 for the RGB sensor 22 changes the image compression method of the encoding circuit 222 in accordance with the operating state of the vehicle 5 notified by the ECU 3 .

[0146] The control circuit 34 of the ECU 3 monitors the operating state of the vehicle 5 and monitors whether the amount of data transmitted through the serial cable 4 overflows, and transmits the monitoring result to the camera module 2 as control data.

[0147] (Image Compression Control Processing According to Operational State of Vehicle 5) Fig. 18 is a flowchart showing a control procedure for executing the image compression control processing by the ECU 3. Fig. 19 is a flowchart showing the detailed control procedure of step ST19g shown in Fig. 18.

[0148] 20 to 24 are screen diagrams showing image compression methods according to the operating state of the vehicle 5. In each of Figs. 20 to 24, (a) shows RGB image data, and (b) shows iTOF image data. In each of Figs. 20 to 24, (a) and (b) show (1) when the vehicle 5 is idling (during authentication), (2) when the vehicle 5 is in operation (driving state), and (3) when the vehicle 5 is stopped. In each of Figs. 20 to 24, (a) and (b) show image areas compressed by lossy compression, indicated by dots.

[0149] In FIG. 18, when the control circuit 34 of the ECU 3 detects that the engine of the vehicle 5 is started or the ignition is on (step ST19a), it sets the image compression method to compression pattern 1 (the compression pattern shown in FIG. 20) (step ST19b). When it confirms that the vehicle 5 is idling, i.e., that the vehicle 5 is stopped, it transmits control data including lossless compression to the control circuit 23 for iTOF of the camera module 2, and also transmits control data including lossless compression to the control circuit 24 for RGB of the camera module 2 (step ST19c).

[0150] Then, the control circuit 34 confirms the positions of people sitting in the seats of the vehicle 5, i.e., the driver, passenger, and the left and right rear seats, through image recognition by the image reproduction circuit 33 (step ST19d). If the driver and passenger are recognized, the control circuit 34 changes the image compression method to, for example, compression pattern 2 (the compression pattern shown in FIG. 21) (step ST19e), and transmits control data including lossy compression to the RGB control circuit 24 of the camera module 2.

[0151] Next, the control circuit 34 constantly monitors the operating state of the vehicle 5 and changes the setting of the image compression method in accordance with the operating state of the vehicle 5 (step ST19f). The control circuit 34 also monitors whether the engine of the vehicle 5 is stopped (step ST19g), and if it determines that the engine of the vehicle 5 is not stopped (No), it proceeds to the processing of step ST19f and continues changing the setting of the image compression method in accordance with the operating state of the vehicle 5. On the other hand, if it determines that the engine of the vehicle 5 has stopped and n minutes have passed since the engine was stopped (Yes), the control circuit 34 ends the image compression control processing.

[0152] (When the vehicle 5 is idling) In FIG. 19 , after changing the image compression method to compression pattern 2 in step ST19e, the control circuit 34 monitors whether the amount of data transmitted through the serial cable 4 overflows, i.e., whether the number of image frames per second exceeds 60 (step ST19f1). If it determines that overflow has occurred (step ST19f1: Yes), it changes the image compression method to, for example, compression pattern 3 (the compression pattern shown in FIG. 22) (step ST19f2) and transmits control data to the iTOF control circuit 23 of the camera module 2, including a compression method that performs lossless compression on the image areas of the driver and passenger and lossy compression on the other areas. Thereafter, the control circuit 34 returns to the processing of step ST19f1 and monitors whether the amount of data transmitted through the serial cable 4 overflows. During idling, the amount of data transmitted through compression pattern 2 is reduced compared to the amount of data transmitted through compression pattern 1. Furthermore, the amount of data transmitted through compression pattern 3 is reduced compared to the amount of data transmitted through compression pattern 2.

[0153] (When the vehicle 5 is driving) On the other hand, if it is not determined that an overflow will occur (step ST19f1: No), the control circuit 34 monitors whether the vehicle 5 is in a driving state (step ST19f3). If it is determined that the vehicle 5 is in a driving state (step ST19f3: Yes), the control circuit 34 proceeds to the processing of step ST19f1 described above and monitors whether the amount of data transmitted through the serial cable 4 will overflow. If it is determined that an overflow will occur (step ST19f1: Yes), the control circuit 34 changes the image compression method to, for example, compression pattern 3 (the compression pattern shown in FIG. 24) (step ST19f2), and returns to the processing of step ST19f1 described above to continue monitoring whether an overflow will occur. When driving, the amount of data transmitted using compression pattern 3 is reduced compared to the amount of data transmitted using compression pattern 2.

[0154] If it is determined that overflow will occur even when the compression pattern is changed to 3 (step ST19f1: Yes), the image compression method is changed to, for example, compression pattern 4 (the compression pattern shown in FIG. 23) (step ST19f2), and control data including a compression method that performs lossless compression on the driver's image area and lossy compression on other areas is transmitted to the iTOF control circuit 23 of the camera module 2. Thereafter, the control circuit 34 returns to the processing of step ST19f1 and monitors whether the amount of data transmitted through the serial cable 4 will overflow. During operation, the amount of data transmitted using compression pattern 4 is reduced compared to the amount of data transmitted using compression pattern 3.

[0155] Furthermore, if it is determined that overflow will occur even when the compression pattern is changed to 4 (step ST19f1: Yes), the image compression method is changed to, for example, compression pattern 5 (the compression pattern shown in FIG. 24) (step ST19f2), and control data including a compression method that performs lossless compression only on the image area of ​​the driver's face and lossy compression on the other areas is transmitted to the iTOF control circuit 23 of the camera module 2. Thereafter, the control circuit 34 returns to the processing of step ST19f1 and monitors whether the amount of data transmitted through the serial cable 4 will overflow. During driving, the amount of data transmitted using compression pattern 5 is reduced compared to the amount of data transmitted using compression pattern 4.

[0156] (When vehicle 5 is stopped) In step ST19f3, if it is determined that the vehicle 5 is not in a driving state (step ST19f3: No), the control circuit 34 monitors whether the vehicle 5 is in an idling state (step ST19f4), and if it is determined that the vehicle 5 is in an idling state (step ST19f4: Yes), it proceeds to the processing of step ST19f1 and monitors whether the amount of data transmitted through the serial cable 4 overflows.

[0157] (When the engine of the vehicle 5 is stopped) If it is determined in step ST19f4 that the vehicle 5 is not in an idling state (step ST19f4: No), the control circuit 34 proceeds to the process of step ST19g. In the control procedure, the compression methods of the iTOF image data and the RGB image data are changed depending on the operating state of the vehicle 5. However, this is not limited to this. The control circuit 34 of the ECU 3 can also control the encoding circuits 212, 222 to fix the compression methods of the iTOF image data and the RGB image data, and operate the iTOF sensor 21 and the RGB sensor 22 from the time the engine of the vehicle 5 is started until the engine is stopped.

[0158] <Effects of the Third Embodiment> As described above, according to the third embodiment, in the camera module 2, by compressing the iTOF image data and RGB image data using a compression pattern that corresponds to the operating state of the vehicle 5, it becomes possible to transfer the iTOF image data and RGB image data to the ECU 3 in the necessary and sufficient data amounts, and the ECU 3 can play back images at the necessary and sufficient resolution to monitor the interior of the vehicle 5.

[0159] Since monitoring the driver's area is important when monitoring the inside of the vehicle, the area for the driver is subjected to lossless compression, while the passenger side is subjected to lossy compression to reduce the amount of data, thereby enabling serial transfer without increasing the number of serial cables 4, and enabling the ECU 3 to monitor the driver's area with the necessary and sufficient resolution.

[0160] Furthermore, according to the third embodiment, by varying the compression method for each of the iTOF image data and the RGB image data depending on the type of image sensor, for example, the RGB image data is compressed using lossy compression, while the iTOF image data obtained by the iTOF sensor 21, which can reliably recognize the object, is compressed by lossless compression of the driver's face and lossy compression of the passenger side, thereby further reducing the amount of data and enabling serial transfer, and making it possible for the ECU 3 to monitor the driver's area with the necessary and sufficient resolution.

[0161] <Fourth embodiment> Fig. 25 is a schematic configuration diagram showing a partial example of an image transmission system 1000A according to a fourth embodiment of the present disclosure. In Fig. 25, the same parts as those in Fig. 1 above are assigned the same reference numerals, and detailed description thereof will be omitted. Fig. 25(a) shows a side view of a vehicle 5, and Fig. 25(b) shows a top view of the vehicle 5. The image transmission system 1000A includes two wide-angle camera modules: a first camera module 2001 and a second camera module 2002.

[0162] In a fourth embodiment of the present disclosure, an example will be described in which two types of image sensors, an iTOF sensor 21 and an RGB sensor 22, having different detection characteristics are used to integrate (fuse) information about an object recognized based on the output of each image sensor to recognize the object with high accuracy.

[0163] 26 and 27 are schematic configuration diagrams showing a partial example of an image transmission system according to a comparative example of the fourth embodiment of the present disclosure. In Fig. 26 and Fig. 27, the same parts as those in Fig. 1 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0164] 26 , the camera module 2 in the comparative example is disposed at the front of the vehicle 5 and captures images of the driver and passengers sitting in the front seats. However, the passengers sitting in the rear seats are not within the imaging field of the camera module 2.

[0165] 27, it is possible to widen the imaging field of view of the camera module 2 so that the passengers sitting in the back seats are included in the imaging field of view. However, widening the imaging field of view of the camera module 2 increases the current consumption, and the camera module 2 itself generates heat.

[0166] <Implementation Means According to Fourth Embodiment> Therefore, in the fourth embodiment of the present disclosure, a first camera module 2001 is installed in the front of the vehicle 5, and a second camera module 2002 is installed in front of the rear seat of the vehicle 5. This allows the first camera module 2001 to capture images of the driver and passenger sitting in the front seat, and the second camera module 2002 to capture images of the passenger sitting in the rear seat.

[0167] <Effects of the fourth embodiment> As described above, according to the fourth embodiment, by arranging the wide-angle first camera module 2001 and the second camera module 2002 at appropriate positions within the vehicle 5, it is possible to sufficiently cover the required monitoring area within the vehicle 5.

[0168] <First Modification of Fourth Embodiment> Fig. 28 is a schematic configuration diagram showing a partial example of an image transmission system 1000B according to a first modification of the fourth embodiment of the present disclosure. In Fig. 28, the same parts as in Fig. 25 are assigned the same reference numerals, and detailed description thereof will be omitted. Fig. 28(a) shows a side view of a vehicle 5, and Fig. 28(b) shows a top view of the vehicle 5.

[0169] In a first modified example of the fourth embodiment of the present disclosure, a first camera module 2001 is installed in the front of the vehicle 5, and a second camera module 2001 is installed behind the rear seat of the vehicle 5. This allows the first camera module 2001 to capture images of the driver and passenger sitting in the front seat, and the second camera module 2001 to capture images of the passenger sitting in the rear seat.

[0170] <Operational Effects of the First Modification of the Fourth Embodiment> As described above, the first modification of the fourth embodiment also provides the same operational effects as the fourth embodiment.

[0171] <Second Modification of Fourth Embodiment> Fig. 29 is a schematic configuration diagram showing a partial example of an image transmission system 1000C according to a second modification of the fourth embodiment of the present disclosure. In Fig. 29, the same parts as in Fig. 25 are assigned the same reference numerals, and detailed description thereof will be omitted. Fig. 29(a) shows a side view of a vehicle 5, and Fig. 29(b) shows a top view of the vehicle 5.

[0172] In a second modified example of the fourth embodiment of the present disclosure, a first camera module 2001 is installed on the left side of the vehicle 5, and a second camera module 2001 is installed on the right side of the vehicle 5. This allows the first camera module 2001 to capture images of the driver sitting in the front seat and the passenger sitting in the rear seat, and the second camera module 2001 to capture images of the passenger sitting in the front seat and the passenger sitting in the rear seat.

[0173] <Operational Effects of the Second Modification of the Fourth Embodiment> As described above, the second modification of the fourth embodiment also provides the same operational effects as the fourth embodiment.

[0174] <Third Modification of Fourth Embodiment> Fig. 30 is a schematic configuration diagram showing a partial example of an image transmission system 1000D according to a third modification of the fourth embodiment of the present disclosure. In Fig. 30, the same parts as in Fig. 25 are assigned the same reference numerals, and detailed description thereof will be omitted. Fig. 30(a) shows a side view of a vehicle 5, and Fig. 30(b) shows a top view of the vehicle 5.

[0175] In a third modified example of the fourth embodiment of the present disclosure, the first camera module 2001 is installed on the left side of the vehicle 5, and the second camera module 2001 is installed on the right side of the vehicle 5, thereby widening the imaging field of view of each. This allows the first camera module 2001 to capture images of a passenger sitting in the front seat and a passenger sitting in the rear seat, and the second camera module 2001 to capture images of a driver sitting in the front seat and a passenger sitting in the rear seat.

[0176] <Operational Effects of the Third Modification of the Fourth Embodiment> As described above, the third modification of the fourth embodiment also provides the same operational effects as the fourth embodiment.

[0177] <Fourth Modification of Fourth Embodiment> Fig. 31 is a schematic configuration diagram showing a partial example of an image transmission system 1000E according to a fourth modification of the fourth embodiment of the present disclosure. In Fig. 31, the same parts as those in Fig. 25 are assigned the same reference numerals, and detailed description thereof will be omitted. Fig. 31(a) shows a side view of a vehicle 5, and Fig. 31(b) shows a top view of the vehicle 5. The image transmission system 1000E includes four wide-angle camera modules: a first camera module 2001, a second camera module 2002, a third camera module 2003, and a fourth camera module 2004.

[0178] In a fourth modified example of the fourth embodiment of the present disclosure, a first camera module 2001 is installed in front of a vehicle 5, a second camera module 2001 is installed in front of a rear seat of the vehicle 5, a third camera module 2003 is installed on the left side of the vehicle 5, and a fourth camera module 2001 is installed on the right side of the vehicle 5. As a result, the first camera module 2001 can capture images of the driver and passenger sitting in the front seats, and the second camera module 2001 can capture images of passengers sitting in the rear seats. Furthermore, the third camera module 2003 can capture images of the driver sitting in the front seat and the passenger sitting in the rear seats, and the fourth camera module 2004 can capture images of the passenger sitting in the front seat (passenger seat) and the passenger sitting in the rear seats.

[0179] <Effects of the Fourth Modification of the Fourth Embodiment> As described above, the fourth modification of the fourth embodiment can also provide the same effects as the fourth embodiment. In particular, the entire monitoring area inside the vehicle 5 can be monitored without any omissions.

[0180] <Fifth Modification of Fourth Embodiment> Fig. 32 is a schematic configuration diagram showing a partial example of an image transmission system 1000F according to a fifth modification of the fourth embodiment of the present disclosure. In Fig. 32, the same parts as in Fig. 31 are assigned the same reference numerals, and detailed description thereof will be omitted. Fig. 32(a) shows a side view of a vehicle 5, and Fig. 32(b) shows a top view of the vehicle 5.

[0181] In a fifth modified example of the fourth embodiment of the present disclosure, a first camera module 2001 is installed in the front of the vehicle 5, and a second camera module 2001 is installed behind the rear seat of the vehicle 5. This allows the first camera module 2001 to capture images of the driver and passenger sitting in the front seat, and the second camera module 2001 to capture images of the passenger sitting in the rear seat.

[0182] <Operational Effects of Fifth Modification of Fourth Embodiment> As described above, the fifth modification of the fourth embodiment also provides the same operational effects as the fourth embodiment.

[0183] <Sixth Modification of Fourth Embodiment> Fig. 33 is a schematic configuration diagram showing a partial example of an image transmission system 1000G according to a fifth modification of the fourth embodiment of the present disclosure. In Fig. 33, the same parts as in Fig. 31 are assigned the same reference numerals, and detailed description thereof will be omitted. Fig. 33(a) shows a side view of a vehicle 5, and Fig. 33(b) shows a top view of the vehicle 5.

[0184] In a sixth modified example of the fourth embodiment of the present disclosure, a first camera module 2001 is installed in front of the vehicle 5, and a second camera module 2001 is installed in front of the rear seat of the vehicle 5. In addition, a third camera module 2003 is installed on the left side of the vehicle 5, and a fourth camera module 2004 is installed on the right side of the vehicle 5, thereby widening the imaging field of view of each.

[0185] <Operational Effects of the Sixth Modification of the Fourth Embodiment> As described above, the sixth modification of the fourth embodiment also provides the same operational effects as the fourth embodiment.

[0186] <Seventh Modification of Fourth Embodiment> Fig. 34 is a schematic configuration diagram showing a partial example of an image transmission system 1000H according to a seventh modification of the fourth embodiment of the present disclosure. In Fig. 34, the same parts as in Fig. 31 are assigned the same reference numerals, and detailed description thereof will be omitted. Fig. 34(a) shows a side view of a vehicle 5, and Fig. 34(b) shows a top view of the vehicle 5.

[0187] In a seventh modified example of the fourth embodiment of the present disclosure, a first camera module 2001 is installed in the front of the vehicle 5, and a second camera module 2001 is installed behind the rear seat of the vehicle 5, thereby widening the imaging field of view of each. In addition, a third camera module 2003 is installed on the left side of the vehicle 5, and a fourth camera module 2004 is installed on the right side of the vehicle 5, thereby widening the imaging field of view of each.

[0188] <Operational Effects of Seventh Modification of Fourth Embodiment> As described above, the seventh modification of the fourth embodiment also provides the same operational effects as the fourth embodiment.

[0189] Fifth Embodiment Fig. 35 is a schematic configuration diagram showing a partial example of an image transmission system 1001A according to a fifth embodiment of the present disclosure. In Fig. 35, the same parts as those in Fig. 1 above are assigned the same reference numerals, and detailed description thereof will be omitted. Fig. 35(a) shows a side view of a vehicle 5, and Fig. 35(b) shows a top view of the vehicle 5. The image transmission system 1001A includes four narrow-angle camera modules: a first camera module 2011, a second camera module 2012, a third camera module 2013, and a fourth camera module 2014.

[0190] In a fifth embodiment of the present disclosure, an example will be described in which two types of image sensors, an iTOF sensor 21 and an RGB sensor 22, having different detection characteristics are used to integrate (fuse) information about an object recognized based on the output of each image sensor to recognize the object with high accuracy.

[0191] In the fifth embodiment of the present disclosure, a first camera module 2011 is installed in front of a vehicle 5, and a second camera module 2012 is installed in front of a rear seat of the vehicle 5. The first camera module 2011 has an imaging field of view that covers the driver and a portion of a passenger (referred to as a front passenger seat passenger). The second camera module 2012 has an imaging field of view that covers a passenger sitting in the rear seat (referred to as a first rear passenger seat) and a portion of a passenger sitting next to him (referred to as a second rear passenger seat).

[0192] A third camera module 2013 is installed in front of the vehicle 5, and a fourth camera module 2014 is installed in front of the rear seats of the vehicle 5. The third camera module 2013 has an imaging field of view that includes a passenger (front seat passenger) and a portion of the driver. The fourth camera module 2014 has an imaging field of view that includes a passenger (second rear seat passenger) sitting in the rear seat and a portion of the passenger sitting next to him (first rear seat passenger). This allows the first camera module 2011 and the third camera module 2013 to capture images of the driver and passenger sitting in the front seats, and the second camera module 2002 and the fourth camera module 2014 to capture images of the passenger sitting in the rear seats.

[0193] <Operation and Effect of Fifth Embodiment> As described above, according to the fifth embodiment, the first camera module 2011, the second camera module 2012, the third camera module 2013, and the fourth camera module 2014 for narrow angles are arranged at appropriate positions within the vehicle 5, thereby making it possible to sufficiently cover a required monitoring area within the vehicle 5. Moreover, the first camera module 2011, the second camera module 2012, the third camera module 2013, and the fourth camera module 2014 for narrow angles are less expensive and consume less power than the camera modules for wide angles.

[0194] <First Modification of Fifth Embodiment> Fig. 36 is a schematic configuration diagram showing a partial example of an image transmission system 1001B according to a first modification of the fifth embodiment of the present disclosure. In Fig. 36, the same parts as in Fig. 35 are assigned the same reference numerals, and detailed description thereof will be omitted. Fig. 36(a) shows a side view of a vehicle 5, and Fig. 36(b) shows a top view of the vehicle 5.

[0195] In a first modified example of the fifth embodiment of the present disclosure, a first camera module 2011 is installed in the front of the vehicle 5, and a second camera module 2012 is installed behind the rear seat of the vehicle 5. In addition, a third camera module 2013 is installed in the front of the vehicle 5, and a fourth camera module 2014 is installed behind the rear seat of the vehicle 5.

[0196] <Operational Effects of the First Modification of the Fifth Embodiment> As described above, the first modification of the fifth embodiment also provides the same operational effects as the fifth embodiment.

[0197] <Second Modification of Fifth Embodiment> Fig. 37 is a schematic configuration diagram showing a partial example of an image transmission system 1001C according to a second modification of the fifth embodiment of the present disclosure. In Fig. 37, the same parts as in Fig. 35 are assigned the same reference numerals, and detailed description thereof will be omitted. Fig. 37(a) shows a side view of a vehicle 5, and Fig. 37(b) shows a top view of the vehicle 5.

[0198] In a second modified example of the fifth embodiment of the present disclosure, a first camera module 2011 is installed in the front of the vehicle 5, and a second camera module 2012 is installed in front of the rear seat of the vehicle 5. In addition, a third camera module 2013 is installed in the front of the vehicle 5, and a fourth camera module 2014 is installed in front of the rear seat of the vehicle 5.

[0199] The first camera module 2011 has an imaging field of view that includes a passenger (front seat passenger) and a portion of the driver. The second camera module 2012 has an imaging field of view that includes a passenger (second rear seat passenger) sitting in the back seat and a portion of the passenger sitting next to him (first rear seat passenger). The third camera module 2013 has an imaging field of view that includes a portion of the passenger (front seat passenger) and the driver. The fourth camera module 2014 has an imaging field of view that includes a passenger (first rear seat passenger) sitting in the back seat and a portion of the passenger sitting next to him (second rear seat passenger). As a result, the first camera module 2011 and the third camera module 2013 can capture images of the driver and passenger sitting in the front seats, and the second camera module 2002 and the fourth camera module 2014 can capture images of the passenger sitting in the back seat.

[0200] <Operational Effects of the Second Modification of the Fifth Embodiment> As described above, the second modification of the fifth embodiment also provides the same operational effects as the fifth embodiment.

[0201] <Third Modification of Fifth Embodiment> Fig. 38 is a schematic configuration diagram showing a partial example of an image transmission system 1001D according to a third modification of the fifth embodiment of the present disclosure. In Fig. 38, the same parts as in Fig. 35 are assigned the same reference numerals, and detailed description thereof will be omitted. Fig. 38(a) shows a side view of a vehicle 5, and Fig. 38(b) shows a top view of the vehicle 5.

[0202] In a third modified example of the fifth embodiment of the present disclosure, a first camera module 2011 is installed in the front of the vehicle 5, and a second camera module 2012 is installed behind the rear seat of the vehicle 5. In addition, a third camera module 2013 is installed in the front of the vehicle 5, and a fourth camera module 2014 is installed behind the rear seat of the vehicle 5.

[0203] The first camera module 2011 has an imaging field of view that includes a passenger (front seat passenger) and a portion of the driver. The second camera module 2012 has an imaging field of view that includes a passenger sitting in the back seat (second rear seat passenger) and a portion of the passenger sitting next to him (first rear seat passenger). The third camera module 2013 has an imaging field of view that includes a portion of the passenger (front seat passenger) and the driver. The fourth camera module 2014 has an imaging field of view that includes a passenger sitting in the back seat (first rear seat passenger) and a portion of the passenger sitting next to him (second rear seat passenger).

[0204] <Operational Effects of the Third Modification of the Fifth Embodiment> As described above, the third modification of the fifth embodiment also provides the same operational effects as the fifth embodiment.

[0205] <Fourth Modification of Fifth Embodiment> Fig. 39 is a schematic configuration diagram showing a partial example of an image transmission system 1001E according to a fourth modification of the fifth embodiment of the present disclosure. In Fig. 39, the same parts as in Fig. 35 are assigned the same reference numerals, and detailed description thereof will be omitted. Fig. 39(a) shows a side view of a vehicle 5, and Fig. 39(b) shows a top view of the vehicle 5.

[0206] In a fourth modified example of the fifth embodiment of the present disclosure, a first camera module 2011 is installed in the front of the vehicle 5, and a second camera module 2012 is installed in front of the rear seat of the vehicle 5. In addition, a third camera module 2013 is installed in the front of the vehicle 5, and a fourth camera module 2014 is installed in front of the rear seat of the vehicle 5.

[0207] The first camera module 2011 has an imaging field of view that includes a passenger (front seat passenger) and a portion of the driver. The second camera module 2012 has an imaging field of view that includes a passenger sitting in the back seat (first rear seat passenger) and a portion of the passenger sitting next to him (second rear seat passenger). The third camera module 2013 has an imaging field of view that includes a portion of the passenger (front seat passenger) and the driver. The fourth camera module 2014 has an imaging field of view that includes a passenger sitting in the back seat (second rear seat passenger) and a portion of the passenger sitting next to him (first rear seat passenger).

[0208] <Operational Effects of the Fourth Modification of the Fifth Embodiment> As described above, the fourth modification of the fifth embodiment also provides the same operational effects as the fifth embodiment.

[0209] <Fifth Modification of Fifth Embodiment> Fig. 40 is a schematic configuration diagram showing a partial example of an image transmission system 1001F according to a fifth modification of the fifth embodiment of the present disclosure. In Fig. 40, the same parts as in Fig. 35 are assigned the same reference numerals, and detailed description thereof will be omitted. Fig. 40(a) shows a side view of a vehicle 5, and Fig. 40(b) shows a top view of the vehicle 5.

[0210] In a fifth modified example of the fifth embodiment of the present disclosure, a first camera module 2011 is installed in the front of the vehicle 5, and a second camera module 2012 is installed in front of the rear seat of the vehicle 5. In addition, a third camera module 2013 is installed in the front of the vehicle 5, and a fourth camera module 2014 is installed in front of the rear seat of the vehicle 5.

[0211] The first camera module 2011 has an imaging field of view that covers the driver and a portion of the passenger (front seat passenger). The second camera module 2012 has an imaging field of view that covers the passenger sitting in the back seat (first rear seat passenger) and a portion of the passenger sitting next to him (second rear seat passenger). The third camera module 2013 has an imaging field of view that covers a portion of the driver and the passenger (front seat passenger). The fourth camera module 2014 has an imaging field of view that covers the passenger sitting in the back seat (second rear seat passenger) and a portion of the passenger sitting next to him (first rear seat passenger).

[0212] <Operational Effects of Fifth Modification of Fifth Embodiment> As described above, the fifth modification of the fifth embodiment also provides the same operational effects as the fifth embodiment. While the fifth embodiment and each modification have described in detail an example of fusion of the iTOF sensor 21 and the RGB sensor 22, the iTOF sensor 21 and an RGB / IR sensor, or the iTOF sensor 21 and an IR sensor, may also be used. Furthermore, a TOF sensor based on a different principle other than the iTOF may also be substituted. Other camera configurations may also include combinations of an RGB sensor and an RGB sensor, an RGB sensor and an IR sensor, an RGB sensor and an RGBIR sensor, an RGBIR sensor and an RGBIR sensor, or an IR sensor and an IR sensor.

[0213] Sixth Embodiment Fig. 41 is a schematic configuration diagram showing a partial example of an image transmission system 1100A according to a sixth embodiment of the present disclosure. In Fig. 41, the same parts as those in Fig. 25 above are assigned the same reference numerals, and detailed description thereof will be omitted. Fig. 41(a) shows a side view of a vehicle 5, and Fig. 41(b) shows a top view of the vehicle 5. The image transmission system 1100A includes two wide-angle camera modules, a first camera module 2001 and a second camera module 2002, a pedal camera module 2100, and a millimeter-wave module 2200.

[0214] In a sixth embodiment of the present disclosure, an example will be described in which two types of image sensors, an iTOF sensor 21 and an RGB sensor 22, having different detection characteristics are used to integrate (fuse) information about an object recognized based on the output of each image sensor to recognize the object with high accuracy.

[0215] The pedal camera module 2100 captures an image of the accelerator pedal and brake pedal of the vehicle 5 within its field of view and transmits pedal image data to the ECU 3. The millimeter wave module 2200 obtains image data including objects such as the driver and passengers within a monitoring area within the vehicle 5 using millimeter waves, and transmits the image data to the ECU 3. Note that the sixth embodiment of the present disclosure may include only the pedal camera module 2100.

[0216] <Effects of the Sixth Embodiment> As described above, the sixth embodiment provides the same effects as the fourth embodiment, and also contributes to the safety of the vehicle 5 and safety measures for the driver and passengers, since the state of the pedals of the vehicle 5 can be confirmed by the pedal camera module 2100 and the state of the driver and passenger can be confirmed by the millimeter-wave module 2200.

[0217] <Modification of Sixth Embodiment> Fig. 42 is a schematic configuration diagram showing a partial example of an image transmission system 1100B according to a modification of the sixth embodiment of the present disclosure. In Fig. 42, the same parts as those in Fig. 41 above are assigned the same reference numerals, and detailed description thereof will be omitted. Fig. 42(a) shows a side view of a vehicle 5, and Fig. 42(b) shows a top view of the vehicle 5. The image transmission system 1100B includes four narrow-angle camera modules: a first camera module 2011, a second camera module 2012, a third camera module 2013, and a fourth camera module 2014.

[0218] <Effects of the Modification of the Sixth Embodiment> As described above, the modification of the sixth embodiment provides the same effects as those of the fifth and sixth embodiments. In the sixth embodiment and its modifications, an example has been described in which the accelerator pedal and brake pedal of the vehicle 5 are used as the imaging field of view and the pedal camera module 2100 is used to generate pedal image data. However, in order to confirm the depression positions of the accelerator pedal and brake pedal of the vehicle 5, only an iTOF module may be used. Furthermore, in the sixth embodiment and its modifications, an example of fusion of the iTOF sensor 21 and the RGB sensor 22 has been described in detail. However, the iTOF sensor 21 and the RGB / IR sensor, or the iTOF sensor 21 and the IR sensor may also be used. Furthermore, a TOF sensor other than the iTOF sensor, which operates based on a different principle, may also be used. Other camera configurations may include combinations of an RGB sensor and an RGB sensor, an RGB sensor and an IR sensor, an RGB sensor and an RGBIR sensor, an RGBIR sensor and an RGBIR sensor, or an IR sensor and an IR sensor.

[0219] Seventh Embodiment The image transmission system 1100A according to the sixth embodiment can be applied to an image transmission system according to a seventh embodiment. An example of the image transmission system according to the seventh embodiment to which the image transmission system 1100A is applied will be described below.

[0220] In the seventh embodiment of the present disclosure, an example will be described in which two types of image sensors, an iTOF sensor 21 and an RGB sensor 22, having different detection characteristics are used to integrate (fuse) information about an object recognized based on the output of each image sensor to recognize the object with high accuracy.

[0221] The control circuit 23 for the iTOF sensor 21 changes the image compression method and image resolution of the encoding circuit 212 in response to the abnormality notification from the ECU 3. The control circuit 24 for the RGB sensor 22 changes the image compression method and image resolution of the encoding circuit 222 in response to the abnormality notification from the ECU 3.

[0222] The control circuit 34 of the ECU 3 monitors for abnormalities in the operating state of the vehicle 5 based on the pedal image data transmitted from the pedal camera module 2100 and the image data transmitted from the millimeter wave module 2200, and if an abnormality is detected, transmits information about the abnormality as control data to the first camera module 2001 and the second camera module 2002.

[0223] (Image Compression Control Processing in Accordance with Abnormal Operation State of Vehicle 5) Fig. 43 is a flowchart showing a control procedure for executing image compression control processing by the ECU 3. Fig. 44 is a flowchart showing the detailed control procedure of step ST43f shown in Fig. 43.

[0224] In Figure 43, when the control circuit 34 of the ECU 3 detects that the engine of the vehicle 5 has started or the ignition has been turned on (step ST43a), it sets the image compression method to compression pattern 1 (compression pattern shown in Figure 20) (step ST43b), and when it confirms that the vehicle 5 is idling, i.e., that the vehicle 5 is stopped, it transmits control data including reversible compression to the control circuits 23 for iTOF of the first camera module 2001 and the second camera module 2002, and transmits control data including reversible compression to the control circuits 24 for RGB of the first camera module 2001 and the second camera module 2002 (step ST43c).

[0225] Then, control circuit 34 confirms the positions of people sitting in the seats of vehicle 5, i.e., the driver, passenger, and the left and right rear seats, through image recognition by image reproduction circuit 33 (step ST43d). If the driver and passenger are recognized, control circuit 34 changes the image compression method to, for example, compression pattern 2 (compression pattern shown in FIG. 21) (step ST43e), and transmits control data including lossy compression to control circuits 24 for RGB of first camera module 2001 and second camera module 2002.

[0226] Next, the control circuit 34 constantly monitors the operating state of the vehicle 5 and changes the setting of the image compression method in accordance with the operating state of the vehicle 5 (step ST43f). The control circuit 34 also monitors whether the engine of the vehicle 5 is stopped (step ST43g), and if it determines that the engine of the vehicle 5 is not stopped (No), it proceeds to the processing of step ST43f and continues changing the setting of the image compression method in accordance with the operating state of the vehicle 5. On the other hand, if it determines that the engine of the vehicle 5 has stopped and n minutes have passed since it was stopped (Yes), the control circuit 34 ends the image compression control processing.

[0227] 44, after changing the image compression method to compression pattern 2 in step ST43e, the control circuit 34 monitors whether the amount of data transmitted through the serial cable 4 will overflow, i.e., whether the number of image frames per second exceeds 60 frames (step ST43f1). If it determines that overflow will occur (step ST43f1: Yes), it changes the image compression method to, for example, compression pattern 3 (the compression pattern shown in FIG. 22) (step ST43f2) and transmits control data to the iTOF control circuit 23 of the first camera module 2001, including a compression method that performs lossless compression on the image areas of the driver and passenger and lossy compression on the other areas. Thereafter, the control circuit 34 returns to the processing of step ST43f1 and monitors whether the amount of data transmitted through the serial cable 4 will overflow.

[0228] (When vehicle 5 is driving) On the other hand, if it is not determined that overflow will occur (step ST43f1: No), the control circuit 34 monitors whether the vehicle 5 is in a driving state (step ST43f3), and if it determines that the vehicle 5 is in a driving state (step ST43f3: Yes), it determines the abnormal mode of the vehicle 5 based on the pedal image data transmitted from the pedal camera module 2100 and the image data transmitted from the millimeter wave module 2200. Here, when the speed of the vehicle 5 exceeds 50 km / h, when the accelerator pedal is pressed more than usual, when abnormalities are observed in the driver such as closing the eyes, removing the hands from the steering wheel, or unstable posture, when the brake pedal is pressed suddenly, when the seat belt is not fastened, or when the window or sunroof is opened while driving, the control circuit 34 determines that the abnormal mode is in effect (step ST43f4: Yes), and transmits control data to the first camera module 2001 and the second camera module 2002 to increase the image resolution of the front or rear of the vehicle 5 from 30 fps to 60 fps (step ST43f5). Thereafter, the control circuit 34 proceeds to the processing of step ST43f1 and monitors whether the amount of data transmitted through the serial cable 4 overflows.

[0229] If it is determined in step ST43f4 that the vehicle is not in abnormal mode (No), the control circuit 34 sends control data to the first camera module 2001 and the second camera module 2002 to reduce the image resolution in front of or behind the vehicle 5 from 60 fps to 30 fps (step ST43f6).

[0230] (When vehicle 5 is stopped) In step ST43f3, if it is determined that the vehicle 5 is not in a driving state (step ST43f3: No), the control circuit 34 monitors whether the vehicle 5 is in an idling state (step ST43f4), and if it is determined that the vehicle 5 is in an idling state (step ST43f4: Yes), it proceeds to the processing of step ST43f1 and monitors whether the amount of data transmitted through the serial cable 4 overflows.

[0231] (When the engine of the vehicle 5 is stopped) When it is determined in step ST43f7 that the vehicle 5 is not in an idling state (step ST43f7: No), the control circuit 34 proceeds to the process of step ST43g.

[0232] <Effects of the Seventh Embodiment> As described above, according to the seventh embodiment, when an abnormality occurs in the vehicle 5 or the driver, the image resolution of the captured image of the driver or passenger is increased based on the pedal image data transmitted from the pedal camera module 2100 and the image data transmitted from the millimeter wave module 2200, so that the cause of the abnormality can be reliably identified.

[0233] Eighth Embodiment At least one of the image transmission system 1001A according to the fifth embodiment, the image transmission system 1001B according to the first modified example of the fifth embodiment, the image transmission system 1001C according to the second modified example of the fifth embodiment, the image transmission system 1001D according to the third modified example of the fifth embodiment, the image transmission system 1001E according to the fourth modified example of the fifth embodiment, and the image transmission system 1001F according to the fifth modified example of the fifth embodiment can be applied to an image transmission system according to the eighth embodiment. An example of an image transmission system according to the eighth embodiment to which the image transmission systems 1001A, 1001B, 1001C, 1001D, 1001E, and 1001F are applied will be described below.

[0234] In the eighth embodiment of the present disclosure, an example will be described in which two types of image sensors, an iTOF sensor 21 and an RGB sensor 22, having different detection characteristics are used to integrate (fuse) information about an object recognized based on the output of each image sensor to recognize the object with high accuracy.

[0235] 45 to 50 are screen diagrams showing image compression methods according to the operating state of the vehicle 5. In each of Figs. 45 to 50, (a) shows RGB image data, and (b) shows iTOF image data. In each of Figs. 45 to 50, (a) and (b) show (1) when the vehicle 5 is idling (during authentication), (2) when the vehicle 5 is in operation (driving state), and (3) when the vehicle 5 is stopped. In each of Figs. 45 to 50, (a) and (b) show image areas compressed using lossless compression, and (b) shows image areas compressed using lossy compression, indicated by dots.

[0236] When the control circuit 34 of the ECU 3 detects that the engine of the vehicle 5 has started or the ignition has been turned on, it sets the image compression method to compression pattern 1 (compression pattern shown in Figure 45), and when it confirms that the vehicle 5 is idling, i.e., that the vehicle 5 is stopped, it transmits control data including reversible compression to the control circuits 23 for iTOF of the first camera module 2011 and the third camera module 2013, and also transmits control data including reversible compression to the control circuits 24 for RGB of the first camera module 2011 and the third camera module 2013.

[0237] The control circuit 34 then confirms the positions of people sitting in the seats of the vehicle 5, i.e., the driver, passenger, and the left and right rear areas, through image recognition by the image reproduction circuit 33. Here, when the driver and passenger are recognized, the control circuit 34 changes the image compression method to, for example, compression pattern 2 (the compression pattern shown in FIG. 46 ), and transmits control data to the iTOF control circuits 23 of the first camera module 2011 and the third camera module 2013 to change the peripheral areas of lossless compression to lossy compression, and also transmits control data to the RGB control circuits 24 of the first camera module 2011 and the third camera module 2013 to change the peripheral areas of lossless compression to lossy compression.

[0238] If the control circuit 34 determines that the amount of data transmitted through the serial cable 4 will overflow after changing the image compression method to compression pattern 2, it changes the image compression method to, for example, compression pattern 3 (the compression pattern shown in FIG. 47 ), and transmits control data to the iTOF control circuits 23 of the first camera module 2011 and the third camera module 2013 to perform lossless compression on the image areas of the driver and passenger, and lossy compression on the RGB control circuits 24. The amount of data transmitted using compression pattern 2 is reduced compared to the amount of data transmitted using compression pattern 1. Furthermore, the amount of data transmitted using compression pattern 3 is reduced compared to the amount of data transmitted using compression pattern 2.

[0239] When the vehicle 5 is being driven, if the control circuit 34 determines that overflow will occur even if the image compression method is changed to compression pattern 3, it changes the image compression method to, for example, compression pattern 4 (the compression pattern shown in FIG. 48 ), and transmits control data to the control circuits 23 for the iTOF of the first camera module 2011 and the third camera module 2013, including a compression method for lossy compression of the areas surrounding the image areas of the driver and passenger.

[0240] If the control circuit 34 determines that overflow will occur even when the image compression method is changed to compression pattern 4 while the vehicle 5 is being driven, it changes the image compression method to, for example, compression pattern 5 (the compression pattern shown in FIG. 49 ), and transmits control data to the iTOF control circuit 23 of the first camera module 2011, the control data including a compression method that performs lossy compression on image areas other than the driver. The amount of data transmitted using compression pattern 4 is reduced compared to the amount of data transmitted using compression pattern 3. Furthermore, the amount of data transmitted using compression pattern 5 is reduced compared to the amount of data transmitted using compression pattern 4.

[0241] When the vehicle 5 is being driven, if the control circuit 34 determines that overflow will occur even if the compression pattern is changed to 5, it changes the image compression method to, for example, compression pattern 6 (the compression pattern shown in Figure 50), and transmits control data to the control circuit 23 for the iTOF of the third camera module 2013, including a compression method that performs lossless compression on only the image area of ​​the driver's face and lossy compression on the other areas.

[0242] <Effects of the Eighth Embodiment> As described above, the eighth embodiment provides the same effects as the third embodiment. While the eighth embodiment describes in detail an example of fusion of the iTOF sensor 21 and the RGB sensor 22, the iTOF sensor 21 and an RGB / IR sensor, or the iTOF sensor 21 and an IR sensor, may also be used. Furthermore, a TOF sensor based on a different principle other than the iTOF may also be substituted. Other camera configurations may include combinations of an RGB sensor and an RGB sensor, an RGB sensor and an IR sensor, an RGB sensor and an RGBIR sensor, an RGBIR sensor and an RGBIR sensor, or an IR sensor and an IR sensor.

[0243] <Other Embodiments> As described above, the present technology has been described using the first to eighth embodiments. However, the descriptions and drawings that form part of this disclosure should not be understood to limit the present technology. Upon understanding the gist of the technical content disclosed in the above embodiments, it will be apparent to those skilled in the art that various alternative embodiments, examples, and operational techniques may be included in the present technology. Furthermore, the configurations disclosed in the first to eighth embodiments may be appropriately combined within a range that does not cause contradictions. For example, configurations disclosed in multiple different embodiments may be combined, or configurations disclosed in multiple different modified examples of the same embodiment may be combined.

[0244] The present disclosure may also be configured as follows: (1) An image transmission device comprising: a signal processing unit connected to an image sensor and performing signal processing on image data output from the image sensor and including an object within an imaging field of view inside a vehicle; a control unit controlling the signal processing of the signal processing unit; a data transmission unit transmitting, as transmission data to an external device, the image data signal-processed by the signal processing unit and control data obtained by control processing of the control unit and necessary for reproducing the image data; and a compression unit compressing at least one of the image data and the control data into compressed data having a data amount smaller than the transmission data, and including the compressed data in the transmission data. (2) The image transmission device described in (1) above, wherein the compression unit compresses at least one of the image data and the control data into the compressed data using either lossless compression or lossy compression. (3) An image receiving device comprising: a data receiving unit that receives transmission data transmitted from an external device; a decoding unit that converts compressed data included in the transmission data back into image data including an object within an imaging field of view inside the automobile or control data required for reproducing the image data; an image reproduction unit that reproduces the image data; and a control unit that controls the reproduction process by the image reproduction unit based on the control data.(4) An image transmission system comprising: an image transmitting device, an image receiving device, and a serial transmission path connecting the image transmitting device and the image receiving device, wherein the image transmitting device comprises: a signal processing unit to which an image sensor is connected and which performs signal processing on image data output from the image sensor and including an object within an imaging field of view inside the automobile; a transmitting-side control unit which controls the signal processing of the signal processing unit; a data transmitting unit which transmits, as transmission data to the image receiving device, image data generated by the signal processing unit and control data obtained by control processing of the transmitting-side control unit and necessary for reproducing the image data; and a compression unit which compresses at least one of the image data and the control data into compressed data with a data amount smaller than the transmission data and includes the compressed data in the transmission data, wherein the image receiving device comprises: a data receiving unit which receives the transmission data transmitted from the image transmitting device; a decoding unit which converts the compressed data included in the transmission data back into image data including an object within an imaging field of view inside the automobile or control data necessary for reproducing the image data; an image reproducing unit which reproduces the image data; and a receiving-side control unit which controls the reproduction processing by the image reproducing unit based on the control data. (5) The image transmission system according to (4) above, wherein the compression unit compresses at least one of the image data and the control data into the compressed data using either lossless compression or lossy compression. (6) The image transmission system according to (4) above, wherein the compression unit encrypts the compressed data with encryption key information to generate encrypted compressed data and includes the encryption key information in the control data, and the decryption unit decrypts the encrypted compressed data included in the transmission data based on the encryption key information included in the control data.(7) A first signal processing unit that connects a first image sensor and performs signal processing on first image data output from the first image sensor and including objects within an imaging field of view inside the automobile; a first control unit that controls the signal processing of the first signal processing unit; a second signal processing unit that connects a second image sensor having a function different from that of the first image sensor and performs signal processing on second image data output from the second image sensor and including objects within an imaging field of view inside the automobile; a second control unit that controls the signal processing of the second signal processing unit; and a data transmission unit that transmits, as transmission data, to an external device, the first image data signal-processed by the first signal processing unit, the second image data signal-processed by the second signal processing unit, first control data obtained by control processing of the first control unit and necessary for reproducing the first image data, and second control data obtained by control processing of the second control unit and necessary for reproducing the second image data. and a compression unit that compresses at least one of the first image data, the second image data, the first control data, and the second control data into compressed data having a data amount smaller than the transmission data, and includes the compressed data in the transmission data. (8) The image transmission device according to (7) above, wherein the compression unit compresses at least one of the first image data, the second image data, the first control data, and the second control data into the compressed data using either lossless compression or lossy compression. (9) The image transmission device according to (7) above, wherein the compression unit compresses at least one of the first to fourth set of data into the compressed data when the first image data and the second image data are defined as a first set of data, the first image data and the second control data are defined as a second set of data, the first control data and the second image data are defined as a third set of data, and the first control data and the second control data are defined as a fourth set of data. (10) The image transmission device according to (9), wherein the compression unit compresses at least one of the first to fourth data sets into the compressed data using either lossless compression or lossy compression.(11) An image receiving device comprising: a data receiving unit that receives transmission data transmitted from an external device; a decoding unit that converts compressed data included in the transmission data back into at least one of first image data obtained by a first image sensor and including an object within an imaging field of view inside a vehicle, first control data required to reproduce the first image data, second image data obtained by a second image sensor having a function different from that of the first image sensor, and second control data required to reproduce the second image data; an image reproduction unit that reproduces the first image data and the second image data; and a control unit that controls reproduction processing by the image reproduction unit based on the first control data and the second control data.(12) An image transmitting device includes an image receiving device, and a serial transmission path connecting the image transmitting device and the image receiving device, wherein the image transmitting device includes: a first signal processing unit connected to a first image sensor and performing signal processing on first image data output from the first image sensor and including an object within an image field of view inside the automobile; a first transmitting side control unit controlling the signal processing of the first signal processing unit; a second image sensor connected to a second image sensor having a function different from that of the first image sensor and performing signal processing on second image data output from the second image sensor and including an object within an image field of view inside the automobile; a second transmitting side control unit controlling the signal processing of the second signal processing unit; and a data transmitting unit transmitting, as transmission data, to the image receiving device, the first image data signal-processed by the first signal processing unit, the second image data signal-processed by the second signal processing unit, first control data obtained by control processing of the first transmitting side control unit and necessary for reproducing the first image data, and second control data obtained by control processing of the second transmitting side control unit and necessary for reproducing the second image data. and a compression unit that compresses at least one of the first image data, the second image data, the first control data, and the second control data into compressed data having a data amount smaller than that of the transmission data, and includes the compressed data in the transmission data, wherein the image receiving device comprises: a data receiving unit that receives the transmission data transmitted from the image transmitting device, a decoding unit that restores the compressed data included in the transmission data to at least one of the first image data, the first control data, the second image data, and the second control data, an image playback unit that plays back the first image data and the second image data, and a receiving-side control unit that controls a playback process by the image playback unit based on the first control data and the second control data. (13) The image transmission system according to (12), wherein the compression unit compresses at least one of the first image data, the second image data, the first control data, and the second control data into the compressed data using either lossless compression or lossy compression.(14) The image transmission system according to (12), wherein the compression unit compresses at least one of the first to fourth set of data into the compressed data, where the first image data and the second image data are defined as a first set of data, the first image data and the second control data are defined as a second set of data, the first control data and the second image data are defined as a third set of data, and the first control data and the second control data are defined as a fourth set of data. (15) The image transmission system according to (14), wherein the compression unit compresses at least one of the first to fourth set of data into the compressed data using either lossless compression or lossy compression. (16) The image transmission system according to (12), wherein the compression unit encrypts the compressed data with encryption key information to generate encrypted compressed data and includes the encryption key information in the control data, and the decryption unit decrypts the encrypted compressed data included in the transmission data based on the encryption key information included in the control data. (17) The image transmission system according to (12) above, wherein a plurality of the image transmission devices are provided, and a first image transmission device among the plurality of image transmission devices is arranged so that its field of view covers at least a part of a first monitoring area inside the vehicle, and a second image transmission device is arranged so that its field of view covers at least a part of a second monitoring area different from the first monitoring area. (18) The image transmission system according to (17) above, wherein the first image sensor and the second image sensor of the first image transmission device have a field of view that covers the first monitoring area, and the first image sensor and the second image sensor of the second image transmission device have a field of view that covers the second monitoring area. (19) The image transmission system according to (18) above, wherein the first image sensor and the second image sensor of the first image transmission device have a field of view that is narrower than the first monitoring area, and the first image sensor and the second image sensor of the second image transmission device have a field of view that is narrower than the second monitoring area.(20) The image transmission system according to (19), further comprising: a third image transmitting device having the first image sensor and the second image sensor whose imaging field of view is the remaining area of ​​the first monitoring area; and a fourth image transmitting device having the first image sensor and the second image sensor whose imaging field of view is the remaining area of ​​the second monitoring area. (21) The image transmission system according to (12), further comprising: a pedal image sensor whose imaging field of view is the pedals of the automobile and which transmits pedal image data to the image receiving device. (22) The image transmission system according to (12), further comprising: a millimeter-wave radar that uses millimeter waves to obtain fourth image data including an object within the imaging field of view inside the automobile and transmits the fourth image data to the image receiving device. (23) An image transmitting device comprising: a first signal processing unit connected to a first image sensor and performing signal processing on first image data output from the first image sensor and including objects in an imaging field of view within the vehicle; a second signal processing unit connected to a second image sensor having a function different from that of the first image sensor and performing signal processing on second image data output from the second image sensor and including objects in an imaging field of view within the vehicle; a data transmitting unit transmitting the first image data signal-processed by the first signal processing unit and the second image data signal-processed by the second signal processing unit as transmission data to an external device; and a compression unit compressing at least one of the first image data and the second image data into compressed data with a data amount smaller than the transmission data in accordance with an operating state of the vehicle monitored by the external device, and including the compressed data in the transmission data. (24) The image transmitting device according to (23), wherein the compression unit compresses the first image data into first compressed data and the second image data into second compressed data in accordance with an operating state of the vehicle monitored by the external device. (25) The image transmission device according to (24), wherein the compression unit compresses the first image data into first compressed data by lossless compression, and compresses the second image data into second compressed data by lossless compression. (26) The image transmission device according to (24), wherein the compression unit compresses the first image data into first compressed data by lossy compression, and compresses the second image data into second compressed data by lossless compression.(27) The image transmitting device according to (24), wherein the compression unit generates the second compressed data by compressing the first image data using lossy compression to first compressed data, compressing at least a portion of data of the object in the second image data using lossless compression, and compressing other data of the second image data using lossy compression. (28) An image receiving device comprising: a data receiving unit that receives transmission data transmitted from an external device; a decoding unit that decodes compressed data included in the transmission data into at least one of first image data obtained by a first image sensor and including an object within an imaging field of view inside the vehicle, and second image data obtained by a second image sensor having a function different from that of the first image sensor; an image reproducing unit that reproduces the first image data and the second image data; and a control unit that monitors the operating state of the vehicle and transmits the monitoring result to the external device as control data. (29) The image receiving device according to (28), wherein the control unit monitors at least one of an idling state, a driving state, and a stopped state of the vehicle.(30) An image transmitting device includes an image receiving device, and a serial transmission path connecting the image transmitting device and the image receiving device, wherein the image transmitting device includes: a first signal processing unit connected to a first image sensor and performing signal processing on first image data output from the first image sensor and including an object in an imaging field of view inside the automobile; a second signal processing unit connected to a second image sensor having a function different from that of the first image sensor and performing signal processing on second image data output from the second image sensor and including an object in an imaging field of view inside the automobile; a data transmitting unit transmitting the first image data signal-processed by the first signal processing unit and the second image data signal-processed by the second signal processing unit as transmission data to the image receiving device via the serial transmission path; and a compression unit compressing at least one of the first image data and the second image data into compressed data with a data amount smaller than the transmission data according to the operating state of the automobile monitored by the image receiving device, and including the compressed data in the transmission data; and the image receiving device includes: a data receiving unit receiving the transmission data transmitted from the image transmitting device; an image transmission system comprising: a decoding unit that converts compressed data included in the transmission data back into at least one of the first image data and the second image data; an image playback unit that plays back the first image data and the second image data; and a control unit that monitors the operating state of the vehicle and transmits the monitoring results as control data to the image transmission device. (31) The image transmission system according to (30) above comprises a plurality of the image transmission devices, and a first image transmission device among the plurality of image transmission devices is arranged so that its field of view is at least a part of a first monitoring area within the vehicle, and a second image transmission device is arranged so that its field of view is at least a part of a second monitoring area different from the first monitoring area. (32) The image transmission system according to (31) above, wherein the first image sensor and the second image sensor of the first image transmission device have field of view that is the first monitoring area, and the first image sensor and the second image sensor of the second image transmission device have field of view that is the second monitoring area.(33) The image transmission system according to (31), wherein the first image sensor and the second image sensor of the first image transmission device have an imaging field of view that is narrower than the first monitoring area, and the first image sensor and the second image sensor of the second image transmission device have an imaging field of view that is narrower than the second monitoring area. (34) The image transmission system according to (33), further comprising: a third image transmission device having the first image sensor and the second image sensor whose imaging field of view is the remaining area of ​​the first monitoring area; and a fourth image transmission device having the first image sensor and the second image sensor whose imaging field of view is the remaining area of ​​the second monitoring area. (35) The image transmission system according to (30), further comprising a pedal image sensor whose imaging field of view is a pedal of the automobile and which transmits pedal image data to the image receiving device as third image data. (36) The image transmission system according to (35), further comprising a millimeter-wave radar that uses millimeter waves to obtain fourth image data including an object within the imaging field of view inside the automobile and transmits the fourth image data to the image receiving device. (37) The image transmission system described in (36) above, wherein the control unit monitors the operating state of the vehicle based on the third image data and the fourth image data, and when an abnormality is detected, transmits information about the abnormality to the image transmitting device as control data, and when the image transmitting device is notified of the abnormality by the image receiving device, the image transmitting device increases the resolution of at least one of the first image data and the second image data. (38) The image transmission system described in (30) above, wherein the compression unit compresses at least one of the first image data and the second image data into the compressed data using either lossless compression or lossy compression. (39) The image transmission system described in (30) above, wherein the compression unit encrypts the compressed data with encryption key information to generate encrypted compressed data and includes the encryption key information in control data to be transmitted to the image receiving device, and the decryption unit decrypts the encrypted compressed data included in the transmission data based on the encryption key information included in the control data.(40) The image transmission system according to (38) or (39), wherein the control unit of the image receiving device controls the first signal processing unit, the second signal processing unit, and the compression unit of the image transmitting device, and sets a compression method for at least one of the first image data and the second image data by the compression unit according to the operating state of the vehicle. (41) The image transmission system according to (40), wherein the control unit of the image receiving device fixes the compression method by the compression unit, and operates the first signal processing unit, the second signal processing unit, and the compression unit of the image transmitting device from the start of the engine of the vehicle to the stop of the engine.

[0245] 1A, 1B, 1C, 1D, 1E, 1F, 1G, 1H, 1I, 1J, 100A, 100B, 100C, 100D, 100E, 100F, 1000A, 1000B, 1000C, 1000D, 1000E, 1000F, 1000G, 1000H, 1001A, 1001B, 1001C, 1001D, 1001E, 1001F, 1100A, 1100B Image transmission system 2, 200 Camera module 4, 41, 42 Serial cable 5 Vehicle 21 iTOF sensor 22 RGB sensor 23, 24, 34 Control circuit 25 Serializer circuit 26, 27, 28 Encoder circuit 29, 36, 209, 291, 292, 306 Control bus 31 Deserializer circuit 32 Decode circuit 33 Image reproduction circuit 35 Decode / decryption circuit 51, 52, 53 Encryption / encode circuit 201 RGB(x) sensor 202, 213, 223 Serializer circuit 203 Control circuit 205, 214, 224 Encryption / encode circuit 212, 222, 2014 Encode circuit 211, 221 Logic circuit 301 Deserializer circuit 302 Decode circuit 303 Image reproduction circuit 304 Control circuit 305 Decode / decryption circuit 400 Serial cable 2001 First camera module 2002 Second camera module 2003 Third camera module 2004 Fourth camera module 2100 Pedal camera module 2200 millimeter wave radar

Claims

1. An image transmitting device comprising: a signal processing unit to which an image sensor is connected and which performs signal processing on image data output from the image sensor and including an object within an imaging field of view within an automobile; a control unit which controls the signal processing of the signal processing unit; a data transmitting unit which transmits the image data signal-processed by the signal processing unit and control data obtained by control processing of the control unit and necessary for reproducing the image data as transmission data to an external device; and a compression unit which compresses at least one of the image data and the control data into compressed data with a data volume smaller than the transmission data, and includes the compressed data in the transmission data.

2. The image transmitting device according to claim 1, wherein the compression section compresses at least one of the image data and the control data into the compressed data by either lossless compression or lossy compression.

3. An image receiving device comprising: a data receiving unit that receives transmission data sent from an external device; a decoding unit that converts compressed data included in the transmission data back into image data including an object within the imaging field of view inside the automobile, or control data required for reproducing the image data; an image reproduction unit that reproduces the image data; and a control unit that controls the reproduction process by the image reproduction unit based on the control data.

4. An image transmission system comprising an image transmitting device, an image receiving device, and a serial transmission path connecting the image transmitting device and the image receiving device, wherein the image transmitting device comprises: a signal processing unit to which an image sensor is connected and which performs signal processing on image data output from the image sensor and including an object within the imaging field of view inside the automobile; a transmitting side control unit which controls the signal processing of the signal processing unit; a data transmitting unit which transmits to the image receiving device as transmission data the image data generated by the signal processing unit and control data obtained by control processing of the transmitting side control unit and necessary for reproducing the image data; and a compression unit which compresses at least one of the image data and the control data into compressed data with a data amount smaller than the transmission data and includes the compressed data in the transmission data, and wherein the image receiving device comprises: a data receiving unit which receives the transmission data transmitted from the image transmitting device; a decoding unit which converts the compressed data included in the transmission data back into image data including an object within the imaging field of view inside the automobile or control data necessary for reproducing the image data; an image reproduction unit which reproduces the image data; and a receiving side control unit which controls the reproduction processing by the image reproduction unit based on the control data.

5. The image transmission system according to claim 4, wherein the compression section compresses at least one of the image data and the control data into the compressed data by either lossless compression or lossy compression.

6. The image transmission system of claim 4, wherein the compression unit encrypts the compressed data with encryption key information to generate encrypted compressed data and includes the encryption key information in the control data, and the decryption unit decrypts the encrypted compressed data included in the transmission data based on the encryption key information included in the control data.

7. A first signal processing unit which connects a first image sensor and performs signal processing on first image data output from the first image sensor and including objects within an imaging field of view within the automobile; a first control unit which controls the signal processing of the first signal processing unit; a second signal processing unit which connects a second image sensor having a function different from that of the first image sensor and performs signal processing on second image data output from the second image sensor and including objects within an imaging field of view within the automobile; a second control unit which controls the signal processing of the second signal processing unit; and a data transmission unit which transmits, as transmission data, to an external device, the first image data signal-processed by the first signal processing unit, the second image data signal-processed by the second signal processing unit, first control data obtained by control processing of the first control unit and necessary for reproducing the first image data, and second control data obtained by control processing of the second control unit and necessary for reproducing the second image data. a compression unit that compresses at least one of the first image data, the second image data, the first control data, and the second control data into compressed data having a data amount smaller than that of the transmission data, and includes the compressed data in the transmission data.

8. The image transmitting device of claim 7, wherein the compression unit compresses at least one of the first image data, the second image data, the first control data, and the second control data into the compressed data using either lossless compression or lossy compression.

9. The image transmitting device of claim 7, wherein the compression unit compresses at least one of the first to fourth set data into the compressed data when the first image data and the second image data are a first set data, the first image data and the second control data are a second set data, the first control data and the second image data are a third set data, and the first control data and the second control data are a fourth set data.

10. The image transmitting device according to claim 9, wherein the compression section compresses at least one of the first to fourth group data into the compressed data by either lossless compression or lossy compression.

11. An image receiving device comprising: a data receiving unit that receives transmission data sent from an external device; a decoding unit that converts compressed data included in the transmission data into at least one of first image data obtained by a first image sensor and including an object within an imaging field of view inside a vehicle, first control data required to reproduce the first image data, second image data obtained by a second image sensor having a function different from that of the first image sensor, and second control data required to reproduce the second image data; an image reproduction unit that reproduces the first image data and the second image data; and a control unit that controls the reproduction process by the image reproduction unit based on the first control data and the second control data.

12. An image transmitting device, an image receiving device, and a serial transmission path connecting the image transmitting device and the image receiving device, the image transmitting device comprising: a first signal processing section for connecting a first image sensor and signal processing first image data output from the first image sensor and including an object within the imaging field of view inside the automobile; a first transmitting side control section for controlling the signal processing of the first signal processing section; a second signal processing section for connecting a second image sensor having a function different from that of the first image sensor and signal processing second image data output from the second image sensor and including an object within the imaging field of view inside the automobile; a second transmitting side control section for controlling the signal processing of the second signal processing section; and a data transmitting section for transmitting, as transmission data, to the image receiving device, the first image data signal-processed by the first signal processing section, the second image data signal-processed by the second signal processing section, first control data obtained by control processing of the first transmitting side control section and necessary for reproducing the first image data, and second control data obtained by control processing of the second transmitting side control section and necessary for reproducing the second image data. an image receiving device comprising: a data receiving unit that receives transmission data transmitted from the image transmitting device; a decoding unit that converts the compressed data included in the transmission data into at least one of the first image data, the second image data, the first control data, and the second control data; an image playback unit that plays back the first image data and the second image data; and a receiving-side control unit that controls playback processing by the image playback unit based on the first control data and the second control data.

13. The image transmission system of claim 12, wherein the compression unit compresses at least one of the first image data, the second image data, the first control data, and the second control data into the compressed data using either lossless compression or lossy compression.

14. The image transmission system of claim 12, wherein the compression unit compresses at least one of the first to fourth set data into the compressed data when the first image data and the second image data are a first set data, the first image data and the second control data are a second set data, the first control data and the second image data are a third set data, and the first control data and the second control data are a fourth set data.

15. The image transmission system according to claim 14, wherein the compression section compresses at least one of the first to fourth group data into the compressed data by either lossless compression or lossy compression.

16. The image transmission system of claim 12, wherein the compression unit encrypts the compressed data with encryption key information to generate encrypted compressed data and includes the encryption key information in the control data, and the decryption unit decrypts the encrypted compressed data included in the transmission data based on the encryption key information included in the control data.

17. The image transmission system of claim 12, further comprising a plurality of the image transmitting devices, wherein a first image transmitting device among the plurality of image transmitting devices is positioned so as to have an imaging field of view that covers at least a portion of a first monitoring area within the vehicle, and a second image transmitting device is positioned so as to have an imaging field of view that covers at least a portion of a second monitoring area different from the first monitoring area.

18. The image transmission system of claim 17, wherein the first image sensor and the second image sensor of the first image transmission device have an imaging field of view that covers the first monitoring area, and the first image sensor and the second image sensor of the second image transmission device have an imaging field of view that covers the second monitoring area.

19. The image transmission system described in claim 18, wherein the first image sensor and the second image sensor of the first image transmission device have an imaging field of view that is narrower than the first monitoring area, and the first image sensor and the second image sensor of the second image transmission device have an imaging field of view that is narrower than the second monitoring area.

20. The image transmission system of claim 19, further comprising: a third image transmitting device having the first image sensor and the second image sensor whose imaging field of view is the remaining area of ​​the first monitoring area; and a fourth image transmitting device having the first image sensor and the second image sensor whose imaging field of view is the remaining area of ​​the second monitoring area.

21. The image transmission system according to claim 12, further comprising a pedal image sensor that has an imaging field of view that includes the pedals of the automobile and transmits pedal image data to the image receiving device.

22. An image transmission system according to claim 12, further comprising a millimeter wave radar which obtains fourth image data including an object within an imaging field of view inside the automobile using millimeter waves, and transmits the fourth image data to the image receiving device.

23. An image transmitting device comprising: a first signal processing unit to which a first image sensor is connected and which performs signal processing on first image data output from the first image sensor and including an object within an imaging field of view within the vehicle; a second signal processing unit to which a second image sensor having a function different from that of the first image sensor is connected and which performs signal processing on second image data output from the second image sensor and including an object within an imaging field of view within the vehicle; a data transmitting unit to transmit the first image data signal-processed by the first signal processing unit and the second image data signal-processed by the second signal processing unit as transmission data to an external device; and a compression unit to compress at least one of the first image data and the second image data into compressed data having a smaller data amount than the transmission data according to the operating state of the vehicle monitored by the external device, and include the compressed data in the transmission data.

24. The image transmitting device according to claim 23, wherein the compression unit compresses the first image data into first compressed data and compresses the second image data into second compressed data according to an operating state of the vehicle monitored by an external device.

25. The image transmitting device according to claim 24, wherein the compression section compresses the first image data into first compressed data by lossless compression, and compresses the second image data into second compressed data by lossless compression.

26. The image transmitting device according to claim 24, wherein said compression section compresses said first image data into first compressed data by lossy compression, and compresses said second image data into second compressed data by lossless compression.

27. The image transmitting device of claim 24, wherein the compression unit generates the second compressed data by compressing the first image data into first compressed data using lossy compression, compressing at least a portion of the data of the object in the second image data using lossless compression, and compressing other data of the second image data using lossy compression.

28. An image receiving device comprising: a data receiving unit that receives transmission data sent from an external device; a decoding unit that converts compressed data included in the transmission data into at least one of first image data obtained by a first image sensor and including an object within an imaging field of view within the vehicle, and second image data obtained by a second image sensor having a function different from that of the first image sensor; an image reproducing unit that reproduces the first image data and the second image data; and a control unit that monitors the operating state of the vehicle and transmits the monitoring results to the external device as control data.

29. The image receiving device according to claim 28, wherein the control unit monitors at least one of an idling state, a driving state, and a stopped state of the automobile.

30. An image transmitting device, an image receiving device, and a serial transmission path connecting the image transmitting device and the image receiving device, wherein the image transmitting device comprises: a first signal processing section for connecting a first image sensor and signal processing first image data output from the first image sensor and including an object within the imaging field of view within the automobile; a second signal processing section for connecting a second image sensor having a function different from that of the first image sensor and signal processing second image data output from the second image sensor and including an object within the imaging field of view within the automobile; a data transmitting section for transmitting the first image data signal-processed by the first signal processing section and the second image data signal-processed by the second signal processing section as transmission data to the image receiving device via the serial transmission path; and a compression section for compressing at least one of the first image data and the second image data into compressed data with a data amount smaller than that of the transmission data according to the operating state of the automobile monitored by the image receiving device, and including the compressed data in the transmission data. The image receiving device comprises: a data receiving section for receiving the transmission data transmitted from the image transmitting device; An image transmission system comprising: a decoding unit that converts compressed data included in the transmission data back into at least one of the first image data and the second image data; an image playback unit that plays back the first image data and the second image data; and a control unit that monitors the operating state of the automobile and transmits the monitoring results to the image transmitting device as control data.

31. The image transmission system of claim 30, further comprising a plurality of the image transmitting devices, wherein a first image transmitting device among the plurality of image transmitting devices is positioned so as to have an imaging field of view that covers at least a portion of a first monitoring area within the vehicle, and a second image transmitting device is positioned so as to have an imaging field of view that covers at least a portion of a second monitoring area different from the first monitoring area.

32. An image transmission system as described in claim 31, wherein the first image sensor and the second image sensor of the first image transmission device have an imaging field of view that covers the first monitoring area, and the first image sensor and the second image sensor of the second image transmission device have an imaging field of view that covers the second monitoring area.

33. An image transmission system as described in claim 31, wherein the first image sensor and the second image sensor of the first image transmission device have an imaging field of view that is narrower than the first monitoring area, and the first image sensor and the second image sensor of the second image transmission device have an imaging field of view that is narrower than the second monitoring area.

34. The image transmission system of claim 33, further comprising: a third image transmitting device having the first image sensor and the second image sensor whose imaging field of view is the remaining area of ​​the first monitoring area; and a fourth image transmitting device having the first image sensor and the second image sensor whose imaging field of view is the remaining area of ​​the second monitoring area.

35. The image transmission system according to claim 30, further comprising a pedal image sensor that has an imaging field of view that includes the pedals of the automobile and transmits pedal image data to the image receiving device as third image data.

36. The image transmission system according to claim 35, further comprising a millimeter wave radar that obtains fourth image data including an object within an imaging field of view within the automobile using millimeter waves, and transmits the fourth image data to the image receiving device.

37. The image transmission system described in claim 36, wherein the control unit monitors the operating condition of the vehicle for abnormalities based on the third image data and the fourth image data, and when an abnormality is detected, transmits information about the abnormality as control data to the image transmitting device, and when notified of the abnormality by the image receiving device, the image transmitting device increases the resolution of at least one of the first image data and the second image data.

38. The image transmission system according to claim 30, wherein the compression section compresses at least one of the first image data and the second image data into the compressed data by either lossless compression or lossy compression.

39. An image transmission system as described in claim 30, wherein the compression unit encrypts the compressed data with encryption key information to generate encrypted compressed data and includes the encryption key information in control data to be transmitted to the image receiving device, and the decryption unit decrypts the encrypted compressed data included in the transmission data based on the encryption key information included in the control data.

40. An image transmission system as described in claim 38 or claim 39, wherein the control unit of the image receiving device controls the first signal processing unit, the second signal processing unit, and the compression unit of the image transmitting device, and sets the compression method of at least one of the first image data and the second image data by the compression unit depending on the operating state of the vehicle.

41. An image transmission system as described in claim 40, wherein the control unit of the image receiving device fixes the compression method used by the compression unit and operates the first signal processing unit, the second signal processing unit, and the compression unit of the image transmitting device from the time the engine of the automobile is started to the time the engine is stopped.