Vehicle-mounted imaging system

Abstract

Provided is a vehicle-mounted imaging system to be installed in a vehicle, wherein the vehicle-mounted imaging system makes it possible to simultaneously reduce size and weight, lower costs, and achieve scalability (versatility). The vehicle-mounted imaging system to be installed in a vehicle is characterized: by being provided with a first camera, a second camera installed at a different location than the first camera, and an image processing unit for processing images acquired by the first camera and the second camera; and in that the first camera and the second camera are connected by a first communication line, and the second camera and the image processing unit are connected by a second communication line.

Description

[Technical field]

[0001] The present invention relates to the configuration of an imaging system mounted on a vehicle, and more particularly to a technique that is effective when applied to a stereo camera that uses a plurality of cameras to generate parallax images and measure distances. [Background technology]

[0002] In order to prevent traffic accidents, advanced driver assistance systems (ADAS) for automobiles and motorcycles are being introduced. For example, in-vehicle cameras are used to monitor the area around the vehicle and recognize other vehicles, and are used to warn drivers (alerts) and avoid collisions.

[0003] The on-board cameras used in ADAS are stereo cameras that can record information about the depth direction by simultaneously capturing images of an object from different directions using multiple cameras (usually two).The size, position, and speed of multiple three-dimensional objects can also be detected by stereoscopic processing of images captured by multiple cameras.

[0004] ADAS is divided into systems for luxury vehicles and systems for general-purpose vehicles, and in-vehicle cameras need to be designed with scalability in mind.

[0005] In addition, with regard to in-vehicle cameras, development is also underway on sensing systems that add an additional camera to improve functionality and redundancy, as opposed to conventional sensing systems that install a monocular camera behind the rear-view mirror, etc.

[0006] Background art of this technical field includes, for example, technology such as that described in Patent Document 1. Patent Document 1 discloses "an information processing device that supports a user in obtaining an image suitable for a photographing instruction of an object given at any time by a camera installed on a moving object."

[0007] Furthermore, Patent Document 2 discloses a "vehicle driving environment detection device capable of accurately detecting objects not only in front of the vehicle but also on the left and right sides of the vehicle, and calculating distance information necessary for driving control."

[0008] Moreover, Patent Document 3 discloses an "image display system capable of improving driving safety."

[0009] Furthermore, Patent Document 4 discloses an "imaging control device capable of achieving highly accurate distance measurement using a pair of cameras aligned vertically." [Prior art documents] [Patent documents]

[0010] [Patent Document 1] JP 2020-106890 A [Patent Document 2] JP 2020-51942 A [Patent Document 3] JP 2013-62657 A [Patent Document 4] International Publication No. 2018 / 180579 Summary of the Invention [Problem to be solved by the invention]

[0011] As mentioned above, in-vehicle cameras for ADAS are required to be designed with scalability (versatility) in mind, as well as to have further improved functionality and redundancy as stereo cameras.

[0012] However, if one tries to achieve stereo vision by simply adding another camera to a conventional monocular camera, the number of electronic control units (ECUs) and communication lines required to control them will increase, which is detrimental to making in-vehicle systems smaller, lighter, and less costly.

[0013] Furthermore, in order to accommodate designs that take scalability (versatility) into consideration, it is desirable for the ECU that controls the in-vehicle camera to have a versatile hardware configuration rather than being specialized for stereo cameras.

[0014] None of Patent Documents 1 to 4 discloses a specific configuration for simultaneously achieving the above-mentioned small, lightweight, low-cost vehicle-mounted camera and scalability (versatility).

[0015] SUMMARY OF THE PRESENT EMBODIMENTS Accordingly, an object of the present invention is to provide an in-vehicle imaging system that is mounted on a vehicle and that can simultaneously achieve small size, light weight, low cost, and scalability (versatility). [Means for solving the problem]

[0016] In order to solve the above problems, the present invention provides an in-vehicle imaging system mounted on a vehicle, comprising a first camera, a second camera installed at a position different from that of the first camera, and an image processing unit that processes images acquired by the first camera and the second camera, the first camera and the second camera being connected by a first communication line, and the second camera and the image processing unit being connected by a second communication line. The second camera generates a parallax image using the image signal of the first camera and the image signal of the second camera, and transmits the parallax image to the image processing unit via the second communication line. It is characterized by the above. Effect of the Invention

[0017] According to the present invention, it is possible to realize an in-vehicle imaging system that is mounted on a vehicle and that is capable of simultaneously achieving small size, light weight, low cost, and scalability (versatility).

[0018] Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0019] [Figure 1] 1 is a diagram showing a vehicle and an on-board camera according to a first embodiment of the present invention. [Diagram 2] 1 is a diagram showing an in-vehicle imaging system according to a first embodiment of the present invention. [Figure 3A] FIG. 11 is a diagram showing a vehicle and an on-board camera according to a second embodiment of the present invention. [Figure 3B] 1 is a diagram conceptually showing an area in which distance information can be obtained by an on-board camera; [Figure 4] FIG. 11 is a diagram showing an in-vehicle imaging system according to a second embodiment of the present invention. [Diagram 5] FIG. 11 is a diagram showing a vehicle and an on-board camera according to a third embodiment of the present invention. [Figure 6] FIG. 11 is a diagram showing an in-vehicle imaging system according to a third embodiment of the present invention. [Figure 7] FIG. 11 is a diagram showing a vehicle and an on-board camera according to a fourth embodiment of the present invention. [Figure 8] FIG. 11 is a diagram showing a vehicle and an on-board camera according to a fifth embodiment of the present invention. [Figure 9] FIG. 13 is a diagram showing an in-vehicle imaging system according to a sixth embodiment of the present invention. [Figure 10] FIG. 13 is a diagram showing an in-vehicle imaging system according to a sixth embodiment of the present invention. [Figure 11A] FIG. 1 is a diagram showing a conventional vehicle-mounted imaging system. [Figure 11B] FIG. 1 is a diagram showing a conventional vehicle-mounted imaging system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same components are designated by the same reference numerals, and detailed description of overlapping parts will be omitted. EXAMPLES

[0021] First, the configuration and problems of a conventional vehicle-mounted imaging system will be described with reference to Fig. 11A and Fig. 11B. Fig. 11A and Fig. 11B are both diagrams showing conventional vehicle-mounted imaging systems. Fig. 11A shows a monocular camera system for general vehicles, and Fig. 11B shows a stereo camera system for luxury vehicles in which one camera is added to the monocular camera to form a stereo camera.

[0022] As shown in FIG. 11A, in a conventional vehicle-mounted imaging system 31 for general-purpose vehicles, an image signal captured by a camera head 2 is transmitted to an image recognition ECU 5, and image processing is performed by a general-purpose SoC 13 mounted in the image recognition ECU 5.

[0023] 11B, a conventional in-vehicle imaging system 32 for luxury vehicles is configured as a stereo camera by adding a camera head 3 and another image recognition ECU 5 to the above configuration. An image signal captured by the camera head 3 is transmitted to another image recognition ECU 5, and image processing is performed by a general-purpose SoC 13 mounted on the image recognition ECU 5, and distortion correction, brightness correction, calibration, stereo matching processing, etc. are performed by an image processing chip 33 mounted on the same image recognition ECU 5.

[0024] The conventional in-vehicle imaging system 32 for luxury vehicles is configured as described above and has many video signal lines, which is disadvantageous in terms of reducing the size and weight of the in-vehicle imaging system and its reliability, and also leads to increased costs.

[0025] In addition, since the image recognition ECU 5 has a stereo processing function, it cannot be used as an ECU for a monocular camera, and becomes an ECU designed specifically for a stereo camera.

[0026] Next, an in-vehicle imaging system according to a first embodiment of the present invention will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is a diagram showing a vehicle and an in-vehicle camera of the present embodiment. Fig. 2 is a diagram showing an in-vehicle imaging system mounted on the vehicle of Fig. 1.

[0027] As shown in FIG. 1, in this embodiment, two camera heads 2 and 3 are mounted in the vicinity of the rearview mirror of a vehicle 1.

[0028] As shown in FIG. 2, the main components of the in-vehicle imaging system 4 mounted on the vehicle 1 are a camera head 2 (first camera), a camera head 3 (second camera) installed in a position different from that of the camera head 2, and an image recognition ECU 5 (image processing unit) that processes images acquired by the camera head 2 and the camera head 3.

[0029] The camera head 2 and the camera head 3 are connected by a communication line 14 (first communication line), and the camera head 3 and the image recognition ECU 5 are connected by a communication line 15 (second communication line).

[0030] The camera head 2 (first camera) transmits an image signal of an image captured by the imaging element 6 to the camera head 3 (second camera) via the serializer 8 and a communication line 14 (first communication line).

[0031] Camera head 3 (second camera) has an image processing chip 11 (parallax image generating unit) that generates a parallax image, and image processing chip 11 generates a parallax image using an image signal of camera head 2 and an image signal of camera head 3.

[0032] The camera head 3 transmits the image signal of the camera head 2 input via the deserializer 9, the image signal of the image captured by the imaging element 7, and the parallax image generated by the image processing chip 11 to the image recognition ECU 5 (image processing unit) via the serializer 10 and a communication line 15 (second communication line).

[0033] The above-mentioned signals input to the image recognition ECU 5 (image processing unit) via the deserializer 12 are subjected to image processing by the general-purpose SoC 13.

[0034] The vehicle-mounted imaging system 4 of this embodiment is configured as described above, and can reduce the number of video signal lines and ECUs compared to the conventional vehicle-mounted imaging system shown in FIG. 11B.

[0035] In addition, since the image recognition ECU 5 does not need to have a hardware configuration specialized for stereo cameras, it can be used in a scalable system.

[0036] Incidentally, the communication line 14 also serves to supply power from the camera head 3 to the camera head 2, so there is no need to provide a separate power supply line for the camera head 2.

[0037] Moreover, the image processing chip 11, which is a semiconductor device, has an ISP (Image Signal Processor) function for processing image signals, as well as a distortion correction function, a brightness correction function (calibration function), and a stereo matching process (parallax image generation function). EXAMPLES

[0038] A vehicle-mounted imaging system according to a second embodiment of the present invention will be described with reference to Fig. 3A to Fig. 4. Fig. 3A is a diagram showing a vehicle and a vehicle-mounted camera of this embodiment. Fig. 3B is a diagram conceptually showing an area in which distance information can be obtained by the vehicle-mounted camera. Fig. 4 is a diagram showing the vehicle-mounted imaging system mounted on the vehicle of Fig. 3A.

[0039] In this embodiment, a more sophisticated vehicle-mounted imaging system will be described.

[0040] In particular, in systems for luxury vehicles, there is a demand for obtaining distance information over a wider range, for more accurate measurement of small debris on the road and unevenness in the road, and for improved distance accuracy over long distances.

[0041] Furthermore, in the vehicle-mounted imaging system described in the first embodiment, the camera head 3 is equipped with the image processing chip 11, and heat generated from the image processing chip 11 may affect reliability.

[0042] 3A, in this embodiment, two camera heads, such as upper camera head 16 and lower camera head 17, are arranged vertically on vehicle 1. In other words, lower camera head 17 is mounted at a position at a different height from upper camera head 16.

[0043] By arranging the two camera heads vertically as in FIG. 3A, the area in which distance information can be obtained can be expanded as shown in FIG. 3B, compared to the horizontal arrangement described in Example 1 (FIG. 1).

[0044] This makes it possible to obtain distance information over almost the entire horizontal angle of view, and makes it easier to detect objects with many horizontal edge components. Also, by arranging the two camera heads vertically, it is possible to increase the baseline length (the distance between the two camera heads) within the wiping range of the wiper, improving the accuracy of distance measurement at long distances.

[0045] 4, in this embodiment, the lower camera head 17 is disposed near an air conditioning mechanism (defroster 21) of the vehicle 1. The defroster 21 and the image recognition ECU 5 (general-purpose SoC 13) are connected by a communication line 20.

[0046] In addition, the lower camera head 17 is provided with a temperature sensor 19, and when the temperature sensor 19 detects a predetermined temperature, the air conditioning mechanism of the vehicle 1 is controlled so that the lower camera head 17 is cooled by a defroster 21.

[0047] In addition, for example, in cold regions, when the temperature drops extremely, it is possible to warm the lower camera head 17 with the defroster 21 to ensure the operation of the lower camera head 17.

[0048] Furthermore, a separate temperature sensor 18 different from the temperature sensor 19 may be provided in the upper camera head 16, and the air conditioning mechanism (defroster 21) of the vehicle 1 may be controlled so that the difference between the value of the temperature sensor 18 and the value of the temperature sensor 19 falls within a predetermined range.

[0049] By suppressing heat generation from lower camera head 17 and controlling the temperatures of the two camera heads to be approximately the same, differences in the optical characteristics of the cameras due to heat can be suppressed, and deterioration of distance measurement accuracy can be suppressed. EXAMPLES

[0050] Third Embodiment A vehicle-mounted imaging system according to a third embodiment of the present invention will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a diagram showing a vehicle and a vehicle-mounted camera according to the third embodiment. Fig. 6 is a diagram showing an in-vehicle imaging system mounted on the vehicle of Fig. 5.

[0051] As shown in FIG. 5, in this embodiment, similarly to the second embodiment (FIG. 3A), two camera heads are arranged vertically.

[0052] However, it differs from the second embodiment (FIG. 3A) in that a lower camera head 22 incorporating an image processing chip is provided instead of the lower camera head 17 of the second embodiment (FIG. 3A).

[0053] FIG. 6 shows a schematic configuration of the lower camera head 22 with a built-in image processing chip.

[0054] The lower camera head 22 with built-in image processing chip has an L-shaped structure in which the camera housing is divided into a sensor board section having a sensor board 25 and a main board section having a main board 23.

[0055] The image processing chip 11 is mounted on the main board 23, and heat generated from the image processing chip 11 is dissipated to the housing via a heat spreader (heat sink) 24. The main board section is cooled by a defroster 21.

[0056] As in this embodiment, by placing the lower camera head 22 with built-in image processing chip on or within the dashboard 27 near the windshield 26, it becomes possible to cool the camera head with built-in image processing chip 11 using the air conditioning mechanism (defroster 21).

[0057] In addition, by installing the main board (board on which image processing chip 11 is mounted) so that it is hidden under the dashboard, it is possible to reduce the protruding portion from dashboard 27. If image processing chip 11 is built into the camera head, the housing of the camera head becomes large, but by using the configuration shown in Fig. 6, it is possible to improve the mountability of the camera head and ensure the driver's field of vision.

[0058] Furthermore, by disposing a non-reflective material 28 on the dashboard 27 near the windshield 26, reflection of the dashboard 27 on the windshield 26 is prevented, allowing a clearer image to be captured. EXAMPLES

[0059] Fourth embodiment An on-vehicle imaging system according to the present invention will be described with reference to Fig. 7. Fig. 7 is a diagram showing a vehicle and an on-vehicle camera according to this embodiment.

[0060] In this embodiment, an example will be described in which a truck is used as the vehicle 29. In this embodiment as well, two camera heads, an upper camera head 16 and a lower camera head 22 incorporating an image processing chip, are arranged vertically.

[0061] Large vehicles such as trucks have ample space underneath (on the dashboard), so as in the third embodiment (FIG. 6), the device is embedded inside the dashboard so as not to obstruct visibility.

[0062] The dashboard also includes an air conditioning system for the vehicle. By disposing a camera head incorporating an image processing chip 11 on the dashboard, cooling by the air conditioning mechanism (defroster 21) becomes possible. EXAMPLES

[0063] Fifth embodiment An on-vehicle imaging system according to a fifth embodiment of the present invention will be described with reference to Fig. 8. Fig. 8 is a diagram showing a vehicle and an on-vehicle camera according to the present embodiment.

[0064] In this embodiment, a motorcycle will be described as an example of the vehicle 30. In this embodiment as well, two camera heads, an upper camera head 16 and a lower camera head 22 incorporating an image processing chip, are arranged vertically.

[0065] The upper camera head 16 is disposed on the upper front side of the cowl, and the lower camera head 22 with an integrated image processing chip is disposed on the lower front side of the cowl.

[0066] By disposing the lower camera head 22 with built-in image processing chip on the front lower side of the cowl, it is possible to avoid direct sunlight and to easily obtain the air-cooling effect by the air flow during riding. EXAMPLES

[0067] A vehicle-mounted imaging system according to a sixth embodiment of the present invention will be described with reference to Fig. 9 and Fig. 10. Fig. 9 and Fig. 10 are both diagrams showing the vehicle-mounted imaging system of this embodiment. Fig. 9 shows a state when camera head 2 (first camera) is malfunctioning, and Fig. 10 shows a state when camera head 3 (second camera) is malfunctioning.

[0068] As shown in Figures 9 and 10, if either camera head 2 (first camera) or camera head 3 (second camera) fails, camera head 3 (second camera) stops generating disparity images in image processing chip 11 (disparity image generation unit) and transmits the image signal of the healthy camera to image recognition ECU 5 (image processing unit).

[0069] With this configuration, if either camera head 2 or camera head 3 fails, it can be used as a conventional monocular camera system as shown in FIG. 11A, thereby providing redundancy to the vehicle-mounted camera system.

[0070] In addition, when the common area between the imaging areas of camera head 2 (first camera) and camera head 3 (second camera) is equal to or larger than a predetermined area, the imaging areas as monocular cameras captured by camera head 2 or camera head 3 can be regarded as being approximately equivalent, so the image signal of either camera head 2 or camera head 3 may be transmitted to image recognition ECU 5 (image processing unit).

[0071] On the other hand, if the common area of ​​the imaging areas of camera head 2 (first camera) and camera head 3 (second camera) is less than a specified area, the imaging areas captured by camera head 2 and camera head 3 as monocular cameras can be considered to be different, and therefore the image signals of both camera head 2 and camera head 3 are transmitted to image recognition ECU 5 (image processing unit).

[0072] With this configuration, when the common area between the imaging areas of camera head 2 (first camera) and camera head 3 (second camera) is greater than a specified area, the processing load on image recognition ECU 5 (image processing unit) can be reduced.

[0073] The present invention is not limited to the above-described embodiments, and includes various modified examples. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the configurations described. It is also possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. [Explanation of symbols]

[0074] 1,29,30...vehicle, 2,3...camera head, 4,31,32...vehicle-mounted imaging system, 5...image recognition ECU, 6,7...imaging element, 8,10...serializer, 9,12...deserializer, 11,33...image processing chip, 13...general-purpose SoC, 14,15,20...communication line, 16...upper camera head, 17...lower camera head, 18,19...temperature sensor, 21...defroster, 22...lower camera head with built-in image processing chip, 23...main board, 24...heat spreader (heat sink), 25...sensor board, 26...windshield, 27...dashboard, 28...non-reflective material

Claims

1. An in-vehicle imaging system mounted on a vehicle, A first camera; a second camera installed at a position different from that of the first camera; an image processing unit that processes images acquired by the first camera and the second camera, the first camera and the second camera are connected by a first communication line; the second camera and the image processing unit are connected by a second communication line, The second camera generates a parallax image using the image signal of the first camera and the image signal of the second camera, and transmits the parallax image to the image processing unit via the second communication line.

2. The vehicle-mounted imaging system according to claim 1 , the second camera has a parallax image generator that generates a parallax image; the first camera transmits an image signal of the first camera to the second camera via the first communication line; the parallax image generating unit generates the parallax image by using an image signal of the first camera and an image signal of the second camera; The second camera transmits the image signal of the first camera, the image signal of the second camera, and the disparity image generated by the disparity image generation unit to the image processing unit via the second communication line.

3. The vehicle-mounted imaging system according to claim 1 , The second camera is mounted at a different height from the first camera in the vehicle-mounted imaging system.

4. The vehicle-mounted imaging system according to claim 1 , An in-vehicle imaging system in which power is supplied from the second camera to the first camera.

5. The vehicle-mounted imaging system according to claim 1 , the second camera has a temperature sensor; When the temperature sensor detects a predetermined temperature, the second camera is cooled or heated by an air conditioning mechanism of the vehicle.

6. The vehicle-mounted imaging system according to claim 5, the first camera has a temperature sensor different from the temperature sensor; An in-vehicle imaging system that controls an air conditioning mechanism of the vehicle so that a difference between a value of the temperature sensor of the first camera and a value of the temperature sensor of the second camera falls within a predetermined range.

7. The vehicle-mounted imaging system according to claim 1 , the second camera has a semiconductor device having a parallax image generating function; performing stereo matching processing between the image of the first camera and the image of the second camera; An in-vehicle imaging system that outputs a distance image including parallax information generated by the semiconductor to the image processing unit.

8. The vehicle-mounted imaging system according to claim 1 , the vehicle is a car or a truck; The second camera is an in-vehicle imaging system located on or within the dashboard of the vehicle.

9. The vehicle-mounted imaging system according to claim 1 , the vehicle is a motorcycle, The first camera is disposed on a front upper side of the cowl, The second camera is an in-vehicle imaging system disposed on the front lower side of the cowl.

10. The vehicle-mounted imaging system according to claim 2, When either the first camera or the second camera fails, The second camera stops generation of parallax images by the parallax image generating unit and transmits an image signal of the other camera that is not malfunctioning to the image processing unit.

11. The vehicle-mounted imaging system according to claim 2, When a common area between the imaging areas of the first camera and the second camera is equal to or larger than a predetermined area, the second camera transmits an image signal of either the first camera or the second camera to the image processing unit; An in-vehicle imaging system that transmits image signals from both the first camera and the second camera to the image processor when a common area between the imaging areas of the first camera and the second camera is less than a predetermined area.

Images