Image processing method for in-vehicle camera, and in-vehicle camera

Abstract

To provide an image processing method for an in-vehicle camera capable of displaying images captured by an imaging device installed in a cabin more clearly, and an in-vehicle camera.SOLUTION: An image processing method for an in-vehicle camera arranged in a cabin of a vehicle, includes: picking up images outside the cabin of the vehicle through glass arranged in the vehicle; acquiring images; acquiring multiple color temperatures from the images; detecting a color temperature range that the in-vehicle camera regards it as white from the multiple color temperatures; adjusting the color temperature range by referring to a table for holding the color temperature range according to spectral characteristics of the glass so that the color temperature range matches the color temperature captured by the in-vehicle camera arranged outside the cabin of the vehicle; adjusting the white balance of the image based on the color temperature range; and outputting the white-balanced image to the outside.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to an image processing method for an in-vehicle camera and an in-vehicle camera. [Background technology]

[0002] In recent years, with the spread of driving assistance systems for vehicles, imaging devices are increasingly being installed in vehicles. In addition, imaging devices and electronic mirrors are being installed in the vehicle interior, and images of the area behind the vehicle captured by the imaging devices are being displayed on the electronic mirrors.

[0003] Furthermore, in vehicles, in order to make it difficult to see inside the vehicle from outside, the rear window mounted on the vehicle is sometimes processed to reduce transmittance or tinted. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6657925 [Patent Document 2] Patent No. 6537385 Summary of the Invention [Problem to be solved by the invention]

[0005] When an imaging device is installed inside a vehicle, it captures an image of the area behind the vehicle through the rear window. Therefore, if the rear window is processed as described above, the color and quality of the image captured by the imaging device may be affected by the rear window, and the image may not be displayed clearly on the electronic mirror.

[0006] The present disclosure provides an image processing method for an in-vehicle camera and an in-vehicle camera that can more clearly display images captured by an imaging device installed in a vehicle cabin. [Means for solving the problem]

[0007] The image processing method for an onboard camera according to the present disclosure is an image processing method for an onboard camera placed inside the cabin of a vehicle, which involves capturing an image of the outside of the vehicle cabin through glass placed in the vehicle, acquiring the image, obtaining multiple color temperatures from the image, detecting a color temperature range that the onboard camera considers to be white from the multiple color temperatures, adjusting the color temperature range by referring to a table that holds color temperature ranges according to the spectral characteristics of the glass so that the color temperature range becomes the color temperature used for capturing images by the onboard camera placed outside the vehicle cabin, adjusting the white balance of the image based on the color temperature range, and outputting the white-balanced image to the outside. [Effects of the Invention]

[0008] According to the image processing method for an in-vehicle camera and the in-vehicle camera of the present disclosure, images captured by an imaging device installed inside the vehicle cabin can be displayed more clearly. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of a vehicle equipped with an electronic mirror system according to the first embodiment. [Figure 2] FIG. 2 is a diagram showing an example of the configuration of the digital mirroring system according to the first embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of the configuration of the imaging device according to the first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of the configuration of the digital mirror according to the first embodiment. [Figure 5] FIG. 5 is a block diagram showing an example of the functional configuration of the imaging device according to the first embodiment. [Figure 6] FIG. 6 is a diagram showing an example of the characteristics of a rear window. [Figure 7] FIG. 7 is a diagram showing an example of a white detection range detected by the imaging device according to the first embodiment. [Figure 8] FIG. 8 is a table showing an example of a color temperature range corrected by the digital mirror system according to the first embodiment. [Figure 9]FIG. 9 is a block diagram showing an example of the functional configuration of the imaging device according to the second embodiment. [Figure 10] FIG. 10 is a flowchart showing an example of the operation of the digital mirroring system according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] (First embodiment) Hereinafter, an embodiment of a digital mirror system according to a first embodiment will be described with reference to the drawings.

[0011] The electronic mirror system according to the first embodiment is composed of an imaging device and an electronic mirror mounted on a vehicle. The electronic mirror displays an image of the rear of the vehicle captured by the imaging device, for example, instead of a mirror for viewing the rear of the vehicle.

[0012] First, the overall configuration of the electronic mirror system 3 will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of a vehicle on which the electronic mirror system 3 according to the embodiment is mounted.

[0013] The electronic mirror system 3 is mounted on a vehicle 5 and includes a camera 1 and an electronic mirror 2. The electronic mirror system 3 is an example of an image processing device according to the present disclosure.

[0014] Camera 1 is provided inside the interior of vehicle 5. For example, camera 1 is provided at the rear of the interior of vehicle 5 and in front of rear window 4, and is installed facing the rear of vehicle 5 (negative side of the X axis). Camera 1 is equipped with an imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Complementary Metal Oxide Semiconductor), and captures an image of the area behind vehicle 5. Camera 1 is an example of an imaging device in the present disclosure.

[0015] The electronic mirror 2 has the function of a general mirror-type rearview mirror and the function of displaying an image of the area behind the vehicle 5 captured by the camera 1. The structure of the electronic mirror 2 will be described in detail later. The electronic mirror 2 is an example of a display device in the present disclosure.

[0016] The electronic mirror 2 is disposed inside the vehicle cabin. For example, the electronic mirror 2 has a display panel 23 on the surface facing the vehicle cabin (the negative direction of the X axis shown in FIG. 1 ), and displays an image captured by the camera 1 on the display panel 23. If the electronic mirror 2 is a rearview electronic mirror, it may be implemented in the form of a rearview mirror, and the shape of the display panel 23 may be the same as the mirror surface of the rearview mirror.

[0017] Next, the configuration of the electronic mirror system 3 will be described with reference to Fig. 2. In the electronic mirror system 3, the camera 1 and the electronic mirror 2 are connected to each other so that they can communicate with each other via a communication medium 8, as shown in Fig. 2. Fig. 2 is a diagram showing the configuration of the electronic mirror system 3 according to this embodiment. The communication medium 8 may be a communication cable such as a serial cable, or may be a wireless communication line such as Bluetooth (registered trademark). Fig. 2 illustrates a configuration in which the communication medium 8 is a communication cable, and one end of the communication medium 8 is connected to the I / F unit 13 of the camera 1 and the other end is connected to the I / F unit 21 of the electronic mirror 2.

[0018] The camera 1 captures an image behind the vehicle body 6 and supplies the acquired image to the electronic mirror 2 via the communication medium 8. As a result, the electronic mirror 2 receives the image captured by the camera 1 and displays the received image on the display panel 23. The electronic mirror 2 may display the entire image captured by the camera 1, or may crop a portion of the image (hereinafter also referred to as a partial image) and display it on the display panel 23.

[0019] 2 illustrates a configuration in which a partial image cut out in the shape of the mirror surface of the rearview mirror from an image captured by the camera 1 is displayed on the display panel 23 of the electronic mirror 2. The electronic mirror 2 may have a housing 28c fixed to the inside of the roof 6b of the vehicle body 6 in a state in which the position of the housing 28c is variable. The electronic mirror 2 may have a housing 28b fixed to the roof 6b of the vehicle body 6 via a support 28a.

[0020] The display panel 23 is, for example, a liquid crystal panel or an organic EL panel, etc. A half mirror 24 is provided so as to face the display surface of the display panel 23. The half mirror 24 is a typical rearview mirror.

[0021] The operation switch 25 is provided at the bottom of the housing 28c. Operating the operation switch 25 switches between the display on the display panel 23 and the reflected image of the rear of the vehicle 5 reflected in the half mirror 24. When the reflected image reflected in the half mirror 24 is displayed, the display panel 23 is in a non-display state.

[0022] Supporting portion 28a is a member that supports housing 28c on the inside of roof portion 6b of the vehicle. Housing 28b is a member that houses display panel 23. Housing 28b is also supported on the inside of roof portion 6b of the vehicle via supporting portion 28a.

[0023] In response to a change in the attitude of the housing 28c, the electronic mirror 2 may finely adjust the position of a partial image to be cut out from the entire image and display it on the display panel 23. This allows the electronic mirror 2 to display an image of the surroundings of the vehicle captured by the camera 1 in accordance with the attitude of the housing 28c, and can operate as a substitute for a mirror.

[0024] Next, the configuration of the camera 1 will be described with reference to Fig. 3. As shown in Fig. 3, the camera 1 includes an image sensor 11, a signal processor (ISP: Image Signal Processor) 12, and an interface (I / F) unit 13. Fig. 3 is a diagram showing the configuration of the camera 1 according to this embodiment. The image sensor 11 includes terminals 11a and 11b. The signal processor 12 has terminals 12a, 12b, 12c, and 12d. The interface unit 13 includes terminals 13a, 13b, and 13c.

[0025] The image sensor 11 acquires, for example, an image of the rear of the vehicle 5, generates a plurality of pixel signals according to the acquired image, and supplies the plurality of pixel signals to the signal processor 12. The image sensor 11 includes a pixel unit 111, an interface (I / F) 112, and a drive control unit 113.

[0026] The pixel unit 111 has a plurality of pixels arranged in a plurality of rows and a plurality of columns, and a plurality of AD converters (a plurality of ADCs) arranged corresponding to the plurality of columns. The pixel unit 111 performs charge accumulation operation in the plurality of pixels according to the subject image (image) formed on its imaging surface, and performs AD conversion of the accumulated signals of the plurality of pixels for each column using the AD converter to generate a plurality of pixel signals.

[0027] The interface 112 exchanges communication control information with the signal processor 12 via the terminals 11b and 12b, and establishes communication with the signal processor 12. The interface 112 may establish communication with the signal processor 12 in accordance with the SPI (Serial Peripheral Interface) method. The interface 112 transmits a plurality of pixel signals output from the pixel unit 111 as one frame of image to the signal processor 12 via the terminals 11a and 12a. The interface 112 can transmit a plurality of time-sequential frames of image as video signals to the signal processor 12 via the terminals 11a and 12a.

[0028] The drive control unit 113 has a vertical scanning circuit and a horizontal scanning circuit. The vertical scanning circuit scans multiple pixels in the vertical direction in response to a vertical drive signal (VD) and controls the charge accumulation operation of the multiple pixels on a row-by-row basis. The horizontal scanning circuit scans multiple pixels in the horizontal direction in response to a horizontal drive signal (HD) and reads out pixel signals from the multiple pixels and performs AD conversion on a column-by-column basis.

[0029] The signal processor 12 receives a plurality of pixel signals corresponding to an image from the image sensor 11 and performs predetermined signal processing on the plurality of pixel signals. The signal processor 12 includes a signal processing unit 121 and an interface (I / F) 122.

[0030] The signal processing unit 121 has an AD converter and an address processing circuit, and performs predetermined signal processing on a plurality of pixel signals using the AD converter and the address processing circuit. The predetermined signal processing includes processing to adjust the dynamic range of the signal and noise reduction correction processing to reduce noise contained in the signal. If the pixel unit 111 supports color, for example, by including a color filter in each pixel, the predetermined signal processing may further include YC signal processing to generate a YC signal containing a luminance component (Y) and a chromaticity component (C) from the pixel signal.

[0031] The signal processing unit 121 includes a processor such as a CPU (Central Processing Unit) and a memory such as a flash memory. The signal processing unit 121 realizes its functions as the signal processing unit 121 by causing the processor to execute a program loaded into the memory. The functions of the signal processing unit 121 will be described later.

[0032] The interface 122 exchanges communication control information with the interface unit 13 via the terminals 12d and 13b, and establishes communication with the interface unit 13. The interface 122 may establish communication with the interface unit 13 in accordance with the I2C (Inter-Integrated Circuit) method.

[0033] The interface 122 transmits a plurality of pixel signals output from the signal processing unit 121 as one frame of image to the interface unit 13 via the terminals 12c and 13a. The interface 122 can transmit a plurality of time-sequential frame images as video signals to the interface unit 13 via the terminals 12c and 13a.

[0034] The interface unit 13 notifies the electronic mirror 2 of any failure detected by the image sensor 11 and / or the signal processor 12. The interface unit 13 is connected to the electronic mirror 2 via a communication medium 8.

[0035] The interface unit 13 converts the format of signals used in communication. If the communication medium 8 supports serial communication, the interface unit 13 may include a serializer 131.

[0036] The serializer 131 converts the video signal received from the image sensor 11 and / or the signal processor 12 via the terminal 13a from parallel format to serial format, and transmits the serial format video signal to the electronic mirror 2 via the terminal 13c and the communication medium 8. The serializer 131 converts the communication control information received from the image sensor 11 and / or the signal processor 12 via the terminal 13b from parallel format to serial format, and transmits the serial format communication control information to the electronic mirror 2 via the terminal 13c and the communication medium 8.

[0037] Next, the configuration of the digital mirror 2 will be described with reference to Fig. 4. As shown in Fig. 4, the digital mirror 2 includes an interface (I / F) unit 21, an image processing unit 22, and a display panel 23. Fig. 4 is a diagram showing the configuration of the digital mirror 2 according to this embodiment.

[0038] The interface unit 21 is connected to the camera 1 via a communication medium 8. The interface unit 21 includes a connector unit 211, and the camera 1 is connected to the connector unit 211 via the communication medium 8. The interface unit 21 converts the format of signals used in communication. If the communication medium 8 supports serial communication, the interface unit 213 may include a deserializer 212.

[0039] The deserializer 212 converts the video signal transmitted from the camera 1 via the communication medium 8 and the connector unit 211 from serial to parallel format, and transmits the parallel format video signal to the video processing unit 22. The video signal may be transmitted from the connector unit 211 to the deserializer 212 by communication conforming to the FPD-Link (Flat Panel Display-Link) standard. The video signal may be transmitted from the deserializer 212 to the video processing unit 22 by communication conforming to the MIPI (Mobile Industry Processor Interface) standard.

[0040] The video processing unit 22 converts the video signal transmitted from the deserializer 212 into a video signal for display and supplies it to the display panel 23. The display panel 23 displays the video signal converted by the video processing unit 22.

[0041] Although not shown in Figure 4, the electronic mirror 2 is equipped with a half mirror 24 superimposed on the display panel 23, as described in Figure 2, and by operating the operation switch 25, the electronic mirror 2 switches between displaying the video signal displayed on the display panel 23 and the reflected image of the rear of the vehicle 5 reflected on the half mirror 24.

[0042] Next, functions of the signal processing unit 121 will be described with reference to Fig. 5. Fig. 5 is a block diagram showing an example of functions of the signal processing unit 121. The signal processing unit 121 includes an image acquisition unit 1211, a color temperature acquisition unit 1212, a detection unit 1213, a first adjustment unit 1214, and an output unit 1215. Note that the functions of the signal processing unit 121 are not limited to these.

[0043] The image acquisition unit 1211 acquires an image captured by the camera 1. Specifically, the image acquisition unit 1 acquires an image of the outside of the vehicle cabin captured by the camera 1 mounted in the vehicle cabin through the rear window 4 provided on the vehicle 5.

[0044] The color temperature acquisition unit 1212 acquires the color temperature from the image captured by the camera 1. Specifically, the color temperature acquisition unit 1212 acquires the color temperature from the image acquired by the image acquisition unit 1211. The color temperature will be described later.

[0045] The detection unit 1213 detects a color temperature range from the color temperature acquired by the color temperature acquisition unit 1212. Specifically, the detection unit 1213 detects a color temperature range that the camera 1 regards as white from the color temperature acquired by the color temperature acquisition unit 1212.

[0046] The first adjustment unit 1214 is an example of an adjustment unit. The first adjustment unit 1214 adjusts the white balance based on the color temperature range detected by the detection unit 1213. Specifically, the first adjustment unit 1214 adjusts the white balance for the color temperature range detected by the detection unit 1213 so that an image captured through the rear window 4 by the camera 1 mounted inside the vehicle has the same color temperature as an image captured with the camera 1 mounted outside the vehicle, that is, an image in a state where the rear window 4 is not present. The result of the white balance adjustment by the first adjustment unit 1214 is stored in a memory.

[0047] The output unit 1215 outputs the image whose white balance has been adjusted by the first adjustment unit 1214. Specifically, the output unit 1215 outputs the image whose white balance has been adjusted by the first adjustment unit 1214 to the digital mirror 2.

[0048] Here, the transmittance characteristics of the rear window 4 will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of the characteristics of the rear window 4. In the graph shown in Fig. 6, the horizontal axis represents wavelength [nm] and the vertical axis represents transmittance [%]. Note that the wavelength state where the transmittance is 100% corresponds to a state where the rear window 4 is not present.

[0049] The characteristics of the rear windshield 4 can be expressed, for example, as the relationship between wavelength and transmittance. For example, a rear windshield 4 having the characteristics of graph G1 exhibits a transmittance of approximately 60% in the wavelength range of 450 nm to 650 nm included in visible light. A rear windshield 4 having the characteristics of graph G2 exhibits a transmittance of 25% in the wavelength range of 450 nm to 650 nm. A rear windshield 4 having the characteristics of graph G3 exhibits a transmittance of approximately 15% in the wavelength range of 450 nm to 650 nm. A rear windshield 4 having the characteristics of graph G4 exhibits a transmittance of approximately 10% in the wavelength range of 450 nm to 650 nm.

[0050] When camera 1 captures an image of the rear of the vehicle through rear window 4, the image is affected by the transmittance of rear window 4, and the color temperature of the captured image differs from the color temperature of an image captured without rear window 4. The greater the difference in transmittance for each wavelength in the image captured by camera 1, the greater the shift in the color temperature range considered to be white, so it is necessary to adjust the white balance according to the characteristics of rear window 4.

[0051] Next, the details of white balance adjustment by the first adjustment unit 1214 will be described with reference to Fig. 7 and Fig. 8. Fig. 7 is a diagram showing an example of a color temperature range detected by the imaging device by the detection unit 1213. Fig. 8 is a table showing an example of adjustment details for adjusting the white balance by the first adjustment unit 1214.

[0052] 7, the horizontal axis represents R / G and the vertical axis represents the spectral sensitivity ratio, which indicates B / G. Color temperatures 71 and 72 are color temperatures when an image is captured by a camera 1 provided outside the vehicle cabin (without going through the rear window 4). Color temperatures 73 and 74 are color temperatures acquired by the color temperature acquisition unit 1212, and are color temperatures when an image is captured by a camera 1 provided inside the vehicle cabin through the rear window 4. The detection unit 1213 detects a color temperature range 75 that the camera 1 regards as white from the color temperatures 73 and 74 acquired by the color temperature acquisition unit 1212.

[0053] Color temperature range 75 is the color temperature range in which white is detected (considered to be white) by the auto white balance function by first adjustment unit 1214. Color temperatures 76 and 77 are the color temperatures when an image is captured through the rear window 4 by camera 1 installed inside the vehicle cabin as a result of adjusting the color temperature by first adjustment unit 1214.

[0054] The color temperatures of color temperature 71 and color temperature 76 are 8800K [Kelvin]. The color temperatures of color temperature 72 and color temperature 77 are 4000K [Kelvin]. The color temperature of color temperature 73 is 15000K [Kelvin]. The color temperature of color temperature 74 is 5250K [Kelvin].

[0055] For example, if the white detection color temperature range for the exterior of the vehicle is set to 4000K to 8000K, when the camera 1 captures an image of the rear of the vehicle through the rear window 4 inside the vehicle, the color temperature range detected by the detection unit 1213 will be equivalent to 5250K to 15000K. In this case, the camera 1 may capture the sky or a sunshade with a high color temperature. When the image captured by the camera 1 is displayed on the electronic mirror 2, the image will be displayed dark.

[0056] Therefore, the first adjustment unit 1214 adjusts the signal levels of the RGB signals included in the video signal output from the image sensor 11 in accordance with the color temperature of the rear window 4. For example, if the color temperature detected by the detection unit 1213 is between 5250K and 15000K, the first adjustment unit 1214 refers to the table shown in Fig. 8 and adjusts the color temperature range to between 4000K and 8800K.

[0057] As a result, the camera 1 installed inside the vehicle can capture an image equivalent to an image captured outside the vehicle when capturing an image within the color temperature range adjusted by the first adjustment unit 1214. Furthermore, the electronic mirror 2 can display an image with adjusted white balance, resulting in a clearer display.

[0058] In the first embodiment, the first adjustment unit 1214 obtains the color temperature from the image captured by the camera 1, then detects the white detection color temperature range, and adjusts the white balance for the color temperature range. However, for example, the user may manually input the color temperature that is considered to be white, and adjust the white balance for the input range.

[0059] As described above, the image processing device of the first embodiment detects the color temperature range regarded as white from the acquired color temperature, adjusts the white balance for the color temperature range, and outputs the result. As a result, the image processing device of the first embodiment can more clearly display images captured by an imaging device installed in the vehicle cabin.

[0060] (Second embodiment) Next, a second embodiment will be described. Explanations of parts common to the first embodiment will be omitted where appropriate. Note that components similar to those in the first embodiment will be given the same reference numerals and explanations will be omitted where appropriate.

[0061] In the first embodiment described above, the camera 1 adjusts the white balance based on the detected color temperature range. In the second embodiment, the camera 1 adjusts the gain based on the input illuminance. The gain is a parameter that adjusts the brightness when capturing an image with the camera 1, and specifically corresponds to the adjustment value of the pixel signal generated by the image sensor 11.

[0062] In order to ensure brightness equivalent to that of an imaging device mounted outside the vehicle cabin, the camera 1 of the second embodiment first sets the exposure time to 16.6 msec (=60 fps), the same as that of the imaging device mounted outside the vehicle cabin. Also, the exposure amount of the camera 1 is set to -12 dB. By setting the exposure amount of the camera 1, it is possible to ensure brightness approximately equivalent to that of an imaging device mounted outside the vehicle cabin.

[0063] 9 is a block diagram showing an example of the functional configuration of the signal processing unit 121 according to the second embodiment. The signal processing unit 121 includes a color temperature acquisition unit 1212, a detection unit 1213, a first adjustment unit 1214, an output unit 1215, an acquired illuminance determination unit 1216, an estimation unit 1217, an estimated illuminance determination unit 1218, a second adjustment unit 1219, and a third adjustment unit 1220. Note that the functions of the signal processing unit 121 are not limited to these.

[0064] The obtained illuminance determination unit 1216 determines whether the input illuminance obtained from the image sensor 11 reaches a target value. The obtained illuminance determination unit 1216 is an example of a determination unit. Specifically, the obtained illuminance determination unit 1216 determines whether the input illuminance obtained from the image sensor 11 reaches a target value indicating an input illuminance equivalent to that outside the vehicle cabin.

[0065] The estimation unit 1217 estimates the estimated illuminance. Specifically, when the acquired illuminance determination unit 1216 determines that the acquired input illuminance does not reach the target value, the estimation unit 1217 estimates the acquired input illuminance as the estimated illuminance.

[0066] The estimated illuminance determination unit 1218 is an example of a determination unit. The estimated illuminance determination unit 1218 determines whether or not the estimated illuminance exceeds a first threshold. Specifically, the estimated illuminance determination unit 1218 determines whether or not the estimated illuminance estimated by the estimation unit 1217 exceeds the first threshold. The estimated illuminance determination unit 1218 also determines whether or not the estimated illuminance exceeds a second threshold. Specifically, the estimated illuminance determination unit 1218 determines whether or not the estimated illuminance estimated by the estimation unit 1217 exceeds the second threshold.

[0067] Furthermore, the estimated illuminance determination unit 1218 determines whether or not the estimated illuminance exceeds a third threshold. Specifically, the estimated illuminance determination unit 1218 determines whether or not the estimated illuminance estimated by the estimation unit 1217 exceeds a third threshold.

[0068] Here, the first threshold, second threshold, and third threshold will be described. The first threshold is, for example, in the range of 100 lux to 200 lux. The second threshold is, for example, in the range of 30 lux to 60 lux. The third threshold is, for example, in the range of 10 lux to 20 lux.

[0069] If the estimated illuminance determination unit 1218 determines that the estimated illuminance is below the first threshold, the second adjustment unit 1219 adjusts the gain to increase. Specifically, if the estimated illuminance determination unit 1218 determines that the estimated illuminance estimated by the estimation unit 1217 is below the first threshold, the second adjustment unit 1219 adjusts the gain of the pixel unit 111 to increase it.

[0070] If the illuminance estimated by the estimation unit 1217 is below the first threshold, the illuminance of the subject being imaged by the camera 1 is low, and the camera 1 cannot capture a clear image. Therefore, for example, the second adjustment unit 1219 adjusts the gain of the pixel unit 111 to increase it by +9 dB from the exposure set for the exterior of the vehicle. As a result, the exposure of the camera 1 becomes -3 dB, and the brightness becomes approximately the same as that of an imaging device installed outside the vehicle. This allows the camera 1 to capture a clear image.

[0071] Furthermore, when the estimated illuminance determination unit 1218 determines that the estimated illuminance exceeds the second threshold, the second adjustment unit 1219 adjusts the gain to decrease. Specifically, when the estimated illuminance determination unit 1218 determines that the estimated illuminance estimated by the estimation unit 1217 exceeds the second threshold, the second adjustment unit 1219 adjusts the gain of the pixel unit 111 to decrease.

[0072] If the illuminance estimated by the estimation unit 1217 exceeds the second threshold, the illuminance of the subject being imaged by the camera 1 is high, and the camera 1 is unable to capture a clear image. Therefore, for example, the second adjustment unit 1219 adjusts the gain of the pixel unit 111 to decrease (i.e., decrease the gain that was once increased by -9 dB). As a result, the exposure of the camera 1 becomes -12 dB, and the brightness becomes approximately the same as that of an imaging device mounted outside the vehicle cabin. The second adjustment unit 1219 may also adjust the pixel unit 111 to increase the brightness by gradation correction. As a result, the brightness can be maintained approximately the same as that set for the outside of the vehicle cabin. This allows the camera 1 to capture a clear image.

[0073] The third adjustment unit 1220 adjusts the exposure time to be longer when the estimated illuminance determination unit 1218 determines that the estimated illuminance is below the second threshold. Specifically, the third adjustment unit 1220 adjusts the exposure time of the pixel unit 111 to be longer when the estimated illuminance determination unit 1218 determines that the estimated illuminance estimated by the estimation unit 1217 is below the second threshold.

[0074] If the illuminance estimated by the estimation unit 1217 is below the second threshold, the illuminance of the subject being imaged by the camera 1 is low, and the camera 1 cannot capture a clear image. Therefore, for example, the third adjustment unit 1220 adjusts the exposure time from 16.6 msec to 33.3 msec to increase the exposure time for the pixel unit 111. As a result, the brightness can be maintained at approximately the same level as the setting for the exterior of the vehicle. This allows the camera 1 to capture a clear image.

[0075] Furthermore, when the estimated illuminance determination unit 1218 determines that the estimated illuminance exceeds the third threshold, the third adjustment unit 1220 adjusts the exposure to be shorter. Specifically, when the estimated illuminance determination unit 1218 determines that the estimated illuminance estimated by the estimation unit 1217 exceeds the third threshold, the third adjustment unit 1220 adjusts the exposure of the pixel unit 111 to be shorter.

[0076] If the illuminance estimated by the estimation unit 1217 exceeds the third threshold, the illuminance of the subject being imaged by the camera 1 is high, and the camera 1 is unable to capture a clear image. Therefore, for example, the third adjustment unit 1220 adjusts the exposure time from 33.3 msec to 16.6 msec to shorten the exposure for the pixel unit 111. The third adjustment unit 1220 may also adjust the pixel unit 111 to increase the brightness by performing gradation correction. As a result, the brightness can be maintained at approximately the same level as the setting for the exterior of the vehicle. This allows the camera 1 to capture a clear image.

[0077] Next, the flow of processing executed in the digital mirroring system 3 configured as above will be described.

[0078] FIG. 10 is a flowchart showing an example of the flow of processing executed in the digital mirroring system 3 according to the second embodiment.

[0079] First, in Figure 10, the exposure time of the camera 1 of the second embodiment is set to 16.6 msec (= 60 fps), the same as that of the imaging device mounted outside the vehicle cabin, in order to ensure brightness equivalent to that of the imaging device mounted outside the vehicle cabin.

[0080] The acquired illuminance determination unit 1216 determines whether the input illuminance acquired from the image sensor 11 reaches the target value (step S201). If the acquired illuminance determination unit 1216 determines that the input illuminance acquired from the image sensor 11 reaches the target value (step S201: Yes), this processing ends. If the acquired illuminance determination unit 1216 determines that the input illuminance acquired from the image sensor 11 does not reach the target value (step S201: No), the processing proceeds to step S202.

[0081] The estimation unit 1217 estimates the input illuminance acquired by the acquired illuminance determination unit 1216 as an estimated illuminance (step S202).

[0082] Next, the estimated illuminance determination unit 1218 determines whether the estimated illuminance exceeds the first threshold value (step S203). If the estimated illuminance determination unit 1218 determines that the estimated illuminance exceeds the first threshold value (step S203: No), this process ends. If the estimated illuminance determination unit 1218 determines that the estimated illuminance is below the first threshold value (step S203: Yes), the process proceeds to step S204.

[0083] The second adjustment unit 1219 adjusts the pixel unit 111 to increase the gain (step S204).

[0084] Next, the acquired illuminance determination unit 1216 determines whether the input illuminance acquired from the image sensor 11 reaches the target value (step S205). If the acquired illuminance determination unit 1216 determines that the input illuminance acquired from the image sensor 11 reaches the target value (step S205: Yes), this processing ends. If the acquired illuminance determination unit 1216 determines that the input illuminance acquired from the image sensor 11 does not reach the target value (step S205: No), the processing proceeds to step S206.

[0085] The estimation unit 1217 estimates the input illuminance acquired by the acquired illuminance determination unit 1216 as the estimated illuminance (step S206).

[0086] Next, the estimated illuminance determination unit 1218 determines whether the estimated illuminance exceeds the second threshold (step S206). If the estimated illuminance determination unit 1218 determines that the estimated illuminance exceeds the second threshold (step S206: No), the process proceeds to step S208. If the estimated illuminance determination unit 1218 determines that the estimated illuminance is lower than the second threshold (step S203: Yes), the process proceeds to step S209.

[0087] The second adjustment unit 1219 adjusts the pixel unit 111 so as to decrease the gain (step S208).

[0088] The third adjustment unit 1220 adjusts the pixel unit 111 to extend the exposure time (step S209).

[0089] Next, the acquired illuminance determination unit 1216 determines whether the input illuminance acquired from the image sensor 11 reaches the target value (step S210). If the acquired illuminance determination unit 1216 determines that the input illuminance acquired from the image sensor 11 reaches the target value (step S210: Yes), this processing ends. If the acquired illuminance determination unit 1216 determines that the input illuminance acquired from the image sensor 11 does not reach the target value (step S210: No), the processing proceeds to step S211.

[0090] The estimation unit 1217 estimates the input illuminance acquired by the acquired illuminance determination unit 1216 as an estimated illuminance (step S211).

[0091] Next, the estimated illuminance determination unit 1218 determines whether the estimated illuminance exceeds the third threshold (step S212). If the estimated illuminance determination unit 1218 determines that the estimated illuminance is below the third threshold (step S212: Yes), this process ends. If the estimated illuminance determination unit 1218 determines that the estimated illuminance exceeds the third threshold (step S203: No), the process proceeds to step S213.

[0092] The third adjustment unit 1220 adjusts the exposure time for the pixel unit 111 to be shorter (step S213), and then the process returns to step S201.

[0093] As described above, the image processing device of the second embodiment adjusts the gain according to the input illuminance acquired by the imaging device. As a result, the image processing device of the second embodiment can clearly display the image of the digital mirror 2 by adjusting the gain.

[0094] The above-described embodiment can be modified as needed by partially changing the configuration or functions of each of the above-described devices. Therefore, several modifications of the above-described embodiment will be described below as other embodiments. The following mainly focuses on differences from the above-described embodiment, and detailed descriptions of commonalities with the content already described will be omitted. The modifications described below may be implemented individually or in appropriate combination.

[0095] In the above-described embodiment, the camera 1 adjusts the exposure time according to the estimated illuminance. However, if the vehicle 5 is traveling in a nighttime environment, the exposure time may be frequently changed. Therefore, hereinafter, a modified example will be described in which the camera 1 adjusts the exposure time according to the estimated illuminance.

[0096] (Variation) For example, if the state displayed on the electronic mirror 2 is set to a state where it is used at night or the displayed image is darkened, the brightness behind the vehicle may vary, for example, when the headlights of a following vehicle 5 turn in and out, or when the headlights of a following vehicle are approaching. To ensure sufficient brightness behind the vehicle, the camera 1 may adjust the exposure time and frequently switch the exposure time. Frequent switching of the exposure time makes the image displayed on the electronic mirror 2 difficult to see.

[0097] Therefore, the third adjustment unit 1220 adjusts the exposure time using histogram auto exposure. Histogram auto exposure adjusts the exposure based on the result of the determination by the estimated illuminance determination unit 1218 as to whether the lowest illuminance among the input illuminances acquired by the pixel unit 111 exceeds the predetermined threshold, for example.

[0098] As a result, the third adjustment unit 1220 can reduce the frequency of switching the exposure time even when the brightness behind the vehicle varies in a state where the brightness behind the vehicle varies.

[0099] The programs executed in the digital mirroring system of this embodiment are provided in advance in a ROM, etc. The programs executed in the digital mirroring system of this embodiment may be provided by being recorded in an installable or executable file format on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a digital versatile disk (DVD).

[0100] Furthermore, the program executed by the digital mirroring system of this embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the program executed by the information processing system of this embodiment may be provided or distributed via a network such as the Internet.

[0101] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention described in the claims and their equivalents. [Explanation of symbols]

[0102] 1 camera 2 Electronic mirrors 3 Electronic mirror system 4 Rear window 5 vehicles 11 Image sensor 12 Signal Processor (ISP) 13 Interface section (I / F section) 21 Interface section (I / F section) 22 Video Processing Unit 23 Display Panel 111 Pixel section 121 Signal Processing Unit 1211 Image acquisition unit 1212 Color temperature acquisition section 1213 Detector 1214 1st adjustment section 1215 Output section 1216 Acquired illuminance judgment unit 1217 Estimation Department 1218 Estimated illuminance determination unit 1219 2nd adjustment section 1220 Third adjustment section

Claims

1. An image processing method for an in-vehicle camera disposed in a vehicle cabin, comprising: An image of the outside of the vehicle is captured through glass disposed in the vehicle; Acquire an image, obtaining a plurality of color temperatures from the image; detecting a color temperature range that the vehicle-mounted camera regards as white from the plurality of color temperatures; adjusting the color temperature range by referring to a table that stores the color temperature range according to the spectral characteristics of glass so that the color temperature range becomes a color temperature captured by an on-board camera disposed outside the passenger compartment of the vehicle; adjusting the white balance of the image based on the color temperature range; outputting the image with the adjusted white balance to an external device; Image processing method for in-vehicle cameras.

2. 2. The image processing method for an in-vehicle camera according to claim 1, outputting the image whose white balance has been adjusted to an electronic mirror disposed in the passenger compartment of the vehicle; Image processing method for in-vehicle cameras.

3. An in-vehicle camera disposed in a vehicle cabin, The vehicle-mounted camera is capturing an image of the outside of the vehicle through glass disposed in the vehicle, and acquiring the image; obtaining a plurality of color temperatures from the image; detecting a color temperature range that the vehicle-mounted camera regards as white from the plurality of color temperatures; adjusting the color temperature range by referring to a table that stores the color temperature range according to the spectral characteristics of glass so that the color temperature range becomes a color temperature captured by an on-board camera disposed outside the passenger compartment of the vehicle; adjusting the white balance of the image based on the color temperature range; outputting the image with the adjusted white balance to an external device; In-car camera.

4. The vehicle-mounted camera according to claim 3, The vehicle-mounted camera outputs the image whose white balance has been adjusted to an electronic mirror disposed in the passenger compartment of the vehicle. In-car camera.

5. The vehicle-mounted camera according to claim 3, The vehicle-mounted camera is disposed at the rear of the cabin of the vehicle and in front of the rear window of the vehicle. In-car camera.

6. The vehicle-mounted camera according to claim 3, The vehicle-mounted camera includes an image sensor and a signal processor, the image sensor captures the image; The signal processor obtaining a plurality of color temperatures from the image; detecting a color temperature range that the vehicle-mounted camera regards as white from the plurality of color temperatures; adjusting the color temperature range by referring to a table that stores the color temperature range according to the spectral characteristics of glass so that the color temperature range becomes a color temperature captured by an on-board camera disposed outside the passenger compartment of the vehicle; adjusting the white balance of the image based on the color temperature range; outputting the image whose white balance has been adjusted to the outside; In-car camera.

7. An image processing method for an in-vehicle camera disposed in a vehicle cabin, comprising: An image of the outside of the vehicle is captured through glass disposed in the vehicle; Acquire an image, obtaining a plurality of color temperatures from the image; detecting a color temperature range that the vehicle-mounted camera regards as white from the plurality of color temperatures; adjusting the color temperature range by referring to a table that stores the color temperature range according to the spectral characteristics of the glass; adjusting the white balance of the image based on the color temperature range; outputting the image with the adjusted white balance to an external device; Image processing method for in-vehicle cameras.

8. 8. The image processing method for an in-vehicle camera according to claim 7, outputting the image whose white balance has been adjusted to an electronic mirror disposed in the passenger compartment of the vehicle; Image processing method for in-vehicle cameras.

Images