Image display device, imaging device, and image processing device

Abstract

To provide an image display device capable of preventing a pickup image displayed on a display from making a passenger feel uncomfortable.SOLUTION: An electronic mirror system 10 including a wiper unit 18, and an auto light unit 24 detects from the pickup image of the camera 32 whether or not there is an environmental condition in which water droplets adhering to the rear glass affects the brightness of the photographed image of the camera 32. When determining that the environmental condition has occurred when water droplets adhering to the rear glass affect the brightness of the image captured by the camera 32, a brightness adjustment unit 40 performs a series of dimming processing to suppress brightness on the pickup image displayed on the inner mirror 28. This prevents the displayed image on the inner mirror 28 from giving a driver discomfort such as glare and flickering.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image display device, an imaging device, and an image processing device. [Background technology]

[0002] The imaging system of Patent Document 1 is provided with an adjustment mechanism that reduces changes in brightness of the captured image output from the imaging element of the camera device. Also, the imaging system is provided with a zoom function in the camera device, and when the zoom position is on the optical side of the threshold value, the adjustment mechanism is stopped while the wiper is operating, thereby preventing the image captured by the camera device from looking unnatural.

[0003] The image display device of Patent Document 2 displays an image of the rear of a vehicle captured by a camera on a display. This image display device detects the illuminance in front of and behind the vehicle, and sets the luminance of the display to a first luminance when the front illuminance is less than a set value and the rear illuminance is equal to or greater than a set value, and sets the luminance of the display to a second luminance lower than the first luminance when the front illuminance is equal to or greater than the set value and the rear illuminance is less than the set value. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5404250 [Patent Document 2] Patent No. 6763324 Summary of the Invention [Problem to be solved by the invention]

[0005] However, raindrops and water films on the windshield scatter light. Therefore, when an image captured through the windshield is displayed on a display, if the light from the headlights of other vehicles shines on the windshield, the raindrops and water films become bright and cause glare to the passengers viewing the display. Some cameras and other devices are equipped with an automatic light adjustment function, which adjusts the brightness to reduce glare.

[0006] However, for example, when the wiper device is in operation, the wiper (wiper blade) repeatedly crosses the shooting area, causing repeated changes in brightness, such as blinking, in the image displayed on the display. Such changes in brightness (blinking) of the display can cause discomfort, such as restlessness, to the occupants viewing the display.

[0007] In addition, brightness adjustment by an automatic dimming function generally requires a certain amount of response time, and cannot respond to sudden blinking such as when a wiper passes over the shooting area. Patent Document 1 was devised in consideration of such problems, but the cause of blinking due to the operation of the wiper device is the influence of the wiper itself blocking the shooting area, as well as the influence of light scattering due to raindrops and water film on the windshield. In particular, the water film has a large influence on light scattering because the water film spreads over a wide area of ​​the glass surface due to the wiper wiping the glass surface, and the spread of the water film due to wiping is eliminated immediately after the wiper passes. For this reason, light scattering occurs only after the wiper passes, and the quick elimination leads to further blinking.

[0008] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide an image display device, an imaging device, and an image processing device that are capable of obtaining captured images that do not cause discomfort to occupants when displayed on a display. [Means for solving the problem]

[0009] In order to achieve the above object, the image display device of claim 1 includes a display means for displaying a display image corresponding to a captured image of one side of the outside of the vehicle through a windshield by an imaging means, an environmental detection means for detecting an environmental condition that allows for estimation of whether or not water droplets are attached to the windshield and whether or not water droplets attached to the windshield will affect the brightness of the image captured by the imaging means, and an adjustment means for performing an adjustment such that the brightness of the display image is reduced when the environmental condition detected by the environmental detection means indicates that water droplets are attached to the windshield and that the water droplets attached to the windshield will affect the brightness of the display image, compared to when the environmental condition detected by the environmental detection means indicates that water droplets are not attached to the windshield or that water droplets attached to the windshield are not estimated to affect the brightness of the display image.

[0010] In the image display device of claim 1, a display image corresponding to a captured image of one side of the vehicle outside through the windshield by the imaging means is displayed on the display means. The environment detection means detects an environmental condition that can estimate whether water droplets are attached to the windshield and whether water droplets attached to the windshield affect the brightness of the captured image by the imaging means. The adjustment means adjusts the brightness of the display image when it is displayed on the display means.

[0011] Here, when the environmental state is estimated to be such that water droplets are attached to the windshield and the water droplets attached to the windshield are estimated to affect the brightness of the displayed image, the adjustment means suppresses the brightness of the displayed image more than when the environmental state is estimated to be such that water droplets are not attached to the windshield or when the water droplets attached to the windshield are estimated to affect the brightness of the displayed image.

[0012] As a result, even if the light from the headlights of another vehicle is irradiated onto the windshield with raindrops in a dark surrounding environment, the brightness of the image displayed on the display means is suppressed, so that the image displayed on the display means is not too bright and dazzling.

[0013] The image display device of claim 2 is in claim 1, wherein the environment detection means includes a light / dark detection means for detecting the brightness outside the vehicle, and a wiper device that wipes water droplets adhering to the windshield when activated, and the adjustment means reduces the brightness of the displayed image when the brightness outside the vehicle is below a predetermined level and the wiper device is activated.

[0014] In the image display device according to claim 2, the environment detection means includes a light / dark detection means for detecting the brightness outside the vehicle, and a wiper device which is operated when it rains to wipe off water droplets adhering to the windshield.

[0015] The adjustment means reduces the brightness of the displayed image when the brightness outside the vehicle detected by the brightness detection means is equal to or lower than a predetermined brightness and the wiper device is in operation.

[0016] This makes it possible to estimate the influence of water droplets adhering to the windshield on the brightness of the image captured by the imaging means, and even if the light of the headlights of another vehicle is irradiated onto the windshield, the image displayed on the display means will not be too bright and will not cause glare.

[0017] According to a third aspect of the present invention, in the image display device of the first or second aspect, the adjustment means reduces the brightness of the display image by suppressing the luminance of the display image when the display image is displayed on the display means.

[0018] In the image display device of claim 3, when the display image is displayed on the display means, the adjustment means suppresses the luminance of the display image to suppress the brightness of the display image. For example, color correction (tone correction) or gamma correction can be applied to suppress the luminance. This prevents the display image on the display means from being perceived as dazzling even if the captured image is too bright.

[0019] According to a fourth aspect of the present invention, in the image display device of any one of the first to third aspects, the adjustment means adjusts the brightness of the display means to thereby adjust the brightness of the displayed image.

[0020] In the image display device of claim 4, the adjustment means reduces the brightness of the display means to reduce the brightness of the displayed image. For example, the brightness of the backlight of an LCD can be reduced to reduce the brightness of the display means. This prevents the image displayed on the display means from being too bright, even if the captured image becomes too bright.

[0021] According to a fifth aspect of the present invention, in the image display device of the first or second aspect, the adjustment means reduces the brightness of the image displayed on the display means, thereby suppressing the brightness of the displayed image.

[0022] In the image display device of claim 5, the adjustment means reduces the brightness of the image displayed on the display means to suppress the brightness of the displayed image. This suppresses the brightness of the displayed image on the display means (darkens it), so that the displayed image on the display means does not cause a sense of glare even if the captured image becomes too bright.

[0023] The imaging device of claim 6 includes imaging means for imaging one side of the outside of the vehicle through a windshield; environmental detection means for detecting environmental conditions that enable estimation of whether or not water droplets are attached to the windshield and whether or not water droplets attached to the windshield will affect the brightness of an image captured by the imaging means; and adjustment means for performing an adjustment such that, when the environmental conditions detected by the environmental detection means indicate that water droplets are attached to the windshield and that the water droplets attached to the windshield will affect the brightness of the image captured by the imaging means, the brightness of the captured image is reduced compared to when the environmental conditions detected by the environmental detection means indicate that water droplets are not attached to the windshield or that water droplets attached to the windshield are not estimated to affect the brightness of the captured image.

[0024] In the imaging device of claim 6, when the environmental state is estimated to be that water droplets are on the windshield and that the water droplets on the windshield affect the brightness of the captured image, the brightness of the captured image is suppressed compared to when the environmental state is estimated to be that water droplets are not on the windshield or that the water droplets on the windshield are not estimated to affect the brightness of the captured image.

[0025] As a result, when an image is captured through raindrop-covered windshield in a dark environment, even if the captured image is illuminated by the light of the vehicle's headlights, the captured image can be prevented from becoming too bright, and even if the captured image is displayed on a display means, the user does not feel any glare.

[0026] The image processing device of claim 7 includes an environmental detection means for detecting environmental conditions that can estimate whether or not water droplets are attached to the windshield and whether or not water droplets attached to the windshield will affect the brightness of an image captured by an imaging means, and an adjustment means for performing an adjustment such that, when the environmental conditions detected by the environmental detection means for a captured image of one side of the outside of the vehicle through the windshield by the imaging means indicate that water droplets are attached to the windshield and that the water droplets attached to the windshield will affect the brightness of the captured image, the brightness of the captured image is reduced compared to when the environmental conditions detected by the environmental detection means indicate that water droplets are not attached to the windshield or that water droplets attached to the windshield are not estimated to affect the brightness of the captured image.

[0027] In the image processing device of claim 7, even if the headlights of a vehicle are shone on windshield that is assumed to have water droplets and that is assumed to affect the brightness of the captured image, the image captured through the windshield can be prevented from becoming dazzling, so that even if the captured image is displayed on a display means, no glare is felt. Effect of the Invention

[0028] As described above, according to the present invention, the image displayed on the display means does not cause glare, and therefore the image displayed on the display means does not cause discomfort. [Brief description of the drawings]

[0029] [Figure 1] 1 is a schematic configuration diagram of a digital mirroring system according to a first embodiment. [Diagram 2] 1 is a plan view showing a schematic configuration of a vehicle according to a first embodiment. [Diagram 3] FIG. 2 is a front view showing an example of an inner mirror. [Figure 4] 4 is a flowchart showing an outline of a dimming process setting according to the first embodiment. [Diagram 5] (A) to (C) are front views of the inner mirror displaying an image at night, where (A) shows an image that has been dimmed when it is raining, (B) shows an image that is displayed when it is not raining, and (C) shows an image that is not dimmed when it is raining. [Figure 6] 4 is a diagram showing an outline of conversion characteristics of the luminance value of an output image relative to the luminance value of an input image, which is applied to luminance adjustment in the first embodiment. FIG. [Figure 7] 7 is a diagram showing an example of color tone characteristics applied to luminance adjustment in place of FIG. 6. FIG. [Figure 8] FIG. 13A is a schematic configuration diagram of a digital mirror system according to a second embodiment, and FIG. 13B is a diagram showing an example of gamma characteristics for a display according to the second embodiment. [Figure 9] FIG. 11 is a schematic configuration diagram of a digital mirroring system according to a third embodiment. [Figure 10] 13 is a flowchart outlining a main part of a dimming process setting according to the third embodiment. [Figure 11] FIG. 11 is a schematic configuration diagram of a digital mirroring system according to a fourth embodiment. [Figure 12] 13 is a flowchart showing an outline of a dimming process setting according to the fourth embodiment. [Figure 13]FIG. 13 is a schematic configuration diagram of a digital mirroring system according to a fifth embodiment. [Figure 14] 13 is a flowchart showing an outline of a dimming process setting according to the fifth embodiment. [Figure 15] FIG. 13 is a schematic configuration diagram of an electronic mirroring system according to a sixth embodiment. [Figure 16] FIG. 13 is a schematic configuration diagram of an imaging device according to a seventh embodiment. [Figure 17] FIG. 13 is a plan view showing a schematic configuration of a vehicle according to a seventh embodiment. [Figure 18] 23 is a flowchart showing an outline of a dimming process setting according to the seventh embodiment. [Figure 19] FIG. 13 is a schematic configuration diagram of an imaging device according to an eighth embodiment. [Figure 20] 13 is a flowchart showing an outline of a dimming process setting according to the eighth embodiment. [Figure 21] 10 is a diagram showing an outline of a change in brightness of a captured image when a wiper device is not in operation and when it is in operation; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. [First embodiment] Fig. 1 shows a schematic configuration diagram of an electronic mirror system 10 according to the first embodiment, and Fig. 2 shows a schematic configuration of a vehicle 12 according to the first embodiment in a plan view. In the drawings, the front side of the vehicle is indicated by an arrow FR, the right side in the vehicle width direction is indicated by an arrow HR, and the upward direction is indicated by an arrow UP.

[0031] As shown in FIG. 2, the vehicle 12 is provided with a front windshield glass (windshield) 16F at the front of the vehicle interior 14, and a rear windshield glass (rear glass) 16R as a windshield at the rear of the vehicle interior 14. In addition, in the vehicle interior 14, a driver's seat 14D, a passenger seat 14P on the left side of the driver's seat 14D in the vehicle width direction, and a row of rear seats 14R at the rear of the driver's seat 14D and the passenger seat 14P are arranged as seats for passengers to sit in. As such a vehicle 12, one-box cars or two-box cars such as station wagons, minivans, and SUVs (Sport Utility Vehicles) can be applied. In addition, various types of automobiles such as three-box cars (so-called sedan types) can be applied to the vehicle 12 without being limited thereto.

[0032] The vehicle 12 is equipped with a wiper device 18 and a rear wiper device 20. The wiper device 18 is installed on the windshield 16F, and when a wiper switch (not shown) is turned on, the wiper 18A (wiper blade) is operated to wipe the outer surface (front surface of the vehicle) of the windshield 16F. This removes raindrops and the like adhering to the windshield 16F.

[0033] In addition, the rear wiper device 20 is installed relative to the rear windshield 16R, and when a rear wiper switch (not shown) is turned on, the wiper 20A (wiper blade) is operated to wipe the outer surface (front surface of the vehicle) of the rear windshield 16R to remove raindrops, etc.

[0034] The vehicle 12 may be equipped with an automatic light device 24 that turns on / off the headlights 22. The automatic light device 24 detects the brightness (darkness) of the surroundings of the vehicle 12 by an illuminance sensor (not shown) or the like, and turns on the headlights 22, etc. (sidelights, taillights, etc. (not shown)) when the surroundings of the vehicle 12 reach a predetermined brightness (darkness that is set when the headlights 22 are turned on).

[0035] 1 and 2, the electronic mirror system 10 includes an imaging device 26, an inner mirror 28 as a display means, and a display controller (display ECU) 30 for displaying an image captured by the imaging device 26 on the inner mirror 28. In the first embodiment, the inner mirror 28 and the display controller 30 function as an image display device, and the display controller 30 functions as an image processing device.

[0036] The imaging device 26 includes a camera (camera device) 32 as an imaging means, and an imaging element (image sensor) such as a CCD image sensor or a CMOS image sensor is used for the camera 32. The camera 32 is installed in the vehicle interior 14 on the rear side of the vehicle (the vehicle interior 14 side of the rear window 16R) and at an upper part (ceiling surface side) of the center in the vehicle width direction, and the camera 32 captures an image of the rear of the vehicle as one side of the outside of the vehicle through the rear window 16R.

[0037] As shown in Fig. 2, the inner mirror 28 is installed in the vehicle interior 14 at the front of the vehicle (on the inside of the windshield 16F of the vehicle interior 14) and at the upper part of the center in the vehicle width direction, and the display surface of the inner mirror 28 faces the rear of the vehicle. This allows the driver seated in the driver's seat 14D to view the displayed image without interfering with the driver's view of the front of the vehicle. Fig. 3 shows a schematic front view of the inner mirror 28 as seen from the rear of the vehicle.

[0038] As shown in Fig. 3, the inner mirror 28 is provided with a display (monitor) 34 constituting a display means, and the display 34 uses an LCD (Liquid Crystal Display), an LED, or the like (both not shown). An electric signal corresponding to image data is input to the display 34, and the inner mirror 28 displays an image (video) corresponding to the image data. Note that the inner mirror 28 is provided with a half mirror that is semi-transparent and can function as a mirror on the front side (rear side of the vehicle) of the display 34, and the half mirror functions as an optical mirror when the display 34 is in an inoperative state (no image is displayed or the backlight is turned off).

[0039] 1, the display controller 30 is formed with an image processing unit 36 ​​and an environment detection unit 38, and a brightness adjustment unit 40 serving as an adjustment means is formed in the image processing unit 36. The image capture device 26 and the inner mirror 28 (display 34) are connected to the display controller 30, and the display controller 30 performs predetermined image processing in the image processing unit 36 ​​on the captured image (image data) captured by the image capture device 26 (camera 32) and outputs the processed image to the inner mirror 28 (display 34).

[0040] Such a display controller 30 includes a microcomputer in which a CPU, a ROM, a RAM, a storage as a non-volatile memory, an input / output interface, and the like are connected to each other via a bus so as to enable data communication, and required functional circuits are connected to the input / output interface (all of which are not shown). In the display controller 30, the CPU reads out programs for image processing and environment detection stored in the ROM and the storage, and executes the programs while expanding them in the RAM, thereby realizing the functions of the image processing unit 36, the environment detection unit 38, and the brightness adjustment unit 40.

[0041] The image processing unit 36 ​​performs required image processing on the image data (video data) of the captured image captured by the camera 32 in order to display an image corresponding to the captured image on the inner mirror 28 (display 34). The required image processing in the image processing unit 36 ​​includes viewpoint conversion for matching the captured image with the viewpoint of the driver, and processing for inverting the captured image from left to right.

[0042] The brightness adjustment unit 40 adjusts the brightness of the captured image so that the brightness (luminance) of the image displayed on the display 34 falls within a predetermined range. The brightness adjustment unit 40 is capable of switching between normal processing and dimming processing in which the brightness (brightness) is suppressed (dimmed) more than in the normal processing. In the electronic mirror system 10, by performing normal processing in the brightness adjustment unit 40, when an image captured by the camera 32 is displayed on the inner mirror 28 (display 34), the brightness is suppressed when the area behind the vehicle is too bright, and the darkness is suppressed (brightened) when the area behind the vehicle is too dark.

[0043] Furthermore, the brightness adjustment section 40 switches between normal processing and dimming processing according to the detection result (determination result) of the environment detection section .

[0044] An environment detection means is connected to the environment detection unit 38, and the environment detection means detects an environmental condition for estimating whether water droplets are attached to the rear window 16R and whether the water droplets attached to the rear window 16R affect the brightness of the image captured by the camera 32. The environment detection means includes a light / dark detection means for estimating the brightness (light / dark or day / night) around the vehicle 12, and a weather detection means (rainfall detection means) for estimating rainfall as a weather condition around the vehicle 12.

[0045] The environment detection unit 38 judges (estimates) whether the brightness of the surroundings of the vehicle 12 has become equal to or lower than a preset brightness based on the detection result of the light / darkness detection means. As the light / darkness detection means, an illuminance sensor (or a sunshine sensor) for detecting the brightness outside the vehicle can be used. In the first embodiment, the automatic light device 24 is applied, and the automatic light device 24 is connected to the environment detection unit 38.

[0046] Furthermore, the brightness of the image of the rear of the vehicle captured by the camera 32 of the imaging device 26 changes according to the brightness of the surroundings, and the captured image becomes darker as the surroundings become darker. Therefore, in the first embodiment, in addition to the auto light device 24, (the captured image of) the imaging device 26 is used as a light / dark detection means, and the imaging device 26 is connected to the environment detection unit 38.

[0047] In addition, since the brightness (light and dark) around the vehicle 12 can be obtained from calendar information indicating sunrise and sunset at the driving location, a GPS device capable of acquiring position information of the vehicle 12 and calendar information corresponding to the position information can be applied to the light and dark detection means.

[0048] The weather detection means detects whether the surrounding weather is rain (which may include snow or fog). A raindrop sensor can be applied as the weather detection means, but in the first embodiment, the weather detection means can be the wiper device 18 that the driver operates in rainy weather. In the first embodiment, the wiper device 18 and the rear wiper device 20 are connected to an environment detection unit 38. Note that if the rear wiper device 20 is not provided, only the wiper device 18 may be provided.

[0049] The environment detection unit 38 is connected to the brightness adjustment unit 40, and the brightness adjustment unit 40 switches from normal processing to dimming processing when the environment detection unit 38 determines that the brightness around the vehicle 12 is equal to or lower than a predetermined value and that rainfall has been detected. In the first embodiment, the environment detection unit 38 sets the normal processing or dimming processing based on the detection result of the environment detection means, and the brightness adjustment unit 40 performs the normal processing or dimming processing based on this setting.

[0050] Next, the operation of the first embodiment will be described. The electronic mirror system 10 is activated and starts operation when an ignition switch (not shown) of the vehicle 12 is turned on, and stops operation when the ignition switch is turned off. When the electronic mirror system 10 is operated, the imaging device 26 captures an image of the rear of the vehicle through the rear window 16R using the camera 32, and displays an image corresponding to the captured image on the inner mirror 28 disposed in front of the driver's seat 14D. This allows the driver to see the rear of the vehicle in the electronic mirror system 10.

[0051] On the other hand, if raindrops or a water film formed by wiping off raindrops are attached to the rear windshield 16R, the water film attached to the rear windshield 16R is reflected (displayed) in the inner mirror 28. In addition, the water film attached to the rear windshield 16R tends to diffusely reflect light that is irradiated. For this reason, for example, when it is dark and raining, such as at night, the rear windshield 16R receives light from the headlights of the vehicle behind, causing the image displayed on the inner mirror 28 to become too bright. This causes the driver viewing the inner mirror 28 to feel dazzled, reducing the driver's visibility behind the vehicle (causing dazzle).

[0052] From this point, brightness adjustment processing for preventing glare (suppressing glare) is executed in the electronic mirror system 10. An outline of the brightness adjustment processing for preventing glare (setting for brightness adjustment) is shown in a flow chart in FIG.

[0053] 4 is executed at a predetermined time interval by the display controller 30 when the electronic mirror system 10 is started, and ends when the electronic mirror system 10 is stopped. When the electronic mirror system 10 is started, the display controller 30 is set so that the brightness adjustment unit 40 performs normal processing (normal brightness processing).

[0054] 4, the display controller 30 performs brightness detection using an image captured by the camera 32 in the first step 200, and checks whether the brightness of the captured image is equal to or lower than a predetermined threshold value in step 202. In addition, in step 204, the display controller 30 performs brightness detection using the operation of the automatic light device 24, and checks whether the automatic light device 24 is operating in step 206.

[0055] A threshold value for determining the brightness (luminance value) detected from the image captured by the camera 32 is set in the display controller 30, and the display controller 30 detects the brightness of the surroundings of the vehicle 12 from the image captured by the camera 32. The brightness (luminance value) of the peripheral part of the captured image (average brightness value) can be applied as the brightness (luminance value) of the captured image for determining the brightness or darkness of the surroundings of the vehicle 12. In addition, the brightness (luminance value) at which a vehicle traveling nearby may turn on its headlights can be applied as the threshold value for the captured image.

[0056] If the display controller 30 makes a negative determination in step 202 because the brightness (luminance value) detected from the image captured by the camera 32 exceeds the threshold value, and further makes a negative determination in step 206 because the auto light device 24 is not operating, the display controller 30 proceeds to step 208. That is, the display controller 30 proceeds to step 208 when it is determined that the brightness of the surroundings of the vehicle 12 exceeds the threshold value in both the image captured by the camera 32 and the auto light device 24.

[0057] The display controller 30 sets the normal processing to be performed as the processing for adjusting the brightness of the captured image in step 208. As a result, the image of the rear of the vehicle captured by the camera 32 is displayed on the inner mirror 28 after being normally processed for brightness.

[0058] When the display controller 30 detects in at least one of the image captured by the camera 32 and the automatic light device 24 that the brightness around the vehicle 12 is below a predetermined level, the display controller 30 makes an affirmative determination in step 202 or step 206 and proceeds to step 210.

[0059] In step 210, the display controller 30 performs rainfall detection using the wiper device 18. In step 210, the display controller 30 acquires the operation of the wiper device 18, and in step 212, checks whether the wiper device 18 is operating. At this time, if the wiper device 18 is turned off (not operating), the display controller 30 makes a negative determination in step 212 and proceeds to step 208. As a result, even if the display controller 30 determines that the surroundings of the vehicle 12 are dark, if rainfall is not detected, the display controller 30 performs brightness adjustment by normal processing.

[0060] In contrast, when it is determined that the surroundings are dark and the driver operates the wiper switch (not shown) to activate the wiper device 18 in order to remove raindrops adhering to the windshield 16F, the display controller 30 makes a positive judgment in step 212 and proceeds to step 214.

[0061] In step 214, the display controller 30 sets the dimming process to be performed as a process for adjusting the brightness of the captured image. That is, the display controller 30 sets the dimming process to be performed as the brightness adjustment when the surrounding brightness is equal to or lower than a threshold value, or when it is estimated that the surroundings have become dark due to the operation of the automatic light device 24, and it can be determined (estimated) that a rainy state is occurring. As a result, an image corresponding to the captured image of the rear of the vehicle captured by the camera 32 is dimmed and displayed on the inner mirror 28.

[0062] When the display controller 30 sets the dimming process for brightness adjustment, it performs light / dark detection using the image captured by the camera 32 in step 216, and checks whether the brightness of the captured image exceeds a threshold in step 218. The display controller 30 also performs light / dark detection using the automatic light device 24 in step 220, and checks whether the surroundings are bright (whether the operation of the automatic light device 24 has been stopped) in step 222. Furthermore, the display controller 30 performs rainfall detection using the wiper device 18 in step 224, and checks whether the rain has stopped (whether the wiper device 18 has been stopped) in step 226.

[0063] As a result, the display controller 30 continues the dimming process if the brightness (luminance value) of the captured image is below the threshold value (negative determination in step 218), if the automatic light device 24 is operating (negative determination in step 222), or if the wiper device 18 is operating (negative determination in step 226).

[0064] Therefore, even if the headlights of the vehicle behind are turned on, the image displayed on the inner mirror 28 does not cause the occupant to feel dazzled. Also, while the surroundings are dark and the wiper device 18 is operating, the brightness of the image displayed on the inner mirror 28 is kept suppressed, so the occupant does not feel the display image flickering.

[0065] Furthermore, if the brightness (luminance value) of the captured image exceeds the threshold value, the operation of the automatic light device 24 is stopped, and the wiper device 18 is stopped, the display controller 30 makes an affirmative determination in each of steps 218, 222, and 226 and proceeds to step 208. As a result, the luminance adjustment in the luminance adjustment unit 40 is returned from the dimming process to the normal process.

[0066] Figure 5(A) from Figure 5(C)Fig. 5 shows a schematic front view of the inner mirror 28 on which an image corresponding to an image captured by the camera 32 at night is displayed, and Fig. 6 shows a schematic diagram of the conversion characteristics used for brightness adjustment, and Fig. 5(A) shows the inner mirror 28 on which an image subjected to dimming processing when it is raining is displayed, Fig. 5(B) shows the inner mirror 28 on which an image when it is not raining (an image with normal brightness processing) is displayed, and Fig. 5(C) shows the inner mirror 28 on which an image not subjected to dimming processing when it is raining is displayed. In Fig. 6, the horizontal axis represents the brightness (brightness value) of the input image, and the vertical axis represents the output (output brightness level), and the conversion characteristics applied to normal processing are shown by dashed lines, and the conversion characteristics applied to suppression processing are shown by solid lines.

[0067] As shown in Fig. 6, the conversion characteristic applied to the dimming process is shifted so that the luminance value is lower than that of the conversion characteristic applied to the normal process. As a result, the image displayed on the inner mirror 28 is less bright overall when the dimming process is applied (see Fig. 5(A)) than when the normal process is applied (see Fig. 5(B)).

[0068] On the other hand, when it is raining, raindrops adhere to the rear windshield 16R, and a water film or the like is formed when the rear wiper device 20 is operated. As shown in Fig. 5(C), when the headlights of a vehicle behind are shone on the rear windshield 16R with water droplets or a water film adhered thereto at night or the like, the water droplets or the like diffuse the light of the headlights. In particular, when the rear wiper device 20 is operated and a water film is formed on the rear windshield 16R, not only does the water film portion become brighter, causing the driver to feel dazzled, but the visibility of the displayed image (the driver's visibility behind the vehicle) is also reduced.

[0069] Here, the electronic mirror system 10 is set to perform dimming processing when it is determined that the surroundings of the vehicle 12 are dark and raining, and performs brightness adjustment so that the brightness value of the displayed image is lowered. Therefore, as shown in Fig. 5(A), the brightness of the image displayed on the inner mirror 28 is suppressed. This prevents the driver from feeling dazzled and does not reduce visibility behind the vehicle.

[0070] In this way, the electronic mirror system 10 uses the image captured by the camera 32 and the automatic light device 24 for light and dark detection, and the wiper device 18 for rain detection. Possible Moreover, since the function of the vehicle 12 is used to determine (estimate) whether or not there is a possibility that the image displayed on the inner mirror 28 is perceived as dazzling, it is not necessary to add a functional component to suppress the image displayed on the inner mirror 28 from being perceived as dazzling.

[0071] In the digital mirror system 10, when performing light / dark detection using the image captured by the camera 32, the brightness (luminance value) of the captured image is applied, but this is not limiting, and the contrast of the captured image may also be applied.

[0072] Moreover, the display controller 30 switches from the dimming process to the normal process when the brightness (luminance value) of the captured image exceeds the threshold value, the operation of the automatic light device 24 is stopped, and the wiper device 18 is stopped (positive judgments in each of steps 218, 222, and 226). However, the dimming process may be switched to the normal process only when the headlights of the rear vehicle or the like are not dazzling. For example, regardless of the brightness of the surroundings, the operation of the wiper device 18 may be stopped, and the normal process may be switched to after a predetermined time (a time when it is determined that the influence of water droplets on the rear windshield 16R has disappeared; for example, a preset time of several minutes to several tens of minutes) has elapsed from a state in which it is determined that the rain has stopped. Moreover, when the surroundings become bright enough to turn off the headlights, the normal process may be switched to regardless of whether it is raining or not (regardless of whether the wiper device 18 is operating / stopped).

[0073] In addition, the luminance adjustment unit 40 uses the conversion characteristics shown in FIG. 6 for the luminance adjustment. However, color tone characteristics may be applied to the luminance adjustment, or gamma characteristics (gamma coefficients) may be applied instead of color tone characteristics. In addition, the luminance adjustment may be performed using a lightness adjustment. Apply Fig. 7 shows an outline of an example of a color tone characteristic (tone curve) applied to brightness adjustment in a diagram. In Fig. 7, the horizontal axis represents the brightness (brightness value) of the input image, and the vertical axis represents the brightness (output brightness level) of the output.

[0074] As shown in Fig. 7, when applying color tone characteristics to brightness adjustment, the color tone characteristics applied to the dimming process (shown by a solid line in Fig. 7) are set to be darker than the color tone characteristics applied to normal processing (shown by a dashed line in Fig. 7) in areas where the brightness of the input image is relatively dark. When applying gamma (gamma correction) to brightness adjustment, the gamma coefficient γ is made smaller for the dimming process than for normal processing. When applying brightness adjustment, the brightness of the captured image is adjusted to be lower in the dimming process than in normal processing.

[0075] Therefore, the dimming process emphasizes black in the image behind the vehicle displayed on the inner mirror 28, and the overall brightness is suppressed (effectively dimmed). As a result, even if the light from the headlights of the vehicle behind is irradiated onto water droplets or a water film adhering to the rear windshield 16R, the water droplets or water film part can be prevented from becoming too bright, and the driver does not feel dazzled when viewing the image displayed on the inner mirror 28.

[0076] Second Embodiment Next, a second embodiment of the present invention will be described. In the second embodiment, the same functional parts as those in the first embodiment are denoted by the same reference numerals as those in the first embodiment, and detailed description thereof will be omitted.

[0077] FIG. 8(A) shows a schematic diagram of a digital mirror system 50 according to the second embodiment. In the second embodiment, brightness adjustment is performed by varying gamma correction for an image displayed on the display 34, and FIG. 8(B) shows a diagram of an example of gamma characteristics applied to the gamma correction for the display 34. In FIG. 8(B), the horizontal axis represents the brightness level of an input image, and the vertical axis represents the brightness level of an output image. In FIG. 8(B), the dashed line represents the gamma characteristics applied to normal processing, and the solid line represents the gamma characteristics applied to dimming processing.

[0078] 8A, in the electronic mirror system 50, a brightness adjustment unit 54 serving as an adjustment means is formed in a display controller 52. The electronic mirror system 50, the display controller 52, and the image processing unit 56 of the second embodiment differ from the electronic mirror system 10, the display controller 30, and the image processing unit 36 ​​of the first embodiment in that the brightness adjustment unit 54 is formed instead of the brightness adjustment unit 40.

[0079] The image processing unit 56 performs required image processing on the image data of the captured image captured by the camera 32 in order to display the captured image on the inner mirror 28 (display 34). The processing performed by the image processing unit 56 includes viewpoint conversion for matching the captured image with the driver's viewpoint, and processing for inverting the captured image from left to right.

[0080] The brightness adjustment unit 54 adjusts the brightness of the image displayed on the display 34 by performing gamma correction as a brightness adjustment (brightness adjustment) on the image (image data) output from the image processing unit 56 to the display 34, using gamma characteristics that are preset for the display 34.

[0081] As shown in Fig. 8(B), the brightness adjustment unit 54 uses different gamma characteristics (gamma coefficients) between normal processing and dimming processing. In dimming processing, the gamma coefficient is set so that the output brightness is lower relative to the input brightness compared to normal processing, and the gamma coefficient applied to dimming processing is set smaller than the gamma coefficient applied to normal processing. As a result, the brightness adjustment unit 54 applies different gamma coefficients between normal processing and dimming processing.

[0082] 8(A), the detection result (luminance processing setting according to the determination result) of the environment detection unit 38 is input to the luminance adjustment unit 54. The luminance adjustment unit 54 switches between normal processing and dimming processing based on the detection result (luminance processing setting) input from the environment detection unit 38.

[0083] In the display controller 52 of the electronic mirror system 50 configured as described above, normal processing is set based on the detection result of the environment detection unit 38, and the brightness adjustment unit 54 adjusts the brightness of the image output from the image processing unit 56 to the display 34 based on the gamma characteristic set for normal processing. In addition, when the brightness around the vehicle 12 is below a predetermined level based on the image captured by the camera 32 and the operating state of the auto light device 24, and it is detected that it is raining (rainfall) based on the operating state of the wiper device 18, the display controller 52 sets to dimming processing (see FIG. 4 of the first embodiment).

[0084] When the dimming process is set, the brightness adjustment unit 54 adjusts the brightness of the image (image data) output from the image processing unit 56 to the display 34 based on the gamma characteristic set for the dimming process. As a result, the brightness is suppressed and the brightness is reduced in the image of the rear of the vehicle displayed on the inner mirror 28. As a result, even if the headlights of the vehicle behind are turned on at night or the like with water droplets, a water film, or the like attached to the rear windshield 16R, the water droplets or water film portion can be prevented from becoming too bright.

[0085] Therefore, in the electronic mirror system 50, like the electronic mirror system 10, the rear image displayed on the inner mirror 28 during rainy hours, such as at night, does not cause the driver to feel dazzled, and does not reduce visibility behind the vehicle.

[0086] On the other hand, in the electronic mirror system 50, the brightness of the image displayed on the inner mirror 28 is adjusted by performing a different specific gamma correction in the brightness adjustment unit 54. The adjustment of the brightness of the image displayed on the inner mirror 28 is not limited to this, and the brightness (luminance) of a backlight (not shown) provided on the display 34 may also be adjusted. In this case, in the dimming process, the brightness of the backlight may be suppressed (the amount of light may be reduced) compared to normal processing. This prevents the water droplets or water film from becoming too bright even if the light from the headlights of the vehicle behind is irradiated onto water droplets or a water film adhering to the rear windshield 16R.

[0087] Third Embodiment Next, a third embodiment of the present invention will be described. In the third embodiment, functional parts similar to those in the first or second embodiment are denoted by the same reference numerals as the corresponding functional parts in the first or second embodiment, and detailed descriptions thereof will be omitted.

[0088] FIG. 9 is a schematic diagram showing the configuration of an electronic mirror system 60 according to the third embodiment. As shown in FIG. 9, the electronic mirror system 60 includes an imaging device 26, an inner mirror 28, and a display controller 62. The display controller 62 includes an image processing unit 66 in which a luminance adjustment unit 64 is formed, an adjustment switching unit 68 as a switching means, and an environment detection unit 70. In the electronic mirror system 60 according to the third embodiment, the environment detection unit 70 of the display controller 62 has a function similar to that of the environment detection unit 38 of the first embodiment. The luminance adjustment unit 64 formed in the image processing unit 66 of the display controller 62 performs normal processing and dimming processing, but differs from the luminance adjustment unit 40 of the first embodiment in that a first dimming processing and a second dimming processing are set in the dimming processing. The display controller 62 also includes an adjustment switching unit 68, and differs from the display controller 30 of the first embodiment in that the luminance adjustment unit 64 is formed in place of the luminance adjustment unit 40 in the display controller 62, and the adjustment switching unit 68 is formed.

[0089] The luminance adjustment unit 64 adjusts the luminance in the same manner as the luminance adjustment unit 40 of the first embodiment. In the luminance adjustment process, the luminance adjustment unit 64 has a normal process, a first dimming process for suppressing (dimming) the luminance (brightness) more than the normal process, and a second dimming process for further suppressing (dimming) the luminance (brightness) more than the first dimming process set. The luminance adjustment unit 64 is switched to the normal process, the first dimming process, or the second dimming process according to the detection result of the environment detection unit 70.

[0090] The adjustment switching unit 68 is connected to a rear wiper device 20 for wiping off raindrops on the rear windshield 16R. The rear wiper device 20 applied to the third embodiment is capable of selecting a continuous mode in which the wiper 20A is operated continuously, and an intermittent mode in which the wiper 20A is operated intermittently at a predetermined time interval. The continuous mode and the intermittent mode are selected by operating a rear wiper switch (not shown).

[0091] When the environment detection unit 70 detects that the brightness around the vehicle 12 is equal to or lower than a predetermined value and estimates that it is raining, it switches from normal processing to dimming processing. When the environment detection unit 70 is set to perform dimming processing, the adjustment switching unit 68 sets the adjustment switching unit 68 to perform dimming processing in response to the operation of the rear wiper device 20.

[0092] The average wiping speed (movement speed) of the wiper 20A (movement time per unit time) differs between the continuous mode and the intermittent mode in the rear wiper device 20, and the average wiping speed is faster in the continuous mode than in the intermittent mode. Water droplets, etc., adhering to the rear windshield 16R are spread on the rear windshield 16R by being wiped away by the wiper 20A to form a water film, etc., but the formed water film gradually narrows. At this time, when there is a lot of rain, the continuous mode is often selected, and even if the wiper 20A wipes away frequently, the water film is likely to remain, and the remaining water film causes glare in the displayed image on the inner mirror 28.

[0093] From here, the adjustment switching unit 68 sets the brightness adjustment unit 64 to perform the second dimming process when the rear wiper device 20 is operated in the continuous mode, and sets the brightness adjustment unit 64 to perform the first dimming process when the rear wiper device 20 is operated in the intermittent mode.

[0094] Next, the dimming process executed by the display controller 62 as the operation of the third embodiment will be described. In the electronic mirror system 60, the flowchart of Fig. 4 is executed in the same manner as in the display controller 30 of the first embodiment. As a result, the display controller 62 switches from normal processing to dimming processing when it is determined that the brightness around the vehicle 12 is equal to or lower than a predetermined brightness and rainfall is detected, and switches from dimming processing to normal processing when the brightness of the surroundings exceeds the predetermined brightness and rainfall is no longer detected.

[0095] Here, when the dimming process is set in the display controller 62, the dimming process is executed in response to the operation of the rear wiper device 20, and the intensity of the dimming process is changed. Fig. 10 is a flow chart showing an outline of the dimming process setting process executed by the display controller 62 in the electronic mirror system 60 by transitioning to step 214 in Fig. 4.

[0096] The flowchart of FIG. 10 is executed at a predetermined time interval when the display controller 62 sets the environment detection unit 70 to perform the dimming process (transition to step 214 in FIG. 4), and detects the operation of the rear wiper device 20 in the first step 230, and checks whether the rear wiper device 20 is operating or not in step 232. At this time, if the rear wiper device 20 is not operating, a negative determination is made in step 232, and the process transitions to step 216 in FIG. 4. This causes the brightness adjustment unit 64 to perform normal processing on the captured image. Note that, when the flowchart of FIG. 4 is applied in the third embodiment, if a negative determination is made in any of steps 218, 222, and 226, the process returns to step 214.

[0097] In contrast, when the rear wiper device 20 is operated, the display controller 62 makes an affirmative determination in step 232 and proceeds to step 234 to check whether the rear wiper device 20 is operating in the intermittent mode. At this time, if the rear wiper device 20 is operating in the intermittent mode, the display controller 62 makes an affirmative determination in step 234 and proceeds to step 236 to set the first dimming process as the dimming process. As a result, the brightness adjustment unit 64 performs the first dimming process on the captured image, and an image with reduced brightness compared to normal processing is displayed on the inner mirror 28.

[0098] Furthermore, if the rear wiper device 20 is operated in the continuous mode, the display controller 62 makes a negative determination in step 234 and proceeds to step 238, where it sets the second dimming process as the dimming process. As a result, the brightness adjustment unit 64 performs the second dimming process, which reduces the brightness of the captured image more than the first dimming process, and an image with even lower brightness than the normal process is displayed on the inner mirror 28.

[0099] Furthermore, the display controller 62 checks in step 240 whether or not the rear wiper device 20 has been stopped, and if the rear wiper device 20 has not been stopped, a negative determination is made in step 240, and the process proceeds to step 234. As a result, the first dimming process or the second dimming process continues.

[0100] In contrast, when the rear wiper device 20 is stopped, the display controller 62 makes an affirmative determination in step 240 and proceeds to step 242. In step 242, the display controller 62 sets the normal processing and ends the setting processing of the dimming processing. Note that, when setting the normal processing in step 242, the display controller 62 sets the normal processing after a predetermined time (for example, about several tens of seconds to several minutes, a time during which the change in the display image of the inner mirror 28 does not bother the driver) has elapsed. As a result, in the electronic mirror system 60, even if the rear wiper device 20 is repeatedly turned on and off or the operation time is short, the brightness of the display image of the inner mirror 28 is suppressed from changing in a flickering manner, thereby preventing the driver from feeling uncomfortable.

[0101] In the third embodiment, the dimming process is set to two stages. However, the dimming process may be set to three stages. In this case, a third dimming process that dims the light less than the first dimming process is provided between the normal process and the first dimming process, and the third dimming process is set when the rear wiper device 20 is not operating (when the negative determination is made in step 232 of FIG. 10).

[0102] In the third embodiment, the intensity of the dimming of the captured image (image displayed on the inner mirror 28) is varied depending on whether the rear wiper device 20 is in the intermittent mode or the continuous mode. However, the wiping speed (average wiping speed, wiping frequency) of the wiper 20A may be detected from the captured image of the camera 32, and the faster the wiping speed is, the stronger the dimming may be.

[0103] In this case, the wiper (wiper blade) 20A can be detected from the captured image by extracting the shadow of the wiper 20A reflected in the captured image. The wiping speed of the wiper 20A can be calculated as an average value of the movement amount per unit time from the change in the extracted position of the wiper 20A by extracting the position of the wiper 20A from each frame of the captured image or from frames at a predetermined time interval.

[0104] In addition, the dimming process may be performed with a strength corresponding to the set dimming ratio, which is set as an index showing the strength of dimming relative to the wiping speed of the wiper 20A so that the dimming ratio increases as the wiping speed increases.

[0105] [Fourth embodiment] Next, a fourth embodiment of the present invention will be described. In the fourth embodiment, functional parts similar to those in any of the first to third embodiments are given the same reference numerals as the corresponding functional parts in any of the first to third embodiments, and detailed descriptions thereof will be omitted.

[0106] FIG. 11 is a schematic diagram showing the configuration of an electronic mirror system 80 according to the fourth embodiment. 11, the electronic mirror system 80 includes an imaging device 26, an inner mirror 28, and a display controller 82. The display controller 82 is provided with an image processing unit 66, an adjustment switching unit 84 as a switching means, and an environment detection unit 38. The electronic mirror system 80 according to the fourth embodiment differs from the electronic mirror system 60 of the third embodiment in that the adjustment switching unit 84 and the environment detection unit 38 are formed in the display controller 82 instead of the adjustment switching unit 68 and the environment detection unit 70 of the display controller 62 of the third embodiment.

[0107] An image captured by the imaging device 26 is input to the adjustment switching unit 84. The adjustment switching unit 84 detects brightness from the captured image, and determines whether or not to perform dimming processing according to the detected brightness, and sets the strength of the dimming processing if dimming processing is performed. The brightness adjustment unit 64 switches between normal processing and dimming processing based on the setting of the adjustment switching unit 84, and also switches the strength of the dimming processing when dimming processing is performed.

[0108] Next, a dimming process executed by the display controller 82 will be described as a fourth embodiment. FIG. 12 is a flow chart showing an outline of the brightness adjustment process that is executed at predetermined time intervals by the display controller 82 when the digital mirror system 80 is started up.

[0109] As shown in FIG. 12, when the brightness of the surroundings of the vehicle 12 is equal to or lower than a predetermined brightness in either the image captured by the camera 32 or the automatic light device 24, and the wiper device 18 is operated to detect a rainy condition, the display controller 82 proceeds to step 250.

[0110] In step 250, the display controller 82 acquires an image captured by the camera 32, and in the next step 252, it is confirmed whether or not the brightness of the acquired captured image is equal to or greater than a preset brightness. The preset brightness may be the brightness of an image captured at night or other times when it is not raining (when there are no raindrops on the rear windshield 16R) with the headlights of the vehicle behind switched on. In addition, the preset brightness may be the brightness at which the driver viewing the image displayed on the inner mirror 28 feels dazzling even when it is raining at night. I don't feel it Brightness may also be applied.

[0111] If the brightness of the captured image does not exceed a preset brightness, it can be determined that the driver does not find the image displayed on the inner mirror 28 too dazzling, so the display controller 82 makes a negative judgment in step 252 and proceeds to step 208.

[0112] Here, when it is raining at night or the like and the headlights of the vehicle behind are turned on, water droplets and the like adhering to the rear window 16R brighten the captured image. If the brightness of the captured image exceeds a preset brightness, the display controller 82 makes an affirmative determination in step 252 and proceeds to step 214 to set a dimming process.

[0113] Thereafter, the display controller 82 detects the brightness of the captured image in step 250. As a result, when the brightness of the captured image becomes equal to or lower than a preset brightness, the display controller 82 makes an affirmative determination in step 252 and proceeds to step 208, where the luminance processing is set to normal processing.

[0114] In this way, the electronic mirror system 80 (display controller 82) performs dimming processing when the image captured by the camera 32 becomes too bright for the driver to perceive as dazzling, even during rainfall at night, etc. This prevents dimming processing from being performed when there is no vehicle behind with its headlights on that would cause glare, and prevents the image displayed on the inner mirror 28 from appearing too dark due to dimming processing.

[0115] In addition, the electronic mirror system 80 (display controller 82) can prevent the image displayed on the inner mirror 28 from being dazzling to the driver due to lighting behind the vehicle 12, not limited to that of the vehicle behind, thereby improving the driver's visibility behind the vehicle. The light reduction process set in step 242 may be set in accordance with the brightness detected from the captured image, with the strength of the light reduction process being set to be stronger as the detected brightness becomes brighter.

[0116] Fifth embodiment Next, a fifth embodiment of the present invention will be described. In the fifth embodiment, functional parts similar to those in any of the first to fourth embodiments will be denoted by the same reference numerals as those corresponding to the functional parts in any of the first to fourth embodiments, and detailed descriptions thereof will be omitted.

[0117] In the fourth embodiment, the process switches from normal processing to dimming processing when the brightness detected from the image captured by the camera 32 becomes equal to or higher than a preset brightness. In contrast, in the fifth embodiment, a rear vehicle is detected, and when the rear vehicle is detected, dimming processing is performed on the captured image (the image displayed on the inner mirror 28).

[0118] FIG. 13 is a schematic diagram showing the configuration of an electronic mirror system 90 according to the fifth embodiment. 13, the electronic mirror system 90 according to the fifth embodiment includes an imaging device 26, an inner mirror 28, and a display controller 92. The display controller 92 is provided with an image processing unit 66, an adjustment switching unit 94 as a switching means, and an environment detection unit 38. The electronic mirror system 90 according to the fifth embodiment differs from the electronic mirror system 80 according to the fourth embodiment in that the adjustment switching unit 94 is provided in the display controller 92 instead of the adjustment switching unit 84 of the fourth embodiment.

[0119] A vehicle detection device 96 that detects a rear vehicle is connected to the adjustment switching unit 94, and the adjustment switching unit 94 switches the brightness adjustment process of the brightness adjustment unit 64 depending on whether a rear vehicle has been detected.

[0120] Various configurations can be applied to the vehicle detection means applied to the vehicle detection device 96 as long as they are capable of detecting a rear vehicle. For example, the vehicle detection means can be a millimeter wave radar method that uses radio waves with frequencies of 30 GHz to 300 GHz called millimeter waves and can measure the position and distance of an object. In addition, the vehicle detection means can be a known method such as a LiDAR (Light Detection Ranging) method that uses light such as infrared light (laser light) to measure the direction of light irradiation and the time it takes for the light reflected by an object to be received.

[0121] In addition, since the rear vehicle is captured in the image captured by camera 32, camera 32 (the image captured by camera 32) can be applied as the vehicle detection means, and in this case, image recognition processing such as extracting an image of the rear vehicle from the image captured by camera 32 can be performed.

[0122] Furthermore, when the rear vehicle has its headlights turned on at night when it is dark, the brightness differs between the front and rear of the vehicle 12, so the vehicle detection means can also use the camera 32 and an imaging means (front imaging means) for capturing images of the front of the vehicle 12, etc. In this case, the brightness (average brightness, or brightness between a preset area such as the center of the captured image, etc.) of the image captured by the camera 32 and the image captured by the front imaging means can be compared to determine whether there is a difference in brightness that allows the presence of a rear vehicle to be estimated.

[0123] The adjustment switching unit 94 detects a rear vehicle using the vehicle detection device 96, and when a rear vehicle is detected, sets (switches) the brightness adjustment unit 64 to perform dimming processing. At this time, the adjustment switching unit 94 may change the strength of the dimming processing depending on whether or not there is a rear vehicle in a rainy condition where the surroundings are dark.

[0124] Next, a dimming process executed by the display controller 92 as an operation of the fifth embodiment will be described. Fig. 14 is a flow chart showing an outline of a setting process for the dimming process executed by the display controller 92 at a predetermined time interval when the electronic mirror system 90 is started.

[0125] As shown in FIG. 14, when the brightness of the surroundings of the vehicle 12 is equal to or lower than a predetermined brightness in both the image captured by the camera 32 and the automatic light device 24, and the wiper device 18 is operated and it is determined that it is raining (positive determination in step 212), the display controller 92 proceeds to step 260.

[0126] In step 260, the display controller 92 checks whether or not the vehicle detection device 96 has detected a rear vehicle. At this time, if the vehicle detection device 96 has not detected a rear vehicle, the display controller 92 makes a negative determination in step 262 and proceeds to step 208.

[0127] In contrast, when the vehicle detection device 96 detects a rear vehicle, the display controller 92 makes an affirmative determination in step 262 and proceeds to step 264. In step 264, the display controller 92 sets the processing in the brightness adjustment unit 64 to dimming processing. This allows the driver to see behind the vehicle without feeling that the image displayed on the inner mirror 28 is dazzling, even if the headlights of the rear vehicle are irradiated onto the rear windshield 16R.

[0128] Furthermore, when the vehicle detection device 96 no longer detects a rear vehicle in a state where the brightness adjustment is set to the dimming process, the display controller 92 makes a negative determination in step 262 and proceeds to step 208. As a result, the brightness adjustment is set to normal processing. Note that even if the vehicle detection device 96 detects a rear vehicle (a state where a positive determination is made in step 252), the display controller 92 proceeds to step 208 (returns to normal processing) when a positive determination is made in each of steps 218, 222, and 226.

[0129] In this way, the electronic mirror system 90 (display controller 92) performs normal processing if no rear vehicle is detected, even during rain at night, etc. This prevents dimming processing from being performed when there is no rear vehicle with its headlights on, which can cause glare, and prevents the image displayed on the inner mirror 28 from appearing dark due to dimming processing.

[0130] Sixth Embodiment Next, a sixth embodiment of the present invention will be described. In the sixth embodiment, functional parts similar to those in any of the first to fifth embodiments will be denoted by the same reference numerals as those corresponding to the functional parts in any of the first to fifth embodiments, and detailed descriptions thereof will be omitted.

[0131] FIG. 15 is a schematic configuration diagram of a digital mirror system 100 according to the fourth embodiment. 15, the electronic mirror system 100 includes an inner mirror 28, a display controller 102 for displaying an image on the inner mirror 28, and an imaging device 104 for capturing an image of the rear of the vehicle through the rear glass 16R. In the sixth embodiment, the imaging device 104 functions as the imaging device of the present invention. The electronic mirror system 100 of the sixth embodiment differs from the electronic mirror systems 10, 50, 60, 80, 90 of the first to fifth embodiments in that the imaging device 104 is provided with a function for adjusting the brightness of the display image on the inner mirror 28 (display 34).

[0132] The display controller 102 of the electronic mirror system 100 is formed with an image processing unit 106, which performs the necessary image processing on the captured image (image data) input from the imaging device 104 in order to display the captured image on the inner mirror 28 (display 34).

[0133] The imaging device 104 includes a camera 32 and an imaging controller (imaging ECU) 108 as an image processing device for controlling imaging using the camera 32 and outputting a captured image. The imaging controller 108 is formed with a brightness adjustment unit 40 as an environment detection means and adjustment means.

[0134] The imaging controller 108 includes a microcomputer in which a CPU, a ROM, a RAM, a storage as a non-volatile memory, an input / output interface, and the like are connected to each other via a bus so as to enable data communication, and a required functional circuit is connected to the input / output interface (all not shown). In the imaging controller 108, the CPU reads out required programs stored in the ROM and the storage and executes them while expanding them in the RAM, thereby realizing a function for imaging using the camera 32, a function for adjusting brightness, and a function for outputting a captured image. As a result, the imaging controller 108 includes an environment detection unit 38 and a brightness adjustment unit 40.

[0135] In the electronic mirror system 100 configured as above, the imaging controller 108 executes the same process for dimming as the display controller 30 of the first embodiment. When the brightness (luminance value) detected from the image captured by the camera 32 is equal to or lower than a threshold value and the wiper device 18 is operated while the auto light device 24 is operating, the imaging controller 108 executes dimming process for the captured image.

[0136] In addition, when the brightness detected from the image captured by the camera 32 exceeds a threshold value, the operation of the automatic light device 24 is stopped (the headlights 22 are turned off), and the wiper device 18 is stopped, the imaging controller 108 performs normal processing for brightness adjustment of the captured image.

[0137] This prevents the headlights of the vehicle behind from making the image displayed on the inner mirror 28 too bright for the driver to perceive as dazzling at night when it is raining, and improves the driver's visibility behind the vehicle even when the vehicle behind is traveling.

[0138] Furthermore, in the imaging device 104, when the imaging controller 108 estimates the environmental state to be nighttime and raining, a dimming process is performed on the image captured by the camera 32. This allows the imaging device 104 to output a captured image that is not dazzling to the driver even when displayed on the inner mirror 28.

[0139] Such an imaging device 104 can be used not only in the electronic mirror system 100 but also in a drive recorder that records captured images in a recording medium such as a non-volatile storage. 。 As a result, when the captured image recorded on the recording medium of the drive recorder is reproduced, even if the image is captured at night on a rainy day, no halation or the like occurs, and a high-quality captured image can be obtained.

[0140] In the sixth embodiment, the imaging device 104 is provided with an adjustment function for the display controller 30. However, the imaging device may be provided with an adjustment function for the display controller 52 according to the second embodiment, an adjustment function for the display controller 62 according to the third embodiment, an adjustment function for the display controller 82 according to the fourth embodiment, or a display controller 92 according to the fifth embodiment.

[0141] In addition, in the camera 32 and the like, the brightness of the captured image can be changed by adjusting the aperture (adjusting the light receiving time of the image sensor). Therefore, in the imaging device, the aperture may be used for the light reduction process, and when the light reduction process is performed, the aperture may be adjusted so that the brightness of the captured image is lower than in the case of normal processing.

[0142] Seventh embodiment Next, a seventh embodiment of the present invention will be described. In the seventh embodiment, functional parts similar to those in any of the first to sixth embodiments will be denoted by the same reference numerals as those corresponding to the functional parts in any of the first to sixth embodiments, and detailed descriptions thereof will be omitted.

[0143] FIG. 16 shows a schematic configuration diagram of an imaging device 110 according to the seventh embodiment, and FIG. 17 shows a schematic configuration of a vehicle 12A according to the seventh embodiment in a plan view.

[0144] 16, the imaging device 110 includes a camera 32 as an imaging means, and an imaging controller 112 that controls imaging using the camera 32 and outputs a captured image. As shown in FIG. 17, in the seventh embodiment, a vehicle 12A is applied instead of the vehicle 12 according to the first embodiment, and the basic configuration of the vehicle 12A is the same as that of the vehicle 12.

[0145] In the vehicle 12A, a camera 32 is disposed in the vehicle interior 14 at the front side of the vehicle (on the inside of the vehicle interior 14 of the windshield 16F) and at the upper part of the center in the vehicle width direction. The camera 32 is directed toward the front of the vehicle, and captures an image of the area in front of the vehicle through the windshield 16F.

[0146] Such an imaging device 110 can be used, for example, as a recording device such as a drive recorder that records captured images of the area in front of a vehicle, or for capturing images to be displayed on a monitor (display) provided on an instrument panel.

[0147] The imaging controller 112 is formed with a brightness adjustment unit 114 as a brightness adjustment means, an environment detection unit 38, and an adjustment switching unit 116. The imaging controller 112 includes a microcomputer in which a CPU, a ROM, a RAM, a storage as a non-volatile memory, an input / output interface, and the like are connected to each other via a bus so as to be capable of data communication, and a required functional circuit is connected to the input / output interface (all of which are omitted in the drawing). In the imaging controller 112, the CPU reads out required programs stored in the ROM and the storage and executes them while expanding them in the RAM, thereby realizing the functions of the environment detection unit 38, the brightness adjustment unit 114, and the adjustment switching unit 116 as well as the function of imaging using the camera 32 and the function of outputting a captured image. At this time, in the imaging controller 112, as the function of the brightness adjustment unit 114, for example, a function similar to that of the brightness adjustment unit 64 in which dimming processing is switched between multiple stages is realized.

[0148] The environment detection unit 38 estimates whether the environmental state can be regarded as nighttime from the image captured by the camera 32 and the operation of the automatic light device 24, and estimates whether the environmental state can be regarded as raining from the operation of the wiper device 18. Furthermore, the environment detection unit 38 sets the brightness adjustment to normal processing or dimming processing according to the estimated environmental state.

[0149] The wiper device 18 can select between intermittent mode and continuous mode by operating a wiper switch (not shown), and can also select the operation interval (intermittent time, interval) in the intermittent mode and the wiping speed in the continuous mode.

[0150] The adjustment switching unit 116 is connected to the wiper device 18, and the adjustment switching unit 116 applies the wiping speed as the operating state of the wiper device 18, and switches the strength of the light reduction process according to the wiping speed.

[0151] The switching adjustment unit 116 is set with a dimming ratio for dimming processing according to the wiping speed of the wiper device 18 (the reciprocating period of the wiper 18A), and the switching adjustment unit 116 is set with a dimming ratio as the strength of the dimming processing according to the wiping speed. The wiping speed is calculated according to the movement amount of the wiper 18A per predetermined time (unit time), and the wiping speed in the intermittent mode is faster when the intermittent time is longer than when the intermittent time is short. is set low (slow). In addition, the light reduction ratio is set so that the higher the wiping speed is, the greater the light reduction ratio becomes compared to a case where the wiping ratio is low. The ratio is The intensity of the dimming process is set to be stronger (brightness is greatly reduced) compared to when the intensity is small. Note that the switching adjustment unit 116 sets the intensity of the dimming process when the wiper device 18 is stopped (wiping speed is zero), and the dimming process at this time is set to be the weakest (it may be the same as the normal process).

[0152] Next, the light reduction process in the imaging device 110 will be described with reference to Fig. 18. Note that Fig. 18 shows an outline of the light reduction process setting executed in the imaging controller 112 in the form of a flow chart.

[0153] As shown in FIG. 18, by executing steps 200 to 206 and steps 210 and 212, the imaging controller 112 proceeds to step 270 when it determines that the brightness around the vehicle 12 in both the image captured by the camera 32 and the automatic light device 24 is equal to or lower than a predetermined brightness and the wiper device 18 is operated to indicate a rainy state (positive determination in step 212).

[0154] In step 270, the imaging controller 112 acquires the speed of the wiper device 18 being operated. That is, whether the wiper device 18 is being operated in intermittent mode or continuous mode, the intermittent time in intermittent mode, and the wiping speed in continuous mode (high speed, low speed, etc.) are acquired, and the dimming ratio is set based on the acquired wiping speed. Next, in step 272, the imaging controller 112 sets the dimming process so that the process is performed with the strength set according to the dimming ratio.

[0155] When the dimming process and the intensity of the dimming process in the brightness adjustment are set in this manner, the imaging controller 112 executes steps 216 to 226, and if an affirmative determination is made in each of steps 218, 222, and 226, the imaging controller 112 proceeds to step 208 (returns to normal processing). If the wiper device 18 is stopped in a state in which a negative determination is made in step 218 or step 222, the imaging controller 112 sets the wiping speed to a minimum value (e.g., 0) and continues the dimming process.

[0156] Here, when the wiper device 18 operates, if the period of the wiper 18A is short, the interval of the brightness change caused by the scattering of light and the shadow of the wiper 18A becomes short. Therefore, when the wiping speed of the wiper device 18 is high, the flickering caused in the captured image becomes larger and the brightness change tends to become larger compared to when the wiping speed is low, which may be annoying when viewing the captured image.

[0157] In this way, when the image capturing device 110 estimates that the environmental state is such that the captured image is perceived as dazzling when viewed, the image capturing device 110 performs a dimming process on the captured image. This prevents the captured image from being perceived as dazzling even if an oncoming vehicle has its headlights turned on.

[0158] Furthermore, the imaging device 110 performs dimming processing with a strength based on the wiping speed of the wiper device 18. At this time, when the wiping speed is high, the strength of the dimming processing is set stronger than when the wiping speed is low. This makes it possible to reduce the change in brightness that occurs in the captured image, and does not cause annoyance when viewing the captured image.

[0159] Eighth embodiment Next, an eighth embodiment of the present invention will be described. In the eighth embodiment, functional parts similar to those in any of the first to seventh embodiments will be given the same reference numerals as those corresponding to the functional parts in any of the first to seventh embodiments, and detailed descriptions thereof will be omitted.

[0160] FIG. 18 is a schematic diagram showing the configuration of an imaging device 120 according to the eighth embodiment. 18, the imaging device 120 is provided with an imaging controller 122, and an adjustment switching unit 124 is formed in the imaging controller 122. The imaging device 120 according to the eighth embodiment differs from the imaging device 110 according to the seventh embodiment in that the adjustment switching unit 124 is formed in the imaging controller 122 instead of the adjustment switching unit 116 in the imaging controller 112 according to the seventh embodiment.

[0161] An image captured by the camera 32 is input to the adjustment switching unit 124 of the imaging controller 122. In the imaging device 120, the wiper device 18 is connected to the adjustment switching unit 124. Furthermore, in the imaging device 120, a defogger device 126 that operates to defog the windshield 16F may be connected.

[0162] The adjustment switching unit 124 acquires (monitors) images captured by the camera 32 capturing an image of the area ahead of the vehicle through the windshield 16F, and calculates an average value (non-operating average value) of the brightness (luminance) of the captured images at a predetermined time interval when the wiper device 18 is not in operation. Furthermore, when the wiper device 18 is in operation, the adjustment switching unit 124 calculates an average value (operating average value) of the brightness of the captured images at a predetermined time interval when the wiper device 18 is in operation. At this time, the predetermined time interval is set to a time shorter than one period of the wiping operation of the wiper 18A of the wiper device 18.

[0163] When setting the dimming process for the captured image, the adjustment switching unit 124 compares the non-operation average value and the operation average value of the captured image (contrasts the brightness) and detects the peak (maximum value) of the difference obtained by the comparison. ) exceeds a preset threshold value and the interval between the peaks is synchronized with the wiping cycle of the wiper device 18, the dimming process is set to be performed.

[0164] Next, the dimming process in the imaging device 120 will be described with reference to Fig. 20 and Fig. 21. Fig. 20 shows an outline of the dimming process setting executed in the imaging controller 122 in a flow chart, and Fig. 21 shows an outline of the change in brightness obtained from the captured image in an environment where the captured image of the camera 32 is dazzling. In Fig. 21, the horizontal axis represents time, and the vertical axis represents the brightness (average luminance) of the captured image.

[0165] As shown in FIG. 20, by executing steps 200 to 206 and steps 210 and 212, the imaging controller 122 proceeds to step 280 when it determines that the brightness around the vehicle 12 is equal to or lower than a predetermined brightness in both the image captured by the camera 32 and the automatic light device 24 and that the wiper device 18 is operated and a rainy state is occurring (positive determination in step 212).

[0166] In step 280, the imaging controller 122 calculates an operating average value of brightness when the wiper device 18 is in operation from the image captured by the camera 32, and compares the calculated operating average value with a non-operating average value calculated before the operation of the wiper device 18. In addition, in step 282, the imaging controller 122 obtains a period (anti-glare period) at which the difference between the non-operating average value and the operating average value is equal to or greater than a preset threshold value. At this time, the threshold value can be set based on the average brightness value (lower limit value) of the captured image at which dazzling is felt compared to the non-operating average value.

[0167] When the imaging controller 122 acquires the anti-glare cycle, it determines whether or not the anti-glare cycle is synchronized with the wiping cycle of the wiper 18A in step 284. At this time, if the anti-glare cycle and the wiping cycle are not synchronized, the imaging controller 122 makes a negative determination in step 284 and proceeds to step 208.

[0168] On the other hand, if the anti-glare cycle and the wiping cycle are synchronized, the imaging controller 122 makes an affirmative decision in step 284 and proceeds to step 286 to set the dimming process to be performed (set to dimming process).

[0169] Thereafter, the imaging controller 122 proceeds to step 208 when the determinations in steps 218 and 222 are both affirmative, or when the determination in step 284 is negative because the wiper device 18 has been stopped, for example.

[0170] As shown in FIG. 21, when raindrops or the like are attached to the windshield 16F at night, the wiper device 18 is operated. By doing The average brightness (operating average brightness) changes in synchronization with the wiping action of the wiper 18A, and the difference with the non-operating average brightness exceeds the threshold value. At this time, the remaining part of the wiper 18A forms a water film or the like, causing the operating average brightness to increase. Also, when the wiper 18A is not wiping the image, the wiper 18A may not be wiping the image. Reflected This causes wiper 18A to cast a shadow, lowering the operating average value.

[0171] Therefore, it is possible to determine from the change in the average brightness of the captured image whether or not the change in brightness of the captured image is caused by the operation of the wiper device 18. Furthermore, if the change in the average brightness of the captured image affects the operation of the wiper device 18, the period (anti-glare period) will be close to the wiping period of the wiper 18A.

[0172] From this, the imaging controller 122 performs a dimming process on the captured image when it is determined that the anti-glare cycle and the wiping cycle are synchronized. This makes it possible to prevent glare from occurring when the captured image is displayed, even if the wiper device 18 is operating.

[0173] In this way, in the imaging device 120, even if the wiper device 18 is operated when it is estimated that the environmental state is such that the captured image is perceived as being dazzling, it is possible to prevent the captured image from being perceived as being dazzling when viewed.

[0174] On the other hand, the vehicle 12A is equipped with a defogger device 126, and by operating this defogger device 126, it is possible to promote the defogging of the windshield 16F. Therefore, in the imaging device 120, the defogger device 126 is operated when performing the dimming process. This makes it possible to effectively suppress glare from occurring in the captured image.

[0175] In addition, the rear window 16R is often provided with a rear defogger. When capturing an image of the rear of the vehicle through the rear window 16R, activating the rear defogger in conjunction with the light reduction process can effectively reduce glare in the captured image.

[0176] In the above-described first to eighth embodiments, the vehicle has been described as an automobile such as the vehicle 12, 12A, etc. However, the vehicle may be any vehicle provided with a wiper device on a windshield, and may be a train such as an electric locomotive.

[0177] In the above-described first to eighth embodiments, the programs executed in the display controllers 30, 52, 62, 82, 92, 102 and the imaging controllers 108, 112, 122 may be pre-installed in a ROM or storage. These programs may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory, or may be downloaded from an external device via a network.

[0178] In addition, the present invention is not limited to the first to eighth embodiments, and various modifications can be made without departing from the spirit of the present invention. For example, the spirit of the present invention will not be lost even if all or part of the components of the display controllers 30, 52, 62, 82, 92, 102 and the imaging controllers 108, 112, 122 are integrated into the inner mirror 28 or the camera 32. [Explanation of symbols]

[0179] 10, 50, 60, 80, 90, 100, Electronic mirror system 16F Windshield (Windshield) 16R Rear glass (window glass) 18 Wiper device (environment detection means) 20 Rear wiper device 24 Auto light device (environment detection means) 26 Imaging device 28 Inner mirror (display means, image display device) 30, 52, 62, 82, 92, 102 Display controller (image display device, image processing device) 32 Camera (imaging means, environment detection means) 38, 70 Environment detection unit (environment detection means) 40, 54, 64, 114 Brightness adjustment unit (adjustment means) 68, 84, 116, 124 Adjustment switch 104, 110, 120 Imaging device 108, 112, 122 Imaging controller (image processing device)

Claims

1. a display means for displaying a display image corresponding to an image captured by the imaging means through the windshield of one side of the vehicle exterior; an environmental detection means for detecting an environmental condition capable of estimating whether water droplets are attached to the window glass and whether water droplets attached to the window glass affect the brightness of an image captured by the imaging means; an adjustment means for performing an adjustment so as to reduce the brightness of the display image when the environmental condition detected by the environmental detection means is estimated to include water droplets on the window glass and the water droplets on the window glass are estimated to affect the brightness of the display image, compared to when the environmental condition detected by the environmental detection means is estimated not to include water droplets on the window glass or the water droplets on the window glass are estimated not to affect the brightness of the display image; An image display device comprising:

2. The environment detection means includes a light / dark detection means for detecting brightness outside the vehicle, and a wiper device for wiping off water droplets adhering to the windshield when the wiper device is activated.

2. The image display device according to claim 1, wherein the adjustment means reduces the brightness of the displayed image when the brightness outside the vehicle is equal to or lower than a predetermined value and the wiper device is in operation.

3. 3. The image display device according to claim 1, wherein the adjustment means adjusts the brightness of the display image by suppressing a luminance of the display image when the display image is displayed on the display means.

4. The image display device according to claim 1 , wherein the adjustment means adjusts the brightness of the display means to thereby adjust the brightness of the displayed image.

5. 3. The image display device according to claim 1, wherein the adjustment means reduces the brightness of the image displayed on the display means by lowering the brightness of the image.

6. An imaging means for imaging one side of the outside of the vehicle through a windshield; an environmental detection means for detecting an environmental condition capable of estimating whether water droplets are attached to the window glass and whether water droplets attached to the window glass affect the brightness of an image captured by the imaging means; an adjustment means for making an adjustment when the environmental condition detected by the environmental detection means is estimated to include water droplets on the window glass and when the water droplets on the window glass are estimated to affect the brightness of the image captured by the imaging means, so as to reduce the brightness of the captured image compared to when the environmental condition detected by the environmental detection means is estimated to include no water droplets on the window glass or when the water droplets on the window glass are estimated to affect the brightness of the captured image; An imaging device comprising:

7. an environmental detection means for detecting an environmental condition capable of estimating whether water droplets are attached to a window glass and whether water droplets attached to the window glass affect the brightness of an image captured by an imaging means; A photographed image of one side of the outside of the vehicle taken through the windshield by the imaging means, an adjustment means for making an adjustment when the environmental condition detected by the environmental detection means is estimated to include water droplets on the window glass and when the water droplets on the window glass are estimated to affect the brightness of the captured image, so as to reduce the brightness of the captured image compared to when the environmental condition detected by the environmental detection means is estimated not to include water droplets on the window glass or not to affect the brightness of the captured image; 2. An image processing device comprising:

Images