Video Display System

Abstract

To provide "video display system" which assists quick cognition of a rear situation.SOLUTION: An image analysis unit 11 identifies the area of an image of a road and the area of an image of an automobile in the video captured with a back camera 3 and sets the area except the road area and the vehicle area as a peripheral landscape area. A cognition assisting processing unit 122 sets a tone weakening degree to be larger with reduction in a distance to another vehicle in the rear. An image processing unit 121 applies an image process of weakening the tone of the peripheral landscape area with increase in the tone weakening degree to the video captured with the back camera 3, and displays the video on a back monitor 7.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for displaying an image of the rear of a vehicle captured by a camera on a monitor. [Background technology]

[0002] A known technology for displaying an image of the rear of a vehicle captured by a camera on a monitor is electronic mirror technology, which displays an image captured by a camera capturing an image of the rear of the vehicle on a rear-view mirror-type monitor positioned above the front windshield (for example, Patent Document 1).

[0003] Also known is an electronic mirror technology that uses a camera to capture an image of the rear of a vehicle from the side and displays the captured image on a monitor installed inside the vehicle (for example, Patent Document 2). Also, in such electronic mirrors, a technology is known in which the area of ​​the image of the following vehicle in the image captured by the camera is cut out and enlarged, and then displayed in association with the image of the following vehicle in the image, at a position that overlaps with the area of ​​images of roadside features such as guardrails, the sky, or roadside strips in the image (for example, Patent Document 3). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-195301 A [Patent Document 2] JP 2011-213186 A [Patent Document 3] JP 2016-220011 A Summary of the Invention [Problem to be solved by the invention]

[0005] When checking the rear situation while driving a car, the driver needs to quickly recognize the rear situation in a short time so as not to lose attention to the front. However, it may be difficult to instantly recognize the situation behind the vehicle, for example, when the color of the vehicle behind is very similar to the color of the surrounding environment, or when the road on which the vehicle is traveling has many lanes.

[0006] On the other hand, with the above-mentioned technology of cutting out and enlarging the area of ​​the image of the following vehicle, the presence of the following vehicle itself can be quickly recognized, but because the display is unnatural and different from reality, it is difficult to intuitively grasp the relative positions of the following vehicle and the own vehicle.

[0007] Therefore, an object of the present invention is to support quick recognition of the rear situation in an image display system that displays an image of the rear of a vehicle captured by a camera on a monitor. [Means for solving the problem]

[0008] In order to achieve the above object, the present invention provides an image display system for displaying an image of the rear of an automobile, which comprises a camera for capturing an image of the rear of the automobile, an area identification unit for identifying an area of ​​the road image and an area of ​​the vehicle image in the image captured by the camera, and setting the area of ​​the image excluding the identified area of ​​the road image and the area of ​​the vehicle image as a surrounding scenery area, a monitor for displaying the image captured by the camera, and an image processing means for selectively applying image processing to the image displayed on the monitor to tone down the surrounding scenery area.

[0009] Here, such an image display system may be provided with a rear other vehicle detection means for detecting other vehicles behind the automobile or other vehicles within a predetermined distance behind the automobile, and the image processing means may be configured so that when a predetermined condition required for at least the rear other vehicle detection means not to detect another vehicle is satisfied, the image displayed on the monitor does not undergo image processing to weaken the tone of the surrounding scenery area, and when at least the rear other vehicle detection means detects another vehicle, the image processing means performs image processing to weaken the tone of the surrounding scenery area on the image displayed on the monitor.

[0010] In this case, the image display system may be configured so that, at least when the rear vehicle detection means detects another vehicle, the image processing means applies image processing to the image displayed on the monitor such that the tone of the surrounding scenery area is weakened the shorter the distance to the other vehicle detected by the rear vehicle detection means.

[0011] Alternatively, in this case, the image display system may be configured so that, in the image processing means, at least when the rear vehicle detection means detects another vehicle, the image displayed on the monitor is subjected to image processing that weakens the tone of the surrounding scenery area more the greater the approach speed of the other vehicle detected by the rear vehicle detection means.

[0012] In addition, the image display system equipped with these rear other vehicle detection means may be configured so that, in the image processing means, at least when the rear other vehicle detection means detects another vehicle, the shorter the distance to the other vehicle detected by the rear other vehicle detection means, the more intense the tone of the area of ​​the image of the other vehicle is.

[0013] Furthermore, this image display system may be configured to include a vehicle speed detection means for detecting the speed of the automobile, and in the image processing means, when a predetermined condition required for at least the vehicle speed detected by the vehicle speed detection means to be less than a predetermined speed is satisfied, image processing for weakening the tone of the surrounding scenery area is not applied to the image displayed on the monitor, and when at least the vehicle speed is equal to or greater than the predetermined speed, image processing for weakening the tone of the surrounding scenery area is applied to the image displayed on the monitor.

[0014] In this case, the image display system may be configured so that, when the vehicle speed detected by the vehicle speed detection means is equal to or greater than the specified speed, the image processing means applies image processing to the image displayed on the monitor such that the greater the vehicle speed, the more the tone of the surrounding scenery area is weakened.

[0015] Furthermore, this video display system may be provided with gaze detection means for detecting the gaze of the driver of the automobile, and gaze time monitoring means for detecting the duration of the driver's gaze at the monitor based on the gaze detection result of the gaze detection means, and the image processing means may be configured so that when a predetermined condition required for at least the duration detected by the gaze time monitoring means to be less than a predetermined time is satisfied, image processing for softening the tone of the surrounding scenery area is not applied to the image displayed on the monitor, and when at least the duration detected by the gaze time monitoring means is equal to or longer than the predetermined time, image processing for softening the tone of the surrounding scenery area is applied to the image displayed on the monitor.

[0016] In addition, this video display system may be configured so that, when a specified condition required for ensuring that at least the area of ​​the road image in the video captured by the camera and the surrounding area are not similar in color, the image processing means does not apply image processing to the image displayed on the monitor to weaken the tone of the surrounding scenery area, and when at least the area of ​​the road image in the video captured by the camera and the surrounding area are similar in color, the image processing means applies image processing to the image displayed on the monitor to weaken the tone of the surrounding scenery area.

[0017] According to the image display system as described above, when there is another vehicle to the rear that should be recognized, when it is difficult to see another vehicle to the rear, or when the driver is trying to check behind, etc., the image captured by the camera can be displayed on the monitor with the tone of the surrounding scenery area, which is the area other than the image of the vehicle and the road, weakened as necessary, so that the driver can quickly recognize the road and vehicle behind from the realistic and natural image displayed on the monitor without being distracted by the image portion of the surrounding scenery area. Effect of the Invention

[0018] As described above, according to the present invention, in an image display system that displays an image of the rear of a vehicle captured by a camera on a monitor, it is possible to assist the driver in quickly recognizing the situation behind the vehicle. [Brief description of the drawings]

[0019] [Figure 1] 1 is a block diagram showing a configuration of a video display system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing the arrangement of various cameras and monitors according to the embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram showing the arrangement of a radar and a driver camera according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing an example of analysis by an image analysis unit according to the embodiment of the present invention. [Diagram 5] 1 is a flowchart showing a cognitive assistance process according to an embodiment of the present invention. [Figure 6] FIG. 11 is a diagram illustrating a processing example of the cognitive assistance processing according to the embodiment of the present invention. [Figure 7] FIG. 11 is a diagram illustrating another example of the cognitive assistance processing according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, an embodiment of the present invention will be described. FIG. 1 shows the configuration of a video display system according to this embodiment. The video display system is a system installed in an automobile, and as shown in the figure, comprises a left side camera 1, a right side camera 2, a rear camera 3, a radar 4, a left side monitor 5, a right side monitor 6, a rear monitor 7, a driver camera 8, a DMS 9 (driver monitoring system 9), a status sensor 10, an image analysis unit 11, and a display control unit 12.

[0021] The display control unit 12 also includes an image processing unit 121 and a cognitive assistance processing unit 122 . In such a configuration of the image display system, as shown in Figures 2a1 and 2a2, the left side camera 1 is disposed on the left side of the vehicle and takes pictures of the rear of the vehicle from that position, the right side camera 2 is disposed on the right side of the vehicle and takes pictures of the rear of the vehicle from that position, and the backup camera 3 is disposed at the rear of the vehicle and takes pictures of the rear of the vehicle from that position.

[0022] The left side monitor 5, right side monitor 6, and rear monitor 7 are displays that display images and are arranged in positions that are visible to the driver inside the vehicle. For example, as shown in Fig. 2b, the left side monitor 5 is arranged to be located to the left of the center of the driver's seat in the left-right direction, and the right side monitor 6 is arranged to be located to the right of the center of the driver's seat in the left-right direction. The rear monitor 7 is arranged in the center in the left-right direction above the front windshield.

[0023] Next, the radar 4 is disposed at the rear of the vehicle as shown in FIG. 3a, and detects the positions of various objects, such as other vehicles, behind the vehicle. Moreover, the driver camera 8 is, for example, a near-infrared camera arranged diagonally above and in front of the driver's seat as shown in FIG. 3b, and captures an image of the head of the driver seated in the driver's seat from in front of the driver.

[0024] The DMS 9 then recognizes the position and direction of the driver's eyes from the image captured by the driver camera 8 to detect the driver's line of sight. Next, the state sensor 10 is a sensor that detects various states of the automobile, such as the vehicle speed, steering angle, acceleration, and angular acceleration. The image analysis unit 11 performs image analysis on the images captured by the left side camera 1, the right side camera 2, and the rear camera 3 to identify the road area, which is the area of ​​the image of the road in the images, and the vehicle area, which is the area of ​​the images of the vehicle and other vehicles in the images, sets the area other than the road area and vehicle area in the images as the surrounding scenery area, and notifies the image processing unit 121 of the display control unit 12.

[0025] That is, taking an image captured by the rear camera 3 as an example, as shown in Fig. 4, the image analysis unit 11 performs image recognition such as white line recognition and boundary recognition on the image captured by the rear camera 3 shown in Fig. 4a, and identifies an area R of the road image as a road area as shown in Fig. 4b. Also, image recognition such as pattern matching is performed on the image captured by the rear camera 3 shown in Fig. 4a, and identifies an area V of the automobile image as a vehicle area as shown in Fig. 4c. Note that the vehicle area may be identified by taking into consideration the position of another vehicle detected by the radar 4 in addition to the image recognition.

[0026] Then, as shown in FIG. 4d, an area P excluding the identified road area R and vehicle area V in the image captured by the backup camera 3 is set as a surrounding scenery area, and this is notified to the image processing unit 121 of the display control unit 12.

[0027] The cognitive assistance processing unit 122 of the display control unit 12 performs the cognitive assistance processing shown in FIG. As shown in the figure, in the cognitive assistance processing, the cognitive assistance processing unit 122 acquires a situation (step 502), calculates an assistance level according to the acquired situation (step 504), and sets the dimming level of the surrounding scenery area in the image processing unit 121 according to the calculated assistance level (step 506), repeating the process.

[0028] Here, in step 502, the vehicle speed detected by the status sensor 10, the distance to other vehicles behind detected by the radar 4, and the time that the driver continues to look at any of the left side monitor 5, right side monitor 6, and rear monitor 7, as indicated by the history of gaze detection results of the DMS 9, are obtained.

[0029] Then, in step 504, the temporary support level 1 is calculated to be higher as the vehicle speed increases, the temporary support level 2 is calculated to be higher as the distance to the other vehicle behind decreases, and the temporary support level 3 is calculated to be higher as the monitor gaze continues for a longer period of time. However, the temporary support level 1 is set to 0 when the vehicle speed is less than a predetermined speed. The temporary support level 2 is set to 0 when the distance to the other vehicle behind is equal to or greater than a predetermined distance, or when no other vehicle behind is detected. The temporary support level 3 is set to 0 when the gaze continues for less than a predetermined period of time.

[0030] Then, the highest level among the provisional support level 1, the provisional support level 2, and the provisional support level 3 is calculated as the support level. In step 506, the degree of weak adjustment is calculated so that the degree of weak adjustment increases as the calculated support level increases, and is set in the image processing unit 121. Note that the minimum value of the degree of weak adjustment is set to 0, and when the support level is 0, the degree of weak adjustment is set to 0.

[0031] Returning to Figure 1, the image processing unit 121 of the display control unit 12 applies image processing to the image captured by the left side camera 1 to generate an image showing the left rear of the vehicle as a left side image to display on the left side monitor 5, applies image processing to the image captured by the right side camera 2 to generate an image showing the right rear of the vehicle as a right side image to display on the right side monitor 6, and applies image processing to the image captured by the rear camera 3 to generate an image showing the rear of the vehicle as a rear image to display on the rear monitor 7.

[0032] Here, the image processing unit 121 performs image processing on each video image to weaken the tone of the surrounding scenery area in the video image according to the weakening level set by the cognitive assistance processing unit 122. In this image processing, the greater the weakening level, the more the tone of the surrounding scenery area is weakened. Also, when the weakening level is 0, the tone of the surrounding scenery area in the video image is not weakened and remains as it is.

[0033] The image processing for weakening the tone of the surrounding scenery region is image processing for increasing the transparency of the surrounding scenery region as the weak tone increases, but other image processing can also be used for weakening the tone of the surrounding scenery region, such as image processing for decreasing the saturation of the surrounding scenery region as the weak tone increases, or image processing for decreasing the contrast of the surrounding scenery region as the weak tone increases.

[0034] As shown in Figs. 6a-d, taking the example of a rear view image generated from an image captured by the rear camera 3 and displayed on the rear monitor 7, such image processing by the image processing unit 121 causes the tone of the image portion of the surrounding scenery area to become weaker as the dimming level set by the recognition assistance processing unit 122 increases, making it possible to recognize the road surface and the vehicle without being distracted by the image portion of the surrounding scenery area.

[0035] Therefore, the dimming level is set to be greater as the assistance level is higher, so the higher the vehicle speed, the shorter the distance to other vehicles behind, and the longer the driver is staring at the monitor, the more the image portion of the surrounding scenery area will be dimmed, allowing the driver to recognize the road surface and vehicle in a shorter time.

[0036] The embodiment of the present invention has been described above. Here, the above embodiment may be modified in various ways. For example, the recognition assistance process shown in Fig. 5 may be such that when the radar 4 does not detect another vehicle within a specified distance behind, the assistance level is unconditionally set to 0, and when another vehicle is present within the specified distance behind, the assistance level is increased as the distance to the other vehicle decreases, and when there is no other vehicle within the specified distance behind, the image portion of the surrounding scenery area is displayed as is without dimming the tone. By doing so, when there is no other vehicle within the specified distance behind that the driver needs to immediately recognize, the driver can confirm the rearward state, including the scenery around the road, from the display on each monitor in the same way as if he or she were looking at it with his or her eyes.

[0037] Alternatively, the recognition support process may be such that, when the radar 4 has not detected another vehicle within a predetermined distance behind and the vehicle speed is equal to or lower than a predetermined speed, the support level is unconditionally set to 0, and when the radar 4 has not detected another vehicle within a predetermined distance behind and the vehicle speed is higher than a predetermined speed, the support level is increased as the vehicle speed increases, and when there is another vehicle within a predetermined distance behind, the support level is increased as the distance to the other vehicle decreases, and when there is no other vehicle behind, the tone of the image portion of the surrounding scenery area may be reduced only when the vehicle is traveling at high speed. This eliminates the annoyance of periodic structures, poles, lighting at night or in a tunnel, and other objects moving within the display of each monitor, making it easier to notice when another vehicle appears behind.

[0038] In these cases, an indicator may be provided to inform the driver of the detection of a rear vehicle, and the indicator may be turned on while the radar 4 detects a vehicle within a specified distance behind. In a vehicle equipped with left and right door mirrors, indicators may be provided in the left and right door mirrors, respectively, and the indicator in the left door mirror may be turned on when a vehicle is detected to the left rear, and the indicator in the right door mirror may be turned on when a vehicle is detected to the right rear.

[0039] Alternatively, the recognition assistance process may be such that, when the approaching speed of the other vehicle detected by the radar 4 from behind is high, the assistance level is increased as the approaching speed increases, and the surrounding scenery area is rapidly weakened when the other vehicle approaches from behind at high speed. By doing so, it is possible to prevent the driver from being confused by the sudden appearance of the other vehicle due to a delay in discovering the other vehicle from the display of each monitor.

[0040] Alternatively, the recognition assistance process may be such that, when another vehicle is detected behind the vehicle, the assistance level is increased if the color of the vehicle area in the images captured by each camera is similar to the color of the surrounding area of ​​the vehicle area, or the vehicle area corresponding to the image of the other vehicle is more emphasized as the distance to the other vehicle behind the vehicle detected by radar 4 becomes shorter, for example by increasing the saturation.

[0041] That is, for example, taking the example of a left side image generated from an image captured by the left side camera 1 and displayed on the left side monitor 5, as shown in FIG. 7, when the images of FIG. 7a1, a2, and a3 are captured sequentially in that order by the left side camera 1, there is an image of a dark-colored other vehicle in the shade in the captured image of FIG. 7a1, which is similar to the color of the vehicle area of ​​the image of the other vehicle and the area around the vehicle area, so the assistance level is increased and the image processing unit 121 is caused to generate the left side image of FIG. 7b1 in which the tone of the surrounding scenery area is weakened to make the other vehicle easier to see, and to display it on the left side monitor 5.

[0042] Then, if other vehicles approach successively as shown in the captured images of Figures 7a2 and a3, the emphasis degree of the vehicle area of ​​the other vehicles is increased, and the image processing unit 121 is caused to generate a left side image in which the image of the approaching other vehicles is successively emphasized more as shown in Figures 7b2 and b3, and to display this on the left side monitor 5.

[0043] In the above embodiment, the radar 4 is used to detect the position and speed of other vehicles behind the vehicle. However, the image analysis unit 11 may detect the position and speed of other vehicles by analyzing the position and movement of the images of other vehicles in the images captured by each camera. [Explanation of symbols]

[0044] 1...left side camera, 2...right side camera, 3...back camera, 4...radar, 5...left side monitor, 6...right side monitor, 7...back monitor, 8...driver camera, 9...DMS, 10...status sensor, 11...image analysis unit, 12...display control unit, 121...image processing unit, 122...cognitive assistance processing unit.

Claims

1. An image display system for displaying an image behind a vehicle, A camera that captures images of the rear of the vehicle; an area identification unit that identifies an area of ​​a road image and an area of ​​a vehicle image in the video captured by the camera, and sets an area of ​​the video excluding the identified area of ​​the road image and the area of ​​the vehicle image as a surrounding scenery area; a monitor for displaying an image captured by the camera; an image processing means for selectively performing a predetermined image processing of the surrounding scenery area on the image displayed on the monitor; A video display system characterized in that the specified image processing of the surrounding scenery area is either image processing that increases the transparency of the surrounding scenery area, image processing that decreases the saturation of the surrounding scenery area, or image processing that decreases the contrast of the surrounding scenery area.

2. 2. The image display system according to claim 1, a rear vehicle detection means for detecting a rear vehicle behind the vehicle or a rear vehicle within a predetermined distance behind the vehicle; the image processing means does not apply the specified image processing of the surrounding scenery area to the image displayed on the monitor when at least a specified condition required for the rear other vehicle detection means to not detect another vehicle is satisfied, and applies the specified image processing of the surrounding scenery area to the image displayed on the monitor at least when the rear other vehicle detection means detects another vehicle.

3. 3. The image display system according to claim 2, the image processing means controls a characteristic of a predetermined image processing of the surrounding scenery area in accordance with a distance to the other vehicle detected by the rear other vehicle detection means at least when the rear other vehicle detection means detects the other vehicle; In the control, the image processing means When the predetermined image processing of the surrounding scenery area is image processing for increasing the transparency of the surrounding scenery area, the characteristic is controlled so that the transparency of the surrounding scenery area increases as the distance to the other vehicle decreases; When the predetermined image processing of the surrounding scenery area is image processing for lowering the saturation of the surrounding scenery area, the characteristic is controlled so that the saturation of the surrounding scenery area becomes lower as the distance to the other vehicle becomes shorter; When the specified image processing of the surrounding scenery area is image processing for lowering the contrast of the surrounding scenery area, the image display system controls the characteristics so that the contrast of the surrounding scenery area becomes lower as the distance to the other vehicle becomes shorter.

4. 3. The image display system according to claim 2, the image processing means controls a characteristic of a predetermined image processing of the surrounding scenery area in accordance with an approaching speed of the other vehicle detected by the rear other vehicle detection means at least when the rear other vehicle detection means detects the other vehicle; In the control, the image processing means when the predetermined image processing of the surrounding scenery area is image processing for increasing the transparency of the surrounding scenery area, the characteristic is controlled so that the transparency of the surrounding scenery area increases as the approaching speed of the other vehicle increases; When the predetermined image processing of the surrounding scenery area is image processing for lowering the saturation of the surrounding scenery area, the characteristic is controlled so that the saturation of the surrounding scenery area becomes lower as the approaching speed of the other vehicle increases. a control unit that controls the characteristics so that the contrast of the surrounding scenery area becomes lower as the approaching speed of the other vehicle increases, when the specified image processing of the surrounding scenery area is image processing for lowering the contrast of the surrounding scenery area.

5. 5. The image display system according to claim 2, 3 or 4, The image display system is characterized in that the image processing means performs image processing such that, at least when the rear other vehicle detection means detects another vehicle, the area of ​​the image of the other vehicle is more strongly emphasized the shorter the distance to the other vehicle detected by the rear other vehicle detection means.

6. 2. The image display system according to claim 1, a vehicle speed detection means for detecting a vehicle speed of the vehicle; The image processing means does not apply the specified image processing of the surrounding scenery area to the image displayed on the monitor when at least a specified condition required for the vehicle speed detected by the vehicle speed detection means to be less than a specified speed is satisfied, and applies the specified image processing of the surrounding scenery area to the image displayed on the monitor when at least the vehicle speed is equal to or greater than the specified speed.

7. 7. The image display system according to claim 6, the image processing means controls a characteristic of a predetermined image processing of the surrounding scenery area in response to the vehicle speed detected by the vehicle speed detection means when the vehicle speed is equal to or higher than the predetermined speed; In the control, the image processing means when the predetermined image processing of the surrounding scenery area is image processing for increasing the transparency of the surrounding scenery area, the characteristic is controlled so that the transparency of the surrounding scenery area increases as the vehicle speed detected by the vehicle speed detection means increases; when the predetermined image processing of the surrounding scenery area is image processing for lowering the saturation of the surrounding scenery area, the characteristic is controlled so that the saturation of the surrounding scenery area becomes lower as the vehicle speed detected by the vehicle speed detection means increases; A video display system characterized in that, when the specified image processing of the surrounding scenery area is image processing to lower the contrast of the surrounding scenery area, the characteristics are controlled so that the contrast of the surrounding scenery area becomes lower as the vehicle speed detected by the vehicle speed detection means increases.

8. 2. The image display system according to claim 1, A gaze detection means for detecting a gaze of a driver of the vehicle; a gaze time monitoring means for detecting a duration of the driver's gaze on the monitor based on a gaze detection result of the gaze detection means, The image processing means does not apply the specified image processing of the surrounding scenery area to the image displayed on the monitor when a specified condition required for at least the duration detected by the gaze time monitoring means to be less than a specified time is satisfied, and applies the specified image processing of the surrounding scenery area to the image displayed on the monitor when at least the duration detected by the gaze time monitoring means is equal to or longer than the specified time.

9. 2. The image display system according to claim 1, The image processing means is characterized in that, when a predetermined condition required for ensuring that the color of at least the area of ​​the road image in the image captured by the camera and the surrounding area are not similar, the image processing means does not apply a predetermined image processing of the surrounding scenery area to the image displayed on the monitor, and when the color of at least the area of ​​the road image in the image captured by the camera and the surrounding area are similar, the image processing means applies a predetermined image processing of the surrounding scenery area to the image displayed on the monitor.

Images