Display control device

Abstract

To provide a display control device in which a shield region image (transparent view) is not displayed when there is no need to check the state of the shield region.SOLUTION: A display control device 100 comprises an obstacle identification unit 10 that identifies an obstacle region R' that includes an obstacle S, a driver information acquisition unit 30 that acquires driver information regarding the visual field of a driver, a shield region identification unit 40 that identifies a shield region N' by a right-side A pillar 152 that shields the visual field of the driver, a shield area ratio calculation unit 50 that calculates the area ratio M1 / M0 of the obstacle region R' that is shielded by the shield region N', and a display control unit 60 that controls the image to be shown to a display R142 that is provided to the right-side A pillar 152. When the shield area ratio M1 / M0 is less than a prescribed value, the display control unit 60 does not show a transparent view P2 to the display R142, which is an image within the shield region N' from a driver viewpoint image P1.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a display control device. [Background technology]

[0002] 2. Description of the Related Art Image display devices are known that use external recognition sensors such as in-vehicle cameras and LiDAR (Light Detection and Ranging) to display images of areas that are blind spots (obstructed areas) for the driver on a display unit.

[0003] In this image display device, a display unit that displays an image of the obstructed area is provided on a vehicle obstacle that creates the obstructed area, such as an A-pillar, and blocks part of the field of view in front of or to the side of the vehicle in the direction of travel. An image display device has been proposed in which this display unit displays an image of the obstructed area that is expected to be visible to the driver if there were no vehicle obstacle (see, for example, Patent Documents 1 and 2).

[0004] Such an image display device converts an image acquired by an external environment recognition sensor installed outside the vehicle cabin into an image seen from the driver's viewpoint, and displays the converted image on a display unit installed on the vehicle body obstacle. Therefore, this image display device can show the driver an image of the blocked area as if it had passed through the vehicle body obstacle. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2017 / 130439 [Patent Document 2] JP 2017-181586 A Summary of the Invention [Problem to be solved by the invention]

[0006] A driver of a vehicle equipped with the above-mentioned image display device sees areas other than obstacles on the vehicle body with his / her own naked eyes through the windshield (front glass), and sees the image displayed on the display unit for the obstacle area on the vehicle body where the view is blocked. In this case, if there is a difference (position deviation, color deviation, brightness deviation, etc.) between the image of the area seen by the naked eye and the image displayed on the display unit, the difference may give the driver a sense of incongruity.

[0007] Even if the driver does not feel uncomfortable, the driver may feel annoyed if the display unit frequently displays images.

[0008] The present invention has been developed in consideration of the above-mentioned circumstances, and aims to provide a display control device that displays an image of a hidden area when the driver cannot fully see objects near the hidden area, and does not display an image of the hidden area when the driver can fully see objects near the hidden area. [Means for solving the problem]

[0009] The present invention is a display control device that controls a display unit provided on a structure of a vehicle body that obstructs the field of vision of a driver of a vehicle, and includes an obstacle identification unit that identifies an obstacle area containing an obstacle in an image acquired from an imaging unit that captures images of the area around the vehicle, a driver information acquisition unit that acquires driver information related to the field of vision of the driver of the vehicle, an obstructed area identification unit that identifies an obstructed area in which the structure obstructs the driver's field of vision based on the driver information, an obstructed area ratio calculation unit that calculates an obstructed area ratio, which is the ratio of an area of ​​the obstacle area that overlaps with the obstructed area to a total area of ​​the obstacle area, and a display control unit that controls an image to be displayed on the display unit, wherein when the obstructed area ratio is equal to or greater than a predetermined value, the display control unit displays on the display unit a driver's viewpoint image obtained by converting the viewpoint of the image captured by the imaging unit based on the driver information, and when the obstructed area ratio is less than the predetermined value, the display control device does not display the driver's viewpoint image on the display unit. Effect of the Invention

[0010] The display control device of the present invention displays an image of a hidden area when the driver cannot fully see objects near the hidden area, and does not display an image of the hidden area when the driver can fully see objects near the hidden area. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a configuration of a display control device according to an embodiment of the present invention. [Diagram 2] 1 is an example of the range of an image P0 captured by a camera R. [Diagram 3] FIG. 2 is a schematic diagram showing an example of the installation position of a driver photographing camera. [Figure 4] This is an example of a driver's viewpoint image in which it is assumed that the subject corresponding to the image P0 captured by the camera R will be visually recognized by the driver's naked eye at the driver's eye position. [Diagram 5] FIG. 2 is a diagram showing an example of an image p1 that is actually viewed by the driver's naked eye. [Figure 6] FIG. 2 is a diagram showing a blocked area N' corresponding to a blocked range N by a right A-pillar in an image p1 visually recognized by the driver's naked eye in a driver's viewpoint image P1. [Figure 7] 13 is a schematic diagram showing a state in which an interior view P5 is displayed on a display R. FIG. [Figure 8] 13 is a schematic diagram showing a state in which the display R is in a blank state P6. [Figure 9] 13 is a schematic diagram showing a state in which an interior view P5 and a warning display P7 are displayed on a display R. FIG. [Figure 10] FIG. 13 is a schematic diagram showing a state in which no display P6 is displayed on the display R and a warning display P7 is displayed. [Figure 11] 1 is a schematic diagram showing a state in which a transparent view P2, which is an image within a blocked area N' of a driver's viewpoint image P1, and a warning display P7 are displayed on a display R. FIG. [Figure 12]11 is a flowchart illustrating a flow of operations of the display control device. [Figure 13] This is a schematic diagram showing a state in which only the left edge of the display R corresponding to the side where the obstacle S is blocked is illuminated as a warning display P7. [Figure 14] 13 is a schematic diagram showing a state in which only the right edge of the display R corresponding to the side where the obstacle S is blocked is illuminated as a warning display P7. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] An exemplary embodiment of a display control device according to the present invention will be described in detail below with reference to the drawings. Fig. 1 is a block diagram showing the configuration of a display control device 100 according to an embodiment of the present invention, Fig. 2 is a schematic diagram showing an example of the range of an image P0 captured by a camera R112, and Fig. 3 is a schematic diagram showing an example of the installation position of a driver-capturing camera 130.

[0013] <Configuration> 1, the vehicle 200 (hereinafter referred to as the vehicle 200) includes an external environment recognition sensor 110 (an example of an imaging unit) that detects the surroundings of the vehicle 200, a vehicle information acquisition unit 120, a driver photographing camera 130, a display 140 (an example of a display unit), and a display control device 100. The external environment recognition sensor 110 may be, for example, a camera L (left) 111, a camera R (right) 112, a sonar 113, a radar 114, a LiDAR 115, or the like, but may be a sensor other than these.

[0014] 2, the camera R112 is provided on the front right side outside the cabin of the vehicle 200 and captures an image P0 of a subject in the range in front of and to the right of the vehicle 200. The camera L111 is provided on the front left side outside the cabin of the vehicle 200 and captures an image P0 of a subject in the range in front of and to the left of the vehicle 200.

[0015] The installation locations of the camera L111 and the camera R112 are not limited to the above-mentioned locations, and the camera may be installed at any location of the vehicle 200 as long as the imaging range can be the forward range and the lateral range of the vehicle 200 that can be seen from the driver's seat of the vehicle 200. As the camera, instead of the camera L111 and the camera R112, a single camera with a wide angle of view having the functions of the camera L111 and the camera R112 may be applied.

[0016] The installation locations of the cameras L111 and R112 are also sensitive to infrared light so that they can capture surrounding subjects even in dark conditions such as at night. The cameras L111 and R112 may each be equipped with an infrared light that irradiates infrared light toward the surroundings.

[0017] A plurality of sonars 113 are provided distributed around the entire perimeter outside the vehicle cabin, and detect obstacles present within a relatively short distance range around the vehicle 200. Radars 114 are provided at the front and rear of the vehicle 200, respectively, and detect obstacles up to a relatively long distance range in front of the vehicle 200 and obstacles up to a relatively long distance range behind the vehicle. LiDARs 115 are provided at the front and rear of the vehicle 200, respectively, and detect obstacles up to a distance range intermediate between those of the sonars 113 and the radars 114 over a wide range in front of and behind the vehicle 200.

[0018] The vehicle information acquisition unit 120 acquires the operating conditions of the vehicle 200 such as the speed, steering angle, etc. detected by various sensors installed in the vehicle 200 .

[0019] As shown in FIG. 3, the driver shooting camera 130 is provided, for example, on a roof part 190 near a rearview mirror 180 (a so-called room mirror for observing the rear of the vehicle 200) in the vehicle interior, facing the driver sitting in the driver's seat. The driver shooting camera 130 shoots the face of the driver sitting in the driver's seat. The driver shooting camera 130 has an angle of view that can shoot the driver's face regardless of the driver's sitting height. The driver shooting camera 130 is also sensitive to infrared rays so that the driver's face can be shot even in a dark situation such as at night. The driver shooting camera 130 may be equipped with an infrared light that irradiates infrared rays toward the driver.

[0020] Display 140 has display L (left) 141 and display R (right) 142. Display L 141 is provided on the surface of left A-pillar 151 facing the interior of the vehicle, and display R 142 is provided on the surface of right A-pillar 152 facing the interior of the vehicle. For example, a liquid crystal display can be used as display 140, and in this case, display 140 is attached to left A-pillar 151 or right A-pillar 152. When display 140 is formed in the shape of a film using, for example, organic EL or the like, it is disposed along the surface of left A-pillar 151 and the surface of right A-pillar 152, respectively.

[0021] The A-pillars are pillar members that are provided on the left and right edges of the windshield (front glass) of the vehicle body and support a loop panel of the vehicle body. The positions of the left A-pillar 151 and the right A-pillar 152 in the vehicle 200 are each stored in advance in the blocked area identification unit 40.

[0022] The display control device 100 of this embodiment controls a display L141 provided on a left A-pillar 151 and a display R142 provided on a right A-pillar 152, which are an example of a vehicle body structure that obstructs the field of view of the driver of a vehicle 200. As shown in FIG. 1 , the display control device 100 includes an obstacle identification unit 10, a collision possibility determination unit 20, a driver information acquisition unit 30, a blocked area identification unit 40, a blocked area ratio calculation unit 50, and a display control unit 60.

[0023] The obstacle identification unit 10 has an obstacle detection unit 11 and an obstacle tracking unit 12. The obstacle detection unit 11 detects an obstacle S detected around the vehicle 200 and its size based on an image P0 captured and acquired by a camera L111 and a camera R112 that capture images of the surroundings, and information such as distance detected by a sonar 113, a radar 114, and a LiDAR 115.

[0024] Obstacles that can be obtained based on image P0 include planar objects such as lines drawn on the road, crosswalks, and arrows, but by combining this with information obtained from sonar 113, radar 114, and LiDAR 115, the obstacles S detected by the obstacle detection unit 11 are narrowed down to three-dimensional objects that may collide with the vehicle 200.

[0025] The detected obstacles S include moving objects such as pedestrians, bicycles, wheelchairs, small mobility vehicles (e.g., kick scooters), motorbikes, and animals, as well as stationary objects such as traffic lights, guardrails, telephone poles, curbs, cones (pylons), and poles.

[0026] The obstacle tracking unit 12 distinguishes whether each obstacle S detected by the obstacle detection unit 11 is a moving object or a stationary object based on the relative positional relationship with the vehicle 200 according to the state of the vehicle 200 (state of moving, stopped, etc.) acquired by the vehicle information acquisition unit 120. Furthermore, when the obstacle S is a moving object, the obstacle tracking unit 12 determines the moving direction and moving speed of the moving object.

[0027] In addition, the obstacle tracking unit 12 also predicts the subsequent movement status of the obstacle S (whether it will continue to move at a constant speed, accelerate, decelerate, or stop, etc.) based on the movement direction and movement speed of the obstacle S over time.

[0028] Then, the obstacle identification unit 10 identifies each obstacle S obtained by the obstacle tracking unit 12 in the image P0. For example, the obstacle identification unit 10 surrounds the obstacle S with a rectangular frame K that is set to be slightly larger than the circumscribing area of ​​the obstacle S, and identifies the obstacle S as an obstacle region R that is the region surrounded by the rectangular frame K. In addition, the frame surrounding the obstacle S may not be rectangular, but may be a frame that has a shape that outlines the contour of the obstacle S.

[0029] The driver information acquisition unit 30 acquires the eye position of the driver as an example of information regarding the field of view of the driver sitting in the driver's seat, based on the face of the driver captured by the driver capturing camera 130. The information acquired by the driver information acquisition unit 30 only needs to be information that can calculate the direction in which the driver is looking, and can further acquire information regarding the position of the driver's head, for example.

[0030] The blocked area identifying unit 40 identifies a blocked area N, which is a range in which the driver's field of vision is blocked by the A-pillars 151, 152. More specifically, the blocked area identifying unit 40 identifies the blocked area N (see FIG. 2), which is a range in which the driver's field of vision is blocked by the right A-pillar 152, within the field of vision in front and on the right side of the vehicle 200 that can be seen from the driver's eye position, based on the relative positional relationship between the driver's eye position acquired by the driver information acquiring unit 30 and the position at which the right A-pillar 152 is installed in the vehicle 200 that is stored in advance.

[0031] Furthermore, the blocked region identifying unit 40 performs viewpoint conversion on the image P0 captured by the camera R112 into an image having the eye position of the driver detected by the driver information acquiring unit 30 as the viewpoint (driver viewpoint image P1).

[0032] Fig. 4 shows an example in which image P0 captured by camera R112 is converted into driver's viewpoint image P1 based on the driver's eye position. As shown in Fig. 4, this driver's viewpoint image P1 includes the subject in image P0 captured by camera R112, and is an image that predicts the field of view as seen by the driver's naked eye at the driver's eye position.

[0033] 5 is a diagram showing an example of an image p1 actually seen by the driver's naked eye. In the image p1 actually seen by the driver's naked eye, the driver cannot see a subject present behind the blocked area N blocked by the right A-pillar 152.

[0034] In contrast, the driver's viewpoint image P1 is an image obtained by converting the viewpoint of the image P0 captured by the camera R112 installed outside the vehicle body structure such as the right A-pillar 152 based on information such as the driver's eye position, and therefore the driver's viewpoint image P1 also includes a subject that exists behind the obstructed area N.

[0035] Then, the blocked area identifying unit 40 identifies, in the driver's viewpoint image P1, a blocked area N' that corresponds to the blocked range N by the right A-pillar 152 in the image p1 visually recognized by the driver's naked eye, as shown in FIG.

[0036] Here, the obstacle identification unit 10 identifies the obstacle S identified in the image P0 as an obstacle region R' in the driver's viewpoint image P1. For example, the obstacle identification unit 10 reflects the rectangular frame K set in the image P0 before the viewpoint conversion in the driver's viewpoint image P1. In this case, the obstacle identification unit 10 may deform the rectangular frame K to a rectangular frame K' set slightly larger than the rectangular frame K' that circumscribes the obstacle S, and identify the region surrounded by the rectangular frame K' as the obstacle region R'. The inside of the region surrounded by the rectangular frame K, the rectangular frame K', or a frame that outlines the contour of the obstacle S is the obstacle region R' in the driver's viewpoint image P1.

[0037] Then, the shielded region identification unit 40 judges whether the obstacle region R' overlaps with the shielded region N'. When either a part of the obstacle region R' falls within the shielded range N blocked by the right A-pillar 152 in the image p1 (FIG. 5) seen with the naked eye according to the driver's eye position, and the part of the obstacle region R' is invisible to the driver, or the whole of the obstacle region R' falls within the shielded range N blocked by the right A-pillar 152, and the whole of the obstacle region R' is invisible to the driver, the shielded region identification unit 40 judges that the obstacle region R' overlaps with the shielded region N'.

[0038] On the other hand, when not even a part of the obstacle area R' is included in the blocked range N blocked by the right A-pillar 152 in the image p1 seen by the naked eye at the driver's eye position, it is determined that the obstacle area R' does not overlap the blocked area N'. This means that the obstacle area R' is not blocked by the right A-pillar 152 and the driver can fully confirm the entire obstacle area R'.

[0039] The shielded area ratio calculation unit 50 calculates a shielded area ratio (=M1 / M0) [%], which is the ratio of an area M1 of the obstacle area R' that is blocked from the driver's field of view by overlapping with the shielded area N' to the entire area M0 of the obstacle area R'. At this time, the shielded area ratio calculation unit 50 calculates the shielded area ratio for the area that the shielded area identification unit 40 determines as the obstacle area R' overlapping with the shielded area N'.

[0040] For example, the total area M0 of the obstacle region R' in the driver's viewpoint image P1 and the area M1 of the range where each obstacle region R' overlaps with the blocked region N' are calculated. Then, the blocked area ratio (=M1 / M0) [%] is calculated, which is the ratio of the area M1 of the range blocked by overlapping with the blocked region N' to the total area M0 of the obstacle region R'.

[0041] When a plurality of obstacle regions R' exist in the driver's viewpoint image P1, the blocked area ratio is calculated for each obstacle region R'.

[0042] The collision possibility determination unit 20 determines the possibility of a collision between the vehicle 200 and an obstacle S based on the state of the vehicle 200 (e.g., speed, steering angle, and operation state of the turn signal) acquired by the vehicle information acquisition unit 120 and each obstacle S identified by the obstacle identification unit 10.

[0043] In other words, if the future position of the vehicle 200 predicted based on the operating state of the vehicle 200 after a predetermined time has elapsed overlaps with the future position of each obstacle region R predicted based on the state (movement direction and movement speed) of each obstacle region R identified by the obstacle identification unit 10, the collision possibility determination unit 20 determines that there is a possibility of a collision between the vehicle 200 and the obstacle S after the predetermined time has elapsed.

[0044] On the other hand, if the future position of vehicle 200 predicted based on the operating state of vehicle 200, such as the speed and steering angle of vehicle 200, after a predetermined time has elapsed does not overlap with the future position of obstacle area R predicted based on the state of obstacle area R identified by obstacle identification unit 10, the collision possibility determination unit 20 determines that there is no possibility of collision between vehicle 200 and obstacle S after a predetermined time has elapsed.

[0045] Based on the state of vehicle 200 acquired by vehicle information acquisition unit 120, collision possibility determination unit 20 predicts the position of vehicle 200 after a predetermined time has elapsed, assuming that the state will continue (for example, if it is operating at a constant speed and a constant steering angle, the constant speed and constant steering angle will continue even after a predetermined time has elapsed, or, for example, if it is operating at a steering angle with a constant acceleration and constant angular acceleration, the constant acceleration and constant angular acceleration will continue even after a predetermined time has elapsed).

[0046] 2, in a situation where vehicle 200 stops temporarily in a right-turn lane in an intersection with the right direction indicator for turning right flashing, and obstacle S is moving across a crosswalk on the right side of the intersection, the future position predicted by collision possibility determination unit 20 on the assumption that the vehicle speed of vehicle 200 is 0 [km / h] will be the same as the current position. Therefore, collision possibility determination unit 20 will determine that there is no possibility of collision with obstacle S moving across the crosswalk on the right side of the intersection.

[0047] However, vehicle 200 stopped in the right-turn lane in an intersection may start moving from a stopped state (vehicle speed 0 [km / h]) and turn right if there is no traffic of other vehicles traveling in the oncoming lane or if it is determined that there is no obstacle S on the crosswalk on the right side of the intersection.

[0048] Therefore, the collision possibility determination unit 20 predicts the future position of the vehicle 200 not simply on the assumption that the state of the vehicle 200 will continue, but also takes into account vehicle information such as the operating state of the turn signal (whether the left or right turn signal light is flashing or off), predicts the future position of the vehicle 200, and determines whether there is a possibility of a collision between the vehicle 200 and the obstacle S after a predetermined time has elapsed.

[0049] The collision possibility determination unit 20 can determine whether or not there is a collision possibility by combining map information acquired by a navigation device (not shown) with vehicle information.

[0050] Fig. 7 is a schematic diagram showing a state in which an interior view P5 is displayed on the display R142, and Fig. 8 is a schematic diagram showing a state in which the display R142 is set to a blank state P6. The display control unit 60 performs control to switch the display contents, such as images, displayed on the display R142 provided over the right A-pillar 152, based on the determination result by the collision possibility determination unit 20 and the blocked area ratio by the blocked area ratio calculation unit 50.

[0051] Specifically, when the collision possibility determination unit 20 determines that there is no possibility of a collision between the vehicle 200 and the obstacle S, the display control unit 60 controls the display R142 to display an interior view P5 configured with a pattern that will be used to decorate the interior of the vehicle, as shown in Fig. 7, or to display no display P6 as shown in Fig. 8, regardless of the covered area ratio calculated by the covered area ratio calculation unit 50. In this case, from the viewpoint of energy saving, it is preferable that the display control unit 60 controls the display R142 to display no display P6, that is, to display nothing.

[0052] In order to prevent the driver from staring at the screen, it is preferable that the interior view P5 is a still image. The interior view P5 is not limited to an image composed of a plurality of patterns, but may be an image composed of a single pattern, or a still image composed of a single color without a pattern, a uniform single color, or a single color with gradation. In addition, when a moving image is applied as the interior view P5, it is preferable that the image moves slowly so as not to attract the driver's attention.

[0053] FIG. 9 is a schematic diagram showing a state in which an interior view P5 and a warning display P7 are displayed on the display R142, and FIG. 10 is a schematic diagram showing a state in which the display R142 is in a blank state P6 and displays the warning display P7.

[0054] When the collision possibility determination unit 20 determines that there is a possibility of a collision between the vehicle 200 and the obstacle S, and the obstructed area ratio calculation unit 50 determines that the obstructed area ratio of the obstacle region R' is less than 80%. The display control unit 60 controls the display R142 not to display the transparent view P2. The transparent view P2 is an image obtained by cutting out the obstructed region N' from the driver's viewpoint image P1.

[0055] Furthermore, the display control unit 60 controls the display R142 to display a warning display P7 that warns the driver of the presence of an obstacle S that is partially obscured by the right A-pillar 152. The warning display P7 is a display that causes the outer periphery of the display R142 to emit light in red, for example.

[0056] The display control unit 60 displays an interior view P5 and a warning display P7 on the display R142, for example, as shown in Fig. 9. Note that the display control unit 60 may cause the display R142 to display no display P6 and to display the warning display P7, as shown in Fig. 10.

[0057] FIG. 11 is a schematic diagram showing a state in which a transparent view P2, which is an image within a blocked area N' of a driver's viewpoint image P1, and a warning display P7 are displayed on a display R142.

[0058] When the collision possibility determination unit 20 determines that there is a possibility of a collision between the vehicle 200 and the obstacle S, and the obstruction area ratio calculation unit 50 determines that the obstruction area ratio of the obstacle area R' is 80% or more, the display control unit 60 cuts out an image within the obstruction area N' from the driver's viewpoint image P1, and displays the cut-out image as a see-through view P2 on the display R142. The display control unit 60 displays the see-through view P2 and a warning display P7, for example, as shown in Fig. 11. Note that the display control unit 60 does not have to display the warning display P7 on the display R142.

[0059] <effect> The display control device 100 configured as above operates as follows. Fig. 12 is a flowchart explaining the flow of the operation of the display control device 100. Note that the following explanation is about a subject in the field of view to the front right of the vehicle 200, but the same applies to a subject in the field of view to the front left and the front right.

[0060] When the vehicle 200 starts up to a drivable state, the display control device 100 is powered on and starts operating. First, the display control device 100 performs an obstacle identification process (S1 in FIG. 2). When the display control device 100 is first powered on, the display R142 is controlled to display, by default, an interior view P5 (see FIG. 7) described below or no display P6 (see FIG. 8).

[0061] Specifically, in the obstacle identification process, the obstacle identification unit 10 detects an obstacle S based on the image P0 captured by the camera R112, and determines whether or not an obstacle S is present around the host vehicle 200 (S2).

[0062] When it is determined that no obstacle S exists around the vehicle 200 (NO in S2), the display control unit 60 does not display the see-through view P2 on the display R142. The display control unit 60 displays the interior view P5 shown in Fig. 7 on the display R142, or sets the display R142 to a non-display P6 shown in Fig. 8 (S11).

[0063] When it is determined that an obstacle S exists around the vehicle 200 (YES in S2), the display control device 100 performs a blocked area identification process (S3). Specifically, in the blocked area identification process, the blocked area identification unit 40 identifies a blocked area N' in the driver's viewpoint image P1 that corresponds to the blocked range N by the right A-pillar 152 when viewed with the naked eye from the driver's eye position, and determines whether or not an obstacle S exists in the blocked area N' (S4). The process of determining whether or not an obstacle S exists in the blocked area N' is a process of determining whether or not an obstacle area R' overlaps with the blocked area N' in the driver's viewpoint image P1.

[0064] When the obstruction area N' overlaps with the obstacle area R', it means that part or all of the obstacle S seen with the naked eye from the driver's eye position is overlapped by the obstruction area N by the right A-pillar 152 and cannot be seen.

[0065] When the obstructed area identification unit 40 determines that the obstructed area N' does not overlap with the obstacle area R' (NO in S4), the display control unit 60 displays the interior view P5 shown in FIG. 7 on the display R142, or sets the display R142 to a non-display P6 shown in FIG. 8 (S11).

[0066] When the obstructed area identification unit 40 determines that the obstructed area N' overlaps with the obstacle area R' (YES in S4), the display control device 100 performs a collision possibility determination process (S5). Specifically, in the collision possibility determination process, the collision possibility determination unit 20 determines whether or not there is a possibility of a collision between the vehicle 200 and the obstacle S (S6).

[0067] Then, when it is determined that there is no possibility of a collision (NO in S6), the display control unit 60 displays the interior view P5 shown in FIG. 7 on the display R142, or sets the display R142 to a no-display P6 shown in FIG. 8 (S11).

[0068] When it is determined that there is a possibility of a collision (YES in S6), the display control device 100 performs an obstacle shielding rate calculation process (S7). Specifically, in the obstacle shielding rate calculation process, the shielded area ratio calculation unit 50 calculates a shielded area ratio (=M1 / M0) [%] based on the area M0 of the obstacle area R' in the driver's viewpoint image P1 and the area M1 of the obstacle area R' overlapping with the shielded area N' in the driver's viewpoint image P1 identified by the shielded area identification unit 40, and determines whether the shielded area ratio is less than 80 [%] (S8).

[0069] When it is determined that the blocked area ratio calculated by the blocked area ratio calculation unit 50 is not less than 80% (NO in S8), the display control unit 60 displays, on the display R142, a transparent view P2 which is an image within the blocked area N' of the driver's viewpoint image P1, as shown in Fig. 11. The display control unit 60 may display, in addition to the transparent view P2 on the display R142, a warning display P7 which causes the outer periphery of the display R142 to emit light in red (S9).

[0070] In this way, when the proportion of the area of ​​the obstacle region R' that is overlapped and obstructed by the obstruction region N' to the total area of ​​the obstacle region R' is greater than a predetermined value (e.g., 80%]), the display control unit 60 displays the transparent view P2 on the display R142.

[0071] In addition, when there are multiple obstacle regions R' overlapping the blocked region N', if there is even one obstacle region R' whose blocked area ratio is equal to or greater than a predetermined value, the display control unit 60 displays the see-through view P2 on the display R142. In this case, the display control unit 60 displays all obstacle regions R' overlapping the blocked region N', including the obstacle regions R' whose blocked area ratio is less than the predetermined value, as the see-through view P2 on the display R142.

[0072] If the proportion of the obstructed area of ​​all of the obstacle regions R' that overlap the obstructed region N' does not exceed a predetermined value, the display control unit 60 does not display the transparent view P2 on the display R142.

[0073] On the other hand, when it is determined that the occlusion area ratio is less than 80% (YES in S8), the display control unit 60 displays the interior view P5 and the warning display P7 on the display R142, as shown in FIG. 9, or displays the warning display P7 after making the display P6 blank (S10).

[0074] In this way, when the proportion of the area of ​​the obstacle region R' that is overlapped and blocked by the blocked region N' to the total area of ​​the obstacle region R' is smaller than a predetermined value (e.g., 80[%]), the display control unit 60 does not display the see-through view P2 on the display R142. By not displaying the see-through view P2 when the driver can see most of the obstacle S with the naked eye, it is possible to reduce the annoyance felt when the see-through view P2 is frequently displayed and the discomfort felt when the see-through view P2 is displayed.

[0075] Then, after displaying on the display R142 by processing either S9, S10 or S11, the process returns to S1 (if NO at S12) until the main power supply to the vehicle 200 is cut off and the power supply to the display control device 100 is turned off (S12), and the process from S1 to S12 is repeated.When the main power supply to the vehicle 200 is cut off and the power supply to the display control device 100 is turned off (if YES at S12), the power supply to the display control device 100 is turned off.

[0076] The above description of the display control device 100 has been mainly about subjects in the field of view to the right in front of the vehicle 200, but the display control device 100 also processes subjects in the field of view to the left in front of the vehicle 200 in the same manner as subjects in the field of view to the right in front of the vehicle. That is, the display control device 100 detects an obstacle S in the range to the left in front of the vehicle based on the image P0 of the left in front of the vehicle captured by the camera L111, specifies an obstacle area R, converts it into a driver's viewpoint image P1, and then switches the image to be displayed on the display L141 installed on the left A-pillar 151 in accordance with the presence or absence of a possibility of a collision between the vehicle 200 and the obstacle S and the area ratio of the obstacle area R' corresponding to the obstacle S with which there is a possibility of a collision that is blocked by the left A-pillar 151.

[0077] <Effects> As described above in detail, the display control device 100 of this embodiment displays an image of the obstacle S (transparent view P2) on the display R142 provided on the right A-pillar 152 when there is a possibility that the vehicle 200 will collide with an obstacle around the vehicle 200 and when the driver sitting in the driver's seat of the vehicle 200 looks at an object to the front right with the naked eye from the eye position of the driver, and the obstacle S is obscured by the right A-pillar 152 by a certain amount (for example, 80% as an area percentage), making it difficult for the driver to see the obstacle S (Figure 11).

[0078] As a result, the driver can see objects in the front right field of view that are not hidden by the right A-pillar 152 as images seen by the driver's own naked eyes, and for the area N obscured by the right A-pillar 152 that cannot be seen by the driver's own naked eyes, the driver can see an image within the obscured area N' in the driver's viewpoint image P1 displayed on the display R142 provided on the right A-pillar 152 as a transparent view P2.

[0079] In other words, the driver is given the illusion that the transparent view P2, which is the image within the shielded area N' displayed on the display R142 mounted on the right A-pillar 152, is an image seen through the right A-pillar 152, and can see the subject in the front right field of vision as a pseudo-single image in which the image seen with the naked eye and the image displayed on the display R142 are connected.

[0080] Therefore, it is possible to reduce the likelihood of the driver overlooking an obstacle S that is difficult for the driver to see and that may cause a collision.

[0081] On the other hand, by displaying the warning display P7, the driver is made aware that the image displayed on the display R142 provided on the right A-pillar 152 is not an image that is actually seen by the driver's naked eye, thereby alerting the driver to overconfidence in the image displayed on the display R142.

[0082] Furthermore, even if there is a possibility that the vehicle 200 will collide with an obstacle around the vehicle 200, when the obstacle area R' overlapping with the occlusion area N' is hidden by less than a certain amount (for example, 80% as an area ratio), the display control device 100 of this embodiment does not display the transparent view P2 of the obstacle area R' on the display R142 provided on the right A-pillar 152, but displays an interior view P5 and a warning display P7 (Figure 9), or displays only the warning display P7 (Figure 10).

[0083] The state in which the obstacle region R' overlapping with the shielded region N' is hidden less than a certain amount corresponds to a state in which the obstacle S is hidden less than a certain amount by the right A-pillar 152 when the driver sitting in the driver's seat of the vehicle 200 looks at an object to the front right with the naked eye from the eye position of the driver. Conversely, in this case, since the obstacle S is exposed outside the shielded region N' to a certain extent or more, the driver can fully see the obstacle S with the naked eye.

[0084] Therefore, when the driver can adequately see the obstacle S with the naked eye, there is little need to display an image of the obstacle S (transparent view P2) on the display R142 provided on the right A-pillar 152.

[0085] In this way, the display control device 100 of the present embodiment, by not displaying an image (transparent view P2) of the obstacle S when there is little need to display the obstacle S on the display R142, can reduce the chances that the driver will feel uncomfortable due to differences (positional deviation, color deviation, brightness deviation, etc.) that may occur between an image seen with the naked eye and an image displayed on the display R142. Furthermore, the display control device 100 can reduce the annoyance felt by the driver due to frequent display of the transparent view P2.

[0086] Furthermore, when there is little need to display the obstacle S on the display R142, the display control device 100 can reduce the power consumption of the display R142 and save energy by setting the display control device 100 to a no-display P6 in which even the interior view P5 is not displayed. Reducing the power consumption of the display R142 can also extend the life of the display R142.

[0087] Even in such a case where there is little need to display the obstacle S on the display R142, since part of the obstacle S is hidden by the right A-pillar 152, the display control device 100 can display a warning display P7 on the display R142 to alert the driver that part of the obstacle S is hidden by the right A-pillar 152.

[0088] The display form of the warning display P7 displayed by the display control device 100 on the display R142 is not limited to the frame-shaped display described above, and various shapes of display may be applied. Furthermore, the warning display P7 is not limited to a continuously lit display, and may be a flashing display or a display that alternates between light and dark.

[0089] Figure 13 is a schematic diagram showing a state in which only the left edge 142L of the display R142 corresponding to the side on which the obstacle S is blocked is illuminated as a warning display P7, and Figure 14 is a schematic diagram showing a state in which only the right edge 142R of the display R142 corresponding to the side on which the obstacle S is blocked is illuminated as a warning display P7.

[0090] In addition, the warning display P7 can also be applied in a form in which only the edge of the left and right edges of the display R142 that corresponds to the boundary between the part of the obstacle S that is hidden by the right A-pillar 152 and the exposed part is illuminated.

[0091] For example, as shown in FIG. 13, when an obstacle S moves from left to right relative to the right A-pillar 152 and begins to be hidden by the right A-pillar 152, the display control device 100 causes only the left edge 142L of the display R142 to light up as a warning display P7, and does not cause the other edges to light up. As shown in FIG. 14, when an obstacle S moves from being hidden by the right A-pillar 152 to beginning to be exposed to the right, the display control device 100 causes only the right edge 142R of the display R142 to light up as a warning display P7, and does not cause the other edges to light up.

[0092] In addition to the above, the display control device 100 may illuminate only an area of ​​the edge of the display R142 that overlaps with the obstacle S. For example, in Fig. 13, since the obstacle S overlaps with the lower half of the left edge 142L, the display control device 100 may illuminate the lower half of the left edge 142L of the display R142 but not the upper half.

[0093] This enables the display control device 100 to provide the driver with an indication of the hidden portion of the obstacle S and the direction in which the obstacle S is moving.

[0094] In addition, when there is no possibility of the vehicle 200 colliding with an obstacle around the vehicle 200, the display control device 100 of this embodiment displays only the interior view P5 on the display R142 provided on the right A-pillar 152 (Figure 7), or displays no view P6 (Figure 8).

[0095] If there is no possibility of the vehicle 200 colliding with an obstacle, when the driver looks at an object on the front right side with the naked eye from the driver's eye position, the driver does not need to pay special attention to the obstacle, regardless of whether the obstacle S is hidden by the right A-pillar 152. In this case, there is no need, or the need is extremely low, to display the see-through view P2 of the obstacle area R' on the display R142 provided on the right A-pillar 152.

[0096] In this way, the display control device 100 of the present embodiment can reduce the chances that the driver will feel uncomfortable due to differences (such as positional deviation, color deviation, or brightness deviation) that may occur between an image seen with the naked eye and an image displayed on the display R142, by not displaying the see-through view P2 when there is little need to display the obstacle S on the display R142. Furthermore, the display control device 100 can reduce the annoyance felt by the driver due to frequent display of the see-through view P2.

[0097] Furthermore, when there is little need to display the transparent view P2 on the display R142, the display control device 100 can reduce the power consumption of the display R142 and save energy by setting the display R142 to a no-display P6 in which even the interior view P5 is not displayed. Reducing the power consumption of the display R142 can also extend the life of the display R142.

[0098] The display control device 100 of this embodiment is equipped with a collision possibility determination unit 20, which performs a collision possibility determination process (S5) and, if it determines that there is a possibility of a collision (YES in S6 of Figure 12), performs a shading rate calculation process (S7). However, the display control device of the present invention does not need to be equipped with a collision possibility determination unit.

[0099] A display control device of the present invention that does not have a collision possibility determination unit performs an obstacle coverage rate calculation process (S7) after the process of determining whether or not there is an obstacle (S4) in the blocked area identification process (S3), and switches the display process (S9 or S10) for the display R142 by the display control unit 60 depending on the blocked area ratio calculated by the blocked area ratio calculation unit 50 (S8).

[0100] The display control device 100 of this embodiment is set so that the power of the display control device 100 is automatically turned on and off in response to the main power of the vehicle 200 being turned on and off, but the display control device of the present invention may also be configured so that the power is turned on and off by the driver manually operating a switch based on the driver's intention.

[0101] In addition, the display control device of the present invention may be configured to turn on the power when the vehicle speed of the vehicle 200 acquired by the vehicle information acquisition unit 120 is in a low speed range so as to operate only when the vehicle speed is low (for example, 30 km / h or less), and to switch the power off when the vehicle speed of the vehicle 200 becomes high and exceeds the low speed.

[0102] In addition, the display control device of the present invention may have multiple cameras for acquiring images, and in the case of an area where the imaging ranges of obstacles captured by the multiple cameras overlap, if any of the cameras whose imaging range covers the overlapping area malfunctions or the lens becomes dirty, the image displayed on display R142 or display L141 may be switched to the image from the camera in better condition depending on the state of the malfunction or dirt.

[0103] The display control device 100 of this embodiment uses the right A-pillar and the left A-pillar as vehicle body structures that block the driver's seat from seeing a subject with the naked eye, but the display control device of the present invention is not limited to the right A-pillar and the left A-pillar as vehicle body structures that block the driver's seat from seeing a subject with the naked eye, and can also use the bonnet, instrument panel, doors, etc. of the vehicle 200.

[0104] In addition, the display control device of the present invention can be applied when reversing a vehicle, and can also be used to cover left and right B-pillars and C-pillars and other vehicle body structures that block the view of a subject in the rearward view of the driver seated in the driver's seat with the naked eye.

[0105] In addition, in the display control device 100 according to the embodiment, the collision possibility determination unit 20 may determine whether or not there is a collision possibility. However, in the display control device according to the present invention, the collision possibility determination unit 20 may determine whether or not there is a collision possibility as a step-likely numerical value. For example, the collision possibility determination unit 20 may calculate the possibility of collision between the vehicle 200 and the obstacle S as a value of 0 to 100% based on the state of the vehicle 200 and each identified obstacle S. In this case, when the collision possibility is equal to or greater than a predetermined value, it is determined that there is a collision possibility, and the display control unit 60 executes the obstacle shielding rate calculation process. When the collision possibility is smaller than the predetermined value, it is determined that there is no collision possibility, and the display control unit 60 does not display the transparent view. [Explanation of symbols]

[0106] 10 Obstacle Identification Section 30 Driver information acquisition unit 40 Shield area identification part 50 Shielding area ratio calculation section 60 Display control unit 100 Display control device 112 Camera R 142 Display R 152 Right A-pillar 200 vehicles M0 area M1 area N′ occluded area P1 Driver's view image P2 Transparent View

Claims

1. A display control device controls a display unit provided on a vehicle body structure that obstructs the driver's view of a vehicle, an obstacle identification unit that identifies an obstacle area including an obstacle from an image acquired from an imaging unit that captures an image of the periphery of the vehicle; A driver information acquisition unit that acquires driver information related to a field of view of a driver of the vehicle; an obstructed area identifying unit that identifies an obstructed area in which the structure obstructs the driver's view based on the driver information; a blocked area ratio calculation unit that calculates a blocked area ratio, which is a ratio of an area of ​​the obstacle area that overlaps with the blocked area to a total area of ​​the obstacle area; A display control unit that controls an image to be displayed on the display unit, The display control device is configured to, depending on the shaded area ratio, not display on the display unit a driver's viewpoint image obtained by converting the viewpoint of the image captured by the imaging unit based on the driver information.

2. The display control unit displays the driver's viewpoint image on the display unit when the shaded area ratio is equal to or greater than a predetermined value, The display control device according to claim 1 , wherein the driver's viewpoint image is not displayed on the display unit when the blocked area ratio is less than a predetermined value.

3. a collision possibility determination unit that determines a possibility of a collision between the vehicle and the obstacle identified by the obstacle identification unit, 3. The display control device according to claim 2, wherein the display control unit displays the driver's viewpoint image on the display unit when there is a possibility of a collision and the occlusion area ratio is equal to or greater than a predetermined value based on the occlusion area ratio and the judgment of the collision possibility judgment unit, and does not display the driver's viewpoint image on the display unit when there is a possibility of a collision and the occlusion area ratio is less than a predetermined value or when there is no possibility of a collision.

4. The display control device according to claim 1 , wherein the display control unit is configured to display a still image or no image on the display unit when the driver's viewpoint image is not displayed on the display unit.

5. The display control device according to claim 2 , wherein the display control unit displays a warning on the display unit when the obstructed area is less than the predetermined value.

6. The display control device according to claim 5 , wherein the display control unit, when the obstructed area is less than the predetermined value, displays the warning on an edge of the display unit corresponding to a side on which the obstacle is obstructed.

Images