Vehicular display device

Abstract

To prevent irradiation to a surrounding person by reflection in a falling object and reduce the influence.SOLUTION: A vehicular display device 100 is provided in a vehicle 200. The vehicular display device includes: a road projector 1 which performs desired projection display to the outside of the vehicle 200; and a CPU 41 of a control unit 4. The CPU 41 executes: a procedure of Step S25 for detecting a falling object R in a projection range Ap by the road projector 1 outside the vehicle 200; and procedures of Step S35 to Step S45 for turning off or dimming an existence expansion region Ae corresponding to at least the falling object R in the projection display by the road projector 1 according to the detection of the falling object R.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a display device for a vehicle that is mounted in a vehicle and performs a desired display. [Background technology]

[0002] Conventionally, there is known a display device as described in, for example, Patent Document 1. This display device projects a holographic reproduction image or the like onto a road surface or the like from a projector mounted on a vehicle, thereby displaying predetermined information to vehicles traveling around the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6344545 Summary of the Invention [Problem to be solved by the invention]

[0004] Projectors used in the display devices of the above-mentioned prior art generally emit high illuminance light, and are therefore considered to avoid irradiating people's faces as much as possible. However, there are cases where unexpected objects have fallen onto the outdoor road surface where the projector projects and displays, and further, there are cases where the fallen object has a glossy surface and reflects the projection light. In such cases, there is a possibility that the projection light projected from the projector onto the road surface will be reflected by the fallen object and irradiated onto the faces of people nearby.

[0005] Therefore, the present invention has been made in consideration of the above problems, and has an object to provide a vehicle display device that can prevent irradiation of surrounding people due to reflection from falling objects, thereby reducing the impact. [Means for solving the problem]

[0006] The present invention is a vehicle display device 100 provided in a vehicle 200, comprising a projection unit 1 that performs a desired projection display outside the vehicle 200, and a control unit 41, wherein the control unit 41 performs a falling object detection process that detects a falling object R within a projection range Ap by the projection unit 1 outside the vehicle 200, and a darkening process that turns off or dims at least the area Ar corresponding to the falling object R in the projection display by the projection unit 1 in response to the detection of the falling object R in the falling object detection process. Effect of the Invention

[0007] According to the present invention, it is possible to provide a display device for a vehicle that can prevent irradiation of people in the vicinity due to reflection from a falling object, thereby reducing the influence. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram showing an overall configuration of a vehicle equipped with a vehicle display device according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a functional block diagram showing a system configuration of the vehicle display device. [Diagram 3] 1A and 1B are diagrams illustrating a situation in which the projection light of a road projector is reflected by objects lying on the road surface and irradiated onto people nearby. [Figure 4] 11 is a diagram showing the relationship between the projection range and projection coordinates of a road projector and the imaging range and imaging coordinates of an exterior camera. [Diagram 5] 11A and 11B are diagrams illustrating examples of detection images that are used as a factor in determining whether a fallen object is a reflective object. [Figure 6] 13 is a diagram showing the state of the projection display when the projection light is turned off or dimmed according to the range of the falling object. [Figure 7] 4 is a flowchart showing a control procedure executed by a CPU of the control unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] Fig. 1(a) shows an overall perspective view of a vehicle equipped with the vehicular display device of this embodiment, as viewed from the diagonal front side, and Fig. 1(b) shows an overall perspective view of the vehicle as viewed from the diagonal rear side. In the following, in order to facilitate understanding of the configuration of the display device, the front side of the vehicle is referred to as "front (Fr.)", the opposite side as "rear (Re.)", the left side as viewed from a passenger in the vehicle as "left (L)", the right side as "right (R)", the upper side as "upper (Up.)", and the lower side as "lower (Lo.)", and each part will be described appropriately (see the arrows in each figure).

[0011] <Overall configuration of vehicle display device> In these Figures 1(a) and 1(b), the vehicle display device 100 is mounted on various vehicles such as private cars, commercial vehicles, buses, etc., and in this example has 12 road projectors 1 (projection units), 12 exterior cameras 2 (imaging units), a touch panel 3, and a control unit 4.

[0012] The road projector 1 is an optical display device that projects and displays a predetermined image on the road surface nearby (see Fig. 3 and Fig. 4 described later), and in the illustrated example, it is provided at a total of 12 locations, including the left end position, center position, right end position of each of the front bumper 201 and rear bumper 202 of the vehicle 200, and below the left and right side mirrors 203 (see Fig. 1). Note that what is projected and displayed may be an image of a design image, or simply white light. The road projector 1 is also capable of controlling the illumination, turning off, and dimming of the projection light at any coordinate position within its projection range.

[0013] The exterior camera 2 is an optical device that is individually installed in a position near each of the 12 road projectors 1 mentioned above, and is capable of capturing images in an imaging range Ac that approximately coincides with the projection range Ap on the road surface of the corresponding road projector 1, and outputting image information (see Figure 4 described below).

[0014] In this example, the touch panel 3 is a device that is arranged on the instrument panel (dashboard) in front of the user (driver) inside the vehicle and is configured to combine a liquid crystal display and a touch pad (see FIG. 1). The touch panel 3 displays predetermined information such as navigation information, and allows the user to input control commands related to the projection display of each road projector 1.

[0015] The control unit 4 is a computer having a CPU 41, a memory 42, etc. therein, and in the illustrated example, is disposed in an instrument panel near the front side of the touch panel 3 (see FIG. 1). FIG. 2 is a functional block diagram showing the overall system configuration of the vehicle display device 100, including the connection configuration between the control unit 4 and each of the above-mentioned components. In FIG. 2, the CPU 41 (control unit) provided in the control unit 4 is capable of connecting to each of the above-mentioned 12 outside-vehicle cameras 2 via a bus 43 to acquire image information, and is also capable of connecting to each of the 12 road projectors 1 to output various control information for projection display. The CPU 41 is also capable of outputting various display information to the touch panel 3, and is also capable of inputting various operation information.

[0016] The CPU 41 executes various programs stored in advance while utilizing the temporary storage function of the memory 42, thereby controlling the entire vehicle display device 100. The memory 42 is a storage device equipped with a RAM and a ROM, not shown. The various programs include a program for executing the procedures shown in FIG. 7, which will be described later. The control unit 4 and each component may be connected via a communication interface, such as a CAN (Controller Area Network), or may be electrically connected via a hard wire.

[0017] <Operation of vehicle display device> The operation of the vehicle display device 100 will be described below with reference to Figs. 3 to 6. For example, as shown in the rear perspective view of Fig. 3, all road projectors 1 basically project light approximately downward toward a predetermined projection range on the road surface nearby to perform a predetermined projection display. However, there are cases where an unexpected fallen object R has fallen on the outdoor road surface where the road projector 1 projects and displays, and further, there are cases where the fallen object R has a glossy surface such as a processed product made of metal or plastic, and has a property of reflecting the projection light. When such a fallen object R with reflective properties is present within the projection range of the road projector 1, a part of the projection light projected from the road projector 1 toward the road surface is reflected by the surface of the fallen object R and directed upward, and may be irradiated onto the face of a person M in the vicinity, causing a sense of glare.

[0018] Here, the specific configuration of the road projector 1 to be installed includes not only a floodlight that simply projects white light, but also a hologram projector, a spatial light modulator (SLM), and the like that uses a laser as a light source in recent years. For this reason, it is generally considered that the projection light of the road projector 1 should be prevented from being directly irradiated onto the face of the person M as much as possible, due to the generally high illuminance. However, on the other hand, in situations where light projection by the road projector 1 is required, such as at night or in a dark place such as an underground, there is also a demand to maintain the light projection state as much as possible.

[0019] For this reason, in this embodiment, first, the control unit 4 executes a falling object detection process to detect the presence of a falling object R in the projection range of each road projector 1 of the vehicle 200, that is, in the range where the projection light is actually irradiated, as shown in Fig. 4. Then, in response to the detection of a falling object R by the falling object detection process, a darkening process is executed to turn off or dim at least the area corresponding to the falling object R in the projection display by each road projector 1, as shown in Fig. 6. This makes it possible to prevent the projection of light reflected by the falling object R on the surrounding people M, thereby reducing the impact, while maintaining the projection display by the road projector 1 on the road surface as much as possible.

[0020] Specifically, first, as shown in Fig. 4(a), each outside camera 2 captures the projection range Ap on the road surface of the corresponding road projector 1. In this embodiment, the corresponding outside camera 2 and road projector 1 are arranged close to each other as described above, and the imaging direction Dc and the projection direction Dp are set to be in substantially the same direction and attitude, so that the imaging range Ac on the road surface and the projection range Ap are substantially the same. Furthermore, the scales of the imaging range Ac and the projection range Ap on the road surface are also matched in the control settings, so that the imaging coordinates Xc-Yc of the outside camera 2 and the projection coordinates Xp-Yp of the road projector 1 are also matched in the control settings, as shown in Fig. 4(b), which is the image information captured in Fig. 4(a). This makes it possible to omit a separate coordinate conversion process and use the position information on the imaging coordinates Xc-Yc of the outside camera 2 as it is as the position information on the projection coordinates Xp-Yp of the road projector 1.

[0021] Then, by analyzing the image information captured by the outside camera 2, the presence or absence of a foreign object such as a fallen object R in the image coordinates Xc-Yc of the outside camera 2, i.e., in the projection coordinates Xp-Yp of the road projector 1, and the area in which it exists are detected, as shown in the example in FIG. 4(b). In the example shown, an arrow mark W pointing in the direction toward the body of the vehicle 200 is displayed as the projection content, and a fallen object R having a substantially pentagonal flat plate shape is present at approximately the center of the arrow mark W, and the outer contour of its existence range Ar is detected in a polygonal range connecting five points P1 to P5. Note that, as a specific image analysis process performed at this time, a known method such as a foreign object detection process or an object detection process using so-called deep learning such as a Convolutional Neural Network (CNN) may be applied.

[0022] Furthermore, in this embodiment, a process of determining whether the detected falling object R is a reflective object that reflects the projection light is also performed, and depending on the result of the determination, the darkening process is switched off or dimmed. In this determination process, when a portrait of a person such as a face shown in Fig. 5(a), the appearance of a tree shown in Fig. 5(b), or a cityscape with buildings shown in Fig. 5(c) is detected within the range Ar of the falling object R in the image information, the falling object R is determined to be a reflective object.

[0023] These detection images are images captured by light reflected from the reflecting object R from the outside world and incident on the outside camera 2 in the opposite direction to the projection light from the road projector 1, and are images that cannot normally be captured directly from the road surface. Therefore, the detection images themselves are upside down, and may be images of portraits of animals such as dogs and cats, or skyscapes such as sunsets and starry skies, in addition to the images shown in the figures. By detecting such detection images in the captured image, it can be determined that the fallen object R is a reflecting object that reflects the projection light from the road projector 1 approximately upward. For the determination process of such detection images, known methods such as the discrimination process using deep learning and object detection process described above may be applied.

[0024] If the falling object R is determined to be a reflective object in the determination process, the projection of the road projector 1 is controlled to completely turn off the projected light in a limited area corresponding to the existence range Ar of the falling object R, as shown in Fig. 6(a). In the illustrated example of this embodiment, the projected light on the projection coordinates Xp-Yp of the road projector 1 is partially turned off in correspondence with the existence extension area Ae (polygonal area connecting five points P1' to P5' in the figure) which is set slightly wider by a margin than the existence range Ar of the falling object R in the image capture coordinates Xc-Yc of the outside-vehicle camera 2 (polygonal area connecting five points P1 to P5; see Fig. 4(b) above).

[0025] Even if the falling object R is determined not to be a reflective object in the determination process, the projection of the road projector 1 is controlled to reduce the brightness of the projection light only in the area corresponding to the existence range Ar of the falling object R, as shown in Fig. 6(b) to be on the safe side. In this way, by switching off or reducing the brightness depending on the determination result of whether the falling object R is a reflective object or not, it is possible to control the projection light according to the possibility of reflection so as to ensure the light projection state as much as possible.

[0026] <Control unit control procedure> In order to realize the above-mentioned darkening process in the vehicle display device 100 of this embodiment, the control procedure executed by the CPU 41 of the control unit 4 will be described with reference to the flowchart of Fig. 7. The procedure of Fig. 7 is started for a specific individual road projector 1 when, for example, a command is issued from the touch panel 3 to perform some kind of projection display on any road projector 1.

[0027] First, in step S5, it is determined whether or not a projection display has been instructed for the individual road projector 1 that is the control target in the execution of the procedure. If a projection display has not been instructed for the target road projector 1, the determination is not satisfied (S5: No), and the process proceeds to step S10.

[0028] In step S10, the target road projector 1 is controlled to completely turn off its entire projection area Ap, and then this flow ends.

[0029] On the other hand, if it is determined in step S5 that projection display has been instructed in the target road projector 1, the condition is satisfied (S5: Yes), and the process proceeds to step S15.

[0030] In step S15, the outside camera 2 corresponding to the road projector 1 in question captures an image of a predetermined range on the road surface that is to be projected and displayed.

[0031] Next, the process proceeds to step S20, where image analysis is performed on the captured image information (imaging result).

[0032] Next, the process proceeds to step S25, where it is determined based on the result of the image analysis whether or not a falling object R exists within the range that the target road projector 1 is about to project. If a falling object R does not exist, the determination is not satisfied (S25: No), and the process proceeds to step S30. The procedure of step S25 corresponds to the falling object detection process described in each claim.

[0033] In step S30, the entire display content that was planned to be displayed in a corresponding predetermined range on the road surface is projected as is by the target road projector 1. Then, this flow ends.

[0034] On the other hand, if it is determined in step S25 that a falling object R is present, the condition is satisfied (S25: Yes), and the process proceeds to step S35.

[0035] In step S35, it is determined whether or not the falling object R is a reflective object based on the result of the image analysis. Specifically, it is determined whether or not a specific image such as those shown in Figs. 5(a) to 5(c) above is detected within the existence range Ar of the falling object R in the image information. If a specific image is not detected within the existence range Ar of the falling object R, the determination is not satisfied (S35: No), and the process proceeds to step S40. Note that the procedure of step S35 corresponds to the determination process described in each claim.

[0036] In step S40, the projection display of the road projector 1 is controlled to be partially dimmed limited to the extended area Ae of the falling object R including the margin (see FIG. 6(b)). Then, this flow ends.

[0037] On the other hand, if it is determined in step S35 that a specific image is detected within the range Ar of the falling object R, the condition is satisfied (S35: Yes), and the process proceeds to step S45.

[0038] In step S45, the projection display of the road projector 1 is controlled to be partially turned off only in the extended area Ae of the falling object R including the margin (see FIG. 6(a)). Then, this flow ends. Note that the above steps S35 to S45 correspond to the darkening process described in each claim.

[0039] The above-described procedure of the flow is repeatedly executed in sequence for each of the twelve road projectors 1 mounted on the vehicle as a control target while some projection display is instructed.

[0040] <Effects of the embodiment> The effects of this embodiment will be described below. As described above, in the vehicle display device 100 of this embodiment, the CPU 41 of the control unit 4 detects a falling object R in the projection range Ap of the road projector 1 outside the vehicle 200 in step S25, and in response to the detection result of the falling object R, executes the darkening process of steps S35 to S45 for turning off or dimming at least the area Ar corresponding to the falling object R in the projection display by the road projector 1.

[0041] This makes it possible to maintain the overall projection display on the road surface by the road projector 1 when a falling object R is not detected. Also, even when a falling object R is detected, it is possible to prevent the light reflected by the falling object R from being irradiated onto nearby people M, while maintaining the projection display on the road surface by the road projector 1 as much as possible, thereby reducing the impact.

[0042] In this embodiment, the vehicle is further provided with an exterior camera 2 that captures an image of the outside of the vehicle 200, and the CPU 41 of the control unit 4 identifies the fallen object R by performing image analysis on the image information captured by the exterior camera 2 in the procedure of step S25. This enables rapid and accurate detection of the fallen object by optical detection such as camera imaging and image analysis. Note that, in addition to the exterior camera 2, other optical object detection methods such as a method of detecting the presence or absence of reflected laser light and a detection method using radar that detects the presence or absence of reflection of electromagnetic waves and the reflection time difference can also be applied.

[0043] In particular, in this embodiment, when a falling object R is detected in the procedure of step S25, the CPU 41 of the control unit 4 further executes the procedure of step S35 to determine whether the falling object R is a reflecting object. When the falling object R is determined to be a reflecting object, the projection display of the road projector 1 is partially turned off in at least an area corresponding to the existence range Ar of the reflecting object R in a darkening process. This makes it possible to reliably prevent the reflection of the projection light from the road projector 1 on the reflecting object R, assuming that there is a high possibility that the projection light from the road projector 1 will be irradiated to the person M.

[0044] In particular, in this embodiment, the projection range Ap of the road projector 1 is the road surface, and the CPU 41 of the control unit 4 determines that the fallen object R is a reflecting object based on image analysis in the procedure of step S35 when at least one of a person, a tree, and a building is included in the captured image. In this way, it is determined that an image that cannot normally be captured directly from the road surface is included, so that it can be determined with high reliability that the fallen object R is a reflecting object.

[0045] In particular, in this embodiment, the CPU 41 of the control unit 4 controls the road projector 1 in the darkening process to turn off the presence extended area Ae including at least the outline of the falling object R in the projection display of the road projector 1. This makes it possible to block only the projection light irradiated onto the falling object R, and reliably prevent the reflection of the projection light on the falling object R while maintaining the projection display by the road projector 1 onto the road surface as much as possible.

[0046] In particular, in this embodiment, the CPU 41 of the control unit 4 controls the road projector 1 in the darkening process so that the brightness of the presence extended area Ae including at least the outline of the falling object R is made lower than the brightness of other areas in the projection display. This makes it possible to reduce the brightness of the projection light irradiated onto the falling object R, and to reduce the influence of the light reflected by the falling object R on the surrounding people M while maintaining the projection display onto the road surface by the road projector 1 as much as possible.

[0047] In addition, even if the projection light of the road projector 1 is reflected by a reflecting object R, there is no problem if the reflected light is irradiated onto trees or buildings other than the human M or animals. For this reason, although not shown in the figure, the CPU 41 of the control unit 4 may execute a human body detection process to detect the presence of a human body in a predetermined range outside the vehicle 200, for example, by image analysis of an image captured by a separate camera, and may perform a darkening process of the road projector 1 only when the presence of a human body is detected. In this way, the darkening process is executed only when there is a high possibility that the reflected light will be irradiated onto the human M, and the projection display by the road projector 1 on the road surface can be maintained as much as possible otherwise. In addition, the detection method in the human body detection process may be temperature detection equivalent to body temperature, dynamic changes in detection information such as laser or radar, or a combination of these.

[0048] Furthermore, the locations and numbers of the road projector 1 and the outside vehicle camera 2 are not limited to those in the above embodiment, and they may be installed in various other locations and in various other numbers.

[0049] In addition to the above, the above-mentioned embodiments and the methods according to the modified examples may be appropriately combined and used. For example, in the present embodiment, when the falling object R is a reflective object, the road projector 1 is controlled to make the brightness of the presence extension area Ae lower than the brightness of other areas as a darkening process, but the brightness of the entire projection range Ap including the presence extension area Ae may be turned off or dimmed.

[0050] Although not specifically illustrated, the present invention can be implemented with various modifications without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0051] 1 Road projector (projection unit) 2. External camera (imaging unit) 3. Touch Panel 4. Control Unit 41 CPU (control unit) 42 Memory 43 Bus 100 Vehicle display device 200 vehicles 201 Front bumper 202 Rear bumper 203 Side Mirror Ac Imaging Range Ae Existence Expansion Area Ap Projection Range Ar Falling object range Dc Imaging direction Dp projection direction M Human R Falling objects, reflective objects W Arrow mark (projection display)

Claims

1. A vehicle display device provided in a vehicle, a projection unit for projecting and displaying a desired image on the outside of the vehicle; A control unit; Equipped with The control unit is a falling object detection process for detecting a falling object within a projection range of the projection unit outside the vehicle; a darkening process for turning off or dimming at least an area corresponding to the falling object in the projection display by the projection unit in response to the detection of the falling object in the falling object detection process; A display device for a vehicle, comprising:

2. An imaging unit that images the outside of the vehicle, The control unit, in the falling object detection process, The fallen object is identified by performing image analysis on the image captured by the imaging unit.

2. The display device for a vehicle according to claim 1.

3. The control unit further includes: executing a determination process for determining whether or not the falling object is a reflective object when the falling object is detected in the falling object detection process; When the falling object is determined to be the reflecting object in the determination process, at least an area corresponding to the reflecting object is turned off or dimmed in the darkening process.

3. The display device for a vehicle according to claim 2.

4. The projection range is a road surface, The control unit, in the determination process, Based on the image analysis, if at least one of a human or animal face, the sky, and a building is included in the image capture result, the fallen object is determined to be a reflective object.

4. The display device for a vehicle according to claim 3.

5. The control unit, in the darkening process, The projection unit is controlled so as to turn off an area including at least the outline of the falling object in the projection display.

5. The display device for a vehicle according to claim 4.

6. The control unit, in the darkening process, The projection unit is controlled so that, in the projection display, the brightness of an area including at least the outline of the falling object is made lower than the brightness of other areas.

5. The display device for a vehicle according to claim 4.

7. The control unit further includes: Executing a human body detection process to detect the presence of a human body within a predetermined range outside the vehicle; In the darkening process, When the presence of a human body is detected by the human body detection process, the darkening process is performed, and when the presence of a human body is not detected, the darkening process is not performed.

7. The display device for a vehicle according to claim 1,

Images