vehicle

Abstract

To project and display various kinds of information on such as a road surface, a wall surface, or own vehicle on the basis of information relating to a vehicle, such as a running state of own vehicle (movable body represented by automobile).SOLUTION: A vehicle includes: an acquisition part for acquiring information; and an image projection part for projecting an image. The image projection part projects or non-projects an image on the basis of information that is acquired by the acquisition part.SELECTED DRAWING: Figure 21

Description

Technical Field

[0001] The present invention relates to a vehicle, an image projection device, and an image projection method.

Background Art

[0002] Video projection devices represented by projectors have already been widely used in a wide range of fields as devices for enlarging and projecting a desired video, and in recent years, they have also been widely used as display devices for personal computers and mobile phones.

[0003] Regarding such video projection devices, the following prior arts related to use in vehicles are already known.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Patent Document 5

Summary of the Invention

Problems to be Solved by the Invention

[0005] Specifically, Patent Document 1 discloses a projection-type display device that uses a vehicle's headlight as an external light source by placing a highly portable LCD projector, which does not have a built-in light source, in front of the vehicle's headlight, which serves as an external light source. Patent Document 2, in order to solve the problems described above, shows an embodiment that realizes a first state in which the projector is pre-installed in front of the headlight and inside the vehicle, and a second state in which the projector or headlight is moved to directly irradiate the light beam from the headlight onto the outside of the vehicle, and furthermore, an embodiment that displays an image on the road.

[0006] Furthermore, Patent Document 3 describes a vehicle driving assistance device that displays information on the road in front of the vehicle using an illumination means (laser) attached to the headlight area at the front of the vehicle, in order to effectively alert the occupants of the vehicle when lane departure is detected.

[0007] Furthermore, according to Patent Document 4, a projector is already known to be attached to the front of a vehicle as a projection means, and a route guidance image that guides the vehicle in the direction of a branching point is projected onto the road surface in front of the vehicle, with the projection angle set, based on route information searched by a navigation system. In addition, according to Patent Document 5, a vehicle driving assistance device is already known that enables recognition of the vehicle's destination by projecting a drawing pattern consisting of a target mark and a tracking line onto the road surface in front of the vehicle based on the vehicle's driving state, thereby enabling appropriate driving based on this recognition.

[0008] However, the conventional technologies described above did not necessarily effectively display the various types of information necessary for the vehicle's operation.

[0009] Therefore, the present invention has been achieved in view of the problems of the prior art described above, and aims to provide a vehicle that can project and display various types of information onto the road surface, walls, on the vehicle itself (hereinafter referred to as "road surface, etc.") based on information about the vehicle, such as the driving status of the vehicle (a mobile body such as an automobile). [Means for solving the problem]

[0010] To solve the above problems, for example, the configuration described in the claims is adopted. The present application includes multiple means for solving the above problems, but one example is a vehicle comprising an information acquisition unit and an image projection unit that projects an image, wherein the image projection unit projects or does not project an image based on the information acquired by the information acquisition unit. [Effects of the Invention]

[0011] According to the present invention, it is possible to provide a vehicle that can project and display information onto the road surface or the like, based on information related to the vehicle. [Brief explanation of the drawing]

[0012] [Figure 1] This is a front-view perspective of a vehicle equipped with an image projection device according to one embodiment of the present invention, projecting images onto the road surface or the like. [Figure 2] This is a rearward perspective view of a vehicle equipped with an image projection device according to one embodiment of the present invention, which is projecting images onto the road surface or the like. [Figure 3] This diagram shows the overall configuration of the light distribution control ECU that makes up the video projection device. [Figure 4] This is a block diagram showing a more detailed configuration example of the light distribution control ECU and its surrounding elements. [Figure 5] This figure shows an example of the configuration of an image projection device according to one embodiment of the present invention. [Figure 6] This is a ray diagram that includes the image plane of the projector. [Figure 7] This figure shows specific examples of various images projected onto the road surface from a projector, depending on the relationship with vehicle information. [Figure 8] This figure shows specific examples of various images projected onto the road surface from a projector, depending on the relationship with vehicle information. [Figure 9] This figure shows specific examples of various images projected onto the road surface from a projector, depending on the relationship with vehicle information. [Figure 10]It is a diagram showing specific examples of various images projected onto the road surface based on the relationship between the projector and vehicle information. [Figure 11] It is a diagram showing specific examples of various images projected onto the road surface based on the relationship between the projector and vehicle information. [Figure 12] It is a diagram showing specific examples of various images projected onto the road surface based on the relationship between the projector and vehicle information. [Figure 13] It is a diagram showing specific examples of various images projected onto the road surface based on the relationship between the projector and vehicle information. [Figure 14] It is a diagram showing specific examples of various images projected onto the road surface based on the relationship between the projector and vehicle information. [Figure 15] It is a diagram showing specific examples of various images projected onto the road surface based on the relationship between the projector and vehicle information. [Figure 16] It is a diagram showing specific examples of various images projected onto the road surface based on the relationship between the projector and vehicle information. [Figure 17] It is a diagram showing specific examples of various images projected onto the road surface based on the relationship between the projector and vehicle information. [Figure 18] It is a diagram showing specific examples of various images projected onto the road surface based on the relationship between the projector and vehicle information. [Figure 19] It is a diagram showing specific examples of various images projected onto the road surface based on the relationship between the projector and vehicle information. [Figure 20] It is a diagram showing specific examples of various images projected onto the road surface based on the relationship between the projector and vehicle information. [Figure 21] It is a diagram showing specific examples of various images projected onto the road surface based on the relationship between the projector and vehicle information. [Figure 22] It is a diagram showing specific examples of various images projected onto the road surface based on the relationship between the projector and vehicle information. [Figure 23] It is a diagram showing specific examples of various images projected onto the road surface based on the relationship between the projector and vehicle information. [Figure 24] It is a diagram showing specific examples of various images projected onto the road surface based on the relationship between the projector and vehicle information. [Figure 25] This figure shows specific examples of various images projected onto the road surface from a projector, depending on the relationship with vehicle information. [Figure 26] This figure shows specific examples of various images projected onto the road surface from a projector, depending on the relationship with vehicle information. [Figure 27] This figure shows more detailed examples of various images projected onto the road surface from a projector, depending on the relationship with vehicle information. [Figure 28] This figure shows more detailed examples of various images projected onto the road surface from a projector, depending on the relationship with vehicle information. [Figure 29] This figure shows more detailed examples of various images projected onto the road surface from a projector, depending on the relationship with vehicle information. [Figure 30] This figure shows more detailed examples of various images projected onto the road surface from a projector, depending on the relationship with vehicle information. [Figure 31] This figure shows more detailed examples of various images projected onto the road surface from a projector, depending on the relationship with vehicle information. [Figure 32] This figure shows more detailed examples of various images projected onto the road surface from a projector, depending on the relationship with vehicle information. [Figure 33] This figure shows more detailed examples of various images projected onto the road surface from a projector, depending on the relationship with vehicle information. [Figure 34] This figure shows more detailed examples of various images projected onto the road surface from a projector, depending on the relationship with vehicle information. [Figure 35] This figure shows more detailed examples of various images projected onto the road surface from a projector, depending on the relationship with vehicle information. [Figure 36] This figure shows more detailed examples of various images projected onto the road surface from a projector, depending on the relationship with vehicle information. [Figure 37] This figure shows more detailed examples of various images projected onto the road surface from a projector, depending on the relationship with vehicle information. [Modes for carrying out the invention]

[0013] The embodiments of the present invention will be described in detail below with reference to the attached drawings.

[0014] <Arrangement of video projection equipment> First, Figures 1(A) and (B) show a passenger car as an example of a vehicle 10 equipped with an image projection device according to one embodiment of the present invention. As shown in these figures, a pair of left and right headlights 11 are provided at the front of the body of the passenger car 10.

[0015] In the example shown in Figure 1(A), although not shown in detail here, lamps, which are light-emitting elements, are incorporated inside the pair of headlights 11. Also in the example shown in Figure 1(A), the vehicle (passenger car) 10 is equipped with a pair of image projection devices, which will be described in detail below. The image light from these image projection devices is projected in front of the vehicle (passenger car) 10, for example, through a transparent window. In this example, the image projected onto the road surface, etc., indicates the current or future direction of travel to pedestrians walking near the vehicle (passenger car) 10, thereby ensuring a higher level of safety.

[0016] Figure 1(B) shows an example in which only one image projection device is mounted at the front of the vehicle body. In this case, the image light from the image projection device is projected in front of the vehicle (passenger car) 10 through a transparent window 12 provided at the front of the vehicle body, for example.

[0017] Furthermore, as shown in Figure 1(A), image projection devices are also mounted on the bottom and sides of the vehicle (passenger car) 10, allowing the desired image to be projected onto the road surface or other adjacent surfaces on both sides of the vehicle (passenger car) 10.

[0018] Next, Figures 2(A) and (B) show the rear of the vehicle (passenger car) 10 equipped with an image projection device according to one embodiment of the present invention. As shown in these figures, red taillights 13 and 13' are provided at the rear of the vehicle body. In the example of Figure 2(A), although not shown in detail here, lamps, which are light-emitting elements, are incorporated inside these taillights 13 and 13'. In the example of Figure 2(A), the image projection device is mounted in a pair on the left and right sides, and the image light from the image projection device is projected to the rear of the vehicle (passenger car) 10, for example, through a transparent window.

[0019] Furthermore, Figure 2(B) shows an example in which the image projection device is mounted, for example, near the roof of the vehicle body. In this example of Figure 2(B), similar to Figure 1(B) above, the image light is projected to the rear of the vehicle 10 through a transparent window provided at the rear end of the vehicle body.

[0020] Furthermore, as shown in Figure 2(A), a video projection device is also installed inside the side mirror 14, which allows the desired image to be projected onto the road surface or other adjacent surfaces on both sides of the vehicle 10.

[0021] The above describes an example in which one or more (for example, a pair) video projection devices are mounted on the front, rear, left, and right sides of the vehicle 10. However, the present invention is not limited to these examples, and the video projection devices may be mounted anywhere on the vehicle 10 (for example, on the roof). The video projection devices may also be integrated into the headlights or taillights. In other words, the present invention only requires that the video projection devices can project a desired image onto the road surface or the like. When the video projection devices are integrated into the headlights or taillights, the light sources of the headlights or taillights can also be used as light sources for projection.

[0022] <Configuration of the Light Distribution Control ECU> Next, Figure 3 shows an example of the configuration of the electronic control unit (light distribution control ECU) installed in the vehicle (passenger car) 10 described above. As is clear from this figure, the light distribution control ECU 40 is equipped with a CPU (central processing unit) 41, memory means such as RAM 42 and ROM 43, and an input / output device (I / O unit) 44. Information from various information acquisition and communication units is input to the light distribution control ECU via the I / O unit 44, and controls the driving of the headlights 11 and the projection of images by the image projection device 500.

[0023] The information acquired from the above-mentioned information acquisition unit includes, for example, a speed signal indicating the vehicle's speed, a signal indicating the engine status (ON / OFF), gear information indicating the gear position, a hazard signal to alert surrounding drivers of the presence of danger, a steering angle signal indicating the steering angle, a turn signal signal indicating the presence or absence of a turn signal (also called a "turn signal") and whether the left or right turn signal is lit / flashing, and lamp lighting information indicating the lit / flashing status of the above-mentioned lamps.

[0024] Furthermore, the information from the various information acquisition units mentioned above also includes, for example, information from an ambient light sensor that detects light outside the vehicle (illuminance signal, chromaticity signal, etc.), video information from a camera mounted on the vehicle, signals from a distance sensor that detects the distance between the vehicle 10 and other vehicles and objects traveling in the surrounding area, such as in front of the vehicle, and signals from an infrared sensor that detects the external conditions of the vehicle at night.

[0025] Furthermore, the information from the above-mentioned communication unit includes, for example, GPS (Global Positioning System) signals for determining the position of the vehicle 10, so-called navigation information from a navigation device that provides route guidance, and information from vehicle-to-vehicle communication between vehicles and roads, as well as vehicle-to-infrastructure communication between vehicles and roads.

[0026] Figure 4 shows a more detailed configuration of the light distribution control ECU 40 and its surrounding elements described above. Specifically, in Figure 4, signals from the turn signal sensor 51, the steering angle sensor 52 that detects the steering angle, the shift position sensor 53, the vehicle speed sensor 54, the accelerator operation sensor 55, the brake operation sensor 56, the illuminance sensor 57, the chromaticity sensor 58, the engine start sensor 59, and the hazard lamp sensor 60 are input to the light distribution control ECU 40. Furthermore, it is equipped with a door handle sensor 68 that detects when the driver or passenger is gripping the door handle, a door lock sensor 69 that detects the open / closed state of the door, including a partially closed door, a human presence sensor 70 that detects, for example, the absence of a person inside the vehicle from the weight on the seat, and a transmission sensor 71 that detects the position of the transmission gear. Signals from the camera 61 are input to the ECU 40 via the image processing unit 62, and signals from the GPS receiver 63 and the map information output unit 64 are input to the ECU 40 via the calculation unit 65.

[0027] Furthermore, the projector 100, which constitutes the above-mentioned video projection device 500, receives control signals from the light distribution control ECU 40 and signals from the projection signal output unit 110 (video signals to be projected onto the road surface, etc.) via the control unit 120, thereby performing the projection of video onto the road surface, etc., as described below.

[0028] In addition, the above-mentioned light distribution control ECU 40 also receives signals from the headlight sensor 66 and the high / low sensor 67.

[0029] <Image projection device> Next, a more detailed example of the configuration of the video projection device 500, including the projector 100, projection signal output unit 110, and control unit 120 shown in Figure 4, will be described below with reference to Figure 5.

[0030] The projection optical system 501 is an optical system that projects an image onto a road surface or the like, and includes lenses and / or mirrors. The display element 502 is an element that generates an image by modulating transmitted or reflected light, and uses, for example, a transmissive liquid crystal panel, a reflective liquid crystal panel, or a DMD (Digital Micromirror Device: registered trademark) panel. The display element drive unit 503 sends a drive signal to the display element 502, causing the display element 502 to generate an image. The light source 505 generates light for image projection and uses, for example, a high-pressure mercury lamp, a xenon lamp, an LED light source, or a laser light source. The power supply 506 supplies power to the light source 505. Furthermore, the power supply 506 supplies the necessary power to each other part. The illumination optical system 504 focuses the light generated by the light source 505 and irradiates the display element 502 with more uniform light. The cooling unit 515 cools each component that becomes hot, such as the light source 505, power supply 506, or display element 502, using air cooling or liquid cooling as needed. The operation input unit 507 is a receiver for operation buttons and remote controls, and receives operation signals from the user.

[0031] The video signal input unit 531 receives video data by connecting an external video output device. The audio signal input unit 533 receives audio data by connecting an external audio output device. The audio output unit 540 can output audio based on the audio data input to the audio signal input unit 533. The audio output unit 540 may also output built-in operation sounds and error warning sounds. The communication unit 532 connects to, for example, an external information processing device and inputs and outputs various control signals.

[0032] The non-volatile memory 508 stores various data used by the projector function. The data stored in the non-volatile memory 508 includes pre-prepared image data and video data for projection onto the street. Memory 509 stores the video data to be projected and control parameters for each part of the device. The control unit 510 controls the operation of each connected part.

[0033] The image adjustment unit 560 performs image processing on video data input by the video signal input unit 531, image data stored in the non-volatile memory 508, and video data. Examples of such image processing include scaling, which involves enlarging, reducing, and transforming images; brightness adjustment, which changes the brightness; contrast adjustment, which changes the contrast curve of an image; and retinex processing, which decomposes an image into its light components and changes the weighting of each component.

[0034] The storage unit 570 records video, images, audio, and various data. For example, video, images, audio, and various data may be pre-recorded at the time of product shipment, or video, images, audio, and various data acquired from external devices or external servers via the communication unit 532 may be recorded. The video, images, and various data recorded in the storage unit 570 can be output as projected video via the display element 502 and projection optical system 501. Audio recorded in the storage unit 570 can be output as audio from the audio output unit 540.

[0035] As explained above, the video projection device 500 can be equipped with various functions. However, the video projection device 500 does not necessarily need to have all of the above-mentioned configurations. Any configuration is acceptable as long as it has the function of projecting images.

[0036] Figure 6 is a ray diagram of the projector, including the image plane. In the figure, the image light emitted from a light source (such as an LED, not shown) and transmitted through the image display element passes through filters, etc., is refracted by various lens systems, and then, depending on the configuration, is reflected before being projected onto the image plane 8 (road surface, etc.).

[0037] Thus, with the aforementioned image projection device 500, the projection distance is 700 mm, and the length of the longer side of the projected image is 10061 - 542 = 9519 ≈ 9520 mm, resulting in a projection ratio of 700 / 9520 = 0.07, achieving an unprecedentedly wide angle.

[0038] In addition, although the above describes a single image projection device 500 and its projection optical system, as mentioned above, in the present invention, one or more projectors (for example, a pair) may be mounted on a vehicle (or integrated into the headlights or taillights) to project the desired image onto the road surface or the like. In particular, as shown in Figures 1(A) and 2(A) above, when multiple image projection devices 500 (for example, a pair) are used, the same image may be projected onto the road surface or the like from each image projection device 500 (in which case the same image will be displayed on the display element 502 in Figure 5), or different images may be projected from the left and right image projection devices 500 and combined on the road surface or the like (in which case the display element 502 in Figure 5 will display an image in which the desired image has been split left and right).

[0039] Furthermore, while the above description described a configuration using a transmissive liquid crystal display element as the image projection device 500 for projecting images onto a road surface or the like, the present invention is not limited to this. Other types of image projection devices 500 can also be used, such as a reflective image projection device 500 consisting of micromirrors, such as a DLP (Digital Light Processing) device, or an image projection device 500 capable of projecting image light from a light-modulating planar light-emitting diode via a projection optical system. In other words, the present invention only requires that the image projection device 500 be capable of projecting a desired image onto a road surface or the like.

[0040] <Projected images of various information to be displayed on road surfaces, etc.> Next, we will explain in detail, with reference to Figures 7-26, specific examples of various images projected onto the road surface, etc., in relation to vehicle information, using the image projection device 500, which has been described in detail above, as an example, mounted on the front and / or rear of the vehicle body as described above.

[0041] The video projection devices 500 installed at various locations on the vehicle 10 as described above can display not only the status of the vehicle 10 as described above, but also the intentions of the driver of the vehicle 10.

[0042] Figure 7 shows an example of displaying a message to other nearby vehicles when the vehicle 10 is stopped, for example, at an intersection. In this example, a projected image 203, for example, "Please turn right first," is displayed in front of the vehicle 10 to an oncoming vehicle 10' that is indicating a right turn by turning its turning lights on.

[0043] In addition, the display is prepared by storing the image to be displayed in advance in the memory 509 (see Figure 5), which is a storage means constituting the above-mentioned light distribution control ECU, and then recalling it by, for example, a switch provided on the dashboard, and displaying it with the image projection device 500. In this case, as also shown in Figure 10, it is preferable to display the content in the opposite direction to normal in order to make it easier for the drivers of other oncoming vehicles 10' to understand.

[0044] Furthermore, when displaying the above message, to make it easy to see which vehicle is displaying the message, the message display area may be made trapezoidal or triangular, as shown in Figure 8, or an arrow 204 indicating the direction of the displaying vehicle or the vehicle's license plate number 205 may be included in part of the projected image 203. Alternatively, although not shown here, animation may be used.

[0045] Alternatively, as shown in Figure 9, in addition to displaying the projected image 203, its content may be pronounced as audio 250 and displayed in a combined manner.

[0046] Furthermore, as shown in Figure 10, the projected image 203 may be displayed in front of another vehicle 10'.

[0047] Furthermore, as shown in Figure 11, the display of the above-mentioned messages, etc., can also be done in the same manner to pedestrians, etc., who are attempting to cross in front of the vehicle 10. In this case, it is possible to transmit the messages, etc., to the other party more effectively by not only displaying the messages 203, etc., as video, but also by transmitting them via sound from a speaker, etc., installed in the vehicle 10. In this case, for example, an audio signal that has been stored in advance in the memory 509 (see Figure 5) may be used, or the voice 250 of the driver of the vehicle 10 may be transmitted through a microphone installed on the dashboard, etc.

[0048] Furthermore, as shown in Figure 12, the above message can be displayed as a message 206 showing advertising content, particularly on the road surface adjacent to the rear door, especially if the vehicle 10 is a taxi. Alternatively, although not shown here, in vehicles providing public services such as buses and trains, the advertising content message 206 can be displayed at the entrance or exit.

[0049] In addition, the display of the advertisement message 206 described above is performed, for example, as shown in Figure 13(A), by the ECU 40 selecting from multiple advertisements stored in RAM 42 upon receiving a stopped signal from the vehicle speed sensor 54 shown in Figure 4, and displaying them via the video projection device 500. In this case, only one or a predetermined number of different advertisements may be displayed each time the vehicle stops, or multiple advertisements may be displayed sequentially for a predetermined amount of time.

[0050] Alternatively, as shown in Figure 13(B), a router 1000 may be provided so that the ECU 40 can acquire advertising information from outside the vehicle 10 via WiFi and display the acquired advertising content.

[0051] Furthermore, especially when displaying an advertisement as message 206, as shown in Figure 14, for example, using the configuration shown in Figure 13(B), the vehicle 10 can receive advertisements via Wi-Fi from advertisers such as department stores located nearby, and display their content as message 206 on the road surface, etc.

[0052] Furthermore, as shown in Figure 15, instead of the advertisements mentioned above, it is also possible to use the navigation information from the communication unit in Figure 3 to display information that the driver of the vehicle 10 wants to know (so-called navigation information), such as "15km to destination," as a message 206 on the sidewalk, lane, road surface, etc. Displaying information on the road surface, including sidewalks and lane, can prevent misleading drivers of vehicles other than the vehicle 10.

[0053] Furthermore, as shown in Figure 16, the information that the driver wants to know can also be displayed as a message 206, which includes so-called navigation information (the direction of travel of the vehicle 10 and the distance to the destination), vehicle information such as the vehicle's speed and fuel level, alerts, and even information related to the driver's mobile device, such as a smartphone, including incoming emails.

[0054] Furthermore, as shown in Figure 17, warning information such as "Caution! Broken-down vehicle" can also be displayed as message 206. In this case, as also shown in Figure 23, since this information is a warning to the driver of another vehicle 10' coming from the opposite direction, it is preferable to display message 206 in front of the other vehicle 10' in the oncoming lane. To achieve this, it is possible to use the wide-angle display of the image projection device 500 or to change the direction of the image projection device 500.

[0055] In addition, the following are also possible contents that can be projected and displayed by the aforementioned video projection device 500.

[0056] While drivers perform various measurements visually during driving, Figure 18 shows an example of a display that assists such measurements, in which a ruler (measuring stick) is projected onto the front of the vehicle 10 as an image 207. In this example, the driver of the vehicle 10 can easily measure the distance (following distance) to another vehicle 10' traveling ahead of it using the ruler (measuring stick) 207 displayed on the front of the vehicle 10, thereby contributing to safe driving.

[0057] Furthermore, Figure 19 shows a display image 207' that enables easy measurement of the height when passing under a structure with height restrictions, such as a bridge or tunnel (a bridge-like object S). Note that Figure 19 shows the vehicle 10 traveling towards the object S as viewed from the rear.

[0058] In this height measurement display, image projection devices 500 (see Figure 1(A)) installed on the headlights on both sides (left and right) of the front of the vehicle 10 project roughly "T" shaped bar images 207', which are a combination of a vertical bar and a horizontal bar, upward toward the road surface in the direction of travel. These pair of bar images 207' are set so that the backs of the vertical bars are separated from each other at a distance greater than a predetermined distance (d > d), and the horizontal bar is set to be at a predetermined height (for example, a height that the vehicle 10 can safely pass) at that predetermined distance (d).

[0059] As a result, if the vehicle 10 is at a distance greater than the set predetermined distance (distance > d), the pair of bar images 207' are displayed separated from each other, as shown in Figure 19(A). Subsequently, when the vehicle 10 reaches the set predetermined distance (distance = d), the pair of "T" shaped bar images 207' merge to form a cross shape, as shown in Figure 19(B). At this time, the horizontal bar of the "T" bar image 207' indicates the height at which the vehicle 10 can safely pass, and the driver can determine whether or not it is possible for their vehicle to pass safely by checking the position of the horizontal bar of this cross-shaped bar image 207'. After that, as shown in Figure 19(C), the pair of "T" shaped bar images 207' are displayed overlapping each other.

[0060] Furthermore, the road surface on which the vehicle 10 displays various information as video may be an unpaved road surface with uneven surfaces, as shown in Figure 20. In that case, as also shown in Figure 20, a road surface sensor 15 such as a camera (for example, an infrared stereo camera) is attached to the front of the vehicle 10 (in the direction of travel), and the condition of the road surface (shape, reflectivity, etc.) is determined from the image captured by the camera. Based on the results, the image projected onto the road surface is corrected. This makes it possible to display an easy-to-view and highly visible image that has been corrected for the shape and reflectivity of the road surface.

[0061] Furthermore, by utilizing the road surface sensors 15 such as the camera mentioned above, the ECU 40 can also determine whether to display or hide the video according to the flow shown in Figure 21. In the example in Figure 21, the video projection device 500 is started (step 210), the lane lines on the left and right of the vehicle 10 are read from the video from the camera to measure the lane width and detect the road width (step 211), and it is determined whether there is enough road width to display the video (step 212). If it is determined in step 212 that there is enough road width to display the video, the video is displayed (step 213), and if it is determined in step 212 that there is not enough road width to display the video, the video is not displayed (step 214).

[0062] Alternatively, by utilizing the road surface sensor 15 such as the camera mentioned above, as shown in Figure 22, the distance to the projected image 206 can be set to a small value in narrow alleys, etc. (see Figure 22(A)), and the distance to the projected image 206 can be increased in wide roads (see Figure 22(B)). This makes it possible to display an image with excellent visibility. The shaded area in the figure indicates the projection area of ​​the image projection device 500.

[0063] Furthermore, as shown in Figure 23, the aforementioned camera can be mounted on the rear of the vehicle 10, and the image of another vehicle 10' behind it, obtained by the camera, can be projected and displayed in front of the vehicle 10. This would be convenient, as it would allow the driver to easily check the situation behind the vehicle 10 without having to turn around, especially when the vehicle 10 is stopped.

[0064] In addition, as shown in Figure 24, the ECU 40 can also display a warning to the following vehicle based on the video footage of the following vehicle 10' obtained from a camera mounted on the rear of the vehicle 10, according to the distance between the vehicles, the driving speed, and a threshold. Specifically, if the driving speed indicates that the distance between the two vehicles is too close, for example, a red arrow will be displayed on the rear of the vehicle 10 (see Figure 24(A)). If the distance is not that close but is still within a distance that requires caution, for example, a yellow arrow will be displayed (see Figure 24(B)). If the distance is deemed sufficient, no warning will be displayed (see Figure 24(C)).

[0065] In the various examples described above, the image light from the image projection device 500 was described as being projected onto the road surface or the like around the vehicle 10. However, the present invention is not limited thereto, and for example, it is also possible to highlight certain objects within the projection area of ​​the image projection device 500.

[0066] As an example, as shown in Figure 25, the ECU 40 detects an obstacle in the direction of travel (a tree in this example) based on video signals from road sensors such as a camera mounted in front of the vehicle 10 (see Figures 25(A) and (B)). When the video projection device 500 projects video light, it selectively displays only the portion of the obstacle (the trunk of the tree in this example) by highlighting it, for example, with blinking, to draw the driver's attention (see Figure 25(C)). In Figure 25(B), the vertical display range of the video light 206 projected from the video projection device 500 near the tree is shown with a light shade, and in Figure 25(C), the area where blinking is turned ON is shown with a dark shade. If the ECU 40 does not determine that there is an obstacle, blinking is turned OFF (see Figure 25(D)).

[0067] Furthermore, as shown in Figure 26, the vehicle 10 can pinpoint and highlight pedestrians and oncoming vehicles 10' detected by infrared illumination, changing the display color (for example, to red) only the corresponding portion of the image light 206. Other methods of highlighting include modulating or flashing the brightness of the image light projected onto and / or around obstacles. Animation may also be used. The highlighted obstacles (for example, trees) will not move even if the vehicle 10 moves. Similar highlighting may also be applied to dangerous objects such as manhole covers and wheel chocks, not just obstacles.

[0068] In the above explanation, the display of various types of information by the video projection device 500 (see Figure 5) was described as being performed as needed. However, in particular, information displayed in front of and behind the vehicle 10 cannot be effectively displayed if the distance to other vehicles in front or behind is not sufficient. Therefore, although not shown here, it is possible to detect the distance between vehicles using the cameras and sensors mentioned above, and to display the projected image only when a sufficient distance is maintained, and to stop the display if the distance is closer than a predetermined distance.

[0069] Furthermore, in relation to Figure 20 mentioned above, the details of how to correct for the shape and reflectivity of road surfaces and other elements to display clear and highly visible images are described below.

[0070] First, in response to distortions (unevenness) in the road surface shape, for example, as shown in Figures 27(A) and (B), a video projection device mounted in front of the vehicle 10 projects an image 201 onto the road surface, in which a grid-like pattern is inserted into the projected image 200. On the other hand, a camera 61 mounted on the vehicle 10 captures an image of the area in front of the vehicle, and the ECU 40 described above performs predetermined image processing on this image to detect distortions in the road surface. By correcting the projected image in accordance with the detected distortions in the road surface, it is possible to obtain a distortion-free, easy-to-view, and highly visible image despite the distortions in the road surface. However, since this requires a predetermined amount of time, it is difficult to perform such processing in real time.

[0071] Therefore, as shown in the flowchart in Figure 28, the distortion correction process described above is switched ON or OFF as appropriate according to the vehicle's speed. In addition, the type of projected image is also used as a condition for switching. This is because, for example, if the displayed information is text, the driver's attention will be diverted to deciphering it, which is undesirable for safe driving.

[0072] Specifically, as shown in the diagram, upon receiving the road projection ON signal (S2801), the system determines whether the vehicle is stopped or not (S2802). If the vehicle is stopped ("YES"), the distortion correction process described above (details below) is performed. On the other hand, if the vehicle is not stopped ("NO"), the system further determines whether the vehicle's speed is less than the set speed (S2803). If the speed is less than the set speed ("YES"), the distortion correction process described above is performed. On the other hand, if the speed is greater than the set speed ("NO"), the system determines whether the projected image is text or not (or whether it contains text or not) (S2804). If the projected image is text ("YES"), the road projection ON signal is switched to OFF (S2805). If the projected image is not text ("NO"), the road projection ON signal is not switched (S2806).

[0073] Furthermore, in the distortion correction process that is executed when the vehicle is stopped ("YES") in the above determination (S2802), and when the driving speed is less than the set speed ("YES") in the above determination (S2802), first, a distortion detection image, that is, an image 201 in which a grid pattern is inserted into the above-mentioned projected image 200, is projected (S2807). Then, the above-mentioned camera captures a comparison between the grid pattern inserted into the projected image 200 and the grid pattern in the image captured by the camera (S2808). Subsequently, if the determination (S2809) determines that the distortion of the grid pattern in the image captured by the camera is greater than a predetermined value (threshold) ("YES") (i.e., the distortion is too great to correct), the above-mentioned road surface projection ON signal is switched to OFF (S2810), that is, road surface projection is stopped. On the other hand, if the distortion is determined to be smaller than a predetermined value (threshold) ("NO"), correction is performed on the road surface projection image (S2811), the road surface projection ON signal is not switched (S2812), and road surface projection is performed. The above-described processing can be carried out, for example, by the CPU (Central Processing Unit) 41 of the light distribution control ECU 40 shown in Figure 4 above.

[0074] As is clear from the above, in the above embodiment, the image is basically projected only at a speed that ensures safe driving, including when the vehicle is stopped, and in particular, the display of the image is suppressed if the projected image contains text. Furthermore, if the distortion on the road surface onto which the image is projected is too large (distortion amount > threshold), the projection of the image onto the road surface is stopped. This is because if the amount of distortion is large, the image distortion for subjects (observers) with different viewpoint positions in the corrected image will also increase, so the display of the image is deliberately stopped.

[0075] Furthermore, the brightness and color of the projected image may deviate from what was intended due to the color of the road surface and patterns such as pedestrian crossings. In such cases, the driver's intentions when projecting the image onto the road may not be properly conveyed to those around them. Therefore, in the embodiment described in detail later, by detecting the uneven distribution of color and illuminance (so-called unevenness) of the projected image on the road surface, brightness correction and color balance correction are performed at each location on the road surface where the image is projected. This makes it possible to project the image intended by the driver onto the road surface by obtaining an image that is easy to see and has excellent visibility.

[0076] Regarding the distortion (unevenness) of the road surface shape described above, specifically, in addition to the distortion detection video shown in Figure 27 above, i.e., the video 201 with a grid-like pattern inserted, which is projected onto the road surface from a video projection device mounted in front of the vehicle 10 (see Figure 29(A)), an all-white image (an image containing R, G, and B color light), which is not shown here, may also be projected onto the road surface. This all-white image may be projected intermittently onto the road surface together with the video 201 with the grid-like pattern inserted, and may be used for brightness correction, as will be described later.

[0077] Furthermore, according to this, it becomes possible to detect the brightness distribution from the illumination unevenness on the road surface and the illumination balance of each color from the color unevenness from the image captured by the camera 61 (see Figure 27 above) as described above. Based on these detection results, brightness correction and color balance correction are performed at each location on the road surface from which the image is projected, making it possible to display the image intended by the driver on the road surface. In the above description, an image of all white is displayed intermittently along with the image 201 with a grid pattern inserted, but the present invention is not limited to this, and it will be obvious to those skilled in the art that the same effect can be obtained by, for example, inserting R, G, and B colored light, for example, in order or intermittently.

[0078] Furthermore, if the aforementioned image projection device is tilted relative to the road surface, which is the projection surface, and the image is displayed by so-called oblique projection, the size of the projected image will change significantly depending on the relationship between the image projection device and the road surface, that is, the relative angle (tilt angle) with the road surface, causing the projected image to be distorted. For this reason, it is preferable that the image projection device and the road surface are parallel to each other.

[0079] For example, when the vehicle 10 is parallel to the road surface, the grid pattern inserted into the projected image is displayed without distortion (in its original shape), as shown in Figure 29(A). In contrast, as shown in Figure 29(B), when the vehicle 10 is tilted relative to the road surface, particularly to the left in the lateral direction relative to the direction of travel, the image projected onto the road surface from the image projection device is deformed and distorted, deviating from its original shape. Similarly, when tilted to the right, the image is distorted, as shown in Figure 29(C). It should be noted that while a grid pattern was described here as the image inserted into the projected image, it goes without saying that road surface distortion can be detected in a similar manner by using a rectangular image or a simple rectangular frame instead.

[0080] Furthermore, as shown in Figure 30, the vehicle 10 is normally required to be positioned parallel to the road surface even in the direction of travel (see Figure 30(A)). However, there are also cases where it may tilt forward (see Figure 30(B)) or backward (see Figure 30(C)). Even when the vehicle body is tilted forward or backward in this way, the image projected onto the road surface from the image projection device will be scaled backward or enlarged forward from its original shape (rectangle), and will still be distorted due to deformation.

[0081] Therefore, for example, before displaying an image, such as when the vehicle starts up, the aforementioned rectangular or grid-shaped image patterns can be inserted, and the image projected onto the road surface can be captured by the aforementioned camera 61 (see Figure 27 above) and subjected to predetermined image processing. This allows for the detection of the tilt of the image projection device, and furthermore, the tilt of the vehicle itself relative to the road surface (tilt in the direction of travel and lateral direction). By correcting the tilt of the image projection device and the road surface based on the tilt of the vehicle itself relative to the road surface detected in this way, it is possible to obtain an image that is distortion-free, easy to see, and has excellent visibility. Furthermore, the tilt of the vehicle itself relative to the road surface detected in this way can also be used for controlling the vehicle's attitude by actuators installed inside the vehicle, similar to the auto-leveling function of the headlights.

[0082] Furthermore, Figure 21 above shows a flow in which the ECU 40 makes a decision on whether to display or hide the video using road surface sensors such as a camera. There, it is determined whether there is sufficient road width to display the video, and if it is determined that there is sufficient road width to display the video, the video is displayed, and on the other hand, if it is determined that there is not sufficient road width to display the video, the video is not displayed. However, the present invention is not limited to this, and the following displays are also possible.

[0083] More specifically, before projecting the desired image onto the road, for example, as shown in Figure 31(A), in places where the road is sufficiently wide, an image 201 with a grid pattern inserted can be projected, and by comparing the projected image with the actual image using a camera, it becomes possible to detect the presence of walls and obstacles on the left and right sides of the road surface. However, in practice, as shown in Figure 31(B), when detected obstacles such as walls overlap with the image to be projected, it is possible to hide the image, or as shown in Figure 31(C), reduce the size of the projected image.

[0084] Furthermore, Figure 32 shows what happens when encountering an oncoming vehicle 10' (obstacle) in a narrow alley or similar location. Note that, as indicated by the arrows in the figure, the area of ​​the image projected from the image projection device (see the thick line in the figure) is limited to the area that can be displayed on the road (see the dashed line in the figure), or it is displayed at a reduced size.

[0085] Figure 33 shows an example of the processing flow in the case described above. First, upon receiving the road surface projection ON signal (S3301), a so-called road width / obstacle detection image, such as the grid-like pattern described above, is inserted into the projected image and projected onto the road surface from the image projection device (S3302). Subsequently, the image projected onto the road surface is photographed by the camera described above, and it is determined whether the width of the photographed projected image is greater than the width of the road (S3303). If it is determined that the width of the projected image is greater than the width of the road ("YES"), the road surface projection of the image is stopped (OFF) (S3304).

[0086] On the other hand, if it is determined that the width of the projected image is smaller than the width of the road ("NO"), the image captured by the camera is used to determine whether or not there is an obstacle on the road surface (S3305). If, as a result, there is no obstacle on the road surface ("NO"), the road surface projection of the image is performed (ON) (S3306). On the other hand, if there is an obstacle ("YES"), it is further determined whether or not it is possible to display the image while avoiding the obstacle (S3307). If it is possible to display the image while avoiding the obstacle ("YES"), the projection position on the road surface is corrected (S3308), and then the road surface projection is performed (ON) (S3309). The above processing can be performed by the CPU (Central Processing Unit) 41 of the light distribution control ECU 40, as shown in Figure 4 above.

[0087] Furthermore, as mentioned in Figure 24 above, the system displays warnings to following vehicles based on the distance, speed, and threshold values ​​of the vehicle following it. The following provides further details on this system.

[0088] Generally, a safe following distance is said to be the driving speed minus 15 meters, up to 60 km / h, in order to stop after detecting danger. Therefore, the vehicle's driving speed is calculated from the vehicle speed pulse, and the distance to the following vehicle is calculated from a camera mounted on the rear of the vehicle. If the following distance is insufficient, a red arrow is displayed in the projected image to warn the following vehicle. If, for example, there is only 10% of the required following distance remaining, a yellow arrow is used to draw attention. Alternatively, the following distance to the following vehicle may be detected using a laser radar instead of the camera mentioned above.

[0089] Furthermore, the stopping distance of a vehicle changes depending on the road surface conditions. Generally, during rainfall, 1.5 times the normal following distance is required, and when the road surface is icy, 3 times the normal following distance is required. Therefore, it is desirable to change the setting of the required following distance calculated from the aforementioned driving speed according to the surrounding environment. For example, rainfall can usually be detected using an infrared sensor, specifically by detecting changes in reflection due to raindrops adhering to the window glass. Road surface icing can be detected by the specular reflectivity of the road surface. That is, on a normal road surface, the diffuse reflection component due to surface irregularities is strong, but when it is icy, a film of ice forms on the surface, so the specular reflection component becomes strong. As a result, for example, the specular reflection of headlights from oncoming vehicles becomes stronger, so it is possible to detect the icy condition by detecting the amount of reflected light from the road surface. This data on safe / cautionary following distances should be stored in memory as a table beforehand.

[0090] In the above explanation, we have described how, in particular, if the distance between your vehicle and another vehicle behind it is insufficient, an effective display cannot be provided. Therefore, we have described how the system detects the distance between vehicles and displays the projected image only when a sufficient distance is maintained, and stops the display if the distance is closer than a predetermined distance. However, as shown in Figure 34, an effective display cannot be provided if there is insufficient distance between your vehicle 10 and the other vehicle 10' in front of it. Figure 34(A) shows the case where there is sufficient distance, Figure 34(B) shows the case where the distance is insufficient, and Figure 34(C) shows the case where there is almost no distance between vehicles.

[0091] Here, Figure 35 shows an example of the processing flow for issuing the warning to the following vehicle as described above. In the figure, first, the distance between the vehicle and the following vehicle is detected (S3501). Next, the driving speed of the vehicle is detected (S3502). Furthermore, the presence or absence of rain is detected by the rain sensor described above (S3503). In addition, road surface freezing is detected by a road surface freezing sensor based on the principle described above (S3504). Subsequently, based on these detection results, the necessary safe / caution distance is searched from the table described above and set accordingly (S3505).

[0092] Subsequently, it is determined whether the distance between vehicles detected above is smaller than the safe distance set above (S3506). If it is determined that the distance is larger than the safe distance ("NO"), it is further determined whether it is smaller than the caution distance (S3507). On the other hand, if it is determined in this determination (S3506) that the distance is smaller than the safe distance ("YES"), the warning video for following vehicles described above is projected onto the road surface (ON) (S3510).

[0093] On the other hand, if the above determination (S3506) is determined to be greater than the safe following distance ("NO"), but the determination (S3507) is determined to be less than the caution following distance ("YES"), then instead of the above warning video, a caution video for following vehicles is projected onto the road surface (ON) (S3508). Furthermore, if the above determination (S3506) is determined to be greater than the safe following distance ("NO"), and the determination (S3507) is also determined to be greater than the caution following distance ("NO"), then the display of the safety / caution information for following vehicles as described above is stopped (OFF) (S3509). Note that the above processing can be carried out by the CPU (Central Processing Unit) 41 of the light distribution control ECU 40 as shown in Figure 4 above.

[0094] In addition to the warning to following vehicles mentioned above, in order to project an image, there must be at least the space of one vehicle; otherwise, depending on the image to be displayed, it may not be possible to display it properly. Therefore, it was found that it is desirable to hide the image display when the distance to the vehicle in front and the vehicle behind is too narrow for the displayed image. For example, if a 4m wide image is displayed on the road surface 10m away, a person who is 170cm tall will see it as a 600mm square image at 10m away. In this case, if the person has a visual acuity of 0.1 or better without glasses, they can recognize an image of 30mm or larger at 10m away, which means that it is possible to display an image with a resolution of 16x16 or higher, which is necessary for displaying text.

[0095] Based on the above-mentioned findings, an example of a processing flow for projecting images onto the road surface is shown in Figure 36. In the figure, first, upon receiving a road surface projection ON signal (S3601), it is determined whether or not the projected image will be displayed in front of the vehicle (S3602).

[0096] If, as a result of the above determination, it is determined that the displayed image is the image displayed in front of the vehicle itself ("YES"), the next step is to detect the distance between the vehicle in front and the vehicle itself (S3603). Next, it is determined whether the size of the image projected in front of the vehicle itself is greater than the distance detected above (S3604). If the size of the image is greater than the distance ("YES"), the projection of the image onto the road surface is stopped (OFF) (S3605). On the other hand, if it is smaller than the distance ("NO"), the projection of the image onto the road surface is started (ON) (S3606).

[0097] On the other hand, if the above determination (S3602) determines that the displayed image is an image displayed behind the vehicle ("NO"), the distance between the following vehicle and the vehicle is detected (S3607). Next, it is determined whether the size of the image projected behind the vehicle is larger than the distance detected above (S3608). As a result, if the size of the image is larger than the distance ("YES"), the projection of the image onto the road surface is stopped (OFF) (S3609), while if it is smaller than the distance ("NO"), the projection of the image onto the road surface is performed (ON) (S3510).

[0098] In the embodiments described above, for example, grid patterns, rectangular images, or simple rectangular frames were described as images inserted into the projected image for detecting road surface distortion or obstacle detection. In such cases, it is preferable to use light with a spectral intensity called "AM1.5," which is light in the wavelength band centered on a wavelength of 1.4 μm. This is because, outside the atmosphere, some wavelength components of sunlight are absorbed by components in the atmosphere and do not reach the ground, and the spectral intensity called "AM1.5" has some spectral components reduced or removed, and in particular, the intensity in the wavelength band centered on a wavelength of 1.4 μm in the near-infrared region is almost zero (0). In other words, on the ground where the vehicle is traveling, the above AM1.5 is not included in sunlight. That is, by using such light, it is possible to stably obtain road surface condition information without being affected by sunlight.

[0099] In addition, the brightness correction of the projected image, as mentioned above, will be explained in detail below with reference to Figure 37. The flowchart in this figure can be performed by the CPU (Central Processing Unit) 41 of the light distribution control ECU 40, as shown in Figure 4 above, in the same manner as described above.

[0100] First, upon receiving a road surface projection ON signal (S3701), the system then projects a road surface illuminance detection image onto the road surface (S3702). This road surface illuminance detection image is the all-white image (an image containing R, G, and B colored light) described above. Furthermore, during this all-white display, the system detects the non-uniformity (so-called variation) of the road surface illuminance from the image captured by the camera (S3703). Subsequently, the detected illuminance variation (or each value) is compared with a predetermined threshold (S3704).

[0101] If the illuminance variation is greater than the threshold ("YES") based on the above comparison, it is further determined whether the illuminance variation is greater than the limit value (S3705). The limit value is the maximum value within which correction is possible by the light source of the image projection device. If it is determined that the illuminance is greater than the limit value ("YES"), the projection of the image onto the road surface is stopped (OFF) (S3706).

[0102] On the other hand, if it is determined to be smaller than the limit value ("NO"), the illuminance of the image projected onto the road surface is corrected (S3707), and the projection of the image onto the road surface is performed (ON) (S3708), similar to when the variation in illuminance is smaller than the threshold value ("NO") in the above comparison (S3704).

[0103] According to the above embodiment, by obtaining an image that is easy to see and has excellent visibility, it becomes possible to reliably and clearly project the image intended by the driver onto the road surface. Furthermore, by using a pattern of single colors, such as red (R), green (G), and blue (B), instead of the all-white image described above as the road surface illumination detection image, and correcting the brightness of the projected image (for example, sequentially), it becomes possible to detect variations in each color, thereby enabling color shift correction.

[0104] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are included. For example, the embodiments described above are detailed explanations of the entire system in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Furthermore, it is possible to replace parts of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add configurations from other embodiments to the configuration of one embodiment. In addition, it is possible to add, delete, or replace parts of the configuration of each embodiment with other configurations. [Explanation of Symbols]

[0105] 10...Own vehicle (passenger car), 10'...Other vehicles, 11...Headlights, 12...Window area, 13, 13'...Taillights, 14...Side mirrors, 40...Light distribution control ECU, 51...Turn signal sensor, 52...Steering angle sensor, 53...Shift position sensor, 54...Vehicle speed sensor, 55...Accelerator operation sensor, 56...Brake operation sensor, 57...Illuminance sensor, 58...Color sensor, 59...Engine start sensor, 60...Hazard lamp sensor, 61...Camera, 62...Image processing unit, 63...GPS receiver, 64...Map information output unit, 66...Headlight sensor, 67...High / low sensor, 68...Door handle sensor, 69...Door lock sensor, 70...Human presence sensor, 71...Transmission sensor, 100...Projector, 110...Projection signal output unit, 120...Control unit, 500...Image projection device, 501...Projection optical system, 502...Display element, 503...Display element drive unit, 504...Illumination optical system, 505...Light source, 531...Video signal input unit, 533...Audio signal input unit, 532...Communication unit.

Claims

1. It is a vehicle, Information acquisition unit, An image projection unit that projects an image, Equipped with, The acquisition unit is a sensor mounted on the vehicle, The image projection unit compares the lane width, which indicates the distance between the left and right lane lines detected by the sensor, with the width of the projected image, and projects or does not project the image based on the result of the comparison. When projecting the aforementioned image, if the vehicle's speed is lower than a preset speed, there is a state in which the projected image is corrected based on the road surface conditions and / or reflectivity detected by the sensor. vehicle.

2. In the vehicle described in claim 1, The image projection unit projects the image if the lane width is wider than the width of the image to be projected. vehicle.

3. In the vehicle described in claim 1, The image projection unit shall not project the image if the lane width is narrower than the width of the image to be projected. vehicle.

Images