Image projection device

Abstract

Provided is an image projection device which enables appropriate content display according to the traveling state or the viewpoint position of a driver. An image projection device (3) projects a projection image onto a display unit for displaying a virtual image, and comprises: an external image acquisition unit (20) that acquires an external image; an image editing unit (40) that edits the projection image on the basis of the external image; and an image illumination unit (10) that illuminates the display unit with the projection image as image light.

Description

Image projection device

[0001] The present invention relates to an image projection device.

[0002] Conventionally, as a device for displaying various information in a vehicle, an instrument panel that lights up icons has been used. In addition, along with an increase in the amount of information to be displayed, it has also been proposed to embed an image display device in the instrument panel or to configure the entire instrument panel with an image display device.

[0003] However, since the instrument panel is located below the front glass (windshield) of the vehicle, it is necessary for passengers such as the driver to move their line of sight downward during driving to visually recognize the information displayed on the instrument panel, which is not preferable. Therefore, an image projection device such as a head-up display (hereinafter referred to as HUD: Head Up Display) has been proposed that projects an image onto the front glass so that information can be read when the passenger views the front of the vehicle. (See, for example, Patent Documents 1 and 2).

[0004] A conventional image projection device irradiates irradiation light including an image from an image irradiation unit, reflects the irradiation light with a free-form surface mirror or the like, and causes an image to be formed in space through a display unit such as a windshield so as to reach the position of the passenger's viewpoint. Thereby, the passenger can recognize that an image is displayed at the imaging position in the depth direction by the irradiation light incident on the viewpoint.

[0005] Japanese Unexamined Patent Application Publication No. 2019-119248, Japanese Unexamined Patent Application Publication No. 2019-119262

[0006] In a conventional image projection device, although the display content may be changed according to the information to be transmitted, predetermined content has been displayed. However, since the situation outside the vehicle changes at any time during the running of the vehicle, it has been difficult to perform appropriate display according to the situation. In addition, even in the same vehicle, the appearance of the visual field range differs depending on the position of the driver's viewpoint, so there may be a sense of discomfort in the case of uniform content display.

[0007] Therefore, the present invention has been made in view of the above conventional problems, and an object thereof is to provide an image projection device capable of appropriately displaying content according to the driving situation and the position of the driver's viewpoint.

[0008] To solve the above problems, the present invention provides an image projection device that projects a projection image onto a display unit for displaying a virtual image, and is characterized by comprising: an external image acquisition unit for acquiring an external image; an image editing unit for editing the projection image based on the external image; and an image illumination unit for irradiating the display unit with the projection image as image light.

[0009] In this image projection device of the present invention, the view from outside the vehicle is acquired as an external image, and the projected image is edited based on the external image and illuminated with image light, making it possible to display appropriate content according to the driving conditions and the driver's viewpoint.

[0010] Furthermore, in one aspect of the present invention, an image recognition unit is provided to recognize an object included in the external image, and the external image editing unit applies a coloring process to the external image according to the object.

[0011] Furthermore, in one aspect of the present invention, the image editing unit colors only the outline of the object if the object is a direct recognition object, and fills in the entire object if it is a drawing-out object.

[0012] Furthermore, in one aspect of the present invention, the area ratio to which the coloring treatment has been applied is 50% or less of the external image.

[0013] Furthermore, in one aspect of the present invention, the image projection unit projects the projected image onto the area of ​​the display unit that overlaps with the vehicle.

[0014] Furthermore, in one aspect of the present invention, the projected image includes an external view mode based on the external image and a driving support mode in which the projected image includes content images that assist in driving, and the image projection unit switches between displaying the external view mode and the driving support mode.

[0015] Furthermore, in one aspect of the present invention, the projected image includes a distant image formed at a first distance from the display unit and a near image formed at a second distance shorter than the first distance, wherein the near image is based on the external image and the distant image includes content images that assist in driving.

[0016] Furthermore, in one aspect of the present invention, there is a depression angle acquisition unit that acquires the depression angle when viewing the projected image from the viewpoint position, and based on the depression angle, the area in the display unit that overlaps with the vehicle is calculated.

[0017] Furthermore, in one aspect of the present invention, the image editing unit adjusts the vanishing point of the projected image, which is the vanishing point of the projection image, based on the depression angle.

[0018] Furthermore, in order to solve the above problems, the present invention provides an image projection device that projects a projection image onto a display unit for displaying a virtual image, and is characterized by comprising: an image illumination unit that illuminates the display unit with the projection image as image light; a depression angle acquisition unit that acquires the depression angle when viewing the projection image from a viewpoint position; and an image editing unit that edits the projection image to adjust the vanishing point of the projection image, which is the vanishing point of the projection image, based on the depression angle.

[0019] Furthermore, in one aspect of the present invention, the system includes a viewpoint position acquisition unit that acquires the viewpoint position, and the image editing unit edits the projected image so as to bring the image vanishing point closer to the background vanishing point, which is the vanishing point of the background on which the projected image is superimposed, based on the viewpoint position and the depression angle.

[0020] The present invention provides an image projection device that can display appropriate content according to driving conditions and the driver's viewpoint.

[0021] This is a schematic diagram showing the projection of virtual images P1 and P2 using the image projection device 3 according to the first embodiment. This is a block diagram showing an example of the configuration of the image projection device 3. This is a flowchart explaining the method of illuminating a projected image using the image projection device 3. This is a schematic diagram showing an example of projecting an edited projection image using the image projection device 3 according to the first embodiment. This is a schematic diagram showing an example of the display of the driving support mode in the image projection device 3 according to the second embodiment, where Figure 5(a) shows a distant image projected as virtual image P1 and Figure 5(b) shows a near image projected as virtual image P2. This is a schematic diagram showing an example of the display of the external view mode in the image projection device 3 according to the second embodiment, where Figure 6(a) shows a distant image projected as virtual image P1 and Figure 6(b) shows a near image projected as virtual image P2. This is a schematic diagram showing the projection of virtual images P1 and P2 in the image projection device 3 according to the third embodiment, where Figure 7(a) shows a distant image without editing and Figure 7(b) shows a distant image with editing. This is a schematic diagram showing the projection of virtual images P1 and P2 in the image projection device 3 according to the fourth embodiment. Figure 8(a) shows a distant image without editing, and Figure 8(b) shows a distant image after editing.

[0022] (First Embodiment) Hereinafter, a first embodiment of the present invention will be described in detail with reference to the drawings. The same or equivalent components, members, and processes shown in each drawing will be denoted by the same reference numerals, and redundant explanations will be omitted as appropriate. Figure 1 is a schematic diagram showing the projection of virtual images P1 and P2 using the image projection device 3 according to this embodiment. As shown in Figure 1, a passenger (driver) 2 is seated in the vehicle 1, and the vehicle 1 is equipped with a windshield WS, an image projection device 3, a headlight 4, a control unit (not shown), and cameras C1, C2, and C3. In Figure 1, the dashed arrows indicate the path of the center position of the first image light L1 that projects the virtual image P1, and the solid arrows indicate the path of the center position of the second image light L2 that projects the virtual image P2.

[0023] The image projection device 3 is the part that irradiates a first image light L1 and a second image light L2 of a predetermined image shape based on image information from the control unit. The specific configuration of the image projection device 3 is not limited, and conventionally known devices such as liquid crystal display devices, organic EL display devices, and combinations of laser light sources and optical modulation elements can be used. The image projection device 3 is equipped with conventionally known projection optics such as free-form mirrors and reflectors, but these are not shown in the illustration.

[0024] In the image projection device 3, each part is controlled by a control unit that is connected to each part for information communication. The configuration of the control unit is not limited, but one example is one that includes a CPU (Central Processing Unit) for information processing, a memory device, a recording medium, an information communication device, etc. The control unit controls the operation of each part according to a predetermined program and sends information including images (image information) to the image projection unit 10.

[0025] The headlight 4 is the part that illuminates the front of the vehicle 1 with either low beam or high beam light. The specific configuration of the headlight 4 is not limited, but it must be able to emit light that is shaped into a predetermined outline. An example of the outline emitted from the headlight 4 is a low beam formed by a cutoff line. Alternatively, the headlight 4 may use ADB light distribution control to emit light as an image of a predetermined shape.

[0026] The windshield WS is a visible light-transmitting part located in front of the driver's seat of the vehicle 1. On the inner surface of the vehicle 1, the windshield WS reflects the first image light L1 and the second image light L2 incident from the image projection device 3 toward the viewpoint direction of the occupant 2, and transmits light from outside the vehicle 1 toward the viewpoint direction, thus corresponding to the display unit in this invention. Here, an example using the windshield WS as the display unit is shown, but a combiner may be prepared as a separate display unit from the windshield WS, and it may reflect light from the image projection device 3 toward the viewpoint direction. Furthermore, it is not limited to being located in front of the vehicle 1, but may be placed to the side or rear as long as it projects an image toward the viewpoint of the occupant 2.

[0027] The virtual images P1 and P2 are images that appear to be projected into space when the first image light L1 and second image light L2, reflected by the windshield WS, reach the viewpoint (eyebox) of the occupant 2. The position where the virtual images P1 and P2 are projected is determined by the angle of spread of the first image light L1 and second image light L2, emitted from the image projection device 3, as they travel in the direction of the viewpoint after being reflected by the windshield WS. The content of the images projected as virtual images P1 and P2 is not limited, but examples include warning images, emergency information, speed, volume indicators, direction of travel guides, and other auxiliary information related to driving.

[0028] As shown in Figure 1, the virtual image P1 projected by the first image light L1 is projected further away from the driver's viewpoint than the virtual image P2 projected by the second image light L2. Therefore, the image projected by the first image light L1 corresponds to the distant image in this invention, and the image projected by the second image light L2 corresponds to the near image in this invention.

[0029] Cameras C1, C2, and C3 are mounted on the vehicle 1 and each acquires images in a specific direction and field of view. The specific configuration of cameras C1, C2, and C3 is not limited, and conventionally known CCD (Charge Coupled Device) cameras, CMOS (Complementary Metal Oxide Semiconductor) cameras, etc., can be used. Furthermore, they may be monochrome cameras that capture monochrome images, or color cameras that capture color images.

[0030] In the example shown in Figure 1, camera C1 is located at the front of vehicle 1 and captures images of the area in front of vehicle 1. Camera C2 is located inside the passenger compartment of vehicle 1, near the upper end of the windshield WS, and captures images of the area in front of vehicle 1. Camera C3 is located inside the passenger compartment of vehicle 1 and captures images in the direction of driver 2. More specifically, camera C1 is located at a front position approximately the same as the headlight 4, but lower than the hood, and captures images of the road surface in front of vehicle 1, including the area immediately surrounding the vehicle, from a position lower than the driver's viewpoint. Camera C2 is located near the upper end of the windshield WS and captures images of the road surface further away than camera C1, from a position higher than the driver's viewpoint. Camera C3 is located near the upper end of the windshield WS and captures images of the driver's upper body or face as a driver image from a position higher than the driver's viewpoint.

[0031] As shown in Figure 1, the first image light L1 and the second image light L2 projected from the image projection device 3 are reflected by the windshield WS and illuminate the driver's 2 viewpoint (eyebox). The driver 2 sees virtual images P1 and P2 formed at a first and second distance from the viewpoint, on the extension of the first image light L1 and the second image light L2 that have reached the viewpoint. In addition, camera C1 captures the front of the vehicle 1 from a position lower than the hood, camera C2 captures the front of the vehicle 1 from the top of the windshield WS, and camera C3 captures the driver's 2 upper body or face.

[0032] Figure 2 is a block diagram showing an example configuration of the image projection device 3. As shown in Figure 2, the image projection device 3 includes an image projection unit 10, an external image acquisition unit 20, an image recognition unit 30, an image editing unit 40, a depression angle acquisition unit 50, and a viewpoint position acquisition unit 60. Each part of the image projection device 3 is realized by a control unit that performs calculation processing and controls various hardware according to a predetermined program.

[0033] The image illumination unit 10 is the part that, based on image information from the control unit, illuminates the windshield WS with first image light and second image light containing an image, and reflects the first image light and second image light, which are then projected onto the windshield WS, by the projection optics unit (not shown). The specific configuration of the image illumination unit 10 is not limited, and one example is the use of a liquid crystal display device.

[0034] The external image acquisition unit 20 is responsible for acquiring images of the nearby road surface captured by camera C1, images of the distant road surface captured by camera C2, and images of the driver captured by camera C3. The nearby road surface image, distant road surface image, and driver image each correspond to external images in this invention. The specific means for acquiring the nearby road surface image, distant road surface image, and driver image are not limited, and as long as cameras C1, C2, and C3 and the control unit are connected by wired or wireless connections, image information of the external images can be acquired using conventionally known communication standards, etc.

[0035] The image recognition unit 30 is the part that recognizes objects included in the external image. The specific configuration of the image recognition unit 30 is not limited, and it uses known image recognition technology to recognize objects included in the nearby road surface image, the distant road surface image, and the driver image, and acquires the region and classification information of the objects included in the image. Examples of objects include pedestrians, bicycles, motorcycles, vehicles in front, oncoming vehicles, lane markings, signs, the driver's viewpoint position, and the driver's line of sight direction.

[0036] The image editing unit 40 is the part that edits the projected image based on the external image. The method of editing the projected image is not limited, and the projected image is obtained by editing the contents of the external image by combining known image editing processes. Examples of image editing include coloring the entire area of ​​the object with a specific color, coloring the outline of the object, adjusting the vanishing point of the projected image, and changing the display position in the projected image.

[0037] The depression angle acquisition unit 50 is the part that acquires the depression angle when the driver 2 views the projected image from their viewpoint. The configuration of the depression angle acquisition unit 50 is not limited, but it may calculate the depression angle when the driver 2 views the virtual images P1 and P2 from the driver's viewpoint and the optical path of the first image light L1 or the second image light L2. In this case, the driver 2's viewpoint can be acquired from the viewpoint position acquisition unit 60. The optical path of the first image light L1 or the second image light L2 can be calculated from the illumination angle from the image projection device 3 and the inclination angle of the windshield WS, etc.

[0038] The viewpoint position acquisition unit 60 is responsible for acquiring the viewpoint position of the driver 2. The configuration of the viewpoint position acquisition unit 60 is not limited, but it may be such that the image recognition unit 30 recognizes the position of the eyes from an image of the driver 2's upper body or face captured by the camera C3, and calculates the viewpoint position from the field of view of the camera C3 and the position of the eyes in the image. In addition, the viewpoint position acquisition unit 60 may acquire not only the viewpoint position of the driver 2 but also the direction of the driver 2's gaze using eye-tracking technology.

[0039] Figure 3 is a flowchart illustrating the method of projecting an image using the image projection device 3. In the image projection device 3, the control unit executes a predetermined program and performs a set of procedures, thereby executing each step of the flowchart shown in Figure 3.

[0040] In the external image acquisition step S1, the external image acquisition unit 20 acquires a nearby road surface image, a distant road surface image, or a driver image as an external image from cameras C1, C2, and C3. Here, an example is shown in which any of the nearby road surface image, distant road surface image, or driver image is acquired as an external image, but multiple images from the nearby road surface image, distant road surface image, and driver image may be acquired simultaneously. The external image acquisition unit 20 also transmits the acquired external image to the image recognition unit 30. The external image acquisition unit 20 may also record the acquired external image on the recording medium of the control unit.

[0041] Next, in the image recognition step of step S2, the image recognition unit 30 performs image recognition processing on the external image to obtain information such as the type and position of the object included in the external image. Further, the image recognition unit 30 transmits the obtained information such as the type and position of the object to the image editing unit 40. Also, the image recognition unit 30 may record the obtained image recognition result on the recording medium of the control unit. In this case, it is preferable to record the external image data and the image recognition result in association with each other.

[0042] Next, in the projected image editing step of step S3, the image editing unit 40 performs editing processing on the external image based on information such as the type and position of the object included in the external image to obtain a projected image. Further, the image editing unit 40 transmits the projected image to the image irradiation unit 10. Also, the image editing unit 40 may record the projected image on the recording medium of the control unit.

[0043] Next, in the image irradiation step of step S4, the image irradiation unit 10 irradiates the windshield WS with the projected image as the first image light L1 or the second image light L2 to form virtual images P1, P2. The control unit of the image projection device 3 repeatedly executes steps S1 to S4 until the interruption process is executed, and continues to project the virtual images P1, P2 in the traveling state of the vehicle 1.

[0044] FIG. 4 is a schematic diagram showing an example of projecting a projected image subjected to editing processing using the image projection device 3 according to the present embodiment. In the present embodiment, a distant road surface image captured by the camera C2 is obtained as an external image, and the externally edited image is irradiated from the image projection device 3 onto the windshield WS. FIG. 4 shows an example in which the entire area of the windshield WS is irradiated with the first image light L1 or the second image light L2, but it is also possible to irradiate a part of the windshield WS with the externally edited image as the first image light L1 or the second image light L2.

[0045] As shown in Figure 4, the distant road surface image captured by camera C2 shows the road surface in the direction of travel of vehicle 1 and its surroundings. The external image acquisition unit 20 acquires the distant road surface image as an external image and transmits the distant road surface image, which is an external image, to the image recognition unit 30. The image recognition unit 30 recognizes objects T1 to T4 included in the external image using known image recognition techniques. As a result of the image recognition, the image recognition unit 30 records information about the region in the external image where objects T1 to T4 exist, as well as classification information of objects T1 to T4, on a recording medium and transmits it to the image editing unit 40.

[0046] In the example shown in Figure 4, the image recognition unit 30 acquires information such as the position and spacing of the vehicle's taillights, indicating that object T1 is a preceding vehicle and is classified as a warning object, and how many meters away it is from vehicle 1. The image recognition unit 30 also acquires information such as the content of the regulations indicated by object T2, indicating that object T2 is a road sign and is classified as a direct recognition object, by comparing it with a pre-recorded sign image. The image recognition unit 30 also acquires information such as the direction of movement of object T3, indicating that object T3 is a pedestrian and is classified as a warning object, using known human body recognition technology. The image recognition unit 30 also acquires information such as the direction of movement of object T4, indicating that object T4 is a lane mark and is classified as a non-highlighted object, using known lane mark detection technology.

[0047] The image editing unit 40 performs an editing process on the external image based on the recognition result by the image recognition unit 30 to obtain a projection image. Further, the image irradiation unit 10 irradiates the projection image toward the windshield WS. In FIG. 4, an example of a projection image subjected to a coloring process according to the objects T1 to T4 is shown as an editing process of the external image, and the colored portions are indicated by slanted hatching. For the object T1 which is a preceding vehicle and the object T3 which is a pedestrian, since they are objects to be alerted, a coloring process is performed to paint over the entire preceding vehicle and pedestrian. Also, for the object T2 which is a road sign, since it is a direct recognition target, a coloring process is performed surrounding the contour portion of the road sign. Also, for the object T4 which is a lane mark, since it is a non-emphasis target, no coloring process is performed. Here, the lane mark is assumed to be a non-emphasis target and no coloring process is performed, but a coloring process may be performed as a direct recognition target or an object to be alerted.

[0048] The specific method of the coloring process is not limited, but for pedestrians, it may be to use yellow, amber, or the like to attract attention. Also, for the vehicle body such as a preceding vehicle, light blue may be used, and for the tail lamp, red or the like may be used to enhance visibility at night or in bad weather. Also, for the lane mark, light blue or yellow may be used to enhance visibility at night or in bad weather. Also, for the road sign, red or yellow may be used according to the regulatory content indicated by the sign to enhance the visibility of the regulatory content.

[0049] Also, when performing a coloring process as an editing process on the external image, the area ratio occupied by the colored area is preferably 50% or less of the external image, more preferably 20% or less. If the area ratio of the colored area is large, the visibility of the external image subjected to the editing process may conversely deteriorate.

[0050] As described above, in the image projection device 3 of the present embodiment, the view outside the vehicle 1 is acquired as an external image, and the projection image is edited based on the external image to irradiate the first image light L1 or the second image light L2, so that appropriate content display according to the driving situation and the viewpoint position of the driver 2 becomes possible.

[0051] (Second Embodiment) Next, a second embodiment of the present invention will be described with reference to Figures 5 and 6. Content that overlaps with the first embodiment will be omitted from the explanation. In this embodiment, a nearby road surface image captured by camera C1 is acquired as an external image, and the projected image is projected from the image projection device 3 onto the area of ​​the windshield WS that overlaps with the vehicle 1. The image projection device 3 also includes an external view mode in which the projected image is based on the external image, and a driving support mode in which the projected image includes content images that assist driving, and switches between the external view mode and the driving support mode for display.

[0052] Figure 5 is a schematic diagram showing an example of the display of the driving support mode in the image projection device 3 according to this embodiment. Figure 5(a) shows a distant image projected as a virtual image P1, and Figure 5(b) shows a near image projected as a virtual image P2. As shown in Figure 5(a), in the driving support mode, a warning icon or the like is used as the distant image, and the virtual image P1 is projected from the image projection unit 10 using the first image light L1. Also, as shown in Figure 5(b), an auxiliary driving information such as emergency information, icons such as speed, volume indicator, and direction of travel guide are used as the near image, and the virtual image P2 is projected from the image projection unit 10 using the second image light L2.

[0053] Figure 6 is a schematic diagram showing an example of the external view mode display in the image projection device 3 according to this embodiment. Figure 6(a) shows a distant image projected as a virtual image P1, and Figure 6(b) shows a near image projected as a virtual image P2. As shown in Figure 6(a), in the external view mode, auxiliary information related to driving, such as emergency information, icons such as speed, volume indicator, and direction of travel guide are used as the distant image, and the virtual image P1 is projected from the image irradiation unit 10 using the first image light L1. Also, as shown in Figure 6(b), a projected image based on the external image is used as the near image, and the virtual image P2 is projected from the image irradiation unit 10 using the second image light L2.

[0054] Here, the external image in the external view mode is an image of the nearby road surface captured by the camera C1 and acquired by the external image acquisition unit 20. The image recognition unit 30 may perform image recognition processing on objects included in the nearby road surface image, which is the external image. The image editing unit 40 performs editing processing on the acquired external image to obtain a projected image. The obtained projected image is projected onto the windshield WS from the image illumination unit 10. At this time, it is preferable that the edited projected image is projected onto an area that overlaps with a part of the vehicle 1 when viewed from the driver's 2 viewpoint. An example of an area that overlaps with a part of the vehicle 1 is the hood, which is visible through the windshield WS.

[0055] In this embodiment, the image projection device 3 uses camera C3 to capture an image of the driver 2, including the driver's face, as an external image. The external image acquisition unit 20 acquires the driver image, which is an external image, and transmits it to the image recognition unit 30. The image recognition unit 30 recognizes the position of the driver 2's eyes in the external image. The viewpoint position acquisition unit 60 calculates the viewpoint position of the driver 2 inside the vehicle 1 from the eye position in the image recognized by the image recognition unit 30 and the field of view of camera C3. The depression angle acquisition unit 50 acquires the depression angle from the viewpoint position calculated by the viewpoint position acquisition unit 60.

[0056] Furthermore, the control unit calculates the position where the hood of vehicle 1 is visible based on the viewpoint position and the downward angle, and projects a near-field image onto the area of ​​the windshield WS that overlaps with the hood. Here, the virtual image P2 of the near-field image is a projected image based on the nearby road surface image captured by the camera C1. In such an image projection device 3, the blind spot area hidden by the hood, etc., from the driver's viewpoint is acquired as a nearby road surface image, and the virtual image P2 of the near-field image is projected onto the hood. This makes it possible to project an image of the blind spot area, which is difficult to see, using a part of vehicle 1 such as the hood that narrows the field of view, and to display appropriate content according to the driving conditions and the driver's viewpoint position.

[0057] Furthermore, it is preferable that the projection optics section of the image projection device 3 is equipped with a tilt mechanism and a slide mechanism on the free-form surface mirror that reflects the second image light L2, so that the irradiation position and irradiation angle of the second image light L2 on the windshield WS can be changed. This allows the driver to project a near image in a position that does not overlap with the vehicle 1 when switching between the driver assistance mode and the external view mode, and to project a near image in a position that overlaps with the vehicle 1 when switching between the driver assistance mode and the external view mode.

[0058] As described above, the image projection device 3 of this embodiment also acquires the view outside the vehicle 1 as an external image, edits the projected image based on the external image, and irradiates with the first image light L1 or second image light L2, making it possible to display appropriate content according to the driving conditions and the driver's viewpoint position.

[0059] (Third Embodiment) Next, a third embodiment of the present invention will be described with reference to Figure 7. Details that overlap with the first embodiment will be omitted from the explanation. In this embodiment, based on the driver image captured by camera C3, the viewpoint position acquisition unit 60 acquires the viewpoint position of the driver 2, and the depression angle acquisition unit 50 acquires the depression angle from the viewpoint position of the driver 2. In addition, the image editing unit 40 performs editing processing to adjust the vanishing point of the projected image, which is the vanishing point of the projected image, based on the depression angle.

[0060] Figure 7 is a schematic diagram showing the projection of virtual images P1 and P2 in the image projection device 3 according to this embodiment. Figure 7(a) shows a distant image without editing, and Figure 7(b) shows a distant image after editing. The solid lines in Figures 7(a) and 7(b) indicate the range of the outside world that the driver 2 can see from the viewpoint position through the windshield WS. The dashed rectangles shown in the figures indicate the projection range of the virtual images P1 and P2 superimposed on the outside world. The images shown within the dashed lines are examples of distant and near images projected as virtual images P1 and P2, respectively.

[0061] In this embodiment, a two-dimensional figure with a sense of depth is drawn using perspective projection as the distant image. Figures 7(a) and 7(b) show examples of pseudo-lane mark shapes that guide the driver's destination. The content and shape of the distant image are not limited; any perspective projection with a vanishing point may be used, such as arrow shapes, three-dimensional shapes, or figures with multiple shapes repeated. The perspective projection used for the distant image may be one-point perspective or two-point perspective, but it is preferable to use one-point perspective to reduce the sense of incongruity with the road on which the vehicle 1 is traveling. In addition, for the near image, planar icons such as auxiliary driving information such as emergency information, speed, volume indicator, and direction of travel guide are used.

[0062] As shown in Figure 7(a), if the distant image is not edited, the vanishing point in the three-dimensional two-dimensional figure and the vanishing point of the outside world as seen from the driver's (2) viewpoint do not coincide, resulting in an unnatural appearance. This is because the height of the viewpoint differs depending on the driver's (2) height and driving position, causing the vanishing point of the outside world to change. Therefore, in this embodiment, as shown in Figure 7(b), the image editing unit 40 edits the distant image so that the vanishing point of the projected image is brought closer to the vanishing point of the outside world.

[0063] In this embodiment, the image projection device 3 uses camera C3 to capture an image of the driver 2, including the driver's face, as an external image. The external image acquisition unit 20 acquires the driver image, which is an external image, and transmits it to the image recognition unit 30. The image recognition unit 30 recognizes the position of the driver 2's eyes in the external image. The viewpoint position acquisition unit 60 calculates the viewpoint position of the driver 2 inside the vehicle 1 from the eye position in the image recognized by the image recognition unit 30 and the field of view of camera C3. The depression angle acquisition unit 50 acquires the depression angle from the viewpoint position calculated by the viewpoint position acquisition unit 60.

[0064] The control unit calculates the vanishing point of the external world as perceived from the driver's 2 viewpoint, based on the viewpoint position calculated by the viewpoint position acquisition unit 60 and the depression angle calculated by the depression angle acquisition unit 50. Alternatively, the image recognition unit 30 may calculate the vanishing point of the external world based on a nearby road surface image captured by camera C1 or a distant road surface image captured by camera C2. The image editing unit 40 performs editing processing to adjust the image vanishing point, which is the vanishing point of the projected image, based on the external vanishing point calculated by the control unit. Examples include a distortion correction method that brings the image vanishing point closer to the external vanishing point, or a method that prepares multiple projected images corresponding to multiple vanishing points in advance and selects the most appropriate one. Furthermore, it is more preferable to perform editing processing to make the image vanishing point and the external vanishing point coincide.

[0065] As shown in Figure 7(b), by applying an editing process to the projected image that brings the vanishing point of the image closer to the vanishing point of the external environment, the sense of incongruity of the projected image superimposed on the external environment can be reduced when the driver 2 views the virtual image P1.

[0066] As described above, the image projection device 3 of this embodiment also acquires the view outside the vehicle 1 as an external image, edits the projected image based on the external image, and irradiates with the first image light L1 or second image light L2, making it possible to display appropriate content according to the driving conditions and the driver's viewpoint position.

[0067] (Fourth Embodiment) Next, a fourth embodiment of the present invention will be described with reference to Figure 8. Details that overlap with the first embodiment will be omitted from the explanation. In this embodiment, based on the driver image captured by camera C3, the viewpoint position acquisition unit 60 acquires the viewpoint position of the driver 2, and the depression angle acquisition unit 50 acquires the depression angle from the viewpoint position of the driver 2. In addition, the image editing unit 40 performs editing processing to adjust the display position of the projected image based on the depression angle.

[0068] Figure 8 is a schematic diagram showing the projection of virtual images P1 and P2 in the image projection device 3 according to this embodiment. Figure 8(a) shows a distant image without editing, and Figure 8(b) shows a distant image after editing. The solid lines shown in Figures 8(a) and 8(b) indicate the range of the outside world that the driver 2 can see from the viewpoint position through the windshield WS. The dashed rectangles shown in the figures indicate the projection range of the virtual images P1 and P2 superimposed on the outside world. The images shown within the dashed lines are examples of distant and near images projected as virtual images P1 and P2, respectively.

[0069] As shown in Figure 8(a), if the distant image is not edited, the structure of the outside world as seen from the driver's (2) viewpoint and the content displayed in the distant image become too close together, reducing the visibility of the distant image. This is because the height of the viewpoint differs depending on the driver's (2) height and driving position, and the superimposition position of the outside world when seen changes. Therefore, in this embodiment, as shown in Figure 8(b), the image editing unit 40 edits the distant image so that the display position of the projected image is moved away from the structure of the outside world.

[0070] In this embodiment, the image projection device 3 uses camera C3 to capture an image of the driver 2, including the driver's face, as an external image. The external image acquisition unit 20 acquires the driver image, which is an external image, and transmits it to the image recognition unit 30. The image recognition unit 30 recognizes the position of the driver 2's eyes in the external image. The viewpoint position acquisition unit 60 calculates the viewpoint position of the driver 2 inside the vehicle 1 from the eye position in the image recognized by the image recognition unit 30 and the field of view of camera C3. The depression angle acquisition unit 50 acquires the depression angle from the viewpoint position calculated by the viewpoint position acquisition unit 60.

[0071] The external image acquisition unit 20 acquires a nearby road surface image captured by camera C1 or a distant road surface image captured by camera C2 as an external image. The image recognition unit 30 recognizes external structures included in the nearby or distant road surface image. The control unit calculates the position of external structures recognized from the driver's 2 viewpoint based on the viewpoint position calculated by the viewpoint position acquisition unit 60, the depression angle calculated by the depression angle acquisition unit 50, and the external structures recognized by the image recognition unit 30. The image editing unit 40 performs editing processing to adjust the display position of content in the projected image based on the position of external structures calculated by the control unit. One example is a correction that moves the display position of content away from external structures.

[0072] As shown in Figure 8(b), by applying editing processing to the projected image to move the display position of the content away from external structures, the visibility of the projected image superimposed on the external environment can be improved when the driver 2 views the virtual image P1.

[0073] As described above, the image projection device 3 of this embodiment also acquires the view outside the vehicle 1 as an external image, edits the projected image based on the external image, and irradiates with the first image light L1 or second image light L2, making it possible to display appropriate content according to the driving conditions and the driver's viewpoint position.

[0074] The present invention is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

[0075] This international application claims priority based on Japanese Patent Application No. 2024-200163, filed on 17 November 2024, and the entire contents of said Japanese Patent Application No. 2024-200163 are incorporated herein by reference.

[0076] The above description of specific embodiments of the present invention is provided for illustrative purposes only. It is not intended to be exhaustive or to limit the invention to the forms described. Numerous modifications and changes are possible in light of the above description, as will be obvious to those skilled in the art.

[0077] 1...Vehicle 2...Passenger (driver) 3...Image projection device 4...Headlight 10...Image illumination unit 20...External image acquisition unit 30...Image recognition unit 40...Image editing unit 50...Depression angle acquisition unit 60...Viewpoint position acquisition unit P1, P2...Virtual image C1-C3...Camera T1-T4...Object

Claims

1. An image projection device for projecting a projection image onto a display unit for displaying a virtual image, comprising: an external image acquisition unit for acquiring an external image; an image editing unit for editing the projection image based on the external image; and an image illumination unit for illuminating the display unit with the projection image as image light.

2. An image projection device according to claim 1, comprising an image recognition unit that recognizes an object included in the external image, and the image editing unit that applies a coloring process to the external image according to the object.

3. An image projection device according to claim 2, wherein the image editing unit colors only the outline of the object when the object is a direct recognition object, and fills in the entire object when the object is a warning object.

4. An image projection device according to claim 2, characterized in that the area ratio of the colored area is 50% or less of the external image.

5. An image projection device according to claim 1, wherein the image irradiation unit projects the projected image onto an area of ​​the display unit that overlaps with the vehicle.

6. An image projection device according to claim 5, wherein the projected image comprises an external view mode based on the external image and a driving support mode in which the projected image includes content images that support driving, and the image projection unit switches between displaying the external view mode and the driving support mode.

7. An image projection device according to claim 5, wherein the projected image comprises a distant image formed at a first distance from the display unit and a near image formed at a second distance shorter than the first distance, wherein the near image is based on the external image and the distant image includes content images that support driving.

8. An image projection device according to claim 5, comprising a depression angle acquisition unit for acquiring the depression angle when viewing the projected image from a viewpoint position, and characterized in that the area in the display unit that overlaps with the vehicle is calculated based on the depression angle.

9. An image projection device according to claim 8, wherein the image editing unit adjusts the vanishing point of the projected image based on the depression angle.

10. An image projection device for projecting a projected image onto a display unit for displaying a virtual image, comprising: an image illumination unit for irradiating the display unit with the projected image as image light; a depression angle acquisition unit for acquiring the depression angle when viewing the projected image from a viewpoint position; and an image editing unit for editing the projected image to adjust the vanishing point of the projected image, which is the vanishing point of the projected image, based on the depression angle.

11. An image projection device according to claim 10, comprising a viewpoint position acquisition unit that acquires the viewpoint position, wherein the image editing unit edits the projected image so as to bring the vanishing point of the image closer to the vanishing point of the background, which is the vanishing point of the background on which the projected image is superimposed, based on the viewpoint position and the depression angle.

Images