Image projection device and image projection method

Abstract

Provided are an image projection device and an image projection method which prevent a feeling of incongruity due to a change in a traveling environment of a vehicle when viewing a front display image. An image projection device (10) projects a projection image onto a windshield (16) for displaying virtual images (17, 18), and is characterized by comprising an image radiation unit (11) that radiates a projection image onto the windshield (16) as image light and a front monitoring unit (20) that monitors a traveling environment in front of a host vehicle, and in that a display-prohibited region in which projection of the projection image is prohibited is set according to the traveling environment.

Description

Image projection device and image projection method

[0001] The present invention relates to an image projection device and an image projection method.

[0002] Conventionally, dashboards that illuminate icons have been used to display various types of information inside vehicles. As the amount of information to be displayed increases, it has been proposed to embed an image display device in the dashboard or to configure the entire dashboard with an image display device.

[0003] However, since the instrument panel is located below the windshield of the vehicle, the driver has to move his or her eyes downward while driving in order to see the information displayed on the instrument panel, which is undesirable. Therefore, a head-up display (hereinafter referred to as HUD) has been proposed, which projects an image onto the windshield so that the driver can read the information when looking ahead of the vehicle (see, for example, Patent Documents 1 and 2).

[0004] JP 2019-119248 A JP 2019-119262 A

[0005] In a vehicle equipped with a HUD, the driver visually views a virtual image (hereinafter sometimes referred to as a "forward display image") displayed in front of the vehicle. The vertical display position of the forward display image is typically set assuming that there is no vehicle ahead (a vehicle traveling ahead of the host vehicle). Therefore, when a vehicle ahead approaches the host vehicle on the road, the forward display image overlaps with the vehicle ahead, making it difficult for the driver to see the forward display image, which can cause the driver to feel uncomfortable. To address this issue, it is possible to shift the position of the forward display image downward in advance during the design stage of the image projection device. However, as is clear from considering the differences in the driver's gaze direction on ordinary roads and expressways, for example, the driver's line of sight changes vertically. Therefore, it is desirable to be able to change the display position of the forward display image depending on the vehicle's driving environment. In particular, when the forward display image is an image that calls the driver's attention, it is expected that the driver's awareness will be delayed due to the above-mentioned discomfort. To address this issue, it is desirable to have a variable display position of the forward display image.

[0006] On the other hand, when going around a curve, the position of the forward display image may deviate from the driver's line of sight. This is because the driver's line of sight naturally moves in the direction of the curve. This kind of deviation may also cause the driver to feel uncomfortable when viewing the forward display image, so it is preferable to eliminate this deviation as much as possible.

[0007] Therefore, the present invention has been made in consideration of the above-mentioned conventional problems, and aims to provide an image projection device and an image projection method that suppress the discomfort felt when viewing a forward display image due to changes in the vehicle's driving environment.

[0008] In order to solve the above problem, the image projection device of the present invention is an image projection device that projects a projection image onto a display unit for displaying a virtual image, and is characterized in that it is equipped with an image irradiation unit that irradiates the projection image onto the display unit as image light, and a monitoring unit that monitors the driving environment ahead of the vehicle, and a display prohibition area in which projection of the projection image is prohibited is set depending on the driving environment.

[0009] The image projection device of the present invention includes an image projection unit that projects a projection image onto a display unit as image light, and a monitoring unit that monitors the driving environment ahead of the vehicle. A display prohibition area in which projection of the projection image is prohibited is set according to the driving environment. As a result, the projection image moves to a position that is easy for the driver to see. This reduces the discomfort in viewing the forward display image caused by changes in the driving environment of the vehicle.

[0010] In one aspect of the present invention, the vehicle further comprises a measurement unit that measures the driver's field of view, and a control unit that, when the driving environment satisfies predetermined conditions, sets a display prohibition area in which projection of the projected image is prohibited based on the field of view, and corrects the display position of the projected image to avoid the display prohibition area.

[0011] In one aspect of the present invention, the control unit corrects the display position of the projection image by moving the projection image below the display prohibited area.

[0012] In one aspect of the present invention, the predetermined condition is a distance between the host vehicle and a vehicle ahead.

[0013] In one aspect of the present invention, the vehicle navigation system further includes an angular velocity measurement unit that measures a gradient of the host vehicle, and the control unit corrects the display prohibited area based on the gradient.

[0014] In one aspect of the present invention, the monitoring unit detects a curvature of the road, and the control unit corrects the display prohibited area based on the curvature.

[0015] In one aspect of the present invention, the control unit corrects the display position of the projection image by changing the display position of the projection image in the image projection unit so as to avoid the display prohibited area.

[0016] In one aspect of the present invention, the control unit changes the position of the image projection unit to correct the display position of the projected image so as to avoid the display prohibited area.

[0017] In order to solve the above problem, the image projection method of the present invention is an image projection method using an image projection device that projects a projection image onto a display unit for displaying a virtual image, the image projection device having an image irradiation unit that irradiates the projection image onto the display unit as image light, and a monitoring unit that monitors the driving environment ahead of the vehicle, and is characterized in that a display prohibited area in which projection of the projection image is prohibited is set depending on the driving environment.

[0018] The present invention can provide an image projection device and an image projection method that suppress the sense of incongruity that occurs when viewing a forward display image due to changes in the vehicle's driving environment.

[0019] 4(a) shows a state in which a preceding vehicle is traveling sufficiently ahead of the own vehicle, FIG. 4(b) shows a state in which the own vehicle and the preceding vehicle are close to each other and the preceding vehicle appears relatively large, and FIG. 4(c) shows a state in which the own vehicle and the preceding vehicle are even closer to each other and the preceding vehicle appears significantly larger. FIG. 4(c) is a flowchart showing the processing flow of a display position correction program executed by the image projection device according to the first embodiment. FIG. 4(a) is a block diagram showing an example of the configuration of a control system of an image projection device according to the second embodiment. FIG. 4(b) is a schematic diagram showing the operation of the image projection device according to the second embodiment. FIG. 7(a) shows a method for setting a display prohibited area in a case in which the road is approximately straight, FIG. 7(b) shows a case in which the road is curved to the right, and FIG. 7(c) shows a case in which the road has an uneven surface. FIG. 7(c) is a flowchart showing the processing flow of a display position correction program executed by the image projection device according to the second embodiment. 9(a) is a schematic diagram showing the operation of an image projection device according to a modified example of the second embodiment. Fig. 9(b) shows a method for setting a display-prohibited area when the road is substantially straight, when the road is curved to the right, and when the road has an uneven surface. Fig. 9(c) is a block diagram showing an example of the configuration of an image projection device according to a third embodiment.

[0020] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The same or equivalent components, members, and processes shown in each drawing will be designated by the same reference numerals, and redundant descriptions will be omitted where appropriate. In the following description, an image projection device according to the present invention will be described, exemplarily, as applied to a HUD mounted on a vehicle or the like. Furthermore, as an image projection device, an image projection device capable of displaying two screens, a far image formed as a virtual image relatively far ahead of the vehicle and a near image formed as a virtual image relatively close ahead of the vehicle, will be described. Each of the far image and the near image is a form of the forward display image described above. Since the discomfort caused by the approach of a leading vehicle described above is often a problem with the far image, which is more frequently viewed, the following description will exemplify how to deal with the far image. Note that the image projection device is not limited to a two-screen image projection device, and may be applied to an image projection device that projects three or more forward display images on one screen, an image projection device that projects one forward display image on one screen, or an image projection device that projects multiple forward display images on multiple screens.

[0021] (First embodiment) An image projection device and an image projection method according to this embodiment will be described with reference to Figs. 1 to 5. Fig. 1 is a block diagram showing an example of the configuration of an image projection device 10 according to this embodiment. The main body of the image projection device 10 is disposed, for example, below the dashboard of a vehicle. The image projection device 10 exemplifies an image projection device employing an image projection method capable of projecting two forward display images on a single screen. That is, the image projection device 10 is capable of displaying a distant image formed as a virtual image relatively far in front of the vehicle, and a near image formed as a virtual image relatively close in front of the vehicle.

[0022] 1, an image projection device 10 according to this embodiment includes an image projection unit 11, a reflecting mirror 12, a free-form surface mirror 13, an optical branching unit 14, a free-form surface mirror 15, a forward monitoring unit 20, a line-of-sight measurement unit 21, an angular velocity measurement unit 22, and a control unit 30. Also shown in Fig. 1 is a windshield 16 onto which a projected image from the image projection device 10 is projected.

[0023] The image projection unit 11 projects an image for forming a virtual far image or a near image ahead of the vehicle. The image projection unit 11 is configured, for example, with a liquid crystal panel, an organic EL panel, a DMD (Digital Micro Mirror Device), an LCOS, or the like. The image projection unit 11 is divided into a far pixel region for displaying an image for forming a far image, and a near pixel region for displaying an image for forming a near image. In FIG. 1 , the optical path of the far image light projected from the far pixel region is indicated by a dashed line, and the optical path of the near image light projected from the near pixel region is indicated by a dashed line.

[0024] The image projection device 10 is configured so that the far image (virtual image 18) displayed in front of the vehicle can be moved up and down (vertically) and left and right (horizontally). As a result, the position of the image displayed in the far pixel area of ​​the image projection unit 11 can be changed. By moving the image displayed in the far pixel area, the projection position of the far image light from the image projection unit 11 onto the windshield 16 is adjusted, and as a result, the far image is adjusted up and down and left and right. Note that the direction of movement in each of the up and down and left and right directions can be selected as needed, and the image may be configured so that it can be moved in either one of the directions.

[0025] The movement of the display image in the far pixel region will be described in more detail with reference to FIG. 2 . FIG. 2 is a schematic diagram showing a pixel region 40 in which an image is displayed by the image projection unit 11. The pixel region 40 is the entire region capable of displaying an image as a display. The pixel region 40 includes a far pixel region 41 that displays a far image formed in the distance, and a near pixel region 43 that displays a near image formed in the distance. In the image projection device 10, far image light that irradiates the far image displayed in the far pixel region 41 is formed as a virtual image 18 in the distance via the optical branching unit 14, etc. Further, near image light that irradiates the near image displayed in the near pixel region 43 is formed as a virtual image 17 in the near direction via the reflecting mirror 12, etc.

[0026] A margin area 42 is provided around the far pixel area 41, and the image displayed in the far pixel area 41 can be moved within the margin area 42 in directions D1, D2, D3, and D4 shown by the arrows in Fig. 2. This adjusts the position at which the far image light is irradiated onto the windshield 16, and as a result, the vertical and horizontal positions of the far image are adjusted.

[0027] As shown in Figure 1, the image projection unit 11 is connected to the control unit 30, and the control unit 30 controls the movement of the position of the image to be displayed in the distant pixel area 41, i.e., the movement of the distant image in the vertical and horizontal directions.

[0028] In this embodiment, the image displayed in the far pixel region 41 is configured to allow the far image to be moved up and down and left and right, but if the near image creates a sense of incongruity, the near image may be moved up and down or left and right by moving the image displayed in the near pixel region. Alternatively, both the far image and the near image may be moved up and down or left and right.

[0029] The optical branching unit 14 is a component that branches the far image light emitted from the far pixel region 41, and is configured, for example, by a prism. As shown in FIG. 1 , the far image light passes through the optical branching unit 14, is reflected by the free-form surface mirror 15, the free-form surface mirror 13, and the windshield 16, and reaches the driver's viewpoint 52. As a result, a virtual image 18 (far image) is formed relatively far in front of the vehicle. The display position of the far image is, for example, about 15 m in front of the vehicle. In FIG. 1 , a "driving assistance display" is illustrated as an example of the content of the far image.

[0030] On the other hand, as shown in Figure 1, near image light is emitted from the near pixel region, and then reflected by reflecting mirror 12, free-form surface mirror 13, and windshield 16, before reaching the driver's viewpoint 52. As a result, a virtual image 17 (near image) is formed relatively close in front of the vehicle. As an example, the display position of the near image is about 3 m in front of the vehicle. In Figure 1, "vehicle speed display, etc." is shown as an example of the content of the near image.

[0031] With the above configuration, the image projection device 10 projects the virtual image 18 in the distance, several degrees below the horizontal, reducing the amount of line-of-sight movement required by the driver to visually recognize the driving assistance information. Furthermore, the virtual image 17 is projected even further below in the near distance, allowing the driver to clearly recognize the vehicle speed display and other information by shifting their line of sight. Of course, the content of the far image and near image exemplified in this embodiment is merely an example, and there is no limitation on the content of the far image and near image.

[0032] Referring again to FIG. 1 , the forward monitoring unit 20 is a component that monitors objects ahead of the vehicle, which in this embodiment mainly monitors a vehicle ahead or the road on which the vehicle is traveling. In this embodiment, as an example of monitoring, the forward monitoring unit 20 monitors the distance and relative position between the vehicle ahead and the vehicle ahead. The forward monitoring unit 20 is configured, for example, with an imaging device such as a camera, and a ranging device such as LiDAR (Light Detection and Ranging) or millimeter-wave radar. As the camera, in addition to a normal camera, a camera for a specific purpose such as an ADAS (Advanced Driver Assistance System) camera may also be used. The forward monitoring unit 20 is connected to a control unit 30 (described later), and information monitored by the forward monitoring unit 20 is sent to the control unit 30. The "forward monitoring unit 20" is an example of a "monitoring unit" according to the present invention.

[0033] The gaze measurement unit 21 is a so-called eye tracking device, and is a component that tracks in real time where the driver is looking. In this embodiment, the gaze measurement unit 21 detects the area where the driver's gaze is focused (hereinafter referred to as the "visible area"), mainly in front of the vehicle. The gaze measurement unit 21 is connected to the control unit 30, and measurement information measured by the gaze measurement unit 21 is sent to the control unit 30.

[0034] The angular velocity measurement unit 22 is a so-called gyro sensor that detects the gradient (tilt) of the vehicle. In a vehicle, the vertical position of the distant image fluctuates due to changes in the vehicle's posture caused by factors such as uneven load distribution, driving on a slope, and unevenness of the road. In this case, changing the display position of the distant image in accordance with the fluctuation in the vertical position may reduce discomfort felt by the driver. Therefore, in this embodiment, the angular velocity measurement unit detects the gradient angle of the vehicle, determines the posture of the vehicle, and moves the display position of the distant image. Note that in this embodiment, controlling the display position of the distant image based on gradient detection is an additional function, and the angular velocity measurement unit 22 is not an essential component.

[0035] The control unit 30 controls the entire image projection device 10 and executes a display position correction program (described later). The control unit 30 includes a CPU, ROM, RAM, etc. (not shown). The control unit 30 may be an ECU (Engine Control Unit) of a vehicle equipped with the image projection device 10.

[0036] As described above, when a leading vehicle approaches the vehicle on the roadway, the forward display image, particularly the distant image, may overlap with the rear of the leading vehicle, making the forward display image difficult to view, which may cause the driver to feel uncomfortable. Therefore, in this embodiment, a display prohibition area is set to prohibit the display of the distant image. The display prohibition area is set inside and outside the display area of ​​the distant image (hereinafter referred to as the "distant image display area") as a virtual image in front of the vehicle. The display prohibition area is controlled so that the distant image is not displayed in the display prohibition area, thereby moving the distant image and eliminating the driver's discomfort. More specifically, the forward monitoring unit 20 measures the distance to the leading vehicle, i.e., the inter-vehicle distance, and adjusts the position of the image to be displayed in the distant pixel area 41 of the image projection unit 11 when the inter-vehicle distance satisfies a predetermined condition. As a result, the projection position of the distant image light from the image projection unit 11 onto the windshield 16 is adjusted, and the vertical or horizontal position of the distant image is adjusted.

[0037] An example of the configuration of the control system of the image projection device 10 according to this embodiment will be described with reference to FIG. 3 . As shown in FIG. 3 , the control unit 30 includes a leading vehicle detection unit 31, a visual recognition area detection unit 32, a gradient detection unit 33, a display prohibited area setting unit 34, a distant image display position correction unit 35, an image control signal generation unit 36, and a storage unit 37. These components are each controlled by the control unit 30. In this embodiment, the leading vehicle detection unit 31, the visual recognition area detection unit 32, the gradient detection unit 33, the display prohibited area setting unit 34, the distant image display position correction unit 35, and the image control signal generation unit 36 ​​are each implemented by software. However, this is not a limitation, and they may also be implemented by hardware such as an ASIC (application specific integrated circuit).

[0038] The vehicle-in-front detection unit 31 receives monitoring information from the forward monitoring unit 20 and detects a vehicle in front 54. Information about the detected vehicle in front 54 is sent to a display-prohibited area setting unit 34, which will be described later. The visible area detection unit 32 receives measurement information from the line-of-sight measurement unit 21 and detects a visible area, which is an area ahead of the driver's line of sight, in front of the vehicle. Information about the detected visible area is sent to a display-prohibited area setting unit 34, which will be described later. The gradient detection unit 33 receives a signal from the angular velocity measurement unit 22 and detects changes in the vertical position of the vehicle. Information about the detected gradient is sent to a display-prohibited area setting unit 34, which will be described later. The memory unit 37 is a memory unit that temporarily stores a display position correction program, which will be described later, data being processed by the control unit 30, and the like. The memory unit 37 may be configured as a storage means, such as a ROM or RAM, separate from the control unit 30.

[0039] The display prohibited area setting unit 34 sets a display prohibited area XA based on information about the leading vehicle 54 from the leading vehicle detection unit 31, information about the visible area from the visible area detection unit 32, and gradient information from the gradient detection unit 33. The far image display position correction unit 35 corrects the display position of the far image so that the far image is displayed in an appropriate position while avoiding the display prohibited area XA set by the display position prohibited area 34. The image control signal generation unit 36 ​​generates a signal for controlling the image projection unit 11, more specifically, a control signal for controlling the far pixel area 41, based on the display position of the far image corrected by the far image display position correction unit 35. Note that in this embodiment, the display prohibited area XA is set based on the visible area detected by the visible area detection unit 32, so that the display prohibited area XA can be set appropriately regardless of differences in the position (height) of the driver's viewpoint 52. Conversely, if differences in the position of the driver's viewpoint 52 are not an issue, it is not necessary to use the gaze measurement unit 21. For example, the display prohibited area XA may be set using an image of the forward driving environment, etc., acquired by the forward monitoring unit 20.

[0040] The function of the above-mentioned display prohibited area XA will be described with reference to Figure 4. Figure 4 is a diagram schematically illustrating the relationship between the state ahead of the vehicle while it is traveling, i.e., the driving environment, and the display state of the distant image. Hereinafter, the rectangle surrounding the contour of the distant image, i.e., the virtual image 18, will be referred to as the "distant image area FG." In contrast, the range within which the distant image area FG can move will be referred to as the "distant image display area FOV." The distant image area FG and the distant image display area FOV correspond to the distant pixel area 41 and pixel area 40 of the image projection unit 11, respectively.

[0041] 4A shows a state in which a leading vehicle 54 is traveling sufficiently ahead of the vehicle. In this case, when the image projection device 10 is not performing any control, i.e., in the normal state, the upper part of the distant image display area FOV and the lower part of the leading vehicle 54 are almost in contact. However, since the leading vehicle 54 appears sufficiently small, the distant image area FG is located below the leading vehicle 54, and the distant image is projected onto the road surface 50 below the leading vehicle 54. Therefore, there is no particular problem with the driver's visibility. In such a case, the display prohibited area XA is not set.

[0042] FIG. 4B shows a state in which the host vehicle and the leading vehicle 54 are approaching each other, with the leading vehicle 54 appearing relatively large. The display-prohibited area setting unit 34, which receives this approach information from the leading vehicle detection unit 31, sets a display-prohibited area XA when the inter-vehicle distance between the host vehicle and the leading vehicle 54 becomes equal to or less than a predetermined distance. That is, the line-of-sight measurement unit 21 first detects the visible area, which is the area where the driver's gaze is focused. In the example shown in FIG. 4B, the driver's gaze is mostly focused on the rear of the leading vehicle 54, so the display-prohibited area XA is set to cover the rear of the leading vehicle 54. As a result, the far image display area FOV moves downward, which in turn moves the far image area FG downward, and the far image is displayed on the road 50. However, in the state shown in FIG. 4B, the position of the far image area FG within the far image display area FOV does not change. As a result, the distant image area FG moves to a position where it does not overlap with the vehicle in front 54, reducing the sense of discomfort felt by the driver when viewing the distant image.

[0043] FIG. 4( c ) shows a state in which the host vehicle and the leading vehicle 54 have come even closer, with the leading vehicle 54 appearing significantly larger. The display-prohibited area setting unit 34, which has received this approach information from the leading vehicle detection unit 31, corrects the display-prohibited area XA to enlarge it. As a result, the far image area FG within the far image display area FOV moves downward, and projection onto the road 50 is maintained. While FIG. 4( c ) shows an example in which this control moves the far image area FG below the leading vehicle 54, the direction of movement of the far image area FG is not limited to below the leading vehicle 54, and it may also be to the left or right. Furthermore, if the inter-vehicle distance becomes even smaller and falls below a predetermined value, or if the size of the leading vehicle 54 exceeds a predetermined size, the display of the far image may be stopped.

[0044] The procedure for correcting the display position of a distant image executed by the image projection device 10 according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the processing flow of a display position correction program, which shows the procedure for the display position correction process executed by the control unit 30 of the image projection device 10. This display position correction program is stored in a storage means such as a ROM (not shown), and is read out by the CPU, expanded into a RAM, etc., and executed.

[0045] In step S10, the control unit 30 controls the leading vehicle detection unit 31 to acquire leading vehicle information.

[0046] In step S11, the control unit 30 controls the visual recognition area detection unit 32 to acquire visual recognition area information, which is a set of areas ahead of the driver's line of sight.

[0047] In step S12, the control unit 30 determines whether it is necessary to set the display prohibited area XA. As described above, whether it is necessary to set the display prohibited area XA is determined, for example, based on the distance (inter-vehicle distance) between the subject vehicle and the preceding vehicle 54 acquired in step S10, and it is determined that it is necessary if the inter-vehicle distance is equal to or less than a predetermined value. The predetermined distance can be, for example, approximately 3 m to 10 m. If the determination is negative, the process returns to step S10 and continues acquiring information about the preceding vehicle. On the other hand, if the determination is positive, the process proceeds to step S13.

[0048] In step S13, the control unit 30 controls the display prohibition area setting unit 34 to set a display prohibition area XA. The display prohibition area XA is set based on the visible area acquired in step S11. More specifically, the display prohibition area XA is set to include, for example, at least a part of the visible area. In this embodiment, the display prohibition area XA is set outside the far image display area FOV, but it may also be set inside the far image display area FOV.

[0049] In step S14, the control unit 30 controls the image control signal generating unit 36 ​​to acquire the display position information of the current distant image.

[0050] In step S15, the control unit 30 controls the gradient detection unit 33 to acquire gradient information, that is, information relating to the vertical position of the vehicle.

[0051] In step S16, the control unit 30 controls the display prohibition area setting unit 34 to correct the display prohibition area XA based on the information on the vertical position of the host vehicle acquired in step S15. For example, if the host vehicle is traveling on a flat road and there is substantially no fluctuation in the vertical position, no processing is performed in this step.

[0052] In step S17, the control unit 30 controls the far display position correction unit 35 to compare the current display position of the far image acquired in step S14 with the display prohibited area XA corrected in step S16. More specifically, it determines whether or not there is an overlapping portion between the far image area FG (far image) and the display prohibited area XA.

[0053] In step S18, the control unit 30 controls the far image display position correction unit 35 to generate correction data for the display position of the far image, i.e., the position of the far image area FG. That is, the control unit 30 generates position correction data for the far image area FG so as to reduce the overlap area between the far image area FG determined in step S17 and the display prohibited area XA. Here, it is not necessary to completely eliminate the overlap between the far image area FG and the display prohibited area XA; some overlap may be allowed depending on the required performance, etc.

[0054] In step S19, the control unit 30 controls the image control signal generation unit 36 ​​to generate an image control signal for controlling the image projection unit 11 based on the correction data for the far image display position generated in step S18.

[0055] In step S20, the control unit 30 controls the image projection unit 11 to project an image.

[0056] In step S21, it is determined whether an end instruction has been issued. If the determination is negative, the process proceeds to step S10, where acquisition of leading vehicle information continues. In other words, in this embodiment, acquisition of leading vehicle information continues at all times, so the relative positional relationship between the leading vehicle 54 and the display position of the distant image (position of the distant image area FG) is continuously and smoothly adjusted. On the other hand, if the determination is positive, the display position correction program is terminated. An end instruction may be, for example, by turning off the power to the image projection device 10.

[0057] As described above in detail, the image projection device and image projection method according to this embodiment make it possible to provide an image projection device and an image projection method that suppress the discomfort felt when viewing the forward display image due to changes in the vehicle's driving environment.

[0058] Second Embodiment An image projection device 10A and an image projection method according to this embodiment will be described with reference to Figures 6 to 8. This embodiment is a modification of the first embodiment, in which the method for setting the display-prohibited area XA is changed. Therefore, since the configuration of the image projection device is the same as that of the image projection device 10, Figures 1 and 2 will be referred to as necessary, and detailed description will be omitted.

[0059] An example of the configuration of the control system of the image projection device 10A will be described with reference to Fig. 6. The image projection device 10A differs from the image projection device 10 shown in Fig. 3 in that it further includes a position information acquisition unit 23 and a map information acquisition unit 24, and the control unit 30 further includes a road shape detection unit 38.

[0060] The road shape detection unit 38 is connected to the forward monitoring unit 20, and detects the shape of the road, which is the driving environment ahead of the vehicle, using image information sent from the forward monitoring unit 20. The detected road shape information is sent to the display prohibited area setting unit 34, which uses the road shape information to correct the settings of the shape, position, etc. of the display prohibited area XA.

[0061] The position information acquisition unit 23 includes a receiver (not shown) and can receive GPS information and the like acquired from GPS (Global Positioning System) satellites. The GPS information can be used to determine the current position (latitude, longitude, etc.) of the vehicle. The position information acquired by the position information acquisition unit 23 is sent to the control unit 30.

[0062] The map information acquisition unit 24 is connected to a communication network (not shown) such as an IP network, and acquires information from a map information provider (not shown) or the like. The map information provider is, for example, a server that stores HD maps or the like. Note that the map information acquisition unit 24 is not limited to this, and may also be a navigation system for the vehicle. The control unit 30 identifies the position of the vehicle based on the position information from the position information acquisition unit 23, and acquires the road environment, such as the road shape at the current position of the vehicle, using the map information acquired by the map information acquisition unit 24. However, as described above, the road shape is basically detected based on image information from the forward monitoring unit 20, and therefore detection using the position information acquisition unit 23 and the map information acquisition unit 24 is auxiliary. Therefore, the position information acquisition unit 23 and the map information acquisition unit 24 are not essential components.

[0063] The operation of the image projection device 10A will be described with reference to FIG. 7 . The road 50 shown in FIG. 7 has one lane on each side, with lane marks LM1 and LM3 indicating both ends of the road 50 and lane mark LM2 indicating the center. Lane marks LM refer to white lines drawn on the road 50 or guardrails or the like installed on the road 50. The vehicle is traveling in the depth direction of the page between lane marks LM1 and LM2. In FIG. 7 , the intersection of an extension line UL at the top edge of the distant image area FG and the lane mark LM1 is shown as a base point BP1, and the intersection of the extension line UL and the lane mark LM2 is shown as a base point BP2. FIG. 7( a ) shows how the display-prohibited area XA is set when the road 50 is substantially straight, FIG. 7( b ) shows how the road 50 curves to the right, and FIG. 7( c ) shows how the road 50 has irregularities. In the following explanation, the preceding vehicle detection unit 31 detects a preceding vehicle, the visible area detection unit 32 detects the visible area, and when the inter-vehicle distance between the host vehicle and preceding vehicle 54 falls below a predetermined distance, the control unit 30 determines that setting of the display prohibited area XA is necessary, and sets the display prohibited area XA, as in the first embodiment. That is, the following explanation shows an embodiment in which the display prohibited area XA is once set, and the positions of the distant image display area FOV and distant image area FG are fixed, and further display prohibited area XA is added (corrected) depending on the road environment.

[0064] As shown in FIG. 7A , when the road 50 is substantially straight, the control unit 30 draws vertical lines (vertical lines) from each of the base points BP1 and BP2 to divide the interior of the far image display area FOV into three sections, and sets display-prohibited areas XA1 and XA2 at both ends of the far image display area FOV. This restricts the position of the far image area FG to between the vertical lines drawn from the base points BP1 and BP2. Therefore, the display position of the far image is set to a position that is easily visible to the driver. The position of the far image area FG may be freely moved left and right, up and down, as long as it is between the base points BP1 and BP2. For example, the far image area FG may be positioned approximately in the center of the far image display area FOV excluding the display-prohibited areas XA1 and XA2. As an example of a method for arranging the far image area FG in this manner, the distance d1 between the base point BP1 and the left side of the far image area FG and the distance d2 between the base point BP2 and the right side of the far image area FG can be set to be equal. However, the present invention is not limited to this, and the far image area FG may be set using the visual recognition area detected by the visual recognition area detection unit 32.

[0065] Next, as shown in FIG. 7B , if the road 50 curves to the right, the control unit 30 draws a vertical line (a line in the vertical direction) from the base point BP1 to divide the interior of the far image display area FOV into two, and sets a display-prohibited area XA1 on the left side of the far image display area FOV. As a result, the position of the far image area FG is restricted between the vertical line drawn from the base point BP1 and the right side of the far image display area FOV. Here, when the road curves, the driver's line of sight tends to shift in the direction of the curve. In the example of FIG. 7B , the driver's line of sight tends to shift to the right from the center of the road width. To accommodate this tendency, in this embodiment, the display-prohibited area XA1 is set in the direction opposite to the curvature of the road 50, so that the far image area FG is shifted to the right. This allows the display position of the far image to be easily recognized by the driver.

[0066] The position of the far image area FG may be freely moved left and right, up and down, and set between the base point BP1 and the right side of the far image display area FOV. For example, the far image area FG may be positioned approximately in the center of the far image display area FOV excluding the display prohibited area XA1. As an example of a method for positioning the far image area FG, the distance d1 between the base point BP1 and the left side of the far image area FG and the distance d2 between the base point BP2 and the right side of the far image area FG may be set to be equal. However, this is not limited to this, and the far image area may also be set using the visual recognition area detected by the visual recognition area detection unit 32. Furthermore, the amount by which the far image area FG is shifted to the right may be predetermined. For example, the distance may be set to approximately 25% of the left-right width of the far image area FG. Note that if the road 50 curves left, the right in the above description can simply be read as left, and detailed description thereof will be omitted.

[0067] Referring to FIG. 7( c), the setting of the display-prohibited area XA when the road 50 is uneven will be described. When the road is uneven, the distant image (distant image area FG) will naturally move (vibrate) up and down. In particular, when the host vehicle approaches a convex portion of the road 50, the distant image will overlap with the vehicle ahead 54 (not shown in FIG. 7), which is expected to cause the driver to feel uncomfortable. Therefore, in this embodiment, the forward monitoring unit 20 detects a vanishing point VP of the road 50 on which the host vehicle is traveling. As shown in FIG. 7( c), the vanishing point VP is detected as the intersection of an extension of the lane mark LM1 and an extension of the lane mark LM2. In this case, the vanishing point VP may be detected in cooperation with the gradient detection unit 33 when the gradient detection unit 33 detects an upward gradient.

[0068] As shown in FIG. 7(c), the area above the vanishing point VP is set as a display prohibited area XA. FIG. 7(c) shows an example of setting the display prohibited area XA when the vanishing point VP has risen to the position of point VP' in the far image display area FOV. The width W may be determined taking into account the width of movement due to unevenness in the far image area FG. Furthermore, display prohibited areas are also set to the left of the vertical line drawn from base point BP1 and to the right of the vertical line drawn from base point BP2. As a result, in this embodiment, the display prohibited area XA is set as a polygon with six vertices, as shown in FIG. 7(c). The far image area FG is then controlled to be located in an area other than the display prohibited area XA within the far image display area FOV. As a result, the display position of the far image is set at a position that is easily visible to the driver.

[0069] Next, a procedure for correcting the display position of a distant image executed by the image projection device 10A according to this embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the processing flow of a display position correction program showing the procedure for the display position correction process executed by the control unit 30 of the image projection device 10A. This display position correction program is stored in a storage means such as a ROM (not shown), and is read out by the CPU, expanded into a RAM, etc., and executed.

[0070] Steps S10 to S21 shown in FIG. 8 are basically the same as steps S10 to S21 shown in FIG. 5, so only the differences will be described.

[0071] After determining in step S12 that it is necessary to set the display prohibited area XA, in step S22 the control unit 30 controls the lane shape detection unit 38 to acquire lane shape information.

[0072] In step S13, the control unit 30 sets the display prohibited area XA. The procedure for setting the display prohibited area XA in this embodiment is as explained above with reference to FIGS.

[0073] In step S16, the control unit 30 analyzes the gradient information acquired in step S15 and determines whether the road is uneven. If it is determined that the road is uneven, the control unit 30 controls the display prohibited area setting unit 34 to correct the display prohibited area XA in the procedure described in FIG. 7C.

[0074] In step S18, the control unit 30 sets the position of the far image area FG so that the far image is displayed in the far image display area FOV excluding the display prohibited area XA set in step S13.

[0075] In step S20, the control unit 30 controls the image projection unit 11 to project an image, so that the distant image is displayed at a position that is easily visible to the driver.

[0076] As described above in detail, the image projection device and image projection method according to this embodiment also make it possible to provide an image projection device and an image projection method that suppress the sense of discomfort felt when viewing the forward display image due to changes in the vehicle's driving environment. In particular, this embodiment makes it possible to provide an image projection device and an image projection method that suppress the sense of discomfort felt when viewing the forward display image even when the road 50 is curved or has unevenness.

[0077] <Modification of Second Embodiment> The operation of an image projection device according to a modification of the second embodiment will be described with reference to FIG. 9 . This embodiment is a modified version of the procedure for setting the display-prohibited area XA in the second embodiment. Therefore, since the configuration of the image projection device is the same as that of the image projection device 10A shown in FIG. 6 , please refer to FIG. 6 as necessary and a detailed description will be omitted. Furthermore, since the procedure for correcting the display position of a distant image according to this embodiment is the same as that shown in the flowchart in FIG. 8 , a detailed description will be omitted. FIG. 9( a ) shows a method for setting the display-prohibited area XA when the road 50 is substantially straight, FIG. 9( b ) shows a method for setting the road 50 when it curves to the right, and FIG. 9( c ) shows a method for setting the road 50 when it has an uneven surface.

[0078] As shown in FIG. 9A , when the road 50 is substantially straight, the control unit 30 sets a display prohibited area XA1 in the left portion of the far image display area FOV cut out by the lane mark LM1 passing through the base point BP1. Similarly, a display prohibited area XA2 is set in the right portion of the far image display area FOV cut out by the lane mark LM2 passing through the base point BP2. The far image area FG is then positioned within the range of the far image display area FOV restricted by the display prohibited area XA1 and the display prohibited area XA2. This allows the far image to be displayed in a position that is less uncomfortable for the driver's view. The far image area FG may be positioned in either the far image display area FOV restricted by the display prohibited area XA1 or the display prohibited area X2. For example, the far image area FG may be positioned approximately in the center of the far image display area FOV excluding the display prohibited areas XA1 and XA2. As an example of a method for arranging the far image area FG in this manner, the distance d1 between the base point BP1 and the left side of the far image area FG and the distance d2 between the base point BP2 and the right side of the far image area FG can be set to be equal. However, the present invention is not limited to this, and the far image area FG may be set using the visual recognition area detected by the visual recognition area detection unit 32.

[0079] Next, as shown in FIG. 9B, if the road 50 curves to the right, the control unit 30 sets a display-prohibited area XA1 in the left portion cut off by the lane mark LM1 that passes through the base point BP1. As a result, the position of the distant image area FG is limited to a position to the right of the lane mark LM1. As mentioned above, when the road is curved, the driver's line of sight tends to shift in the direction of the curve. In the example of FIG. 9B, the driver's line of sight tends to shift to the right from the center of the road width. To address this tendency, in this embodiment, the display-prohibited area XA1 is set in the direction opposite to the curvature of the road 50, so the distant image area FG is shifted to the right. As a result, the display position of the distant image is set to a position that is easily visible to the driver.

[0080] The position of the far image area FG may be freely moved left and right, up and down, and set between the display prohibited area XA1 and the right side of the far image display area FOV. For example, the far image area FG may be positioned approximately in the center of the far image display area FOV excluding the display prohibited area XA1. As an example of a method for positioning the far image area FG, the distance d1 between the base point BP1 and the left side of the far image area FG and the distance d2 between the base point BP2 and the right side of the far image area FG may be set to be equal. However, this is not limited to this, and the far image area may be set using the visual recognition area detected by the visual recognition area detection unit 32. Furthermore, the amount by which the far image area FG is shifted to the right may be predetermined. For example, the distance may be set to approximately 25% of the left-right width of the far image area FG. Note that if the road 50 curves left, the right in the above description can be replaced with left, and detailed description thereof will be omitted.

[0081] Referring to FIG. 9( c), the setting of the display-prohibited area XA when the road 50 is uneven will be described. When the road is uneven, the distant image (distant image area FG) will naturally move (vibrate) in the vertical direction. In particular, when the host vehicle approaches a convex portion of the road 50, the distant image will overlap with the vehicle ahead 54 (not shown in FIG. 9), which is expected to cause the driver to feel uncomfortable. Therefore, in this embodiment, the forward monitoring unit 20 detects a vanishing point VP of the road 50 on which the host vehicle is traveling. As shown in FIG. 9( c), the vanishing point VP is detected as the intersection of the extension of the lane mark LM1 and the extension of the lane mark LM2. In this case, the vanishing point VP may be detected in cooperation with the gradient detection unit 33 when the gradient detection unit 33 detects an upward gradient.

[0082] As shown in FIG. 9C, the area above the vanishing point VP is set as a display prohibited area XA. FIG. 9C shows an example of setting the display prohibited area XA when the vanishing point VP has risen to the position of point VP' in the far image display area FOV. The width W may be determined taking into consideration the width of movement due to unevenness in the far image area FG. Furthermore, display prohibited areas are set to the left of the lane mark LM1 passing through the base point BP1 and to the right of the lane mark LM2 passing through the base point BP2. As a result, in this embodiment, the display prohibited area XA is set as a polygon with six vertices, as shown in FIG. 9C. The far image area FG is controlled to be located in an area other than the display prohibited area XA in the far image display area FOV. As a result, the display position of the far image is set at a position that is easily visible to the driver. As described above, this embodiment can also achieve the same effects as the second embodiment.

[0083] Third Embodiment An image projection device and an image projection method according to this embodiment will be described with reference to Fig. 10. An image projection device 10B according to this embodiment is a configuration in which the method of moving a distant image is changed in the image projection device 10 according to the above embodiment. Therefore, the same components as those in Fig. 1 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0084] 10, in an image projection device 10B, a free-form surface mirror 13 is connected to a control unit 30 instead of the image projection unit 11. The free-form surface mirror 13 is provided with a mechanism (not shown) for adjusting the tilt angle, and by adjusting this tilt angle, the projection position of the distant image light from the image projection unit 11 onto the windshield 16 is adjusted, and as a result, the vertical direction of the distant image is adjusted. The specific method and procedure for correcting the display position of the forward display image are the same as those described with reference to FIGS. 3 and 5.

[0085] In the case of the image projection device 10B shown in Figure 10, adjusting the tilt angle of the free-form surface mirror 13 simultaneously moves the near image. However, since the display position of the near image is less sensitive to changes in the tilt angle than the display position of the far image, this is not a problem in most cases. However, if you want to move only the far image, you can move the optical branching unit 14 or the free-form surface mirror 15. Alternatively, separate free-form surface mirrors 13 may be provided for the far image light and the near image light.

[0086] As described above, the image projection device and image projection method according to this embodiment also make it possible to provide an image projection device and image projection method that suppresses discomfort in viewing the forward display image due to changes in the vehicle's driving environment. In particular, the image projection device and image projection method according to this embodiment have the advantage that, compared to the first and second embodiments, the range of movement of the forward display image in the up and down direction can be increased, and there is no need to increase the pixel area 40 of the image projection unit 11. While this embodiment has been described with reference to an example in which the tilt angle is changed by the free-form surface mirror 13, this is not limiting. The image projection device and image projection method may also be configured to change the angle in the left and right directions, allowing the distant image to move up and down and left and right.

[0087] Although the image projection device and image projection method according to each embodiment have been described individually in the above embodiments, multiple embodiments may be combined. For example, the first and third embodiments may be combined to enable both vertical and horizontal movement of the distant image by the image projection unit 11 and vertical movement of the distant image by the free-form surface mirror 13. In this case, a priority may be set between the movement of the distant image by the image projection unit 11 and the movement of the distant image by the free-form surface mirror 13. For example, the movement of the distant image by the image projection unit 11 may be prioritized, and the movement of the distant image by the free-form surface mirror 13 may be applied to a range beyond the image projection unit 11's capabilities.

[0088] In the above embodiments, the distant image is moved by moving the distant pixel region 41 of the image projection unit 11 or by changing the tilt angle of the free-form surface mirror 13. However, the present invention is not limited to this, and the distant image may be moved by moving the image projection unit 11 itself. According to this embodiment, the range of movement of the distant image can be increased.

[0089] Furthermore, in each of the above embodiments, an example is given of setting the display prohibited area XA based on the visual area detected by the visual area detection unit 32, but this is not limited to this, and the display prohibited area XA may also be set using forward road environment information acquired by the forward monitoring unit 20, such as the rear area of ​​the leading vehicle 54.

[0090] This international application claims priority based on Japanese Patent Application No. 2023-187320, filed on October 31, 2023, the entire contents of which are incorporated herein by reference.

[0091] The above descriptions of specific embodiments of the present invention have been presented for purposes of illustration. They are not intended to be exhaustive or to limit the invention to the precise forms described. Numerous modifications and variations will be apparent to those skilled in the art in light of the above description.

[0092] DESCRIPTION OF SYMBOLS 10, 10A, 10B...Image projection device 11...Image irradiation unit 12...Reflector 13...Free-form surface mirror 14...Light branching unit 15...Free-form surface mirror 16...Windshield 17, 18...Virtual image 20...Forward monitoring unit 21...Gaze measurement unit 22...Angular velocity measurement unit 23...Position information acquisition unit 24...Map information acquisition unit 30...Control unit 31...Front vehicle detection unit 32...Viewable area detection unit 33...Slope detection unit 34...Display prohibited area setting unit 35...Far image display position correction unit 36...Image control signal generation unit 37...Memory unit 38...Road shape detection unit 40...Pixel area 41...Far pixel area 42...Margin area 43...Near pixel area 50...Road 52...Viewpoint 54...Front vehicle BP1, BP2...Base point FG...Far image area D1, D2, D3, D4... Direction FOV... Far image display area UL... Extension line LM, LM1, LM2, LM3... Lane mark UL... Top edge VP... Vanishing point VP'... Point W... Width XA, XA1, XA2... Display prohibited area

Claims

1. An image projection device that projects a projection image onto a display unit for displaying a virtual image, comprising: an image irradiation unit that irradiates the projection image onto the display unit as image light; and a monitoring unit that monitors the driving environment ahead of the vehicle, wherein a display prohibited area in which projection of the projection image is prohibited is set according to the driving environment.

2. An image projection device as described in claim 1, further comprising: a measurement unit that measures the driver's visual field; and a control unit that, when the driving environment satisfies predetermined conditions, sets a display prohibition area in which projection of the projected image is prohibited based on the visual field, and corrects the display position of the projected image so as to avoid the display prohibition area.

3. An image projection device according to claim 2, wherein the control unit corrects the display position of the projected image by moving the projected image below the display prohibited area.

4. An image projection device according to claim 2, wherein the predetermined condition is a distance between the vehicle and a preceding vehicle.

5. An image projection device according to claim 2, further comprising an angular velocity measuring unit that measures a gradient of the vehicle, and the control unit corrects the display prohibited area based on the gradient.

6. An image projection device according to claim 2, wherein the monitoring unit detects a curvature of the road, and the control unit corrects the display-prohibited area based on the curvature.

7. An image projection device as described in claim 2, characterized in that the control unit corrects the display position of the projection image so as to avoid the display prohibited area by changing the display position of the projection image in the image irradiation unit.

8. An image projection device according to claim 2, characterized in that the control unit changes the position of the image projection unit to correct the display position of the projected image so as to avoid the display prohibited area.

9. An image projection method using an image projection device that projects a projection image onto a display unit for displaying a virtual image, the image projection device comprising: an image projection unit that irradiates the projection image onto the display unit as image light; and a monitoring unit that monitors the driving environment ahead of the vehicle, the image projection method being characterized in that a display prohibited area in which projection of the projection image is prohibited is set according to the driving environment.

Images