Imge irradiation apparatus

Abstract

An image irradiation apparatus (100) which displays an image is provided with a detection unit (12) that detects a sign installed in a predetermined position and indicating road information, and an image irradiation unit (14) that irradiates an image onto a display unit (25) on the basis of the predetermined position detected by the detection unit (12).

Description

Image irradiation device

[0001] The present disclosure relates to an image projection device that can be mounted on a vehicle.

[0002] Conventionally, technologies have been developed for projecting images to assist vehicle driving so as to be superimposed on real scenery. For example, International Publication No. 2020 / 031912 (Patent Document 1) discloses a vehicle display system provided in a vehicle, the vehicle display system including a first display device configured to emit a light pattern toward a road surface outside the vehicle, and a second display device located inside the vehicle and configured to display predetermined information toward an occupant of the vehicle so that the predetermined information is superimposed on the real space outside the vehicle.

[0003] International Publication No. 2020 / 031912

[0004] There is a need for a technology that goes beyond the technology described in Patent Document 1 and that can display an image to be superimposed on a real landscape in a more appropriate position.

[0005] The present disclosure has been made to solve the above-mentioned problems, and its purpose is to provide an image projection device that can display an image to be superimposed on a real landscape at a more appropriate position.

[0006] In order to achieve the above object, an image projection device according to one aspect of the present disclosure is an image projection device that displays an image, and includes a detection unit that detects a sign indicating road information installed at a predetermined position, and an image projection unit that projects the image onto a display unit based on the predetermined position detected by the detection unit.

[0007] According to the present disclosure, an image to be superimposed on a real landscape can be displayed at a more appropriate position.

[0008] Fig. 1 is a diagram illustrating a configuration of an image projection device according to an embodiment of the present disclosure. Fig. 2 is a diagram illustrating an example of an image displayed by the image projection unit illustrated in Fig. 1. Fig. 3A is a diagram illustrating an example of position estimation of a colored line by a position estimation unit in the image projection unit illustrated in Fig. 1. Fig. 3B is a diagram illustrating another example of position estimation of a colored line by the position estimation unit in the image projection unit illustrated in Fig. 1. Fig. 4A is a diagram illustrating another example of position estimation of a colored line by the position estimation unit in the image projection unit illustrated in Fig. 1. Fig. 4B is a diagram illustrating another example of position estimation of a colored line by the position estimation unit in the image projection unit illustrated in Fig. 1. Fig. 5 is a flowchart illustrating an operation procedure when an image projection device according to an embodiment of the present disclosure projects an image of pseudo-colored lines onto a display unit.

[0009] <Configuration and Basic Operation> [Configuration of Image Projection Device] The present disclosure will be described with reference to the drawings. Fig. 1 is a diagram showing the configuration of an image projection device according to an embodiment of the present disclosure.

[0010] 1, the image projection device 100 is a HUD (Head-Up Display) that is mounted on a vehicle 1 and displays an image related to the traveling of the vehicle 1 so that the image is superimposed on an external real space as seen by an occupant of the vehicle 1. Hereinafter, the image projection device 100 is also referred to as "HUD 100." The HUD 100 is disposed, for example, in a dashboard inside the vehicle 1.

[0011] In addition to the HUD 100, the vehicle 1 further includes an illuminance sensor 21, a camera 22, a LiDAR (Light Detection and Ranging) 23, a navigation device 24, and a display unit 25. The HUD 100 includes a visibility estimation unit 11, a detection unit 12, a storage unit 13, and an image projection unit 14.

[0012] The HUD 100 displays, for example, an image that imitates a colored line such as a white line painted on a road surface, or a sign that indicates traffic restrictions painted on the road surface, on the display unit 25. In this example, a case will be described in which the HUD 100 displays, on the display unit 25, an image of a pseudo-colored line that imitates a colored line such as a white line painted on a road surface.

[0013] (Visibility Estimation Unit) The illuminance sensor 21 measures the illuminance around the vehicle 1 and transmits the measurement result to the HUD 100. The camera 22 captures an image of the area around the vehicle 1 and transmits the captured image to the HUD 100. The LiDAR 23 measures the distance and direction from the vehicle 1 to an object present around the vehicle 1 and transmits the measurement result to the HUD 100.

[0014] The visibility estimation unit 11 determines whether visibility is poor around the vehicle 1 based on at least one of the measurement result from the illuminance sensor 21 and the captured image from the camera 22. For example, assume that the visibility estimation unit 11 receives a measurement result from the illuminance sensor 21 indicating that the illuminance around the vehicle 1 is less than a predetermined value. In this case, the visibility estimation unit 11 determines that visibility is poor outside the vehicle 1 due to factors such as the current time being during the nighttime hours or the current weather being poor, such as rain or cloudy. The visibility estimation unit 11 then outputs the determination result indicating poor visibility to the detection unit 12.

[0015] Meanwhile, suppose that the visibility estimation unit 11 receives a measurement result from the illuminance sensor 21 indicating that the illuminance around the vehicle 1 is equal to or greater than a predetermined value. In this case, the visibility estimation unit 11 determines that the visibility outside the vehicle 1 is good. Then, the visibility estimation unit 11 outputs the determination result indicating that the visibility is good to the detection unit 12.

[0016] For example, suppose that the average brightness value per unit area in the image captured by the camera 22 is less than a predetermined value. In this case, the visibility estimation unit 11 determines that visibility outside the vehicle 1 is poor due to factors such as the current time being during the nighttime or the current weather being poor. The visibility estimation unit 11 then outputs the determination result indicating poor visibility to the detection unit 12.

[0017] On the other hand, if the average in the image captured by the camera 22 is equal to or greater than a predetermined value, the visibility estimation unit 11 determines that the visibility outside the vehicle 1 is good. Then, the visibility estimation unit 11 outputs the determination result that the visibility is good to the detection unit 12.

[0018] (Detection Unit) When the detection unit 12 receives a judgment result indicating poor visibility from the visibility estimation unit 11, it detects signs (hereinafter referred to as "traffic signs") that indicate road information and are installed at predetermined locations.

[0019] More specifically, the memory unit 13 stores traffic sign information indicating, for example, the specified range of height of a delineator (visual delineator) D, which is an example of a traffic sign, the specified range of diameter of the reflector provided on the head Db of the delineator D, the reflective performance of the reflector, and the installation interval of the delineators D.

[0020] The detection unit 12 detects a plurality of consecutive delineators D included in the image captured by the camera 22 using various information related to the delineators D indicated by the traffic sign information stored in the memory unit 13 .

[0021] Specifically, when three or more lights are lined up in the captured image, the detection unit 12 determines whether the diameter of each light is 100 mm or less, which is included in the specified range of the diameter of the reflector provided on the head Db of the delineator D. Then, when the diameter of each light is 100 mm or less, the detection unit 12 determines that these lights are reflectors of the delineator D, for example, and outputs the detection result including the position of each delineator D to the image projection unit 14.

[0022] The detection unit 12 may detect a traffic sign such as a delineator D regardless of the determination result from the visibility estimation unit 11. In this case, the HUD 100 does not need to include the visibility estimation unit 11.

[0023] The detection unit 12 may detect not only the delineator D but also lane dividers and the like as traffic signs. In this case, the memory unit 13 stores information such as a specified range of lane divider heights. That is, in this example, a traffic sign refers to a sign that provides traffic information and is installed consecutively along a road, such as a roadway or a sidewalk, at a predetermined position relative to the road (for example, at the center in the width direction or a predetermined distance inward from an edge in the width direction). In this example, a traffic sign refers to a sign that is required to have a predetermined height by law or regulation. Note that traffic signs are not necessarily installed on roads, but may be installed outside the width direction of the road.

[0024] (Image Projection Unit) Fig. 2 is a diagram showing an example of an image displayed by the image projection unit shown in Fig. 1. With reference to Fig. 1 and Fig. 2, the image projection unit 14 has a position estimation unit 31, an image generation unit 32, and an projection unit 33, and displays an image such as a pseudo-colored line on the display unit 25 based on the position of the traffic sign detected by the detection unit 12.

[0025] More specifically, the position estimation unit 31 estimates, for example, the position of a colored line drawn on the road surface based on the detection result of the delineator D from the detection unit 12 and the measurement result from the LiDAR 23. Details of the estimation of the position of the colored line by the position estimation unit 31, i.e., the estimation of the display position of the image G of the pseudo colored line, will be described later.

[0026] The image generation unit 32 generates an image G of pseudo colored lines that imitate the colored lines so that the image G is superimposed on the position of the colored lines estimated by the position estimation unit 31 when an occupant of the vehicle 1 looks at the display unit 25. The irradiation unit 33 displays the image G generated by the image generation unit 32 on the display unit 25. The display unit 25 is provided, for example, on the windshield of the vehicle 1 or on a combiner provided in the passenger compartment.

[0027] [Position Estimation of Colored Lines] (When the Gradient of the Road on Which the Vehicle Travels is the Same as the Gradient of the Road Surface on Which the Delineator is Located) FIGS. 3A and 3B are diagrams for explaining an example of the position estimation of colored lines by the position estimation unit in the image projection unit shown in FIG. 1. A plurality of traffic signs are provided along the road at predetermined positions determined in advance with respect to the road. This makes it possible to estimate the shape of the road based on the positions of the traffic signs, and to estimate the positions of the colored lines based on the estimated road shape. In this example, a case will be described in which the detection unit 12 detects a plurality of delineators D as traffic signs. In addition, this example will describe a case in which the gradient of the road on which the vehicle 1 travels is the same as the gradient of the road surface on which the delineator D is located.

[0028] Referring to Figures 3A and 3B, the position estimation unit 31 assumes that a colored line exists at the lower end Da of each delineator D detected by the detection unit 12, and estimates the position of the colored line based on the detection results of the delineator D from the detection unit 12 and the measurement results from the LiDAR 23.

[0029] More specifically, for each delineator D detected by the detection unit 12, the position estimation unit 31 determines, based on the measurement results from the LiDAR 23, the distance L from the occupant of the vehicle 1 to the head Db of the delineator D and the angle θdel, which is the angle of elevation or depression when the occupant views the delineator D. In addition, the position estimation unit 31 checks the height hdel of the delineator D by referring to traffic sign information stored in the memory unit 13.

[0030] Then, the position estimation unit 31 calculates the depression angle θgr when the occupant looks down at the lower end Da of the delineator D, i.e., the position where the colored line is located, by calculating equation (1) using the identified distance L and angle θdel, as well as the confirmed height hdel.

[0031] In equation (1), as shown in Fig. 3A, when the head Db of the delineator D is located above the line Ho extending horizontally from the occupant, that is, when the occupant looks up at the head Db of the delineator D, the angle θdel, which is the elevation angle, is a positive value. On the other hand, as shown in Fig. 3B, when the head Db of the delineator D is located below the line Ho, that is, when the occupant looks down at the head Db of the delineator D, the angle θdel, which is the depression angle, is a negative value.

[0032] In this way, the position estimation unit 31 can estimate the position of the colored line by calculating the depression angle θgr for each delineator D.

[0033] For each depression angle θgr calculated by the position estimation unit 31, the image generation unit 32 identifies the point where a straight line having an angle θgr with the line Ho intersects with the display unit 25, and generates an image G of pseudo-colored lines so that the image G is superimposed on the identified points.

[0034] The position estimation unit 31 may estimate the position of the colored line based on the detection result from the detection unit 12 and the measurement result from the LiDAR 23, as well as on the position information and map information indicating the current traveling position of the vehicle 1 output from the navigation device 24. That is, the position estimation unit 31 may comprehensively use the position of the colored line estimated by calculating the depression angle θgr for each delineator D and the position of the colored line estimated based on the current traveling position of the vehicle 1 and the map information.

[0035] (When the gradient of the road on which the vehicle is traveling is different from the gradient of the road surface on which the delineator is located) The position estimation unit 31 may estimate the position of the colored line based on the gradient of the road on which the vehicle 1 is traveling, in addition to the detection results of the delineator D from the detection unit 12 and the measurement results from the LiDAR 23.

[0036] 4A and 4B are diagrams for explaining another example of the position estimation of the colored line by the position estimation unit in the image projection unit shown in Fig. 1. Here, a case will be described in which the gradient of the road on which the vehicle 1 is traveling differs from the gradient of the road surface on which the delineator D is located.

[0037] Referring to Figures 4A and 4B, the position estimation unit 31 acquires the angle θr, which is the difference between the gradient of the road on which the vehicle 1 is traveling and the gradient of the road surface on which the delineator D is located, based on, for example, map information output from the navigation device 24 installed in the vehicle 1.

[0038] In this case, the position estimation unit 31 calculates the depression angle θgr when the occupant looks down at the lower end Da of the delineator D by calculating equation (2) using the distance L from the occupant of the vehicle 1 to the head Db of the delineator D, the angle θdel which is the elevation or depression angle when the occupant looks at the delineator D, and the height hdel of the delineator D, as well as the acquired angle θr.

[0039] In equation (2), as in Fig. 3A, when the occupant looks up at the head Db of the delineator D as shown in Fig. 4A, the angle θdel, which is the elevation angle, takes a positive value. On the other hand, when the occupant looks down at the head Db of the delineator D as shown in Fig. 4B, the angle θdel, which is the depression angle, takes a negative value in equation (2). In addition, the angle θr takes a positive value when the gradient of the road surface on which the delineator D is located is an uphill slope as viewed from the road on which the vehicle 1 is traveling, and takes a negative value when the gradient is a downhill slope.

[0040] In this way, the position estimation unit 31 can estimate the position of the colored line by calculating the depression angle θgr for each delineator D.

[0041] In the case shown in Figure 4A or 4B, as in the case shown in Figure 3A or 3B, the position estimation unit 31 may estimate the position where the colored line exists based on the detection results from the detection unit 12 and the measurement results from the LiDAR 23, as well as the current traveling position of the vehicle 1 and map information output from the navigation device 24.

[0042] <Flow of Operation> Next, a description will be given of an operation performed by the HUD 100 according to an embodiment of the present disclosure when projecting an image G of pseudo-colored lines onto the display unit 25. Fig. 5 is a flowchart showing an operation procedure performed by the HUD according to an embodiment of the present disclosure when projecting an image of pseudo-colored lines onto the display unit.

[0043] 5 , first, when the ignition switch of the vehicle 1 is turned on, for example, the visibility estimation unit 11 in the HUD 100 acquires the measurement result from the illuminance sensor 21, the captured image from the camera 22, and the measurement result from the LiDAR 23 (step S101). The measurement result from the illuminance sensor 21, the captured image from the camera 22, and the measurement result from the LiDAR 23 are transmitted to the HUD 100 periodically or irregularly.

[0044] Next, the visibility estimation unit 11 determines whether or not visibility is poor outside the vehicle 1 based on the measurement results obtained from the illuminance sensor 21 and the captured image from the camera 22, and outputs the determination result to the detection unit 12 (step S102).

[0045] Next, when the detection unit 12 receives a judgment result from the visibility estimation unit 11 indicating that visibility outside the vehicle 1 is poor ("YES" in step S102), the detection unit 12 checks, for example, whether three or more lights are lined up in the image captured by the camera 22 acquired in step S101 (step S103).

[0046] Next, if three or more lights are lined up in the captured image ("YES" in step S103), the detection unit 12 checks, for example, by referring to traffic sign information stored in the memory unit 13, whether the diameter of each light shown in the captured image is within the specified range of the diameter of the reflector of the delineator D (step S104).

[0047] Next, if the diameter of each light reflected in the captured image is within the specified range of the diameter of the reflector of the delineator D ('YES' in step S104), the detection unit 12, for example, identifies the position of each delineator D in the captured image and outputs the detection result including the identified position to the image projection unit 14 (step S105).

[0048] Next, the position estimation unit 31 in the image projection unit 14 acquires position information and map information indicating the current traveling position of the vehicle 1 from the navigation device 24 (step S106).

[0049] Next, based on the acquired location information and map information, the position estimation unit 31 checks whether the gradient of the road on which the vehicle 1 is traveling is the same as the gradient of the road surface on which the delineator D is located as detected by the detection unit 12 (step S107).

[0050] Next, if the gradient of the road on which the vehicle 1 is traveling is the same as the gradient of the road surface on which the delineator D detected by the detection unit 12 is located ('YES' in step S107), the position estimation unit 31 estimates the position of the colored line by using the above-mentioned equation (1) (step S108).

[0051] On the other hand, if the gradient of the road on which the vehicle 1 is traveling is different from the gradient of the road surface on which the delineator D detected by the detection unit 12 is located ("NO" in step S107), the position estimation unit 31 estimates the position of the colored line by using the above-mentioned equation (2) (step S109).

[0052] Next, based on the positions of the colored lines estimated by the position estimation unit 31, the image generation unit 32 generates the image G of the pseudo colored lines so that the image G is superimposed on the positions of the actual colored lines when an occupant of the vehicle 1 looks at the display unit 25. Then, the projection unit 33 displays the image G generated by the image generation unit 32 on the display unit 25 (step S110).

[0053] In addition, if the visibility estimation unit 11 determines in step S102 that the visibility outside the vehicle 1 is good ("NO" in step S102), if three or more lights are not lined up in the captured image in step S103 ("NO" in step S103), or if the diameter of each light shown in the captured image in step S104 is not within the specified range of the diameter of the reflector of the delineator D ("NO" in step S104), the processing for displaying the image by the HUD 100 is terminated.

[0054] Furthermore, the HUD 100 repeats the operations of steps S101 to S110 described above until, for example, the ignition switch of the vehicle 1 is turned off.

[0055] However, there is a need for a technology that goes beyond the technology described in Patent Document 1 and can display an image to be superimposed on a real landscape at a more appropriate position. In response to this need, in the HUD 100 according to the embodiment of the present disclosure, as described above, the detection unit 12 detects a sign indicating road information that is installed at a predetermined position. Furthermore, the image projection unit 14 projects the image onto the display unit 25 based on the predetermined position detected by the detection unit 12.

[0056] In this way, by displaying images based on the locations of traffic signs indicating road information, it is possible to display images to assist driving, such as images that mimic colored lines painted on the road, in more appropriate locations, which is particularly effective at night, in poor weather, when there are no street lights, or when driving on winding roads such as mountain roads.

[0057] Furthermore, in the HUD 100 according to an embodiment of the present disclosure, the detection unit 12 detects multiple traffic signs that are arranged consecutively and have known heights. The image projection unit 14 projects an image onto the display unit 25 based on the position of each traffic sign detected by the detection unit 12. In this manner, a configuration that detects multiple objects that are arranged consecutively and have known heights as traffic signs can prevent erroneous detection of construction lights or residential lights as traffic signs.

[0058] Furthermore, in the HUD 100 according to the embodiment of the present disclosure, the image displayed on the display unit 25 is an image of pseudo-colored lines that imitate colored lines painted on the road surface. The image projection unit 14 estimates the position of the colored lines based on the current traveling position of the vehicle 1 and map information in addition to the position of the traffic sign, and projects the image onto the display unit 25 so that the image of the pseudo-colored lines is superimposed on the estimated position of the colored lines.

[0059] This configuration allows the position of the actual colored line to be more accurately estimated, and the image of the pseudo colored line can be displayed so that it is more accurately superimposed on the actual colored line as seen by the occupant. Also, since the image is displayed at a different position from the actual traffic sign as seen by the occupant, it is possible to avoid the actual traffic sign becoming invisible.

[0060] Furthermore, in the HUD 100 according to the embodiment of the present disclosure, the image projection unit 14 projects an image onto the display unit 25 based on the distance from the head of the occupant of the vehicle 1 to the head of the traffic sign, the elevation angle or depression angle when the occupant views the traffic sign, and the height of the traffic sign. With this configuration, it is possible to display an image at a more appropriate position that takes into account how the image is viewed by the occupant.

[0061] Furthermore, in the HUD 100 according to the embodiment of the present disclosure, the image projection unit 14 further projects an image onto the display unit 25 based on the gradient of the road on which the vehicle 1 is traveling. With this configuration, it is possible to display an image at a more appropriate position that takes into account the gradient of the road on which the vehicle 1 is traveling.

[0062] In the HUD 100 according to the embodiment of the present disclosure, the traffic signs detected by the detection unit 12 are delineators. In this manner, the size range and height of the delineators, which have known reflectors, can be detected easily and accurately.

[0063] Although the embodiments of the present disclosure have been described above, it goes without saying that the technical scope of the present disclosure should not be interpreted as being limited by the description of the present embodiments. The present embodiments are merely examples, and it will be understood by those skilled in the art that various modifications of the embodiments are possible within the scope of the invention described in the claims. The technical scope of the present disclosure should be determined based on the scope of the invention described in the claims and its equivalents.

[0064] For example, in the above-described embodiment, the pseudo colored lines are displayed as images superimposed on the actual colored lines. However, the present disclosure is not limited to this. For example, the pseudo colored lines may be displayed without being superimposed on the actual colored lines. Alternatively, the shape of the road may be estimated based on signs installed at predetermined positions relative to the road, and a display based on the estimated road shape may be performed. For example, the image may display text or shapes that convey the magnitude of the road's curvature radius or a warning based on the curvature radius (e.g., a warning of a sharp curve), or a warning based on the magnitude or gradient of the road (e.g., a warning of a steep gradient or a warning to reduce speed on a gentle uphill slope).

[0065] This application is based on Japanese Patent Application No. 2022-056297, filed on March 30, 2022, the contents of which are incorporated herein by reference.

Claims

1. An image projection device for displaying an image, a detection unit that detects signs indicating road information installed at predetermined positions; an image projection unit that projects the image onto a display unit based on the predetermined position detected by the detection unit.

2. the detection unit detects a plurality of the markers that are provided consecutively and have known heights; The image projection device according to claim 1 , wherein the image projection unit projects the image onto the display unit based on the position of each of the markers detected by the detection unit.

3. the image is an image of pseudo-colored lines that mimic colored lines painted on a road surface, 3. The image projection device according to claim 1, wherein the image projection unit estimates a position where the colored line is located based on a current traveling position of a vehicle equipped with the image projection device and map information in addition to the position of the sign, and projects the image onto the display unit so that the image is superimposed and displayed at the estimated position.

4. 3. The image projection device according to claim 1, wherein the image projection unit projects the image onto the display unit based on, for each sign, a distance from the head of an occupant of a vehicle equipped with the image projection device to the head of the sign, an elevation angle or depression angle when the occupant views the sign, and a height of the sign.

5. The image projection device according to claim 4 , wherein the image projection unit further projects the image onto the display unit based on a gradient of a road on which the vehicle is traveling.

6. 3. The image projection device according to claim 1, wherein the mark is a delineator.