Display system, information presentation system with a display system, method for controlling a display system, recording medium and mobile body with a display system

Abstract

A display system that projects an image so that a person visually perceives a virtual image displayed in a target space in front of a mobile body, wherein the display system comprises: a projector designed to project the image; and a controller designed to control the projection of the image, wherein the control regulates the projection of the image according to an inclination of the mobile body, which is obtained from data measured by the mobile body, such that the virtual image is displayed in such a way that it contains initial attribute information representing an inclination of a road surface on which the mobile body is moving, and The projection of the image into the target space is controlled in such a way that it has a partially different light transmission within itself, in order to appear partially transparent, so that the road surface can be seen.

Description

Technical field

[0001] The present disclosure relates generally to a display system, an information presentation system with a display system, a method for controlling a display system, a program, and a mobile body with a display system. In particular, the present disclosure relates to a display system that projects a virtual image into a target space, an information presentation system with a display system, a method for controlling a display system, a program, and a mobile body with a display system. Technical background

[0002] In order to provide a display device for a vehicle capable of assisting a driver's safe driving operation, JP 2006-243888A proposes that this device comprise a display that shows a moving road shape based on map information consisting of three-dimensional information stored in a map information storage device; a position detection device that detects a current position of a vehicle; and a control device that, based on current position data of the vehicle acquired by the position detection device and the map information, displays the moving road shape in front of the vehicle on the display.

[0003] This device further comprises an imaging device that captures an image of the front of the vehicle and a temperature sensor that detects the ambient temperature of the vehicle. Based on the data captured by the imaging device and the temperature data detected by the temperature sensor, the control unit displays a section (road surface) that may freeze over in the driving road shape shown on the display.

[0004] German patent application DE 10 2014 219 575 A1 provides embodiments of techniques or systems for three-dimensional (3D) navigation. A head-up display (HUD) component can project, render, display, or present graphic elements on focal planes around an environment surrounding a vehicle. The HUD component can cause these graphic elements to appear volumetric or three-dimensional by moving or adjusting the distance between a focal plane and the vehicle. Objects within the environment are tracked and identified, and corresponding graphic elements can be projected onto, near, or around the respective objects. For example, the HUD component can project graphic elements or pointers onto pedestrians in such a way as to alert the driver or operator of the vehicle to their presence. These pointers can "stick" to the pedestrian if he or she is walking within the environment.Metadata related to objects can be presented, such as address information, ratings, telephone numbers, logos, etc.

[0005] A conventional vehicle display device (display system) is known that reflects a display image projected from a main display unit onto an inner surface of a windshield via a reflector in order to display the image remotely as a virtual image. Such a device is disclosed, for example, in patent specification 1. A vehicle display device disclosed in patent specification 1 displays route guidance information (for example, displaying a road map near the current position and direction of travel) and guidance indices (for example, arrow indices indicating straight ahead, left turns, and right turns) as virtual images. List of quotations Patent literature

[0006] Patent Specification 1: Unexamined Japanese Patent Publication No. JP 2004-168 230 A Summary of the invention

[0007] The present disclosure aims to provide a display system that makes it easy to detect a condition of a road surface on which the mobile body moves, an information presentation system with a display system, a method for controlling a display system, a program and a mobile body with a display system.

[0008] A display system according to one aspect of the present disclosure is defined in claim 1.

[0009] According to another aspect of the present disclosure, an information presentation system includes the aforementioned display system and a detection system. The detection system detects an object in the vicinity of the mobile body.

[0010] A method for controlling a display system according to yet another aspect of the present disclosure is defined in claim 7.

[0011] A program according to one aspect of the present disclosure is a program to cause a computer system to execute the above-mentioned method for controlling the display system.

[0012] A mobile body, according to one aspect of the present disclosure, contains the aforementioned display system and a reflective element. The reflective element has optical transparency and reflects light emitted by the projector.

[0013] The present disclosure has the advantage of allowing a simple determination of a condition of a road surface on which a mobile body moves. Brief description of the drawing Fig. Figure 1 is a concept diagram of a vehicle that includes a display system according to an exemplary embodiment of the present disclosure. Fig. Figure 2 is a concept diagram showing configurations of the display system and an information presentation system. Fig. Figure 3 is a concept diagram that represents a user's field of view when using the display system. Fig. Figure 4 is a concept diagram that represents the user's field of view when a virtual image, which has an initial content example, is projected using the display system. Fig. Figure 5 is a concept diagram that represents the user's field of view when the virtual image, which features the first content example (including a recommended route), is projected using the display system. Fig. Figure 6 is a flowchart that illustrates the operation of the display system when the virtual image is projected with the first content example. Fig. 7A is a bird's-eye view representing a condition of a road surface when a virtual image, which has a second content example, is projected using the display system. Fig. 7B is a bird's-eye view representing a condition of a road surface when the virtual image, which features the second content example, is projected using the display system. Fig. Figure 8 is a concept diagram that represents the user's field of view when the virtual image, which features the second content example, is projected using the display system. Fig. Figure 9 is a concept diagram representing the user's field of view when the virtual image, which features the second content example (including additional markers), is projected using the display system. Fig. Figure 10 is a flowchart that represents a function of the display system when the virtual image is projected with the second content example. Fig. 11A is a concept diagram that represents the user's field of view when a virtual image, which has a third content example, is projected using the display system. Fig. 11B is a concept diagram that represents the user's field of view when the virtual image, which features the third content example, is projected using the display system. Fig. 12A is a concept diagram representing a condition in which a vehicle is facing an incline. Fig. 12B is a concept diagram that represents the user's field of view when the virtual image, which features the third content example, is projected using the display system in the condition where the vehicle is in front of the incline. Fig. 13A is a concept diagram representing a condition in which the vehicle is located at the starting point of the incline. Fig. 13B is a concept diagram that represents the user's field of view when the virtual image, which features the third content example, is projected using the display system in the condition where the vehicle is at the starting point of the incline. Fig. 14A is a concept diagram representing a condition in which the vehicle is located in the middle of the incline. Fig. 14B is a concept diagram that represents the user's field of view when the virtual image, which features the third content example, is projected using the display system in the condition where the vehicle is in the middle of the incline. Fig. 15A is a concept diagram that represents the user's field of view when the virtual image, which features the third content example (including an additional marker), is projected using the display system. Fig. Figure 15B is a concept diagram representing the user's field of view when a partially transparent virtual image, featuring the third content example, is projected using the display system. Fig. Figure 16 is a concept diagram representing the user's field of view when a virtual image, featuring the third content example (traffic sign), is projected using the display system. Fig. 17A is a concept diagram that represents the user's field of view when a virtual image, which has a fourth content example (street name), is projected using the display system. Fig. 17B is a concept diagram that represents the user's field of view when a virtual image, featuring the fourth content example (street name), is projected using the display system. Fig. Figure 18 is a concept diagram representing the user's field of view when a virtual image, featuring the fourth content example (street direction), is projected using the display system. Fig. Figure 19 is a concept diagram representing the user's field of view when a virtual image, which has a fifth content example, is projected using the display system. Fig. Figure 20 is a concept diagram representing the user's field of view when a virtual image, which has a sixth content example, is projected using the display system. Fig. Figure 21 is a concept diagram representing the user's field of view when a virtual image, which has a seventh content example, is projected using the display system. Fig. Figure 22 is a concept diagram that represents the user's field of view when a virtual image, which has an eighth content example, is projected using the display system. Fig. Figure 23 is a concept diagram representing the user's field of view when a virtual image, which has a ninth content example, is projected using the display system. Description of the embodiment

[0014] Before describing an exemplary embodiment of the present invention, problems with conventional devices are briefly described. A driver of a vehicle (mobile body) can determine a route to a destination, but has difficulty determining the condition of a road surface on which the vehicle is moving. (1) Overview

[0015] The display system 10 according to an exemplary embodiment of the present disclosure, the information presentation system 1000 containing the display system 10, and a mobile body (in this case the vehicle 100, own vehicle) containing the display system 10 are below with reference to Fig. 1 to 3 described. The display system 10 according to this exemplary embodiment is, for example, a head-up display (HUD) used for the vehicle 100, as described in the Fig. Figures 1 to 3 are shown. The display system 10 is installed in the interior of vehicle 100 to project an image from below onto the windshield 101 of vehicle 100. In an example in Fig. The display system 10 is located in the dashboard 102 below the windshield 101. When an image from the display system 10 is projected onto the windshield 101, a user 200 (the driver of the vehicle) visually perceives an image reflected by the windshield 101. That is, the vehicle 100 (the mobile body) contains the display system 10 and a reflective element (in this case, the windshield 101). The reflective element has optical transparency and reflects light emitted by the projector 40 (described later) contained within the display system 10.

[0016] According to the display system 10, the user 200 visually perceives a virtual image 300 through the windshield 101, which is projected into a target space 400 located in front of the vehicle 100 (outside the vehicle). In this case, "virtual image" refers to an image formed by deflected light rays emitted by the display system and refracted by a reflector such as the windshield 101, creating the impression of an actual object. Therefore, the user 200 can see the virtual image 300 projected by the display system 10, so that it is superimposed on a real space extending in front of the vehicle 100.Accordingly, the display system 10 can show 300 different driver assistance information as a virtual image, such as vehicle speed information, navigation information, pedestrian information, information about a vehicle ahead, lane departure information, and vehicle status information, and allows the user 200 to visually perceive the displayed information. This allows the user 200 to visually perceive driver assistance information simply by slightly moving their line of sight, which is directed in front of the windshield 101.

[0017] In this exemplary embodiment, as in Fig. As shown in Figure 2, the display system is part of the information presentation system 1000 and receives information from an Advanced Driver Assistance System (ADAS) (described later) as part of the driver assistance information from the detection system 7. The detection system 7 is designed to detect an object in the vicinity of the vehicle 100. In other words, the information presentation system 1000 comprises the display system 10 and the detection system 7.

[0018] In the display system 10 according to this exemplary embodiment, the virtual image 300 generated in the target space 400 contains at least two types of virtual images, namely a first virtual image 301 and a second virtual image 302, as shown in the Fig. 1 and Fig. 3 shown. The “first virtual image” mentioned here is the virtual image 300 (301) generated on a first virtual plane 501. The “first virtual plane” is a virtual plane whose inclination angle α with respect to an optical axis 500 of the display system 10 is smaller than a predefined value γ (α < γ). Furthermore, the “second virtual image” mentioned here is a virtual image 300 (302) generated on a second virtual plane 502. The “second virtual plane” is a virtual plane whose inclination angle β with respect to the optical axis 500 of the display system 10 is greater than a predefined value γ (β > γ). The “optical axis” mentioned here is an optical axis of an optical system of the optical projection system 4 (see Fig. 2), which is described below, and denotes an axis that passes through a center of the target space 400 and extends along an optical path of the virtual image 300. An example of the specified value γ is 45°, and an example of the inclination angle β is 90°.

[0019] In the display system 10 according to this exemplary embodiment, the virtual image 300, which is generated in the target space 400, contains, in addition to the first virtual image 301 and the second virtual image 302, a third virtual image 303 (see Fig. 3) The “third virtual image” is similar to the second virtual image 302, the virtual image 300 (303) formed on the second virtual plane 500, whose inclination angle β with respect to the optical axis 500 is greater than the specified value γ. Among the virtual images 300 formed on the second virtual plane 502, a virtual image formed by light transmitted through a moving screen 1a is a second virtual image 302, and a virtual image formed by light transmitted through a stationary screen 1b is a third virtual image 303, as will be described in detail later.

[0020] In this exemplary embodiment, the optical axis 500 extends along the road surface 600 in front of the vehicle 100 in the target space 400 in front of the vehicle 100. The first virtual image 301 is generated on the first virtual plane 501, which is substantially parallel to the road surface 600, and the second virtual image 302 and the third virtual image 303 are generated on the second virtual plane 502, which is substantially perpendicular to the road surface 600. If the road surface 600 is, for example, a horizontal plane, the first virtual image 301 is displayed along the horizontal plane, and the second virtual image 302 and the third virtual image 303 are displayed along a vertical plane. In this case, as shown in Fig. As shown in Figure 1, the distances (also called "visual distances") from one eye (eye point) of the user 200 to the furthest and nearest parts of the first virtual image 301 are different. That is, the first virtual image 301 is a virtual image that has a depth in the direction of travel of a mobile body (vehicle 100). In other words, the depth of the first virtual image 301 varies parallel to the direction of travel of the mobile body.

[0021] Fig. Figure 3 is a concept diagram representing a user's field of vision. That is, as in Fig. As shown in Figure 3, the display system 10 according to this exemplary embodiment can display the first virtual image 301, the second virtual image 302, and the third virtual image 303. The user 200 visually perceives the first virtual image 301 as having depth along the road surface 600. The user 200 perceives the second virtual image 302 and the third virtual image 303 as being perpendicular to the road surface 600 at predetermined distances from the user 200. Therefore, in the eyes of the user 200, the first virtual image 301 appears to be on a plane that is substantially parallel to the road surface 600, and the second virtual image 302 and the third virtual image 303 appear to be on planes that are substantially perpendicular to the road surface 600.An example of content in the first virtual image 301 is navigation information indicating the direction of travel for the vehicle 100, which may be represented, for example, by an arrow on the road surface 600 indicating a right-turn or left-turn point. An example of content in the second virtual image 302 is information indicating the distance to a vehicle ahead or a pedestrian, which may be represented, for example, by a distance to the vehicle ahead (intermediate vehicle distance) on the vehicle ahead for the user 200.An example of the content of the third virtual image 303 is a current time, vehicle speed information and vehicle status information, which can be displayed to the user 200 using this information, for example with letters, numbers and symbols or an instrument such as a fuel gauge.

[0022] In the display system 10 according to this exemplary embodiment, in particular, the content of the virtual image 300 has a depth in the direction of travel of the vehicle 100 and contains attribute information of the road surface 600 on which at least the vehicle 100 is traveling. Consequently, by using the virtual image 300, it is possible, for example, to present the user 200 with a condition of the road surface 600 on which the vehicle 100 is traveling. The "attribute information of the road surface 600" mentioned here includes information about objects on the road surface 600 in addition to information about the road surface 600 itself, such as the gradient of the road surface 600 and a name of the road that contains the road surface 600.Objects on the road surface 600 include, for example, other mobile bodies (in this case, other vehicles) that are different from the vehicle 100 moving on the road surface 600, and obstacles (for example, pedestrians and construction sites) that are different from the other mobile bodies on the road surface 600. Such objects are described in detail later in “(4) Content of a virtual image”. (2) Interpretation

[0023] As in Fig. As shown in Figure 2, the display system 10, according to the exemplary embodiment, comprises a plurality of screens 1a, 1b, the drive unit 2, the projection unit 3, the optical projection system 4, the control unit 5, and the detection unit 6. According to the exemplary embodiment, the optical projection system 4 forms the projector 40, which projects the virtual image 300 (see Figure 2). Fig. 1) together with the radiating unit 3 into the target room 400 (see Fig. 1) projected.

[0024] The array of screens 1a, 1b comprises the fixed screen 1b and the movable screen 1a. The fixed screen 1b is mounted in a predetermined position relative to a housing and the like of the display system 10. The movable screen 1a is inclined at an angle θ relative to the reference plane 503. Furthermore, the movable screen 1a is designed to be movable in the direction of movement X orthogonal to the reference plane 503 (a direction in Fig. 2 (direction indicated by the arrow X1-X2). The “reference plane” mentioned here is not a real plane, but a virtual flat plane that defines a direction of movement for the movable screen 1a. The movable screen 1a is designed to move in a straight line in the direction of movement X, while maintaining a position inclined at an angle θ with respect to the reference plane 503. In the following description, any of the multiple screens 1a, 1b can be called “screen 1” if the movable screen 1a and the stationary screen 1b are not specifically distinguished from one another.

[0025] The screen 1 (movable screen 1a and fixed screen 1b) has a light transmittance and generates an image to represent the virtual image 300 (see Fig. 1) to generate in the target space 400 (see Fig. 1) That is, an image is drawn on the screen 1 by the light from the emitting unit 3, and the virtual image 300 is generated in the target space 400 by the light transmitted through the screen 1. The screen 1 consists, for example, of a rectangular, plate-shaped element with light-scattering properties. The screen 1 is arranged between the emitting unit 3 and the optical projection system 4.

[0026] The drive unit 2 moves the movable screen 1a in the direction of movement X. The drive unit 2 can move the movable screen 1a both towards and away from the optical projection system 4 along the direction of movement X. The drive unit 2 is, for example, an electrically driven actuator such as a voice coil motor and operates according to a first control signal from the controller 5.

[0027] The emission unit 3 is a scanning light emission unit and illuminates the movable screen 1a or the stationary screen 1b with light. The emission unit 3 contains the light source 31 and the scanner 32. In the emission unit 3, the light source 31 and the scanner 32 each operate according to a second control signal from the controller 5.

[0028] The light source 31 consists of a laser module that emits laser light. The light source 31 contains a red laser diode that emits a red laser beam (R), a green laser diode that emits a green laser beam (G), and a blue laser diode that emits a blue laser beam (B). The laser beams of the three colors emitted by these three types of laser diodes are synthesized, for example, by a dichroic mirror and strike the scanner 32.

[0029] During scanning, the scanner 32 illuminates a surface of the movable screen 1a or the stationary screen 1b with light from the light source 31. In this case, the scanner 32 performs a raster scan, in which a surface of the movable screen 1a or the stationary screen 1b is scanned two-dimensionally with light.

[0030] When light emitted by the emitting unit 3 is passed through the screen 1 and enters the optical projection system 4 as incident light, the optical projection system 4 projects the virtual image 300 (see Fig. 1) by the incident light into the target area 400 (see Fig. 1) In this case, the optical projection system 4 is arranged in line with the screen 1 in the direction of movement X of the movable screen 1a. The optical projection system 4 includes a magnifying lens 41, a first mirror 42 and a second mirror 43, as shown in Fig. 2 shown.

[0031] A magnifying lens 41, the first mirror 42, and the second mirror 43 are arranged in this order along the path of the light passing through the screen 1. The magnifying lens 41 is positioned in the direction of movement X on the side opposite the emitting unit 3 (side of the first direction X1), as seen from the screen 1, so that light emitted from the screen 1 in the direction of movement X enters the magnifying lens 41. The magnifying lens 41 magnifies an image produced by light from the emitting unit 3 on the screen 1 and outputs the image to the first mirror 42. The first mirror 42 reflects light from the magnifying lens 41 to the second mirror 43. The second mirror 43 reflects light from the first mirror 42 to the windshield 101 (see figure). Fig. 1) That is, the optical projection system 4, with its magnifying lens 41, enlarges an image generated by light from the emitting unit 3 on the screen 1 and projects the image onto the windshield 101, thereby projecting the virtual image 300 into the target space 400. One optical axis of the magnifying lens 41 corresponds to the optical axis 500 of the optical projection system 4.

[0032] The Controller 5 consists of a microcomputer, primarily containing components such as a central processing unit (CPU) and memory. In other words, the Controller 5 is implemented as a computer with a CPU and memory. The CPU executes a program stored in memory, enabling the computer to function as the Controller 5. In this case, the programs are pre-recorded in the Controller 5's memory. However, the programs can also be provided via a telecommunications connection, such as the internet, or by being recorded on a storage medium like a memory card.

[0033] The controller 5 controls the display of the virtual image 300, which is projected into the target space 400, by controlling the drive unit 2 and the projection unit 3. The controller 5 controls the drive unit 2 with the first control signal and controls the projection unit 3 with the second control signal. The controller 5 is designed to synchronize the operation of the drive unit 2 with the operation of the projection unit 3. As in Fig. As shown in Figure 2, the control unit 5 also functions as a drive control unit 51 and as a display control unit 52.

[0034] The drive control 51 moves the movable screen 1a relative to the reference position by controlling the drive unit 2. The "reference plane" mentioned here is a position defined at a predetermined location within the movement range of the movable screen 1a. The drive control 51 moves the movable screen 1a to project the first virtual image 301 and the second virtual image 302 into the target space 400 using light passing through the movable screen 1a. The drive control 51 controls the drive unit 2 synchronously with the drawing on the movable screen 1a by the projection unit 3.

[0035] The display control 52 controls the drive unit 2 and the projection unit 3 to display the virtual image 300, which has content based on one or more pieces of information (also referred to as "mobile body information") acquired by the sensing unit 6, in cooperation with the drive control 51. In this exemplary embodiment, the virtual image 300 (in this case, the first virtual image 301) has a depth in the direction of travel of the vehicle 100 (mobile body) and includes content that provides attribute information of the road surface 600 on which at least the vehicle 100 is traveling.

[0036] The acquisition unit 6 acquires one or more pieces of information about the mobile body, including information (also referred to as "position information") relating to the position of the vehicle 100 (mobile body), information (also referred to as "ADAS information") relating to an object in the vicinity of the vehicle 100, and information (also referred to as "vehicle information") relating to the state of the vehicle 100. That is, attribute information contained in the content of the virtual image 300 comprises one or more pieces of information from position information, ADAS information, and vehicle information.

[0037] ADAS information is information that can be acquired by a camera, sonar sensor, radar, LiDAR, or the like, which serves as a detector for an Advanced Driver Assistance System (ADAS). In this exemplary embodiment, the acquisition unit 6 acquires ADAS information from the detection system 7, which includes the image acquisition device 71 and the laser radar 72. The image acquisition device 71 captures an image of a space around the vehicle 100, which includes the target space 400. The laser radar 72 measures the distance between an object located in a space near the vehicle 100 and the vehicle 100, a property of the object, and the like.Specific examples of ADAS information include the distance from the moving object to a vehicle traveling alongside vehicle 100, the vehicle's relative coordinates with respect to vehicle 100, distances between a variety of vehicles, and the relative speeds of those vehicles. In this case, objects near vehicle 100 in the ADAS information include a vehicle traveling alongside or stopped next to vehicle 100, a structure such as a guardrail or similar, a pedestrian, and a small animal.

[0038] Vehicle information is information that represents a local state of the vehicle 100 itself and can be captured by a sensor mounted in the vehicle 100. Specific examples of vehicle information include the vehicle 100's speed (travel speed), the acceleration applied to the vehicle 100, the degree of accelerator pedal depressor (degree of accelerator pedal opening), the degree of brake pedal depressor, the steering angle, the vehicle's roll, and the tilt of a seat backrest in the vehicle. Vehicle information also includes the driver's heart rate, facial expression, and line of sight, measured by a driver monitoring sensor. Furthermore, vehicle information includes data unique to the vehicle 100, such as its width, height, overall length, and eye position.

[0039] The position information is based on the vehicle's position (100) and includes, for example, road information at the vehicle's location, which can be captured by a positioning system such as a Global Positioning System (GPS). Specific examples of position information include the number of lanes on a road at the vehicle's location, information indicating whether a road is an intersection, information indicating whether a road is a T-junction, information indicating whether a road is a one-way street, road width, information indicating whether a sidewalk is present, gradient, and curve curvature. (3) Operation

[0040] A basic operation of the display system 10 in this exemplary embodiment is described below. The control unit 5 controls the emitting unit 3 to illuminate the movable screen 1a with light from the emitting unit 3. At this moment, the emitting unit 3 emits light that scans a surface of the movable screen 1a. Thus, an image is generated (projected) onto the movable screen 1a. Light from the emitting unit 3 is transmitted through the movable screen 1a and illuminates the windshield 101 from the optical projection system 4. In this way, the image generated on the movable screen 1a is projected from below the windshield 101 into the interior of the vehicle 100 onto the windshield 101.

[0041] When the image from the optical projection system 4 is projected onto the windshield 101, the windshield 101 reflects the light from the optical projection system 4 to the user 200 (driver) inside the vehicle. This allows the user 200 to visually perceive the image reflected by the windshield 101. Consequently, the user 200 can visually perceive the virtual image 300 (first virtual image 301 or second virtual image 302) projected in front of the vehicle 100 (outside the vehicle) through the windshield 301.

[0042] Specifically, while the movable screen 1a is stationary, the controller 5 scans a surface of the movable screen 1a with light in the direction of movement X to generate the first virtual image 301, which is perceived by the user 200 with a depth impression along the road surface 600. The controller 5 scans a surface of the movable screen 1a with light while moving the movable screen 1a to establish a constant distance in direction X between a luminous point on a surface of the movable screen 1a and the optical projection system 4. Consequently, the second virtual image 302 is generated, which is perceived as being perpendicular to the road surface 600 at a predetermined distance from the user 200.

[0043] While the movable screen 1a is illuminated with light from the emitter 3, the controller 5 controls the drive unit 2 to move the movable screen 1a in the direction of movement X. If the position of the light from the emitter 3 on a surface of the movable screen 1a, i.e., the position of a luminous point, remains constant when the movable screen 1a moves in the first direction X1, the distance from the user's eye (eye point) 200 to the virtual image 300 decreases. Conversely, if the position of the luminous point on a surface of the movable screen 1a remains constant when the screen 1a moves in the second direction X2, the viewing distance to the virtual image 300 increases (becomes longer). That is, the viewing distance to the virtual image 300 changes according to the position of the movable screen 1a in the direction of movement X.

[0044] To change the viewing distance to the first virtual image 301, the controller 5, for example, moves the movable screen 1a in the X direction according to the viewing distance, locks the movable screen 1a in a position after the movement, and scans a surface of the movable screen 1a with light. To change the viewing distance to the second virtual image 302, the controller 5 moves the movable screen 1a in the X direction according to the viewing distance. The controller 5 scans a surface of the movable screen 1a with light while moving the movable screen 1a to establish a constant distance in the X direction between the light point and the optical projection system 4 with respect to the position after the movement.

[0045] The controller 5 controls the emitter 3 to illuminate the stationary screen 1b with light from the emitter 3. At this moment, the emitter 3 emits light that scans a surface of the stationary screen 1b. Similar to the case where the movable screen 1a is illuminated with light, this process creates (projects) an image on the stationary screen 1b and projects it onto the windshield 101. As a result, the user 200 can visually perceive the virtual image 300 (third virtual image 303) projected in front of the vehicle 100 (outside the vehicle) through the windshield 101.Since the third virtual image 303 is generated by light projected onto the stationary screen 1b, which is fixed in a position, the third virtual image 303 is visually perceived by the user as being perpendicular to the road surface 600 at a predetermined distance (for example, 2 m to 3 m) from the user 200.

[0046] The display system 10 according to this exemplary embodiment can project the first virtual image 301, the second virtual image 302, and the third virtual image 303 all in one cycle, during which the scanner 32 moves back and forth once in the vertical direction of the movable screen 1a (a direction inclined to the reference plane 503 of the movable screen 1a). In particular, in a "forward path" in which the movable screen 1a and the stationary screen 1b are scanned with light in that order, the projector 40 first illuminates the movable screen 1a with light to project the first virtual image 301, and then illuminates the stationary screen 1b with light to project the third virtual image 303.Then, in a “reverse path” in which the fixed screen 1b and the movable screen 1a are scanned with light in that order, the projector 40 first illuminates the fixed screen 1b with light to project the third virtual image 303, and then illuminates the movable screen 1a with light to project the second virtual image 302.

[0047] Consequently, the first virtual image 301, the third virtual image 303, and the second virtual image 302 are projected into the target space 400 in a cycle in which the scanner 32 performs longitudinal scanning. Longitudinal scanning using the projection unit 3 at a relatively high speed allows the user 200 to visually perceive the first virtual image 301, the third virtual image 303, and the second virtual image 302 as if the images were displayed simultaneously. For example, the longitudinal scanning frequency in the projection unit 3 is greater than or equal to 60 Hz. (4) Content of a virtual image

[0048] Examples of the content of the virtual image 300, which is projected into the target space 400 by the display system 10 according to this exemplary embodiment, are described below. The examples of the content of the virtual image 300 described below can be used in combination with one another. Furthermore, the virtual image 300 described below can be projected into the target space 400 regardless of the use of a navigation system, with the exception of a ninth content example. (4.1) First content example

[0049] In the first content example, as in Fig. As shown in Figure 4, a virtual image 300 is formed by the marker 310. The marker 310 is the first virtual image 301 and represents a pre-calculated route of the vehicle 100 (mobile body). The "pre-calculated route" mentioned here is a route along which the vehicle 100 is expected to travel if the user 200 maintains current steering control. Furthermore, the marker 310 has a shape that avoids the other vehicle 110 in a specific area A1 (see a dashed line in Figure 4). Fig. 4; the same applies to Fig. 5) A boundary line of marker 310 with respect to the other vehicle 110 in the display area A1 is a relatively thick line. In this case, the specific area A1 is an area located in the direction of travel of vehicle 100, where a pre-calculated route overlaps the other vehicle. The specific area A1 has a shape that avoids the other vehicle 110, as described above, with a boundary line with respect to the other vehicle 110 being a relatively thick line, thus indicating that vehicle 100 cannot enter the specific area A1. That is, in this content example, attribute information of the virtual image 300 is information that represents whether vehicle 100 can enter the specific area A1, which is located in the direction of travel of vehicle 100.

[0050] In this example content, virtual image 300 is displayed when, for example, road surface 600 meets a predefined criterion. The predefined criterion is met, for example, when a large number of vehicles cannot drive side-by-side because road surface 600 is located in a bottleneck or construction zone. Furthermore, in this example content, if vehicle 100 cannot enter the designated area A1, control 5 displays marker 311 (virtual image 300), which represents a recommended route allowing vehicle 100 to bypass the designated area A1, as shown in... Fig. 5 shown.

[0051] An operational example of display system 10 when the virtual image 300, which contains this content example, is projected, is shown below with reference to Fig. 6 described. The processing from “START” to “END” in Fig. Step 6 is repeated according to a data acquisition cycle (for example, 30 times per second) of the data acquisition unit 6. First, the controller 5 acquires one or more pieces of information from the mobile body (for example, road information at a position of the vehicle and object information in the vicinity of the vehicle) from the data acquisition unit 6 (step S101). The controller 5 then constructs a pseudo-space (virtual 3D space) based on the acquired one or more pieces of information from the mobile body, which imitates the target space 400 (step S102).

[0052] The controller 5 then determines whether the road surface 600, on which the vehicle 100 is traveling, meets the specified criterion (step S103). If the road surface 600 does not meet the specified criterion (step S103: No), the controller 5 repeats step S101 instead of performing the subsequent processing, because the road surface 600 is not located in a bottleneck. In contrast, if the road surface 600 does meet the specified criterion (step S103: Yes), the controller 5 plots a pre-calculated route in pseudo-space based on one or more pieces of information about the mobile body (for example, vehicle width, steering angle, and azimuth information) that were captured by the sensing unit 6 (step S104).

[0053] The controller 5 then determines whether an object is located on the pre-calculated route (step S105). If an object (for example, another vehicle 110) is present on the pre-calculated route (step S105: Yes), the controller 5 draws information indicating that vehicle 100 cannot enter the pseudo-space (step S106). This information indicates that vehicle 100 cannot enter the specified area A1. If there is a route that can bypass the object, the controller 5 draws a recommended route in the pseudo-space (step S107). Subsequently, the controller 5 controls the drive unit 2 and the projection unit 3 to project content, which is drawn as a virtual image 300 in the pseudo-space, into the target space 400 (step S108).This function projects marker 310, which represents a pre-calculated route, and marker 311, which represents a recommended route, onto the road surface 600 in the target area 400.

[0054] If no object is present on the pre-calculated route (step S105: No), the controller 5 determines whether an object exists in the target space 400 (step S109). If an object exists in the target space 400 (step S109: Yes), the controller 5 first draws a recommended route into the pseudo-space (step S107) and then executes step S108. If no object exists in the target space 400 (step S109: No), the controller 5 executes step S108 without drawing a recommended route into the pseudo-space.

[0055] A condition is described under which step S109 is executed. For example, if user 200 operates a steering wheel according to marker 311 after marker 311 has been projected into the target space 400, the pre-calculated route changes. If the pre-calculated route is almost identical to a recommended route, there is no object on the pre-calculated route. Under this condition, if an object is in the target space 400 (in this case, another vehicle 110 is in front of vehicle 100), the projection of marker 311 into the target space 400 is maintained. If there is no object in the target space 400, for example, after vehicle 100 has passed the other vehicle 110, the projection of marker 311 into the target space 400 is terminated.

[0056] As described above, in this content example, the virtual image 300, which contains information indicating whether vehicle 100 can enter the specified area A1 in its direction of travel, is projected into the target space 400. Accordingly, this content example is useful in simplifying the determination of whether it is necessary to change the direction of travel of vehicle 100 by allowing user 200 to visually perceive the virtual image 300. Furthermore, in this content example, if vehicle 100 can enter the specified area A1, the virtual image 300, which contains information indicating a detour route, is projected into the target space 400. The content example is therefore useful in that it allows user 200 to easily select the detour around the specified area A1, which is inaccessible, by allowing user 200 to visually perceive the virtual image 300.

[0057] In this example of the processing performed by controller 5, the processing in steps S101 and S102 can be executed after the processing in step S103 instead of before the processing in step S103. In this case, the processing in steps S101 and S102 is only executed if the road surface 600 meets a predefined criterion.

[0058] In this example, the designated area A1 in marking 310 can have a color that differs from the color of a part that is not designated area A1 if vehicle 100 cannot enter designated area A1. Based on this aspect, user 200 can easily recognize that vehicle 100 cannot enter designated area A1.

[0059] In this content example, if there is no route that can bypass an object, the controller 5 can project the virtual image 300, which contains content depicting the stopping of vehicle 100, into the target space 400. Based on this, user 200 can easily see that vehicle 100 should stop. (4.2) Second content example

[0060] In a second content example, as in the Fig. 7A and Fig. As shown in Figure 7B, the virtual image 300 is projected into the target space 400 in a condition in which the vehicle 100 (mobile body) performs a lane change from a first lane 601, on which the vehicle 100 is driving, to another lane 602. Fig. 7A represents a road surface condition 600 that allows the vehicle 100 to perform a lane change without acceleration or braking. Fig. 7B represents a condition of the road surface 600 on which another vehicle 110 is driving alongside vehicle 100, and vehicle 100 cannot change lanes without accelerating.

[0061] In the Fig. In the condition shown in 7A, if the user 200 activates a direction indicator to change lanes, the virtual image 300, consisting of the two markers 302, 313, is projected into the target space 400, as shown in Fig. Figure 8 shows that marker 312 is the first virtual image 301 and represents a specific area A1 on the second lane 602, which is distinct from the first lane 601 where vehicle 100 is traveling. The color (or shape) of the specific area A1 changes depending on whether vehicle 100 can enter the specific area A1 (that is, perform a lane change). Marker 313 is the first virtual image 301 and represents a set (in this case, two) of arrows to instruct user 200 to perform a lane change. Marker 313 is projected into the target space only if vehicle 100 can enter the specific area A1.

[0062] In the Fig. In the condition shown in Figure 7A, when the user 200 activates the turn signal, the control unit 5 determines, based on one or more pieces of information from the vehicle body that are detected by the sensing unit 6 (for example, vehicle speed and object information in the vicinity of the vehicle), whether the vehicle 100 can enter the specified area A1. In this case, the control unit 5 also uses detection information obtained from a blind spot monitoring system (BSM) as part of the information from the vehicle body. The hatching in Fig. 7A represents a detection area of ​​the BSM system.

[0063] In contrast, in the Fig. In the condition shown in Figure 7B, when the user 200 activates the direction indicator to change lanes, the virtual image 300, consisting of three markers 312, 313 and 314, is projected into the target space 400, as shown in Figure 7B. Fig. 9 shown. Marker 314 is the first virtual image 301 and represents a multitude (in this case three) of arrows to prompt the user 200 to accelerate or decelerate (in this case to accelerate).

[0064] In the Fig. In the condition shown in Figure 7B, when the user 200 activates the turn signal, the controller 5 determines, based on one or more pieces of information from the mobile body, which are acquired by the detection unit 6, whether the vehicle 100 can enter the specified area A1 by accelerating or braking. In this case, the controller 5 uses information from the mobile body such as, for example, the vehicle speed, object information in the vicinity of its own vehicle, and a relative speed with respect to the other vehicle 110. Obviously, the controller 5 can use detection information obtained from the BSM system.

[0065] This means that, as in the first content example, this example contains attribute information of the virtual image 300, which indicates whether vehicle 100 can enter the specified area A1 in the direction of travel of vehicle 100. Furthermore, in this content example, the specified area A1 is located on a lane (second lane 602) that differs from a lane (first lane 601) on which vehicle 100 is traveling.

[0066] An operational example of display system 10 when the virtual image 300, which contains this content example, is projected, is shown below with reference to Fig. 10 described. The processing from “START” to “END” in Fig. Step 10 is repeated according to a capture cycle (for example, 30 times per second) of the capture unit 6. First, the controller 5 captures one or more pieces of information about the mobile body from the capture unit 6 (step S201). The controller 5 then constructs a pseudo-space (virtual 3D space) based on the captured one or more pieces of information about the mobile body, which imitates the target space 400 (step S202).

[0067] The controller 5 then determines whether the turn signal is working (step S203). If the turn signal is not working (step S203: No), this indicates that the user 200 does not intend to change lanes. Consequently, the controller 5 repeats step S201 instead of performing the subsequent processing. In contrast, if the turn signal is working (step S203: Yes), the controller 5 determines, based on one or more pieces of information from the vehicle body, which are captured by the detection unit 6, whether there is a specific area A1 on the second lane 602 into which the vehicle can enter (step S204).

[0068] If there is a specific area A1 on the second lane 602 into which a vehicle can enter (step S204: Yes), the controller 5 draws the specific area A1 into which a vehicle can enter in a pseudo-space on the second lane 602 (step S205). The controller 5 draws information prompting the user to change lanes onto the first lane 601 in the pseudo-space (step S206). Subsequently, the controller 5 controls the drive unit 2 and the projection unit 3 to project content, drawn as a virtual image 300 in the pseudo-space, into the target space 400 (step S207). In this process, the marker 312, representing the defined area A1, is projected into the target area 400 onto the second lane 602, and the second marker 313, requesting the user 602 to change lanes, is projected onto the first lane 601.

[0069] In contrast, if there is no specific area A1 on the second lane 602 into which the vehicle can enter (step S204: No), the controller 5 determines whether the specific area A1 into which the vehicle can enter is created when the vehicle 100 accelerates or brakes (step S208). If the specific area A1 into which the vehicle can enter is created (step S208: Yes), the controller 5 draws the specific area A1 into which the vehicle can enter in a pseudo-space on the second lane 602 (step S209). Additionally, the controller 5 draws information prompting the user to change lanes and information prompting the user to accelerate or brake onto the first lane 601 in the pseudo-space (step S210).The controller 5 then controls the drive unit 2 and the projection unit 3 to project content, drawn as a virtual image 300 in the pseudo-space, into the target space 400 (step S207). This process projects marker 312, representing the defined area A1, onto the second lane 602 in the target space 400, and marker 313, prompting the user to change lanes, and marker 314, prompting the user to accelerate or brake, onto the first lane 601.

[0070] In contrast, if the specified area A1, into which the vehicle can enter, is not created (step S208: No), the controller 5 draws information indicating that the vehicle cannot enter a pseudo-space (step S211). This information indicates that the vehicle 100 cannot enter the specified area A1 (i.e., cannot change lanes). The controller 5 then controls the drive unit 2 and the projection unit 3 to project content, drawn as a virtual image 300 in the pseudo-space, into the target space 400 (step S207). This function projects the marker 312, representing the specified area A1 into which the vehicle cannot enter, onto the second lane 602 in the target space 400.

[0071] As described above, in this content example, the virtual image 300, which contains information indicating whether vehicle 100 can enter the specified area A1 on the second lane 602, is projected into the target space 400. Accordingly, this content example is useful in simplifying the determination of whether a lane change is necessary by allowing user 200 to visually perceive the virtual image 300.

[0072] In this example of the processing performed by controller 5, the processing in steps S201 and S202 can be executed after the processing in step S203 instead of before the processing in step S203. In this case, the processing in steps S201 and S202 is only executed if it is determined that the direction indicator is working.

[0073] The virtual image 300 with this content example can be projected into the target space 400 under a condition in which the vehicle 100 is parked, for example, on a road shoulder. The reason is that the user 200 activates the turn signal both when the vehicle 100 is to be parked on a road shoulder and when the vehicle 100 is changing lanes. In this case, the specific area A1 is located on a road shoulder of the first lane 601 instead of the second lane 602. (4.3) Third content example

[0074] In a third content example, as in the Fig. 11A and Fig. As shown in Figure 11B, the virtual image 300 is formed by the marker 315. The marker 315 is the first virtual image 301 in the form of an arrow and represents a slope of the road surface 600 on which the vehicle 100 (mobile body) travels. With reference to Fig. Mark 315 on 11A is an arrow whose shaft tapers from front to back and bears the inscription "Upwards". With reference to Fig. 11A represents marking 315, indicating that the road surface 600 has an increasing gradient of 610. With reference to Fig. Mark 315 on 11B is an arrow whose shaft thickens from front to back and bears the inscription "Downwards". With reference to Fig. 11B represents the marker 315 indicating that the road surface 600 has a downward slope 611. That is, in this content example, attribute information of the virtual image 300 is information representing the slope of the road surface 600 on which the vehicle 100 is moving.

[0075] In this example, the virtual image 300 is displayed, for instance, when the vehicle 100 is approaching an incline, until the vehicle 100 reaches the point where the road surface 600 is level. Specifically, the controller 5 controls the display of the virtual image 300 to overlay the marker 315 and project it onto the road surface 600 in a target space 400, based on one or more pieces of information from the mobile body (for example, road information about the vehicle's position, speed, and inclination) that were captured by the sensing unit 6.

[0076] As described above, in this content example, the virtual image 300, which contains content representing the incline of the road surface 600 on which the vehicle 100 is traveling, is projected into the target space 400. Consequently, the content example is useful in allowing the user 200 to understand whether the road surface 600 has an ascending incline 610 or a descending incline 611 by visually perceiving the virtual image 300 and easily adjusting their driving accordingly. In particular, the virtual image 300 with the content example is effective when it is difficult for the user 200 to notice the presence of an incline due to the structure of the road surface 600 or the surrounding landscape. For example, if the user 200 drives without noticing the presence of an incline 610, the vehicle 100 will naturally slow down. This can cause a traffic jam.Furthermore, if the user is driving at 200, for example, without noticing the presence of a downward slope 611, the vehicle will naturally accelerate 100. This can unnecessarily reduce the distance to a vehicle in front. Even in such cases, the user can accelerate or brake correctly by visually perceiving the virtual image 300.

[0077] The in the Fig. 11A and Fig. The markings 315 shown in 11B bear the inscriptions "Up" and "Down". However, this is not complete. For example, the markings shown in the Fig. 11A and Fig. The markings shown in 11B do not contain any letter sequences. Furthermore, the markings shown in the Fig. 11A and Fig. The markings shown in 11B 315 may be formed from moving images instead of still images. For example, the one shown in Fig. The marker 315 shown in Figure 11A is a moving image that undulates towards a nearby side over time. Furthermore, for example, the one shown in Figure 11A could be a moving image that undulates towards a nearby side. Fig. The marker 315 shown in 11B is a moving image that undulates over time towards a distant side.

[0078] In this case, attribute information of the virtual image 300 preferably changes with the position of the vehicle 100 relative to a slope. In particular, the controller 5 calculates the slope of the road surface 600 ahead and the coordinates to which the slope changes in advance, based on one or more pieces of information about the mobile body that were acquired by the sensing unit 6. The controller 5 changes the shape of the marker 315 (in this case, the shape of the shaft of an arrow) with the calculated coordinates as the starting point, according to a slope angle.

[0079] A specific example of virtual image 300, when the vehicle is traveling at 100 on an ascending slope 610, is shown below with reference to the Fig. 12A to 14B are described. First, as in Fig. 12A shows when the vehicle 100 approaches the rising incline 610, which is in Fig. Mark 315, shown in Figure 12B, is projected into the target area 400. Mark 315A, on the near side of mark 315, differs from mark 315B, on the far side, in shaft width and rate of change of width, with a starting point of the rising slope 610 forming a boundary. Subsequently, as shown in the Fig. 13A and Fig. Figure 14A shows when vehicle 100 begins to travel on the ascending slope 610, which is in the Fig. 13B and Fig. Mark 315 shown in Figure 14B is projected into the target area 400. At each mark 315, the width of an arrow shaft decreases as the inclination of the vehicle 100 approaches the angle of inclination of the rising slope 610.

[0080] As described above, if the virtual image 300 changes according to the position of the vehicle 100 relative to an incline, the user 200 can detect the presence of the incline before the vehicle 100 approaches it by visually perceiving the virtual image 300. This configuration is advantageous because it allows the user 200 to drive easily while perceiving the incline of the road surface 600 by perceiving the virtual image 300.

[0081] In this content example, as in Fig. As shown in Figure 15A, for example, an additional marker 316 can be projected into the target space 400 in addition to the marker 315. The additional marker 316 is the first virtual image 301 and has an arc-shaped arrow form that curves upwards. In the Fig. In the example shown in Figure 15A, the additional marking 316 indicates that the road surface 600 has a downward slope 611. This example is advantageous because it allows the user 200 to easily and intuitively recognize the slope of the road surface 600 by perceiving both the additional marking 316 and the marking 315.

[0082] In this content example, as in Fig. As shown in Figure 15B, for example, marker 315, which has a partially different light transmission or color, can be projected into target space 400 to appear partially transparent. In the Fig. In the example shown in Figure 15B, part of the marking 315, which extends towards the descending slope 611, is rendered as if seen through the road surface 600. This configuration is advantageous because it allows the user 200 to easily and intuitively perceive the slope of the road surface 600 by visually perceiving the marking 315.

[0083] In this content example, the arrow-shaped marker 315 is projected as a virtual image 300 into the target space 400. However, this is not complete. As in Fig. As shown in Figure 16, the marking 315, representing a traffic sign indicating a speed limit (in this case, 50 kilometers per hour) on the road surface 600, can be projected as a virtual image 300 into the target space 400. This example is advantageous because it allows the user 200 to easily recognize two pieces of information: the gradient of the road surface 600 and the speed limit on that surface, by visually perceiving the marking 315. Furthermore, the marking 315 could, for example, consist of a symbol that is not a traffic sign, or it could be a string of characters such as "gradient present," or the like. At a minimum, the marking 315 must have a shape that represents the gradient of the road surface 600. (4.4) Fourth content example

[0084] In a fourth content example, as in Fig. As shown in Figure 17A, a virtual image 300 is formed by marker 317, which is itself formed by two markers 317A and 317B. Marker 317A is the first virtual image 301 and represents a pre-calculated route for vehicle 100 (mobile body). Marker 317B is the first virtual image 301 and represents the name of a road (in this case, Route AAA) on which vehicle 100 travels. Marker 317B is superimposed and projected onto marker 317A. That is, in this content example, the attribute information of virtual image 300 is information representing the name of a road that encompasses the road surface 600 on which vehicle 100 is moving.

[0085] In this example, the virtual image 300 is always displayed while the vehicle is traveling at 100. Specifically, the controller 5 controls the display of the virtual image 300 to overlay the marker 317 and project it onto the road surface 600 into a target space 400, based on one or more pieces of information from the mobile body (for example, a steering angle and road information at the vehicle's position) that were captured by the detection unit 6.

[0086] As described above, in this content example, the virtual image 300, which contains the name of the road on which vehicle 100 is traveling, is projected into the target space 400. Consequently, the content example is advantageous because it allows user 200 to drive easily while recognizing where they are going by visually perceiving the virtual image 300.

[0087] In this content example, the street name represented by marker 317B is the name of a route. However, this marker could represent the name of a highway, freeway, city street, or the like. Furthermore, in this content example, the street name represented by marker 317B could, for instance, be the name of a street (in this case, BB St.), as in Fig. 17B shown, or may be the name of an avenue, street, expressway, diversion, or the like.

[0088] In this content example, as in Fig. As shown in Figure 18, for example, marker 317B can represent a direction in which vehicle 100 is traveling (in this case, "Towards CC"). That is, in the content example, attribute information of the virtual image 300 is information representing a direction of a road that encompasses the road surface 600 on which vehicle 100 is moving. In this case, in addition to marker 317, which represents a direction of one lane (lane 601) in which vehicle 100 is traveling, a marker 317, representing a direction of another lane (second lane 602) (in this case, "Towards DD"), can be projected into the target space 400. (4.5) Fifth content example

[0089] In a fifth content example, as in Fig. As shown in Figure 19, a virtual image 300 is formed by a plurality of (in this case, three) markings 318 and is a moving image. In the content example, each marking 318 is not the first virtual image 301, which has a depth in the direction of travel of the vehicle 100, but is the second virtual image 302, which is visually perceived as standing upright on the road surface 600 at a predetermined distance from the user 200. It should be noted that in the content example, the virtual image 300 is visually perceived by the user 200 as a whole as an image having a depth in the direction of travel of the vehicle 100, because the plurality of markings 318 is projected as a moving image into the target space 400. In the following description, of the three markings 318, the nearest marking 318 and the furthest away marking 318 are also referred to as "marking 318A" and "marking 318A," respectively.“Mark 318C”, and a mark between marks 318A, 318C is also referred to as “Mark 318B”.

[0090] In this example, the virtual image 300 is projected into the target space 400 when the vehicle 100 is in front of the curve 603, which has a curvature greater than or equal to a predefined value. Specifically, the controller 5 determines, based on one or more pieces of information from the mobile body (for example, road information at the vehicle's position) acquired by the detection unit 6, whether the curve 603 is in front of the vehicle 100. If it determines that the curve 603 is in front of the vehicle 100, the controller 5 controls the display of the virtual image 300, so that three markers 318 above the curve 603 are projected into the target space (in Fig. 19 (over a guardrail). In this case, curve 603, as seen from vehicle 100, bends to the left; three markers are projected into target area 400 as a symbol with an arrow-shaped shape pointing to the left. Obviously, if curve 603, as seen from vehicle 100, bends to the right, three markers are projected into target area 400 as a symbol with an arrow-shaped shape pointing to the right.

[0091] In this case, the controller 5 controls the display of the virtual image 300, so that three markers 318, namely markers 318A, 318B, 318C, 318A, ..., are projected in this order at predetermined time intervals (for example, one or more individual frames). Furthermore, three markers 318 are projected into the target space 400, so that the symbols move to the left and become smaller as the distance from the vehicle 100 increases (that is, as the viewing distance increases). Consequently, the user 200 visually perceives the three markers 318 as if the markers were moving uniformly along the curve 603.

[0092] As described above, in this content example, when the vehicle 100 is approaching curve 603, which has a curvature greater than or equal to a predefined value, the virtual image 300 is projected as a moving image into the target space 400. Consequently, this content example is advantageous because it allows the user 200 to drive easily while intuitively perceiving changes in the depth of the road surface 600, the curvature of curve 603, and the direction of the curve 603 by visually perceiving the virtual image 300. The virtual image 300 is particularly useful at night, when it is more difficult for the user 200 to have a sense of distance than during the day. That is, the user 200 can notice the presence of curve 603 by visually perceiving the virtual image 300, even if the user does not notice the presence of curve 603 due to darkness.

[0093] In this content example, because the virtual image 300 is a moving image, the multitude of markers 318 for sub-images can be projected in one-to-one correspondence. If the display system 10 is designed to project the multitude of markers 318 simultaneously in a sub-image, an increase in the volume of the display system 10 could make it impossible to mount the display system 10 in the vehicle 100. In contrast, according to this content example, the display system 10 is designed to project the multitude of markers 318 in the sub-images in one-to-one correspondence, and thus it is unlikely that the volume of the display system 10 will increase. This can reduce the probability that the display system cannot be mounted in the vehicle 100.Furthermore, the content example is useful because it allows the user 200 to easily focus on a condition of the road surface 600 represented by the virtual image 300, compared to the virtual image 300 being a still image.

[0094] According to this content example, the virtual image 300 above the curve 603 is projected into the target space 400. However, this is not complete. For example, the virtual image 300 can be superimposed and projected onto the road surface 600 into the target space 400. In this case, the multitude of markings 318 can each be projected as the first virtual image 301 into the target space 400. (4.6) Sixth content example

[0095] In a sixth content example, as in Fig. As shown in Figure 20, a virtual image 300 consists of two markers 319A and 319B. In the following description, two markers 319A and 319B are sometimes referred to as "marker 319" when they are not particularly distinct from one another. Mark 319A is the first virtual image 301 and represents another road surface 604 that is related to the road surface 600 on which the vehicle 100 (mobile body) travels and which allows the vehicle 100 to enter. In this case, the other road surface 604 is a highway entrance. Mark 319B is the second virtual image 302 and represents a destination of another road surface 604 indicated by mark 319A. In this case, marking 319B is a symbol that schematically represents a motorway and is superimposed and projected onto marking 319A.

[0096] That is, in this content example, attribute information of the virtual image 300 is information that represents another road surface 604, which allows the vehicle 100 to enter, among other road surfaces 604 that are related to the road surface 600 on which the vehicle 100 is moving. Accordingly, although in Fig. 20 not shown, marking 319 not projected onto another road surface 604 (for example, a road surface of a one-way street) that is related to road surface 600, but does not allow vehicle 100 to enter.

[0097] In this example, the virtual image 300 is always displayed while the vehicle is traveling at 100. Specifically, the controller 5 controls the display of the virtual image 300 to overlay the marker 319 and project it onto the road surface 604 in the target space 400, based on one or more pieces of information from the mobile body (for example, road information at the vehicle's position) that were captured by the detection unit 6.

[0098] As described above, in this content example, the virtual image 300, which contains content representing a different road surface 604 related to the road surface 600 on which vehicle 100 is traveling and which allows vehicle 100 to enter, is projected into the target space 400. Consequently, the content example is advantageous because it allows user 200 to easily navigate while considering other routes that differ from the current route of vehicle 100 by visually perceiving the virtual image 300. For example, the content example is advantageous because it allows user 200 to select a route, such as whether to travel on a highway or a motorway, when they recognize a highway entrance by visually perceiving the virtual image 300.

[0099] In this content example, another road surface 604, which allows vehicle 100 to enter, is, for example, a parking lot entrance or a shop entrance. Furthermore, in this content example, marker 319B, which indicates a destination of another road surface 604 that allows vehicle 100 to enter, is projected into the destination space 400. However, marker 319B does not have to be projected. (4.7) Seventh content example

[0100] In a seventh content example, as in Fig. As shown in Figure 21, the virtual image 300 consists of a multitude of markers 320. Each of the multitude of markers 320 is projected into the target space 400 as an arrow pointing in the direction of travel of the vehicle 100 (mobile body). The multitude of markers 320, which are visually perceived by the user 200, can give the user 200 an illusion of acceleration and thus have an effect that causes the user 200 to believe that the speed of the vehicle is greater than its actual speed. That is, in this content example, one element of the content of the virtual image 300 is information that gives the user 200 the illusion that the vehicle 100 driven by the user 200 is accelerating.

[0101] In this content example, the virtual image 300 indicates, for instance, that the vehicle's speed 100 exceeds a speed limit (in this case, a speed limit on a road where the vehicle is traveling at 100). Specifically, the controller 5 controls the display of the virtual image 300 to overlay the multitude of markings 320 and project them onto the road surface 600 in the target space 400, based on one or more pieces of information from the mobile body (for example, road information about the vehicle's position and speed) that were captured by the sensor unit 6.

[0102] As described above, in this content example, if, for example, the speed of vehicle 100 exceeds a speed limit, the virtual image 300, which gives user 200 an illusion of acceleration, is projected into the target space 400. Consequently, user 200 visually perceives the virtual image 300 to recognize that the current vehicle speed is too high and thus attempts to reduce the current vehicle speed, for example, by pressing the brake pedal. That is, the content example is advantageous because it is able to prompt user 200 to reduce the current vehicle speed to a speed limit or less by causing user 200 to visually perceive the virtual image 300.

[0103] In this case, if vehicle 100 is configured to emit a warning tone when it exceeds the speed limit, the user 200 might be disturbed by frequently generated warning tones. In contrast, this content example is advantageous because it is able to reduce the likelihood of disturbing the user 200 by displaying the virtual image 300 instead of generating warning tones. Although the illusion of acceleration is sometimes conveyed to a user 200 on specific roads, such as in tunnels, by lane markings, such locations with lane markings are limited. In contrast, this content example is advantageous because it is able to always convey an illusion of acceleration to the user 200, regardless of such lane markings on roads.

[0104] In this example, the virtual image 300 can have a different shape, instead of being limited to the variety of markers 320 described above. That is, the virtual image 300 must have at least one shape that can give the user 200 the illusion of acceleration. (4.8) Eighth content example

[0105] In an eighth content example, as in Fig. As shown in Figure 22, a virtual image 300 is formed by marker 321, which is formed by two markers 321A and 321B. Marker 321A is the first virtual image 301 and represents a past route on which the vehicle 100 (mobile body) actually traveled. Let us assume that a starting position and a destination position of a current route are the same as those of a past route, as in a round trip or commuting, and that the vehicle 100 is traveling on the same route. Marker 321B is the first virtual image 301 and represents a difference between the time required to travel from a starting position to a current position on the current route and the time required on the past route.In this case, because the time required on the current route exceeds the time required on the previous route by 0.5 s, marker 321B, which consists of the string "+0.5 s", is superimposed and projected onto marker 321A. That is, in this content example, the content of virtual image 300 is information representing a past route whose departure and destination positions are the same as those of a current route on which vehicle 100 is moving, and which is the same as the road on which vehicle 100 is moving.

[0106] In this example, the virtual image 300 is displayed, for instance, at the start time of the journey of vehicle 100. Specifically, the controller 5 stores a past route among its past routes, one that required the shortest time from the departure position to the destination position. When vehicle 100 begins its journey, the controller 5 adjusts the display of the virtual image 300 so that the marker 321A is superimposed and projected onto the road surface 600 in the target area 400. The information contained in the marker 321A is based on one or more pieces of information from the mobile body (for example, road information at the vehicle's current position) that were captured by the sensor unit 6.Furthermore, the controller 5 compares the time required on a current route from a departure position to a destination position with the time required on a previous route in real time and controls the display of virtual image 300 so that marker 321B is superimposed and projected onto marker 321A. Subsequently, when vehicle 100 arrives at a destination position, the controller 5 saves the current route as a previous route if the time required on the current route from the departure position to the destination position is shorter than the time required on the previous route.

[0107] As described above, in this content example, if, for example, user 200 begins driving vehicle 100, the virtual image 300, which contains content representing a past route, is projected into the target space 400. Consequently, this content example is advantageous because it allows user 200 to drive while searching for an optimal route to a target position by visually perceiving the virtual image 300 and comparing a current route with a past route.

[0108] User 200 can specify whether the projection of the virtual image 300, which contains this content example, should begin when vehicle 100 starts its journey. For example, if user 200 is driving vehicle 100 on a route different from a commuter route, such as on a vacation trip, comparing it to past routes is pointless. In such a case, display system 10 does not need to project the virtual image 300, which contains the content example, into the target space 400. (4.9) Ninth content example

[0109] In a ninth content example, as in Fig.As shown in Figure 23, a virtual image 300 is formed by marker 322 and marker 323, which is formed by two markers 323A and 323B. Marker 322 is the first virtual image 301 and represents a current route displayed by a navigation system. Marker 323A is the first virtual image 301 and represents a route after the navigation system has recalculated the path. Marker 323B is the first virtual image 301 and represents the difference between the time calculated as the travel time from a current position to a target position on the current route and the time calculated as the travel time on the route after the navigation system has recalculated the path.In this case, because the time calculated as required on the rerouted route after re-searching is 5 minutes shorter than the time calculated as required on the current route, marker 323B, which consists of the string "-5 min", is superimposed and projected onto marker 323A. That is, in this example, the content of virtual image 300 contains information representing the current route displayed by the navigation system and the route after re-searching.

[0110] In this example, the virtual image 300 is displayed when, for instance, a traffic jam occurs on a current route and the user 200 again searches for a route to a destination using the navigation system. Specifically, the controller 5 controls the display of the virtual image 300 to project the markers 321 and 322 onto the road surface 600. This is based on one or more pieces of information from the mobile body (for example, road information at the vehicle's current position) that were captured by the sensor unit 6. In this case, marker 321 is projected onto the first lane 601 on the road surface 600, where the traffic jam occurred, and marker 322 is projected onto the second lane 602, which branches off from the first lane 601.

[0111] As described above, in this content example, when user 200 searches again for a route to a target location using the navigation system, the virtual image 300, which contains information about the current route and the route after the re-search, is projected into the target space 400. Consequently, this content example is advantageous because it allows user 200 to easily grasp the advantage (or disadvantage) of a route after the re-search by visually perceiving the virtual image 300 and comparing the current route with the route after the re-search. (5) Modification

[0112] The exemplary embodiment described above is merely one of several exemplary embodiments of the present disclosure. The exemplary embodiment described above can be modified in various ways according to designs and the like, as long as the objective of the present disclosure can be achieved. For example, the same function of the display system 10 can be performed by a method for controlling the display system 10, a computer program, or a recording medium that stores a program.

[0113] A method for controlling the display system 10 according to one aspect is a method for controlling the display system 10 that includes a projector 40 and a controller 5. The projector 40 projects the virtual image 300 into the target space 400. The controller 5 controls the display of the virtual image 300. The method for controlling the display system 10 displays the virtual image 300, which has a depth in a direction of travel of a mobile body (in this case, the vehicle 100) and contains attribute information of the road surface 600 on which at least the mobile body is moving.

[0114] A program according to one aspect is a program to cause a computer system to execute a procedure for controlling the display system 10 described above.

[0115] The display system 10, or a unit that performs the control method according to the present disclosure, includes a computer system. The computer system consists mainly of a processor as hardware and a memory. When a processor executes a program recorded in a memory of the computer system, functions of the system or of a unit that performs the method according to the present disclosure are implemented. The program may be pre-recorded in memory or may be provided via a telecommunications line. The program may be provided by being recorded on a non-volatile recording medium that can be read by the computer system, such as a memory card, an optical disc, or a hard disk drive.The processor of a computer system consists of one or more electronic circuits, which may contain an integrated semiconductor circuit (IC) or a highly integrated circuit (LSI). The multitude of electronic circuits can be integrated onto a single chip or distributed across multiple chips. The multitude of chips can be integrated into a single device or distributed across multiple devices.

[0116] The functions of the controller 5 in the display system 10 can be distributed across a variety of systems (devices). At least some of the functions of the controller 5 can, for example, be implemented using cloud computing.

[0117] The display system 10 can use a so-called vehicle-to-everything (V2X) communication technology, in which communication takes place between vehicles (vehicle-to-vehicle communication) or between a vehicle and infrastructure, such as a traffic light and a road sign (road-to-vehicle communication). According to the V2X communication technology, for example, the vehicle 100 can receive information about its mobile body from surrounding vehicles or from infrastructure. Content for the virtual image 300, which is to be projected into the target space 400, can be received from infrastructure. In this case, it is not necessary for the vehicle 100 to contain at least part of the control system 5.

[0118] Furthermore, the display system 10 is not necessarily designed to project the virtual image 300 into the target space 400, which lies in front of a mobile body (vehicle 100) in the direction of travel. For example, the display system 10 can project the virtual image 300 into a lateral area, rear area, upper area, or the like in the direction of travel of the mobile body.

[0119] Furthermore, the display system 10 is not limited to the head-up display used in the vehicle 100. For example, the display system 10 can also be used in a mobile body other than the vehicle 100, such as a motorcycle, train, aircraft, construction machine, watercraft, or the like. In addition, the display system 10 can be used not only for a mobile body, but also, for example, for amusement facilities or as a wearable device, such as a head-mounted display (HMD). The display system 10 can also be used in medical facilities or as a stationary device.

[0120] The display system 10 is not necessarily designed to project the virtual image 300 using laser light. For example, the display system 10 may be designed to project an image (virtual image 300) from the back of a screen 1 onto the diffuse transmission screen 1 using a projector. Alternatively, the display system 10 may project the virtual image 300, which corresponds to an image displayed by a liquid crystal display, via a projector 40. (Summary)

[0121] As described above, the display system (10) comprises, according to a first aspect, a projector (40) and a controller (5). The projector (40) projects the virtual image (300) into the target space (400). The controller (5) controls the display of the virtual image (300). The controller (5) controls the display of a virtual image (300) that has a depth in the direction of travel of a mobile body (for example, a vehicle (100)) and contains content representing attribute information of the road surface (600) on which at least the mobile body is moving.

[0122] This aspect is advantageous because it allows the user (200) to easily recognize a condition of the road surface (600) by perceiving the virtual image (300) displayed in the target space (400). Furthermore, according to this aspect, the virtual image (300), which contains content representing attribute information of the road surface (600), is displayed in the target space (400) regardless of the use of a navigation system. Consequently, this aspect is advantageous because it allows the user (200) to easily recognize a condition of the road surface (600) without much thought.

[0123] Furthermore, this aspect is advantageous because it allows the user (200) to easily and intuitively recognize a condition of the road surface (600) because the virtual image (300), which has depth in the direction of travel of a mobile body, is projected into the target space (400). That is, if a virtual image (300) that has no depth in the direction of travel of the mobile body is displayed in the target space (400), the user (200) perceives the virtual image (300) as an object that has a planar shape. That is, the virtual image (300) is perceived by the user (200) with a shape that differs from real space as a collection of stereoscopic objects, and thus the user (200) may have a strange sensation.When the user (200) sees such a virtual image (300), it is difficult for the user (200) to intuitively grasp the virtual image (300) because the user's (200) brain processes the perception of the virtual image (300) as an object in a real space.

[0124] In contrast, according to this aspect, because the virtual image (300) has depth in the direction of travel of the mobile body, the user (200) easily perceives the virtual image (300) as a stereoscopic object, that is, in the form of real space. Consequently, this aspect is advantageous because it allows the user (200) to easily and intuitively perceive the virtual image (300) when the user (200) sees the virtual image (300), because the aforementioned processing load in the user's (200) brain is reduced.

[0125] From the standpoint of supporting the user (200) while driving, there is a tendency for immediacy to be required more for the user (200) to perceive a condition of a road surface (600) than for the user (200) to perceive directional information regarding a destination. This aspect is advantageous because it allows the user (200) to easily and intuitively perceive the virtual image (300) and thus be able to easily meet the aforementioned requirement for immediacy, thereby improving a function of supporting the user (200) while driving.

[0126] In the display system (10) according to a second aspect, attribute information in the first aspect is information that is based on one or more pieces of information from information relating to a position of a mobile body, information relating to an object in the environment of the mobile body and information relating to a state of the mobile body.

[0127] This aspect is advantageous because it easily presents the user (200) with different road surface conditions (600).

[0128] In the display system (10) according to a third aspect, attribute information of the first or second aspect is information that represents whether a mobile body can enter the specified area (A1) in a pre-calculated direction of travel of the mobile body.

[0129] This aspect is advantageous because it allows the user (200) to easily determine whether it is necessary to change the direction of travel of the mobile body.

[0130] In the display system (10) according to a fourth aspect, the control (5) in the third aspect performs the control to display the virtual image (300) which has content that provides information about a route that allows a mobile body to bypass the specified area (A1) if the mobile body cannot enter the specified area (A1).

[0131] This aspect is advantageous because it allows the user (200) to easily select the function of bypassing the prohibition area of ​​the specified area (A1).

[0132] In the display system (10) according to a fifth aspect, the specified area (A1) of the third aspect is located on a different lane (602) which is different from the lane (601) on which a mobile body is moving.

[0133] This aspect is advantageous because it allows the user (200) to easily determine whether it is necessary to perform a lane change.

[0134] In the display system (10) according to a sixth aspect, attribute information in one of the first to fifth aspects is information that represents an inclination of a road surface (600) on which a mobile body is moving.

[0135] This aspect is advantageous because it allows the user (200) to easily drive according to the gradient of the road surface (600).

[0136] In the display system (10) according to a seventh aspect, attribute information in any of the first to sixth aspects is information that represents a name of a street or a direction of a street that includes the road surface (600) on which a mobile body moves in any of the first to sixth aspects.

[0137] This aspect is advantageous because it allows the user (200) to drive easily while recognizing where he is going.

[0138] In the display system (10) according to an eighth aspect, the virtual image (300) is a moving image in each of the first to seventh aspects.

[0139] This aspect is advantageous because it allows the user (200) to easily pay attention to a condition of the road surface (600) represented by the virtual image 300, compared to a case where the virtual image (300) is a still image.

[0140] In the display system (10) according to a ninth aspect, attribute information in one of the first to eighth aspects is information that represents another road surface (604) that allows the entry of a mobile body, under other road surfaces (604) related to the road surface (600) on which the mobile body is moving.

[0141] This aspect is advantageous because it allows the user (200) to drive easily while taking into account other routes that differ from a current route of a mobile body.

[0142] An information presentation system (1000) according to a tenth aspect includes the display system (10) according to one of the first to ninth aspects and the recognition system (7). The recognition system (7) recognizes an object in the environment of a mobile body.

[0143] This aspect is advantageous because it allows the user (200) to easily recognize a condition of the road surface (600) by perceiving the virtual image (300) displayed in the target space (400).

[0144] A method for controlling the display system (10) according to an 11th aspect is a method for controlling the display system (10) that includes a projector (40) and a controller (5). The projector (40) projects the virtual image (300) into the target space (400). The controller (5) controls the display of the virtual image (300). The method for controlling the display system (10) displays the virtual image (300), which has depth in a direction of travel of a mobile body and contains attribute information of the road surface (600) on which at least the mobile body is moving.

[0145] This aspect is advantageous because it allows the user (200) to easily recognize a condition of the road surface (600) by perceiving the virtual image (300) displayed in the target space (400).

[0146] A program according to a 12th aspect is a program to cause a computer system to execute the procedure for controlling the display system (10) according to the 11th aspect.

[0147] This aspect is advantageous because it allows the user (200) to easily recognize a condition of the road surface (600) by perceiving the virtual image (300) displayed in the target space (400).

[0148] A mobile body according to a 13th aspect contains the display system (10) in one of the first to ninth aspects and a reflective element (for example, a windshield (101)).

[0149] This aspect is advantageous because it allows the user (200) to easily recognize a condition of the road surface (600) by perceiving the virtual image (300) displayed in the target space (400).

[0150] Without being limited to the above aspects, various arrangements (including modifications) of the display system (10) according to an exemplary embodiment can be embodied by a method for controlling the display system (10) and a (computer) program.

[0151] The interpretations according to the second to ninth aspects are not essential interpretations for the display system (10) and may be omitted if necessary. Reference symbols in the drawing 1 screen 1a movable screen 1b fixed screen 2 Drive unit 3 Radiation unit 4 Optical Projection System 6 recording units 10 Display system 31 Light source 32 probes 40 projector 41 Magnifying lens 42, 43 mirrors 51 Drive control 52 Display control 71 Image capture device 72 Laser radar 5 Control 7 Recognition system 100, 110 Vehicle (mobile body) 101 Windshield (reflective element) 102 Dashboard 200 users 300 virtual images 301 first virtual image 302 second virtual image 303 third virtual image 400 Target area 600 road surface 601 first lane (lane) 602 second lane (other lane) 604 other road surfaces 603 Curve 610, 611 gradient 1000 information presentation system

Claims

[1] Display system that projects an image so that a person visually perceives a virtual image displayed in a target space in front of a mobile body, the display system comprising: a projector designed to project the image; and a controller designed to control the projection of the image, wherein the control regulates the projection of the image according to an inclination of the mobile body, which is obtained from data measured by the mobile body, such that the virtual image is displayed in such a way that it contains initial attribute information representing an inclination of a road surface on which the mobile body is moving, and The projection of the image into the target space is controlled in such a way that it has a partially different light transmission within itself, in order to appear partially transparent, so that the road surface can be seen. [2] Display system according to claim 1, wherein the image further contains second attribute information for guiding a route of the mobile body. [3] Display system according to claim 1 or 2, wherein the image further contains second attribute information representing a name or direction of a road containing the road surface on which the mobile body is moving. [4] Display system according to any one of claims 1 to 3, wherein the control further controls the display of an image that presents information which gives the user an illusion of acceleration. [5] Display system according to any one of claims 1 to 4, wherein the virtual image is a moving image. [6] Information presentation system, comprehensive: the display system according to any one of claims 1 to 5; and a detection system designed to detect an object in the vicinity of the mobile body. [7] Method for controlling a display system that projects an image so that a person visually perceives a virtual image displayed in a target space, the display system comprising: a projector designed to project the image; and a control designed to control the projection of the image, the control method comprising: Controlling the projection of the image according to a tilt of the mobile body, which is obtained from data measured by the mobile body, so that the virtual image is displayed in such a way that it contains attribute information representing a tilt of a road surface on which the mobile body is moving. the projection of the image into the target space is controlled in such a way that it has a partially different light transmission in order to appear partially transparent, so that the road surface can be seen. [8] Non-volatile, machine-readable recording medium that stores a program to cause a computer system to execute the method for controlling a display system according to claim 7. [9] Mobile body, encompassing: the display system according to any one of claims 1 to 5; and a reflective element that has optical transparency and reflects light emitted by the projector.

Images