Image generation device, display system, image generation method, and program

Abstract

To provide an image generation device and the like which can reduce a sense of discomfort of a user to a display object.SOLUTION: An image generation device (vehicle information acquisition unit 110 and control unit 120) is the image generation device which is used for a display system 100 that projects an image to a display medium of a vehicle 2 to make a user 1 visually recognize a virtual image, and comprises: a vehicle information acquisition unit 110 which acquires vehicle position information indicating a current position of the vehicle 2 and route information indicating a route for guiding the vehicle 2 to the destination; a map image generation unit 125 which generates a map image of the periphery of the current position on the basis of the acquired vehicle position information; a guide image generation unit 121 which generates a guide image for indicating a route at the current position on the basis of the acquired vehicle position information and route information; and an image composition unit 124 which generates a composite image for projecting the generated guide image and map image in a mutually-associated mode.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a display system that allows a user to view an image as a virtual image, an image generation device used in the display system, an image generation method, and a program. [Background technology]

[0002] Conventionally, a display system has been proposed in which light representing an image is projected onto a translucent, plate-like display medium, and the light is reflected (projected) from the display medium, allowing the user to view the image as a virtual image while viewing a background through the display medium. Such a display system uses so-called augmented reality (AR) and can display an image related to a real background within the real background. In particular, in the automotive field, a so-called head-up display (HUD) has been developed that displays images indicating speed and various warnings as virtual images in front of the windshield while driving (see, for example, Patent Document 1).

[0003] Using such a display system, the user (driver) can view images related to driving (e.g., maps, speedometer, navigation directions, etc.) while looking at the outside world ahead, without having to move their eyes significantly, allowing for safer driving. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2015 / 118859 Summary of the Invention [Problem to be solved by the invention]

[0005] However, there is a problem in that the user may feel uncomfortable with the displayed objects.

[0006] Therefore, the present disclosure provides a display system that can reduce the sense of discomfort felt by the user when viewing a displayed object, an image generating device used in the display system, an image generating method, and a program. [Means for solving the problem]

[0007] An image generation device according to one aspect of the present disclosure is an image generation device used in a display system that projects an image onto a display medium of a vehicle to allow a user to view a virtual image, and includes: a vehicle information acquisition unit that acquires vehicle position information indicating the current position of the vehicle and route information indicating a route for guiding the vehicle to a destination; a map image generation unit that generates a map image of the area around the current position based on the acquired vehicle position information; a guidance image generation unit that generates a guidance image for indicating the route at the current position based on the acquired vehicle position information and route information; and an image synthesis unit that generates a synthetic image for projecting the generated guidance image and the map image in a manner that corresponds to each other.

[0008] Furthermore, a display system according to one aspect of the present disclosure includes the image generating device described above, a display unit that displays the composite image, and a projection unit that projects light representing the composite image displayed by the display unit onto the display medium.

[0009] Furthermore, an image generation method according to one aspect of the present disclosure is an image generation method that uses a computer to generate an image that allows a user to view a virtual image by projecting it onto a display medium of a vehicle, the image generation method acquiring vehicle position information indicating the current position of the vehicle and route information indicating a route for guiding the vehicle to a destination, generating a map image of the area around the current position based on the acquired vehicle position information, generating a guide image for indicating the route at the current position based on the acquired vehicle position information and route information, and generating a composite image for projecting the generated guide image and the map image in a manner that corresponds to each other.

[0010] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of the system, the method, the integrated circuit, the computer program, and the recording medium. The recording medium may also be a non-transitory recording medium. [Effects of the Invention]

[0011] The image generating device and the like of the present disclosure can reduce the sense of discomfort felt by the user with respect to a displayed object.

[0012] Further advantages and benefits of certain aspects of the present disclosure will become apparent from the specification and drawings. Such advantages and / or benefits may be provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all of them may be provided to obtain one or more identical features. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing an example of use of a display system according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of the interior of a vehicle equipped with a display system according to the embodiment. [Figure 3] FIG. 3 is a block diagram showing a functional configuration of the display system according to the embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a display object according to the embodiment. [Figure 5A] FIG. 5A is a diagram showing a specific example of a display object superimposed on a road surface by the display system according to the embodiment. [Figure 5B] FIG. 5B is a diagram showing a specific example of a display object superimposed on a road surface by the display system according to the embodiment. [Figure 6] FIG. 6 is a diagram showing another specific example of a display object superimposed on the road surface by the display system according to the embodiment. [Figure 7]FIG. 7 is a diagram showing yet another specific example of a display object superimposed on a road surface by the display system according to the embodiment. [Figure 8] FIG. 8 is a diagram showing yet another specific example of a display object superimposed on a road surface by the display system according to the embodiment. [Figure 9] FIG. 9 is a diagram showing yet another specific example of a display object superimposed on a road surface by the display system according to the embodiment. [Figure 10] FIG. 10 is a flowchart showing the processing operation of the image generating apparatus according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] (Findings that formed the basis of this disclosure) The present inventors have found that the display system of Patent Document 1 described in the "Background Art" section may have the following problems.

[0015] The display system in Patent Document 1 displays guidance for vehicle navigation. That is, the display system superimposes display objects such as arrows on the road surface. Such guidance display is expressed as AR, and the display object for the guidance display is in the form of a long carpet.

[0016] However, with such guidance displays, if the accuracy of the map or the detection accuracy of the sensor used for navigation is insufficient, the displayed objects of the guidance display may deviate from the driving lane or may not indicate the correct route. In other words, there is a possibility of overlay misalignment. For this reason, attempts have been made to provide more accurate information to the user by displaying map information in addition to such guidance displays. However, when a large amount of information is visually displayed at once, the user is forced to select and select the information they need and further construct a composite information that comprehensively interprets the multiple pieces of information, which is cumbersome for the user. This complexity can cause the user to feel uncomfortable, and this discomfort may lead to the user navigating the vehicle incorrectly.

[0017] In order to solve such problems, an image generation device according to one aspect of the present disclosure is an image generation device used in a display system that projects an image onto a display medium of a vehicle to allow a user to view a virtual image, and includes: a vehicle information acquisition unit that acquires vehicle position information indicating the current position of the vehicle and route information indicating a route for guiding the vehicle to a destination; a map image generation unit that generates a map image of the area around the current position based on the acquired vehicle position information; a guidance image generation unit that generates a guidance image for indicating a route at the current position based on the acquired vehicle position information and route information; and an image synthesis unit that generates a synthetic image for projecting the generated guidance image and map image in a manner that corresponds to each other.

[0018] This allows the generated guide image and map image to be projected in a manner that correlates them with each other. Since the correlation between the guide image and the map image can be read naturally, it is possible to reduce the sense of incongruity felt by the user even when the guide image and the map image are displayed together.

[0019] Furthermore, for example, the map image generating unit may generate a map image that is visually recognized by the user so as to follow the road surface ahead of the vehicle.

[0020] This naturally conveys to the user that the map image needs to be read as information associated with the road surface, further reducing the sense of discomfort felt by the user.

[0021] In addition, for example, the device may further include an auxiliary image generation unit that generates an auxiliary image that corresponds to the guide image on the map image, and in the composite image, the guide image and the map image may be corresponded to each other by the auxiliary image superimposed on the map image.

[0022] This allows the correspondence between the guide image and the map image to be naturally interpreted using the auxiliary image.

[0023] Furthermore, for example, the device may further include a display position setting unit that sets the display position of the guide image so that the virtual image of the guide image is visible to the user at a position where it overlaps with a route point on the route, and the auxiliary image may be superimposed at a position on the map image corresponding to the route point where the virtual image of the guide image is displayed.

[0024] This allows the correspondence between the guide image and the map image to be naturally interpreted by associating the display position of the guide image with the auxiliary image superimposed at the corresponding position on the map image.

[0025] Furthermore, for example, the auxiliary image generating unit may generate an auxiliary image that is visually recognized by the user in a shape that corresponds to the shape of the guide image.

[0026] As a result, since the guide image and the auxiliary image have the same shape, the correspondence between the guide image and the auxiliary image on the map image can be naturally read.

[0027] Also, for example, the auxiliary image may be superimposed on a position on the map image that corresponds to the current position.

[0028] This allows the user to naturally read the auxiliary image that indicates the position on the map image that corresponds to the current position.

[0029] Furthermore, for example, the device may further include a display position setting unit that sets the display position of the guide image so that the virtual image of the guide image is visible to the user at a position where it overlaps with a route point on the route, and the auxiliary image generation unit generates two auxiliary images, a first auxiliary image and a second auxiliary image, the first auxiliary image being superimposed on a position on the map image corresponding to the route point where the virtual image of the guide image is displayed, and the second auxiliary image being superimposed on a position on the map image corresponding to the current position.

[0030] This further enhances the effect of allowing the first auxiliary image and the second auxiliary image to naturally interpret the association between the guide image and the map image from multiple viewpoints using auxiliary images.

[0031] Furthermore, for example, the auxiliary image generating unit may generate a first auxiliary image that is visually recognized by the user in a shape that corresponds to the shape of the guide image.

[0032] As a result, since the guide image and the first auxiliary image have the same shape, the correspondence between the guide image and the first auxiliary image on the map image can be naturally read.

[0033] Furthermore, for example, the guide image generation unit may determine at least one of the posture and shape of the virtual image of the guide image that is visible to the user based on the route information, and generate a guide image that is visible to the user using at least one of the determined posture and shape.

[0034] This allows the user to view the guidance image in at least one of the orientation and shape determined as at least one of the orientation and shape based on the route information.

[0035] Furthermore, for example, the map image generating unit may generate the map image so that the upward direction of the map image corresponds to the traveling direction of the vehicle.

[0036] This has the advantage that the user can easily understand that the vehicle is moving in the upward direction on the map image.

[0037] Furthermore, for example, the map image generating unit may generate the map image so that the center of the map image corresponds to the current location.

[0038] This allows the map image to be associated with information visually obtained as the scenery from the vehicle, while obtaining additional information such as what landmarks are in the surrounding area, with the current position of the vehicle at the center.

[0039] Furthermore, for example, the map image generation unit may generate the map image so that the center of the map image corresponds to a first position on the route, and when the vehicle passes the first position, the center corresponds to a second position on the route, which is closer to the destination than the first position.

[0040] This makes it possible to generate a map image that can guide the vehicle so that it approaches the destination while sequentially tracing a plurality of positions on the route (for example, right and left turning points, intermediate stores, etc.).

[0041] Furthermore, for example, the guidance image generation unit may change the appearance of the guidance image that is viewed by the user depending on the distance from the vehicle's current position to a right or left turning point on the route, and the map image generation unit may generate a map image that is viewed by the user in an appearance that corresponds to the change in the appearance of the guidance image.

[0042] This allows the correspondence between the guide image and the map image to be naturally read, with the guide image changing in accordance with the distance from the vehicle's current position to the right or left turning point on the route, and the map image being associated in accordance with this change.

[0043] Furthermore, for example, the guidance image generation unit may change the color of the guidance image visually recognized by the user depending on the distance from the current position of the vehicle to a right or left turn point on the route.

[0044] This allows the user to naturally interpret the correspondence between the guide image and the map image, as the guide image changes color depending on the distance from the vehicle's current position to a right or left turn point on the route.

[0045] Also, for example, the map image generation unit may color the section on the map image from the vehicle's current position to the right or left turn point on the route so that the color changes depending on the distance to the right or left turn point, and the guidance image generation unit may change the color of the guidance image so that the color corresponds to the color colored at the position on the map image that corresponds to the vehicle's current position.

[0046] This allows the correspondence between the guide image and the map image to be naturally readable, using a map image that is colored in a manner that changes depending on the distance to the right or left turn point in the section on the map image from the vehicle's current position to the right or left turn point on the route, and a guide image that changes color to match the color colored at the corresponding position on the map image.

[0047] Furthermore, a display system according to one aspect of the present disclosure includes the image generating device described above, a display unit that displays a composite image, and a projection unit that projects light representing the composite image displayed by the display unit onto a display medium.

[0048] This makes it possible to realize a display system that can achieve the effects of the image generating device described above.

[0049] Furthermore, an image generation method according to one aspect of the present disclosure is an image generation method that uses a computer to generate an image that allows a user to view a virtual image by projecting it onto a display medium of a vehicle, and that acquires vehicle position information indicating the current position of the vehicle and route information indicating a route for guiding the vehicle to a destination, generates a map image of the area around the current position based on the acquired vehicle position information, generates a guidance image for indicating the route from the current position based on the acquired vehicle position information and route information, and generates a composite image for projecting the generated guidance image and map image in a manner that corresponds to each other.

[0050] This makes it possible to achieve the same effects as the image generating device described above.

[0051] Furthermore, a program according to one aspect of the present disclosure is a program for causing a computer to execute the image generating method described above.

[0052] This makes it possible to achieve the same effect as the image generating method described above using a computer.

[0053] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of the system, the method, the integrated circuit, the computer program, or the recording medium. The recording medium may also be a non-transitory recording medium.

[0054] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0055] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concepts are described as optional components.

[0056] In addition, each drawing is a schematic diagram and is not necessarily an exact illustration. In addition, the same components are denoted by the same reference numerals in each drawing.

[0057] (Embodiment) [Overall configuration] FIG. 1 is a diagram showing an example of use of a display system according to this embodiment.

[0058] The display system 100 in this embodiment is configured as a head-up display (HUD) and is mounted on the vehicle 2. In a specific example, the display system 100 is built into the dashboard 2b of the vehicle 2.

[0059] The display system 100 projects image light representing the display object 10 onto the windshield 2a of the vehicle 2. As a result, the image light is reflected by the windshield 2a and directed toward, for example, the user 1, who is the driver of the vehicle 2. As a result, the user 1 views the display object 10 as a virtual image through the windshield 2a. In other words, the display system 100 allows the user 1 to view the display object 10 as a virtual image. Note that allowing the user 1 to view the display object 10 as a virtual image in this manner is hereinafter also referred to as displaying the display object 10, and the operation of projecting the image light is synonymous with the operation of displaying the display object 10. The windshield 2a is an example of a display medium. In this embodiment, the display medium is the windshield 2a, but if the vehicle 2 is equipped with a combiner, the display system 100 may project the image light onto the combiner as the display medium.

[0060] The windshield 2a is a translucent, plate-like display medium. Therefore, the display system 100 allows the user 1 to view the display object 10 as a virtual image while showing the background, such as the road surface, through the windshield 2a. In other words, the AR allows the display object 10 to be displayed against the actual background.

[0061] The display object 10 is an image shaped to point in one direction, and a specific example is an image of a three-dimensional shape like an arrowhead and an image of a planar map shaped along the traveling surface (road surface) in the traveling direction of the vehicle 2. This one direction is the direction of the tip of the arrowhead, and will hereinafter also be referred to as the pointing direction. The pointing direction of this display object 10 is oriented in a direction to guide the vehicle 2 to the destination, i.e., the navigation direction.

[0062] Therefore, by using such a display system 100, the user 1, who is the driver, can see the display object 10 while looking at the outside world ahead without moving his or her eyes significantly, and can therefore drive more safely while understanding the navigation direction and map.

[0063] FIG. 2 is a diagram showing an example of the interior of a vehicle 2 equipped with a display system 100 according to this embodiment.

[0064] The display system 100 is hidden within the dashboard 2b and projects image light onto the windshield 2a. For example, the projection of image light by the display system 100 causes a display object 10 to appear as a virtual image within a display range d1 on the windshield 2a.

[0065] FIG. 3 is a block diagram showing the functional configuration of display system 100 according to this embodiment.

[0066] The display system 100 includes a vehicle information acquisition unit 110, a control unit 120, and a drawing unit .

[0067] The vehicle information acquisition unit 110 is a part of the functions of the image generation device, and acquires vehicle-related information from the navigation device 21, the vehicle control device 22, and the sensor 23 provided in the vehicle 2.

[0068] The navigation device 21 uses a satellite positioning system such as a GPS (Global Positioning System). The navigation device 21 is a device for navigating the vehicle 2 to a destination using a navigation system. Such a navigation device 21 outputs, as the above-mentioned vehicle-related information, vehicle position information indicating the current position of the vehicle 2, route information indicating the route from the current position of the vehicle 2 to the destination, and vehicle orientation information indicating the traveling direction of the vehicle 2.

[0069] The vehicle control device 22 is configured as, for example, an ECU (Electronic Control Unit) mounted on the vehicle 2, and receives vehicle speed information indicating the traveling speed of the vehicle 2 as the above-mentioned vehicle-related information. Output.

[0070] The sensor 23 detects people or other vehicles around the vehicle 2 and outputs sensing information indicating the detection results as the vehicle-related information described above. For example, the sensor 23 may detect people or other vehicles by LiDAR (light detection and ranging). stomach.

[0071] The control unit 120 is a part of the functions of the image generation device, and uses the vehicle-related information acquired by the vehicle information acquisition unit 110 to determine the display form of the display object 10 and generate a composite image. The image generation device is realized by combining the vehicle information acquisition unit 110 and the control unit 120. Specifically, the control unit 120 includes a guide image generation unit 121, a map image generation unit 125, an auxiliary image generation unit 122, a display position setting unit 123, and an image synthesis unit 124. The control unit 120 also includes a tilt processing unit, a design processing unit, a shape processing unit, and a route point determination unit, all of which are not shown.

[0072] The guide image generating unit 121 is a processing unit that generates a guide image for indicating a route from the current position based on the acquired vehicle position information and route information. The guide image generating unit 121 generates a guide image that mainly shows information equivalent to navigation information.

[0073] The map image generation unit 125 is a processing unit that generates a map image of the area around the current position based on the acquired vehicle position information. The map image generation unit 125 also generates a guide image that mainly displays information equivalent to navigation information. However, the map image generated by the map image generation unit 125 is a map image of the area around the current position, and the area around the current position includes the area from the current position of the vehicle 2 to the next right or left turn point. In other words, the map image may be an image that reflects the next right or left turn point, or may be an image that reflects an area approximately within the user's field of view centered on the current position of the vehicle 2. In the example of the image that reflects the next right or left turn point, when the vehicle 2 reaches the next right or left turn point, a map image corresponding to the next right or left turn point is generated.

[0074] The auxiliary image generation unit 122 is a processing unit that generates an auxiliary image that associates a guide image on a map image. The auxiliary images are generated as two auxiliary images, a first auxiliary image and a second auxiliary image, which will be described in detail later. The generated auxiliary images are superimposed on the map image, thereby associating the map image and the guide image with each other.

[0075] The display position setting unit 123 sets the display position of the guide image so that the virtual image of the guide image is visually recognized by the user at a position where it overlaps with a route point on the route. As a result, for example, when the guide image instructs the user to turn right at a right turn point, the display position of the arrowhead of the guide image can be made to correspond to the right turn point. However, since there is a limit to the speed of human reflexes, the instruction to turn right using such a guide image is started from a point some distance before the right turn point. Furthermore, before that, the image is simply fixed at a position about tens to hundreds of meters ahead of the vehicle and is made to point in the straight ahead direction at the same speed as vehicle 2.

[0076] The image synthesis unit 124 is a processing unit that generates a composite image for projecting a guide image and a map image in a manner that correlates them with each other. Specifically, the image synthesis unit 124 processes and synthesizes the guide image, map image, and auxiliary image so that the guide image and map image are correlated using the auxiliary image, or the guide image and map image are correlated by matching the display manner. More specific generation of the composite image by the image synthesis unit 124 will be described in detail later.

[0077] The route point determination unit determines route points on the route indicated in the above-mentioned route information, and notifies the display position setting unit 123 and the tilt processing unit of the determined route points.

[0078] The display position setting unit 123 uses the route points determined by the route point determination unit to determine the position of the display object 10 visually recognized by the user 1. This position is a position in three-dimensional space, and includes the lateral position in the width direction of the vehicle 2, the depth position in the traveling direction of the vehicle 2, and the height from the road surface.

[0079] The tilt processing unit uses the path points determined by the path point determination unit to determine the tilt mode of the display object 10. The tilt mode (in other words, the attitude of the virtual image) is determined by the yaw angle, roll angle, and pitch angle of the display object 10.

[0080] The design processing unit determines the design of the display object 10. Note that the design of the display object 10 in this embodiment refers to the color or brightness of the display object 10, and includes dynamic changes in the color or brightness.

[0081] The shape processing unit determines the shape of the display object 10. In this embodiment, the shape of the display object 10 is defined by the overall length or width of the display object 10. The overall length is the length of the display object 10 along the direction indicated by the display object 10, and the width is the length of the display object 10 along the direction perpendicular to the indicated direction. The shape processing unit determines the shape of the display object 10 by changing the ratio between the overall length and the width.

[0082] The control unit 120 outputs to the drawing unit 130 display form information indicating the display form including the above-mentioned position, tilt state, design, and shape.

[0083] The rendering unit 130 is an example of a display unit and a projection unit. Therefore, the rendering unit 130 has both the function of displaying a composite image and the function of projecting light representing the composite image displayed by the display unit onto a display medium. To this end, the rendering unit 130 acquires display form information from the control unit 120 and renders the display object 10 according to the display form information. For example, the rendering unit 130 includes a light source and an optical system, and generates image light representing the display object 10 in the display form indicated by the display form information so that the display object 10 is visually recognized by the user 1 in that display form. The rendering unit 130 then projects the image light onto the windshield 2a. As a result, the display object 10 with the determined tilt, design, and shape is visually recognized by the user 1 at the determined position. In other words, with regard to the tilted state, the drawing unit 130 projects image light showing the display object 10 with the tilted state determined by the tilt processing unit of the control unit 120 onto the windshield 2a provided in the vehicle 2, causing the windshield 2a to reflect the image light toward the user 1 inside the vehicle 2, allowing the user 1 to view the display object 10 with the tilted state as a virtual image through the windshield 2a.

[0084] Fig. 4 is a diagram showing an example of a display object 10 in this embodiment. Note that Fig. 4(a) shows a top view of a guide image which is one of the display objects 10, Fig. 4(b) shows a perspective view of the guide image which is one of the display objects 10, and Fig. 4(c) shows the position of the guide image which is one of the display objects 10 in the traveling direction and height direction of the vehicle 2.

[0085] As shown in Figures 4(a) and 4(b), the display object 10 is flat and formed in a substantially V-shape or a substantially inverted V-shape. The tilt processing unit of the control unit 120 determines the tilt mode of the display object 10 by controlling the yaw angle ψ, roll angle φ, and pitch angle θ of the display object 10. Although Figure 4 focuses on the guide image of the display object 10, the attitude of the virtual image is similarly controlled for the map image, which is another type of display object 10, and the auxiliary image superimposed on the map image.

[0086] The yaw angle ψ is the angle at which the display object 10 rotates around a yaw axis that is aligned with the thickness direction of the display object 10. For example, when the pointing direction of the display object 10 is aligned with the traveling direction of the vehicle 2, the yaw angle ψ of the display object 10 is 0°.

[0087] The roll angle φ is the angle at which the display object 10 rotates around a roll axis that is aligned with the pointing direction of the display object 10. For example, when the display object 10 is aligned with a horizontal plane, the roll angle φ of the display object 10 is 0°.

[0088] The pitch angle θ is the angle at which the display object 10 rotates around the pitch axis, which is perpendicular to the yaw axis and the roll axis. For example, when the display object 10 is aligned with a horizontal plane, the pitch angle θ of the display object 10 is 0°.

[0089] 4(c), the display position setting unit 123 of the control unit 120 determines a position y of the display object 10 in the traveling direction of the vehicle 2, for example, based on vehicle speed information. The position y is expressed as a distance from the vehicle 2 in the traveling direction. Furthermore, the display position setting unit 123 determines a position z as the height of the display object 10. The position z is expressed as a height from the road surface on which the vehicle 2 is traveling.

[0090] [Example of display items] The generation of a composite image by the image composition unit 124 will be described in detail below with reference to several examples. FIGS. 5A and 5B are diagrams showing specific examples of display objects superimposed on the road surface by the display system of the embodiment. FIG. 5A shows a landscape including display object 10 when viewed across the entire windshield 2a. In this example, a so-called heading-up display is achieved, in which the upward direction of display object 10 corresponds to the traveling direction of vehicle 2. However, according to the embodiment, it is sufficient to show display range d1 in which the composite image is displayed, and therefore in the following explanation (including FIGS. 6 to 9 in addition to FIG. 5B), only a portion of the windshield including display range d1 will be shown and used for explanation.

[0091] 5B, (a) shows the state of the periphery of display range d1 at a first time point, (b) shows the state of the periphery of display range d1 at a second time point after the first time point, (c) shows the state of the periphery of display range d1 at a third time point after the second time point, (d) shows the state of the periphery of display range d1 at a fourth time point after the third time point, and (e) shows the state of the periphery of display range d1 at a fifth time point after the fourth time point. This configuration of the drawings is similar to that of the subsequent FIGS. 6 to 9.

[0092] In the example shown in FIG. 5B, a guide image 10a, a map image 10b, and a first auxiliary image 10c are displayed as the display object 10. The first auxiliary image 10c is superimposed on the map image 10b at a position corresponding to the route point where the virtual image of the guide image 10a is displayed. This allows the guide image 10a and the map image 10b to be associated with each other in terms of the position of the virtual image of the guide image 10a on the map image. Furthermore, the guide image 10a and the first auxiliary image 10c correspond in shape (e.g., a similarity relationship or a projection relationship), so that it is clear at a glance that the first auxiliary image 10c corresponds to the guide image 10a. Furthermore, the first auxiliary image 10c is configured (its orientation is changed) to have an orientation that corresponds to the direction indicated by the guide image 10a.

[0093] In the example shown in FIG. 6, the configuration of the map image 10b is different from the example shown in FIG. 5B. Specifically, in the configuration of FIG. 5B, the map image 10b is displayed with the current position of the vehicle 2 at its center (strictly speaking, the current position of the vehicle 2 corresponds to a position near the center). In contrast, in the example of FIG. 6, the map image 10b is a map image that is fixed within a predetermined range centered on the next right or left turn point. Therefore, the primary auxiliary image 10c, which indicates the position of the guide image 10a displayed near the vehicle 2 at the first time point, does not appear on the map image 10b. The first auxiliary image 10c arrives within the predetermined range from the right or left turn point at or after the second time point.

[0094] In the example shown in Fig. 7, a second auxiliary image 10d is also displayed compared to the example shown in Fig. 5B. The second auxiliary image 10d is superimposed on the position on the map image corresponding to the current position, so that the position of the vehicle 2 and the position of the virtual image of the guide image 10a preceding the vehicle 2 can be seen at a glance from the first auxiliary image 10c and the second auxiliary image 10d.

[0095] In the example shown in FIG. 8, a guide image 10a, a map image 10b, and a second auxiliary image 10d are displayed as the display object 10. The route portion of the map image 10b is colored (in the figure, the coloring is represented by hatching or dot hatching). The display mode of the guide image 10a changes in accordance with the current position of the vehicle 2 in the second auxiliary image 10d. For example, at a first point in time, the guide image 10a is in a color corresponding to the hatching made up of diagonal lines from the upper right to the lower left in the figure (although the second auxiliary image 10d is not displayed in the figure). At a subsequent second point in time, the guide image 10a is displayed in the same color (for example, the color corresponding to the hatching made up of diagonal lines from the upper left to the lower right in the figure) as the coloring of the map image 10b corresponding to the current position of the vehicle 2. Furthermore, the same is true at a third point in time. At the subsequent fourth time point, the map image 10b corresponding to the current position of the vehicle 2 is displayed in a different color and in the same color (the color corresponding to the dot hatching in the figure). The same is true at the fifth time point. Since the color of the map image 10b changes as the vehicle approaches the right or left turn point, the above configuration allows the user to estimate the distance to the right or left turn point using only the guide image 10a.

[0096] Similarly, in the example shown in Fig. 9, the color of guide image 10a changes to correspond to the coloring of map image 10b. However, whereas in the example shown in Fig. 8 the color of the entire area of ​​guide image 10a changes uniformly from top to bottom, in the example shown in Fig. 9 the color of guide image 10a is displayed in divided sections corresponding to the distance from the current position of vehicle 2 to the right or left turn point, such that the higher up on guide image 10a the color is closer to the right or left turn point, and the lower up on guide image 10a the color is closer to the current position of vehicle 2.

[0097] [Processing flow] Fig. 10 is a flowchart showing the processing operation of the image generating device according to the embodiment. As shown in Fig. 10, in this embodiment, first, the vehicle information acquisition unit 110 acquires vehicle-related information of the vehicle 2 (S11). The vehicle-related information includes vehicle position information indicating the current position of the vehicle 2 and route information indicating a route for guiding the vehicle 2 to the destination. Therefore, in step S11, the vehicle information acquisition unit 110 can be said to acquire vehicle position information or route information.

[0098] Then, the map image generation unit 125 generates a map image of the surroundings of the current position based on the acquired vehicle position information (S12), while the guidance image generation unit 121 generates a guidance image for indicating a route from the current position based on the acquired vehicle position information and route information (S13).

[0099] Thereafter, the image synthesis unit 124 generates a synthesis image for projecting the generated guide image and map image in a manner that associates them with each other (S14).

[0100] When the composite image generated in this way is projected, the user can see the guide image and map image in a correlated state, so there is no shortage of displayed information and the relationship between the information is easy to understand, which reduces the user's sense of discomfort with the displayed object.

[0101] (Other aspects) While the display system according to one or more aspects of the present disclosure has been described above based on the embodiments, the present disclosure is not limited to those embodiments. As long as the modifications do not deviate from the spirit of the present disclosure, various modifications conceivable by those skilled in the art to the above embodiments may also be included in the present disclosure.

[0102] In the above-described embodiment, each component is configured by dedicated hardware, but may also be realized by executing a software program appropriate for each component. Each component may also be realized by a program execution unit such as a CPU (Central Processing Unit) or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Here, the software program that realizes the display system 100 of the above-described embodiment causes a computer to execute each step included in the flowchart shown in FIG. 10, for example.

[0103] Furthermore, in the above embodiment, an image in the shape of an arrowhead is displayed as the display object, but the shape of the display object is not limited to an arrowhead shape and may be any shape that indicates the route guidance direction, such as a triangle or an arrow. The display object may also be a long carpet-like object that is displayed so as to be superimposed on the road surface. While the display object is displayed to guide the vehicle to the destination, it may also be displayed for other purposes or uses as long as it is displayed based on the vehicle's orientation. For example, the display object may be an image that indicates the vehicle's orientation itself, or an image that indicates the vehicle's planned traveling direction estimated from the vehicle's orientation and the steering angle.

[0104] Furthermore, in the above-described embodiment, a configuration is described in which the navigation device, vehicle control device, sensor, and display system are each provided as separate devices, but this is not limited to this. For example, the display system may be provided as a single device integrated with at least one of the other devices, such as in an example in which the navigation device and the display system are provided as a single integrated device. Furthermore, it is within the scope of the present disclosure to provide a group of one or more devices in which the above-described navigation device, vehicle control device, sensor, and display system are combined in any combination.

[0105] The following cases are also included in this disclosure:

[0106] (1) The at least one device is specifically a computer system comprising a microprocessor, a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk unit, a display unit, a keyboard, a mouse, etc. A computer program is stored in the RAM or hard disk unit. The at least one device achieves its function when the microprocessor operates in accordance with the computer program. Here, the computer program is composed of a combination of multiple instruction codes that indicate instructions to the computer to achieve a predetermined function.

[0107] (2) A part or all of the components constituting at least one of the above devices are considered to be composed of a single system LSI (Large Scale Integration). A system LSI is an ultra-multifunctional LSI manufactured by integrating multiple components on a single chip, and is specifically a computer system comprising a microprocessor, ROM, RAM, etc. The RAM stores a computer program. The system LSI achieves its functions when the microprocessor operates in accordance with the computer program.

[0108] (3) Some or all of the components constituting at least one of the above devices may be configured as an IC card or a standalone module that can be attached to the device. The IC card or module is a computer system composed of a microprocessor, ROM, RAM, etc. The IC card or module may include the above-mentioned ultra-multifunctional LSI. The IC card or module achieves its functions when the microprocessor operates according to a computer program. This IC card or module may be tamper-resistant.

[0109] (4) The present disclosure may be embodied as the methods described above, a computer program for implementing these methods on a computer, or a digital signal comprising the computer program.

[0110] The present disclosure may also be a computer program or a digital signal recorded on a computer-readable recording medium, such as a flexible disk, a hard disk, a CD (Compact Disc)-ROM, a DVD, a DVD-ROM, a DVD-RAM, a BD (Blu-ray (registered trademark) Disc), a semiconductor memory, etc. Alternatively, the present disclosure may be a digital signal recorded on such a recording medium.

[0111] The present disclosure may also be applied to transmitting a computer program or digital signal via a telecommunications line, a wireless or wired communication line, a network such as the Internet, data broadcasting, or the like.

[0112] Furthermore, the program or digital signal may be recorded on a recording medium and transferred, or the program or digital signal may be transferred via a network or the like, so that the program or digital signal may be implemented by another independent computer system. [Industrial Applicability]

[0113] The display system of the present disclosure has the effect of reducing the sense of discomfort felt by the user when viewing displayed objects, and can be applied to, for example, in-vehicle head-up displays. [Explanation of symbols]

[0114] 1 user 2 vehicles 2a Windshield 2b Dashboard 10 Display items 21 Navigation devices 22 Vehicle control device 23 Sensors 100 Display System 110 Vehicle information acquisition unit (part of image generation device) 120 Control unit (part of image generating device) 121 Guidance image generation unit 122 Auxiliary Image Generation Unit 123 Display position setting section 124 Image synthesis unit 125 Map image generation unit 130 Drawing section d1 Display range d2 Display plane range

Claims

1. An image generating device used in a display system that projects an image onto a display medium of a vehicle to allow a user to view a virtual image, a vehicle information acquisition unit that acquires vehicle position information indicating a current position of the vehicle and route information indicating a route for guiding the vehicle to a destination; a map image generating unit that generates a map image of the surrounding area of ​​the current position based on the acquired vehicle position information; a guide image generating unit that generates a guide image for indicating the route from the current location based on the acquired vehicle position information and the acquired route information; an image synthesis unit that generates a synthetic image for projecting the generated guide image and the generated map image in a manner that correlates them with each other; an auxiliary image generating unit that generates an auxiliary image that corresponds to the guide image on the map image; a display position setting unit that sets a display position of the guidance image so that a virtual image of the guidance image is visually recognized by the user at a position that overlaps with a right or left turn point on the route, In the composite image, the guide image and the map image are associated with each other by the auxiliary image superimposed on the map image, The auxiliary image is superimposed on the map image at a position corresponding to the right or left turn point where the virtual image of the guide image is displayed. Image generating device.

2. The map image generating unit generates the map image that is visually recognized by the user along a road surface ahead of the vehicle. The image generating device of claim 1 .

3. The guidance image generation unit determines a shape of the guidance image to be viewed by the user based on the route information, and generates the guidance image to be viewed by the user in the determined shape; The auxiliary image generating unit generates the auxiliary image that is visually recognized by the user in a shape that corresponds to a shape of the guide image. The image generating device of claim 1 .

4. a display position setting unit that sets a display position of the guidance image so that a virtual image of the guidance image is visually recognized by the user at a position that overlaps with a right or left turn point on the route; the auxiliary image generation unit generates two auxiliary images, a first auxiliary image and a second auxiliary image, the first auxiliary image is superimposed on the map image at a position corresponding to the right / left turn point where a virtual image of the guidance image is displayed, The second auxiliary image is superimposed on the map image at a position corresponding to the current location. The image generating device of claim 1 .

5. The auxiliary image generating unit generates the first auxiliary image that is visually recognized by the user in a shape corresponding to a shape of the guide image.

5. The image generating device of claim 4.

6. The guide image generation unit determines a posture of a virtual image of the guide image to be viewed by the user based on the route information, and generates the guide image to be viewed by the user in the determined posture. The image generating device according to any one of claims 1 to 5.

7. The map image generating unit generates the map image so that the upward direction of the map image corresponds to the traveling direction of the vehicle. The image generating device according to any one of claims 1 to 6.

8. The map image generating unit generates the map image so that the center of the map image corresponds to the current location. The image generating device according to any one of claims 1 to 6.

9. The map image generation unit generating the map image so that a center portion of the map image corresponds to a first position on the route; The map image is generated so that when the vehicle passes the first position, the center portion corresponds to a second position on the route, the second position being closer to the destination than the first position. The image generating device according to any one of claims 1 to 6.

10. the guidance image generation unit changes a state of the guidance image visually recognized by the user in accordance with a distance from the current position of the vehicle to a right or left turn point on the route, The map image generating unit generates the map image that is visually recognized by the user in a manner associated with the change in the manner of the guide image.

3. The image generating device according to claim 1 or 2.

11. The guidance image generation unit changes a color of the guidance image visually recognized by the user according to a distance from the current position of the vehicle to a right or left turn point on the route. The image generating device of claim 10.

12. the map image generation unit applies color to a section on the map image from the current position of the vehicle to the right or left turn point on the route in a manner that changes depending on a distance to the right or left turn point, The guide image generation unit changes the color of the guide image to a color corresponding to a color assigned to a position on the map image that corresponds to the current position of the vehicle. The image generating device of claim 11.

13. An image generating device according to any one of claims 1 to 12; a display unit that displays the composite image; a projection unit that projects light representing the composite image displayed by the display unit onto the display medium. Display system.

14. An image generation method for generating an image for allowing a user to visually recognize a virtual image by projecting it onto a display medium of a vehicle using an image generation device including a vehicle information acquisition unit, a map image generation unit, a guidance image generation unit, an image synthesis unit, an auxiliary image generation unit, and a display position setting unit, the vehicle information acquisition unit acquires vehicle position information indicating a current position of the vehicle and route information indicating a route for guiding the vehicle to a destination; the map image generating unit generates a map image of the surrounding area of ​​the current position based on the acquired vehicle position information; the guidance image generation unit generates a guidance image for indicating the route from the current location based on the acquired vehicle position information and the acquired route information; the image synthesis unit generates a synthesis image for projecting the generated guide image and the generated map image in a manner that they correspond to each other; the auxiliary image generating unit generates an auxiliary image to which the guide image is associated on the map image; the display position setting unit sets a display position of the guidance image so that a virtual image of the guidance image is visually recognized by the user at a position overlapping a right or left turn point on the route; In the composite image, the guide image and the map image are associated with each other by the auxiliary image superimposed on the map image, The auxiliary image is superimposed on the map image at a position corresponding to the right or left turn point where the virtual image of the guide image is displayed. Image generation method.

15. A method for causing the image generating device to execute the image generating method according to claim 14. program.

Images