In-vehicle projection device, control method for in-vehicle projection device, program, and in-vehicle display system

Abstract

The present invention makes it possible to display an optical flow image that is easy to view without using a large display panel. 　An in-vehicle projection device 100: projects, via a projection optical system 220, display light emitted from a projector 200 housed in approximately the center of a dashboard of a vehicle onto a projection surface that is set to the front part of the vehicle, thereby allowing a viewer to view the display light; and comprises a processing unit 311 that generates an optical flow image which moves from approximately the center of a display 210 included in the projector 200 toward both left and right directions perpendicular to a travel direction of the vehicle, and that displays the generated optical flow image on the display, and projects the image as display light onto the projection surface via the projection optical system 220.

Description

In-vehicle projection device, control method for in-vehicle projection device, program, and in-vehicle display system

[0001] The present invention relates to an in-vehicle projection device that projects display light emitted from a projector housed approximately in the center of the vehicle's dashboard onto a projection surface set in front of the vehicle via a projection optical system, thereby allowing the display light to be seen by the viewer.

[0002] For example, Patent Document 1 describes a technology for an in-vehicle information provision device that displays an optical flow image consisting of multiple dot images on a display, such as an exterior view that appears to recede in accordance with the movement of the vehicle, thereby inducing self-motion perception in the occupant observing the provided information (suppressing motion sickness) even when pitch motion is canceled in an actual moving vehicle.

[0003] Japanese Patent Publication No. 2006-248450 (see paragraphs

[0006] and

[0007] )

[0004] Incidentally, the in-vehicle information provision device described in Patent Document 1 displays information on a relatively large display panel installed in a part of the vehicle, making it difficult to see without shifting one's gaze while driving. Furthermore, there was the problem that the layout of the vehicle interior was restricted when a large display panel was installed in the vehicle.

[0005] Therefore, the object of the present invention is to provide an in-vehicle projection device, etc., that can display easily viewable optical flow images without using a large display panel.

[0006] Other objects of the present invention will become apparent to those skilled in the art by referring to the embodiments and best embodiments described below, as well as the accompanying drawings.

[0007] The following are examples of embodiments of the present invention to facilitate understanding of its outline.

[0008] The first embodiment is an in-vehicle projection device that allows a viewer to see display light emitted from a projector housed approximately in the center of the vehicle's dashboard, by projecting the display light onto a projection surface set in front of the vehicle via a projection optical system, and comprises a processing unit that generates an optical flow image moving from approximately the center of a display unit included in the projector toward the left and right directions perpendicular to the direction of travel of the vehicle, displays the generated optical flow image on the display unit, and projects the display light onto the projection surface via the projection optical system.

[0009] Here, "optical flow image" refers to an image of a two-dimensional motion vector field on an image plane formed by the motion vectors of arbitrary points in the external scenery observed from a vehicle window while the vehicle is in motion. For example, the flow image generation unit 311b shown in Figure 5 calculates the apparent velocity distribution of the external scenery flowing through the window based on changes in vehicle speed and trajectory estimated according to the vehicle speed and yaw rate, and generates an optical flow image that displays the calculated velocity distribution as motion vectors for the aforementioned multiple points in a video. Note that the optical flow image is not limited to images in which the motion speed moves continuously based on changes in vehicle speed and trajectory, but also includes images that move in stages, or images that move at a constant speed regardless of changes in vehicle speed and trajectory.

[0010] In the first embodiment, the processing unit generates an optical flow image that moves from approximately the center of the display unit toward the left and right directions perpendicular to the direction of vehicle movement, displays the generated optical flow image on the display unit, and projects it as a displayed image light toward the left and right directions of the vehicle via a projection optical system. Therefore, an easy-to-view optical flow image can be displayed without using a large display panel.

[0011] In a second embodiment dependent on the first embodiment, the projection optical system includes a reflective member consisting of a mirror or prism having a reflective surface that is provided facing the display surface of the display and reflects the display light to each region set in the left and right directions, and one or more lenses that focus the optical flow image onto the projection surface, wherein the processing unit may perform control to display the generated optical flow image on the display and to project the optical flow image, which has been reflected by the reflective surface and focused, onto the projection surface.

[0012] In the second embodiment, the projection optical system consists of a mirror or prism having a reflective surface that is provided opposite the display surface of the display unit, and projects the optical flow image reflected by the reflective surface onto the projection surface (left projection surface and right projection surface) set up in front of the vehicle. Therefore, the optical flow image can be viewed from the front of the vehicle, which is the driver's gaze, without having to move their eyes, without using a large display panel that requires eye movement while driving. Thus, it can contribute to safe driving.

[0013] In a third embodiment dependent on the first or second embodiment, the processing unit may generate the optical flow image representing the vehicle motion of the vehicle, including translational and rotational motion of the vehicle estimated based on externally input information regarding vehicle speed and yaw rate.

[0014] In the third embodiment, the processing unit generates an optical flow image that represents the vehicle motion, including the translational and rotational movements of the vehicle, which are estimated based on information on the vehicle speed and yaw rate input from an external source (for example, the vehicle monitoring device 400 shown in Figure 4). For example, based on the estimated changes in vehicle speed and changes in direction, the processing unit calculates the apparent speed distribution of the scenery outside the vehicle flowing past the window, and can easily generate an optical flow image that displays the speed distribution as motion vectors for multiple points.

[0015] In a fourth embodiment dependent on the first to third embodiments, the processing unit may emit the optical flow image obliquely onto the reflective surface of the projection optical system, thereby controlling the apparent speed of the optical flow image moving from approximately the center of the vehicle toward each of the left and right directions so that it is perceived by the viewer as it moves toward each of the left and right directions.

[0016] By projecting the optical flow image at an angle, as shown in the projection plane 60 (left projection plane 61, right projection plane 62) in Figure 7, for example, the optical flow image is projected to expand in the horizontal (X-axis direction) and vertical (Y-axis direction) directions of the projection plane 60 as it moves from approximately the center of the vehicle toward the left and right directions perpendicular to the direction of vehicle movement. Therefore, when the optical flow image is moved at a constant speed, the apparent speed of movement increases as the optical flow image moves toward the left and right directions. Thus, when an optical flow image generated on a display is moved at a constant speed and projected at an angle by the projection optical system, the speed of movement changes according to the X-axis (it becomes faster the further it is from the projection direction). Here, let Vα(x) be the velocity function on the X-axis. However, since the velocity function Vα(x) alone is expected to result in a slower speed outside the flowing representation of the external scenery (outside the vehicle), a supplementary function Vβ(x) is necessary. Therefore, by setting the movement velocity of the optical flow image to Vα(x) + Vβ(x), the natural flow of the optical flow image can be represented (see, for example, Figure 7(a)).

[0017] In the fourth embodiment, the processing unit emits the optical flow image obliquely using a projection optical system, thereby controlling the apparent movement speed of the optical flow image, which is moving from approximately the center of the vehicle toward the left and right directions of the vehicle, to accelerate (velocity function Vα(x) + Vβ(x)) as it moves toward the left and right directions of the vehicle, so that it is perceived by the viewer. This allows the viewer, the occupant (driver or passenger), to perceive the optical flow image as accelerating as it flows naturally.

[0018] In a fifth embodiment dependent on the first to fourth embodiments, the processing unit may acquire information regarding the vehicle speed from an external source, and if a speed difference occurs between the acquired vehicle speed and the apparent moving speed of the optical flow image, it may perform control to adjust in a direction that eliminates the speed difference.

[0019] In the fifth embodiment, if the processing unit detects a speed difference between the vehicle speed obtained from an external source (for example, the vehicle monitoring device 400 shown in Figure 4) and the apparent movement speed (display speed) of the optical flow image, it controls the apparent movement speed of the optical flow image to approach a natural appearance by adjusting it in a direction that eliminates the speed difference, thereby presenting an optical flow image to the occupants (driver or passenger) in the vehicle that suppresses the sense of discomfort or unease caused to the viewer by the speed difference.

[0020] In a sixth embodiment dependent on the first to fifth embodiments, when the processing unit displays the optical flow image and the non-optical flow image together on the display, it may control the apparent movement speed of the non-optical flow image to decelerate as it moves from the approximate center of the vehicle toward the left and right directions, so that the apparent movement speed of the optical flow image and the apparent movement speed of the non-optical flow image are perceived equally by the viewer.

[0021] For example, if we consider a moving icon as a non-optical flow image, and move it on the display in the left-right direction of the vehicle at the same velocity as the optical flow image, the display light will be projected at an angle by the projection optical system, causing the icon itself to accelerate. In the sixth embodiment, to cancel this, control is performed to gradually decelerate when moving the icon on the display in the left-right direction of the vehicle (decelerating in a way that cancels out the velocity function Vα(x)) (see, for example, Figure 7(e)). In this way, by canceling the acceleration caused by the oblique projection of the display light, it is possible to project the non-optical flow image with an apparent constant velocity movement or with a representation that suppresses the acceleration caused by the oblique projection of the display light.

[0022] In the sixth embodiment, when the processing unit combines and displays an optical flow image and a non-optical flow image, it controls the apparent movement speed of the non-optical flow image to decrease as it moves in the left-right direction relative to the direction of vehicle travel, so that the apparent movement speed of the optical flow image and the apparent movement speed of the non-optical flow image are perceived by the viewer to be the same. By creating a difference in the apparent movement speeds of the optical flow image and the non-optical flow image in this way, the visibility of the non-optical flow image, which is represented by characters, symbols (icons), etc., that appear to move at a constant speed, can be improved.

[0023] In a seventh embodiment dependent on the first to sixth embodiments, the processing unit may perform image correction to narrow the distortion of the non-optical flow image, which appears to expand in the vertical and horizontal directions of the projection surface as it moves from approximately the center of the vehicle toward the left and right directions, by the amount of expansion, when the optical flow image and non-optical flow image are combined and displayed on the display unit.

[0024] Incidentally, the optical flow image projected onto the projection surface and the non-optical flow image, which appears to be moving at a constant velocity and is visible, are distorted in the horizontal and vertical directions as they move in the left-right direction (x-axis direction) perpendicular to the direction of vehicle movement. For this reason, in the seventh embodiment, the processing unit performs image correction to narrow the distortion of the non-optical flow image, which appears to be spreading in the vertical and horizontal directions on the projection surface as it moves from approximately the center of the vehicle toward the left-right direction of the vehicle, by the amount of spreading. In this way, by applying distortion correction so that the shape of non-optical flow images such as characters and symbols, which appear to be moving at a constant velocity with the optical flow image, is not distorted when they flow outside the projection area, the visibility of the non-optical flow image can be improved.

[0025] In an eighth embodiment dependent on the first to seventh embodiments, the processing unit may, when displaying the optical flow image and the warning image combined on the display unit, perform image correction to narrow the distortion of the warning image, which appears to expand in the vertical and horizontal directions of the projection surface as it moves from approximately the center of the vehicle toward the left and right directions, by the amount by which the warning image appears to be displayed in a fixed position.

[0026] In the eighth aspect, when the processing unit combines and displays an optical flow image and a warning image on a display unit, it performs image correction to narrow the distortion that appears to spread vertically and horizontally on the projection surface as it moves from approximately the center of the vehicle toward the left and right directions of the vehicle, so that the warning image appears to be displayed in a fixed position. According to the eighth aspect, the visibility of the warning image displayed in a fixed position is improved by performing inverse correction to narrow the amount that has spread toward the left and right directions of the vehicle, so that image data that is easier to see when projected in an accurate shape, such as a warning image, in a fixed position does not appear to be distorted horizontally and vertically on the projection surface.

[0027] In a ninth embodiment dependent on the first to fourth embodiments, when the projector displays a warning image at a fixed position on the display, the projector may correct the warning image projected onto the projection surface according to the display position of the warning image.

[0028] In the ninth embodiment, when a warning image is displayed at the fixed position, the original warning image data can be used as is by correcting it with a projector without processing the image data of the warning image (correction of the warning image data), and the shape of the characters and symbols of the warning content can be immediately seen.

[0029] A tenth aspect is a control method for an in-vehicle projection device having a projector including a display unit housed approximately in the center of the vehicle's dashboard, and a processing unit that causes a viewer to see the display light emitted from the projector by reflecting the display light onto a projection surface set in front of the vehicle via a projection optical system, the control method comprising: a step of the processing unit generating an optical flow image that moves from approximately the center of the display unit toward the left and right directions perpendicular to the direction of travel of the vehicle; and a step of the processing unit displaying the generated optical flow image on the display unit and projecting it onto the projection surface as the display light via the projection optical system.

[0030] In the tenth embodiment, the processing unit generates an optical flow image that moves from approximately the center of the display unit toward the left and right directions perpendicular to the direction of vehicle movement, displays the generated optical flow image on the display unit, and projects it as displayed image light toward the left and right directions of the vehicle via a projection optical system. As a result, a large display panel is not used, and the optical flow image projected toward the front of the vehicle can be viewed without changing the line of sight.

[0031] The eleventh embodiment is a program for an in-vehicle projection device having a projector including a display unit housed approximately in the center of the vehicle's dashboard, and a processing unit that reflects the display light emitted from the projector onto a projection surface set in front of the vehicle via a projection optical system, thereby making the display light visible to a viewer, wherein the program causes a processor in the processing unit to perform the steps of: generating an optical flow image that moves from approximately the center of the display unit toward the left and right directions perpendicular to the direction of travel of the vehicle; and displaying the generated optical flow image on the display unit and projecting it onto the projection surface as the display light via the projection optical system.

[0032] In the eleventh embodiment, the processor in the processing unit reads and executes a program stored in memory to generate an optical flow image that moves from approximately the center of the display unit toward the left and right directions perpendicular to the direction of vehicle movement. The generated optical flow image is then displayed on the display unit and projected as display image light toward the left and right directions of the vehicle via a projection optical system. As a result, an optical flow image can be displayed that can be viewed without changing the line of sight, without using a large display panel, and the optical flow image projected toward the front of the vehicle can be viewed.

[0033] The twelfth embodiment is an in-vehicle display system that allows a viewer to see the display light emitted from a projector including a display unit housed approximately in the center of the vehicle's dashboard, by reflecting the display light off a projection surface set in front of the vehicle via a projection optical system, and comprising: a vehicle monitoring device that monitors the behavior of the vehicle; and an in-vehicle projection device connected to the vehicle monitoring device via an input / output interface, which estimates the motion state of the vehicle from the behavior of the vehicle acquired from the vehicle monitoring device, generates an optical flow image that moves from approximately the center of the display unit of the vehicle toward the left and right directions perpendicular to the direction of travel of the vehicle, displays the generated optical flow image on the display unit, and projects the display light onto the projection surface via the projection optical system.

[0034] In the twelfth embodiment, the in-vehicle projection device generates an optical flow image that moves from approximately the center of the display toward the left and right directions perpendicular to the direction of vehicle movement, in accordance with the behavior of the vehicle acquired from the vehicle monitoring device, and displays the generated optical flow image on the display and projects it as displayed image light toward the left and right directions of the vehicle via the projection optical system. Therefore, an in-vehicle display system can be provided that allows the optical flow image projected toward the front of the vehicle to be viewed without changing the line of sight, without using a large display panel.

[0035] Those skilled in the art will readily understand that the embodiments of the present invention illustrated can be further modified without departing from the spirit of the invention.

[0036] Figure 1 is a cross-sectional view showing an example of the application of an in-vehicle projection device according to an embodiment of the present invention to a vehicle. Figure 2 is a view from above the vehicle showing an example of the application of an in-vehicle projection device according to an embodiment of the present invention to a vehicle. Figure 3 is a view from the perspective of a viewer (driver or passenger) showing an example of the application of an in-vehicle projection device according to an embodiment of the present invention to a vehicle. Figure 4 is a block diagram showing an example configuration of an in-vehicle display system including an in-vehicle projection device according to an embodiment of the present invention. Figure 5 is a block diagram showing the functional structure of the software (program) executed by the processing unit shown in Figure 4. Figure 6 is a flowchart showing the operation of an in-vehicle projection device according to an embodiment of the present invention. Figure 7 is a diagram included to deepen the understanding of the speed adjustment of images generated by the in-vehicle projection device according to an embodiment of the present invention.

[0037] The best embodiments described below are used to facilitate understanding of the present invention. Therefore, those skilled in the art should note that the present invention is not unduly limited by the embodiments described below (hereinafter referred to as "these embodiments").

[0038] (Configuration of the Embodiment) Refer to Figures 1 to 3. Figure 1 is a cross-sectional view showing an example of the application of the in-vehicle projection device 100 of this embodiment to a vehicle CR. Figure 2 is a view of the application of the in-vehicle projection device 100 of this embodiment to a vehicle CR from the top surface of the vehicle CR, and Figure 3 is a view of the application of the in-vehicle projection device 100 of this embodiment to a vehicle CR from the perspective of the viewer 2 (driver DR or passenger).

[0039] As shown in Figures 1 to 3, the in-vehicle projection device 100 of this embodiment projects display light L emitted from a projector 200 located approximately in the center of the dashboard 50 of the vehicle CR in a left-right direction perpendicular to the direction of travel of the vehicle CR (in this case, the projection surface 60) via a projection optical system 220, thereby allowing the display light L to be seen by the viewer 2.

[0040] The projector 200 includes a display (see 210 in FIG. 4), and for each region of the projection surface 60 (for example, the left projection region 211 corresponding to the left projection surface 61 shown in FIG. 7, the right projection region 212 corresponding to the right projection surface 62) set in each of the left - right directions orthogonal to the traveling direction of the vehicle CR from the approximate center of the display 210, it moves toward the optical flow image (OF in FIG. 3 Ｌ , OF Ｒ ), generates the generated optical flow image OF Ｌ , OF Ｒ , and displays it on the display 210 and projects it as display light L in the left - right direction (windshield WS) of the vehicle CR via the projection optical system 220.

[0041] Note that the projection optical system 220 is provided facing the display surface of the display 210, and has a mirror or prism having a reflecting surface that reflects the display light L to each region of the projection surface 60 (for example, the left projection region 211 corresponding to the left projection surface 61 shown in FIG. 7, the right projection region 212 corresponding to the right projection surface 62) set in each of the left - right directions orthogonal to the traveling direction of the vehicle CR, and one or more lenses that focus the optical flow image OF Ｌ , OF Ｒ on the projection surface 60. The projection optical system 220 projects the optical flow image OF Ｌ , OF Ｒ generated by the display 210, reflected by the reflecting surface, and focused, as display light L, through an opening provided in a cover 70 covering the projector 200 onto the projection surface 60 (or windshield WS) set in front of the vehicle CR.

[0042] Here, "optical flow" is used as an image - processing method in the field of computer vision, is used for the purpose of detecting the movement of objects in an image, and is a method of estimating the amount of movement from the intensity change of the image. The optical flow can calculate the optical flow of each part using methods such as the gradient method or block matching using the gradient of the image as a clue. Also, "the optical flow image OF Ｌ , OF Ｒ" is, for example, an image of a two-dimensional motion vector field on an image plane formed by the motion vectors of arbitrary multiple points in the external scenery observed from a vehicle window while the vehicle is in motion. For example, the flow image generation unit 311b shown in Figure 5 calculates the apparent velocity distribution of the external scenery flowing through the window based on changes in vehicle speed and changes in course estimated according to the vehicle speed and yaw rate, and displays the calculated velocity distribution as motion vectors for the aforementioned multiple points in an optical flow image OF Ｌ OF Ｒ This generates an optical flow image OF. Note that this is not limited to images where the movement speed is continuously changing based on changes in vehicle speed or direction; it also includes images that move in stages, or images that move at a constant speed regardless of changes in vehicle speed or direction. Furthermore, in the following explanation, this optical flow image OF will be referred to as OF. Ｌ OF Ｒ Unless otherwise specified, flow images OF Ｌ OF Ｒ The explanation is given under the guise of this.

[0043] Refer to Figure 4. Figure 4 is a block diagram showing the configuration of an in-vehicle display system 1000 including the in-vehicle projection device 100 of this embodiment. As shown in Figure 4, the in-vehicle display system 1000 of this embodiment includes a vehicle monitoring device 400 and the in-vehicle projection device 100 of this embodiment, which are connected via a vehicle LAN (Local Area Network) such as a CAN (Control Area Network) 500.

[0044] The vehicle monitoring device 400 primarily monitors the behavior of the vehicle CR and communicates with the in-vehicle projection device 100 based on the CAN standard (CAN communication) via the ECUs (here, the steering control ECU 401 and the driving control ECU 403) located within the vehicle CR. The vehicle monitoring device 400 can transmit the steering angle of the steering wheel 80 detected by the steering angle sensor 402 to the in-vehicle projection device 100 of this embodiment via the steering control ECU 401 and CAN 500. It can also transmit the rotational angular velocity of the vehicle CR detected by the gyro sensor 404, the longitudinal G and lateral G acting on the vehicle CR detected by the acceleration sensor 405, and the rotational speed of the wheels detected by the wheel speed sensor 406 to the in-vehicle projection device 100 of this embodiment via the driving control ECU 403 and CAN 500.

[0045] The steering control ECU 401 primarily controls the steering system (not shown) of the vehicle CR, and can output the yaw rate, estimated by calculation based on the steering angle of the steering wheel 80 detected by the steering angle sensor 402, which represents the speed at which the rotational motion (yawing) of the vehicle body changes around the vertical axis, to the in-vehicle projection device 100 via CAN 500. The driving control ECU 403 primarily controls the drive and braking systems (not shown) of the vehicle CR, and can provide the vehicle speed, estimated by calculation from the wheel rotation speed detected by the wheel speed sensor 406, to the in-vehicle projection device 100 via CAN 500. Note that the vehicle speed and yaw rate may be detected by separately provided speed sensors and yaw rate sensors, rather than by calculation.

[0046] The vehicle monitoring device 400 further includes a camera that captures the forward view (actual scenery) of the vehicle CR in order to recognize the surrounding driving environment, including the area in front of the vehicle CR, and a LiDAR (Light Detection And Ranging) that uses near-infrared light, visible light, and ultraviolet light to illuminate obstacles, etc., located in front of the vehicle CR as captured by the camera, captures the reflected light with a light sensor, and determines the distance to the obstacle based on the time difference. Note that the camera and LiDAR are not shown in the illustration.

[0047] The in-vehicle projection device 100 of this embodiment projects display light L emitted from a projector 200 housed approximately in the center of the dashboard 50 of the vehicle CR onto a projection surface 60 (left projection surface 61, right projection surface 62) set in front of the vehicle CR via a projection optical system 220, thereby allowing the display light L to be viewed by the driver DR or passengers. As described above with reference to Figures 1 to 3, the in-vehicle projection device 100 of this embodiment includes a projector 200 and a display control device 300. The projector 200 is housed approximately in the center of the dashboard 50 of the vehicle CR and includes a display unit 210 and a projection optical system 220.

[0048] The display unit 210 is composed of, for example, a DMD (Digital Micro-mirror), an LCoS (Liquid Crystal on silicon), or an LCD (Liquid Crystal Display), and the flow image OF is generated by the input display signal. Ｌ OF Ｒ The projection optical system 220 is provided opposite the display surface of the display unit 210 and consists of a reflective member made of a mirror or prism having a reflective surface that reflects the display light L to projection surfaces 60 (left projection surface 61, right projection surface 62) set in the left and right directions perpendicular to the direction of travel of the vehicle CR, and the flow image OF Ｌ OF Ｒ The system includes one or more lenses that focus on the projection plane 60 (left projection plane 61 and right projection plane 62 respectively), and the flow image OF generated by the display 210, reflected by the reflective surface, and focused on the projection plane 60 (left projection plane 61 and right projection plane 62 respectively) set in front of the vehicle CR, is projected onto the projection plane 60 (left projection plane 61 and right projection plane 62 respectively). Ｌ OF Ｒ It can be projected.

[0049] The display control device 300 includes a computer equipped with a processing unit 311, a RAM 312, a storage unit 313, an input / output interface (I / O) 314, and an internal bus connecting these. The processing unit 311 is hardware for arithmetic processing coupled with the RAM 312. The processing unit 311 includes at least one arithmetic core such as a processor (CPU: Central Processing Unit) and a GPU (Graphics Processing Unit), and operates in cooperation with the GPU by sequentially reading and executing programs stored in the storage unit 313, thereby enabling the display control device 210 included in the projector 200 to display a flow image OF moving from approximately the center toward the left and right directions perpendicular to the direction of travel of the vehicle CR. Ｌ OF Ｒ Generates the generated flow image OF Ｌ OF Ｒ The data can be displayed on the display unit 210 and projected onto the projection surface 60 as display light L via the projection optical system 220.

[0050] Furthermore, the processing unit 311 generates a flow image OF that represents the vehicle motion of the vehicle CR, including the translational and rotational movements of the vehicle CR, which are estimated based on information on the vehicle speed and yaw rate input from an external source (vehicle monitoring device 400). Ｌ OF Ｒ It can generate [this].

[0051] Furthermore, the processing unit 311 processes the flow image OF Ｌ OF Ｒ By projecting the light obliquely onto the reflective surface of the reflective member of the projection optical system 220, a flow image OF moves from approximately the center of the vehicle CR in the left and right directions perpendicular to the direction of travel of the vehicle CR. Ｌ OF Ｒ The apparent speed of movement can be controlled to accelerate as it moves in each direction (left or right) so that it is visible to the driver (DR) or passengers.

[0052] Furthermore, the processing unit 311 acquires information regarding the vehicle speed from an external source (vehicle monitoring device 400), and processes the acquired vehicle speed and flow image OF Ｌ OF ＲIf a speed difference occurs between the apparent speed of movement and the actual speed, control can be performed to adjust the speed to eliminate the speed difference.

[0053] Furthermore, the processing unit 311 displays the flow image OF on the display unit 210. Ｌ OF Ｒ When displaying a composite of non-flow images such as ambient images, the flow image OF moves from approximately the center of the vehicle CR toward the left and right directions perpendicular to the direction of travel of the vehicle CR. Ｌ OF Ｒ To ensure that the apparent speed of the non-flow image and the apparent speed of the non-flow image are perceived similarly by the driver (DR) or passengers, control can be implemented to decelerate the apparent speed of the non-flow image as it moves from approximately the center of the vehicle (CR) towards the left and right.

[0054] Furthermore, the processing unit 311 can perform image correction to narrow the distortion of the non-flow image, which appears to expand vertically and horizontally in the projection plane 60 (left projection plane 61, right projection plane 62) as it moves from approximately the center of the vehicle CR toward the left and right directions perpendicular to the direction of travel of the vehicle CR, by the amount of expansion.

[0055] Furthermore, the processing unit 311 displays the flow image OF on the display unit 210. Ｌ OF Ｒ When displaying a warning image such as an ABS (Anti-lock Brake System) warning, the distortion of the warning image, which appears to expand vertically and horizontally as it moves from approximately the center of the vehicle CR toward the left and right directions perpendicular to the direction of travel of the vehicle CR, can be corrected by narrowing the distortion by the amount that it has expanded so that the warning image appears to be displayed in a fixed position.

[0056] Furthermore, when displaying a warning image in a fixed position, the projector 200 can correct the warning image projected onto the projection surface 60 (left projection surface 61, right projection surface 62) according to the display position of the warning image, without requiring image correction by the processing unit 311.

[0057] The processing unit 311 may further include an FPGA (Field-Programmable Gate Array), an NPU (Neural Network Processing Unit), and an IP core (Intellectual Property Core) equipped with other dedicated functions. The processing unit 311 accesses the RAM 312 and performs various processes to realize the functions of each function block described later. A non-volatile storage medium is implemented in the storage unit 313. Various software (programs) executed by the processing unit 311 are stored in the storage unit 313.

[0058] The input / output interface (I / O) 314 manages the interface with the CAN 500 and transmits information regarding the behavior of the vehicle CR, such as yaw rate and vehicle speed, acquired from the vehicle monitoring device 400 (steering control ECU 401, driving control ECU 403) via CAN communication to the processing unit 311.

[0059] Refer to Figure 5. Figure 5 is a block diagram showing the functional structure of the software (program) executed by the processing unit 311 shown in Figure 4. As shown in Figure 5, the processing unit 311 includes a vehicle motion estimation unit 311a, a flow image generation unit 311b, a warning image generation unit 311c, a non-flow image generation unit 311d, a display speed adjustment unit 311e, an image correction unit 311f, and an image synthesis unit 311g.

[0060] The vehicle motion estimation unit 311a has the function of estimating the vehicle motion of the vehicle CR, including the translational and rotational movements of the vehicle CR, based on information on vehicle speed and yaw rate input via CAN 500 from the steering control ECU 401 and driving control ECU 403 of the vehicle monitoring device 400.

[0061] Furthermore, the flow image generation unit 311b calculates, for example, the apparent speed distribution of the scenery outside the car flowing past the car window, based on the changes in vehicle speed and yaw rate estimated by the vehicle motion estimation unit 311a, and displays the calculated speed distribution as a motion vector for multiple points in a flow image OF Ｌ OF Ｒ It has the function of generating [something].

[0062] Furthermore, the warning image generation unit 311c has the function of generating warning images projected onto a fixed position on the projection surface 60, such as the ABS warning, half-door warning, and fuel level warning mentioned above, while the non-flow image generation unit 311d has the function of generating moving text information and icons, such as map information notifications.

[0063] The display speed adjustment unit 311e controls the flow image OF Ｌ OF Ｒ By projecting the light obliquely onto the reflective surface of the projection optical system 220, a flow image OF moves from approximately the center of the vehicle CR toward the left and right directions of the vehicle CR. Ｌ OF Ｒ The apparent speed of the vehicle can be controlled to accelerate as it moves in the left-right direction of the vehicle CR, so that it is visible to the driver DR or passengers.

[0064] The display speed adjustment unit 311e acquires information regarding the vehicle speed from the driving control ECU 403 of the vehicle monitoring device 400, and the acquired vehicle speed and flow image OF Ｌ OF Ｒ If a speed difference occurs between the apparent moving speed and the actual speed, control can be performed to adjust the speed to eliminate the speed difference. In addition, the display speed adjustment unit 311e displays a flow image OF on the display unit 210. Ｌ OF Ｒ When displaying a composite image of a flow image and a non-flow image, the flow image OF moves from approximately the center of the vehicle CR toward the left and right directions of the vehicle CR. Ｌ OF Ｒ To ensure that the apparent speed of the non-flow image and the apparent speed of the non-flow image are perceived similarly by the driver (DR) or passengers, control can be implemented to decelerate the apparent speed of the non-flow image as it moves from approximately the center of the vehicle (CR) towards the left and right.

[0065] The image correction unit 311f outputs the flow image OF by the flow image generation unit 311b, the warning image generation unit 311c, and the non-flow image generation unit 311d. Ｌ OF ＲImage correction is performed to cancel the distortion of the warning image and the non-flow image, and the results can be output to the image synthesis unit 311g. The image correction unit 311f can perform image correction to narrow the distortion of the non-flow image, which appears to widen in the vertical and horizontal directions of the projection plane 60 (left projection plane 61, right projection plane 62) as it moves from approximately the center of the vehicle CR toward the left and right directions of the vehicle CR, by the amount of widening.

[0066] Furthermore, the image correction unit 311f displays the flow image OF on the display unit 210. Ｌ OF Ｒ When displaying a warning image in combination with the warning image, the distortion of the warning image, which appears to expand vertically and horizontally on the projection surface 60 (left projection surface 61, right projection surface 62) as it moves from approximately the center of the vehicle CR toward the left and right directions of the vehicle CR, can be corrected separately from the non-flowga image by narrowing the distortion by the amount that it has expanded so that the warning image appears to be displayed in a fixed position. Furthermore, when the projector 200 displays the warning image in a fixed position without using the image correction unit 311f, the warning image projected onto the projection surface 60 can be corrected according to the display position of the warning image.

[0067] The image synthesis unit 311g processes the flow image OF, which has been corrected by the image correction unit 311f to cancel out distortion. Ｌ OF Ｒ It has a function to combine warning images and non-flow images and output them to the projector 200 (display unit 210).

[0068] (Operation of the Embodiment) The operation of the in-vehicle projection device 100 of this embodiment, shown in Figures 1 to 5, will be described in detail below with reference to Figures 6 and 7.

[0069] Refer to Figure 6. Figure 6 is a flowchart showing the operation of the in-vehicle projection device 100 of this embodiment. In the in-vehicle projection device 100 of this embodiment, the processing unit 311 starts displaying the flow image when predetermined conditions are met. Here, the predetermined conditions are, for example, (1) when the vehicle CR starts moving, (2) when the vehicle CR is traveling at a predetermined speed or higher, and (3) when the driver DR, who is the viewer, operates the flow image OF Ｌ OFＲ The display is initiated in the following cases: (4) when the driving control ECU 403 of the vehicle monitoring device 400 determines that the display should be initiated based on the driving environment of the vehicle CR; (5) when the display is determined to be initiated based on the driving state of the driver DR who is the viewer; or a combination thereof (however, it is not limited to these cases).

[0070] As shown in Figure 6, when the predetermined conditions described above are met, the processing unit 311 acquires information regarding vehicle speed and yaw rate, which are generated by the driving control ECU 403 and steering control ECU 401 of the vehicle monitoring device 400, via the CAN 500 and input / output interface 314 (I / O) (step ST101).

[0071] Next, the processing unit 311, specifically the vehicle motion estimation unit 311a, estimates the motion state of the vehicle CR, such as changes in vehicle speed and changes in direction, based on information about the vehicle CR's behavior acquired from the vehicle monitoring device 400, for example, the vehicle speed acquired from the driving control ECU 403 and the yaw rate acquired from the steering control ECU 401 (step ST102). Then, the flow image generation unit 311b calculates, for example, the apparent speed distribution of the scenery outside the vehicle flowing past the window, based on the changes in vehicle speed and changes in direction estimated by the vehicle motion estimation unit 311a, and displays the calculated speed distribution as a motion vector for multiple points in a flow image OF. Ｌ OF Ｒ Generate (step ST103). Flow image OF generated here Ｌ OF Ｒ The warning images, such as ABS warnings, which are preferably viewed at a fixed position and are generated by the warning image generation unit 311c, and / or non-flow images, such as text information and icons, generated by the non-flow image generation unit 311d for map information notifications, are output to the image correction unit 311f.

[0072] Next, the processing unit 311 adjusts the display speed adjustment unit 311e to the flow image OF generated by the flow image generation unit 311b. Ｌ OF Ｒ By projecting the light obliquely onto the reflective surface of the projection optical system 220, a flow image OF moves from approximately the center of the vehicle CR toward the left and right directions perpendicular to the direction of travel of the vehicle CR.Ｌ OF Ｒ The apparent speed of the vehicle is controlled so that it appears to accelerate to the driver DR, who is the observer, as it moves in each direction to the left and right. At this time, the display speed adjustment unit 311e acquires information on the vehicle speed from the vehicle monitoring device 400 (driving control ECU 403), and the acquired vehicle speed and flow image OF Ｌ OF Ｒ If a speed difference occurs between the apparent movement speed and the displayed speed (step ST104 "YES"), the displayed speed is adjusted to eliminate the speed difference (step ST105). If there is no speed difference (step ST104 "NO"), the process proceeds to step ST106.

[0073] Furthermore, the display speed adjustment unit 311e displays the flow image OF to the display unit 210. Ｌ OF Ｒ When a non-flow image, such as a moving icon, is generated by the non-flow image generation unit 311d and combined with the flow image OF, the flow image OF moves from approximately the center of the vehicle CR toward the left and right directions perpendicular to the direction of travel of the vehicle CR. Ｌ OF Ｒ Speed ​​adjustment may be performed to control the apparent speed of the non-flow image as it moves from approximately the center of the vehicle CR toward the left and right directions, so that the apparent speed of the non-flow image and the apparent speed of the non-flow image are perceived by the driver DR, who is the viewer, as moving at the same constant speed.

[0074] By the way, flow image OF Ｌ OF Ｒ By emitting it at an angle, the flow image OF Ｌ OF Ｒ As the projection extends from approximately the center of the vehicle CR toward the left and right directions perpendicular to the direction of travel of the vehicle CR, it expands in the lateral direction (X-axis direction) and vertical direction (Y-axis direction) of the projection plane 60. Therefore, the flow image OF Ｌ OF Ｒ When moved at a constant speed, the flow image OF Ｌ OF ＲAs it moves in each of the left - right directions, the apparent moving speed increases. Thus, on the display 210, when the generated flow images OF Ｌ , OF Ｒ are obliquely projected by the projection optical system 220 when moved at a constant speed, the moving speed changes according to the X - coordinate axis (here, let it be the speed function Vα(x) on the X - coordinate axis). However, if it is only the speed function Vα(x), it is assumed that the speed outside the expression of the flowing exterior scene will be slower, so a supplementary function Vβ(x) is required. Therefore, by setting the moving speed of the flow images OF Ｌ , OF Ｒ to Vα(x)+Vβ(x), a natural flow of the flow images OF Ｌ , OF Ｒ can be expressed.

[0075] Also, for example, assuming an icon that moves as a non - flow image, when the non - flow image and the flow images OF Ｌ , OF Ｒ are each moved at a constant speed in the left - right direction orthogonal to the traveling direction of the vehicle CR, the display light L is obliquely projected by the projection optical system 220, causing the icon image to accelerate. Therefore, when moving it in the left - right direction on the display 210 to cancel this, control is performed to gradually decelerate (decelerate so as to cancel the speed function Vα(x)). Thus, by canceling the acceleration due to the oblique projection of the display light L, it is possible to project in a manner that suppresses the apparent constant - speed movement of the non - flow image or the acceleration due to the oblique projection of the display light L.

[0076] Specifically, if there is a non - flow image to be synthesized with the flow images OF Ｌ , OF Ｒ generated by the flow image generation unit 311b (step ST106 “YES”), the display speed adjustment unit 311e acquires the non - flow image (icon) generated by the non - flow image generation unit 311d (step ST107). Then, the display speed adjustment unit 311e moves the flow images OF Ｌ , OF Ｒ from the approximate center of the vehicle CR in each of the left - right directions.The apparent movement speed of the non-flow image is adjusted and controlled so that the apparent movement speed of the vehicle and the apparent movement speed of the acquired non-flow image are visually recognized by the driver DR, who is a viewer, at the same speed. Specifically, the apparent movement speed of the non-flow image is decelerated as it moves from the approximate center of the vehicle CR in both the left and right directions (step ST108).

[0077] A supplementary explanation of the speed adjustment by the processing unit 311 (display speed adjustment unit 311e) will be given with reference to FIG. 7. FIG. 7 is a diagram cited to deepen the understanding of the speed adjustment of the non-flow image generated by the in-vehicle projection device 100 of the present embodiment. In FIG. 7(a), reference numeral 210 denotes a projection area (display area) set corresponding to the projection surface 60 (left projection surface 61, right projection surface 62). Reference numeral 211 denotes a left projection area provided corresponding to the left projection surface 61, and reference numeral 212 denotes a right projection area provided corresponding to the right projection surface 62. For example, they are assigned to a predetermined area of the RAM 312 shown in FIG. 4 and stored. Also, in the figure, the arrows indicate the direction in which the flow image OF Ｌ , OF Ｒ and the non-flow image flow.

[0078] FIGS. 7(b) to 7(e) are graphs showing the change in the movement speed of the non-flow image (the state of speed adjustment) when the horizontal axis (x-axis) is the time axis t and the vertical axis (y-axis) is scaled with the movement speed V of the non-flow image. Here, the movement speed of the non-flow image on the display 210 is shown by a thin solid line, and the movement speed of the non-flow image projected onto the projection surface 60 (left projection surface 61, right projection surface 62) is shown by a thick solid line.

[0079] In FIG. 7(b), on the left projection surface 61 and the right projection surface 62, the non-flow image on the display 210 is moved at the same speed as the flow image OF Ｌ , OF Ｒ in the direction corresponding to the left and right direction (X-axis direction) orthogonal to the traveling direction of the vehicle CR from the center of the vehicle CR. The natural acceleration situation of the flow image OF Ｌ , OF Ｒ and the non-flow image by projection is shown. Also, in FIG. 7(c), in the left and right direction (X-axis direction), the flow image OF Ｌ , OF ＲThe diagram also shows a case where a non-flow image is moved while being accelerated (natural acceleration by projection + acceleration by the display unit 210).

[0080] Furthermore, Figure 7(d) shows a case in which the movement speed V of the non-flow image is adjusted according to the vehicle speed. The processing unit 311 can change the movement speed V of the non-flow image displayed on the display unit 210 according to the vehicle speed (which may also be acceleration) acquired from the vehicle speed (or acceleration) of the vehicle monitoring device 400, etc. In this case, the system controls the movement speed V of the non-flow image to increase as the vehicle speed increases (or acceleration increases). In the figure, the movement speed V of the non-flow image, which changes according to the vehicle speed or acceleration, is shown with a solid line, as well as a dotted line and a dashed line. Also, in Figure 7(e), when moving a non-flow image consisting of character information or symbols, such as for map information notification, the processing unit 311 (display speed adjustment unit 311e) moves the non-flow image while decelerating it in the direction of the display unit 210 toward the X-axis direction (left and right direction perpendicular to the direction of travel of the vehicle CR) of the left projection plane 61 and the right projection plane 62, in order to suppress natural acceleration due to projection.

[0081] The explanation returns to Figure 6. The non-flow image after speed adjustment is output to the image correction unit 311f. The image correction unit 311f performs image correction to cancel the distortion of the non-flow image generated by the non-flow image generation unit 311d, and can output it to the image synthesis unit 311g. The image correction unit 311f performs correction processing on the non-flow image data to narrow the distortion that appears to spread in the vertical and horizontal directions of the projection plane 60 as the non-flow image moves from approximately the center of the vehicle CR toward the left and right directions perpendicular to the direction of travel of the vehicle CR, by the amount of the spread (step ST109).

[0082] Next, the processing unit 311 processes the flow image OF generated by the flow image generation unit 311b. Ｌ OF Ｒ Then, by outputting the non-flow image, whose distortion has been canceled by the image correction unit 311f, to the image synthesis unit 311g, a composite image is generated in the image synthesis unit 311g (step ST110), and output to the display unit 210 of the projector 200. The display unit 210 displays the flow image OFＬ OF Ｒ A composite image of the non-flow image is displayed, and the composite image can be projected onto the projection surface 60 (left projection surface 61, right projection surface 62) as display light L via the projection optical system 220 (step ST111).

[0083] Meanwhile, in step ST106, the flow image OF Ｌ OF Ｒ If it is determined that there is no non-flow image to synthesize with (step ST106 "NO"), the processing unit 311 further processes the flow image OF Ｌ OF Ｒ The presence or absence of a warning image to be combined with the flow image is determined (step ST112). Here, the flow image OF Ｌ OF Ｒ If there is a warning image to be combined with the flow image (step ST112 "YES"), that is, when the flow image and the warning image are combined and displayed on the display unit 210, the image correction unit 311f corrects the warning image data separately from the non-flow image by narrowing the distortion of the warning image, which appears to spread in the vertical and horizontal directions of the projection surface 60 as it moves from approximately the center of the vehicle CR toward the left and right directions perpendicular to the direction of travel of the vehicle CR, so that the warning image appears to be displayed in a fixed position (step ST113).

[0084] Next, the processing unit 311 processes the flow image OF generated by the flow image generation unit 311b in the image synthesis unit 311g. Ｌ OF Ｒ Then, the image correction unit 311f outputs the warning image, whose distortion has been canceled, to the image synthesis unit 311g, which generates a composite image (step ST110), and outputs it to the display unit 210 of the projector 200. As a result, the display unit 210 displays the flow image OF Ｌ OF Ｒ A composite image of the warning image and the display image can be shown and projected onto the projection surface 60 as display light L via the projection optical system 220 (step ST111).

[0085] Furthermore, the display unit 210 shows the flow image OF Ｌ OF ＲWhen displaying a combined warning image, as described above, the original warning image data can be used as is by correcting it with the projector 200 without processing the warning image data (correcting the warning image data).

[0086] Furthermore, the flowchart shown in Figure 6 above does not limit the processing procedure according to the present invention to the procedure shown in the flowchart, and additional, deleted, or rearranged steps may be added or deleted without departing from the spirit and technical idea of ​​the invention.

[0087] (Effects of the Embodiment) As described above, the in-vehicle projection device of this embodiment is an in-vehicle projection device 100 that allows viewers (including the driver DR and passengers) to view the display light L emitted from a projector 200 housed approximately in the center of the dashboard 50 of the vehicle CR, as shown in Figures 1 to 4, by projecting the display light L onto a projection surface 60 (left projection surface 61, right projection surface 62) set in front of the vehicle CR via a projection optical system 220. The in-vehicle projection device 100 then displays an optical flow image OF that moves from approximately the center of the display unit 210 included in the projector 200 toward the left and right directions perpendicular to the direction of travel of the vehicle CR. Ｌ OF Ｒ Generates the optical flow image OF Ｌ OF Ｒ The system includes a processing unit 311 that displays the data on a display unit 210 and projects it as display light L onto a projection surface 60 (left projection surface 61, right projection surface 62) via a projection optical system 220.

[0088] Here, "Optical flow image OF Ｌ OF Ｒ"This refers to an optical flow image (OF), which is an image of a two-dimensional motion vector field on an image plane formed by the motion vectors of arbitrary points in the external scenery observed from the window of a vehicle while it is moving. For example, the flow image generation unit 311b shown in Figure 5 calculates the apparent velocity distribution of the external scenery flowing through the window based on changes in vehicle speed and changes in course estimated according to the vehicle speed and yaw rate, and displays the calculated velocity distribution as motion vectors for the aforementioned multiple points in an optical flow image (OF). Ｌ OF Ｒ It generates an optical flow image OF. Ｌ OF Ｒ This is not limited to images where the speed of movement is continuously changing based on changes in vehicle speed or direction, but also includes images that move in stages, or images that move at a constant speed regardless of changes in vehicle speed or direction.

[0089] According to the in-vehicle projection device 100 of this embodiment, the processing unit 311 generates an optical flow image OF that moves from approximately the center of the display unit 210 toward the left and right directions perpendicular to the direction of travel of the vehicle. Ｌ OF Ｒ Generates the optical flow image OF Ｌ OF Ｒ The process involves displaying the image on the display unit 210 and projecting it as display light L in the left-right direction of the vehicle CR via the projection optical system 220, thus providing an easy-to-view optical flow image OF without using a large display panel. Ｌ OF Ｒ We can provide an in-vehicle projection device 100 that can display [something].

[0090] Furthermore, according to the in-vehicle projection device 100 of this embodiment, the projection optical system 220 consists of a mirror or prism having a reflective surface that is provided opposite the display surface of the display unit 210, and the optical flow image OF reflected by the reflective surface is projected onto the projection surface 60 set in front of the vehicle CR. Ｌ OF ＲBecause it takes the form of projecting an optical flow image OF without requiring eye movement, it does not use a large display panel that is viewed with eye movement while driving, but rather allows the driver to view the optical flow image OF from the front of the vehicle CR without eye movement while driving. Ｌ OF Ｒ Because it allows for visual confirmation, it can contribute to safer driving.

[0091] Furthermore, according to the in-vehicle projection device 100 of this embodiment, the processing unit 311 generates an optical flow image OF that represents the vehicle motion of the vehicle CR, including the translational and rotational movements of the vehicle CR, which are estimated based on information on the vehicle speed and yaw rate input from an external source (for example, the driving control ECU 403 and steering control ECU 401 of the vehicle monitoring device 400). Ｌ OF Ｒ To generate this, for example, based on the estimated changes in vehicle speed and changes in direction, the apparent speed distribution of the scenery outside the car window is calculated, and this speed distribution is displayed as a moving vector for multiple points in an optical flow image (OF). Ｌ OF Ｒ It can be easily generated.

[0092] Furthermore, according to the in-vehicle projection device 100 of this embodiment, the processing unit 311 processes the optical flow image OF Ｌ OF Ｒ By projecting the light obliquely through the projection optical system 220, an optical flow image OF is projected from approximately the center of the vehicle CR toward the left and right directions perpendicular to the direction of travel of the vehicle CR. Ｌ OF Ｒ To control the apparent speed of movement so that it appears to the viewer as accelerating in each of the left and right directions (velocity function Vα(x) + Vβ(x)), an optical flow image OF that flows naturally to the viewer (occupant, driver or passenger) Ｌ OF Ｒ This can be made to appear as if it is accelerating (see Figure 7(b)).

[0093] Furthermore, according to the in-vehicle projection device 100 of this embodiment, the processing unit 311 receives the vehicle speed and optical flow image OF from an external source.Ｌ OF Ｒ When a speed difference occurs between the apparent movement speed (display speed) and the optical flow image OF Ｌ OF Ｒ By adjusting the apparent speed of movement to eliminate the speed difference and control it to create a natural appearance, the optical flow image OF suppresses the sense of unease or discomfort caused to the viewer by the speed difference, for the occupants (driver DR or passengers) in the vehicle CR. Ｌ OF Ｒ It is possible to present this.

[0094] Furthermore, according to the in-vehicle projection device 100 of this embodiment, the processing unit 311 processes the optical flow image OF Ｌ OF Ｒ When displaying a composite image of the optical flow image OF and a non-optical flow image, the optical flow images OF move in the left and right directions relative to the direction of travel of the vehicle CR. Ｌ OF Ｒ To ensure that the apparent movement speed of the optical flow image and the apparent movement speed of the non-optical flow image are perceived by the viewer as being the same, control is performed to decelerate the apparent movement speed of the non-optical flow image as it moves in the left-right direction (see, for example, Figure 7(e)). In this way, the optical flow image OF Ｌ OF Ｒ By creating a difference in the apparent movement speed of non-optical flow images, the visibility of non-optical flow images, such as characters or symbols (icons) that appear to move at a constant speed, can be improved.

[0095] Furthermore, according to the in-vehicle projection device 100 of this embodiment, the processing unit 311 performs image correction to narrow the distortion of the non-optical flow image, which appears to widen in the vertical and horizontal directions of the projection surface 60 as it moves from approximately the center of the vehicle CR toward the left and right directions perpendicular to the direction of travel of the vehicle CR, by the amount of widening. In this way, the apparent optical flow image OF Ｌ OF ＲBy applying distortion correction to prevent the shape of non-optical flow images such as characters and symbols, which appear to be moving at a constant speed, from being distorted when they flow outside the projection surface 60 (left projection surface 61, right projection surface 62), the visibility of non-optical flow images can be improved.

[0096] Furthermore, according to the in-vehicle projection device 100 of this embodiment, the processing unit 311 displays an optical flow image OF on the display unit 210. Ｌ OF Ｒ When displaying a warning image in combination with a warning image, image correction is performed to narrow the distortion that appears to spread vertically and horizontally on the projection surface 60 as it moves from approximately the center of the vehicle CR toward the left and right directions perpendicular to the direction of travel of the vehicle CR, so that the warning image appears to be displayed in a fixed position. In this way, the visibility of the warning image displayed in a fixed position is improved by performing an inverse correction that narrows the amount that spreads as it moves toward the left and right directions of the vehicle CR, so that image data that is easier to see when projected in an accurate shape to a fixed position, such as a warning image, does not appear to be distorted horizontally and vertically on the projection surface 60 (left projection surface 61, right projection surface 62).

[0097] Furthermore, according to the in-vehicle projection device 100 of this embodiment, when displaying a warning image at a fixed position, the original warning image data can be used as is by correcting it with the projector 200 without processing the warning image data (correction of the warning image data), and the shape of the characters and symbols of the warning content can be immediately seen.

[0098] The control method for the in-vehicle projection device of this embodiment is a control method for an in-vehicle projection device 100 having, for example as shown in Figures 1 to 4, a projector 200 including a display 210 housed approximately in the center of the dashboard 50 of the vehicle CR, and a processing unit 311 that reflects the display light L emitted from the projector 200 to projection surfaces 60 (left projection surface 61, right projection surface 62) set in front of the vehicle CR via a projection optical system 220, thereby allowing the display light L to be viewed by a viewer (passenger including the driver DR). The control method is, for example as shown in Figure 6, an optical flow image OF that moves from approximately the center of the display 210 toward the left and right directions perpendicular to the direction of travel of the vehicle CR. Ｌ OF Ｒ Steps to generate the optical flow image OF (ST101 to ST103) and the processing unit 311 Ｌ OF Ｒ The process includes steps (ST104 to ST113) of displaying the data on the display unit 210 and projecting it as display light L onto the projection surface 60 (left projection surface 61, right projection surface 62) via the projection optical system 220.

[0099] According to the control method for the in-vehicle projection device of this embodiment, the processing unit 311 controls the optical flow image OF that moves from approximately the center of the display unit 210 toward the left and right directions perpendicular to the direction of travel of the vehicle CR. Ｌ OF Ｒ Generates the optical flow image OF Ｌ OF Ｒ The optical flow image OF is displayed on the display unit 210 and projected as display light L in the left-right direction of the vehicle CR via the projection optical system 220, thus eliminating the need for a large display panel and allowing the optical flow image OF to be projected in front of the vehicle CR. Ｌ OF Ｒ This allows for visual confirmation without changing the driver's gaze. Therefore, it can contribute to safer driving.

[0100] The program of this embodiment is a program (stored in RAM 312) for an in-vehicle projection device 100, which includes a projector 200 containing a display 210 housed approximately in the center of the vehicle CR's dashboard 50, as shown in Figures 1 to 4, and a processing unit 311 that reflects the display light L emitted from the projector 200 onto projection surfaces 60 (left projection surface 61, right projection surface 62) set in front of the vehicle CR via a projection optical system 220, thereby projecting the display light L to be visible to the viewer (including the driver Dr and other passengers). The program is then used to generate an optical flow image OF that moves from approximately the center of the display 210 toward the left and right directions perpendicular to the direction of travel of the vehicle CR, as shown in Figure 6. Ｌ OF Ｒ The process of generating the optical flow image OF (steps ST101 to ST104) and the generated optical flow image OF Ｌ OF Ｒ The process involves displaying the output on the display unit 210 and projecting it as display light L onto the projection surface 60 (left projection surface 61, right projection surface 62) via the projection optical system 220 (steps ST105 to ST113).

[0101] According to the program of this embodiment, the processor in the processing unit 311 reads and executes the program stored in the memory (RAM 312), thereby generating an optical flow image OF that moves from approximately the center of the display unit 210 toward the left and right directions perpendicular to the direction of travel of the vehicle CR. Ｌ OF Ｒ Generates the optical flow image OF Ｌ OF Ｒ The optical flow image OF is displayed on the display unit 210 and projected as display light L in the left-right direction of the vehicle CR via the projection optical system 220, thus eliminating the need for a large display panel and allowing the optical flow image OF to be projected in front of the vehicle CR. Ｌ OF Ｒ This allows for visual confirmation without changing the driver's gaze. Therefore, it can contribute to safer driving.

[0102] The in-vehicle display system of this embodiment is an in-vehicle display system 1000 that allows viewers (including the driver and passengers) to view the display light L emitted from a projector 200 including a display unit 210 housed approximately in the center of the vehicle CR's dashboard 50, by reflecting the light L off a projection surface 60 (left projection surface 61, right projection surface 62) set in front of the vehicle CR via a projection optical system 220. The in-vehicle display system 1000 is connected to a vehicle monitoring device 400 that monitors the behavior of the vehicle CR via an input / output interface (e.g., CAN 500), estimates the motion state of the vehicle CR from the behavior of the vehicle CR acquired from the vehicle monitoring device 400, and displays an optical flow image OF that moves from approximately the center of the display unit 210 toward the left and right directions perpendicular to the direction of travel of the vehicle CR. Ｌ OF Ｒ Generates the optical flow image OF Ｌ OF Ｒ The vehicle-mounted projection device 100 performs a process of displaying on a display unit 210 and projecting it onto a projection surface 60 as display light L via a projection optical system 220.

[0103] According to the in-vehicle display system 1000 of this embodiment, the in-vehicle projection device 100 displays an optical flow image OF that moves from approximately the center of the display unit 210 toward the left and right directions perpendicular to the direction of travel of the vehicle CR, in accordance with the behavior of the vehicle CR acquired from the vehicle monitoring device 400. Ｌ OF Ｒ Generates the optical flow image OF Ｌ OF Ｒ The optical flow image OF is displayed on the display unit 210 and projected as display light L via the projection optical system 220 in a left-right direction perpendicular to the direction of travel of the vehicle CR. Therefore, a large display panel is not used, and the optical flow image OF is projected in front of the vehicle CR. Ｌ OF Ｒ This provides an in-vehicle display system 1000 that allows users to view the information without changing their line of sight.

[0104] The present invention is not limited to the exemplary embodiments described above, and those skilled in the art will be able to easily modify the exemplary embodiments described above to the extent included in the claims.

[0105] 50...Dashboard, 60...Projection surface, 61...Left projection surface, 62...Right projection surface, 80...Steering wheel, 100...In-vehicle projection device, 200...Projector, 210...Display unit, 220...Projection optical system, 300...Display control device, 311...Processing unit, 311a...Vehicle motion estimation unit, 311b...Flow image generation unit, 311c...Warning image generation unit, 311d...Non-flow image generation unit, 311e...Display speed adjustment unit, 31 1f...Image correction unit, 311g...Image synthesis unit, 312...RAM, 313...Storage unit, 314...Input / output interface (I / O), 400...Vehicle monitoring device, 401...Steering control ECU, 402...Steering angle sensor, 403...Driving control ECU, 404...Gyro sensor, 405...Accelerometer, 406...Wheel speed sensor, 500...CAN, 1000...In-vehicle display system, WS...Windshield, L...Display light, OF Ｌ OF Ｒ ...Optical flow image (flow image)

Claims

1. An in-vehicle projection device that allows a viewer to see display light emitted from a projector housed approximately in the center of the vehicle's dashboard, by projecting the display light onto a projection surface set in front of the vehicle via a projection optical system, the in-vehicle projection device comprising: a processing unit that generates an optical flow image moving from approximately the center of a display unit included in the projector toward the left and right directions perpendicular to the direction of travel of the vehicle, displays the generated optical flow image on the display unit, and projects the display light onto the projection surface via the projection optical system.

2. The in-vehicle projection device according to claim 1, wherein the projection optical system includes a reflective member consisting of a mirror or prism having a reflective surface that reflects the display light to each region set in the left and right directions, and one or more lenses that focus the optical flow image onto the projection surface, and the processing unit controls the display of the generated optical flow image on the display and the projection of the optical flow image reflected by the reflective surface and focused onto the projection surface.

3. The in-vehicle projection apparatus according to claim 1 or 2, wherein the processing unit generates an optical flow image representing the vehicle motion of the vehicle, including translational and rotational motion of the vehicle estimated based on information on the vehicle speed and yaw rate input from an external source.

4. The in-vehicle projection device according to claim 2, wherein the processing unit emits the optical flow image obliquely onto the reflective surface of the projection optical system, thereby controlling the apparent movement speed of the optical flow image moving from approximately the center of the vehicle toward each of the left and right directions so that it is perceived by the viewer as it moves toward each of the left and right directions.

5. The in-vehicle projection apparatus according to claim 4, wherein the processing unit acquires information regarding the vehicle speed from an external source, and if a speed difference occurs between the acquired vehicle speed and the apparent moving speed of the optical flow image, it performs control to adjust in a direction that eliminates the speed difference.

6. The in-vehicle projection device according to claim 4, wherein when the processing unit displays the optical flow image and the non-optical flow image together on the display unit, it controls the apparent movement speed of the non-optical flow image to decelerate as it moves from the approximate center of the vehicle toward the left and right directions, so that the apparent movement speed of the optical flow image and the apparent movement speed of the non-optical flow image are perceived by the viewer to be the same.

7. The in-vehicle projection device according to claim 4, wherein the processing unit, when displaying the optical flow image and the non-optical flow image combined on the display unit, performs image correction to narrow the distortion of the non-optical flow image, which appears to expand in the vertical and horizontal directions of the projection surface as it moves from approximately the center of the vehicle toward the left and right directions, by the amount of expansion.

8. The in-vehicle projection device according to claim 4, wherein the processing unit, when displaying the optical flow image and the warning image combined on the display unit, performs image correction to narrow the distortion of the warning image, which appears to spread in the vertical and horizontal directions of the projection surface as it moves from approximately the center of the vehicle toward the left and right directions, by the amount by which the warning image appears to be displayed in a fixed position.

9. The in-vehicle projection device according to claim 1, wherein when the projector displays a warning image at a fixed position on the display unit, the projector corrects the warning image projected onto the projection surface according to the display position of the warning image.

10. A control method for an in-vehicle projection device having a projector including a display unit housed approximately in the center of the vehicle's dashboard, and a processing unit that causes a viewer to see the display light emitted from the projector by reflecting the display light onto a projection surface set in front of the vehicle via a projection optical system, the control method comprising: the step of the processing unit generating an optical flow image that moves from approximately the center of the display unit toward the left and right directions perpendicular to the direction of travel of the vehicle; and the step of the processing unit displaying the generated optical flow image on the display unit and projecting it onto the projection surface as the display light via the projection optical system.

11. A program for an in-vehicle projection device having a projector including a display unit housed approximately in the center of the vehicle's dashboard, and a processing unit that projects display light emitted from the projector onto a projection surface set in front of the vehicle via a projection optical system, thereby making the display light visible to a viewer, the program causing a processor in the processing unit to perform the steps of: generating an optical flow image that moves from approximately the center of the display unit toward the left and right directions perpendicular to the direction of travel of the vehicle; and displaying the generated optical flow image on the display unit and projecting it onto the projection surface as the display light via the projection optical system.

12. An in-vehicle display system that allows a viewer to see display light emitted from a projector including a display unit housed approximately in the center of the vehicle's dashboard, by reflecting the display light off a projection surface set in front of the vehicle via a projection optical system, the system comprising: a vehicle monitoring device that monitors the behavior of the vehicle; and an in-vehicle projection device connected to the vehicle monitoring device via an input / output interface, which estimates the motion state of the vehicle from the behavior of the vehicle acquired from the vehicle monitoring device, generates an optical flow image that moves from approximately the center of the vehicle's display unit toward the left and right directions perpendicular to the direction of travel of the vehicle, displays the generated optical flow image on the display unit, and projects the display light onto the projection surface via the projection optical system.

Images