Head-up display, image correction method, and image correction program

Abstract

To provide a head-up display, an image correction method, and a program, which reduce the sense of incongruity felt by a driver due to the deviation of a virtual image during vehicle vibration with a simple configuration.SOLUTION: A head-up display is a head-up display mounted on a movable body. The head-up display includes: an image acquisition section which acquires an image including a diagram; a vibration information acquisition section which acquires vibration information indicating vibration applied to the movable body; an image correction section which performs vignetting correction for vignetting the contour of the diagram within the image on the image and forms a correction image on the basis of the vibration information; and a display section which displays the correction image as a virtual image in a space.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image correction device, and more particularly to a head-up display mounted in a vehicle, and an image correction method and program therefor. [Background technology]

[0002] 2. Description of the Related Art In recent years, technologies such as head-up displays have become known that assist drivers in moving objects such as automobiles by superimposing a virtual digital image on a real image in the driver's field of vision.

[0003] For example, Patent Document 1 discloses a head-up display device that illuminates a display light representing a display image displayed by a display device onto the windshield of the vehicle, allowing the user to view a virtual image of the display image superimposed on the actual view in front of the vehicle. Patent Document 1 also discloses a technology that, when the vehicle vibrates, identifies the vertical component of the displacement of the vehicle caused by the vibration from vibration information, reads out a display position correction amount, and moves the display position of an image drawn based on the display position correction amount in the vertical direction opposite to the direction in which the vehicle is displaced. This makes it possible to reduce the discomfort of the display given to the user due to the displacement of the virtual image when the vehicle vibrates. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-13590 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, when a correction is made to move the display position of the virtual image in response to the vibration of the vehicle, which changes from time to time, the load on the semiconductor device that performs the correction process increases, which may result in a decrease in processing speed. In addition, in order to prevent the decrease in processing speed, a large-scale semiconductor device must be installed, which requires an increase in the size of the device and measures to prevent heat generation from the device.

[0006] The present invention has been made in consideration of the above points, and aims to provide a head-up display, an image correction method, and an image correction program that use a simple configuration and processing to reduce the discomfort felt by the driver due to the displacement of a virtual image when the vehicle vibrates. [Means for solving the problem]

[0007] The head-up display of the present invention is a head-up display mounted on a moving body, and is characterized by including an image acquisition unit that acquires an image including a figure, a vibration information acquisition unit that acquires vibration information indicating vibrations applied to the moving body, an image correction unit that applies blurring correction to the image based on the vibration information to blur the contours of the figure in the image to generate a corrected image, and a display unit that displays the corrected image as a virtual image in space.

[0008] In addition, the image correction method of the present invention is an image correction method for a head-up display mounted on a moving body and including a display unit that displays a corrected image as a virtual image in space, and is characterized by including an image acquisition step of acquiring an image including a figure, a vibration information acquisition step of acquiring vibration information indicating vibrations applied to the moving body, and an image correction step of applying a blur correction to the image based on the vibration information to blur the contours of the figure in the image, thereby generating the corrected image.

[0009] In addition, the image correction program of the present invention is characterized in that it causes a computer mounted on a head-up display that is mounted on a moving body and includes a display unit that displays a corrected image as a virtual image in space to execute an image acquisition step of acquiring an image including a figure, a vibration information acquisition step of acquiring vibration information indicating vibrations applied to the moving body, and an image correction step of applying a blur correction to the image to blur the contours of the figure in the image based on the vibration information, thereby generating the corrected image. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram showing a side view of a front seat portion of a passenger compartment of an automobile as a vehicle equipped with a HUD according to a first embodiment of the present invention. [Diagram 2] 1 is a block diagram showing a configuration of a HUD according to a first embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram showing an example of a figure before correction processing of the HUD according to the first embodiment of the present invention is performed. [Figure 4] 4 is a graph showing the intensity distribution of luminance and chroma before and after correction processing on the line AA of the diagram in FIG. 3. [Diagram 5] 4A to 4C are diagrams illustrating an example of how a figure appears when displayed as a virtual image before and after a correction process is performed on the HUD according to the first embodiment of the present invention. [Figure 6] 3 is a diagram showing vibrations detected by a vibration sensor of the HUD according to the first embodiment of the present application, and the vibration directions when the vibrations are separated into horizontal vibrations and vertical vibrations. FIG. [Figure 7] FIG. 4 is a diagram showing a virtual image display routine as an example of an operation routine of the HUD according to the first embodiment of the present invention. [Figure 8] 4 is a graph showing the intensity distribution of luminance and saturation before and after correction processing is performed on the line AA of the figure of FIG. 3 of the HUD according to the second embodiment. [Figure 9] 4 is a graph showing the intensity distribution of luminance and saturation before and after correction processing is performed on the line AA of the figure of FIG. 3 of the HUD according to the second embodiment. [Figure 10]FIG. 11 is a diagram illustrating a virtual image display routine as an example of an operation routine of a HUD according to a second embodiment of the present invention. [Figure 11] 1A to 1C are diagrams illustrating examples of display of virtual images projected by a HUD according to the present invention. [Figure 12] 12A and 12B are diagrams showing the intensity distributions of luminance and saturation in the vertical direction of each of the virtual images showing the white line marks shown in FIG. 11. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following description of each embodiment and the accompanying drawings, substantially the same or equivalent parts are designated by the same reference numerals. EXAMPLES

[0012] Hereinafter, a head-up display (hereinafter, referred to as HUD) 10 according to Example 1 of the present application as a first embodiment will be described with reference to the accompanying drawings. In Example 1, the case where the HUD 10 is mounted on an automobile M will be described as an example.

[0013] Fig. 1 is a diagram showing a side view of a front seat portion of a passenger compartment of an automobile M as a vehicle equipped with a HUD 10 according to a first embodiment of the present invention. Fig. 1 shows, as an example of an installation, a case in which the HUD 10 is installed in an instrument panel portion of the automobile M. In other words, the HUD 10 is a head-up display installed in the automobile M as a moving object.

[0014] The HUD 10 includes a light-projecting unit 20 as a display unit. The light-projecting unit 20 is communicably connected to the HUD 10, and can project display light LM onto the windshield WS based on the control of the HUD 10, for example, to allow the driver D to view the image as a virtual image V in the target space FS. The light-projecting unit 20 may be a light projection method such as Digital Light Processing (DLP) using a MEMS (Micro Electro Mechanical Systems) mirror or laser scanning. That is, the light-projecting unit 20 may be any unit that projects an image to be displayed as a virtual image V onto the windshield WS as display light LM and allows the driver D to view the virtual image V by the display light LM reflected by the windshield WS.

[0015] The light projecting unit 20 does not have to be configured to project the display light LM onto the windshield WS, and may be, for example, a display device such as a liquid crystal display or an organic EL display capable of displaying images.

[0016] The vehicle-mounted camera 30 is, for example, a camera capable of capturing an image of the front of the automobile M (target space FS). The vehicle-mounted camera 30 is, for example, provided on a rearview mirror or on the upper end of the windshield WS or on a ceiling portion near the upper end. The vehicle-mounted camera 30 is communicably connected to the HUD 10 and is capable of transmitting a signal of a captured image to the HUD 10. The HUD 10 may be disposed at a position capable of capturing an image of at least the space in which the virtual image V is generated, that is, the front of the automobile M. The vehicle-mounted camera 30 may be, for example, provided on the front grille portion or each of the side mirrors of the automobile M and oriented so as to capture the target space FS.

[0017] The GPS receiver 40 is a device that receives signals (GPS signals) from GPS (Global Positioning System) satellites. The GPS receiver 40 is disposed, for example, on a dashboard. The GPS receiver 40 may be disposed anywhere as long as it can receive GPS signals. The GPS receiver 40 is communicably connected to the HUD 10 and can transmit the received GPS signals to the HUD 10. The HUD 10 also has a navigation function that searches for a route to a set destination and provides route instructions in conjunction with the GPS receiver 40.

[0018] For example, the HUD 10 detects information such as white lines and median strips indicating the lane in which the automobile M is currently traveling, information about other vehicles traveling ahead, and information about pedestrians walking ahead of the automobile M, from an image of the target space FS acquired by the in-vehicle camera 30. In addition, the HUD 10 displays a virtual image V indicating such information superimposed on a real image of the target space FS visually recognized by the driver D, and allows the driver D to visually recognize the virtual image V.

[0019] The image displayed by the HUD 10 as the virtual image V is, for example, an image including figures indicating a white line mark that emphasizes a white line, a vehicle mark that emphasizes another vehicle ahead, a distance between the vehicle and the other vehicle ahead, and a pedestrian mark that emphasizes a pedestrian walking in front of the automobile M. The image displayed by the HUD 10 as the virtual image V is, for example, an image including a guide indicated by a navigation function, that is, a route indication arrow mark that indicates going straight, turning right or left, etc. at an intersection in the travel direction, or figures indicating the distance to the destination, the estimated arrival time, etc.

[0020] The vibration sensor 50 as a vibration information acquisition unit is a sensor that detects vibration information while the automobile M is being driven. The vibration sensor 50 includes, for example, an acceleration sensor or a gyro sensor, and detects vibration information including the vibration direction, vibration amount, and vibration frequency of the automobile M while the automobile M is being driven. The vibration information of the automobile M includes, for example, vertical vibration information including the amplitude and frequency of vibration in the vertical direction, and horizontal vibration information including the amplitude and frequency of vibration in the horizontal direction. Note that the vertical direction here is, for example, the vertical direction that is a direction along the direction of gravity. Also, the horizontal direction is, for example, a direction along a direction perpendicular to the vertical direction.

[0021] The vibration sensor 50 is communicably connected to the HUD 10 and can transmit the detected vibration information to the HUD 10. The vibration sensor 50 may be disposed inside the housing of the HUD 10. The cause of the blurring of the virtual image V displayed by the HUD 10 is that the vibration of the automobile M causes the light projecting unit 20 to vibrate, and the projection position of the display light LM on the windshield WS to be blurred, causing the virtual image V to blur. By disposing the vibration sensor 50 near the light projecting unit 20, the vibration sensor 50 can acquire vibration information similar to the vibration applied to the light projecting unit 20. In other words, the HUD 10 includes the vibration sensor 50 as a vibration information acquisition unit that acquires vibration information indicating the vibration applied to the automobile M. The vibration information also includes horizontal vibration information related to the horizontal vibration applied to the automobile M and vertical vibration information related to the vertical vibration.

[0022] The HUD 10 performs a correction, described later, on the image displayed as the virtual image V in response to the vibration information received from the vibration sensor 50. As a result, even if the real image in front of the driver D and the virtual image V are blurred due to the vibration of the automobile M, the HUD 10 can visually reduce the blur and reduce the discomfort felt by the driver D.

[0023] 2 is a block diagram showing a configuration of a HUD 10 according to a first embodiment of the present invention. The HUD 10 is a device in which a storage unit 103, a control unit 105, an input unit 107, and an output unit 109 cooperate with each other via a system bus 101. In addition, an MCU (Micro Controller Unit) 70 is connected to the control unit 105 of the HUD 10.

[0024] The MCU 70 is a control unit that controls the entire HUD 10. The contents of control by the MCU 70 include, for example, starting and stopping the operation of the HUD 10, and the operation timing of the processing performed by the control unit 105 of the HUD 10.

[0025] The storage unit 103 is configured with, for example, a hard disk device, a solid state drive (SSD), a flash memory, etc., and stores various programs such as an operating system and software for terminals. The storage unit 103 also stores images including figures to be displayed as the virtual image V and map information used for the navigation function. The various programs, images, and map information may be acquired, for example, from another server device or the like via a network, or may be recorded on a recording medium and read via various drive devices. That is, the various programs (including a program for executing processing in the HUD 10 described later) stored in the storage unit 103 can be transmitted via a network, and can also be recorded on a computer-readable recording medium and transferred.

[0026] The control unit 105 is configured with a central processing unit (CPU) 105A, a read only memory (ROM) 105B, a random access memory (RAM) 105C, etc., and functions as a computer. The CPU 105A reads out and executes various programs stored in the ROM 105B and the storage unit 103 to realize various functions.

[0027] The input unit 107 is an interface unit that communicatively connects the HUD 10 to the vehicle-mounted camera 30, the GPS receiver 40, and the vibration sensor 50. The HUD 10 can receive a forward image of the automobile M from the vehicle-mounted camera 30, receive current position information of the automobile M from the GPS receiver 40, and receive vibration information of the automobile M from the vibration sensor 50 through the input unit 107. In this embodiment 1, a case where the image captured by the vehicle-mounted camera 30 is directly input to the HUD 10 will be described. In this case, the CPU 105A of the control unit 105 generates image data from information of the image signal transmitted from the vehicle-mounted camera 30, and sequentially writes or reads data blocks including the image data to the RAM 105C. However, the HUD 10 may be provided with a graphic generation unit between the vehicle-mounted camera 30 and the input unit 107 that generates image data from information of the image signal transmitted from the vehicle-mounted camera 30. In this case, the HUD 10 sequentially receives image data from the graphics generation unit, writes data blocks including the image data to the RAM 105C, and sequentially reads out the written data blocks. This makes it possible to reduce the load on the control unit 105 of the HUD 10.

[0028] The output unit 109 is communicatively connected to the light-projecting unit 20, and is capable of displaying a virtual image V in the target space FS by transmitting a signal of an image to be displayed as a virtual image V to the light-projecting unit 20 and projecting the display light LM.

[0029] Furthermore, the control unit 105 reads out from the storage unit 103 an image including figures such as the white line mark, other vehicle mark, distance between the vehicle and another vehicle, pedestrian mark, or route indication arrow mark to be displayed as the virtual image V. At this time, the control unit 105 functions as an image acquisition unit.

[0030] Furthermore, the control unit 105 generates a correction image from the read image, which the light projecting unit 20 displays as a virtual image V, and sends a signal of the correction image to the light projecting unit 20 via the output unit 109. The light projecting unit 20 projects the received signal of the correction image onto the windshield WS as display light LM, so that the driver D can visually recognize the virtual image V.

[0031] When generating a corrected image, the control unit 105 performs correction processing on the figures in the image read from the storage unit 103 based on the vibration information received from the vibration sensor 50 to generate the corrected image. Specifically, the control unit 105 generates a corrected image in which correction is performed so as to blur the contours of the figures included in the image according to the amount of vibration in the vertical and horizontal directions from the vibration information received from the vibration sensor 50. At this time, the control unit 105 functions as an image correction unit of the HUD 10.

[0032] Next, the correction process of the HUD 10 according to the first embodiment of the present invention will be described with reference to FIGS.

[0033] Fig. 3 is a diagram showing an example of a figure IM before correction processing of the HUD 10 according to the first embodiment of the present invention. In this embodiment, a case where correction is performed on a rectangular figure IM as shown in Fig. 3 will be described. Also, in this embodiment, a case where correction is performed on vibration in the vertical direction (direction along line AA) in Fig. 3 will be described.

[0034] Fig. 4 is a graph showing the intensity distribution of luminance and chroma before and after correction processing on line AA of figure IM in Fig. 3. The vertical axis of Fig. 4 corresponds to line AA, and the horizontal axis represents the intensity of luminance and chroma when displayed as virtual image V. The left side of Fig. 4 shows the intensity distribution of luminance and chroma in figure IM before correction processing, and the right side shows the intensity distribution of luminance and chroma in the corrected image after correction processing.

[0035] The luminance distribution on line AA of figure IM before correction processing is displayed with uniform luminance within the display area defined by the contour EG, for example, as shown on the left side of Fig. 4. That is, in figure IM before correction processing, the luminance and chroma intensities within the display area are uniformly 100%, and the luminance and chroma intensities outside the display area are uniformly 0%.

[0036] When the figure IM with a clear contour EG is projected as the virtual image V in this way, the contour EG is displayed intermittently with a blur (vibration) according to the frequency of the vibration of the automobile M. Therefore, when the figure IM before the correction process is displayed as the virtual image V, the virtual image V may, for example, have a thick line width or flicker around the contour EG. This causes the driver D to recognize the virtual image V as having a different shape from the figure IM, causing the driver D to feel uncomfortable.

[0037] In other words, if a virtual image V that causes discomfort to the driver D is displayed, the visibility of the virtual image V will be reduced, and the driver D may have to look closely to see the virtual image V or may be unable to recognize the information indicated by the virtual image V, which may affect driving safety.

[0038] In the HUD 10 according to the first embodiment of the present application, as shown on the right side of Fig. 4, a blur correction area CA is formed on the periphery of the contour EG within the display area defined by the contour EG. The control unit 105 performs correction so as to gradually decrease the brightness and saturation of the figure IM from a correction start point CS within the display area of ​​the blur correction area CA to a correction end point CE outside the display area. That is, the control unit 105 performs processing on the figure IM to blur the contour EG by gradually decreasing the brightness and saturation around the contour EG, thereby generating a corrected image.

[0039] The luminance and chroma to be reduced at this time are, for example, in the YCbCr color space. Specifically, the control unit 105 reduces the luminance (Y) in the blur correction area CA from the correction start point CS to the correction end point CE in a stepwise manner from the value held by the figure IM to Y=0. In addition, the control unit 105 performs a correction process so as to linearly move the chroma (CbCr) from the value held by the figure IM to the origin (Cb=0, Cr=0) in the CbCr color plane. That is, in the virtual image V displayed by the corrected figure IM, the brightness gradually decreases and the color gradually becomes lighter around the contour EG of the virtual image V from the display area toward the outside.

[0040] Also, the control unit 105 sets the position of the correction start point CS, for example, at a predetermined ratio inward from the contour EG of the figure IM according to the amplitude of vibration of the width from the contour EG along the correction direction of the figure. That is, the correction start point CS is set so as to be located more inward from the figure IM as the amplitude of the vibration increases. The position of the correction start point CS can be set arbitrarily. For example, the control unit 105 may set the position of the correction start point CS at a distance corresponding to 1 / 2 the amplitude of the vibration inward from the contour EG. Also, the control unit 105 may set the position of the correction start point CS at a distance corresponding to a predetermined ratio of the amplitude of the vibration inward from the contour EG. The position of the correction start point CS may be set so that the visibility of the virtual image V after the correction process is good.

[0041] In addition, the control unit 105 sets the position of the correction end point CE outside the contour EG by a predetermined ratio according to the amplitude of the vibration of the width along the correction direction of the figure IM, for example, similar to the position of the correction start point CS. That is, the correction end point CE is set so as to be located inside the figure IM as the amplitude of the vibration increases. The position of the correction end point CE can be set arbitrarily. For example, the control unit 105 may set it at a position at a distance equivalent to 1 / 2 the vibration amplitude outside the contour EG of the figure IM, or the control unit 105 may set the position of the correction start point CS at a distance of a predetermined ratio of the vibration amplitude outside the contour EG. In addition, for example, the control unit 105 may adjust the position of the correction end point CE so that it does not overlap with the display area of ​​another figure according to the shape and display position of the figure, and it is sufficient that it is set so that the visibility of the virtual image V after the correction process is good.

[0042] In addition, in this embodiment 1, the brightness and saturation are reduced in two stages in the blur correction area CA, a section in which the brightness and saturation are reduced steeply and a section in which the brightness and saturation are reduced gently. This makes it possible to improve the visibility of the virtual image V after the correction process. Note that the method of reducing the brightness and saturation is not limited to this. The brightness and saturation may be reduced linearly from the correction start point CS to the correction end point CE, or may be reduced in multiple stages. The brightness and saturation may also be reduced in a curved manner from the correction start point CS to the correction end point CE.

[0043] In addition, in this embodiment, the case where the range of the correction area CA is changed according to the amplitude of the vibration has been described. However, the range of the correction area CA may be changed according to the frequency of the vibration. For example, when the frequency of the vibration is low and the control unit 105 determines that the frequency does not affect the visibility of the driver D, the range of the correction area CA may be reduced or correction may not be performed.

[0044] FIG. 5 is a diagram showing an example of how the figure IM appears when it is displayed as a virtual image V before and after correction processing. FIG. 5 explains a case where correction is performed on vibration in a direction along the line AA, as in FIG. 3. The left side of FIG. 5 shows how the figure IM appears when it is displayed as a virtual image V before correction processing is performed, and the right side of FIG. 5 shows how the figure IM appears when it is displayed as a virtual image V after correction processing is performed. Note that, on the right side of FIG. 5, the area showing a predetermined appearance is shown as being displayed in a stepped manner, but in reality, the brightness and saturation gradually decrease from the correction start point CS to the correction end point CE, and it is displayed as if it has a gradation.

[0045] When the figure IM before the correction process is applied as shown on the left side of Fig. 5 and displayed as a virtual image V, as described above, the outline EG is clearly blurred in the area outside the outline EG due to the vibration of the automobile M. Furthermore, this blur is displayed as flickering according to the frequency of the vibration of the automobile M. Therefore, when the figure IM before the correction process is applied and displayed as a virtual image V, the visibility of the figure IM is deteriorated, as shown on the left side of Fig. 5.

[0046] As described above, the figure IM after the correction process on the right side of FIG. 5 is corrected by gradually lowering the brightness and saturation around the contour EG to blur it. Therefore, when the figure IM after the correction process is displayed as a virtual image V, it is displayed with a gradation from the correction start point CS of the blur correction area CA inside the contour EG to the outside of the contour EG even when the automobile M is vibrating. That is, as shown on the left side of FIG. 5, the contour EG is not clearly blurred, and the outer edge of the blurred contour EG is displayed in a manner that is difficult to distinguish. Therefore, as shown on the right side of FIG. 5, the visibility of the figure IM is very high. That is, it is possible to reduce the blur of the virtual image V caused by the vibration of the automobile M.

[0047] In the first embodiment, a correction is applied to an image including a figure to be displayed as a virtual image V, so that the brightness and saturation around the contour of the figure are gradually reduced and the contour is blurred in accordance with the vibration of the automobile M. This makes it possible to reduce the decrease in visibility caused by the blurring of the virtual image V due to the vibration of the automobile M, and to reduce the discomfort felt by the driver D.

[0048] 3 to 5 in the first embodiment, an example in which correction is applied only to vibrations in the vertical direction has been described. However, as described above, in practice, a similar correction process is also applied to vibrations in the horizontal direction.

[0049] FIG. 6 is a diagram showing vibration RM detected by the vibration sensor 50 and the vibration directions when the vibration RM is separated into horizontal vibration RX and vertical vibration RY.

[0050] The acceleration sensor and gyro sensor included in the vibration sensor 50 can separate the vibration RM having a tilt into a horizontal vibration RX and a vertical vibration RY as shown in FIG. 6. The control unit 105 performs a horizontal blur correction process on the horizontal vibration RX according to the vibration amount, and performs a vertical blur correction process on the vertical vibration RY according to the vibration amount, for the contour of the figure of the image read from the storage unit 103. That is, it is possible to improve the visibility of the figure of the virtual image V even with horizontal vibration. As a result, it is possible to reduce the decrease in visibility due to the blur of the virtual image V, and reduce the discomfort felt by the driver D, even when vibration occurs when the automobile M is tilted when turning a curve or vibration occurs due to a crosswind.

[0051] In the first embodiment, the correction for blurring the contour EG of the figure IM is described as being performed by gradually decreasing the brightness and saturation when displayed as a virtual image V. However, the correction for blurring the contour EG may be performed by changing only the brightness. That is, the contour EG of the figure IM can be blurred by gradually decreasing the brightness in the blur correction area CA. This can reduce the decrease in visibility caused by the blurring of the virtual image V due to the vibration of the automobile M, and can reduce the discomfort felt by the driver D.

[0052] An example of an operation routine up to displaying the virtual image V of the HUD 10 will be described below.

[0053] 7 is a diagram showing a virtual image display routine R1 as an example of an operation routine of the HUD 10 according to the first embodiment of the present invention. The virtual image display routine R1 is repeatedly executed when the power of the HUD 10 is turned on, for example.

[0054] When the virtual image display routine R1 is started, the control unit 105 acquires, from the vehicle-mounted camera 30, an image in front of the automobile M as a target space FS in which the virtual image V captured by the vehicle-mounted camera 30 is to be displayed (step S101).

[0055] Next, the control unit 105 distinguishes white lines indicating lanes on the road, other vehicles ahead of the automobile M, or pedestrians walking ahead of the automobile M from the acquired forward image of the automobile M, and determines whether there is information to be displayed as a virtual image V in the target space FS of the automobile M (step S102). At this time, route instructions indicating a right turn, a left turn, or going straight by the navigation function operating in cooperation with the above-mentioned GPS receiver 40 are also determined as information to be displayed as a virtual image V in the target space FS of the automobile M.

[0056] In step S102, if the control unit 105 determines that there is no information to be displayed in the target space FS of the automobile M (step S102: No), the control unit 105 ends the virtual image display routine R1 and then executes it again. That is, the control unit 105 repeatedly executes the process of step S101 to constantly acquire a forward image of the automobile M from the vehicle-mounted camera 30 and determine whether there is information to be displayed as a virtual image V in the target space FS of the automobile M.

[0057] Furthermore, in step S102, if the control unit 105 determines that there is information to be displayed in the target space FS of the automobile M (step S102: Yes), as an image acquisition step, the control unit 105 reads out an image including a figure IM corresponding to the information from the storage unit 103 (step S103). In other words, the image correction method of the HUD 10 includes an image acquisition step of acquiring an image including a figure.

[0058] Next, as a vibration information acquiring step, the control unit 105 acquires vibration information of the automobile M from the vibration sensor 50 (step S104). In other words, the image correction method for the HUD 10 includes a vibration information acquiring step of acquiring vibration information indicating vibrations applied to the moving object.

[0059] Next, the control unit 105 calculates the amount of correction to be applied to the figure IM included in the image based on the vibration information acquired from the vibration sensor 50 (step S105). In this embodiment, the amount of correction indicates the range amount of the blur correction area CA in the blur correction. In this step, the control unit 105 separates the vibration RM, which is the vibration information acquired from the vibration sensor 50, into a horizontal vibration RX and a vertical vibration RY, as shown in Fig. 6. In addition, the control unit 105 calculates each of the horizontal and vertical blur correction areas CA for the contour EG of the figure IM from the amplitude of each of the separated horizontal vibration RX and vertical vibration RY.

[0060] Next, the control unit 105 generates a corrected image in which the contour EG of the figure IM is subjected to correction for blurring the contour EG (step S106). The correction applied to the figure IM is to blur the contour EG by gradually decreasing the brightness and saturation toward the outside of the figure IM within the range of each of the blur correction areas CA in the horizontal and vertical directions calculated in step S105.

[0061] That is, steps S105 and S106 function as image correction steps in the virtual image display routine R1. In other words, the image correction method of the HUD 10 includes an image correction step of performing blur correction on the image based on the vibration information to blur the contour of the figure IM in the image to generate a corrected image.

[0062] Next, the control unit 105 supplies the generated corrected image to the light projecting unit 20 via the output unit 109, and causes the light projecting unit 20 to output display light LM of the corrected image, thereby displaying the virtual image V (step S107).

[0063] By repeatedly performing the processing of the virtual image display routine R1 described above, the HUD 10 can display a virtual image V with good visibility according to the vibration direction and vibration amount of the automobile M. In other words, the virtual image display routine R1 is an image correction method for a head-up display that is mounted on a moving body and includes a light projecting unit 20 that displays a corrected image as a virtual image V in space.

[0064] According to the first embodiment, the HUD 10 can reduce the decrease in visibility caused by the blurring of the virtual image V due to the vibration of the automobile M by using a relatively simple configuration and correction processing as described above, thereby making it possible to reduce the discomfort felt by the driver D.

[0065] As described above, the virtual image display routine R1 is recorded in the storage unit 103 as a software program of the HUD 10, and the CPU 105A, ROM 105B, and RAM 105C as the control unit 105 read and execute it. In addition, the program can be recorded in a computer-readable recording medium and transferred. In other words, the virtual image display routine R1 is an image correction program that causes a computer mounted on the HUD 10, which is mounted on the automobile M as a moving body and includes the light projecting unit 20 as a display unit that displays a corrected image as a virtual image V in space, to execute an image acquisition step of acquiring an image including a figure IM, a vibration information acquisition step of acquiring vibration information indicating vibration applied to the automobile M, and an image correction step of applying blur correction to the image to blur the contour of the figure IM in the image based on the vibration information to generate a corrected image. In addition, the virtual image display routine R1 is a computer-readable recording medium that stores the image correction program of the virtual image display routine R1. EXAMPLES

[0066] In the first embodiment, a correction process is performed to gradually reduce the brightness and saturation around the contour EG of the figure IM in order to improve the visibility of the virtual image V. However, the correction process method for the figure IM is not limited to this. As a second embodiment, for example, the brightness and saturation of a part of the display area may be increased to enhance the visibility of the virtual image V.

[0067] FIG. 8 is a graph showing the intensity distribution of luminance and chroma before and after the correction process is performed on the line AA of the figure IM in FIG. 3 of the HUD 10 according to Example 2 as the second embodiment. In FIG. 8, as in FIG. 4, the vertical axis corresponds to the line AA, and the horizontal axis represents the intensity of luminance and chroma when displayed as a virtual image V. The left side of FIG. 8 shows the intensity distribution of luminance and chroma in the figure IM before the correction process is performed, and the right side shows the intensity distribution of luminance and chroma in the corrected image after the correction process is performed. Note that the intensity distribution of luminance and chroma in the figure IM before the correction process shown on the left side of FIG. 8 is the same as that shown on the left side of FIG. 4, and therefore a description thereof will be omitted. The configuration of the HUD 10 in this Example 2 is the same as that of Example 1. In this Example 2, the processing content of the correction process is different from that of Example 1, and therefore a description of the configuration of the HUD 10 will be omitted.

[0068] In the intensity distribution of brightness and chroma in the corrected image after the correction process on the right side of FIG. 8, the range of the blur correction area CA and the correction method within the blur correction area CA are the same as those in the first embodiment. In this embodiment, the control unit 105 forms an emphasis correction area EA1 for the section between the blur correction areas CA in each of the contours EG. The control unit 105 applies emphasis correction to the emphasis correction area EA1, which gradually increases the brightness and chroma from the outer edge of the emphasis correction area EA1 toward the center of the figure IM according to the amplitude of the vibration. In other words, the control unit 105 as the image correction unit further applies emphasis correction to the image IM, which emphasizes a part of the figure IM, based on the vibration information. Also, the control unit 105 as the image correction unit performs emphasis correction by gradually increasing the brightness and chroma of the figure IM toward the center of the figure, based on the vibration information.

[0069] In this second embodiment, as in the first embodiment, an example in which correction is applied only to vibrations in the vertical direction will be described. However, as described above, in practice, a similar correction process is also applied to vibrations in the horizontal direction. In other words, the control unit 105 as an image correction unit performs emphasis correction by changing the luminance and saturation of the figure IM in the horizontal direction based on the horizontal vibration information, and changing the luminance and saturation of the figure IM in the vertical direction based on the vertical vibration information.

[0070] The luminance and chroma increased at this time are, for example, in the YCbCr color space. Specifically, the control unit 105 increases the luminance (Y) stepwise from the value stored in the figure IM from the correction start point CS to the correction end point CE in the blur correction area CA. In addition, the control unit 105 performs a correction process on the chroma (CbCr) in the CbCr color plane on a straight line passing through the value stored in the figure IM and the origin (Cb=0, Cr=0) so that the chroma moves away from the origin. That is, in the virtual image V displayed by the corrected figure IM, the brightness gradually increases and the color gradually becomes darker around the contour EG of the virtual image V from the display area toward the outside.

[0071] Furthermore, the control unit 105 increases the maximum values ​​of brightness and saturation in the emphasis correction area EA1 by, for example, a predetermined ratio according to the amplitude of the vibration from the brightness and saturation originally held by the figure IM. That is, the greater the amplitude of the vibration, the stronger the emphasis within the emphasis correction area EA1 of the figure IM. The maximum values ​​of brightness and saturation in the emphasis correction area EA1 can be set arbitrarily. For example, the maximum values ​​of brightness and saturation in the emphasis correction area EA1 may be increased by a predetermined value from the YCbCr values ​​held by the figure IM according to the amplitude of the vibration. The maximum values ​​of brightness and saturation in the emphasis correction area EA1 may be set so that the visibility of the virtual image V after the correction process is good.

[0072] 9, only a portion including the center of the figure IM displayed as the virtual image V may be emphasized. For example, a range of a predetermined percentage of the width of the figure IM along the correction direction from the center of the figure IM according to the amplitude of vibration may be set as an emphasis correction area EA2, and correction may be performed to increase the brightness and saturation stepwise from the outer edge of the emphasis correction area EA2 toward the center of the figure IM.

[0073] According to the second embodiment, the HUD 10 forms an emphasis correction area EA1 or EA2 in a section between blur correction areas CA, which are the central parts of the display area, and increases the luminance and saturation in the emphasis area. The HUD 10 relatively lowers the luminance and saturation of the correction start point CS (i.e., the periphery of the emphasis correction area EA1 or EA2) in the blur correction area CA to emphasize a part of the center of the figure IM, thereby making it possible to further improve the visibility of the figure IM. In addition, by performing a correction to emphasize a part of the center of the figure IM, it is possible to further improve the visibility of, for example, a figure including a relatively thin straight line such as a white line emphasis mark, a number or a letter indicating the distance between another vehicle ahead and a right turn or a left turn during navigation, and the like.

[0074] 10 is a diagram showing a virtual image display routine R2 as an example of an operation routine of the HUD 10 according to the embodiment 2. The virtual image display routine R2 is repeatedly executed when the power of the HUD 10 is turned on, for example.

[0075] It should be noted that steps S201 to S204 and step S207 of the virtual image display routine R2 are similar to steps S101 to S104 and step S107 of the virtual image display routine R1 shown in FIG. 7, and therefore a description thereof will be omitted.

[0076] In step S205, the control unit 105 calculates the amount of correction to be applied to the figure IM included in the image based on the vibration information acquired from the vibration sensor 50. In this embodiment, the amount of correction indicates the range amount of the blur correction area CA and the emphasis correction area EA in the blur correction, and the maximum values ​​of brightness and saturation in the emphasis correction area EA. The control unit 105 calculates each of the blur correction areas CA in the horizontal and vertical directions for the contour EG of the figure IM from the amplitudes of the horizontal vibration RX and the vertical vibration RY. The control unit 105 also calculates each of the emphasis correction areas EA1 or EA2 in the horizontal and vertical directions for the center of the figure IM from the amplitudes of the horizontal vibration RX and the vertical vibration RY.

[0077] Next, in step S206, the control unit 105 generates a corrected image in which the contour EG of the figure IM is blurred and the center of the figure IM is emphasized. The control unit 105 blurs the contour EG by gradually decreasing the brightness and saturation toward the outside of the figure IM within the blur correction area CA in the horizontal and vertical directions calculated in step S205. The control unit 105 also emphasizes the center of the figure IM by gradually increasing the brightness and saturation toward the center of the figure IM within the emphasis correction area EA1 or EA2 based on the amplitude of each of the horizontal vibration RX and the vertical vibration RY.

[0078] As in the first embodiment, steps S205 and S206 function as image correction steps in the virtual image display routine R2.

[0079] By executing the processing of the virtual image display routine R2 including the above steps S205 and S206, the control unit 105 is able to blur the contour EG of the figure IM, emphasize the central part, and display a virtual image V with improved visibility in the target space FS.

[0080] The correction amounts of the blur correction and emphasis correction described above may be changed according to the distance in the target space FS onto which the virtual image V is projected.

[0081] FIG. 11 is a diagram showing a display example of a virtual image projected by the HUD 10 of the present invention.

[0082] The virtual images VA, VB, and VC are virtual images showing, for example, a white line mark that emphasizes the position of the white line in the lane in which the automobile M is traveling.

[0083] In the target space FS where the virtual images VA, VB, and VC are projected, the distances (the distances from the driver D to the real image where the virtual images VA, VB, and VC are superimposed) are in the order of VA, VB, and VC being far. At this time, the control unit 105 performs correction with a correction amount that increases as the distance at which each virtual image is projected is farther, so that the virtual image projected at a far position can have high visibility.

[0084] FIG. 12 is a diagram showing the intensity distributions of the luminance and chroma in the vertical direction of each of the virtual images VA, VB, and VC showing the white line marks shown in FIG. 11.

[0085] As described above, the virtual images VA, VB, and VC are displayed at positions farther from the driver D in the order of VA, VB, and VC in the target space FS.

[0086] The correction amount of each of the virtual images VA, VB, and VC changes the strength of the correction in the relationship of VA < VB < VC according to the distance from the driver D. Specifically, the virtual image VA displayed at a short distance is subjected to a correction process with a small correction amount, and the virtual image VC displayed at a long distance is subjected to a correction process with a large correction amount.

[0087] That is, as shown in FIG. 12, the control unit 105 increases the ranges of the blurring correction regions CAA, CAB, and CAC of each of the virtual images VA, VB, and VC in the order of CAA < CAB < CAC, and blurs the contour EG stronger for the virtual image displayed at a farther distance.

[0088] In addition, in each of the enhancement correction regions EAA, EAB, and EAC of the virtual images VA, VB, and VC, the control unit 105 corrects the maximum values of the luminance and chroma, which are the enhancement amounts, so that they increase in the order of VA < VB < VC, and the virtual image displayed at a farther distance is enhanced more strongly.

[0089] Note that the correction for changing the strength of the correction according to the display distance of the virtual image in the target space FS may be either only the blur correction or only the enhancement correction. It is sufficient to perform correction so as to improve the visibility of the virtual image to be displayed according to the display distance of the virtual image in the target space FS.

[0090] As described above, according to the HUD 10, according to the distance of the virtual image to be displayed in the target space FS, the blur correction region for the figure of the image to be displayed as the virtual image and the maximum values of the luminance and chroma of the enhancement correction are changed. As a result, the visibility of the virtual image displayed far away can be improved, and the discomfort felt by the driver due to the shift of the virtual image during the vibration of the vehicle M can be reduced.

[0091] Note that the correction process in this specification is not limited to the blur correction and the enhancement correction as described above. The luminance and chroma in the vertical direction with respect to the figure of the image may be measured, and the front and back of the figure portion with a large change amount of the luminance and chroma may be corrected so that the change amount becomes small. Also, the formation regions of the vertical figures may be made different in the horizontal direction with respect to the figure of the image.

[0092] Note that the above-described correction process may be performed by the light projecting unit 20. The case where the above-described correction process is performed by the light projecting unit 20 and the light projecting unit 20 is a liquid crystal display will be described. For example, in the control unit 105, an image including the figure IM read from the storage unit 103 and a correction region signal indicating the blur correction region CA and the enhancement correction region EA in the horizontal and vertical directions based on the vibration information are supplied to the drive driver of the light projecting unit 20.

[0093] The drive driver of the light projecting unit 20 extracts and holds the display data of the cells for displaying the image from the supplied image.

[0094] The driver of the light projection unit 20 detects cells with large changes in luminance and chroma in the horizontal and vertical directions from the extracted display data. The driver of the light projection unit 20 corrects the display data so as to gradually decrease the luminance and chroma in cells within the range of the blur correction area CA, starting from the cell with large changes, based on the supplied correction area signal. The driver of the light projection unit 20 corrects the display data so as to gradually increase the luminance and chroma in cells within the range of the emphasis correction area EA, starting from the cell with large changes, based on the supplied correction area signal.

[0095] The driver of the light-projecting unit 20 generates a drive voltage corresponding to the display data for each cell of the liquid crystal panel of the light-projecting unit 20 from the display data corrected in this manner, and supplies it to the liquid crystal panel of the light-projecting unit 20, thereby making it possible to project display light LM that has been blur-corrected and highlighted in the horizontal and vertical directions from the liquid crystal panel of the light-projecting unit 20.

[0096] It should be noted that even if the light projecting unit 20 is a self-emitting panel type such as an organic EL display, or a light projection type such as DLP or laser scanning using a MEMS mirror, the correction can be performed on the driver as described above. In other words, the correction for reducing the decrease in visibility of the virtual image V of the HUD 10 of the present application caused by the vibration of the automobile M or the decrease in the discomfort felt by the driver D can be applied to HUDs using any display type. [Explanation of symbols]

[0097] 10 HUD 20 Light projector 30 In-vehicle camera 40 GPS receiver 50 Vibration Sensor 103 Storage section 105 Control section 107 Input section 109 Output section

Claims

1. A head-up display mounted on a moving object, an image acquisition unit for acquiring an image including a figure; a vibration information acquisition unit that acquires vibration information indicating vibrations applied to the moving object; an image correction unit that performs blur correction on the image based on the vibration information to blur the contour of the figure in the image, thereby generating a corrected image; A display unit that displays the corrected image as a virtual image in space, the vibration information includes horizontal vibration information related to horizontal vibrations applied to the moving object and vertical vibration information related to vertical vibrations applied to the moving object, The head-up display is characterized in that the image correction unit performs blur correction along the horizontal direction of the contour of the figure in the image based on the horizontal vibration information, and performs blur correction along the vertical direction of the contour of the figure in the image based on the vertical vibration information.

2. The image correction unit includes: Based on the horizontal vibration information, the brightness and saturation of the figure are gradually decreased toward the outside of the figure to perform blur correction along the horizontal direction of the outline of the figure; and The head-up display according to claim 1, characterized in that the blur correction is performed by gradually reducing the brightness and saturation of the figure toward the outside of the figure based on the vertical vibration information, thereby performing blur correction along the vertical direction of the contour of the figure.

3. 3 . The head-up display according to claim 1 , wherein the image correction unit performs the blur correction by changing a range in which the contour of the figure is blurred, based on the vibration information. 4 .

4. The head-up display according to any one of claims 1 to 3, characterized in that the image correction unit performs the blur correction by changing the range in which the contour of the figure is blurred based on the position of the virtual image to be displayed in the depth direction of the space.

5. 4. The head-up display according to claim 1, wherein the image correction unit further applies emphasis correction to the image to emphasize a part of the figure based on the vibration information.

6. The head-up display according to claim 5, characterized in that the image correction unit performs the emphasis correction by gradually increasing the brightness and saturation of the figure toward the center of the figure based on the vibration information to emphasize a part of the figure.

7. The head-up display according to any one of claims 5 and 6, characterized in that the image correction unit performs the emphasis correction by changing the maximum values ​​of luminance and saturation of the figure of the emphasis correction applied to the image based on the position of the virtual image to be displayed in the depth direction of the space.

8. A head-up display as described in Claim 6 or 7, characterized in that the image correction unit performs the emphasis correction by changing the brightness and saturation of the figure in the horizontal direction based on the horizontal vibration information, and changing the brightness and saturation of the figure in the vertical direction based on the vertical vibration information.

9. A method for correcting an image of a head-up display including a display unit mounted on a moving object and displaying a corrected image as a virtual image in space, comprising: An image acquisition step of acquiring an image including a figure; a vibration information acquiring step of acquiring vibration information indicating vibration applied to the moving object; an image correction step of performing blur correction on the image based on the vibration information to blur the contour of the figure in the image to generate the corrected image; the vibration information includes horizontal vibration information related to horizontal vibrations applied to the moving object and vertical vibration information related to vertical vibrations applied to the moving object, The image correction method is characterized in that, in the image correction step, blur correction is performed along the horizontal direction of the contour of the figure in the image based on the horizontal vibration information, and blur correction is performed along the vertical direction of the contour of the figure in the image based on the vertical vibration information.

10. A computer is mounted on a head-up display including a display unit that is mounted on a moving object and displays a corrected image as a virtual image in space. An image acquisition step of acquiring an image including a figure; a vibration information acquiring step of acquiring vibration information indicating vibration applied to the moving object; an image correction step of performing blur correction on the image based on the vibration information to blur the contour of the figure in the image to generate the corrected image; the vibration information includes horizontal vibration information related to horizontal vibrations applied to the moving object and vertical vibration information related to vertical vibrations applied to the moving object, The image correction program is characterized in that, in the image correction step, blur correction is performed along the horizontal direction of the contour of the figure in the image based on the horizontal vibration information, and blur correction is performed along the vertical direction of the contour of the figure in the image based on the vertical vibration information.

11. A computer-readable recording medium storing the image correction program according to claim 10.

Images