Head-up display virtual image evaluation method and virtual image evaluation device

Abstract

The present invention precisely evaluates whether correction of distortion in a virtual image is performed in a desired manner. Disclosed is a method for evaluating a virtual image of a head-up display, the method comprising capturing an image of a projection object when an image for inspection displayed by an image output device is projected onto the projection object, and evaluating distortion of a virtual image from the projected image. This method includes: a step for setting a plurality of target positions corresponding to a plurality of evaluation points for distortion evaluation; a step for arranging a plurality of reference pixel groups, each constituted from a plurality of contiguous pixels, in accordance with the plurality of target positions in the image for inspection; and a step for setting a pixel value of each pixel constituting each reference pixel group in the image for inspection so that a center of gravity G based on the respective pixel values of the plurality of reference pixel groups matches or approaches a target position.

Description

[Technical Field]

[0001] The present disclosure relates to a method and device for evaluating a virtual image of a head-up display. [Background technology]

[0002] A known technique for adjusting the virtual image of a head-up display involves photographing a virtual image of a test image and a normal reference point, and adjusting the relative position between the test image and the magnifying glass based on the positional deviation between the virtual image of the test image obtained from the photographed image and the photographed reference point. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-218346 Summary of the Invention [Problem to be solved by the invention]

[0004] In head-up displays that display a virtual image on a vehicle windshield, the virtual image tends to be distorted due to the curved shape of the windshield. Known techniques for canceling out such distortion include correction using a free-form concave mirror that cancels out the distortion, correction by transforming the displayed image to cancel out the distortion (image correction by image warping), and a combination of these methods.

[0005] However, there is no known technology for accurately evaluating whether the distortion of such a virtual image has been corrected in a desired manner.

[0006] Therefore, an object of the present disclosure is to enable accurate evaluation of whether correction of distortion of a virtual image has been achieved in a desired manner. [Means for solving the problem]

[0007] According to one aspect, there is provided a method for evaluating a virtual image of a head-up display, the method comprising: an image output device that outputs an image; and an optical system that enlarges the image and projects it onto a windshield of a vehicle; and projecting the enlarged image onto the windshield to display a virtual image of the image so as to be visible from inside the vehicle, the method comprising: an inspection image display step of displaying an inspection image on the image output device; an imaging step of imaging a projection object corresponding to the windshield when the test image is projected onto the projection object; an evaluation step of evaluating distortion of the virtual image from the captured image of the projection object obtained in the imaging step, The inspection image display step includes: setting a plurality of target positions corresponding to a plurality of evaluation points for distortion evaluation; setting a plurality of reference pixel groups, each of which is composed of a plurality of consecutive pixels, at positions spaced apart from one another in the test image and corresponding to the plurality of target positions; a step of setting the pixel values ​​of each pixel constituting the reference pixel group so that, for each reference pixel group, the center of gravity G expressed by the formula 1 below coincides with or is close to the corresponding target position, where, in the xy coordinate system of the test image, Bxi is the pixel row value for the y-direction pixel row where x coordinate = xi, Byi is the pixel row value for the x-direction pixel row where y coordinate = yi, and nx is the number of y-direction pixel rows constituting a corresponding reference pixel group, and ny is the number of x-direction pixel rows. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to accurately evaluate whether the distortion of a virtual image has been corrected in a desired manner. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram of a head-up display according to one embodiment. [Figure 2] FIG. 2 is a cross-sectional view of a head-up display. [Figure 3] 1 is a diagram illustrating an inspection system including a virtual image evaluation device according to an embodiment of the present invention. [Figure 4] FIG. 10 is an explanatory diagram of an example of an inspection image. [Figure 4A] FIG. 2 is an explanatory diagram of an example of pixels that constitute one reference pixel group. [Figure 5] FIG. 10 is an explanatory diagram of an example of a camera image. [Figure 6] FIG. 10 is an explanatory diagram of a target position and its resolution for one reference pixel group. [Figure 7] FIG. 2 is an explanatory diagram of a two-dimensional xy coordinate system of an inspection image. [Figure 8] 8 is an explanatory diagram of the movement of the center of gravity when different luminance values ​​are assigned to some pixels in the reference pixel group shown in FIG. 7. FIG. [Figure 9] FIG. 10 is an explanatory diagram of a method for selecting a table for setting the luminance value of each pixel. [Figure 10A] FIG. 1 is an explanatory diagram (part 1) of a setting table. [Figure 10B] FIG. 10 is an explanatory diagram (part 2) of the setting table. [Figure 10C] FIG. 10 is an explanatory diagram (part 3) of the setting table. [Figure 10D] FIG. 10 is an explanatory diagram (part 4) of the setting table. [Figure 11A] FIG. 5 is an explanatory diagram (part 5) of the setting table. [Figure 11B] FIG. 6 is an explanatory diagram (part 6) of the setting table. [Figure 11C] FIG. 7 is an explanatory diagram (part 7) of the setting table. [Figure 11D] FIG. 8 is an explanatory diagram of the setting table. [Figure 12] FIG. 9 is an explanatory diagram of the setting table. [Figure 13] FIG. 2 is an explanatory diagram of the imaging characteristics of a camera. [Figure 14] FIG. 10 is an explanatory diagram of coefficients used in brightness correction based on the imaging characteristics of a camera. [Figure 15] FIG. 2 is a functional diagram illustrating functions of the virtual image evaluation device. [Figure 16] 1 is a schematic flowchart showing the flow of a virtual image evaluation method according to the present embodiment. [Figure 17] FIG. 10 is a diagram showing a setting table for table number “51.” [Figure 18] FIG. 10 is a diagram illustrating an example of setting pixel values ​​related to one reference pixel group. DETAILED DESCRIPTION OF THE INVENTION

[0010] Each embodiment will be described in detail below with reference to the accompanying drawings.

[0011] In the following, first, a head-up display to which the virtual image evaluation method and virtual image evaluation device of this embodiment can be applied will be outlined, and then the virtual image evaluation method and virtual image evaluation device of this embodiment will be described.

[0012] [Configuration of Head-Up Display] Fig. 1 is a schematic diagram of a head-up display 12 according to this embodiment. Fig. 2 is a schematic cross-sectional view of the head-up display 12 according to this embodiment.

[0013] As shown in Fig. 1, the head-up display 12 is disposed inside an instrument panel 11 of a vehicle 10. The head-up display 12 reflects projected display light L by a windshield 13 of the vehicle 10 in the direction of a driver (user) 14 of the vehicle 10, thereby displaying a virtual image V. That is, the head-up display 12 emits (projects) display light L emitted from a liquid crystal display 20 (described later) onto the windshield 13 (projection member), and allows the driver 14 to view a display image (virtual image) V obtained by this emission. This allows the driver 14 to view the virtual image V superimposed on the scenery.

[0014] As shown in FIG. 2, the head-up display 12 includes a liquid crystal display 20, a plane mirror 30, a concave mirror 40, and a housing 50.

[0015] The liquid crystal display 20 includes a light source 21, for example, a light emitting diode, mounted on a wiring board R, and a liquid crystal display element 22, for example, a TFT (Thin Film Transistor) type, located in front of (directly above) the light source 21 so as to transmit illumination light from the light source 21 to form display light L. The light source 21 is disposed behind the liquid crystal display element 22, and the liquid crystal display element 22 displays vehicle information using light emitted from the light source 21.

[0016] The plane mirror 30 has a cold mirror 31 and a mounting member 32 for mounting and fixing the cold mirror 31. The cold mirror 31 may include, for example, a substantially rectangular glass substrate 31a and a first reflective layer 31b formed on one surface of the glass substrate 31a (the surface facing the reflective member 41 of the concave mirror 40). The cold mirror 31 may be disposed in an inclined state at a position such that it reflects the display light L emitted by the liquid crystal display 20 toward the concave mirror 40 (the reflective member 41).

[0017] As shown in FIG. 2, the concave mirror 40 includes a reflecting member 41 that reflects the display light L from the cold mirror 31 (that is, the liquid crystal display element 22), and a holding member 42 that holds the reflecting member 41.

[0018] The reflective member 41 may be formed by depositing a second reflective layer 41a on a substrate having a concave surface. The substrate having a concave surface may be integrated with the holding member 42. The reflective member 41 may be disposed in an inclined state at a position where the second reflective layer 41a faces the cold mirror 31 and the light-transmitting cover 54 and can be seen from the light-transmitting cover 54.

[0019] 1 and 2 show a head-up display 12 with a specific configuration, the virtual image evaluation method and virtual image evaluation device of this embodiment described below can be applied to head-up displays with other configurations. Specifically, this embodiment can be applied to any head-up display that includes an image output device (e.g., liquid crystal display 20) that outputs an image related to display light L, and an optical system (e.g., concave mirror 40) that enlarges the image and projects it onto the windshield 13 of the vehicle, and that projects the enlarged image onto the windshield 13 to display a display image (virtual image) V of the image so that it can be seen from inside the vehicle.

[0020] [Virtual Image Evaluation Method and Virtual Image Evaluation Apparatus] Next, a virtual image evaluation method and virtual image evaluation apparatus according to this embodiment will be described with reference to FIG. 3 and subsequent figures.

[0021] FIG. 3 is a diagram schematically illustrating an inspection system 1 including a virtual image evaluation device 100 according to this embodiment. FIG. 3 also illustrates a head-up display 12 to be inspected. Hereinafter, the head-up display 12 refers to the head-up display 12 to be inspected unless otherwise specified. FIG. 4 is an explanatory diagram of an example of an inspection image m1. FIG. 5 is an explanatory diagram of an example of a camera image m4.

[0022] In addition to the virtual image evaluation device 100, the inspection system 1 includes a master windshield 130 having the same configuration as the actual windshield 13, such as curvature, and a camera 140.

[0023] The virtual image evaluation device 100 is a processing device that performs various processes and may be configured by, for example, a computer. The virtual image evaluation device 100 is connected to the head-up display 12 and the camera 140 so as to be able to communicate with each other.

[0024] The master windshield 130 is the object onto which the test image m1 is projected from the head-up display 12, and may be positioned with respect to the head-up display 12 in the same relative positional relationship as the actual windshield 13.

[0025] During inspection, camera 140 captures an inspection image m1 projected onto master windshield 130. Camera 140 may be positioned relative to master windshield 130 so that it has an optical axis that corresponds to the line of sight of driver 14. Hereinafter, the image of master windshield 130 obtained by imaging with camera 140 will also be referred to as a "camera image m4."

[0026] During testing, the virtual image evaluation device 100 outputs a test image m1 to the liquid crystal display 20 of the head-up display 12. The test image m1 has a plurality of reference pixel groups m2, each consisting of a plurality of consecutive pixels, spaced apart from one another, as shown in FIG. 4, for example. FIG. 4A is an explanatory diagram of an example of pixels that make up one reference pixel group m2. In this embodiment, as an example, one reference pixel group m2 consists of 7 × 7 pixels, as shown in FIG. 4A.

[0027] In the test image m1, the multiple reference pixel groups m2 are arranged spaced apart from one another. The position of each reference pixel group m2 is set so that the center of gravity G, which will be described later, coincides with or is close to a target position in the coordinate system of the test image m1.

[0028] A target position is set for each of the plurality of reference pixel groups m2. The target positions are determined in advance so that distortion of the projected image of the test image m1 captured by the camera 140 is reduced or eliminated in a desired manner (see FIG. 5). That is, the target positions are determined based on various design values ​​of the head-up display 12 so that correction of distortion of the virtual image is achieved in a desired manner. In this case, the target positions may be determined with a resolution of, for example, 0.05 pixels.

[0029] During inspection, the virtual image evaluation device 100 acquires a camera image when the inspection image m1 (see FIG. 4) output to the liquid crystal display 20 as described above is projected onto the master windshield 130. Then, based on the acquired camera image, the virtual image evaluation device 100 evaluates whether the distortion of the virtual image has been corrected in a desired manner.

[0030] As shown in FIG. 5, the camera image includes pixel groups (hereinafter also referred to as "inspection pixel groups m5") corresponding to the plurality of reference pixel groups m2. When the distortion of the virtual image is corrected in a desired manner, the positions of the inspection pixel group m5 coincide with the positions of the evaluation points for distortion evaluation. In other words, target positions for the plurality of reference pixel groups m2 in the inspection image m1 are determined so that the positions of the inspection pixel group m5 coincide with the positions of the evaluation points for distortion evaluation. Therefore, when the positions of the inspection pixel group m5 are calculated based on the camera image and the calculated positions of the inspection pixel group m5 coincide with the positions of the evaluation points for distortion evaluation, it can be determined that the distortion of the virtual image has been corrected in a desired manner. Note that the allowable error in determining whether the calculated positions of the inspection pixel group m5 coincide with the positions of the evaluation points for distortion evaluation may be adjusted as appropriate.

[0031] In this evaluation method, the smaller the resolution of the target position, the more accurate the evaluation can be. In other words, the smaller the resolution of the target position, the more accurately it can be evaluated whether the distortion of the virtual image has been corrected in the desired manner.

[0032] Therefore, in this embodiment, the test image m1 is generated and output in a manner that allows the realizable resolution of the target position to be substantially reduced. A method for generating such a test image m1 will be described in detail below.

[0033] FIG. 6 is an explanatory diagram of the target position and its resolution for one reference pixel group m2. As described above, the target position for one reference pixel group m2 is determined so that the distortion of the virtual image is corrected in the desired manner, whereas the physical position of one reference pixel group m2 in the test image m1 is determined in pixel units. Therefore, if the target position does not coincide with the center position P0 of one pixel, such as position P1 shown in FIG. 6, the reference pixel group m2 cannot be set with the target position as its center. In this case, the deviation between the center of the reference pixel group m2 and the target position is at most one-half of a pixel. In other words, the achievable target position resolution is 0.5 pixels in each of the x and y directions.

[0034] Here, if the brightness values ​​of all the pixels that make up one reference pixel group m2 are set to the same value, the position of one inspection pixel group m5 corresponding to that reference pixel group m2 is determined according to the center position of that reference pixel group m2.

[0035] On the other hand, if some of the brightness values ​​of the pixels that make up one reference pixel group m2 are changed, the position of one inspection pixel group m5 corresponding to that reference pixel group m2 will shift from the position corresponding to the center position of that reference pixel group m2.

[0036] In this embodiment, such characteristics are utilized to define the center of gravity G, which can be expressed by the following formula, and the brightness values ​​of each pixel that makes up one reference pixel group m2 are adjusted so that the center of gravity G of the reference pixel group m2 coincides with or is close to the target position.

[0037]

number

[0038] FIG. 7 is an explanatory diagram of a pixel-based xy coordinate system of a test image. In the pixel-based xy coordinate system shown in FIG. 7, the x coordinate of the ith pixel in the x direction is x=i, and the y coordinate of the ith pixel in the y direction is y=i. Therefore, the x coordinate of the center pixel (pixel marked "C") of the reference pixel group m2 shown in FIG. 7 is x=8, and the y coordinate is y=5. In the following, the position of each pixel in the test image will be expressed in the pixel-based coordinate system.

[0039] 7, the x-direction pixel column for y coordinate = 2 is made up of seven pixels lined up in the x direction from (5,2) to (11,2), the x-direction pixel column for y coordinate = 3 is made up of seven pixels lined up in the x direction from (5,3) to (11,3), and so on. Also, the y-direction pixel column for x coordinate = 5 is made up of seven pixels lined up in the y direction from (5,2) to (5,8), the y-direction pixel column for x coordinate = 6 is made up of seven pixels lined up in the y direction from (6,2) to (6,8), and so on.

[0040] Therefore, when applying equation 1 to the reference pixel group m2 shown in FIG. 7, xi varies from xi=5 to 11, and nx=7, and yi varies from yi=2 to 8, and ny=7.

[0041] In the example shown in FIG. 7, the color information of each pixel in the reference pixel group m2 is the same: R=255, G=255, B=255. To calculate the center of gravity G, any one piece of color information from RGB (here, the color information of G) may be used. In the following, the value of the color information of G will also be simply referred to as the "luminance value 1" to avoid confusion with the center of gravity G. In a grayscale image, each RGB color has the same value, and the value of the color information of G is substantially the same as the luminance value.

[0042] In this case, the center of gravity G is calculated using equation 1 as follows: The denominator, the total brightness value (ΣBxi, ΣByi) = 255*7*7 = 12495 The numerator, the total X-coordinate brightness value (Σ(xi×Bxi)) = ((255*5)+(255*6)+(255*7)+(255*8)+(255*9)+(255*10)+(255*11))*7 = 99960 The numerator, the total Y-coordinate brightness value (Σ(yi×Byi)) = ((255*2)+(255*3)+(255*4)+(255*5)+(255*6)+(255*7)+(255*8))*7 = 62475 Therefore, the center of gravity G can be calculated as follows: G = (99960 / 12495, 62475 / 12495) = (8,5). Hereinafter, the center of gravity G when the luminance values ​​of all pixels in the reference pixel group m2 are the same predetermined reference value (here, the pixel value with the maximum luminance, luminance value l=255) will also be referred to as the "nominal center of gravity."

[0043] As described above, it can be seen that the center of gravity G of the reference pixel group m2 corresponds to the center position of the reference pixel group m2 in terms of brightness values. The position of each inspection pixel group m5 in the camera image m4 is calculated based on brightness. Therefore, it can be seen that by adjusting the brightness values ​​of each pixel constituting one reference pixel group m2 so that the center of gravity G of that reference pixel group m2 coincides with or is close to the target position, it is possible to accurately evaluate whether the distortion of the virtual image has been corrected in the desired manner.

[0044] The reference pixel group m2 is a part of the inspection image m1, and the inspection pixel group m5 is similarly a part of the camera image m4. To accurately detect the position of such inspection pixel group m5, it is desirable to increase the luminance difference between the inspection pixel group m5 and its surrounding pixels. Therefore, in this embodiment, the luminance value l of pixels other than the reference pixel group m2 in the inspection image m1 (see background pixel m3 in FIG. 4 ) may be the minimum value (“0”). Furthermore, the luminance value l of each pixel of the reference pixel group m2 is basically set to a predetermined reference value. Here, the predetermined reference value is the pixel value with the maximum luminance (luminance value l=255). As will be described later, some pixels of the reference pixel group m2 (correction pixels px1 and px2, described later) may have a luminance value l that is smaller than the predetermined reference value.

[0045] FIG. 8 is an explanatory diagram of the movement of the center of gravity G, and is an explanatory diagram of the center of gravity G when different luminance values ​​1 are assigned to some pixels in the reference pixel group m2 shown in FIG.

[0046] 7, the brightness value l of each pixel in the reference pixel group m2 is the same, i.e., 255, so the center of gravity G is (8, 5), as described above. However, if it is desired to shift the center of gravity G toward the negative x-direction (see arrow R8), the brightness value l of the pixel on the positive x-direction relative to the center of gravity G before the shift can be reduced and / or the brightness value l of the pixel on the negative x-direction can be increased. For example, if the brightness value l of each pixel in the y-direction pixel column at x coordinate = 11 is changed from 255 to 0, the center of gravity G becomes (7.5, 5), and the center of gravity G can be shifted by 0.5 pixels toward the negative x-direction.

[0047] As can be seen from the above, by varying the brightness value l of each pixel in various ways, the minimum unit by which the center of gravity G can be shifted (i.e., the resolution of the target position that can be achieved) can be made smaller. On the other hand, since varying the brightness value l of each pixel in various ways increases the calculation load, it is desirable to appropriately set a change pattern (correction pattern) for the brightness value l of each pixel, taking these trade-offs into consideration.

[0048] Therefore, in this embodiment, the brightness value l of each pixel is set according to the corresponding target position for each reference pixel group m2, in such a way that the smallest unit by which the center of gravity G can be shifted (i.e., the resolution of the achievable target position) is 0.05 pixels.

[0049] FIG. 9 is an explanatory diagram of a method for selecting a table for setting the luminance value of each pixel.

[0050] 9, the first horizontal column from the top indicates the amount of deviation of the center of gravity G in the x direction, with "0" indicating no deviation from the nominal center of gravity and "-0.05" indicating a deviation of 0.05 pixels to the negative x side from the nominal center of gravity. As described above, the nominal center of gravity refers to the center of gravity G when the luminance values ​​of all pixels in the reference pixel group m2 are the same predetermined reference value (here, the pixel value with the maximum luminance, luminance value 1=255).

[0051] Also, in Figure 9, the first column on the left in the vertical direction represents the amount of deviation of the center of gravity G in the y direction, where "0" means no deviation from the nominal center of gravity and "-0.05" means a deviation of 0.05 pixels to the negative side of the y direction from the nominal center of gravity.

[0052] 9, the other columns are assigned table numbers from "0" to "440" that identify the corresponding setting tables. That is, in this case, a total of 441 setting tables are prepared.

[0053] Therefore, for example, to achieve a center of gravity G that is shifted by 0.05 pixels in the negative x direction and by 0.05 pixels in the negative y direction from the nominal center of gravity, it is sufficient to set the brightness value of each pixel using the setting table with table number "198."

[0054] A total of 441 setting tables can be created in advance based on the principle of movement of the center of gravity G explained in Fig. 8. Here, only some of the 441 setting tables will be explained instead of explaining all of them. Note that a person skilled in the art would be able to create the 441 setting tables in advance based on such a partial explanation.

[0055] Below, some of the 441 setting tables will be described with reference to Figures 10A to 12. Also, Figures 10A to 12 describe setting tables that can be applied to the reference pixel group m2 having a 7x7 pixel configuration, such as that shown in Figure 7 etc.

[0056] 10A is an explanatory diagram of setting tables such as table numbers "10" and "31," and is a diagram illustrating a reference pixel px0 and a correction pixel px1 in a reference pixel group m2. The reference pixel px0 is a pixel whose luminance value is set to a predetermined reference value (here, the pixel value of maximum luminance, luminance value 1=255), and the correction pixel px1 is a pixel whose luminance value is set to a value different from the predetermined reference value (in this embodiment, a value lower than the predetermined reference value, i.e., luminance value 1<255).

[0057] The setting table shown in FIG. 10A is a table that defines the luminance value l to be set for the correction pixel px1. For example, in the setting table with table number "10," a luminance value l=0 is set for each correction pixel px1 shown in FIG. 10A. Note that xg and yg shown in FIG. 10A represent the center of gravity G=(xg, yg) realized by the luminance value pattern defined in the setting table. However, xg and yg shown in FIG. 10A (and similarly in FIG. 10B and subsequent figures described later) are coordinate values ​​for the reference pixel group m2 with a nominal center of gravity of (7,7). This also applies to the following FIGS. 10B to 12. For a reference pixel group m2 whose nominal center of gravity is different from (7,7), the same can be applied by replacing the nominal center of gravity.

[0058] In the example shown in Fig. 10A, the correction pixel px1 is adjacent to the lower edge (outer edge) of the reference pixel px0, but if the correction pixel px1 is adjacent to another outer edge of the reference pixel px0 as shown in Fig. 10B to Fig. 10D, other setting tables can be created accordingly. For example, in the example shown in Fig. 10B, table numbers "241", "262", etc. can be created, and so on.

[0059] FIG. 11A is an explanatory diagram of setting tables such as table numbers "198" and "176," and illustrates a reference pixel px0, a correction pixel px1, and a correction pixel px2 in the reference pixel group m2. Like the correction pixel px1, the correction pixel px2 is a pixel whose luminance value is set to a value different from a predetermined reference value (in this embodiment, a value lower than the predetermined reference value, i.e., a luminance value 1<255). Unlike the correction pixel px1, which is arranged in the x direction, the correction pixel px2 is arranged in the y direction. In other words, it constitutes a y-direction pixel column. Note that the x-direction pixel column including the correction pixel px1 and the y-direction pixel column including the correction pixel px2 share a corner pixel.

[0060] The setting table shown in Fig. 11A is a table that specifies the luminance value l to be set for the correction pixel px1 and the luminance value l to be set for the correction pixel px2. For example, in the setting table with table number "198", the luminance value l = 222 is set for each correction pixel px1 shown in Fig. 11A, and the luminance value l = 222 is set for each correction pixel px2 shown in Fig. 11A.

[0061] In the example shown in Fig. 11A, the x-direction pixel row including correction pixel px1 and the y-direction pixel row including correction pixel px2 are adjacent to the bottom edge (outer edge) and right edge (outer edge) of reference pixel px0, but as shown in Fig. 11B to Fig. 11D, if the x-direction pixel row including correction pixel px1 and the y-direction pixel row including correction pixel px2 are adjacent to other outer edges of reference pixel px0, other setting tables can be created accordingly. For example, in the example shown in Fig. 11B, table numbers "200", "180", etc. can be created, and so on.

[0062] 10A to 11D correspond to the setting tables indicated by hatched areas R90 and R91 in FIG. 9, among the 441 setting tables shown in FIG. 9. The remaining setting tables can be created similarly by appropriately setting the luminance values ​​of the correction pixels px1 and px2. In this case, the luminance values ​​of the correction pixels px1 and px2 do not need to be the same and may be different from each other. When the luminance values ​​of the correction pixels px1 and px2 are to be different, in FIGS. 11A to 11D, one pixel at the corner where the x-direction pixel row including the correction pixel px1 and the y-direction pixel row including the correction pixel px2 overlap is set as the correction pixel px1, but patterns 1201 to 1204 (see FIG. 12) in which the pixel is set as the correction pixel px2 may be used.

[0063] Next, a brightness correction method based on the imaging characteristics of the camera 140 will be described with reference to FIGS.

[0064] As described above, in this embodiment, the brightness value is set based on the setting table, but in the camera image m4 of the camera 140, a brightness value consistent with the brightness value based on the setting table may not be obtained due to the imaging characteristics of the camera 140.

[0065] For example, Fig. 13 is an explanatory diagram of the imaging characteristics of camera 140, and shows a curve 1302 that indicates the imaging characteristics of camera 140 when the horizontal axis represents input luminance and the vertical axis represents output luminance. Fig. 13 also shows a curve 1300 that indicates the imaging characteristics where input luminance = output luminance.

[0066] 13, the imaging characteristics of camera 140 deviate from the imaging characteristics (curve 1300) where input luminance = output luminance. If this deviation is not compensated for, the camera image m4 of camera 140 will not have a luminance value that matches the luminance value based on the setting table.

[0067] Therefore, in this embodiment, preferably, the luminance value based on the setting table may be further corrected based on the imaging characteristics of camera 140. That is, the luminance value based on the setting table may be further corrected so as to compensate for the deviation of the imaging characteristics of camera 140 from the imaging characteristics (curve 1300) where input luminance = output luminance.

[0068] The correction method is arbitrary, and the correction may be realized so that the curve 1302 matches or approaches the curve 1300.

[0069] In this embodiment, as an example, correction is performed by linear interpolation so that the curve 1302 approaches the curve 1300. Specifically, the corrected brightness value is obtained based on the following interpolation formula: Corrected brightness value=y1+((y2-y1)*((brightness value before correction)-x1) / (x2-x1))) Equation (1) where y1, y2, x1, and x2 are coefficients for linear interpolation and are adapted based on the imaging characteristics of camera 140 so that curve 1302 approaches curve 1300. For the imaging characteristics (curve 1302) of camera 140 shown in FIG. 13, y1, y2, x1, and x2 as shown in FIG. 14 may be used. In FIG. 14, the values ​​of y1, y2, x1, and x2 are associated with sections 1 to 15, respectively. Sections 1 to 15 each represent a luminance section from x1 to x2 to which the pre-correction luminance value belongs. For example, if the pre-correction luminance value is "18," it belongs to section 4, and the corrected luminance value is derived based on the values ​​of y1, y2, x1, and x2 associated with section 4. If the pre-correction luminance value differs from the luminance reference value, the values ​​of y1, y2, x1, and x2 associated with the section corresponding to the closest luminance reference value may be used.

[0070] By performing such brightness correction based on the imaging characteristics of the camera 140, it is possible to accurately calculate the position of the inspection pixel group m5 corresponding to the center of gravity G of the reference pixel group m2 based on the camera image m4. As a result, it is possible to accurately and reliably evaluate whether the distortion of the virtual image has been corrected in the desired manner based on the camera image m4.

[0071] Next, the virtual image evaluation device 100 according to this embodiment and an example of its operation (virtual image evaluation method) will be described with reference to FIGS.

[0072] Fig. 15 is a functional diagram showing the functions of virtual image evaluation device 100. As shown in Fig. 15, virtual image evaluation device 100 includes an inspection image display control unit 150, an image acquisition unit 152, and an evaluation unit 154. Note that inspection image display control unit 150, image acquisition unit 152, and evaluation unit 154 can be realized by a CPU of a computer (not shown) constituting virtual image evaluation device 100 executing one or more programs in a storage device.

[0073] The inspection image display control unit 150 includes a target position setting unit 1501 , a reference pixel group setting unit 1502 , a pixel value setting unit 1503 , a pixel value correction unit 1504 , and an image signal output unit 1505 .

[0074] The operation of each part of the examination image display control unit 150 will be described with reference to FIG.

[0075] FIG. 16 is a schematic flowchart showing the flow of the virtual image evaluation method according to this embodiment.

[0076] This virtual image evaluation method first includes a step (step S150) of placing the head-up display 12 to be inspected in the inspection system 1 shown in FIG.

[0077] Next, the present virtual image evaluation method includes a test image display step (step S152) of displaying a test image on the liquid crystal display element 22 of the head-up display 12 to be inspected.

[0078] The inspection image display process (step S152) includes a target position setting process (step S1521), a reference pixel group setting process (step S1522), a pixel value setting process (step S1523), a pixel value correction process (an example of a correction process) (step S1524), and an output process (step S1525).

[0079] The test image display step (step S152) is realized by the virtual image evaluation device 100.

[0080] Specifically, in the target position setting step (step S1521), the target position setting unit 1501 sets a plurality of target positions corresponding to a plurality of evaluation points for distortion evaluation. For example, when obtaining the camera image m4 shown in FIG. 5, a plurality of target positions corresponding to 21 × 9 evaluation points are set. As described above, the target positions are determined based on various design values ​​so that the distortion of the virtual image is corrected in a desired manner. Note that the method of correcting the distortion of the virtual image may be, as described above, correction using a free-form concave mirror 40 that cancels out the distortion, correction by deformation of the displayed image that cancels out the distortion (image correction by image warping), or a combination of these.

[0081] In the target position setting step (step S1521), the target position setting unit 1501 may set the target position with a resolution of 0.05 pixels. For example, if the ideal position for one target position is (19.955, 10.622) in the x-y coordinate system, the target position may be set as follows. First, the difference (-0.045, -0.378) is calculated from (20, 11), which is obtained by rounding (19.955, 10.622) to the nearest tenth. This difference is doubled (-0.09, -0.756), and then rounded to the nearest tenth to obtain (-0.1, -0.8). Next, this value (-0.1, -0.8) is halved to obtain the center of gravity movement amount (-0.05, -0.4). In this case, the target position is (19.95, 10.6) which achieves a center of gravity movement amount of (-0.05, -0.4) relative to (20, 11).

[0082] In the reference pixel group setting step (step S1522), the reference pixel group setting unit 1502 arranges multiple reference pixel groups in the test image in association with each target position (e.g., 21 × 9 target positions) obtained in the target position setting step (step S1521). That is, the positions of multiple reference pixel groups in the test image are determined. In this case, if the ideal position for one target position is (19.955, 10.622) in the xy coordinate system, (19.955, 10.622) may be rounded to one decimal place to obtain (20, 11), which is the position of one reference pixel group associated with that one target position. In this case, the one reference pixel group is positioned so that its center pixel (see the pixel having the center position P0 in FIG. 6) is located at (20, 11).

[0083] In the pixel value setting step (step S1523), the pixel value setting unit 1503 adjusts the luminance value of each pixel constituting each reference pixel group m2 (for example, a 21 × 9 reference pixel group m2) based on the above-mentioned formula 1. Note that the luminance value of the background pixel m3 (see FIG. 4) is set to "0".

[0084] Specifically, the pixel value setting unit 1503 adjusts the luminance value of each pixel in each reference pixel group so that the center of gravity G, expressed by Equation 1, coincides with or approaches the target position. For example, in the example described above, if the central pixel of the reference pixel group is located at (20, 11) relative to the target position (19.95, 10.6), the pixel value setting unit 1503 adjusts the luminance value of each pixel in the reference pixel group to achieve a center of gravity shift of (-0.05, -0.4). In this case, table number "51" shown in FIG. 9, which corresponds to the center of gravity shift of (-0.05, -0.4), is used. Based on the setting table with table number "51," the pixel value setting unit 1503 sets the pixel value of the correction pixel (such as correction pixel px1) specified in the setting table as the pixel value of the corresponding pixel in the reference pixel group. An example of the setting table with table number "51" is shown in FIG. 17. In this case, the setting table with table number "51" is a setting table based on pattern 1201 shown in Fig. 12. Therefore, in this case, a luminance value l of "18" is set to the pixel in the reference pixel group that corresponds to correction pixel px1, and a luminance value l of "196" is set to the pixel in the reference pixel group that corresponds to correction pixel px2.

[0085] In the pixel value correction step (step S1524), the pixel value correction unit 1504 corrects each pixel value set in the pixel value setting step (step S1523) based on the imaging characteristics of the camera 140 of the inspection system 1. The correction method based on the imaging characteristics of the camera 140 is as described above with reference to FIGS. 13 and 14. For example, for a pixel to which the luminance value 1 of "18" is set based on the setting table of table number "51" shown in FIG. 17 in the pixel value setting step (step S1523), the corrected luminance value is derived based on the values ​​of y1, y2, x1, and x2 associated with section 4, as shown in FIG. 14. Similarly, for a pixel to which the luminance value 1 of "196" is set based on the setting table of table number "51" shown in FIG. 17 in the pixel value setting step (step S1523), the corrected luminance value is derived based on the values ​​of y1, y2, x1, and x2 associated with section 13, as shown in FIG. 14. In this case, the luminance value 1 of each pixel after correction is as shown in FIG.

[0086] In the output step (step S1525), the image signal output unit 1505 generates a test image m1 having each reference pixel group m2 at the position determined in the reference pixel group setting step (step S1522), and in which each pixel of each reference pixel group m2 has the luminance value set in the pixel value setting step (step S1523) or the pixel value correction step (step S1524). Specifically, the image signal output unit 1505 supplies an image signal for generating such test image m1 to the head-up display 12, and outputs the test image m1 via the head-up display 12. The test image m1 output by the head-up display 12 in this manner is projected onto the master windshield 130 of the inspection system 1.

[0087] Next, this virtual image evaluation method includes an imaging step (step S154) of imaging the master windshield 130 with the camera 140 when the inspection image is projected onto the master windshield 130. In the imaging step (step S154), the image acquisition unit 152 acquires a camera image m4 (see FIG. 5) via the camera 140.

[0088] Next, the present virtual image evaluation method includes an evaluation step (step S156) of evaluating distortion of the virtual image when display light L is projected from the head-up display 12 onto the windshield 13, based on the camera image m4.

[0089] In the evaluation step (step S156), the evaluation unit 154 calculates the position of each inspection pixel group m5 (e.g., an inspection pixel group m5 of 21 × 9 points) from the camera image m4, and evaluates the distortion of the virtual image by comparing each calculated position with each position of the evaluation points for distortion evaluation. That is, the evaluation unit 154 evaluates whether the distortion of the virtual image has been corrected in a desired manner based on the position of each inspection pixel group m5 (e.g., an inspection pixel group m5 of 21 × 9 points) from the camera image m4.

[0090] The position of each inspection pixel group m5 from the camera image m4 changes depending on the center of gravity G of the reference pixel group m2 to which the luminance value 1 of each pixel is set, as described above. Therefore, according to this embodiment, the distortion of the virtual image can be evaluated with high accuracy.

[0091] Although each embodiment has been described in detail above, it is not limited to the specific embodiment, and various modifications and changes are possible within the scope of the claims. It is also possible to combine all or a plurality of components of the above-described embodiments.

[0092] For example, in the above-described embodiment, the shape of one reference pixel group m2 was rectangular (more specifically, a 7x7 pixel square), but it may be any shape other than rectangular (e.g., circular, elliptical, or polygonal).

[0093] In the above-described embodiment, the correction pixels px1 and px2 are arranged on the outer edge of the reference pixel px0 (the outer edge of the pixel region consisting of the set of reference pixels px0) so that the number of correction pixels px1 and px2 in the reference pixel group m2 does not become excessive, but this is not limiting. For example, in addition to or instead of the correction pixels px1 and px2, correction pixels may be set inside the pixel region consisting of the set of reference pixels px0.

[0094] In the above-described embodiment, the luminance value 1 is used as the pixel value for evaluating distortion, but instead of or in addition to this, a characteristic value other than the luminance value (for example, a color value) may be used. When a color value is used as the characteristic value other than the luminance value, the color values ​​of each RGB color of the pixel value may be treated as the pixel value for evaluation or the correction pixel. [Explanation of symbols]

[0095] 1 Inspection system 12 Head-up display 13 Windshield 20 Liquid crystal display 21 Light source 22 Liquid crystal display element (an example of an image output device) 30 Plane mirror (an example of an optical system) 31 Cold mirror 31a Glass substrate 31b First reflective layer 32 Mounting member 40 Concave mirror (an example of an optical system) 41 Reflective member 41a Second reflective layer 42 Holding member 50 Housing 54 Light-transmitting cover 100 Virtual image evaluation device 130 Master windshield (an example of a projection object) 140 Camera (an example of an imaging means) 150 Inspection image display control unit 1501 Target position setting unit 1502 Reference pixel group setting unit 1503 Pixel value setting unit 1504 Pixel value correction unit 1505 Image signal output unit 152 Image acquisition unit 154 Evaluation unit m1 Inspection image m2 Reference pixel group m3 Background pixel m4 Camera image m5 Inspection pixel group

Claims

1. A method for evaluating a virtual image of a head-up display (12) comprising: an image output device (22) that outputs an image; and an optical system that enlarges the image and projects it onto a windshield (13) of a vehicle, wherein the enlarged image is projected onto the windshield to display a virtual image of the image so as to be visible from inside the vehicle, an inspection image display step of displaying an inspection image (m1) on the image output device; an imaging step of imaging a projection object (130) corresponding to the windshield when the test image is projected onto the projection object; an evaluation step of evaluating distortion of the virtual image from the captured image of the projection object obtained in the imaging step, The inspection image display step includes: setting a plurality of target positions corresponding to a plurality of evaluation points for distortion evaluation; setting a plurality of reference pixel groups, each of which is composed of a plurality of consecutive pixels, at positions spaced apart from one another in the test image and corresponding to the plurality of target positions; and setting pixel values ​​of each pixel constituting the reference pixel group so that, for each reference pixel group, a center of gravity G expressed by the following formula coincides with or is close to the corresponding target position, where Bxi is a pixel row value for a y-direction pixel row where x coordinate = xi, Byi is a pixel row value for an x-direction pixel row where y coordinate = yi, and nx is the number of y-direction pixel rows constituting a corresponding reference pixel group, and ny is the number of x-direction pixel rows: [Equation 1] A method for evaluating a virtual image of a head-up display.

2. The pixel column value Bxi includes an integrated value of pixel values ​​of each pixel constituting a y-direction pixel column of x coordinate = xi, The pixel row value Byi includes an integrated value of pixel values ​​of each pixel constituting an x-direction pixel row of y coordinate = yi.

2. The virtual image evaluation method according to claim 1.

3. the test image is composed of background pixels and the reference pixel group, the reference pixel group is composed of reference pixels whose pixel values ​​are set to a predetermined reference value and correction pixels whose pixel values ​​are set to a correction pixel value different from the predetermined reference value; 3. The virtual image evaluation method according to claim 2.

4. The correction pixel is adjacent to the outer edge of the reference pixel.

4. The virtual image evaluation method according to claim 3.

5. the pixel value of the background pixel is the pixel value with the lowest brightness within a variable range; the predetermined reference value is a pixel value with maximum brightness within a variable range; 5. The virtual image evaluation method according to claim 4.

6. the reference pixel group is a rectangular pixel group; 6. The virtual image evaluation method according to claim 5.

7. a correction step of correcting the pixel values ​​of the reference pixel group of the test image based on the imaging characteristics of an imaging means (140) used in the imaging step; 7. The virtual image evaluation method according to claim 1, further comprising:

8. A virtual image evaluation device (100) for a head-up display (12) includes an image output device (22) that outputs an image, and an optical system that enlarges the image and projects it onto a windshield (13) of a vehicle, and projects the enlarged image onto the windshield to display a virtual image of the image so as to be visible from inside the vehicle, an inspection image display control unit that causes the image output device to display an inspection image; an image acquisition unit that acquires a captured image of a projection object corresponding to the windshield when the test image is projected onto the projection object; an evaluation unit that evaluates distortion of the virtual image based on the captured image acquired by the image acquisition unit, The inspection image display control unit a target position setting unit that sets a plurality of target positions that serve as a plurality of reference points for distortion evaluation; a reference pixel group setting unit that sets a plurality of reference pixel groups, each of which is composed of a plurality of consecutive pixels, at positions spaced apart from one another and corresponding to the plurality of target positions in the test image; the inspection image display control unit includes a pixel value setting unit that sets, for each of the reference pixel groups, a pixel value of each pixel constituting the reference pixel group such that a center of gravity G, expressed by the following formula, coincides with or is close to the corresponding target position, where Bxi is a pixel row value for a y-direction pixel row at x coordinate = xi, Byi is a pixel row value for an x-direction pixel row at y coordinate = yi, nx is the number of y-direction pixel rows constituting a corresponding reference pixel group, and ny is the number of x-direction pixel rows: [Equation 2] A virtual image evaluation device characterized by:

9. the inspection image display control unit sets pixel values ​​of each pixel constituting each of the reference pixel groups in the inspection image so that the center of gravity G expressed by the formula is a distance less than 0.05 times the distance of one pixel in the inspection image.

9. The virtual image evaluation device according to claim 8.

Images