Image generation device, image generation method, and image generation program

Abstract

To provide an image generation device, etc., capable of generating an image in which halation does not occur (more hardly occurs) even in a case where a contrast difference between a road surface drawing pattern and a peripheral region is large and exceeds a handleable range of an imaging apparatus with an HDR function.SOLUTION: An image generation device comprises: an image acquisition section 64a for acquiring a first image group including a road surface drawing pattern and a peripheral region which are imaged in a first gain range and a second image group including the road surface drawing pattern and the peripheral region which are imaged in a second gain range shifted from the first gain range by a fixed value in a direction where a gain is suppressed; a storage section storing a coordinate group defining a contour shape corresponding to the road surface drawing pattern; and an HDR image generation section 64b for generating a first HDR image based on each peripheral region image portion outside of the contour shape in the first image group and generating a second HDR image based on each road surface drawing pattern image portion inside of the contour shape in the second image group.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present disclosure relates to an image generation device, an image generation method, and an image generation program. [Background technology]

[0002] A vehicle system is known that, when the gear position is in reverse, displays on an in-vehicle monitor an image (video) including a road surface drawing pattern captured by an imaging device and formed on the road surface behind the vehicle, as well as the surrounding area (surrounding environment) (see, for example, Patent Document 1).

[0003] The present inventors have considered using an imaging device with a high dynamic range (HDR) function. The HDR function is a function that captures multiple images while changing the exposure within a gain range according to the brightness of the subject (or appropriate for the subject), performs HDR processing based on the multiple images, and combines the multiple images to generate an HDR image with an expanded dynamic range. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-108190 Summary of the Invention [Problem to be solved by the invention]

[0005] However, if the difference in brightness between the road surface drawing pattern and the surrounding area is large and exceeds the range that can be handled by an imaging device with HDR function, there is a problem that halation may occur in the image including the road surface drawing pattern and the surrounding area (especially in the image part corresponding to the relatively bright road surface drawing pattern).

[0006] The present disclosure has been made to solve such problems, and aims to provide an image generation device, an image generation method, and an image generation program that can generate an image in which no halation occurs (or occurs almost no halation) even when the difference in brightness between the road surface drawing pattern and the surrounding area is large and exceeds the range that can be handled by an imaging device with HDR function. [Means for solving the problem]

[0007] The image generating device according to the present disclosure is an image generating device mounted on a vehicle, which generates an image including a road surface drawing pattern and its surrounding area formed on a road surface by a lighting device mounted on the vehicle and which is displayed on a display device mounted on the vehicle, and includes an image acquisition unit that acquires a first group of images including the road surface drawing pattern and its surrounding area captured by an imaging device mounted on the vehicle while changing exposure within a first gain range, and a second group of images including the road surface drawing pattern and its surrounding area captured by the imaging device while changing exposure within a second gain range that is shifted by a certain value in a direction that suppresses gain from the first gain range, and an image acquisition unit that defines a contour shape corresponding to the road surface drawing pattern. the image generation unit generating a first HDR image by performing predetermined image processing based on the peripheral area image portion outside the contour shape of each of the first image group, and a second HDR image by performing HDR processing based on the road surface drawing pattern image portion inside the contour shape of each of the second image group; a brightness adjustment unit adjusting the brightness of the second HDR image; and a composite image generation unit generating a composite image, as the image to be displayed on the display device, by combining the peripheral area image portion outside the contour shape of the first HDR image with the road surface drawing pattern image portion inside the contour shape of the second HDR image after brightness adjustment.

[0008] With this configuration, even if the difference in brightness between the road surface pattern and the surrounding area is large and exceeds the range that can be handled by an imaging device with HDR functionality, it is possible to generate an image in which no halation occurs (or in which halation occurs very little).

[0009] In the image generating device, the brightness adjustment unit may adjust the brightness of the second HDR image so that the brightness is similar to the brightness of the first HDR image.

[0010] In addition, in the above image generating device, when the ambient illuminance detected by an illuminance sensor attached to the vehicle and detecting the ambient illuminance of the vehicle is equal to or lower than a threshold value, the image acquisition unit may acquire the first image group and the second image group from the imaging device.

[0011] In addition, in the above image generating device, the constant value may be determined by comparing a peripheral area image portion outside the contour shape of the image captured by the imaging device with a road surface drawing pattern image portion inside the contour shape.

[0012] In the image generating device, the contour shape defined by the coordinate group may be slightly larger than the outer shape of the road surface drawing pattern formed on a road surface parallel to a horizontal plane.

[0013] In addition, the image generating device may further include a memory unit that pre-stores the constant value for each brightness difference between the surrounding area and the road surface drawing pattern, and when the constant value is required, the brightness difference between the surrounding area and the road surface drawing pattern may be calculated and the constant value corresponding to that brightness difference may be read out from the memory unit.

[0014] An image generation method according to the present disclosure is an image generation method that is mounted on a vehicle and generates an image that includes a road surface drawing pattern and a surrounding area formed on a road surface by a lighting device mounted on the vehicle and is displayed on a display device mounted on the vehicle, the image generation method comprising the steps of: acquiring a first group of images that include the road surface drawing pattern and the surrounding area, captured by an imaging device mounted on the vehicle while changing exposure within a first gain range; and acquiring a second group of images that include the road surface drawing pattern and the surrounding area, captured by the imaging device while changing exposure within a second gain range that is shifted by a certain value in a direction that suppresses gain from the first gain range, A first HDR image is generated by performing a predetermined image processing based on the peripheral area image portion outside the contour shape defined by the coordinate group stored in a memory unit for each of the first image group, and a second HDR image is generated by performing HDR processing based on the road surface drawing pattern image portion inside the contour shape for each of the second image group, and the brightness of the second HDR image is adjusted to generate a composite image as the image to be displayed on the display device, by combining the peripheral area image portion outside the contour shape of the first HDR image with the road surface drawing pattern image portion inside the contour shape of the second HDR image after brightness adjustment.

[0015] The image generation program according to the present disclosure is an image generation program that is mounted on a vehicle and generates an image including a road surface drawing pattern and a surrounding area formed on a road surface by a lighting device mounted on the vehicle and that is displayed on a display device mounted on the vehicle, the image generation program including: a process of acquiring a first group of images including the road surface drawing pattern and the surrounding area captured by an imaging device mounted on the vehicle while changing exposure within a first gain range; and a second group of images including the road surface drawing pattern and the surrounding area captured by the imaging device while changing exposure within a second gain range that is shifted by a certain value in a direction that suppresses gain from the first gain range; the first HDR image by performing predetermined image processing based on a peripheral area image portion outside the contour shape defined by a coordinate group stored in a unit, and the second HDR image by performing HDR processing based on a road surface drawing pattern image portion inside the contour shape of each of the second image group; adjusting the brightness of the second HDR image; and generating, as the image to be displayed on the display device, a composite image by combining the peripheral area image portion outside the contour shape of the first HDR image with the road surface drawing pattern image portion inside the contour shape of the second HDR image after brightness adjustment. [Effects of the Invention]

[0016] The present disclosure makes it possible to provide an image generation device, an image generation method, and an image generation program that can generate an image in which no halation occurs (or occurs very little) even when the difference in brightness between a road surface drawing pattern and the surrounding area is large and exceeds the range that can be handled by an imaging device with HDR functionality. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a schematic diagram of a vehicle system 1 including an image generating device 60. FIG. [Figure 2] FIG. 2(a) is an example of a road surface pattern A1 formed on the road surface behind the vehicle V, and FIG. 2(b) is a top view of FIG. 2(b). [Figure 3] 1 is an example of an image (video) displayed on a monitor 31 mounted on a vehicle V. [Figure 4] (a) An example of a group of normal images (1 / 60, 2 / 60...) captured while changing the exposure within a gain range (e.g., see gain A in Figure 5) corresponding to the brightness of the peripheral area A2 (or suitable for the peripheral area A2); (b) An example of a first group of images (1 / 60, 3 / 60...) captured while changing the exposure within a first gain range (e.g., see gain C in Figure 5) corresponding to the brightness of the peripheral area A2 (or suitable for the peripheral area A2); and (c) an example of a second group of images (2 / 60, 4 / 60...) captured while changing the exposure within a second gain range (e.g., see gain A in Figure 5) that is shifted by a "constant value" in the direction of suppressing the gain compared to the first gain range (e.g., see gain C in Figure 5). [Figure 5] FIG. 2 is a diagram showing the range that the imaging device 11 can handle. [Figure 6] FIG. 4 is a schematic diagram of a road surface drawing pattern A1 extracted from FIG. 3. [Figure 7] (a) A diagram showing the relationship between a road surface drawing pattern A1 formed on a road surface parallel to the horizontal plane and a road surface drawing pattern A1 formed on a road surface inclined at an angle θ2 (e.g., -1°) with respect to the horizontal plane; (b) A diagram showing the relationship between a road surface drawing pattern A1 formed on a road surface parallel to the horizontal plane and a road surface drawing pattern A1 formed on a road surface inclined at an angle θ3 (e.g., +1°) with respect to the horizontal plane. [Figure 8] FIG. 2 is a functional block diagram of an image generating device 60. [Figure 9] 2 is a schematic diagram of an example of the operation of the vehicle system 1. FIG. [Figure 10] 4 is a flowchart of an example of the operation of the vehicle system 1. [Figure 11] 10 is a diagram for explaining the effects of the vehicle system 1 (image generating device 60) configured as described above, in comparison with a comparative example. [Figure 12] (a) is an example of an image including a road surface drawing pattern A1 and a pedestrian U in which halation occurs, and (b) is an example of an image including a road surface drawing pattern A1 and a pedestrian U in which halation does not occur. [Figure 13] 10 is a flowchart of a first modified example of the operation example of the vehicle system 1. [Figure 14] 10 is a flowchart of a second modified example of the operation example of the vehicle system 1. [Figure 15] 10 is a flowchart of a third modified example of the operation example of the vehicle system 1. [Figure 16] 10A and 10B are top views of a road surface drawing pattern A10, which is a modified example of the road surface drawing pattern A1; and FIG. 10B is an example of the road surface drawing pattern A10 displayed on a monitor 31 mounted on a vehicle V. [Figure 17] 10A is a top view of a road surface drawing pattern A11, which is a modified example of the road surface drawing pattern A1, and FIG. 10B is an example of the road surface drawing pattern A11 displayed on a monitor 31 mounted on a vehicle V. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0018] A vehicle system 1 including an image generation device 60 according to an embodiment of the present disclosure will be described below with reference to the accompanying drawings. Corresponding components in each drawing are designated by the same reference numerals, and redundant description will be omitted.

[0019] FIG. 1 is a schematic diagram of a vehicle system 1 including an image generating device 60. As shown in FIG.

[0020] As shown in FIG. 1, a vehicle system 1 is mounted on a vehicle V such as an automobile.

[0021] First, an overview of the vehicle system 1 will be described.

[0022] When the gear position of the vehicle V is in reverse, the vehicle system 1 turns on a road illumination unit 42 (an example of a lighting device of the present disclosure) mounted on the vehicle V, thereby forming a road surface drawing pattern A1 on the road surface behind the vehicle V. FIG. 2(a) is an example of the road surface drawing pattern A1 formed on the road surface behind the vehicle V. FIG. 2(b) is a top view of FIG. 2(b). The road surface drawing pattern A1 is formed on the road surface behind the vehicle V mainly for the purpose of notifying surrounding people (e.g., pedestrians and other vehicles) of the presence of the reversing vehicle V.

[0023] The vehicle system 1 captures images including a road surface drawing pattern A1 and a surrounding area A2 (surrounding environment) at, for example, 60 fps using an imaging device 11 mounted on the rear end of the vehicle V. The vehicle system 1 displays the images (video) captured by the imaging device 11 on a monitor 31 (an example of a display device of the present disclosure) mounted on the vehicle V.

[0024] Fig. 3 is an example of an image (video) displayed on the monitor 31 mounted on the vehicle V. As shown in Fig. 3, the image (video) displayed on the monitor 31 includes a road surface drawing pattern A1, a surrounding area A2, and guidelines GL for assisting the parking of the vehicle V as it backs up. The driver of the vehicle V backs up the vehicle V while visually checking this screen (video) as appropriate.

[0025] Next, the vehicle system 1 will be described in detail.

[0026] 1, the vehicle system 1 includes a surrounding information detection unit 10, a vehicle state detection unit 20, an IVI (In-Vehicle Infotainment) system 30, a road surface illumination system 40, a backup lamp 50, and an image generation device 60. These can communicate with each other via an in-vehicle network NW.

[0027] The surrounding information detection unit 10 includes an imaging device 11 and an illuminance sensor 12 .

[0028] The imaging device 11 is an imaging device that captures an image including a road surface drawing pattern A1 formed on the road surface behind the vehicle V and a surrounding area A2, and is provided at the rear end of the vehicle V (see Figures 2(a) and 2(b)).

[0029] The imaging device 11 is provided, for example, at a height H1 from the road surface (see FIG. 2(a)) and at the center of the rear end of the vehicle V in the vehicle width direction (see FIG. 2(b)). The height H1 from the road surface is, for example, 700 mm. The optical axis AX of the imaging device 11 11 is tilted downward at an angle θ1 with respect to the horizontal. The angle θ1 is, for example, 10°.

[0030] Although not shown, the imaging device 11 includes an imaging element, such as a CCD sensor or a CMOS sensor.

[0031] The imaging device 11 is an imaging device (e.g., a video camera) with an HDR function. The imaging device 11 periodically captures images (e.g., at 60 fps) including a road surface drawing pattern A1 and a surrounding area A2. At this time, the positional relationship between the imaging device 11 and the road surface illumination unit 42 does not change, so the road surface drawing pattern A1 is displayed at the same position (a fixed position) in each image.

[0032] When the ambient illuminance detected by the illuminance sensor 12 exceeds a threshold value (e.g., about 1 Lx), i.e., when the surrounding environment is bright, the imaging device 11 captures a group of images (hereinafter referred to as a group of normal images) including the road surface painting pattern A1 and the surrounding area A2 while changing the exposure within a gain range (e.g., see gain A in FIG. 5) corresponding to the brightness of the surrounding area A2 (or appropriate for the surrounding area A2), just like a typical imaging device. FIG. 4(a) shows an example of a group of normal images (1 / 60, 2 / 60, etc.) captured while changing the exposure within a gain range (e.g., see gain A in FIG. 5) corresponding to the brightness of the surrounding area A2 (or appropriate for the surrounding area A2). FIG. 5 is a diagram showing the range that the imaging device 11 can handle.

[0033] On the other hand, when the ambient illuminance detected by the illuminance sensor 12 is below a threshold value (for example, about 1 Lx), i.e., when the surrounding environment is dark, the imaging device 11 captures a first group of images including the road surface pattern A1 and the surrounding area A2 while changing the exposure within a first gain range (for example, see gain C in Figure 5) corresponding to the brightness of the surrounding area A2 (or appropriate for the surrounding area A2), and captures a second group of images including the road surface pattern A1 and the surrounding area A2 while changing the exposure within a second gain range (for example, see gain A in Figure 5) shifted by a "constant value" in the direction of suppressing the gain from the first gain range (for example, see gain C in Figure 5). Figure 4(b) shows an example of a first group of images (1 / 60, 3 / 60...) captured while changing the exposure within a first gain range (e.g., see gain C in Figure 5) that corresponds to the brightness of the peripheral area A2 (or is appropriate for the peripheral area A2), and a second group of images (2 / 60, 4 / 60...) captured while changing the exposure within a second gain range (e.g., see gain A in Figure 5) that is shifted by a "constant value" in the direction of suppressing the gain compared to the first gain range (e.g., see gain C in Figure 5).

[0034] Although FIG. 4(b) shows an example in which the first image and the second image are alternately captured, the present invention is not limited to this, and the first image and the second image may be captured in a different order.

[0035] The first gain range (see, for example, gain C in FIG. 5) is a gain range that corresponds to the brightness of the peripheral area A2 (or is suitable for the peripheral area A2). On the other hand, the second gain range (see, for example, gain A in FIG. 5) is a gain range that is shifted by a "constant value" in the direction of suppressing the gain from the first gain range, that is, a gain range that corresponds to the brightness of the road surface drawing pattern A1 (or is suitable for the road surface drawing pattern A1). The "constant value" can be specified, for example, as follows. Note that a "constant value" shift in the direction of suppression basically means adjusting the gain (GAIN) to shift the center value of HDR by a "constant value."

[0036] For example, first, the image portion (hereinafter referred to as the surrounding area image portion) outside the contour shape (contour shape defined by the coordinate group 62b) of the image (e.g., the first image) captured by the imaging device 11 is compared with the image portion (hereinafter referred to as the road surface drawing pattern image portion) inside the contour shape (contour shape defined by the coordinate group 62b), and the brightness difference between the two is calculated.

[0037] Once this brightness difference is determined, it is possible to calculate how much (a "constant value") the gain needs to be shifted in the direction of suppressing it from the first gain range (for example, see gain C in Figure 5) to achieve proper exposure (a gain range suitable for road surface drawing pattern A1).

[0038] Therefore, for example, shift amounts ("constant values") that result in proper exposure corresponding to each of the brightness differences (plurality) between the surrounding area image portion and the road surface drawing pattern image portion are stored in advance (for example, before product shipment) in the storage unit 62 or the like. Then, when a "constant value" is needed, the brightness difference between the surrounding area image portion and the road surface drawing pattern image portion is calculated, and the "constant value" that corresponds to that brightness difference is read out from the storage unit 62 or the like. In this way, the "constant value" can be identified. Therefore, it is no longer necessary to calculate the constant value each time, and the calculation load for calculating the constant value can be reduced.

[0039] The normal image group, the first image group, and the second image group captured as described above are transmitted to the image generating device 60.

[0040] The illuminance sensor 12 is a sensor that detects the ambient illuminance of the vehicle V, and is attached, for example, to a dashboard inside the vehicle V. The illuminance (ambient illuminance) detected by the illuminance sensor 12 is transmitted to the image generating device 60.

[0041] The vehicle state detection unit 20 includes a vehicle speed sensor 21 and a gear position sensor 22 .

[0042] The vehicle speed sensor 21 is a sensor that detects the speed of the vehicle V and is attached to a predetermined location on the vehicle V. The vehicle speed detected by the vehicle speed sensor 21 is transmitted to the image generation device 60. Note that the vehicle speed sensor 21 is used in Modification 3, which will be described later, but is not used in this embodiment. To make this clear, the vehicle speed sensor 21 is represented by a dotted rectangle in FIG. 1.

[0043] The gear position sensor 22 is a sensor that detects the gear position of the transmission of the vehicle V, and is attached to a predetermined location of the vehicle V. The gear position detected by the gear position sensor 22 is transmitted to the image generation device 60.

[0044] The IVI (In-Vehicle Infotainment) system 30 includes a monitor 31, map information 32, a GPS sensor 33, and a road surface illumination ON / OFF switch .

[0045] The monitor 31 is, for example, an image display device including a display surface such as a liquid crystal display, and is attached, for example, to the dashboard inside the vehicle V. The monitor 31 (display surface) displays various images (or videos). For example, the monitor 31 (display surface) displays an image including the road surface drawing pattern A1 and the surrounding area A2 (a composite image synthesized by a composite image generation unit 64d, which will be described later), and a Function display (for example, an audio screen, a navigation screen, or a TV screen).

[0046] The map information 32 and the GPS sensor 33 (GPS receiver) constitute part of a navigation system (not shown). For example, the current position of the vehicle V is calculated based on radio waves received by the GPS sensor 33 from GPS satellites, a map corresponding to the current position of the vehicle V is read from the map information 32, and an image symbolizing the vehicle V and a route to the destination (navigation screen) are displayed on the monitor 31 together with the map. Note that the map information 32 and the GPS sensor 33 are used in Modification 1, which will be described later, but are not used in this embodiment. To clarify this, the map information 32 and the GPS sensor 33 are represented by dotted rectangles in FIG. 1.

[0047] The road surface illumination ON / OFF switch 34 is a switch for turning the road surface illumination unit 42 on and off, and is attached, for example, to a dashboard inside the passenger compartment of the vehicle V. Note that the road surface illumination ON / OFF switch 34 is used in Modification 2, which will be described later, but is not used in this embodiment. To make this clear, the road surface illumination ON / OFF switch 34 is represented by a dotted rectangle in FIG. 1.

[0048] The road surface illumination system 40 includes a road surface illumination control device 41 and a road surface illumination unit 42.

[0049] When the gear position detected by the gear position sensor 22 is reverse gear, the road surface illumination control device 41, in accordance with control from the image generation device 60 (road surface illumination control device control unit 64f), controls the road surface illumination unit 42 so that the road surface illumination unit 42 is turned on. On the other hand, when the gear position detected by the gear position sensor 22 is other than reverse gear, the road surface illumination control device 41, in accordance with control from the image generation device 60 (road surface illumination control device control unit 64f), controls the road surface illumination unit 42 so that the road surface illumination unit 42 is turned off.

[0050] The road surface illumination unit 42 is turned on (or off) under control of the road surface illumination control device 41, and forms a road surface drawing pattern A1 on the road surface behind the vehicle V. The illuminance of the road surface drawing pattern A1 is, for example, 100 to 300 Lx. The road surface drawing pattern A1 has a trapezoidal shape, as shown in FIG. 2(b). Note that the shape of the road surface drawing pattern A1 is not limited to a trapezoidal shape.

[0051] The road surface illumination units 42 are provided on both the left and right sides of the rear end of the vehicle V. For example, the road surface illumination units 42 are provided at a height H2 (see FIG. 2(a)) from the road surface. The height H2 from the road surface is, for example, 1000 mm. The distance L1 (see FIG. 2(b)) from the imaging device 11 in the vehicle width direction is, for example, 900 mm.

[0052] The number of road surface drawing patterns A1 formed by the road surface illumination unit 42 on the road surface behind the vehicle V is, for example, three (see FIGS. 2(a) and 2(b)). The three road surface drawing patterns A1 are arranged along straight lines extending in the fore-and-aft direction of the vehicle (see, for example, the straight lines indicated by the symbols Line 1 and Line 2 in FIG. 2) within a range of distance L2 (see FIG. 2(a)) from the vehicle V toward the rear of the vehicle. Distance L2 is, for example, 2555 mm.

[0053] Although not shown, for example, the road surface illumination unit 42 is configured to include three LED light sources and three projection lenses corresponding to the three LED light sources, and each projection lens projects a corresponding LED light source (rectangular light-emitting surface) to form the three road surface drawing patterns A1 on the road surface behind the vehicle V. However, the road surface illumination unit 42 is not limited to this, and may have any configuration as long as it can form one or more (two or four or more) road surface drawing patterns A1 on the road surface behind the vehicle V.

[0054] The backup lamps 50 are typical backup lamps that are turned on when the gear position of the vehicle V is in reverse gear and turned off when the gear position is not reverse gear. The illuminance of the road surface illuminated by the backup lamps 50 is, for example, 40 Lx or less.

[0055] The image generating device 60 is, for example, an ECU (Electronic Control Unit) including a processor 61, a storage unit 62, a memory 63, and the like.

[0056] The memory unit 62 stores a program 62a and a coordinate group 62b. The program 62a is a program executed by the processor 61. The coordinate group 62b is a group of coordinates (coordinate values) that define the contour shape corresponding to the road surface drawing pattern A1. FIG. 6 is a schematic diagram of the road surface drawing pattern A1 extracted from FIG. 3. In FIG. 6, the inverted trapezoid drawn with a solid line represents the outer shape of the road surface drawing pattern A1. Meanwhile, the inverted trapezoid drawn with a dotted line, which is one size larger, represents the contour shape defined by the coordinate group 62b.

[0057] As shown in Fig. 6, the contour shape defined by the coordinate group 62b (see the inverted trapezoid drawn by the dotted line in Fig. 6) is set to be slightly larger than the outer shape of the road surface drawing pattern A1 (see the inverted trapezoid drawn by the solid line in Fig. 6), for the following reason.

[0058] 7(a) is a diagram showing the relationship between a road surface drawing pattern A1 formed on a road surface parallel to the horizontal plane and a road surface inclined at an angle θ2 (e.g., −1°) with respect to the horizontal plane. FIG. 7(b) is a diagram showing the relationship between a road surface drawing pattern A1 formed on a road surface parallel to the horizontal plane and a road surface inclined at an angle θ3 (e.g., +1°) with respect to the horizontal plane.

[0059] In Fig. 7(a), the rectangle drawn with solid lines represents a road surface pattern A1 formed on a road surface parallel to the horizontal plane, while the rectangle drawn with dotted lines represents a road surface pattern A1 formed on a road surface inclined at an angle θ2 (for example, -1°) with respect to the horizontal plane.

[0060] Similarly, in Fig. 7(b), the rectangle drawn with solid lines represents a road surface pattern A1 formed on a road surface parallel to the horizontal plane, while the rectangle drawn with dotted lines represents a road surface pattern A1 formed on a road surface inclined at an angle θ3 (e.g., +1°) with respect to the horizontal plane.

[0061] Referring to Figures 7(a) and 7(b), when the road surface behind the vehicle V is inclined at an angle θ2 or θ3 with respect to the horizontal plane, the road surface drawing pattern A1 is formed at a position slightly shifted toward or away from the vehicle V relative to the road surface drawing pattern A1 formed on a road surface parallel to the horizontal plane.

[0062] In order to accommodate the road surface drawing pattern A1 formed at the shifted position as described above, the contour shape defined by the coordinate group 62b (see the inverted trapezoid drawn by dotted lines in Figure 6) is set to be slightly larger than the outer shape of the road surface drawing pattern A1 formed on the road surface parallel to the horizontal plane (see the inverted trapezoid drawn by solid lines in Figure 6).

[0063] For example, the upper width W1 (see Figure 6) between the contour shape defined by the coordinate group 62b and the outer shape of the road surface drawing pattern A1 is 25 mm, the lower width W2 (see Figure 6) is 22 mm, and the widths W3 (see Figure 6) on both the left and right sides are 17 mm.

[0064] FIG. 8 is a functional block diagram of the image generating device 60.

[0065] The processor 61 executes a program 62a loaded from a storage unit 62 (e.g., ROM) into a memory 63 (e.g., RAM), thereby functioning as an image acquisition unit 64a, an HDR image generation unit 64b, a brightness adjustment unit 64c, a composite image generation unit 64d, a display control unit 64e, and a road surface illumination control device control unit 64f, as shown in Fig. 8. Some or all of these may be realized by hardware.

[0066] The image acquisition unit 64a acquires the normal image group, the first image group, and the second image group that are captured by the imaging device 11 and transmitted.

[0067] When the image acquisition unit 64a acquires a group of normal images, the HDR image generation unit 64b performs predetermined image processing, for example, HDR (High Dynamic Range) processing, based on the group of normal images, in the same way as a general imaging device, to generate an HDR image (hereinafter referred to as a normal HDR image) that combines the group of normal images and expands the dynamic range.

[0068] On the other hand, when the image acquisition unit 64a acquires the first image group, the HDR image generation unit 64b performs predetermined image processing based on the peripheral area image portions (range-limited images) outside the outline shapes (outline shapes defined by the coordinate group 62b) of each of the first image group, thereby synthesizing the peripheral area image portions (group) and generating a first HDR image with an expanded dynamic range. Below, an example will be described in which the predetermined image processing is HDR processing.

[0069] In addition, when the image acquisition unit 64a acquires the second image group, the HDR image generation unit 64b performs HDR processing based on the road surface drawing pattern image portion (range-limited image) inside the contour shape (contour shape defined by the coordinate group 62b) of each of the second image group, thereby synthesizing the road surface drawing pattern image portion(s) and generating a second HDR image with an expanded dynamic range.

[0070] As described above, the range to be subjected to HDR processing is limited not to the entire first image but to a portion of the first image, i.e., the peripheral area image portion outside the contour shape (the contour shape defined by the coordinate group 62b). Similarly, the range to be subjected to HDR processing is limited not to the entire second image but to a portion of the second image, i.e., the road surface drawing pattern image portion inside the contour shape (the contour shape defined by the coordinate group 62b). This reduces the calculation load of HDR processing compared to when the entire first image and the entire second image are subjected to HDR processing. Furthermore, it is possible to relatively reduce the phenomenon in which the image displayed on the monitor 31 momentarily turns white due to HDR processing being unable to keep up.

[0071] The brightness adjustment unit 64c adjusts the brightness of the second HDR image generated by the HDR image generation unit 64b. For example, the brightness adjustment unit 64c adjusts the brightness of the second HDR image (the road surface drawing pattern image portion inside the contour shape (the contour shape defined by the coordinate group 62b)) by performing predetermined image processing on the second HDR image so that the brightness becomes similar to that of the first HDR image.

[0072] For example, assume that the illuminance of the road surface illuminated by the backup lamp 50 is 40 Lx, the illuminance of the road surface drawing pattern A1 formed on the road surface by the road surface illumination unit 42 is 100 Lx, and the illuminance (ambient illuminance) detected by the illuminance sensor 12 is 1 Lx.

[0073] In this case, for example, the brightness adjustment unit 64c adjusts the brightness of the second HDR image (the road surface drawing pattern image portion inside the contour shape (contour shape defined by the coordinate group 62b)) by performing image processing such that the brightness value of each pixel constituting the road surface drawing pattern image portion inside the contour shape (contour shape defined by the coordinate group 62b) of the second HDR image = brightness value of each pixel × (40Lx + 1Lx) / (100Lx + 1Lx).

[0074] The composite image generation unit 64d generates a composite image (video) to be displayed on the monitor 31 by combining the peripheral area image portion outside the contour shape (contour shape defined by the coordinate group 62b) of the first HDR image with the road surface drawing pattern image portion inside the contour shape (contour shape defined by the coordinate group 62b) of the second HDR image after brightness adjustment.

[0075] The display control unit 64e controls the monitor 31 so that the normal HDR image (video) generated by the HDR image generation unit 64b or the composite image (video) generated by the composite image generation unit 64d is displayed. The display control unit 64e also controls the monitor 31 so that a Function display (for example, an audio screen, a navigation screen, or a TV screen) is displayed.

[0076] The road surface illumination control device control unit 64f controls the road surface illumination control device 41 so that the road surface illumination unit 42 is turned on when the gear position detected by the gear position sensor 22 is reverse gear. On the other hand, the road surface illumination control device control unit 64f controls the road surface illumination control device 41 so that the road surface illumination unit 42 is turned off when the gear position detected by the gear position sensor 22 is other than reverse gear.

[0077] Next, an outline of an example of the operation of the vehicle system 1 configured as above will be described.

[0078] FIG. 9 is a schematic diagram of an example of the operation of the vehicle system 1.

[0079] The following processing is mainly performed by the processor 61 executing the program 62 a that has been read from the storage unit 62 into the memory 63 .

[0080] 9, first, the imaging device 11 acquires a group of first images I1 including the road surface pattern A1 and the surrounding area A2 captured within a first gain range (e.g., see gain C in FIG. 5) (step S1), and then acquires a group of second images I2 including the road surface pattern A1 and the surrounding area A2 captured within a second gain range (e.g., see gain A in FIG. 5) that is shifted by a certain value in a direction that suppresses the gain from the first gain range (step S2). This is achieved by the image acquisition unit 64a.

[0081] Next, HDR processing (step S3) is performed based on the peripheral area image portions (range-limited images) outside the contour shapes (contour shapes defined by the coordinate group 62b) of the first images I1 group acquired as described above, to synthesize the peripheral area image portions (groups) and generate a first HDR image I1 with an expanded dynamic range. HDR In addition, HDR processing (step S5) is performed based on the road surface drawing pattern image portions (range-limited images) inside the contour shapes (contour shapes defined by the coordinate group 62b) of the second images I2 acquired as described above, thereby generating a second HDR image I2 with an expanded dynamic range by combining the road surface drawing pattern image portions (groups). HDR (Step S6) The processes of steps S3 to S6 are realized by the HDR image generating unit 64b.

[0082] Next, the second HDR image I2 HDR (Step S7). For example, the brightness adjustment unit 64c adjusts the brightness of the second HDR image (the portion of the road surface drawing pattern image inside the contour shape (the contour shape defined by the coordinate group 62b)) by performing a predetermined image processing on the second HDR image so that the brightness becomes similar to that of the first HDR image. This is achieved by the brightness adjustment unit 64c.

[0083] Next, by executing a synthesis process (step S8), the first HDR image I1 is displayed as an image (video) on the monitor 31. HDR The image portion of the peripheral area outside the contour shape (contour shape defined by the coordinate group 62b) of the second HDR image I2 after brightness adjustment HDR The road surface drawing pattern image portion inside the contour shape (contour shape defined by the coordinate group 62b) is combined with the road surface drawing pattern image portion inside the contour shape (contour shape defined by the coordinate group 62b) to form a composite image I HDR Generate.

[0084] Next, an example of the operation of the vehicle system 1 having the above configuration will be described in detail.

[0085] FIG. 10 is a flowchart of an example of the operation of the vehicle system 1.

[0086] The following processing is mainly performed by the processor 61 executing the program 62 a that has been read from the storage unit 62 into the memory 63 .

[0087] First, it is determined whether the gear position is reverse gear or not (step S10). This is determined based on the gear position detected by the gear position sensor 22.

[0088] Next, if it is determined that the gear position is other than reverse gear (step S10: NO), a Function display (for example, an audio screen, a navigation screen, or a TV screen) is displayed on the monitor 31 (display surface) (step S11). This is realized by the display control unit 64e.

[0089] Next, it is determined whether or not the ignition (IG) is ON (step S12).

[0090] Next, if it is determined that the IG (ignition) is ON (step S12: YES), the process returns to step S10, and the processes from step S10 onward are repeatedly executed. On the other hand, if it is determined that the IG (ignition) is OFF (step S12: NO), the process of FIG. 10 ends.

[0091] On the other hand, if it is determined that the gear position is reverse (step S10: YES), the backup lamps 50 and the road surface illumination unit 42 are turned on (step S13). By turning on the backup lamps 50, the road surface behind the vehicle V is illuminated. In addition, by turning on the road surface illumination unit 42, a road surface drawing pattern A1 is formed on the road surface behind the vehicle V.

[0092] Next, it is determined whether the ambient illuminance is equal to or less than a threshold value (step S14). This determination is made by comparing the illuminance (ambient illuminance) detected by the illuminance sensor 12 with the threshold value. The threshold value may be, for example, one stored in the storage unit 62.

[0093] If it is determined that the ambient illuminance exceeds a threshold value (for example, about 1 Lx) (step S14: NO), that is, if the ambient environment is bright, a normal image is acquired (step S15). This is achieved by the image acquisition unit 64a.

[0094] Specifically, if it is determined that the ambient illuminance exceeds a threshold value (e.g., about 1 Lx) (step S14: NO), the imaging device 11, like a general imaging device, captures a group of images (a group of normal images) including the road surface pattern A1 and the surrounding area A2 while changing the exposure within a gain range (e.g., see gain A in FIG. 5) according to the brightness of the surrounding area A2 (or appropriate for the surrounding area A2). The captured group of normal images is transmitted to the image generation device 60. The image acquisition unit 64a acquires the group of normal images (normal videos) (step S15).

[0095] Next, HDR processing is performed (step S16). This is realized by the HDR image generation unit 64b. Like a general imaging device, the HDR image generation unit 64b performs HDR processing based on the normal image group (normal video) acquired in step S15, thereby synthesizing the normal image group and generating an HDR image (normal HDR image) with an expanded dynamic range.

[0096] Next, the rear view of the vehicle is displayed on the monitor 31 (step S17). This is achieved by the display control unit 64e. The rear view of the vehicle displayed here is composed of a normal HDR image generated as a result of the HDR processing in step S16.

[0097] Next, it is determined whether the gear position is reverse gear or not (step S18). This is determined based on the gear position detected by the gear position sensor 22.

[0098] Next, if the gear position is determined to be reverse gear (step S18: YES), the process returns to step S13, and the processes from step S13 onward are repeatedly executed. On the other hand, if the gear position is determined to be other than reverse gear (step S18: NO), the backup lamps 50 and the road surface illumination unit 42 are turned off (step S19), and the processes from step S11 onward are executed.

[0099] On the other hand, if it is determined in step S14 that the ambient illuminance is equal to or lower than the threshold value (for example, about 1 Lx) (step S14: YES), that is, if the ambient environment is dark, a range-limited image is acquired (step S20). This is achieved by the image acquisition unit 64a.

[0100] Specifically, if the ambient illuminance is determined to be equal to or lower than a threshold value (e.g., about 1 Lx) (step S14: YES), the imaging device 11 captures a first group of images I1 while changing the exposure within a first gain range (e.g., see gain C in FIG. 5) corresponding to the brightness of the peripheral area A2 (or appropriate for the peripheral area A2), and captures a second group of images I2 while changing the exposure within a second gain range (e.g., see gain A in FIG. 5) that is shifted by a "constant value" in a direction that suppresses the gain from the first gain range (e.g., see gain C in FIG. 5). The captured first group of images I1 and second group of images I2 are transmitted to the image generating device 60. The image acquiring unit 64a acquires the first group of images I1 and second group of images I2.

[0101] Next, range-limited HDR processing is performed (step S21). This is realized by the HDR image generation unit 64b. The HDR image generation unit 64b performs HDR processing based on the peripheral area image portions (range-limited images) outside the contour shapes (contour shapes defined by the coordinate group 62b) of each of the first images I1 acquired in step S20, to synthesize the peripheral area image portions (groups) and generate a first HDR image I1 with an expanded dynamic range. HDR Generate.

[0102] In addition, the HDR image generation unit 64b performs HDR processing based on the road surface drawing pattern image portion (range-limited image) inside the outline shape (outline shape defined by the coordinate group 62b) of each of the second images I2 acquired in step S20, and synthesizes the road surface drawing pattern image portion (group) to generate a second HDR image I2 with an expanded dynamic range. HDR Generate.

[0103] Next, the brightness of the second HDR image I2 generated in step S21 is adjusted (step S22). This is achieved by the brightness adjustment unit 64c. HDR For example, the brightness adjustment unit 64c adjusts the brightness of the first HDR image I1 HDR The second HDR image I2 HDR By performing predetermined image processing on the second HDR image I2 HDR The brightness of the road surface drawing pattern image portion inside the outline shape (the outline shape defined by the coordinate group 62b) is adjusted.

[0104] Next, the first HDR image I1 HDR and the second HDR image I2 HDR This is realized by the composite image generation unit 64d. The composite image generation unit 64d generates the first HDR image I1 as the image (video) to be displayed on the monitor 31. HDR The image portion of the peripheral area outside the contour shape (contour shape defined by the coordinate group 62b) of the second HDR image I2 after brightness adjustment HDRThe road surface drawing pattern image portion inside the contour shape (contour shape defined by the coordinate group 62b) is combined with the road surface drawing pattern image portion inside the contour shape (contour shape defined by the coordinate group 62b) to form a composite image I HDR Generate.

[0105] Next, the rear view of the vehicle is displayed on the monitor 31 (step S17). This is realized by the display control unit 64e. The rear view of the vehicle displayed here is the composite image I generated as a result of the processing of step S23. HDR FIG. 3 shows a composite image I displayed on a monitor 31. HDR This is an example.

[0106] Thereafter, the same process as in step S18 is repeatedly executed as described above.

[0107] Next, the effects of the vehicle system 1 (image generating device 60) configured as described above will be described in comparison with a comparative example.

[0108] FIG. 11 is a diagram for explaining the effects of the vehicle system 1 (image generating device 60) configured as described above in comparison with a comparative example.

[0109] Comparative Example 1 is an example in which the imaging device 11 is a general imaging device with an HDR function.

[0110] In Comparative Example 1, multiple images are captured while changing the exposure within a gain range corresponding to the brightness of the subject (for example, see Gain A, Gain B, or Gain C in Figure 11), and HDR processing is performed based on the multiple images to generate an HDR image with an expanded dynamic range by combining the multiple images.

[0111] However, in Comparative Example 1, when the road surface luminance of the road surface pattern A1 is at a bright position (bright position) in FIG. 11 and the illuminance of the surrounding area A2 is at a bright position (dark position) in FIG. 11, that is, when the difference in brightness between the two is large and exceeds the range that can be handled by the imaging device of Comparative Example 1 (for example, see gain C in FIG. 11), there is a problem that halation may occur in the image including the road surface pattern A1 and the surrounding area A2 (particularly, the image portion corresponding to the relatively bright road surface pattern A1). The reason for this is as follows. That is, in Comparative Example 1, as shown in FIG. 11, a wide difference in brightness cannot be covered, so processing is performed by determining a priority range by adjusting the gain. Therefore, in the case of a setting that prioritizes dark areas (for example, see gain C in FIG. 11), halation occurs when the luminance (road surface pattern A1) is outside the range that can be handled by the imaging device of Comparative Example 1 when displayed on the monitor 31.

[0112] Comparative Example 2 is an example in which the imaging device 11 is a high-performance imaging device with an HDR function.

[0113] In Comparative Example 2, multiple images are captured while changing the exposure within a gain range corresponding to the brightness of the subject (for example, see gain D in Figure 11), and HDR processing is performed based on the multiple images to generate an HDR image with an expanded dynamic range by combining the multiple images.

[0114] In this case, the range that the imaging device of Comparative Example 2 can handle is wider than the range that the imaging device 11 of this embodiment can handle and the range that Comparative Example 1 can handle (see gain D in Fig. 11). Therefore, even when the road surface luminance of road surface drawing pattern A1 is at a bright position (bright position) in Fig. 11 and the illuminance of surrounding area A2 is at a bright position (dark position) in Fig. 11, that is, even when there is a large difference in brightness between the two, halation is prevented from occurring in the image that includes road surface drawing pattern A1 and surrounding area A2 (particularly in the image portion that corresponds to the relatively bright road surface drawing pattern A1).

[0115] However, in Comparative Example 2, since the range that can be handled is wide, the number of images that are captured while changing the exposure within a gain range according to the brightness of the subject (for example, see gain D in FIG. 11) increases, which places a heavy load on the HDR processing. This poses a problem of delays in display on the monitor 31. Another problem is that image processing performance (for example, a high-performance processor) that can perform HDR processing on a large number of images at high speed is required, which results in a significant increase in costs.

[0116] In contrast to this, in this embodiment, the range that the imaging device 11 can handle is the same as the range that the imaging device of Comparative Example 1 can handle.

[0117] However, in this embodiment, the imaging device 11 captures multiple images (first image group) while changing the exposure within a gain range (e.g., see gain C in FIG. 11 ) corresponding to the brightness of the surrounding area A2, and performs HDR processing based on the multiple images (first image group) to generate a first HDR image with an expanded dynamic range (step S100). The imaging device 11 also captures multiple images (second image group) while changing the exposure within a gain range (e.g., see gain A in FIG. 11 ) corresponding to the brightness of the road surface pattern A1, and performs HDR processing based on the multiple images (second image group) to generate a second HDR image with an expanded dynamic range (step S101). The second HDR image is darkened by a certain value through image processing by the brightness adjustment unit 64c (step S102). Then, the composite image generation unit 64d generates a composite image by combining the peripheral area image portion outside the contour shape (contour shape defined by the coordinate group 62b) of the first HDR image with the road surface drawing pattern image portion inside the contour shape (contour shape defined by the coordinate group 62b) of the second HDR image after brightness adjustment, and this composite image (video) is displayed on the monitor 31 (step S103).

[0118] Therefore, according to this embodiment, even when the road surface luminance of the road surface pattern A1 is at the bright position (bright position) in Fig. 11 and the illuminance of the surrounding area A2 is at the bright position (dark position) in Fig. 11, that is, even when the difference in brightness between the two is large and exceeds the range that the imaging device 11 can handle (for example, see gain C in Fig. 11), halation is prevented from occurring in the image including the road surface pattern A1 and the surrounding area A2 (especially the image portion corresponding to the relatively bright road surface pattern A1). This solves the problem of the above-mentioned comparative example 1.

[0119] Furthermore, according to this embodiment, the range that the imaging device 11 can handle is similar to that of the imaging device of Comparative Example 1, but is narrower than that of the imaging device of Comparative Example 2, so the number of images that can be captured can be smaller than when using the imaging device of Comparative Example 2. This also solves the problem of Comparative Example 2.

[0120] As described above, according to this embodiment, even if the difference in brightness between the road surface drawing pattern A1 and the surrounding area A2 (surrounding environment) is large and exceeds the range that can be handled by the imaging device 11 with HDR function, it is possible to generate an image (synthetic image) in which no halation occurs (or in which halation occurs very little).

[0121] If the composite image generated as described above includes a road surface drawing pattern A1 and a pedestrian U and halation occurs (see FIG. 12(a)), even if a predetermined image processing is performed on the composite image, it may be impossible to distinguish between the road surface drawing pattern A1 and the pedestrian U, resulting in a false detection of the pedestrian U. FIG. 12(a) is an example of an image including a road surface drawing pattern A1 and a pedestrian U in which halation occurs.

[0122] In contrast, according to this embodiment, a composite image (see FIG. 12(b)) without halation is finally generated. FIG. 12(b) is an example of an image including a road surface drawing pattern A1 and a pedestrian U without halation. Therefore, for example, when a parking support brake, which is part of an ADAS (Advanced Driver-Assistance Systems), performs predetermined image processing on the composite image (a composite image including the road surface drawing pattern A1 and the pedestrian U), it becomes possible to distinguish between the road surface drawing pattern A1 and the pedestrian U. This reduces false detection of the pedestrian U. The parking support brake is a system that issues a warning or performs automatic brake control when it detects a pedestrian U behind the vehicle V based on an image captured by an imaging device (for example, an image (composite image) captured by the imaging device 11). In this way, the image generation device 60 and the image generation method of this embodiment can be used in combination with an ADAS. Note that the composite image finally generated in this embodiment can also be used in systems other than the parking support brake. That is, the image generation device 60 and the image generation method of this embodiment can also be used in combination with devices or systems other than ADAS.

[0123] Next, Modification 1 will be described.

[0124] FIG. 13 is a flowchart of a first modification of the operation example of the vehicle system 1.

[0125] 13, if it is determined that the gear position is reverse (step S10: YES), it is determined whether or not the current position of the vehicle V is within the road surface illumination area (step S30). If it is determined that the current position of the vehicle V is outside the road surface illumination area (step S30: NO), only the backup lamps 50 are turned on (step S31), and then, as in the above embodiment, the processing from step S15 onwards may be executed. On the other hand, if it is determined that the current position of the vehicle V is within the road surface illumination area (step S30: YES), as in the above embodiment, the processing from step S13 onwards may be executed. In this case, in step S19A, only the backup lamps 50 may be turned off. The road surface illumination area is, for example, an area other than a public road, such as a parking lot.

[0126] The determination in step S30, i.e., whether the current position of the vehicle V is within the road surface illumination area, can be made, for example, by comparing the current position of the vehicle V calculated based on radio waves from a GPS satellite received by the GPS sensor 33 with information identifying the road surface illumination area contained in the map information 32 (for example, a group of coordinates identifying the road surface illumination area).

[0127] According to the first modification, in addition to the effects of the above embodiment, the composite image (video) can be displayed on the monitor 31 only when the vehicle V is present in the road surface illumination possible area.

[0128] Next, a second modification will be described.

[0129] FIG. 14 is a flowchart of a second modification of the operation example of the vehicle system 1.

[0130] 14, if it is determined that the gear position is reverse (step S10: YES), it is further determined whether the road surface illumination ON / OFF switch 34 is ON (step S30A). If it is determined that the road surface illumination ON / OFF switch 34 is OFF (step S30A: NO), only the backup lamps 50 may be turned on (step S31), and then, as in the above embodiment, the processing from step S15 onwards may be executed. On the other hand, if it is determined that the road surface illumination ON / OFF switch 34 is ON (step S30A: YES), as in the above embodiment, the processing from step S13 onwards may be executed. In this case, in step S19A, only the backup lamps 50 may be turned off.

[0131] According to the second modification, in addition to the effects of the above embodiment, it is possible to further reduce unnecessary processing.

[0132] Next, a third modification will be described.

[0133] FIG. 15 is a flowchart of a third modification of the operation example of the vehicle system 1.

[0134] 15, if it is determined that the gear position is reverse (step S10: YES), it is determined whether the vehicle speed of the vehicle V detected by the vehicle speed sensor 21 is equal to or less than a threshold value (step S30B). If it is determined that the vehicle speed of the vehicle V exceeds the threshold value (step S30B: NO), only the backup lamps 50 may be turned on (step S31), and then, as in the above embodiment, the processing from step S15 onwards may be executed. On the other hand, if it is determined that the vehicle speed of the vehicle V is equal to or less than the threshold value (step S30B: YES), as in the above embodiment, the processing from step S13 onwards may be executed. In this case, in step S19A, only the backup lamps 50 may be turned off.

[0135] According to the third modification, in addition to the effects of the above embodiment, it is possible to further reduce unnecessary processing when the vehicle V is traveling at a high speed.

[0136] Next, a modified example of the road surface drawing pattern A1 will be described.

[0137] 16(a) is a top view of a road surface drawing pattern A10, which is a modified example of the road surface drawing pattern A1. FIG. 16(b) is an example of the road surface drawing pattern A10 displayed on the monitor 31 mounted on the vehicle V.

[0138] In the above embodiment, an example has been described in which a trapezoidal road surface drawing pattern A1 is used as the road surface drawing pattern, but this is not limiting. That is, road surface drawing patterns of various shapes other than trapezoidal shapes (for example, see road surface drawing pattern A10 shown in FIG. 16(a) and road surface drawing pattern A11 shown in FIG. 17(a)) may be used as the road surface drawing pattern. Note that the dotted line in FIG. 16(b) represents a contour shape that is one size larger than the outer shape of the road surface drawing pattern A10, which is defined by the coordinate group 62b.

[0139] Next, modified examples of the formation location (drawing location) of the road surface drawing pattern A1 will be described.

[0140] 17(a) is a top view of a road surface drawing pattern A11, which is a modified example of the road surface drawing pattern A1. FIG. 17(b) is an example of the road surface drawing pattern A11 displayed on the monitor 31 mounted on the vehicle V.

[0141] In the above embodiment, an example has been described in which the road surface drawing pattern A1 is formed on the road surface behind the vehicle V, but this is not limiting. For example, as shown in FIGS. 17(a) and 17(b), the road surface drawing pattern A11 may be formed (drawn) on the road surface ahead of the vehicle V. Note that the dotted line in FIG. 17(b) represents a contour shape that is one size larger than the outline of the road surface drawing pattern A11, defined by the coordinate group 62b. Furthermore, although not shown, the road surface drawing pattern A11 may be formed (drawn) on a location other than the road surface behind the vehicle V or the road surface ahead of the vehicle V (for example, on the road surface to the side of the vehicle V).

[0142] All the numerical values ​​shown in the above embodiment are merely examples, and it goes without saying that other appropriate numerical values ​​can be used.

[0143] The above-described embodiments are merely examples in all respects. The present disclosure should not be construed as being limited by the description of the above-described embodiments. The present disclosure can be implemented in various other forms without departing from the spirit or main characteristics thereof. [Explanation of symbols]

[0144] 1 Vehicle System 10 Surrounding information detection unit 11 Imaging device 12 Illuminance sensor 20 Vehicle condition detection unit 21 Vehicle speed sensor 22 Gear position sensor 30 IVI systems 31 Monitor 32 Map Information 33 GPS sensor 34 Road surface illumination ON / OFF switch 40 Road surface illumination system 41 Road surface illumination control device 42 Road surface illumination unit 50 Backup lamp 60 Image generation device 61 processors 62 Memory section 62a Program 62b Coordinate group 63 memory 64a Image acquisition unit 64b Image generation unit 64c adjustment part 64d Synthetic image generation unit 64e Display control unit 64f Road surface illumination control device control unit A1, A10, A11 road surface drawing patterns A2 Peripheral area AX 11 optical axis GL Guidelines I1 First image I1 HDR First HDR image I2 Second image I2 HDR Second HDR image I HDR composite image NW In-vehicle network U Pedestrian V vehicle

Claims

1. An image generating device that is mounted on a vehicle and generates an image including a road surface drawing pattern and a surrounding area formed on a road surface by a lighting device mounted on the vehicle, and that is displayed on a display device mounted on the vehicle, an image acquisition unit that acquires a first group of images including the road surface pattern and the surrounding area captured by an imaging device mounted on the vehicle while changing exposure within a first gain range, and a second group of images including the road surface pattern and the surrounding area captured by the imaging device while changing exposure within a second gain range that is shifted by a certain value in a direction that suppresses gain from the first gain range; a storage unit that stores a group of coordinates that define a contour shape corresponding to the road surface drawing pattern; an HDR image generating unit that generates a first HDR image by performing predetermined image processing on a peripheral area image portion outside the contour shape of each of the first image group, and generates a second HDR image by performing predetermined image processing on a road surface drawing pattern image portion inside the contour shape of each of the second image group; a brightness adjustment unit that adjusts the brightness of the second HDR image; an image generating unit that generates a composite image, as the image to be displayed on the display device, by combining a peripheral area image portion outside the contour shape of the first HDR image with a road surface drawing pattern image portion inside the contour shape of the second HDR image after brightness adjustment.

2. The image generating device according to claim 1 , wherein the brightness adjusting unit adjusts the brightness of the second HDR image so that the brightness is similar to that of the first HDR image.

3. 2. The image generating device according to claim 1, wherein when the ambient illuminance detected by an illuminance sensor attached to the vehicle and detecting the ambient illuminance of the vehicle is equal to or lower than a threshold value, the image acquisition unit acquires the first image group and the second image group from the imaging device.

4. 2. The image generating device according to claim 1, wherein the constant value is determined by comparing a peripheral area image portion outside the contour shape of the image captured by the imaging device with a road surface drawing pattern image portion inside the contour shape.

5. 2. The image generating device according to claim 1, wherein the contour shape defined by the coordinate group is slightly larger than the outer shape of the road surface drawing pattern formed on a road surface parallel to a horizontal plane.

6. a storage unit that stores in advance the constant value for each brightness difference between the surrounding area and the road surface drawing pattern, 2. The image generating device according to claim 1, wherein when the constant value is required, a luminance difference between the surrounding area and the road surface pattern is calculated, and the constant value corresponding to the luminance difference is read from the storage unit.

7. 1. An image generation method for generating an image that is mounted on a vehicle and includes a road surface drawing pattern and a surrounding area formed on a road surface by a lighting device mounted on the vehicle, and is displayed on a display device mounted on the vehicle, comprising: a first group of images including the road surface pattern and the surrounding area captured by an imaging device mounted on the vehicle while changing exposure within a first gain range, and a second group of images including the road surface pattern and the surrounding area captured by the imaging device while changing exposure within a second gain range that is shifted by a certain value in a direction that suppresses gain from the first gain range, generating a first HDR image by performing predetermined image processing based on a peripheral area image portion outside the contour shape defined by the coordinate group stored in a storage unit for each of the first image group, and generating a second HDR image by performing predetermined image processing based on a road surface drawing pattern image portion inside the contour shape for each of the second image group; adjusting the brightness of the second HDR image; An image generation method for generating a composite image, as the image to be displayed on the display device, by combining a peripheral area image portion outside the contour shape of the first HDR image with a road surface drawing pattern image portion inside the contour shape of the second HDR image after brightness adjustment.

8. An image generation program that is mounted on a vehicle and generates an image that includes a road surface drawing pattern and a surrounding area formed on a road surface by a lighting device mounted on the vehicle and is displayed on a display device mounted on the vehicle, a process of acquiring a first group of images including the road surface pattern and the surrounding area captured by an imaging device mounted on the vehicle while changing exposure within a first gain range, and a second group of images including the road surface pattern and the surrounding area captured by the imaging device while changing exposure within a second gain range that is shifted by a certain value in a direction to suppress gain from the first gain range; generating a first HDR image by performing predetermined image processing based on a peripheral area image portion outside the contour shape defined by the coordinate group stored in a storage unit for each of the first image group, and generating a second HDR image by performing predetermined image processing based on a road surface drawing pattern image portion inside the contour shape for each of the second image group; adjusting the brightness of the second HDR image; An image generation program that causes a computer to execute a process of generating a composite image, as the image to be displayed on the display device, by combining a peripheral area image portion outside the contour shape of the first HDR image with a road surface drawing pattern image portion inside the contour shape of the second HDR image after brightness adjustment.

Images