Image generation device, image generation method, and image generation program

Abstract

Provided is an image generation device, for example, which is capable of generating an image having no (or almost no) halation even when the brightness difference between a road surface-drawn pattern and the surrounding region is large and exceeds the range that can be accommodated by an imaging device with an HDR function.　The image generation device comprises: an image acquisition unit (64a) that acquires a first group of images which is captured in a first gain range and includes a road surface-drawn pattern and a surrounding region, and a second group of images which is captured in a second gain range and includes the road surface-drawn pattern and the surrounding region, the second gain range being shifted by a fixed value in a direction in which the gain is suppressed from the first gain range; a storage unit that stores a coordinate group that defines a contour shape corresponding to the road surface-drawn pattern; and an HDR image generation unit (64b) that generates a first HDR image on the basis of a surrounding region image portion on the outside of the contour shape of each of the images of the first group of images, and generates a second HDR image on the basis of a road surface-drawn pattern image portion on the inside of the contour shape of each of the images of the second group of images.

Description

Image generation device, image generation method, and image generation program

[0001] The present disclosure relates to an image generation device, an image generation method, and an image generation program.

[0002] A vehicle system is known that, when the gear position is in reverse, displays on an in-vehicle monitor an image (video) including a road surface drawing pattern captured by an imaging device and formed on the road surface behind the vehicle, as well as the surrounding area (surrounding environment) (see, for example, Patent Document 1).

[0003] The present inventors have considered using an imaging device with a high dynamic range (HDR) function. The HDR function is a function that captures multiple images while changing the exposure within a gain range according to the brightness of the subject (or appropriate for the subject), performs HDR processing based on the multiple images, and combines the multiple images to generate an HDR image with an expanded dynamic range.

[0004] Japanese Patent Application Laid-Open No. 2022-108190

[0005] However, if the difference in brightness between the road surface drawing pattern and the surrounding area is large and exceeds the range that can be handled by an imaging device with HDR function, there is a problem that halation may occur in the image including the road surface drawing pattern and the surrounding area (especially in the image portion corresponding to the relatively bright road surface drawing pattern).

[0006] The present disclosure has been made to solve such problems, and aims to provide an image generation device, an image generation method, and an image generation program that can generate images in which no halation occurs (or in which halation occurs very little) even when the difference in brightness between a road surface drawing pattern and the surrounding area is large and exceeds the range that can be handled by an imaging device with HDR functionality.

[0007] The image generating device according to the present disclosure is an image generating device mounted on a vehicle, which generates an image including a road surface drawing pattern and a surrounding area formed on a road surface by a lighting device mounted on the vehicle and which is displayed on a display device mounted on the vehicle, and includes an image acquisition unit that acquires a first group of images including the road surface drawing pattern and the surrounding area captured by an imaging device mounted on the vehicle while changing exposure within a first gain range, and a second group of images including the road surface drawing pattern and the surrounding area captured by the imaging device while changing exposure within a second gain range that is shifted by a certain value in a direction that suppresses gain from the first gain range, and an image acquisition unit that defines a contour shape corresponding to the road surface drawing pattern. the image generation unit generating a first HDR image by performing predetermined image processing based on the peripheral area image portion outside the contour shape of each of the first image group, and generating a second HDR image by performing HDR processing based on the road surface drawing pattern image portion inside the contour shape of each of the second image group; a brightness adjustment unit adjusting the brightness of the second HDR image; and a composite image generation unit generating a composite image, as the image to be displayed on the display device, by combining the peripheral area image portion outside the contour shape of the first HDR image with the road surface drawing pattern image portion inside the contour shape of the second HDR image after brightness adjustment.

[0008] With this configuration, even if the difference in brightness between the road surface drawing pattern and the surrounding area is large and exceeds the range that can be handled by an imaging device with HDR function, it is possible to generate an image in which no halation occurs (or in which halation occurs very little).

[0009] In the image generating device, the brightness adjustment unit may adjust the brightness of the second HDR image so that the brightness is the same as the brightness of the first HDR image.

[0010] In addition, in the above-mentioned image generating device, when the ambient illuminance detected by an illuminance sensor attached to the vehicle and detecting the ambient illuminance of the vehicle is below a threshold value, the image acquisition unit may acquire the first image group and the second image group from the imaging device.

[0011] In addition, in the image generating device, the constant value may be determined by comparing a peripheral area image portion outside the contour shape of the image captured by the imaging device with a road surface drawing pattern image portion inside the contour shape.

[0012] In the image generating device, the contour shape defined by the coordinate group may be slightly larger than the outer shape of the road surface drawing pattern formed on a road surface parallel to a horizontal plane.

[0013] In addition, the image generating device may further include a memory unit that pre-stores the constant value for each brightness difference between the surrounding area and the road surface drawing pattern, and when the constant value is required, the brightness difference between the surrounding area and the road surface drawing pattern may be calculated and the constant value corresponding to that brightness difference may be read out from the memory unit.

[0014] An image generation method according to the present disclosure is an image generation method that is mounted on a vehicle and generates an image including a road surface drawing pattern and a surrounding area formed on a road surface by a lighting device mounted on the vehicle, and that is displayed on a display device mounted on the vehicle, the image generation method comprising: acquiring a first group of images including the road surface drawing pattern and the surrounding area, which are captured by an imaging device mounted on the vehicle while changing exposure within a first gain range; and acquiring a second group of images including the road surface drawing pattern and the surrounding area, which are captured by the imaging device while changing exposure within a second gain range that is shifted by a certain value in a direction that suppresses gain from the first gain range; A first HDR image is generated by performing a predetermined image processing based on the peripheral area image portion outside the contour shape defined by the coordinate group stored in a memory unit for each of the first image group, and a second HDR image is generated by performing HDR processing based on the road surface drawing pattern image portion inside the contour shape for each of the second image group, and the brightness of the second HDR image is adjusted to generate a composite image as the image to be displayed on the display device, which is a composite of the peripheral area image portion outside the contour shape of the first HDR image and the road surface drawing pattern image portion inside the contour shape of the second HDR image after brightness adjustment.

[0015] The image generation program according to the present disclosure is an image generation program that is mounted on a vehicle and generates an image including a road surface drawing pattern and a surrounding area formed on a road surface by a lighting device mounted on the vehicle and that is displayed on a display device mounted on the vehicle, the image generation program including: a process of acquiring a first group of images including the road surface drawing pattern and the surrounding area captured by an imaging device mounted on the vehicle while changing exposure within a first gain range; and a second group of images including the road surface drawing pattern and the surrounding area captured by the imaging device while changing exposure within a second gain range that is shifted by a certain value in a direction that suppresses gain from the first gain range; and generating a second HDR image by performing HDR processing on a road surface drawing pattern image portion inside the contour shape of each of the second image group; adjusting the brightness of the second HDR image; and generating, as the image to be displayed on the display device, a composite image by combining the surrounding area image portion outside the contour shape of the first HDR image and the road surface drawing pattern image portion inside the contour shape of the second HDR image after brightness adjustment.

[0016] The present disclosure makes it possible to provide an image generation device, an image generation method, and an image generation program that can generate images in which no halation occurs (or occurs very little) even when the difference in brightness between a road surface drawing pattern and the surrounding area is large and exceeds the range that can be handled by an imaging device with HDR functionality.

[0017] 2A is a schematic diagram of a vehicle system 1 including an image generating device 60. (a) An example of a road surface drawing pattern A1 formed on a road surface behind a vehicle V, and (b) a top view of FIG. 2A. (c) An example of an image (video) displayed on a monitor 31 mounted on a vehicle V. (d) An example of a group of normal images (1 / 60, 2 / 60, ...) captured while changing the exposure within a gain range (e.g., see gain A in FIG. 5) corresponding to the brightness of a peripheral area A2 (or suitable for the peripheral area A2). (e.g., see gain C in FIG. 5) corresponding to the brightness of the peripheral area A2 (or suitable for the peripheral area A2). (f) An example of a group of first images (1 / 60, 3 / 60, ...) captured while changing the exposure within a first gain range (e.g., see gain C in FIG. 5) corresponding to the brightness of the peripheral area A2 (or suitable for the peripheral area A2), and (f) an example of a group of second images (2 / 60, 4 / 60, ...) captured while changing the exposure within a second gain range (e.g., see gain A in FIG. 5) shifted by a "constant value" in a direction to suppress the gain from the first gain range (e.g., see gain C in FIG. 5). (f) A diagram showing the ranges that can be handled by an imaging device 11. 1 is a schematic diagram of a road surface drawing pattern A1 extracted from FIG. 3. FIG. 2 is a diagram showing the relationship between a road surface drawing pattern A1 formed on a road surface parallel to the horizontal plane and a road surface drawing pattern A1 formed on a road surface inclined at an angle θ2 (for example, −1°) with respect to the horizontal plane. FIG. 3 is a diagram showing the relationship between a road surface drawing pattern A1 formed on a road surface parallel to the horizontal plane and a road surface drawing pattern A1 formed on a road surface inclined at an angle θ3 (for example, +1°) with respect to the horizontal plane. FIG. 4 is a functional block diagram of an image generating device 60. FIG. 5 is a schematic diagram of an operation example of the vehicle system 1. FIG. 6 is a flowchart of an operation example of the vehicle system 1. FIG. 7 is a diagram for explaining the effect of the vehicle system 1 (image generating device 60) configured as described above in comparison with a comparative example. FIG. 8 is an example image including a road surface drawing pattern A1 and a pedestrian U in which halation occurs. FIG. 9 is an example image including a road surface drawing pattern A1 and a pedestrian U in which halation does not occur. FIG. 10 is a flowchart of a first modified example of an operation example of the vehicle system 1. FIG. 11 is a flowchart of a second modified example of an operation example of the vehicle system 1. FIG. 12 is a flowchart of a third modified example of an operation example of the vehicle system 1. 1 is a top view of a road surface drawing pattern A10, which is a modified example of the road surface drawing pattern A1; FIG. 2 is an example of the road surface drawing pattern A10 displayed on a monitor 31 mounted on a vehicle V; and FIG. 3 is a top view of a road surface drawing pattern A11, which is a modified example of the road surface drawing pattern A1.10 is an example of a road surface drawing pattern A11 displayed on a monitor 31 mounted on a vehicle V.

[0018] Hereinafter, a vehicle system 1 including an image generating device 60 according to an embodiment of the present disclosure will be described with reference to the accompanying drawings. Corresponding components in each drawing are designated by the same reference numerals, and redundant description will be omitted.

[0019] FIG. 1 is a schematic diagram of a vehicle system 1 including an image generating device 60.

[0020] As shown in FIG. 1, a vehicle system 1 is mounted on a vehicle V such as an automobile.

[0021] First, an outline of the vehicle system 1 will be described.

[0022] When the gear position of the vehicle V is in reverse, the vehicle system 1 turns on a road illumination unit 42 (an example of a lighting device of the present disclosure) mounted on the vehicle V, thereby forming a road surface drawing pattern A1 on the road surface behind the vehicle V. Fig. 2(a) is an example of the road surface drawing pattern A1 formed on the road surface behind the vehicle V. Fig. 2(b) is a top view of Fig. 2(a). The road surface drawing pattern A1 is formed on the road surface behind the vehicle V mainly for the purpose of notifying surrounding people (e.g., pedestrians and other vehicles) of the presence of the reversing vehicle V.

[0023] The vehicle system 1 captures an image including a road surface drawing pattern A1 and a surrounding area A2 (surrounding environment) at, for example, 60 fps using an imaging device 11 mounted on the rear end of the vehicle V. The vehicle system 1 displays the image (video) captured by the imaging device 11 on a monitor 31 (an example of a display device of the present disclosure) mounted on the vehicle V.

[0024] 3 is an example of an image (video) displayed on the monitor 31 mounted on the vehicle V. As shown in Fig. 3, the image (video) displayed on the monitor 31 includes a road surface drawing pattern A1, a surrounding area A2, and guidelines GL for assisting the parking of the vehicle V as it backs up. The driver of the vehicle V backs up the vehicle V while visually checking the screen (video) as needed.

[0025] Next, the vehicle system 1 will be described in detail.

[0026] 1, the vehicle system 1 includes a surrounding information detection unit 10, a vehicle state detection unit 20, an IVI (In-Vehicle Infotainment) system 30, a road surface illumination system 40, a backup lamp 50, and an image generation device 60. These components can communicate with each other via an in-vehicle network NW.

[0027] The surrounding information detection unit 10 includes an imaging device 11 and an illuminance sensor 12 .

[0028] The imaging device 11 is an imaging device that captures an image including a road surface drawing pattern A1 and a surrounding area A2 formed on the road surface behind the vehicle V, and is provided at the rear end of the vehicle V (see Figures 2(a) and 2(b)).

[0029] The imaging device 11 is provided at a height H1 from the road surface (see FIG. 2A) and at the center of the rear end of the vehicle V in the vehicle width direction (see FIG. 2B). The height H1 from the road surface is, for example, 700 mm. The optical axis AX of the imaging device 11 １１ is tilted downward at an angle θ1 with respect to the horizontal. The angle θ1 is, for example, 10°.

[0030] The imaging device 11 includes an imaging element (not shown), which may be, for example, a CCD sensor or a CMOS sensor.

[0031] The imaging device 11 is an imaging device (e.g., a video camera) with an HDR function. The imaging device 11 periodically captures images (e.g., at 60 fps) including the road surface drawing pattern A1 and the surrounding area A2. During this capture, the positional relationship between the imaging device 11 and the road surface illumination unit 42 remains unchanged, so the road surface drawing pattern A1 is displayed at the same position (a fixed position) in each image.

[0032] When the ambient illuminance detected by the illuminance sensor 12 exceeds a threshold value (e.g., about 1 Lx), i.e., when the surrounding environment is bright, the imaging device 11 captures a group of images (hereinafter referred to as a group of normal images) including the road surface painting pattern A1 and the surrounding area A2 while changing the exposure within a gain range (e.g., see gain A in FIG. 5 ) corresponding to the brightness of the surrounding area A2 (or appropriate for the surrounding area A2), just like a typical imaging device. Figure 4A shows an example of a group of normal images (1 / 60, 2 / 60, ...) captured while changing the exposure within a gain range (e.g., see gain A in FIG. 5 ) corresponding to the brightness of the surrounding area A2 (or appropriate for the surrounding area A2). Figure 5 is a diagram showing the range that the imaging device 11 can handle.

[0033] On the other hand, when the ambient illuminance detected by the illuminance sensor 12 is below a threshold value (e.g., about 1 Lx), i.e., when the surrounding environment is dark, the imaging device 11 captures a first group of images including the road surface drawing pattern A1 and the surrounding area A2 while changing the exposure within a first gain range (e.g., see gain C in Figure 5) that corresponds to the brightness of the surrounding area A2 (or is appropriate for the surrounding area A2), and captures a second group of images including the road surface drawing pattern A1 and the surrounding area A2 while changing the exposure within a second gain range (e.g., see gain A in Figure 5) that is shifted by a "constant value" in the direction of suppressing the gain from the first gain range (e.g., see gain C in Figure 5). Figure 4B is an example of a first group of images (1 / 60, 3 / 60, etc.) captured while changing the exposure within a first gain range (e.g., see gain C in Figure 5) that corresponds to the brightness of the peripheral area A2 (or is appropriate for the peripheral area A2), and a second group of images (2 / 60, 4 / 60, etc.) captured while changing the exposure within a second gain range (e.g., see gain A in Figure 5) that is shifted by a "constant value" in the direction of suppressing the gain from the first gain range (e.g., see gain C in Figure 5).

[0034] Note that, although FIG. 4B shows an example in which the first image and the second image are alternately captured, the present invention is not limited to this, and the first image and the second image may be captured in a different order.

[0035] The first gain range (see, for example, gain C in FIG. 5 ) is a gain range that corresponds to the brightness of the peripheral area A2 (or is suitable for the peripheral area A2). On the other hand, the second gain range (see, for example, gain A in FIG. 5 ) is a gain range that is shifted by a "constant value" from the first gain range in a direction that suppresses the gain, i.e., a gain range that corresponds to the brightness of the road surface drawing pattern A1 (or is suitable for the road surface drawing pattern A1). The "constant value" can be specified, for example, as follows. Note that a "constant value" shift in the suppression direction basically means adjusting the gain (GAIN) to shift the center value of the HDR by a "constant value."

[0036] For example, first, the image portion outside the contour shape (contour shape defined by coordinate group 62b) of the image (e.g., the first image) captured by the imaging device 11 (hereinafter referred to as the surrounding area image portion) is compared with the image portion inside the contour shape (contour shape defined by coordinate group 62b) (hereinafter referred to as the road surface drawing pattern image portion), and the brightness difference between the two is calculated.

[0037] Once this brightness difference is determined, it is possible to calculate how much (a "constant value") the gain needs to be shifted in the direction of suppressing it from the first gain range (for example, see gain C in Figure 5) to achieve proper exposure (a gain range suitable for road surface drawing pattern A1).

[0038] Therefore, for example, shift amounts ("constant values") that result in proper exposure corresponding to each of the luminance differences (plurality) between the surrounding area image portion and the road surface drawing pattern image portion are stored in advance (for example, before product shipment) in the storage unit 62 or the like. Then, when a "constant value" is needed, the luminance difference between the surrounding area image portion and the road surface drawing pattern image portion is calculated, and the "constant value" that corresponds to that luminance difference is read from the storage unit 62 or the like. In this way, the "constant value" can be identified. Therefore, it is no longer necessary to calculate the constant value each time, and the calculation load for calculating the constant value can be reduced.

[0039] The normal image group, the first image group, and the second image group captured as described above are transmitted to the image generating device 60 .

[0040] The illuminance sensor 12 is a sensor that detects the ambient illuminance of the vehicle V, and is attached, for example, to a dashboard inside the passenger compartment of the vehicle V. The illuminance (ambient illuminance) detected by the illuminance sensor 12 is transmitted to the image generation device 60.

[0041] The vehicle state detection unit 20 includes a vehicle speed sensor 21 and a gear position sensor 22 .

[0042] The vehicle speed sensor 21 is a sensor that detects the speed of the vehicle V and is attached to a predetermined location on the vehicle V. The vehicle speed detected by the vehicle speed sensor 21 is transmitted to the image generation device 60. Note that the vehicle speed sensor 21 is used in Modification 3, which will be described later, but is not used in this embodiment. To make this clear, the vehicle speed sensor 21 is represented by a dotted rectangle in FIG. 1.

[0043] The gear position sensor 22 is a sensor that detects the gear position of the transmission of the vehicle V, and is attached to a predetermined location on the vehicle V. The gear position detected by the gear position sensor 22 is transmitted to the image generation device 60.

[0044] The IVI (In-Vehicle Infotainment) system 30 includes a monitor 31, map information 32, a GPS sensor 33, and a road surface illumination ON / OFF switch .

[0045] The monitor 31 is an image display device including a display surface such as a liquid crystal display, and is attached, for example, to a dashboard inside the vehicle V. The monitor 31 (display surface) displays various images (or videos). For example, the monitor 31 (display surface) displays an image including the road surface drawing pattern A1 and the surrounding area A2 (a composite image synthesized by a composite image generation unit 64d, which will be described later), and a function display (for example, an audio screen, a navigation screen, or a TV screen).

[0046] The map information 32 and the GPS sensor 33 (GPS receiver) constitute part of a navigation system (not shown). For example, the current position of the vehicle V is calculated based on radio waves received by the GPS sensor 33 from GPS satellites, a map corresponding to the current position of the vehicle V is read from the map information 32, and an image symbolizing the vehicle V and a route to the destination (navigation screen) are displayed on the monitor 31 together with the map. Note that the map information 32 and the GPS sensor 33 are used in a modified example 1 described below, but are not used in this embodiment. To clarify this, the map information 32 and the GPS sensor 33 are represented by dotted rectangles in FIG. 1.

[0047] The road surface illumination ON / OFF switch 34 is a switch for turning the road surface illumination unit 42 on and off, and is attached, for example, to the dashboard inside the passenger compartment of the vehicle V. Note that the road surface illumination ON / OFF switch 34 is used in Modification 2, which will be described later, but is not used in this embodiment. To make this clear, the road surface illumination ON / OFF switch 34 is represented by a dotted rectangle in Figure 1.

[0048] The road surface illumination system 40 includes a road surface illumination control device 41 and a road surface illumination unit 42 .

[0049] When the gear position detected by the gear position sensor 22 is reverse gear, the road surface illumination control device 41, in accordance with control from the image generation device 60 (road surface illumination control device control unit 64f), controls the road surface illumination unit 42 so that the road surface illumination unit 42 is turned on. On the other hand, when the gear position detected by the gear position sensor 22 is other than reverse gear, the road surface illumination control device 41, in accordance with control from the image generation device 60 (road surface illumination control device control unit 64f), controls the road surface illumination unit 42 so that the road surface illumination unit 42 is turned off.

[0050] The road surface illumination unit 42 is turned on (or off) under control of the road surface illumination control device 41, and forms a road surface drawing pattern A1 on the road surface behind the vehicle V. The illuminance of the road surface drawing pattern A1 is, for example, 100 to 300 Lx. The road surface drawing pattern A1 has a trapezoidal shape, as shown in FIG. 2(b), for example. Note that the shape of the road surface drawing pattern A1 is not limited to a trapezoidal shape.

[0051] The road surface illumination units 42 are provided on both the left and right sides of the rear end of the vehicle V. For example, the road surface illumination units 42 are provided at a height H2 (see FIG. 2(a)) from the road surface. The height H2 from the road surface is, for example, 1000 mm. The distance L1 (see FIG. 2(b)) from the imaging device 11 in the vehicle width direction is, for example, 900 mm.

[0052] The number of road surface drawing patterns A1 formed by the road surface illumination unit 42 on the road surface behind the vehicle V is, for example, three (see FIGS. 2A and 2B). The three road surface drawing patterns A1 are arranged along straight lines extending in the fore-and-aft direction of the vehicle (see, for example, the straight lines indicated by Line 1 and Line 2 in FIG. 2) within a range of distance L2 (see FIG. 2A) from the vehicle V toward the rear of the vehicle. Distance L2 is, for example, 2555 mm.

[0053] Although not shown, for example, the road surface illumination unit 42 is configured to include three LED light sources and three projection lenses corresponding to the three LED light sources, and to form the three road surface drawing patterns A1 on the road surface behind the vehicle V by projecting the light from each LED light source (rectangular light-emitting surface) using each projection lens. However, the road surface illumination unit 42 is not limited to this, and may have any configuration as long as it can form one or more (two or four or more) road surface drawing patterns A1 on the road surface behind the vehicle V.

[0054] The backup lamps 50 are typical backup lamps that are turned on when the vehicle V is in reverse gear and turned off when the vehicle V is not in reverse gear. The illuminance of the road surface illuminated by the backup lamps 50 is, for example, 40 Lx or less.

[0055] The image generating device 60 is, for example, an ECU (Electronic Control Unit) including a processor 61, a storage unit 62, a memory 63, and the like.

[0056] The memory unit 62 stores a program 62a and a coordinate group 62b. The program 62a is a program executed by the processor 61. The coordinate group 62b is a group of coordinates (coordinate values) that define the contour shape corresponding to the road surface drawing pattern A1. FIG. 6 is a schematic diagram of the road surface drawing pattern A1 extracted from FIG. 3. In FIG. 6, the inverted trapezoid drawn with a solid line represents the outer shape of the road surface drawing pattern A1. Meanwhile, the inverted trapezoid drawn with a dotted line, which is one size larger, represents the contour shape defined by the coordinate group 62b.

[0057] As shown in Fig. 6, the outline shape defined by the coordinate group 62b (see the inverted trapezoid drawn by the dotted line in Fig. 6) is set to be slightly larger than the outline of the road surface drawing pattern A1 (see the inverted trapezoid drawn by the solid line in Fig. 6) for the following reason.

[0058] 7A is a diagram showing the relationship between a road surface drawing pattern A1 formed on a road surface parallel to the horizontal plane and a road surface inclined at an angle θ2 (e.g., −1°) with respect to the horizontal plane, and FIG. 7B is a diagram showing the relationship between a road surface drawing pattern A1 formed on a road surface parallel to the horizontal plane and a road surface inclined at an angle θ3 (e.g., +1°) with respect to the horizontal plane.

[0059] 7A, the rectangle drawn with solid lines represents the road surface drawing pattern A1 formed on a road surface parallel to the horizontal plane, while the rectangle drawn with dotted lines represents the road surface drawing pattern A1 formed on a road surface inclined at an angle θ2 (e.g., −1°) with respect to the horizontal plane.

[0060] Similarly, in Fig. 7B, the rectangle drawn with solid lines represents the road surface pattern A1 formed on a road surface parallel to the horizontal plane, while the rectangle drawn with dotted lines represents the road surface pattern A1 formed on a road surface inclined at an angle θ3 (e.g., +1°) with respect to the horizontal plane.

[0061] Referring to Figures 7A and 7B, when the road surface behind the vehicle V is inclined at an angle θ2 or θ3 with respect to the horizontal plane, the road surface drawing pattern A1 is formed at a position slightly shifted toward or away from the vehicle V relative to the road surface drawing pattern A1 formed on a road surface parallel to the horizontal plane.

[0062] In order to accommodate the road surface drawing pattern A1 formed at the shifted position as described above, the contour shape defined by the coordinate group 62b (see the inverted trapezoid drawn by dotted lines in Figure 6) is set to be slightly larger than the outer shape of the road surface drawing pattern A1 formed on the road surface parallel to the horizontal plane (see the inverted trapezoid drawn by solid lines in Figure 6).

[0063] For example, the upper width W1 (see Figure 6) between the contour shape defined by the coordinate group 62b and the outer shape of the road surface drawing pattern A1 is 25 mm, the lower width W2 (see Figure 6) is 22 mm, and the width W3 (see Figure 6) on both the left and right sides is 17 mm.

[0064] FIG. 8 is a functional block diagram of the image generating device 60.

[0065] The processor 61 executes a program 62a loaded from a storage unit 62 (e.g., ROM) into a memory 63 (e.g., RAM), thereby functioning as an image acquisition unit 64a, an HDR image generation unit 64b, a brightness adjustment unit 64c, a composite image generation unit 64d, a display control unit 64e, and a road surface illumination control device control unit 64f, as shown in Fig. 8. Some or all of these may be realized by hardware.

[0066] The image acquisition unit 64 a acquires the normal image group, the first image group, and the second image group captured and transmitted by the imaging device 11 .

[0067] When the image acquisition unit 64a acquires a group of normal images, the HDR image generation unit 64b performs predetermined image processing, for example HDR (High Dynamic Range) processing, based on the group of normal images, in the same way as a general imaging device, to generate an HDR image (hereinafter referred to as a normal HDR image) that combines the group of normal images and expands the dynamic range.

[0068] On the other hand, when the image acquisition unit 64a acquires the first image group, the HDR image generation unit 64b performs predetermined image processing based on the peripheral area image portions (range-limited images) outside the outline shapes (outline shapes defined by the coordinate group 62b) of each of the first image group, thereby generating a first HDR image with an expanded dynamic range by combining the peripheral area image portions (group). Below, an example will be described in which the predetermined image processing is HDR processing.

[0069] In addition, when the image acquisition unit 64a acquires the second image group, the HDR image generation unit 64b performs HDR processing based on the road surface drawing pattern image portion (range-limited image) inside the contour shape (contour shape defined by the coordinate group 62b) of each of the second image groups, thereby synthesizing the road surface drawing pattern image portion(s) and generating a second HDR image with an expanded dynamic range.

[0070] As described above, the range to be subjected to HDR processing is limited not to the entire first image but to a portion of the first image, i.e., the peripheral area image portion outside the contour shape (the contour shape defined by the coordinate group 62b). Similarly, the range to be subjected to HDR processing is limited not to the entire second image but to a portion of the second image, i.e., the road surface drawing pattern image portion inside the contour shape (the contour shape defined by the coordinate group 62b). This reduces the calculation load of HDR processing compared to when the entire first image and the entire second image are subjected to HDR processing. Furthermore, it is also possible to relatively reduce the phenomenon in which the image displayed on the monitor 31 momentarily turns white due to HDR processing not being able to keep up.

[0071] The brightness adjustment unit 64c adjusts the brightness of the second HDR image generated by the HDR image generation unit 64b. For example, the brightness adjustment unit 64c adjusts the brightness of the second HDR image (the portion of the road surface drawing pattern image inside the contour shape (the contour shape defined by the coordinate group 62b)) by performing predetermined image processing on the second HDR image so that the brightness is similar to that of the first HDR image.

[0072] For example, suppose the illuminance of the road surface illuminated by the backup lamp 50 is 40 Lx, the illuminance of the road surface drawing pattern A1 formed on the road surface by the road surface illumination unit 42 is 100 Lx, and the illuminance (ambient illuminance) detected by the illuminance sensor 12 is 1 Lx.

[0073] In this case, for example, the brightness adjustment unit 64c adjusts the brightness of the second HDR image (the road surface drawing pattern image portion inside the contour shape (contour shape defined by the coordinate group 62b)) by performing image processing such that the brightness value of each pixel constituting the road surface drawing pattern image portion inside the contour shape (contour shape defined by the coordinate group 62b) of the second HDR image = brightness value of each pixel × (40Lx + 1Lx) / (100Lx + 1Lx).

[0074] The composite image generation unit 64d generates a composite image (video) to be displayed on the monitor 31 by combining the peripheral area image portion outside the contour shape (contour shape defined by the coordinate group 62b) of the first HDR image with the road surface drawing pattern image portion inside the contour shape (contour shape defined by the coordinate group 62b) of the second HDR image after brightness adjustment.

[0075] The display control unit 64e controls the monitor 31 so that the normal HDR image (video) generated by the HDR image generation unit 64b or the composite image (video) generated by the composite image generation unit 64d is displayed. The display control unit 64e also controls the monitor 31 so that a function display (for example, an audio screen, a navigation screen, or a TV screen) is displayed.

[0076] When the gear position detected by the gear position sensor 22 is reverse gear, the road surface illumination control device control unit 64f controls the road surface illumination control device 41 to turn on the road surface illumination unit 42. On the other hand, when the gear position detected by the gear position sensor 22 is other than reverse gear, the road surface illumination control device control unit 64f controls the road surface illumination control device 41 to turn off the road surface illumination unit 42.

[0077] Next, an outline of an example of the operation of the vehicle system 1 having the above configuration will be described.

[0078] FIG. 9 is a schematic diagram of an example of the operation of the vehicle system 1.

[0079] The following processing is mainly performed by the processor 61 executing the program 62 a that has been read from the storage unit 62 into the memory 63 .

[0080] 9, first, a group of first images I1 including the road surface pattern A1 and its surrounding area A2 captured by the imaging device 11 within a first gain range (e.g., see gain C in FIG. 5) is acquired (step S1), and then a group of second images I2 including the road surface pattern A1 and its surrounding area A2 are acquired by the imaging device 11 within a second gain range (e.g., see gain A in FIG. 5), which is shifted by a certain value from the first gain range in a direction that suppresses the gain (step S2). This is achieved by the image acquisition unit 64a.

[0081] Next, HDR processing (step S3) is performed based on the peripheral area image portions (range-limited images) outside the contour shapes (contour shapes defined by the coordinate group 62b) of the first images I1 acquired as described above, thereby combining the peripheral area image portions (groups) to generate a first HDR image I1 with an expanded dynamic range. ＨＤＲ In addition, HDR processing (step S5) is performed based on the road surface drawing pattern image portions (range-limited images) inside the contour shapes (contour shapes defined by the coordinate group 62b) of the second images I2 acquired as described above, thereby generating a second HDR image I2 with an expanded dynamic range by combining the road surface drawing pattern image portions (group). ＨＤＲ (Step S6) The processes of steps S3 to S6 are realized by the HDR image generating unit 64b.

[0082] Next, the second HDR image I2 ＨＤＲ (Step S7). For example, the brightness adjustment unit 64c adjusts the brightness of the second HDR image (the portion of the road surface drawing pattern image inside the contour shape (the contour shape defined by the coordinate group 62b)) by performing predetermined image processing on the second HDR image so that the brightness becomes similar to that of the first HDR image. This is achieved by the brightness adjustment unit 64c.

[0083] Next, by executing a synthesis process (step S8), the first HDR image I1 is displayed as an image (video) on the monitor 31.ＨＤＲ The image portion of the peripheral region outside the contour shape (the contour shape defined by the coordinate group 62b) of the second HDR image I2 after brightness adjustment. ＨＤＲ The road surface drawing pattern image portion inside the contour shape (contour shape defined by the coordinate group 62b) is combined with the road surface drawing pattern image portion inside the contour shape (contour shape defined by the coordinate group 62b). ＨＤＲ Generate.

[0084] Next, an example of the operation of the vehicle system 1 having the above configuration will be described in detail.

[0085] FIG. 10 is a flowchart of an example of the operation of the vehicle system 1.

[0086] The following processing is mainly performed by the processor 61 executing the program 62 a that has been read from the storage unit 62 into the memory 63 .

[0087] First, it is determined whether the gear position is reverse gear or not (step S10). This determination is made based on the gear position detected by the gear position sensor 22.

[0088] Next, if it is determined that the gear position is other than reverse (step S10: NO), a function display (for example, an audio screen, a navigation screen, or a TV screen) is displayed on the monitor 31 (display surface) (step S11). This is realized by the display control unit 64e.

[0089] Next, it is determined whether the ignition (IG) is ON or not (step S12).

[0090] Next, if it is determined that the ignition (IG) is ON (step S12: YES), the process returns to step S10, and the processes from step S10 onward are repeatedly executed. On the other hand, if it is determined that the ignition (IG) is OFF (step S12: NO), the process of FIG. 10 ends.

[0091] On the other hand, if it is determined that the gear position is reverse (step S10: YES), the backup lamps 50 and the road surface illumination unit 42 are turned on (step S13). By turning on the backup lamps 50, the road surface behind the vehicle V is illuminated. In addition, by turning on the road surface illumination unit 42, a road surface drawing pattern A1 is formed on the road surface behind the vehicle V.

[0092] Next, it is determined whether the ambient illuminance is equal to or less than a threshold value (step S14). This determination is made by comparing the illuminance (ambient illuminance) detected by the illuminance sensor 12 with the threshold value. The threshold value may be, for example, a value stored in the storage unit 62.

[0093] If it is determined that the ambient illuminance exceeds a threshold value (for example, about 1 Lx) (step S14: NO), that is, if the ambient environment is bright, a normal image is acquired (step S15). This is achieved by the image acquisition unit 64a.

[0094] Specifically, if it is determined that the ambient illuminance exceeds a threshold value (e.g., about 1 Lx) (step S14: NO), the imaging device 11, like a general imaging device, captures a group of images (a group of normal images) including the road surface drawing pattern A1 and the surrounding area A2 while changing the exposure within a gain range (e.g., see gain A in FIG. 5 ) according to the brightness of the surrounding area A2 (or appropriate for the surrounding area A2). The captured group of normal images is transmitted to the image generation device 60. The image acquisition unit 64a acquires the group of normal images (normal videos) (step S15).

[0095] Next, HDR processing is performed (step S16). This is realized by the HDR image generation unit 64b. Like a general imaging device, the HDR image generation unit 64b performs HDR processing on the normal image group (normal video) acquired in step S15, thereby generating an HDR image (normal HDR image) with an expanded dynamic range by combining the normal image group.

[0096] Next, the rear view of the vehicle is displayed on the monitor 31 (step S17). This is achieved by the display control unit 64e. The rear view of the vehicle displayed here is composed of the normal HDR image generated as a result of the HDR processing in step S16.

[0097] Next, it is determined whether the gear position is reverse gear or not (step S18). This determination is made based on the gear position detected by the gear position sensor 22.

[0098] Next, if the gear position is determined to be reverse gear (step S18: YES), the process returns to step S13, and the processes from step S13 onward are repeatedly executed. On the other hand, if the gear position is determined to be other than reverse gear (step S18: NO), the backup lamps 50 and the road surface illumination unit 42 are turned off (step S19), and the processes from step S11 onward are executed.

[0099] On the other hand, if it is determined in step S14 that the ambient illuminance is equal to or less than the threshold value (for example, about 1 Lx) (step S14: YES), that is, if the ambient environment is dark, a range-limited image is acquired (step S20). This is achieved by the image acquisition unit 64a.

[0100] Specifically, if the ambient illuminance is determined to be equal to or lower than a threshold value (e.g., about 1 Lx) (step S14: YES), the imaging device 11 captures a first group of images I1 while changing the exposure within a first gain range (e.g., see gain C in FIG. 5 ) corresponding to the brightness of the peripheral area A2 (or appropriate for the peripheral area A2), and captures a second group of images I2 while changing the exposure within a second gain range (e.g., see gain A in FIG. 5 ) that is shifted by a “constant value” in a direction that suppresses the gain from the first gain range (e.g., see gain C in FIG. 5 ). The captured first group of images I1 and second group of images I2 are transmitted to the image generating device 60. The image acquiring unit 64a acquires the first group of images I1 and second group of images I2.

[0101] Next, range-limited HDR processing is performed (step S21). This is realized by the HDR image generation unit 64b. The HDR image generation unit 64b performs HDR processing based on the peripheral area image portions (range-limited images) outside the contour shapes (contour shapes defined by the coordinate group 62b) of each of the first images I1 acquired in step S20, thereby combining the peripheral area image portions (group) and generating a first HDR image I1 with an expanded dynamic range. ＨＤＲ Generate.

[0102] In addition, the HDR image generating unit 64b performs HDR processing based on the road surface drawing pattern image portions (range-limited images) inside the contour shapes (contour shapes defined by the coordinate group 62b) of each of the second images I2 acquired in step S20, thereby synthesizing the road surface drawing pattern image portions (group) and generating a second HDR image I2 with an expanded dynamic range. ＨＤＲ Generate.

[0103] Next, the brightness of the second HDR image I2 generated in step S21 is adjusted (step S22). This is achieved by the brightness adjustment unit 64c. ＨＤＲ For example, the brightness adjuster 64c adjusts the brightness of the first HDR image I1 ＨＤＲ The second HDR image I2 ＨＤＲ By performing predetermined image processing on the second HDR image I2 ＨＤＲ The brightness of the road surface pattern image portion inside the outline shape (the outline shape defined by the coordinate group 62b) is adjusted.

[0104] Next, the first HDR image I1 ＨＤＲ and the second HDR image I2 ＨＤＲ (Step S23). This is realized by the composite image generating unit 64d. The composite image generating unit 64d generates the first HDR image I1 as the image (video) to be displayed on the monitor 31. ＨＤＲ The image portion of the peripheral region outside the contour shape (the contour shape defined by the coordinate group 62b) of the second HDR image I2 after brightness adjustment. ＨＤＲ The road surface drawing pattern image portion inside the contour shape (contour shape defined by the coordinate group 62b) is combined with the road surface drawing pattern image portion inside the contour shape (contour shape defined by the coordinate group 62b). ＨＤＲ Generate.

[0105] Next, the rear view of the vehicle is displayed on the monitor 31 (step S17). This is realized by the display control unit 64e. The rear view of the vehicle displayed here is the composite image I generated as a result of the processing of step S23. ＨＤＲ FIG. 3 shows a composite image I displayed on the monitor 31. ＨＤＲ This is an example.

[0106] Thereafter, the same process as in step S18 is repeatedly executed.

[0107] Next, the effects of the vehicle system 1 (image generating device 60) configured as described above will be described in comparison with a comparative example.

[0108] FIG. 11 is a diagram for explaining the effects of the vehicle system 1 (image generating device 60) configured as described above in comparison with a comparative example.

[0109] Comparative Example 1 is an example in which a general imaging device with an HDR function is used as the imaging device 11.

[0110] In Comparative Example 1, multiple images are captured while changing the exposure within a gain range corresponding to the brightness of the subject (for example, see Gain A, Gain B, or Gain C in Figure 11), and HDR processing is performed based on the multiple images to generate an HDR image with an expanded dynamic range by combining the multiple images.

[0111] However, in Comparative Example 1, when the road surface luminance of the road surface pattern A1 is at the bright position (bright position) in FIG. 11 and the illuminance of the surrounding area A2 is at the bright position (dark position) in FIG. 11 , i.e., when the difference in brightness between the two is large and exceeds the range that can be handled by the imaging device of Comparative Example 1 (e.g., see gain C in FIG. 11 ), there is a problem in that halation may occur in the image including the road surface pattern A1 and the surrounding area A2 (especially the image portion corresponding to the relatively bright road surface pattern A1). The reason for this is as follows. That is, in Comparative Example 1, as shown in FIG. 11 , a wide difference in brightness cannot be covered, so processing is performed by determining a priority range by adjusting the gain. Therefore, in the case of a setting that prioritizes dark areas (e.g., see gain C in FIG. 11 ), halation may occur when the image is displayed on the monitor 31 if the luminance (road surface pattern A1) is outside the range that can be handled.

[0112] Comparative Example 2 is an example in which a high-performance imaging device with an HDR function is used as the imaging device 11.

[0113] In Comparative Example 2, multiple images are captured while changing the exposure within a gain range corresponding to the brightness of the subject (for example, see gain D in Figure 11), and HDR processing is performed based on the multiple images, thereby combining the multiple images to generate an HDR image with an expanded dynamic range.

[0114] In this case, the range that the imaging device of Comparative Example 2 can handle is wider than the range that the imaging device 11 of this embodiment can handle and the range that Comparative Example 1 can handle (see gain D in Fig. 11 ). Therefore, even when the road surface luminance of road surface drawing pattern A1 is at a bright position (bright position) in Fig. 11 and the illuminance of surrounding area A2 is at a bright position (dark position) in Fig. 11 , that is, even when there is a large difference in brightness between the two, halation is prevented from occurring in the image that includes road surface drawing pattern A1 and surrounding area A2 (particularly in the image portion that corresponds to the relatively bright road surface drawing pattern A1).

[0115] However, in Comparative Example 2, because the applicable range is wide, the number of images captured while changing the exposure within a gain range corresponding to the brightness of the subject (for example, see gain D in FIG. 11 ) increases, which places a heavy load on the HDR processing. This poses a problem of delays in display on the monitor 31. In addition, image processing performance (for example, a high-performance processor) capable of performing HDR processing on a large number of images at high speed is required, which poses a problem of significantly increased costs.

[0116] In contrast to this, in this embodiment, the range that the imaging device 11 can handle is the same as the range that the imaging device of Comparative Example 1 can handle.

[0117] However, in this embodiment, the image capture device 11 captures multiple images (first image group) while changing the exposure within a gain range corresponding to the brightness of the surrounding area A2 (e.g., see gain C in FIG. 11 ), and performs HDR processing based on the multiple images (first image group) to generate a first HDR image with an expanded dynamic range (step S100). The image capture device 11 also captures multiple images (second image group) while changing the exposure within a gain range corresponding to the brightness of the road surface pattern A1 (e.g., see gain A in FIG. 11 ), and performs HDR processing based on the multiple images (second image group) to generate a second HDR image with an expanded dynamic range (step S101). The second HDR image is then darkened by a certain value through image processing by the brightness adjustment unit 64c (step S102). Then, the composite image generation unit 64d generates a composite image by combining the peripheral area image portion outside the contour shape (contour shape defined by the coordinate group 62b) of the first HDR image with the road surface drawing pattern image portion inside the contour shape (contour shape defined by the coordinate group 62b) of the second HDR image after brightness adjustment, and this composite image (video) is displayed on the monitor 31 (step S103).

[0118] Therefore, according to this embodiment, even when the road surface luminance of the road surface pattern A1 is at the bright position (bright position) in Fig. 11 and the illuminance of the surrounding area A2 is at the bright position (dark position) in Fig. 11, i.e., even when the difference in brightness between the two is large and exceeds the range that can be handled by the imaging device 11 (for example, see gain C in Fig. 11), halation is prevented from occurring in the image including the road surface pattern A1 and the surrounding area A2 (especially in the image portion corresponding to the relatively bright road surface pattern A1), thereby solving the problem of the above-mentioned comparative example 1.

[0119] Furthermore, according to this embodiment, the range that the imaging device 11 can handle is the same as that of the imaging device of Comparative Example 1, but is narrower than that of the imaging device of Comparative Example 2, so the number of images that can be captured can be smaller than when using the imaging device of Comparative Example 2. This also solves the problem of Comparative Example 2.

[0120] As described above, according to this embodiment, even if the difference in brightness between the road surface drawing pattern A1 and the surrounding area A2 (surrounding environment) is large and exceeds the range that can be handled by the imaging device 11 with HDR function, it is possible to generate an image (synthetic image) in which no halation occurs (or in which halation occurs very little).

[0121] If the composite image generated as described above includes the road surface drawing pattern A1 and the pedestrian U and halation occurs (see FIG. 12A ), even if a predetermined image processing is performed on the composite image, it may be impossible to distinguish between the road surface drawing pattern A1 and the pedestrian U, resulting in a false detection of the pedestrian U. FIG. 12A is an example of an image including the road surface drawing pattern A1 and the pedestrian U in which halation occurs.

[0122] In contrast, according to this embodiment, a composite image (see FIG. 12B ) without halation is finally generated. FIG. 12B illustrates an example image including a road surface drawing pattern A1 and a pedestrian U without halation. Therefore, for example, when a parking support brake, which is part of an ADAS (Advanced Driver-Assistance Systems), performs predetermined image processing on the composite image (a composite image including the road surface drawing pattern A1 and the pedestrian U), it becomes possible to distinguish between the road surface drawing pattern A1 and the pedestrian U. This reduces false detection of the pedestrian U. The parking support brake is a system that issues a warning or performs automatic brake control when it detects a pedestrian U behind the vehicle V based on an image captured by an imaging device (e.g., an image (composite image) captured by the imaging device 11). In this way, the image generation device 60 and the image generation method of this embodiment can be used in conjunction with an ADAS. Note that the composite image finally generated in this embodiment can also be used in systems other than the parking support brake. That is, the image generation device 60 and the image generation method of this embodiment can also be used in conjunction with devices or systems other than the ADAS.

[0123] Next, Modification 1 will be described.

[0124] FIG. 13 is a flowchart of a first modification of the operation example of the vehicle system 1.

[0125] 13 , if it is determined that the gear position is reverse (step S10: YES), it is determined whether the current position of the vehicle V is within the road surface illumination area (step S30). If it is determined that the current position of the vehicle V is outside the road surface illumination area (step S30: NO), only the backup lamps 50 are turned on (step S31), and then, as in the above embodiment, the processing from step S15 onward may be executed. On the other hand, if it is determined that the current position of the vehicle V is within the road surface illumination area (step S30: YES), as in the above embodiment, the processing from step S13 onward may be executed. In this case, in step S19A, only the backup lamps 50 may be turned off. The road surface illumination area is, for example, an area other than a public road, such as a parking lot.

[0126] The determination in step S30, i.e., whether the current position of the vehicle V is within the area where road surface illumination is possible, can be made, for example, by comparing the current position of the vehicle V calculated based on radio waves from GPS satellites received by the GPS sensor 33 with information identifying the area where road surface illumination is possible contained in the map information 32 (for example, a group of coordinates identifying the area where road surface illumination is possible).

[0127] According to the first modification, in addition to the effects of the above embodiment, a composite image (video) can be displayed on the monitor 31 only when the vehicle V is present in the road surface illumination area.

[0128] Next, a second modification will be described.

[0129] FIG. 14 is a flowchart of a second modification of the operation example of the vehicle system 1.

[0130] 14 , if it is determined that the gear position is reverse (step S10: YES), it is further determined whether the road surface illumination ON / OFF switch 34 is ON (step S30A). If it is determined that the road surface illumination ON / OFF switch 34 is OFF (step S30A: NO), only the backup lamps 50 may be turned on (step S31), and then, as in the above embodiment, the processing from step S15 onward may be executed. On the other hand, if it is determined that the road surface illumination ON / OFF switch 34 is ON (step S30A: YES), as in the above embodiment, the processing from step S13 onward may be executed. In this case, in step S19A, only the backup lamps 50 may be turned off.

[0131] According to the second modification, in addition to the effects of the above embodiment, it is possible to further reduce unnecessary processing.

[0132] Next, Modification 3 will be described.

[0133] FIG. 15 is a flowchart of a third modified example of the operation example of the vehicle system 1.

[0134] 15 , if it is determined that the gear position is reverse (step S10: YES), it is determined whether the vehicle speed of the vehicle V detected by the vehicle speed sensor 21 is equal to or less than a threshold value (step S30B). If it is determined that the vehicle speed of the vehicle V exceeds the threshold value (step S30B: NO), only the backup lamps 50 may be turned on (step S31), and then, as in the above embodiment, the processing from step S15 onward may be executed. On the other hand, if it is determined that the vehicle speed of the vehicle V is equal to or less than the threshold value (step S30B: YES), as in the above embodiment, the processing from step S13 onward may be executed. In this case, only the backup lamps 50 may be turned off in step S19A.

[0135] According to the third modification, in addition to the effects of the above embodiment, it is possible to further reduce unnecessary processing when the vehicle V is traveling at a high speed.

[0136] Next, a modified example of the road surface drawing pattern A1 will be described.

[0137] 16A is a top view of a road surface drawing pattern A10, which is a modified example of the road surface drawing pattern A1. FIG. 16B is an example of the road surface drawing pattern A10 displayed on the monitor 31 mounted on the vehicle V.

[0138] In the above embodiment, an example has been described in which the trapezoidal road surface drawing pattern A1 is used as the road surface drawing pattern, but this is not limiting. In other words, road surface drawing patterns of various shapes other than trapezoidal shapes (e.g., see the road surface drawing pattern A10 shown in FIG. 16A and the road surface drawing pattern A11 shown in FIG. 17A ) may also be used as the road surface drawing pattern. The dotted line in FIG. 16B represents a contour shape that is slightly larger than the outline of the road surface drawing pattern A10, as defined by the coordinate group 62b.

[0139] Next, modified examples of the formation location (drawing location) of the road surface drawing pattern A1 will be described.

[0140] 17A is a top view of a road surface drawing pattern A11, which is a modified example of the road surface drawing pattern A1. FIG. 17B is an example of the road surface drawing pattern A11 displayed on the monitor 31 mounted on the vehicle V.

[0141] In the above embodiment, an example has been described in which the road surface drawing pattern A1 is formed on the road surface behind the vehicle V, but this is not limiting. For example, as shown in Figures 17A and 17B, the road surface drawing pattern A11 may be formed (drawn) on the road surface ahead of the vehicle V. Note that the dotted line in Figure 17B represents a contour shape that is slightly larger than the outline of the road surface drawing pattern A11, as defined by the coordinate group 62b. Furthermore, although not shown, the road surface drawing pattern A11 may be formed (drawn) on a location other than the road surface behind the vehicle V or the road surface ahead of the vehicle V (for example, on the road surface to the side of the vehicle V).

[0142] All the numerical values ​​shown in the above embodiment are merely examples, and it goes without saying that other appropriate numerical values ​​can be used.

[0143] The above-described embodiments are merely examples in all respects. The present disclosure should not be construed as being limited by the description of the above-described embodiments. The present disclosure can be implemented in various other forms without departing from the spirit or main characteristics thereof.

[0144] This application claims priority based on Japanese Patent Application No. 2024-18630, filed February 9, 2024, the disclosure of which is incorporated herein by reference in its entirety.

[0145] REFERENCE SIGNS LIST 1 Vehicle system 10 Surrounding information detection unit 11 Imaging device 12 Illuminance sensor 20 Vehicle state detection unit 21 Vehicle speed sensor 22 Gear position sensor 30 IVI system 31 Monitor 32 Map information 33 GPS sensor 34 Road surface illumination ON / OFF switch 40 Road surface illumination system 41 Road surface illumination control device 42 Road surface illumination unit 50 Back lamp 60 Image generation device 61 Processor 62 Storage unit 62a Program 62b Coordinate group 63 Memory 64a Image acquisition unit 64b Image generation unit 64c Adjustment unit 64d Composite image generation unit 64e Display control unit 64f Road surface illumination control device control unit A1, A10, A11 Road surface drawing pattern A2 Surrounding area AX １１　 Optical axis GL Guideline I1 First image I1 ＨＤＲ First HDR image I2 Second image I2 ＨＤＲ Second HDR image I ＨＤＲ Synthetic image NW Vehicle network U Pedestrian V Vehicle

Claims

1. An image generating device mounted on a vehicle that generates images including a road surface drawing pattern and its surrounding area formed on a road surface by a lighting device mounted on the vehicle and that are to be displayed on a display device mounted on the vehicle, comprising: an image acquisition unit that acquires a first group of images including the road surface drawing pattern and its surrounding area captured by an imaging device mounted on the vehicle while changing exposure within a first gain range, and a second group of images including the road surface drawing pattern and its surrounding area captured by the imaging device while changing exposure within a second gain range that is shifted by a certain value in a direction that suppresses gain from the first gain range; a memory unit that stores a group of coordinates that define a contour shape corresponding to the road surface drawing pattern; an HDR image generating unit that generates a first HDR image by performing predetermined image processing on a peripheral area image portion outside the contour shape of each of the first group of images, and generates a second HDR image by performing predetermined image processing on a road surface drawing pattern image portion inside the contour shape of each of the second group of images; and a brightness adjustment unit that adjusts the brightness of the second HDR image. an image generating unit that generates a composite image, as the image to be displayed on the display device, by combining a peripheral area image portion outside the contour shape of the first HDR image with a road surface drawing pattern image portion inside the contour shape of the second HDR image after brightness adjustment.

2. The image generating device according to claim 1, wherein the brightness adjustment unit adjusts the brightness of the second HDR image so that the brightness is similar to that of the first HDR image.

3. An image generating device as described in claim 1, wherein when the ambient illuminance detected by an illuminance sensor attached to the vehicle and detecting the ambient illuminance of the vehicle is below a threshold, the image acquisition unit acquires the first image group and the second image group from the imaging device.

4. An image generating device as described in claim 1, wherein the constant value is determined by comparing the peripheral area image portion outside the contour shape of the image captured by the imaging device with the road surface drawing pattern image portion inside the contour shape.

5. The image generating device according to claim 1, wherein the contour shape defined by the coordinate group is slightly larger than the outer shape of the road surface drawing pattern formed on a road surface parallel to a horizontal plane.

6. An image generating device as described in claim 1, further comprising a memory unit that pre-stores the constant value for each brightness difference between the surrounding area and the road surface drawing pattern, and when the constant value is required, calculates the brightness difference between the surrounding area and the road surface drawing pattern and reads out the constant value corresponding to that brightness difference from the memory unit.

7. An image generation method mounted on a vehicle for generating an image including a road surface pattern and its surrounding area formed on a road surface by a lighting device mounted on the vehicle and displayed on a display device mounted on the vehicle, comprising: acquiring a first group of images including the road surface pattern and its surrounding area captured by an imaging device mounted on the vehicle while changing exposure within a first gain range; and a second group of images including the road surface pattern and its surrounding area captured by the imaging device while changing exposure within a second gain range shifted by a certain value in a direction that suppresses gain from the first gain range; generating a first HDR image by performing predetermined image processing on the surrounding area image portion outside the contour shape defined by a coordinate group stored in a memory unit for each of the first group of images; and generating a second HDR image by performing predetermined image processing on the road surface pattern image portion inside the contour shape for each of the second group of images; and adjusting the brightness of the second HDR images. An image generation method for generating a composite image, as the image to be displayed on the display device, by combining a peripheral area image portion outside the contour shape of the first HDR image with a road surface drawing pattern image portion inside the contour shape of the second HDR image after brightness adjustment.

8. An image generation program that is mounted on a vehicle and generates an image including a road surface drawing pattern and its surrounding area formed on a road surface by a lighting device mounted on the vehicle, and that is displayed on a display device mounted on the vehicle, comprising: a process for acquiring a first group of images including the road surface drawing pattern and its surrounding area, which are captured by an imaging device mounted on the vehicle while changing the exposure within a first gain range, and a second group of images including the road surface drawing pattern and its surrounding area, which are captured by the imaging device while changing the exposure within a second gain range that is shifted by a certain value in a direction that suppresses the gain from the first gain range; a process for generating a first HDR image by performing predetermined image processing on the surrounding area image portion outside the contour shape defined by a coordinate group stored in a memory unit for each of the first group of images, and a process for generating a second HDR image by performing predetermined image processing on the road surface drawing pattern image portion inside the contour shape for each of the second group of images; a process for adjusting the brightness of the second HDR images; An image generation program that causes a computer to execute a process of generating a composite image, as the image to be displayed on the display device, by combining a peripheral area image portion outside the contour shape of the first HDR image with a road surface drawing pattern image portion inside the contour shape of the second HDR image after brightness adjustment.

Images