Driving assistance device and vehicle

Abstract

To provide a driving assistance device and a vehicle which improve detection easiness or easiness of visual recognition of a visible laser pattern.SOLUTION: A driving assistance device according to one embodiment of the present disclosure can execute the following two functions: (1) controlling irradiation of a travel road surface in front of a vehicle with a visible laser pattern, by a visible laser light source, on the basis of an irradiation pattern set value of the visible laser light source; and (2) controlling irradiation with a light distribution variable type headlamp, on the basis of an initial set value of a non-irradiation region or an irradiation region of the light distribution variable type headlamp, in which the initial set value is set so that a predetermined region including the visible laser pattern emitted to the travel road surface becomes the non-irradiation region, or does not overlap the irradiation region, when assuming that the travel load surface in front of the vehicle is a flat surface.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to a driving assistance device mounted on a vehicle, and a vehicle equipped with such a driving assistance device. [Background technology]

[0002] There may be unevenness on the road surface, such as depressions or fallen obstacles. In this case, when a vehicle goes over the unevenness, depending on the depth and height of the unevenness, not only may the ride be uncomfortable for the occupants, but the vehicle may also get stuck in the depression or collide with an obstacle and run off the road. As a result, driving safety may be reduced.

[0003] For example, Patent Documents 1 and 2 disclose that a laser beam is irradiated onto the road surface ahead of a vehicle in order to detect the shape of the road surface and obstacles on the road. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6962464 [Patent Document 2] Japanese Patent Application Laid-Open No. 2010-18080 Summary of the Invention [Means for solving the problem]

[0005] A driving assistance device according to an embodiment of the present disclosure includes an acquisition unit capable of acquiring image data of an area ahead of a vehicle obtained by imaging, a control unit capable of controlling a visible laser light source and an adjustable light distribution headlight, and a storage unit that stores an irradiation pattern setting value of the visible laser light source and an initial setting value of a non-irradiated area or an illuminated area of ​​the adjustable light distribution headlight. The control unit is capable of performing the following two functions. (1) Controlling the irradiation of a visible laser pattern onto a road surface ahead of the vehicle by the visible laser light source based on the irradiation pattern setting value of the visible laser light source. (2) Controlling the illumination of the variable light distribution headlight based on the initial setting value of the non-illuminated area or illuminated area of ​​the variable light distribution headlight. Here, the initial setting value is set so that, assuming that the road surface ahead of the vehicle is flat, a predetermined area including the visible laser pattern projected onto the road surface becomes a non-illuminated area or does not overlap with the illuminated area.

[0006] A vehicle according to an embodiment of the present disclosure includes a driving assistance device, a visible laser light source, and a variable light distribution headlight. The driving assistance device has a configuration common to the driving assistance device described above. [Brief explanation of the drawings]

[0007] The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification. The drawings illustrate one embodiment and, together with the description, serve to explain the principles of the disclosure.

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the appearance of a front portion of a vehicle according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a diagram showing an example of the internal configuration of the left and right headlights of the vehicle shown in FIG. [Figure 3] FIG. 3 is a diagram showing an example of a view ahead of the vehicle when viewed from the driver's seat of the vehicle shown in FIG. [Figure 4] FIG. 4 is a diagram showing an example of the vehicle and the road surface as viewed from above the vehicle of FIG. [Figure 5] FIG. 5 is a diagram showing an example of a view ahead of the vehicle when viewed from the driver's seat of the vehicle of FIG. [Figure 6]FIG. 6 is a diagram showing an example of the vehicle and the road surface as viewed from above the vehicle of FIG. [Figure 7] FIG. 7 is a diagram illustrating an example of functional blocks of a driving control device mounted on the vehicle of FIG. [Figure 8] FIG. 8 is a diagram illustrating an example of a network environment capable of communicating with the driving control device of FIG. [Figure 9] Figure 9 is a diagram showing an example of a visible laser pattern and headlight light contained in an image obtained by capturing an image in front of the vehicle while the vehicle in Figure 1 is irradiating the visible laser pattern and headlight light onto the road surface. [Figure 10] FIG. 10 is a diagram showing an example of a visible laser pattern and headlight light contained in an image obtained by capturing an image ahead of the vehicle when the vehicle in FIG. 1 is irradiating the visible laser pattern and headlight light onto the road surface. [Figure 11] FIG. 11 is a diagram for explaining a method for calculating the depth and position of the groove. [Figure 12] FIG. 12 is a diagram for explaining a method for calculating the height and position of an obstacle. [Figure 13] FIG. 13 is a diagram showing an example of how an image obtained by capturing an image of the area ahead of the vehicle of FIG. 1 is displayed on a display screen of the vehicle of FIG. [Figure 14] Figure 14 is a diagram showing an example of a visible laser pattern and headlight light contained in an image obtained by capturing an image in front of the vehicle while the vehicle in Figure 1 is irradiating the visible laser pattern and headlight light onto the road surface. [Figure 15] Figure 15 is a diagram showing an example of a visible laser pattern and headlight light contained in an image obtained by capturing an image in front of the vehicle while the vehicle in Figure 1 is irradiating the visible laser pattern and headlight light onto the road surface. [Figure 16] Figure 16 is a diagram showing an example of a visible laser pattern and headlight light contained in an image obtained by capturing an image in front of the vehicle while the vehicle in Figure 1 is irradiating the visible laser pattern and headlight light onto the road surface. [Figure 17] Figure 17 is a diagram showing an example of a visible laser pattern and headlight light contained in an image obtained by capturing an image in front of the vehicle while the vehicle in Figure 1 is irradiating the visible laser pattern and headlight light onto the road surface. [Figure 18] FIG. 18 is a diagram for explaining an example of a driving assistance procedure in the vehicle of FIG. [Figure 19] FIG. 19 is a diagram for explaining an example of the driving assistance procedure following FIG. DETAILED DESCRIPTION OF THE INVENTION

[0009] There may be unevenness on the road surface, such as depressions or fallen obstacles. In this case, when a vehicle goes over the unevenness, depending on the depth and height of the unevenness, not only may the ride be uncomfortable for the occupants, but the vehicle may also get stuck in the depression or collide with an obstacle and run off the road. As a result, driving safety may be reduced.

[0010] In order to make it easier to detect or see unevenness in front of the vehicle, it is conceivable to project a visible laser pattern onto the road surface in front of the vehicle and cause the visible laser pattern to bend or break in accordance with the unevenness in front of the vehicle. However, at night, the influence of the light from the vehicle's headlights, neon signs on signs, lights leaking from stores, etc. can make it difficult for a sensor such as a camera to detect the visible laser pattern or for the driver of the vehicle to see the visible laser pattern. It is desirable to provide a driving assistance device and a vehicle that improve the ease of detection or visibility of the visible laser pattern.

[0011] Some exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Note that the following description illustrates one specific example of the present disclosure and should not be construed as limiting the present disclosure. For example, each element, including numerical values, shapes, materials, parts, the position of each part, and the connection method of each part, is merely an example and should not be construed as limiting the present disclosure. Furthermore, in the following exemplary embodiments, components not described in independent claims based on the highest concept of the present disclosure are optional and may be provided as needed. The drawings are schematic and are not intended to be drawn to scale. Throughout this specification and the drawings, components having substantially the same function and configuration are designated by the same reference numerals, and redundant description will be omitted. Furthermore, components not directly related to one embodiment of the present disclosure are not shown in the drawings.

[0012] [composition] FIG. 1 illustrates an example of the appearance of a front portion of a vehicle 100 according to an embodiment of the present disclosure. The drive system of the vehicle 100 is not particularly limited, and the vehicle 100 can be driven, for example, by at least one of an engine and a motor. As shown in FIG. 1, the vehicle 100 includes a pair of left and right headlights (a right headlight HLa and a left headlight HLb) at the front portion, and a front window FW. A stereo camera 110 including a main camera 111 and a sub-camera 112 is provided in the vehicle interior CR, which is visible from the outside through the front window FW. FIG. 1 illustrates an example of the vehicle 100 traveling on a road surface 300.

[0013] 2 shows an example of the internal configuration of a pair of headlights (right headlight HLa and left headlight HLb). The right headlight HLa is provided with a variable light distribution right headlight 130a, and the left headlight HLb is provided with a variable light distribution left headlight 130b. The variable light distribution right headlights 130a and 130b are headlights known as ADB (Adaptive Driving Beam) that can illuminate the area ahead of the vehicle 100 while reducing the amount of light irradiated onto oncoming vehicles and preceding vehicles based on image data Ia or distance image data Ib obtained from a stereo camera 110 (described below). In this embodiment, the variable light distribution right headlights 130a, 130b are headlights that can illuminate the area ahead of the vehicle 100 based on image data Ia or distance image data Ib obtained from the stereo camera 110 described below, while further reducing the amount of light irradiated onto a visible laser pattern LP (described below) irradiated onto the road surface ahead of the vehicle 100.

[0014] The variable light distribution right headlamp 130a, 130b includes, for example, a light source capable of emitting visible light, a light emission control driver capable of controlling the light emission of the light source, an optical system capable of scanning the light emitted from the light source on the road surface 300, and a scan control driver capable of controlling the scanning of the laser light by the optical system. The light emission control driver is capable of controlling the light emission of the light source under control of a control unit 210. The scan control driver is capable of controlling the operation of the optical system under control of the control unit 210. The light source includes, for example, a laser diode (LD) or a light emitting diode (LED). The optical system includes, for example, a plurality of micro light emitting diodes (LEDs) arranged two-dimensionally. The configuration of the variable light distribution right headlamp 130a, 130b is not limited to the above configuration.

[0015] Of the pair of headlights (right headlight HLa, left headlight HLb), the headlight farther from the roadside (in Japan, the right headlight HLa) is provided inside with a visible laser irradiator 120. The visible laser irradiator 120 may be provided at a location on the front of the vehicle 100 that is different from the location described above. The visible laser irradiator 120 irradiates a laser beam L onto a road surface 300 ahead of the vehicle 100, thereby generating a linear visible laser pattern LP on the road surface 300.

[0016] Fig. 3 shows an example of the view ahead of vehicle 100 when the driver looks ahead from the driver's seat while vehicle 100 is traveling on road surface 300. Fig. 4 shows an example of the appearance of vehicle 100 and road surface 300 when viewed from above vehicle 100. For ease of explanation, broken lines are drawn in Figs. 3 and 4 at locations where the left and right tires TR of vehicle 100 are expected to pass (expected passage areas 310). Figs. 3 and 4 also show, by way of example, the presence of side strips 320 on both sides of road surface 300.

[0017] As shown in FIGS. 3 and 4 , a depression 330 exists in front of the vehicle 100. However, it may not be easy for the driver to see the depression 330. Furthermore, it may not be easy for the stereo camera 110, which will be described later, to detect the depression 330. For example, during a snowstorm, at twilight, or at night when there are no street lights, it is extremely difficult for the driver to see the depression 330, and it is also extremely difficult for the stereo camera 110 to detect the depression 330. Therefore, as shown in FIGS. 3 and 4 , the vehicle 100 uses the laser irradiation unit 120 to irradiate a laser beam L onto a road surface 300 ahead of the vehicle, thereby generating a linear visible laser pattern LP on the road surface 300. The visible laser pattern LP makes the depression 330 appear three-dimensionally, thereby making it easier for the driver to see the depression 330 and for the stereo camera 110 to detect it.

[0018] 3 and 4 illustrate an example in which the visible laser pattern LP is projected onto a portion of the travel road surface 300 where the predicted passage area 310 is likely to exist. When there are no irregularities on the travel road surface 300, the visible laser pattern LP is a straight line without the bent line LPa or interruption γ described below. When there are irregularities on the travel road surface 300, the visible laser pattern LP includes, in addition to a straight line, bent lines LPa and LPb and interruptions α and γ described below.

[0019] The visible laser pattern LP generated by irradiating the laser beam L onto the travel road surface 300 is linear in order to reduce the image processing load. Furthermore, the visible laser pattern LP extends in a direction that obliquely intersects with the longitudinal direction of the expected passage area 310 (i.e., the traveling direction of the vehicle 100) in order to make it easier to detect bends and discontinuities in the visible laser pattern LP on the image.

[0020] As shown in FIGS. 3 and 4 , when light emitted from the variable light distribution right headlights 130a and 130b illuminates the headlight illumination area α, a visible laser pattern LP is illuminated within the headlight illumination area α. At this time, the light emitted from the variable light distribution right headlights 130a and 130b is superimposed on the visible laser pattern LP. Therefore, the contrast of the visible laser pattern LP is significantly reduced compared to when light emitted from the variable light distribution right headlights 130a and 130b is not illuminating the headlight illumination area α. Furthermore, when the vehicle 100 is traveling in a city, neon signs and lights leaking from stores are also superimposed on the visible laser pattern LP. This further reduces the contrast of the visible laser pattern LP. As a result, the visible laser pattern LP may be obscured by the light emitted from the variable light distribution right headlights 130a and 130b, the neon signs, and the light leaking from stores, making it difficult for the driver to see the visible laser pattern LP or for the stereo camera 110 to detect the visible laser pattern LP.

[0021] Therefore, in this embodiment, the variable light distribution right headlights 130a, 130b are capable of illuminating the area ahead of the vehicle 100 while suppressing the amount of light irradiation on the visible laser pattern LP irradiated on the road surface ahead of the vehicle 100, based on the image data Ia or distance image data Ib obtained from the stereo camera 110. The variable light distribution right headlights 130a, 130b are capable of irradiating the road surface ahead of the vehicle 100 with light so that a predetermined area including the visible laser pattern LP becomes a headlight non-irradiation area β or so that it does not overlap with the headlight irradiation area α, as shown in FIGS. 5 and 6, for example. The configuration of the cruise control device 1000 for realizing such light irradiation in the variable light distribution right headlights 130a, 130b will be described in detail below.

[0022] Fig. 7 shows an example of functional blocks of the cruise control device 1000. The vehicle 100 is equipped with the cruise control device 1000. The cruise control device 1000 corresponds to a specific example of a "driving assistance device" according to an embodiment of the present disclosure. Fig. 8 shows an example of functional blocks of a control device 2000 provided in a network environment NW to which the cruise control device 1000 is connected via wireless communication.

[0023] The control device 2000 is capable of sequentially integrating and updating the road map information transmitted from the cruise control devices 1000 of the respective vehicles, and transmitting the updated road map information to the respective vehicles. The control device 2000 includes, for example, a road map information integration ECU 510 and a transceiver 520.

[0024] The road map information integration_ECU 510 is capable of sequentially updating road map information surrounding a vehicle on a road by integrating road map information collected from multiple vehicles via the transceiver 520. The road map information is, for example, a dynamic map, and has static information and quasi-static information that mainly constitute road information, and quasi-dynamic information and dynamic information that mainly constitute traffic information.

[0025] Static information that makes up road information includes information that needs to be updated within one month, such as roads, road structures, lane information, road surface information, and permanent traffic regulations. "Roads" include, for example, road locations and shapes, intersections, and road attributes (e.g., national roads, prefectural roads, city roads, private roads, priority roads, non-priority roads, general roads, and expressways). "Road structures" include, for example, traffic signs, traffic lights, convex mirrors, and pedestrian bridges.

[0026] The quasi-static information that constitutes road information is composed of information that needs to be updated every hour, such as traffic regulation information due to road construction or events, wide-area weather information, and traffic congestion forecasts.

[0027] The semi-dynamic information that makes up traffic information is composed of information that must be updated within one minute, such as the actual traffic congestion situation at the time of observation, driving restrictions, temporary driving obstructions such as fallen objects and obstacles, actual accident conditions, and narrow-area weather information.

[0028] The dynamic information that constitutes the traffic information is composed of information that needs to be updated every second, such as information transmitted and exchanged between moving objects, information on currently displayed traffic signals, information on pedestrians and bicycles at intersections, information on vehicles traveling on roads, etc. Such road map information is maintained and updated periodically until the next information is received from each vehicle, and the updated road map information is transmitted to each vehicle as appropriate via the transceiver 520.

[0029] As shown in FIG. 7 , the cruise control device 1000 includes, for example, a stereo camera 110, a visible laser irradiator 120, a headlight irradiator 130, a vehicle state quantity sensor 140, a GNSS receiver 150, a transceiver 160, a control flag input unit 170, a notification unit 180, and a high-precision road map DB 190. The visible laser irradiator 120 corresponds to a specific example of a “visible laser light source” according to an embodiment of the present disclosure. The headlight irradiator 130 corresponds to a specific example of a “variable light distribution headlight” according to an embodiment of the present disclosure. The headlight irradiator 130 includes a variable light distribution right headlight 130a and a variable light distribution left headlight 130b. The cruise control device 1000 may further include components other than those shown in FIG. 7 . FIG. 7 illustrates an example of a portion of the configuration of the cruise control device 1000.

[0030] The stereo camera 110 is fixed, for example, to the upper center of the vehicle interior CR and includes, for example, a main camera 111 and a sub-camera 112. The main camera 111 and the sub-camera 112 are autonomous sensors that sense the real space ahead of the vehicle 100. The main camera 111 and the sub-camera 112 are, for example, arranged at symmetrical positions on either side of the central portion in the width direction of the vehicle 100, and are capable of capturing stereo images of the area ahead of the vehicle 100 from different viewpoints. The main camera 111 and the sub-camera 112 are capable of acquiring stereo image data of the area ahead of the vehicle 100 under control of a control unit 210 (described later). The stereo camera 110 is further capable of outputting, to the control unit 210, stereo image data of the area ahead of the vehicle 100 obtained by capturing images with the main camera 111 and the sub-camera 112. The stereo image data is composed of image data Ia of the visible region obtained by the main camera 111 and image data of the visible region obtained by the sub-camera 112. Note that the image data obtained by the sub-camera 112 may be image data Ia. Image data Ia corresponds to a specific example of "image data" in the present disclosure. The stereo camera 110 can further generate distance image data Ib calculated from the amount of displacement of the positions of corresponding objects based on the obtained stereo image data, and output the distance image data Ib to the control unit 210. The distance image data Ib corresponds to a specific example of "image data" in the present disclosure.

[0031] The visible laser irradiator 120 is provided inside the headlight HLa, for example, as shown in Fig. 1. The visible laser irradiator 120 scans a laser beam L in the visible range on the road surface ahead of the vehicle 100 (traveling road surface 300) under the control of the control unit 210, thereby generating a linear visible laser pattern LP on the traveling road surface 300. Under the control of the control unit 210, the visible laser irradiator 120 is able to scan the laser beam L at a location on the traveling road surface 300 where an expected passage area 310 is likely to exist. Under the control of the control unit 210, the visible laser irradiator 120 is able to scan the laser beam L in a direction obliquely intersecting the longitudinal direction of the expected passage area 310 (i.e., the traveling direction of the vehicle 100). This enables the visible laser irradiation unit 120 to generate a linear visible laser pattern LP extending in a direction that diagonally intersects with the longitudinal direction of the expected passage area 310 (i.e., the traveling direction of the vehicle 100) at a location on the travel road surface 300 where the expected passage area 310 is likely to exist. The visible laser irradiation unit 120 is capable of emitting, as the laser beam L, a single-wavelength laser beam included in a wavelength band (e.g., a green wavelength band) different from the wavelength band of colors generally used on the travel road surface 300. Examples of "colors generally used on the travel road surface 300" include the colors of paved or unpaved roads, and the colors of dividing lines on paved roads.

[0032] The visible laser irradiation unit 120 has, for example, a laser emission unit capable of emitting visible laser light (laser beam L), an emission control driver capable of controlling the emission of the laser emission unit, an optical system capable of scanning the laser light on the road surface 300, and a scan control driver capable of controlling the scanning of the laser light by the optical system. The emission control driver is capable of controlling the emission of the laser emission unit under control of the control unit 210. The scan control driver is capable of controlling the operation of the optical system under control of the control unit 210. The laser emission unit has, for example, a semiconductor laser that emits visible laser light (laser beam L). The optical system is configured to include, for example, a polygon mirror and an fθ lens. The polygon mirror reflects the visible laser light emitted from the laser emission unit and is capable of scanning the reflected light of the visible laser light on the road surface 300 via the fθ lens.

[0033] The vehicle state quantity sensor 140 is configured to include various sensors such as an acceleration sensor, a vehicle speed sensor, and a gyro sensor. The vehicle state quantity sensor 140 is capable of outputting detection signals obtained by the various sensors to the control unit 210. The GNSS receiver 150 is capable of receiving positioning signals transmitted from a plurality of positioning satellites. The GNSS receiver 150 is capable of outputting the received positioning signals to the control unit 210.

[0034] The control flag input unit 170 is capable of receiving input of a control flag 412 from the driver. The control flag input unit 170 is, for example, a paddle shifter attached to a steering wheel. For example, when the driver simultaneously presses and holds the left and right paddle shifters, the control flag input unit 170 can store "1" as the control flag 412 in the storage unit 410. For example, when the driver simultaneously presses and holds the left and right paddle shifters again after previously storing "1" as the control flag 412 in the storage unit 410, the control flag input unit 170 can store "0" as the control flag 412 in the storage unit 410. For example, when the driver simultaneously presses and holds the left and right paddle shifters again after previously storing "0" as the control flag 412 in the storage unit 410, the control flag input unit 170 can store "1" as the control flag 412 in the storage unit 410.

[0035] When the control flag 412 is "1", it means, for example, that the mode is laser irradiation mode. When the control flag 412 is "0", it means, for example, that the mode is normal mode in which laser irradiation is not performed automatically. Note that the values ​​that the control flag 412 can take are not limited to those mentioned above.

[0036] The notification unit 180 includes, for example, a liquid crystal display panel or an organic EL display panel and a speaker. The notification unit 180 can display an image on the display screen 170A based on a video signal input from the notification control unit 23 (described later), and can output a sound based on a sound signal input from the notification control unit 23 (described later).

[0037] The high-precision road map DB 190 is stored in a large-capacity storage medium such as an HDD. The high-precision road map DB 190 includes high-precision road map information (dynamic map). This high-precision road map information, like the road map information included in the road map information integration_ECU 510, includes static information and quasi-static information that mainly constitute road information, and quasi-dynamic information and dynamic information that mainly constitute traffic information.

[0038] The storage unit 410 is configured, for example, by a nonvolatile memory. The storage unit 410 stores, for example, drawing data 411, a control flag 412, a resolution table 413, and a front position table 414. The storage unit 410 corresponds to a specific example of a "storage unit" in the present disclosure. The drawing data 411 includes an irradiation pattern setting value for generating a visible laser pattern LP at a predetermined location on the road surface (traveling road surface 300) ahead of the vehicle 100, and a headlamp irradiation area setting value for defining a headlamp irradiation area α. The headlamp irradiation area setting value is set so that, assuming that the traveling road surface 300 ahead of the vehicle 100 is flat, a predetermined area (headlamp non-irradiation area β) including the visible laser pattern LP to be irradiated onto the traveling road surface 300 becomes a non-irradiation area or does not overlap with the headlamp irradiation area α. The control flag 412 includes a flag (for example, "0" or "1") input from the control flag input unit 170. The irradiation pattern setting value corresponds to a specific example of an "irradiation pattern setting value" in the present disclosure. The headlamp irradiation area setting value corresponds to a specific example of an "initial setting value" in the present disclosure.

[0039] The resolution table 413 includes size data in a predetermined pixel unit (e.g., 1 pixel) for the image data Ia or the range image data Ib. The resolution table 413 includes Y-direction resolution in a direction (Y direction in FIG. 9 described later) corresponding to the traveling direction of the vehicle 100 (Y direction in FIG. 6) for the image data Ia or the range image data Ib, for each predetermined pixel unit (e.g., 1 pixel). The resolution table 413 further includes X-direction resolution in a direction (X direction in FIG. 9 described later) corresponding to the direction (X direction in FIG. 6) perpendicular to the traveling direction of the vehicle 100 (Y direction in FIG. 6) for the image data Ia or the range image data Ib. The resolution table 413 stores, for example, (2 mm, 3 mm) as the pixel resolution (X-direction resolution, Y-direction resolution) corresponding to a position 20 m ahead from the front end of the vehicle 100. In the resolution table 413, for example, (1 mm, 2 mm) is stored as the pixel resolution (X direction resolution, Y direction resolution) corresponding to a position 5 m ahead from the front end of the vehicle 100.

[0040] The front position table 414 includes position data of the road surface 300 ahead of the vehicle 100 for each pixel in the image data Ia or the range image data Ib. Assume that the image data Ia or the range image data Ib is composed of m×n pixels, with m pixels in the X direction and n pixels in the Y direction. In this case, the front position table 414 specifies, for example, that the position data of the (m / 2)th pixel in the X direction of the image data Ia or the range image data Ib and the n-th pixel in the Y direction of the image data Ia is the center position in the width direction of the vehicle 100 and a position 20 m ahead from the front end of the vehicle 100. The front position table 414 also specifies, for example, that the position data of the (m / 2)th pixel in the X direction of the image data Ia or the range image data Ib and the first pixel in the Y direction of the image data Ia or the range image data Ib is the center position in the width direction of the vehicle 100 and a position 5 m ahead from the front end of the vehicle 100.

[0041] The cruise control device 1000 further includes, for example, a control unit 210, a throttle actuator 270, a brake actuator 280, and a steering actuator 290, as shown in FIG. 7 . The control unit 210 corresponds to a specific example of an “acquisition unit” and “control unit” in the present disclosure. The control unit 210 is capable of controlling the entire vehicle 100. The control unit 210 is, for example, a so-called ECU (Electronic Control Unit) and is configured to include, for example, one or more processors and one or more memories. The control unit 210 may be configured to include, for example, a CPU (Central Processing Unit). In this case, the control unit 210 may be capable of controlling the entire vehicle 100 by, for example, executing a program stored in a storage unit 410.

[0042] The control unit 210 has, for example, a driving assistance unit 220 as shown in Fig. 7. The driving assistance unit 220 is capable of assisting the driver in driving the vehicle 100. The driving assistance unit 220 has, for example, an illumination control unit 21, a road surface shape estimation unit 22, and a notification control unit 23 as shown in Fig. 7.

[0043] The irradiation control unit 21 is capable of controlling the irradiation (drawing) of the laser beam L from the visible laser irradiation unit 120 and the irradiation (drawing) of light from the headlamp irradiation unit 130. The irradiation control unit 21 is capable of generating a control signal required for the irradiation (drawing) of the visible laser pattern LP based on, for example, drawing data 411 (irradiation pattern setting value) in the storage unit 410, and outputting the control signal to the visible laser irradiation unit 120. The irradiation control unit 21 is further capable of generating a control signal required for the irradiation (drawing) of light from the headlamp irradiation unit 130 based on, for example, drawing data 411 (headlamp irradiation area setting value) in the storage unit 410, and outputting the control signal to the headlamp irradiation unit 130.

[0044] The irradiation control unit 21 is capable of generating, for example, based on the drawing data 411 (irradiation pattern setting values), a control signal required to scan the laser beam L on the road surface ahead of the vehicle 100 out of the travel road surface 300, and outputting the control signal to the variable laser irradiation unit 120. The irradiation control unit 21 is capable of generating, for example, based on the drawing data 411 (irradiation pattern setting values), a control signal required to scan the laser beam L at least at a location on the travel road surface 300 ahead of the vehicle 100 where the expected passage area 310 is likely to exist. The irradiation control unit 21 is capable of generating, for example, based on the drawing data 411 (irradiation pattern setting values), a control signal required to scan the laser beam L in a direction obliquely intersecting the longitudinal direction of the expected passage area 310 (i.e., the traveling direction of the vehicle 100).

[0045] For example, based on the drawing data 411 (headlamp illumination area setting value), the illumination control unit 21 is capable of generating a control signal necessary to ensure that, when the road surface 300 ahead of the vehicle 100 is assumed to be flat, at least a predetermined area (headlamp non-illumination area β) of the road surface 300 ahead of the vehicle 100, including the visible laser pattern LP to be illuminated onto the road surface 300, becomes a non-illumination area or does not overlap with the headlamp illumination area α.

[0046] The control unit 210, when the irradiation control unit 21 is outputting the above control signals to the variable laser irradiation unit 120 and the headlamp irradiation unit 130, outputs a signal to control imaging to the stereo camera 110, thereby making it possible for the stereo camera 110 to acquire image data Ia including the visible laser pattern LP or distance image data Ib. When the irradiation control unit 21 is outputting the above control signal to the variable laser irradiation unit 120, the control unit 210, when it is outputting a signal to control imaging to the stereo camera 110, makes it possible for the road surface shape estimation unit 22 to acquire image data Ia including the visible laser pattern LP or distance image data Ib obtained by the stereo camera 110.

[0047] The road surface shape estimation unit 22 is capable of acquiring image data Ia or distance image data Ib including the visible laser pattern LP obtained by the stereo camera 110. Hereinafter, the image data Ia or distance image data Ib including the visible laser pattern LP obtained by the stereo camera 110 will be referred to as image data ILP. The road surface shape estimation unit 22 is capable of detecting unevenness on the road surface (traveling road surface 300) ahead of the vehicle 100 based on the image data ILP obtained by the stereo camera 110. The road surface shape estimation unit 22 is capable of detecting the presence or absence of unevenness on the road surface (traveling road surface 300) ahead of the vehicle 100 based on, for example, the image data ILP obtained by the stereo camera 110.

[0048] The road surface shape estimation unit 22 is capable of detecting the presence or absence of irregularities on the road surface (traveling road surface 300) ahead of the vehicle 100 in the image data ILP by the following method: For example, when there are bends and discontinuities in the visible laser pattern LP included in the image data ILP, the road surface shape estimation unit 22 is capable of determining that there are irregularities on the road surface (traveling road surface 300) ahead of the vehicle 100 in the image data ILP.

[0049] The characteristics of the "bent line" differ depending on whether the irregularity is a depression 330 or an obstacle 340 on the road surface 300. Fig. 9 shows an example of a visible laser pattern LP included in the image data ILP when the irregularity is a depression 330. Fig. 10 shows an example of a visible laser pattern LP included in the image data ILP when the irregularity is an obstacle 340 on the road surface 300.

[0050] 9, in the case where the irregularity is a depression 330, the end of the bent line LPa that is relatively far from the vehicle 100 is connected to a straight portion of the visible laser pattern LP, and the end that is relatively closer to the vehicle 100 is not connected to the straight portion of the visible laser pattern LP and is interrupted. In other words, there is an interruption α between one end of the bent line LPa (the end that is relatively closer to the vehicle 100) and the visible laser pattern LP. When the road surface shape estimation unit 22 detects the above-mentioned features (the bent line LPa, the interruption α) from the visible laser pattern LP in the image data ILP, it is possible to determine that a depression 330 exists on the road surface (traveling road surface 300) ahead of the vehicle 100.

[0051] 10, in the case where the unevenness is an obstacle 340 on the traveling road surface 300, the end of the bending line LPb that is relatively closer to the vehicle 100 is connected to a straight portion of the visible laser pattern LP, and the end that is relatively farther from the vehicle 100 is not connected to the straight portion of the visible laser pattern LP and is interrupted. In other words, there is an interruption γ between one end of the bending line LPb (the end that is relatively farther from the vehicle 100) and the visible laser pattern LP. When the road surface shape estimation unit 22 detects the above-mentioned features (bending line LPb, interruption γ) from the visible laser pattern LP in the image data ILP, it is possible to determine that an obstacle 340 exists on the road surface (traveling road surface 300) ahead of the vehicle 100.

[0052] The road surface shape estimation unit 22 is capable of calculating a group of pixel coordinates in the image data ILP for a predetermined region including the detected unevenness. After calculating the group of pixel coordinates, the road surface shape estimation unit 22 is further capable of extracting partial image data (partial image data Ic) of the group of pixel coordinates from the image data ILP.

[0053] The road surface shape estimation unit 22 is capable of detecting bending lines LPa and LPb in the extracted partial image data Ic. When the road surface shape estimation unit 22 detects a bending line LPa in the extracted partial image data Ic, it is possible to generate a straight line (a virtual straight line LV) on the assumption that no bending is present in the visible laser pattern LP, as shown in Fig. 11, for example, and to set a gap in the vertical direction (Y direction) in the partial image data Ic between the generated virtual straight line LV and the bending line LPa as a specific region δa.

[0054] When the road surface shape estimation unit 22 detects a bending line LPb in the extracted partial image data Ic, it generates a straight line (virtual straight line LV) assuming that there is no bending in the visible laser pattern LP, as shown in Figure 12, and is able to set the gap in the vertical direction (Y direction) in the partial image data Ic between the generated virtual straight line LV and the bending line LPb as a specific region δb.

[0055] The road surface shape estimation unit 22 is capable of calculating the number of pixels Np1 of the generated specific region δa and acquiring position data D1 of the specific region δa in the partial image data Ic. The number of pixels Np1 is the number of pixels corresponding to the distance in the Y-axis direction (vertical length Lv1) between, for example, the end (end A) of the bending line LPa that is not in contact with the imaginary straight line LV and the point (point P1) where the bending line LPa and the imaginary straight line LV are connected to each other.

[0056] The road surface shape estimation unit 22 is capable of acquiring, as the position data D1 of the specific region β, for example, the coordinates (x1, y1) of the point (point P1) in the specific region δa where the Y coordinate is maximum, and the coordinates (x2, y2) of the point (point P2) in the specific region β where the Y coordinate is minimum. Here, point P1 is the portion where the virtual straight line Lv and the bent line LPa are connected, and is the starting point of the bent line LPa. Point P2 is the portion where the straight line portion of the visible laser pattern LP is interrupted. The road surface shape estimation unit 22 is capable of estimating the position of the depression 330 ahead of the vehicle 100 based on the acquired position data D1 and the forward position table 414.

[0057] The road surface shape estimation unit 22 is capable of calculating the number of pixels Np2 of the generated specific region δb and acquiring position data D2 of the specific region δb in the partial image data Ic. The number of pixels Np2 is the number of pixels corresponding to the distance in the Y-axis direction (vertical length Lv2) between the bending point (point P3) included in the bending line LPb and the point (point P4) where the virtual straight line LV and the bending line LPb are connected to each other.

[0058] The road surface shape estimation unit 22 is capable of acquiring, as the position data D2 of the specific region δb, for example, the coordinates (x3, y3) of the point in the specific region δb where the Y coordinate is maximum (end B of the bending line LPb) and the coordinates (x4, y4) of the point in the specific region δb where the Y coordinate is minimum (point P4). Here, the end B is a portion of the bending line LPa that is not connected to the imaginary straight line Lv. The point P4 is a portion where the bending line LPa and the imaginary straight line Lv are connected to each other. The road surface shape estimation unit 22 is capable of estimating the position of the obstacle 340 ahead of the vehicle 100 based on the acquired position data D2 and the forward position table 414.

[0059] The road surface shape estimation unit 22 is capable of estimating the depth and size of the depression 330 based on the number of pixels Np1, the position data D1, the resolution table 413, and the front position table 414. The road surface shape estimation unit 22 is capable of, for example, reading out vertical size data in predetermined pixel units (for example, one pixel) that corresponds to the position data D1 from the resolution table 413, and estimating the depth and size of the depression 330 based on the read size data, the number of pixels Np1, and the front position table 414.

[0060] The road surface shape estimation unit 22 is capable of estimating the height and size of the obstacle 340 based on the number of pixels Np2, the position data D2, the resolution table 413, and the front position table 414. The road surface shape estimation unit 22 is capable of reading, for example, vertical size data in a predetermined pixel unit (for example, one pixel) that corresponds to the position data D1 from the resolution table 413, and estimating the height and size of the obstacle 340 based on the read size data, the number of pixels Np2, and the front position table 414.

[0061] The notification control unit 23 is capable of performing notification control in accordance with the depth of the depression 330 or the height of the obstacle 340 obtained as a result of estimation by the road surface shape estimation unit 22. When the depth of the depression 330 or the height of the obstacle 340 exceeds a predetermined threshold, the notification control unit 23 is capable of performing control to notify the driver to avoid the depression 330 or the obstacle 340. When the depth of the depression 330 or the height of the obstacle 340 does not exceed the predetermined threshold, the notification control unit 23 is capable of performing control to notify the driver of the presence of the depression 330 or the obstacle 340.

[0062] In both a manual driving mode (to be described later) and a driving control mode (to be described later), the notification control unit 23 is capable of outputting to the notification unit 180 a video signal for displaying an image in which a marker indicating the position of a depression 330 or an obstacle 340 (for example, an enclosing marker MK surrounding the depression 330 or the obstacle 340) is superimposed on the image data Ia. The notification unit 180 is capable of displaying, on the display screen 170A, an image in which the enclosing marker MK is superimposed, based on the video signal input from the notification control unit 23, as shown in Fig. 13, for example.

[0063] In a driving control mode described below, when the depth of the depression 330 or the height of the obstacle 340 exceeds a predetermined threshold, the notification control unit 23 can, for example, output a video signal to the notification unit 180 for displaying an image including a marker indicating the position of the depression 330 or the obstacle 340, and simultaneously output an audio signal to the notification unit 180 to indicate that driving control will be performed to avoid the depression 330 or the obstacle 340. In a driving control mode described below, when the depth of the depression 330 or the height of the obstacle 340 does not exceed a predetermined threshold, the notification control unit 23 can, for example, output a video signal to the notification unit 180 for displaying an image including a marker indicating the position of the depression 330 or the obstacle 340, and simultaneously output an audio signal to the notification unit 180 to indicate that driving control will be performed to cause the vehicle 100 to travel along the travel path without avoiding the depression 330 or the obstacle 340. The notification control unit 23 may paint the area of ​​the depression 330 or the obstacle 340 with a specific color to serve as a marker indicating the position of the depression 330 or the obstacle 340 .

[0064] Based on the image data ILP obtained by the stereo camera 110, the road surface shape estimation unit 22 is able to detect whether or not a part of the headlamp illumination area α illuminated by the light emitted from the headlamp illumination unit 130 and a part of the visible laser pattern LP generated by the visible laser illumination unit 120 overlap with each other on the road surface (traveling road surface 300) ahead of the vehicle 100. For example, as shown in FIGS. 9 and 10 , it is assumed that a part of the headlamp illumination area α illuminated by the light emitted from the headlamp illumination unit 130 and a part of the visible laser pattern LP generated by the visible laser illumination unit 120 do not overlap with each other. In this case, based on the image data ILP obtained by the stereo camera 110, the road surface shape estimation unit 22 is able to detect unevenness of the road surface (traveling road surface 300) ahead of the vehicle 100.

[0065] On the other hand, for example, as shown in FIGS. 14 and 15 , assume that a portion of the headlamp illumination area α illuminated by the light emitted from the headlamp illumination unit 130 and a portion of the visible laser pattern LP generated by the visible laser illumination unit 120 overlap with each other. In this case, the road surface shape estimation unit 22 can set a setting value that can reduce or eliminate the area of ​​the overlapping area where the portion of the headlamp illumination area α and the portion of the visible laser pattern LP overlap with each other, as shown in FIGS. 16 and 17 , as the latest setting value of the non-illumination area (headlamp non-illumination area β) of the headlamp illumination unit 130 or the headlamp illumination area α, and store the latest setting value in the storage unit 410 as the headlamp illumination area setting value. The latest setting value corresponds to a specific example of the “latest setting value” of the present disclosure. When the latest setting value is stored in the storage unit 410 as the headlamp illumination area setting value, the illumination control unit 21 can control the illumination of the headlamp illumination unit 130 based on the latest setting value.

[0066] Control unit 210 further includes a driving control unit 230, for example, as shown in Fig. 7. Driving control unit 230 is capable of controlling driving of vehicle 100. Driving control unit 230 includes a driving environment detection unit 31, a locator calculation unit 32, and an avoidance control unit 33, for example, as shown in Fig. 7.

[0067] The driving control unit 230 is capable of controlling the vehicle 100 according to, for example, a driving mode. Examples of the driving modes include a manual driving mode and a driving control mode. The manual driving mode is a driving mode that requires the driver to maintain steering, and is a driving mode in which the vehicle 100 is driven according to driving operations such as steering, accelerator, and brake operations by the driver. The driving control mode is a driving mode that supports the driver in driving operations by the driver to increase the safety of pedestrians, vehicles, and the like around the vehicle 100.

[0068] The driving environment detection unit 31 is capable of determining lane markings that divide the road around the vehicle 100 based on the range image data Ib received from the stereo camera 110. The driving environment detection unit 31 is further capable of determining, for example, the road curvature [1 / m] of the markings that divide the left and right sides of the road (driving lane) on which the vehicle 100 is traveling, and the width between the left and right markings (vehicle width). The driving environment detection unit 31 is also capable of detecting lanes and three-dimensional objects such as structures that exist around the vehicle 100, for example, by performing predetermined pattern matching on the range image data Ib.

[0069] Here, the detection of a three-dimensional object in the driving environment detection unit 31 includes, for example, detecting the type of the three-dimensional object, the distance to the three-dimensional object, the speed of the three-dimensional object, the relative speed between the three-dimensional object and the vehicle (host vehicle), etc. Examples of three-dimensional objects to be detected include traffic lights, intersections, road signs, stop lines, other vehicles, pedestrians, and various buildings.

[0070] Locator calculation unit 32 estimates the position of vehicle 100 on a road map (host vehicle position) and is capable of estimating the host vehicle position. Vehicle state quantity sensor 140 and GNSS receiver 150, which are required to estimate the position of vehicle 100 (host vehicle position), are connected to the input side of locator calculation unit 32.

[0071] In the driving control mode, the avoidance control unit 33 is capable of performing driving control in accordance with the depth of the depression 330 or the height of the obstacle 340 obtained as a result of estimation by the road surface shape estimation unit 22. When the depth of the depression 330 or the height of the obstacle 340 exceeds a predetermined threshold, the avoidance control unit 33 is capable of performing driving control to avoid the depression 330 or the obstacle 340. When the depth of the depression 330 or the height of the obstacle 340 does not exceed the predetermined threshold, the avoidance control unit 33 is capable of performing driving control to follow the driving path.

[0072] If the depth of the depression 330 or the height of the obstacle 340 exceeds a predetermined threshold, the avoidance control unit 33 is capable of performing driving control to avoid the depression 330 or the obstacle 340 based on, for example, the distance image data Ib obtained from the stereo camera 110, various data obtained from the vehicle state quantity sensor 140, the positioning signal obtained from the GNSS receiver 150, road map information read from the high-precision road map DB 190, and the position of the depression 330 or the obstacle 340 obtained from the road surface shape estimation unit 22.

[0073] When the depth of the depression 330 or the height of the obstacle 340 exceeds a predetermined threshold, the avoidance control unit 33, for example, when it determines that engine control is necessary, is capable of transmitting an engine control command to the engine control unit 240 as driving control for stopping the vehicle 100 in front of the depression 330 or the obstacle 340. When the depth of the depression 330 or the height of the obstacle 340 exceeds a predetermined threshold, the avoidance control unit 33, for example, when it determines that braking control is necessary, is capable of transmitting braking control as driving control for stopping the vehicle 100 in front of the depression 330 or the obstacle 340. At this time, the avoidance control unit 33, for example, is capable of transmitting a braking control command to the brake control unit 250 as braking control. When the depth of the depression 330 or the height of the obstacle 340 exceeds a predetermined threshold, for example, if the avoidance control unit 33 determines that steering control is necessary, the avoidance control unit 33 can perform, as travel control, steering control to stop the vehicle 100 in front of the depression 330 or the obstacle 340, or to travel while avoiding the depression 330 or the obstacle 340. At this time, the avoidance control unit 33 can, for example, transmit a steering control command to the steering control unit 260 as steering control.

[0074] A throttle actuator 270 is connected to the output side of the engine control unit 240. The throttle actuator 270 opens and closes a throttle valve of an electronically controlled throttle provided in a throttle body of the engine. The engine control unit 240 is able to control the operation of the throttle actuator 270 by outputting a drive signal to the throttle actuator 270. The throttle actuator 270 opens and closes the throttle valve based on the drive signal from the engine control unit 240 to adjust the intake air flow rate, thereby generating a desired engine output.

[0075] A brake actuator 280 is connected to the output side of the brake control unit 250. The brake actuator 280 is capable of adjusting the brake hydraulic pressure supplied to the brake wheel cylinders provided on each wheel. The brake control unit 250 is capable of controlling the operation of the brake actuator 280 by outputting a drive signal to the brake actuator 280. Based on the drive signal from the brake control unit 250, the brake actuator 280 generates a braking force on each wheel using the brake wheel cylinder, thereby forcibly decelerating the vehicle.

[0076] A steering actuator 290 is connected to the output side of the steering control unit 260. The steering actuator 290 is capable of adjusting the steering angle of the steering wheel. The steering control unit 260 is capable of controlling the operation of the steering actuator 290 by outputting a drive signal to the steering actuator 290. The steering actuator 290 is capable of generating a steering torque on the steering wheel based on the drive signal from the steering control unit 260, and forcibly rotating the steering wheel.

[0077] [Operation] Next, the operation of the driving control device 1000 will be described with reference to Figures 18 and 19. Figures 18 and 19 are diagrams for explaining an example of a driving assistance procedure in the driving control device 1000.

[0078] The driving control device 1000 determines whether the control flag 412 is on ("1") (step S101). If the control flag 412 is off ("0") (step S101; N), the driving control device 1000 outputs a control signal to the headlamp irradiation unit 130 to cause the headlamp irradiation unit 130 to irradiate light onto a headlamp irradiation area α on the road surface (traveling road surface 300) ahead of the vehicle 100. As a result, the headlamp irradiation unit 130 irradiates light onto the headlamp irradiation area α in accordance with the input control signal (step S102).

[0079] On the other hand, if the control flag 412 is on (“1”) (step S101; Y), the driving control device 1000 outputs a control signal to the visible laser irradiation unit 120 to scan the laser beam L on the road surface (traveling road surface 300) ahead of the vehicle 100, and outputs a control signal to the headlamp irradiation unit 130 to irradiate a headlamp irradiation area α of the road surface (traveling road surface 300) ahead of the vehicle 100 with light. As a result, the visible laser irradiation unit 120 irradiates the laser beam L onto the road surface (traveling road surface 300) ahead of the vehicle 100 in accordance with the input control signal (step S103). As a result, a linear visible laser pattern LP is drawn on the road surface (traveling road surface 300) ahead of the vehicle 100. Furthermore, the headlamp irradiation unit 130 irradiates the headlamp irradiation area α with light in accordance with the input control signal (step S103).

[0080] Next, the driving control device 1000 acquires the visible laser pattern LP and image data ILP including the visible laser pattern LP captured while light from the headlamp irradiation unit 130 is being irradiated onto the road surface (driving road surface 300) ahead of the vehicle 100 (step S104). Based on the acquired image data ILP, the driving control device 1000 determines whether or not a margin exists between the headlamp irradiation area α irradiated by the light emitted from the headlamp irradiation unit 130 and the visible laser pattern LP generated by the visible laser irradiation unit 120 (step S105).

[0081] If the margin exists (step S105; Y), the driving control device 1000 detects unevenness on the road surface (traveling road surface 300) ahead of the vehicle 100 based on the image data ILP obtained by the stereo camera 110, and estimates the depth or height and position of the detected unevenness (step S109). On the other hand, if the margin does not exist (step S105; Y), the driving control device 1000 calculates a headlamp illumination area setting value so that the margin exists, and stores the calculated headlamp illumination area setting value in the storage unit 410. In this way, the driving control device 1000 corrects the headlamp illumination area setting value (step S106).

[0082] The driving control device 1000 reads out the corrected headlamp illumination area setting value from the storage unit 410 and controls the illumination of the headlamp illumination unit 130 based on the read-out headlamp illumination area setting value. As a result, the headlamp illumination unit 130 emits light onto the headlamp illumination area α set in accordance with the corrected headlamp illumination area setting value (step S107). Next, the driving control device 1000 emits light from the headlamp illumination unit 130 onto the headlamp illumination area α set in accordance with the corrected headlamp illumination area setting value, of the road surface ahead of the vehicle 100 (traveling road surface 300), and re-acquires image data ILP including the visible laser pattern LP that was captured while the visible laser pattern LP generated by the visible laser illumination unit 120 was being irradiated onto the road surface ahead of the vehicle 100 (traveling road surface 300) (step S108).

[0083] The driving control device 1000 detects irregularities on the road surface (traveling road surface 300) ahead of the vehicle 100 based on the reacquired image data ILP, and estimates the depth or height and position of the detected irregularities (step S109).

[0084] The driving control device 1000 determines whether or not the unevenness should be avoided based on the depth or height and position of the unevenness obtained as a result of the estimation (step S110). If the driving control device 1000 determines that the unevenness does not need to be avoided (step S110; N), it performs control to notify the driver of the presence of the unevenness (step S111). The notification unit 180 displays, for example, an image on which an enclosing marker MK is superimposed on the display screen 170A. The driving control device 1000 further performs driving control to keep the vehicle along the road (step S112).

[0085] On the other hand, when it is determined that the unevenness should be avoided (step S110; Y), the driving control device 1000 performs control to notify the driver of the existence and avoidance of the unevenness (step S113). The notification unit 180, for example, displays an image with an enclosing marker MK superimposed on the display screen 170A, and outputs a sound to notify the driver of the existence and avoidance of the unevenness. The driving control device 1000 further performs driving control to avoid the unevenness (step S114). In this way, driving assistance in the driving control device 1000 is performed.

[0086] [effect] Next, the effects of vehicle 100 according to this embodiment will be described.

[0087] In this embodiment, the irradiation of the visible laser pattern LP by the visible laser irradiation unit 120 onto the road surface 300 ahead of the vehicle 100 is controlled based on the drawing data 411 (irradiation pattern setting value) read out from the storage unit 310, and the irradiation of the headlight irradiation unit 130 is controlled based on the drawing data 411 (initial setting value) read out from the storage unit 310. Here, the initial setting value is set so that, assuming that the road surface 300 ahead of the vehicle 100 is flat, a predetermined area including the visible laser pattern LP to be irradiated onto the road surface 300 ahead of the vehicle 100 is a non-irradiated area of ​​the headlight irradiation unit 130 or so that the area does not overlap with the irradiation area of ​​the headlight irradiation unit 130. As a result, the light from the headlight irradiation unit 130 of the vehicle 100 is not superimposed on the visible laser pattern LP, thereby achieving high contrast in the visible laser pattern LP. Furthermore, even if neon signs on signboards, light leaking from stores, etc. are superimposed on the visible laser pattern LP at night, high contrast is achieved in the visible laser pattern LP. As a result, the visible laser pattern LP can be more easily detected by the stereo camera 110, and the visible laser pattern LP can be more easily recognized by the driver of the vehicle 100.

[0088] Furthermore, in this embodiment, when a portion of the headlamp illumination area α illuminated by the light emitted from the headlamp illumination unit 130 and a portion of the visible laser pattern LP generated by the visible laser illumination unit 120 overlap with each other, a setting value that can reduce or eliminate the area of ​​the overlapping area where the portion of the headlamp illumination area α and a portion of the visible laser pattern LP overlap with each other is set as the latest setting value of the non-illuminated area (headlamp non-illuminated area β) of the headlamp illumination unit 130 or the headlamp illumination area α, and the latest setting value is stored as the headlamp illumination area setting value in the storage unit 310. As a result, even if the portion of the headlamp illumination area α illuminated by the light emitted from the headlamp illumination unit 130 and a portion of the visible laser pattern LP generated by the visible laser illumination unit 120 overlap with each other due to the influence of unevenness of the road surface 300, it is possible to immediately reduce or eliminate the overlap between the portion of the headlamp illumination area α and the portion of the visible laser pattern LP. This prevents the light from the headlamp irradiation unit 130 of the vehicle 100 from being continuously superimposed on the visible laser pattern LP, thereby providing high contrast in the visible laser pattern LP. Furthermore, even if neon signs on signs, lights leaking from stores, or the like are superimposed on the visible laser pattern LP at night, high contrast can be provided in the visible laser pattern LP. As a result, the visible laser pattern LP can be more easily detected by the stereo camera 110, and the visible laser pattern LP can be more easily recognized by the driver of the vehicle 100.

[0089] Furthermore, in this embodiment, when the latest setting value is stored in the storage unit 310 as the headlamp illumination area setting value, the illumination of the headlamp illumination unit 130 is controlled based on the latest setting value. This prevents the light of the headlamp illumination unit 130 of the vehicle 100 from being continuously superimposed on the visible laser pattern LP, thereby achieving high contrast in the visible laser pattern LP. Furthermore, even if neon signs on signboards, lights leaking from stores, and the like are superimposed on the visible laser pattern LP at night, high contrast can be achieved in the visible laser pattern LP. As a result, the visible laser pattern LP can be more easily detected by the stereo camera 110, and the visible laser pattern LP can be more easily recognized by the driver of the vehicle 100.

[0090] In this embodiment, the visible laser irradiation unit 120 may be configured to irradiate a plurality of visible laser patterns LP onto the road surface 300. In this case, it is possible to detect unevenness on the road surface 300 more precisely.

[0091] Although the present disclosure has been described above using embodiments, the present disclosure is not limited to these embodiments and various modifications are possible. The effects described in this specification are merely examples, and the effects of the present disclosure are not limited to the effects described in this specification. Therefore, other effects may be obtained with respect to the present disclosure.

[0092] Furthermore, the present disclosure may take the following aspects. (1) an acquisition unit capable of acquiring image data of an area ahead of the vehicle obtained by imaging; a control unit capable of controlling the visible laser light source and the variable light distribution headlamp; a storage unit that stores an irradiation pattern setting value of the visible laser light source and an initial setting value of a non-irradiated area or an irradiated area of ​​the variable light distribution headlamp; Equipped with The control unit Controlling the irradiation of a visible laser pattern onto a road surface ahead of the vehicle by the visible laser light source based on the irradiation pattern setting value; controlling illumination of the variable light distribution headlamp based on the initial setting value; It is possible to execute The initial setting value is set so that a predetermined area including the visible laser pattern to be irradiated onto the road surface ahead of the vehicle is the non-irradiated area or does not overlap with the irradiated area, assuming that the road surface ahead of the vehicle is flat. Driving aids. (2) When the control unit detects, based on image data of the road surface ahead of the vehicle, that a part of the road surface illumination area illuminated by the variable light distribution headlight and a part of the visible laser pattern generated by the visible laser light source overlap with each other, the control unit sets a setting value that can reduce or eliminate the area of ​​the overlapping area where the part of the road surface illumination area and the part of the visible laser pattern overlap with each other as the latest setting value of the non-illuminated area or illuminated area of ​​the variable light distribution headlight, and stores the latest setting value in the storage unit. (1) A driving assistance device as described above. (3) When the latest setting value is stored in the storage unit, the control unit is capable of controlling the illumination of the variable light distribution headlamp based on the latest setting value. (2) A driving assistance device as described above. (4) The variable headlight is composed of multiple micro LEDs arranged two-dimensionally. A driving assistance device according to any one of (1) to (3). (5) The variable headlights are provided on both the left and right headlights. A driving assistance device according to any one of (1) to (4). (6) The visible laser light source is provided in one of the left and right headlights, which is relatively far away from a non-traveling road surface adjacent to the traveling road surface. A driving assistance device according to any one of (1) to (5). (7) The visible laser pattern is linear A driving assistance device according to any one of (1) to (6). (8) The visible laser pattern comprises one or more linear patterns. A driving assistance device according to any one of (1) to (6). (9) The control unit detects unevenness of the road surface in front of the vehicle based on image data of the road surface including the visible laser pattern, and is capable of performing notification control or driving control based on the detection result. A driving assistance device according to any one of (1) to (8). (10) A driving assistance device, a visible laser light source, and a variable light distribution headlamp are provided. The driving assistance device is an acquisition unit capable of acquiring image data of an area ahead of the vehicle obtained by imaging; a control unit capable of controlling the visible laser light source and the variable light distribution headlamp; a storage unit that stores an irradiation pattern setting value of the visible laser light source and an initial setting value of a non-irradiated area or an irradiated area of ​​the variable light distribution headlamp; and The control unit Controlling the irradiation of a visible laser pattern onto a road surface ahead of the vehicle by the visible laser light source based on the irradiation pattern setting value; controlling illumination of the variable light distribution headlamp based on the initial setting value; It is possible to execute The initial setting value is set so that a predetermined area including the visible laser pattern to be irradiated onto the road surface ahead of the vehicle is the non-irradiated area or does not overlap with the irradiated area, assuming that the road surface ahead of the vehicle is flat. vehicle. (11) A driving assistance device according to any one of (1) to (9), a visible laser light source; Variable light distribution headlights Equipped with vehicle.

[0093] The control unit 210 shown in FIG. 5 can be implemented by circuitry including at least one semiconductor integrated circuit, such as at least one processor (e.g., a central processing unit (CPU)), at least one application-specific integrated circuit (ASIC), and / or at least one field-programmable gate array (FPGA). The at least one processor can be configured to perform all or a portion of the various functions of the control unit 210 shown in FIG. 5 by reading instructions from at least one non-transitory, tangible computer-readable medium. Such medium can take various forms, including, but not limited to, various magnetic media such as hard disks, various optical media such as CDs or DVDs, and various semiconductor memories (i.e., semiconductor circuits) such as volatile or non-volatile memories. Volatile memories can include DRAM and SRAM. Non-volatile memories can include ROM and NVRAM. An ASIC is an integrated circuit (IC) specialized to perform all or a portion of the various functions of the control unit 210 shown in FIG. 5. An FPGA is an integrated circuit designed to be configurable after manufacture to perform all or a portion of the various functions of the control unit 210 shown in FIG. 5. [Explanation of symbols]

[0094] 21...Illumination control unit, 22...Road surface shape estimation unit, 23...Notification control unit, 31...Driving environment detection unit, 32...Locator calculation unit, 33...Avoidance control unit, 100...Vehicle, 110...Stereo camera, 111...Main camera, 112...Sub camera, 120...Visible laser irradiation unit, 130...Headlight irradiation unit, 130a...Variable light distribution right headlight, 130b...Variable light distribution left headlight, 140...Vehicle state quantity sensor, 150...GNSS receiver, 160...Transceiver, 170...Control flag input unit, 170A...Display screen, 180...Notification unit, 190...High-precision road map DB, 210...Control unit, 220...Driving assistance unit, 230...Driving control unit, 240...Engine control unit, 250...Brake control unit, 260...Steering control unit, 270...Throttle actuator , 280...Brake actuator, 290...Steering actuator, 300...Travel surface, 310...Expected passage area, 320...Roadside strip, 330...Pot, 340...Obstacle, 410...Memory unit, 411...Drawing data, 412...Control flag, 413...Resolution table, 414...Forward position table, 510...Road map information integration_ECU, 520...Transmitter / receiver, 1000...Driving control device, 2000...Control device, FW...Front window, HLa, HLb...Headlight, Ia...Image data, Ib...Distance image data, L...Laser beam, LP...Visible laser pattern, LPa, LPb...Bent line, MK...Enclosing marker, NW...Network environment, TR...Tire, α...Headlight illuminated area, β...Headlight non-illuminated area, γ...Discontinuity.

Claims

1. an acquisition unit capable of acquiring image data of an area ahead of the vehicle obtained by imaging; a control unit capable of controlling the visible laser light source and the variable light distribution headlamp; a storage unit that stores an irradiation pattern setting value of the visible laser light source and an initial setting value of a non-irradiated area or an irradiated area of ​​the variable light distribution headlamp; Equipped with The control unit Controlling the irradiation of a visible laser pattern onto a road surface ahead of the vehicle by the visible laser light source based on the irradiation pattern setting value; controlling illumination of the variable light distribution headlamp based on the initial setting value; It is possible to execute The initial setting value is set so that a predetermined area including the visible laser pattern to be irradiated onto the road surface ahead of the vehicle is the non-irradiated area or does not overlap with the irradiated area, assuming that the road surface ahead of the vehicle is flat. Driving aids.

2. When the control unit detects, based on image data of the road surface ahead of the vehicle, that a part of the road surface illumination area illuminated by the variable light distribution headlight and a part of the visible laser pattern generated by the visible laser light source overlap with each other, the control unit sets a setting value that can reduce or eliminate the area of ​​the overlapping area where the part of the road surface illumination area and the part of the visible laser pattern overlap with each other as the latest setting value of the non-illuminated area or illuminated area of ​​the variable light distribution headlight, and stores the latest setting value in the storage unit. The driving assistance device according to claim 1 .

3. When the latest setting value is stored in the storage unit, the control unit is capable of controlling the illumination of the variable light distribution headlamp based on the latest setting value. The driving assistance device according to claim 2 .

4. A driving assistance device, a visible laser light source, and a variable light distribution headlamp are provided. The driving assistance device is an acquisition unit capable of acquiring image data of an area ahead of the vehicle obtained by imaging; a control unit capable of controlling the visible laser light source and the variable light distribution headlamp; a storage unit that stores an irradiation pattern setting value of the visible laser light source and an initial setting value of a non-irradiated area or an irradiated area of ​​the variable light distribution headlamp; and The control unit Controlling the irradiation of a visible laser pattern onto a road surface ahead of the vehicle by the visible laser light source based on the irradiation pattern setting value; controlling illumination of the variable light distribution headlamp based on the initial setting value; It is possible to execute The initial setting value is set so that a predetermined area including the visible laser pattern to be irradiated onto the road surface ahead of the vehicle is the non-irradiated area or does not overlap with the irradiated area, assuming that the road surface ahead of the vehicle is flat. vehicle.

Images