Driving assistance device, vehicle, and driving assistance method

Abstract

A driving assistance device according to one embodiment of the present disclosure is provided with a control unit capable of assisting with driving. The control unit is capable of: controlling projection of a visible laser pattern or an infrared laser pattern; acquiring image data of a snow surface which is in front of a vehicle and onto which the visible laser pattern or the infrared laser pattern is projected; and estimating the shape of the snow surface in front of the vehicle on the basis of the acquired image data.

Description

Driving assistance device, vehicle, and driving assistance method

[0001] The present disclosure relates to a driving assistance device mounted on a vehicle, a vehicle equipped with such a driving assistance device, and a driving assistance method implemented in such a driving assistance device.

[0002] In snowy regions, when a vehicle is traveling in a snowstorm or at dusk, the driver's entire field of vision may be blocked by snowfall or the setting sun, making it difficult for the driver to accurately grasp the road surface shape or obstacles ahead. In such cases, driving safety may be reduced, for example, due to tire wobble caused by ruts or the vehicle hitting a snow bank formed by snow removal. Technologies that enable safe driving even in a snowstorm or at dusk have been proposed, for example, in Patent Documents 1 and 2.

[0003] Japanese Patent No. 6962646 Japanese Patent Application Laid-Open No. 2022-53409

[0004] A first driving assistance device according to an embodiment of the present disclosure includes a control unit capable of providing driving assistance, which is capable of controlling the irradiation of a visible laser pattern or an infrared laser pattern, acquiring image data of a snow surface in front of the vehicle onto which the visible laser pattern or the infrared laser pattern is irradiated, and estimating the shape of the snow surface in front of the vehicle based on the acquired image data.

[0005] A second driving assistance device according to an embodiment of the present disclosure includes a control unit capable of providing driving assistance. The control unit is capable of controlling the irradiation of a visible laser pattern or an infrared laser pattern and acquiring image data of a snow surface in front of a vehicle onto which the visible laser pattern or the infrared laser pattern is irradiated. The visible laser pattern or the infrared laser pattern is a two-dimensional geometric pattern or a mesh-like pattern.

[0006] A vehicle according to an embodiment of the present disclosure includes a control unit capable of providing driving assistance, and a light source unit capable of irradiating a visible laser pattern or an infrared laser pattern in accordance with control by the control unit. The control unit is capable of controlling the light source unit to irradiate the visible laser pattern or the infrared laser pattern, acquiring image data of the snow surface in front of the vehicle onto which the visible laser pattern or the infrared laser pattern is irradiated, and estimating the shape of the snow surface in front of the vehicle based on the acquired image data.

[0007] A driving assistance method according to an embodiment of the present disclosure includes the following three steps: (A) projecting a visible laser pattern or an infrared laser pattern onto a snow surface in front of a vehicle; (B) acquiring image data of the snow surface in front of the vehicle onto which the visible laser pattern or the infrared laser pattern is projected; and (C) estimating the shape of the snow surface in front of the vehicle based on the acquired image data.

[0008] The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification. The drawings illustrate one embodiment and, together with the description, serve to explain the principles of the disclosure.

[0009] FIG. 1 is a diagram illustrating an example of the appearance of a front portion of a vehicle equipped with a driving assistance device according to a first embodiment of the present disclosure. FIG. 2 is a diagram illustrating another example of the appearance of the front portion of the vehicle of FIG. 1. FIG. 3 is a diagram illustrating an example of a view ahead of the vehicle when viewed from the driver's seat. FIG. 4 is a diagram illustrating an example of a visible laser pattern being irradiated from the vehicle of FIG. 1 onto a snow surface on which ruts have been formed. FIG. 5 is a diagram illustrating an example of a visible laser pattern being irradiated from the vehicle of FIG. 1 onto a snow surface including a step (snow bank) formed on the side of a road by snow removal. FIG. 6 is a diagram illustrating an example of a visible laser pattern being irradiated onto a snow surface including a snow bank. FIG. 7 is a diagram illustrating an example of a visible laser pattern being irradiated onto a snow surface including a snow bank. FIG. 8 is a diagram illustrating an example of a visible laser pattern being irradiated onto a snow surface including a snow bank. FIG. 9 is a diagram illustrating an example of a visible laser pattern being irradiated onto a snow surface including a snow bank. FIG. 10 is a diagram illustrating an example of a visible laser pattern being irradiated onto a snow surface including a snow bank. FIG. 11 is a diagram showing an example of a visible laser pattern irradiated onto a snow surface including a snow bank. FIG. 12 is a diagram showing an example of a visible laser pattern irradiated onto a snow surface including a snow bank. FIG. 13 is a diagram showing an example of a visible laser pattern irradiated onto a snow surface including a snow bank. FIG. 14 is a diagram showing an example of a visible laser pattern irradiated onto a snow surface including a snow bank. FIG. 15 is a diagram showing an example of a functional block of a cruise control device mounted on the vehicle of FIG. 1. FIG. 16 is a diagram showing an example of a situation in which a visible laser pattern including the outline of a wheel track is irradiated onto a snow surface from the vehicle of FIG. 1. FIG. 17 is a diagram showing an example of a situation in which a visible laser pattern including the outline of a snow bank is irradiated onto a snow surface from the vehicle of FIG. 1. FIG. 18 is a diagram showing an example of a situation in which a visible laser pattern including the outlines of wheel track and snow banks is irradiated onto a snow surface from the vehicle of FIG. 1. FIG. 19 is a diagram showing an example of an image displayed on the display screen of the vehicle of FIG. 1 when a visible laser pattern including the outline of a wheel track is irradiated onto a snow surface from the vehicle of FIG. 1. FIG. 20 is a diagram showing an example of an image displayed on the display screen of the vehicle of FIG. 1 when a visible laser pattern including the outline of a snow bank is projected onto the snow surface from the vehicle of FIG.FIG. 21 is a diagram illustrating an example of an image displayed on the display screen of the vehicle of FIG. 1 when a visible laser pattern including the outlines of ruts and snow banks is irradiated onto a snow surface from the vehicle of FIG. 1. FIG. 22 is a diagram illustrating an example of a procedure for irradiating a visible laser pattern in the cruise control device of FIG. 15. FIG. 23 is a diagram illustrating an example of a procedure for irradiating a visible laser pattern including the outlines of ruts and snow banks or a procedure for displaying an image in the cruise control device of FIG. 15. FIG. 24 is a diagram illustrating an example of a functional block of a cruise control device mounted on a vehicle according to a second embodiment of the present disclosure. FIG. 25 is a diagram illustrating an example of a network environment capable of communicating with the cruise control device of FIG. 24. FIG. 26 is a diagram illustrating a modified example of the functional block of the cruise control device of FIG. 24. FIG. 27 is a diagram illustrating a method for irradiating a visible laser pattern in the cruise control device of FIG. 26. FIG. 28(A) is a diagram illustrating an example of a visible laser pattern when the angle between the driver's line of sight and the laser beam is smaller than a threshold. FIG. 28(B) is a diagram illustrating an example of a visible laser pattern when the angle between the driver's line of sight and the laser beam is equal to or greater than a threshold. FIG. 29 is a diagram for explaining a method of irradiating a visible laser pattern in the driving control device of FIG. 26. FIG. 30(A) is a diagram illustrating an example of a visible laser pattern when the angle between the driver's line of sight and the laser beam is smaller than a threshold. FIG. 30(B) is a diagram illustrating an example of a visible laser pattern when the angle between the driver's line of sight and the laser beam is equal to or greater than a threshold. FIG. 31 is a diagram illustrating an example of a method of calculating the height of a snow bank using a linear visible laser pattern. FIG. 32 is a diagram illustrating an example of a method of calculating the height of a snow bank using a linear visible laser pattern. FIG. 33 is a diagram illustrating an example of a method of calculating the height of a snow bank using a linear visible laser pattern. FIG. 34 is a diagram illustrating an example of a method of calculating the height of a snow bank using a linear visible laser pattern. FIG. 35 is a diagram illustrating an example of a method of calculating the height of a snow bank using a two-dimensional geometric pattern or a mesh-like pattern. Figure 36 is a diagram showing an example of a method for calculating the height of a snow bank using a two-dimensional geometric pattern or a mesh-like pattern. Figure 37 is a diagram showing an example of a method for calculating the height of a snow bank using a two-dimensional geometric pattern or a mesh-like pattern.Fig. 38 is a diagram showing an example of a method for calculating the height of a snow bank using a two-dimensional geometric pattern or a mesh pattern. Fig. 39 is a diagram showing a snow bank meandering relative to a travel path. Fig. 40 is a diagram for explaining an example of a travel control procedure in the travel control device of Fig. 26.

[0010] In snowy regions, when a driver drives a vehicle in a snowstorm or at dusk, the driver's entire field of vision is blocked by snowfall or the setting sun, making it difficult for the driver to accurately grasp the road surface shape and obstacles ahead. In such cases, driving safety may be reduced, for example, due to tire wobble caused by ruts or the vehicle hitting a snowbank formed by snow removal. It is desirable to provide a driving assistance device that enables safe driving even in a snowstorm or at dusk, a vehicle equipped with such a driving assistance device, and a driving assistance method implemented in such a driving assistance device.

[0011] Some exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Note that the following description illustrates one specific example of the present disclosure and should not be construed as limiting the present disclosure. For example, each element, including numerical values, shapes, materials, parts, the position of each part, and the connection method of each part, is merely an example and should not be construed as limiting the present disclosure. Furthermore, in the following exemplary embodiments, components not described in independent claims based on the highest concept of the present disclosure are optional and may be provided as needed. The drawings are schematic and are not intended to be drawn to scale. Throughout this specification and the drawings, components having substantially the same function and configuration are designated by the same reference numerals, and redundant description will be omitted. Furthermore, components not directly related to one embodiment of the present disclosure are not shown in the drawings.

[0012] 1. First Embodiment [Configuration] FIG. 1 illustrates an example of the appearance of a front portion of a vehicle 100 according to a first embodiment of the present disclosure. The drive system of the vehicle 100 is not particularly limited, and the vehicle 100 can be driven, for example, by at least one of an engine and a motor. For example, as shown in FIG. 1 , the vehicle 100 includes a pair of left and right headlights 110 and a front window FW at its front portion. A laser irradiation unit 130 is provided inside the headlight 110, and a stereo camera 120 including a main camera 121 and a sub-camera 122 is provided in the vehicle interior CR visible from the outside through the front window FW. The location of the laser irradiation unit 130 is not particularly limited as long as it is located at the front portion of the vehicle 100. For example, the laser irradiation unit 130 may be provided inside the headlight 110 that is closer to the roadside strip, or, for example, as shown in FIG. 2 , the laser irradiation unit 130 may be provided in the vehicle interior CR.

[0013] 1 and 2 show an example of a vehicle 100 traveling on a road surface 200 in a snowy region. The surface of the road surface 200 is covered with snow 300, and ruts 310 and snow banks 320 have been formed in the snow 300. Therefore, the surface of the snow 300 (snow surface S) has various irregularities formed by the ruts 310 and snow banks 320.

[0014] 3 shows an example of the view ahead of the vehicle as seen from the driver's seat while the vehicle 100 is traveling on a road surface 200 covered with snow 300. For ease of explanation, dashed lines are drawn in FIG. 3 where ruts 310 and snow banks 320 exist.

[0015] As shown in FIG. 3 , there are ruts 310 and snow banks 320 ahead of the vehicle. However, it is not easy for the driver to see them. It is particularly difficult for the driver to see the ruts 310 and snow banks 320 during a snowstorm or at dusk. Therefore, as shown in FIGS. 4 and 5 , the vehicle 100 uses the laser irradiation unit 130 to irradiate the snow surface S with a visible laser pattern LP, such as a grid pattern, to make the unevenness of the snow surface S stand out in a three-dimensional manner, thereby making the ruts 310 and snow banks 320 more easily visible to the driver. Note that FIG. 4 illustrates an example in which the grid-shaped visible laser pattern LP is irradiated onto a portion of the snow surface S where ruts are likely to exist, thereby making the shape of the ruts stand out in a three-dimensional manner. Figure 5 also illustrates an example in which a grid-shaped visible laser pattern LP is irradiated onto areas of the snow surface S where snow banks are likely to exist, thereby making the shape of the snow banks appear three-dimensional.

[0016] The shape of the visible laser pattern LP need only be such that it can make the ruts 310 and snow banks 320 appear three-dimensionally, and is not limited to the shapes exemplified in FIGS. 4 and 5 . The visible laser pattern LP may be, for example, a single straight line that intersects the snow bank 320 at an angle, as shown in FIG. 6 . The visible laser pattern LP may be, for example, a two-dimensional pattern formed by two radial straight lines that intersect the snow bank 320 at an angle and at different angles to each other, as shown in FIG. 7 . The visible laser pattern LP may be, for example, a two-dimensional pattern formed by three radial straight lines that intersect the snow bank 320 at an angle and at different angles to each other, as shown in FIG. 8 . In these cases, one or more straight line patterns included in the visible laser pattern LP intersect obliquely with the traveling direction of the vehicle 100. The direction of travel of vehicle 100 is parallel to the extension direction of the road surface when the road surface is straight, and is parallel to the tangent to the road surface when viewed in a plane when the road surface is curved.

[0017] The visible laser pattern LP may be, for example, a two-dimensional pattern made up of a single straight line that intersects the snow bank 320 obliquely and a circle or ellipse that straddles the straight line, as shown in Fig. 9 . The visible laser pattern LP may be, for example, a two-dimensional pattern made up of two straight lines that intersect the snow bank 320 obliquely and at different angles to each other, as shown in Fig. 10 . The visible laser pattern LP may be, for example, a two-dimensional pattern made up of a single straight line that intersects the snow bank 320 obliquely and a striped pattern that intersects with this straight line, as shown in Fig. 11 . In these cases, one or more straight line patterns included in the visible laser pattern LP intersect obliquely with the traveling direction of the vehicle 100.

[0018] The visible laser pattern LP may be, for example, a two-dimensional pattern made up of a single straight line that intersects the snow bank 320 obliquely and a Fibonacci sequence curve that crosses over that straight line, as shown in Fig. 12 . The visible laser pattern LP may be, for example, a grid pattern that intersects the snow bank 320 obliquely, as shown in Fig. 13 . The visible laser pattern LP may be, for example, a grid pattern that intersects the snow bank 320 at a substantially right angle, as shown in Fig. 14 . In these cases, one or more straight line patterns included in the visible laser pattern LP intersect obliquely with the traveling direction of the vehicle 100.

[0019] In this way, the visible laser pattern LP may be, for example, a two-dimensional geometric pattern as shown in Figures 6 to 14. Also, the visible laser pattern LP may be, for example, a mesh-like pattern as shown in Figures 13 and 14.

[0020] The following describes in detail the cruise control device 1000 mounted on the vehicle 100, which is capable of irradiating the above-described visible laser pattern LP onto the snow surface S. The cruise control device 1000 corresponds to a specific example of a "driving assistance device" according to an embodiment of the present disclosure.

[0021] FIG. 15 shows an example of functional blocks of the cruise control device 1000. The cruise control device 1000 is configured to be able to project the above-described visible laser pattern LP onto the snow surface S. The cruise control device 1000 includes, for example, a stereo camera 120, a laser projection unit 130, an outside air temperature sensor 140, a control unit 150, a memory unit 160, a control flag input unit 170, and a display unit 180. The cruise control device 1000 may further include components other than those shown in FIG. 15. FIG. 15 illustrates an example of a portion of the configuration of the cruise control device 1000.

[0022] The stereo camera 120 is fixed, for example, to the upper center of the vehicle interior CR and includes, for example, a main camera 121 and a sub-camera 122. The main camera 121 and the sub-camera 122 are autonomous sensors that sense the real space ahead of the vehicle 100. The main camera 121 and the sub-camera 122 are, for example, arranged symmetrically on either side of the center portion in the width direction of the vehicle 100, enabling stereo imaging of the area ahead of the vehicle 100 from different viewpoints. The stereo camera 120 is further capable of outputting, for example, a pair of stereo images Ia of the area ahead of the vehicle 100 captured by the main camera 121 and the sub-camera 122 to the control unit 150 described below. The stereo camera 120 is further capable of generating a distance image Ib calculated from the amount of displacement between the positions of corresponding objects based on the pair of stereo images Ia obtained, and outputting the distance image Ib to the control unit 150 described below.

[0023] The laser irradiation unit 130 is provided, for example, inside the headlight 110 or inside the vehicle interior CR, as shown in Figure 1 or Figure 2. The laser irradiation unit 130 is capable of irradiating (drawing) a visible laser pattern LP on the snow surface S under the control of the control unit 150.

[0024] The laser irradiation unit 130 includes, for example, a laser emitter capable of emitting visible laser light, an emission control driver capable of controlling the emission of the laser emitter, an optical system capable of scanning the snow surface S with the laser light, and a scan control driver capable of controlling the scanning of the laser light by the optical system. The emission control driver is capable of controlling the emission of the laser emitter under control of the control unit 150. The scan control driver is capable of controlling the operation of the optical system under control of the control unit 150. The laser emitter includes, for example, a semiconductor laser that emits visible laser light. The optical system is configured to include, for example, a polygon mirror and an fθ lens. The polygon mirror reflects the visible laser light emitted from the laser emitter and is capable of scanning the reflected visible laser light on the snow surface S via the fθ lens.

[0025] The outside air temperature sensor 140 is capable of detecting the temperature outside the vehicle 100 (outside air temperature To) and outputting it to the control unit 150. The outside air temperature sensor 140 has, for example, a thermometer capable of detecting the outside air temperature To and an output section capable of outputting the outside air temperature To obtained by the thermometer to the control unit 150.

[0026] The control unit 150 is capable of controlling the entire vehicle 100. The control unit 150 is, for example, a so-called ECU (Electronic Control Unit) and is configured to include, for example, one or more processors and one or more memories. The control unit 150 may be configured to include, for example, a CPU (Central Processing Unit). In this case, the control unit 150 may be capable of controlling the entire vehicle 100 by, for example, executing a program stored in the storage unit 160.

[0027] The control unit 150 has, for example, a driving assistance unit 151 as shown in Fig. 15. The driving assistance unit 151 is capable of assisting the driver in driving the vehicle 100. The driving assistance unit 151 has, for example, a laser irradiation control unit 52, a snow surface shape estimation unit 53, and a display screen control unit 54 as shown in Fig. 15.

[0028] The snow surface shape estimation unit 53 is capable of estimating the shape of the snow surface S in front of the vehicle 100 based on the distance image Ib obtained from the stereo camera 120. The snow surface shape estimation unit 53 is capable of estimating the shape of the snow surface S based on, for example, the distance image Ib obtained when the visible laser pattern LP is not irradiated onto the snow surface S. The snow surface shape estimation unit 53 is capable of estimating the shape of the snow surface S based on, for example, the distance image Ib obtained when the visible laser pattern LP is irradiated onto the snow surface S.

[0029] The snow surface shape estimation unit 53 is capable of estimating the position and shape of the rut 310 in front of the vehicle 100, or the position and shape of the snow bank 320 in front of the vehicle 100, based on data about the shape of the snow surface S obtained by estimation (snow surface shape data).

[0030] If the snow surface shape estimation unit 53 is able to estimate the position and shape of the rut 310 in front of the vehicle 100, it can output the estimation results (rut data) about the position and shape of the rut 310 in front of the vehicle 100 to the laser irradiation control unit 52 and the display screen control unit 54. If the snow surface shape estimation unit 53 is able to estimate the position and shape of the snow bank 320 in front of the vehicle 100, it can output the estimation results (snow bank data) about the position and shape of the snow bank 320 in front of the vehicle 100 to the laser irradiation control unit 52 and the display screen control unit 54. If the snow surface shape estimation unit 53 is able to estimate the positions and shapes of both the rut 310 and the snow bank 320 in front of the vehicle 100, it can output the estimation results (rut data and snow bank data) about the positions and shapes of the rut 310 and the snow bank 320 in front of the vehicle 100 to the laser irradiation control unit 52 and the display screen control unit 54.

[0031] The laser irradiation control unit 52 is capable of controlling the irradiation (drawing) of the visible laser pattern LP. The laser irradiation control unit 52 is capable of generating a control signal required for the irradiation (drawing) of the visible laser pattern LP based on, for example, drawing data 161 (described later) in the storage unit 160, and outputting the control signal to the laser irradiation unit 130.

[0032] For example, assume that neither rut data nor snow bank data has been input from the snow surface shape estimation unit 53. In this case, the laser irradiation control unit 52 is capable of generating, for example, based on the drawing data 161, a control signal necessary to irradiate (draw) a visible laser pattern LP at a location where a rut 310 is likely to exist. In this case, the laser irradiation unit 130 is capable of irradiating (drawing), for example, a visible laser pattern LP such as that shown in FIG. 4. Furthermore, the laser irradiation control unit 52 is capable of generating, for example, based on the drawing data 161, a control signal necessary to irradiate (draw) a visible laser pattern LP at a location where a snow bank 320 is likely to exist. In this case, the laser irradiation unit 130 is capable of irradiating (drawing), for example, a visible laser pattern LP such as that shown in FIG. 5.

[0033] Also, for example, assume that the estimation results regarding the locations and shapes of ruts and snow banks are input from the snow surface shape estimation unit 53. In this case, the laser irradiation control unit 52 can generate a control signal for controlling the visible laser pattern LP based on, for example, the estimation results and the drawing data 161 so that the visible laser pattern LP becomes a pattern that takes into account the estimation results.

[0034] For example, assume that rut data is input from the snow surface shape estimation unit 53. At this time, the laser irradiation control unit 52 is capable of generating a control signal necessary to irradiate (draw) a visible laser pattern LP including the outline of the rut 310, for example, based on the rut data and the drawing data 161. At this time, the laser irradiation unit 130 is capable of irradiating (drawing) a visible laser pattern LP including the outline of the rut 310 (the dashed line portion in the figure), for example, as shown in FIG.

[0035] Also, for example, assume that snow bank data is input from the snow surface shape estimation unit 53. At this time, the laser irradiation control unit 52 is able to generate a control signal necessary to irradiate (draw) a visible laser pattern LP including the outline of the snow bank 320, for example, based on the snow bank data and the drawing data 161. At this time, the laser irradiation unit 130 is able to irradiate (draw) a visible laser pattern LP including the outline of the snow bank 320 (the dashed line portion in the figure), for example, as shown in Fig. 17 .

[0036] Also, for example, assume that rut data and snow bank data are input from the snow surface shape estimation unit 53. At this time, the laser irradiation control unit 52 is able to generate a control signal necessary to irradiate (draw) a visible laser pattern LP including the outlines of the ruts 310 and snow banks 320, for example, based on the rut data, snow bank data, and drawing data 161. At this time, the laser irradiation unit 130 is able to irradiate (draw) a visible laser pattern LP including the outlines of the ruts 310 and snow banks 320 (the dashed lines in the figure), for example, as shown in FIG.

[0037] In addition, in Figures 16, 17, and 18, the shape of the visible laser pattern LP excluding the outlines of the ruts 310 and snow banks 320 may be, for example, any of the patterns described in Figures 5 to 14.

[0038] The display screen control unit 54 is capable of generating a video signal based on, for example, a stereo image Ia obtained from the stereo camera 120 (e.g., an image obtained from the main camera 121) and outputting the signal to the display unit 180. For example, suppose that the stereo image Ia (e.g., an image obtained from the main camera 121) obtained when the visible laser pattern LP is irradiated (drawn) on the snow surface S is input to the display screen control unit 54. At this time, the display screen control unit 54 is capable of generating a video signal for displaying an image including the visible laser pattern LP.

[0039] It is assumed that the snow surface shape estimation unit 53 inputs the estimated results regarding the locations and shapes of ruts and snow banks to the display screen control unit 54. At this time, the display screen control unit 54 can generate a video signal based on, for example, the stereo image Ia (e.g., an image obtained from the main camera 121) and the estimated results input from the snow surface shape estimation unit 53, and output the video signal to the display unit 180.

[0040] For example, suppose that a stereo image Ia (e.g., an image obtained from the main camera 121) obtained when the visible laser pattern LP is irradiated (drawn) on the snow surface S is input to the display screen control unit 54. Furthermore, suppose that rut data is input to the display screen control unit 54 from the snow surface shape estimation unit 53. At this time, the display screen control unit 54 is capable of generating a video signal for displaying an image including the visible laser pattern LP including the outline of the rut 310. At this time, the display unit 180 is capable of displaying an image including the visible laser pattern LP including the outline of the rut 310 on the display screen 180A, as shown in FIG. 19 . Note that at this time, the outline of the rut 310 is not actually drawn on the snow surface S by the visible laser pattern LP.

[0041] For example, suppose that a stereo image Ia (e.g., an image obtained from the main camera 121) obtained when the visible laser pattern LP is irradiated (drawn) on the snow surface S is input to the display screen control unit 54. Furthermore, suppose that snow bank data is input to the display screen control unit 54 from the snow surface shape estimation unit 53. At this time, the display screen control unit 54 is capable of generating a video signal for displaying an image including the visible laser pattern LP, including the outline of the snow bank 320. At this time, the display unit 180 is capable of displaying an image including the visible laser pattern LP, including the outline of the snow bank 320, on the display screen 180A, as shown in FIG. 20 . Note that at this time, the outline of the snow bank 320 is not actually drawn on the snow surface S by the visible laser pattern LP.

[0042] For example, suppose that a stereo image Ia (e.g., an image obtained from the main camera 121) obtained while the visible laser pattern LP was being irradiated (drawn) on the snow surface S is input to the display screen control unit 54. Furthermore, suppose that rut data and snow bank data are input to the display screen control unit 54 from the snow surface shape estimation unit 53. At this time, the display screen control unit 54 is capable of generating a video signal for displaying an image including the visible laser pattern LP, which includes the outlines of the ruts 310 and the snow banks 320. At this time, the display unit 180 is capable of displaying an image including the visible laser pattern LP, which includes the outlines of the ruts 310 and the snow banks 320, on the display screen 180A, as shown in FIG. 21 . Note that at this time, the outlines of the ruts 310 and the snow banks 320 are not actually drawn on the snow surface S by the visible laser pattern LP.

[0043] In addition, in Figures 19, 20, and 21, the shape of the visible laser pattern LP excluding the outlines of the ruts 310 and snow banks 320 may be, for example, any of the patterns described in Figures 5 to 14.

[0044] The storage unit 160 stores, for example, drawing data 161 required for drawing the visible laser pattern LP. The drawing data 161 includes, for example, two-dimensional coordinate data of the visible laser pattern LP. The storage unit 160 further stores, for example, a control flag 162 input from the control flag input unit 170. The storage unit 160 may also store, for example, a program executed by the control unit 150. This program causes the control unit 150 to execute a series of procedures for controlling the entire vehicle 100. The storage unit 160 is configured, for example, by a random access memory (RAM), a read-only memory (ROM), an auxiliary storage device (such as a hard disk), etc.

[0045] The control flag input unit 170 is capable of receiving input of the control flag 162 from the driver. The control flag input unit 170 is, for example, a paddle shifter attached to the steering wheel. For example, when the driver simultaneously presses and holds the left and right paddle shifters, the control flag input unit 170 can store "1" as the control flag 162 in the storage unit 160. For example, when the driver simultaneously presses and holds the left and right paddle shifters again after previously storing "1" as the control flag 162 in the storage unit 160, the control flag input unit 170 can store "0" as the control flag 162 in the storage unit 160. For example, when the driver simultaneously presses and holds the left and right paddle shifters again after previously storing "0" as the control flag 162 in the storage unit 160, the control flag input unit 170 can store "1" as the control flag 162 in the storage unit 160.

[0046] When the control flag 162 is "1", it means that the mode is, for example, laser irradiation mode. When the control flag 162 is "0", it means that the mode is, for example, normal mode in which laser irradiation is not performed automatically. Note that the values ​​that the control flag 162 can take are not limited to those mentioned above.

[0047] [Operation] Next, the operation of the driving control device 1000 will be described with reference to Fig. 22. Fig. 22 is a diagram for explaining an example of a visible laser pattern irradiation procedure in the driving control device 1000.

[0048] The driving assistance unit 151 acquires the outside air temperature To from the outside air temperature sensor 140. The driving assistance unit 151 determines whether the outside air temperature To is equal to or higher than 0°C (step S101). If the outside air temperature To is greater than 0°C (step S101; N), the driving assistance unit 151 determines whether the control flag 162 is on ("1") (step S102). If the control flag 162 is on ("1") (step S102; Y) or if the outside air temperature To is equal to or lower than 0°C (step S101; Y), the driving assistance unit 151 determines whether there is an abnormality in the stereo image Ia obtained from the stereo camera 120 (step S103).

[0049] In step S103, the driving assistance unit 151 determines whether the scenery ahead of the vehicle 100 included in the stereo image Ia has poor visibility due to, for example, a snowstorm or twilight (a situation similar to a whiteout). If the scenery ahead of the vehicle 100 included in the stereo image Ia has poor visibility due to a snowstorm or twilight (step S103; N), the driving assistance unit 151 generates a control signal necessary to irradiate (draw) a visible laser pattern LP at a location where a snow bank 320 is likely to exist, based on, for example, the drawing data 161 in the storage unit 160, and outputs the generated signal to the laser irradiation unit 130. The laser irradiation unit 130 irradiates (draws) a visible laser pattern LP at a location where a snow bank 320 is likely to exist, based on the control signal input from the driving assistance unit 151 (step S107).

[0050] On the other hand, if the view ahead of the vehicle 100 included in the stereo image Ia is not poorly visible due to a snowstorm or twilight (step S103; Y), the driving assistance unit 151 determines whether the shape of the snow surface S can be estimated, for example, based on the distance image Ib obtained from the stereo camera 120 (step S104). If the shape of the snow surface S cannot be estimated (step S104; N), the driving assistance unit 151 generates a control signal, for example, based on the drawing data 161 in the storage unit 160, necessary to irradiate (draw) a visible laser pattern LP at a location where a snow bank 320 is likely to exist, and outputs the control signal to the laser irradiation unit 130. Based on the control signal input from the driving assistance unit 151, the laser irradiation unit 130 irradiates (draws) a visible laser pattern LP at a location where a snow bank 320 is likely to exist (step S107).

[0051] If the driving assistance unit 151 is able to estimate the shape of the snow surface S in step S104 (step S104; Y), it determines whether or not it is possible to detect the ruts 310 (step S105). For example, the driving assistance unit 151 determines whether or not the shape of the ruts 310 is included in the shape of the snow surface S estimated in step S104. As a result, for example, if the shape of the ruts 310 is not included in the shape of the snow surface S estimated in step S104 (step S105; N), the driving assistance unit 151 generates a control signal necessary to irradiate (draw) a visible laser pattern LP at a location where a snow bank 320 is likely to exist, based on the drawing data 161 in the storage unit 160, and outputs the generated signal to the laser irradiation unit 130. Based on the control signal input from the driving assistance unit 151, the laser irradiation unit 130 irradiates (draws) a visible laser pattern LP at a location where a snow bank 320 is likely to exist (step S107).

[0052] On the other hand, for example, if the shape of the snow surface S estimated in step S104 includes the shape of a rut 310 (step S105; Y), the driving assistance unit 151 generates a control signal necessary to irradiate (draw) a visible laser pattern LP at a location where the rut 310 is likely to exist, based on the drawing data 161 in the storage unit 160, and outputs the control signal to the laser irradiation unit 130. The laser irradiation unit 130 irradiates (draws) a visible laser pattern LP at a location where the rut 310 is likely to exist, based on the control signal input from the driving assistance unit 151 (step S106).

[0053] Next, the driving assistance unit 151 estimates the shape of the snow surface S ahead of the vehicle 100 based on the distance image Ib obtained from the stereo camera 120. At this time, the distance image Ib is an image obtained from the stereo camera 120 while the visible laser pattern LP is irradiated onto the snow surface S. That is, the driving assistance unit 151 estimates the shape of the snow surface S using the visible laser pattern LP (step S108). Next, the driving assistance unit 151 estimates the positions and shapes of the ruts 310 and snow banks 320 ahead of the vehicle 100 based on data on the estimated shape of the snow surface S (snow surface shape data) (step S109).

[0054] The driving assistance unit 151 determines whether to reflect the estimated results of the positions and shapes of the ruts 310 and snow banks 320 in the visible laser pattern LP (step S110). If it is set to reflect the estimated results in the visible laser pattern LP (step S110; Y), the driving assistance unit 151 generates a control signal necessary for irradiating (drawing) the visible laser pattern LP including the outlines of the ruts 310 and snow banks 320 based on the estimated results of the positions and shapes of the ruts 310 and snow banks 320 (rut data and snow bank data) and the drawing data 161, and outputs the control signal to the laser irradiation unit 130. The laser irradiation unit 130 irradiates (draws) the visible laser pattern LP including the outlines of the ruts 310 and snow banks 320 onto the snow surface S (step S111). The laser irradiation unit 130 irradiates (draws) a visible laser pattern LP including the outlines of a rut 310 and a snow bank 320 onto the snow surface S, as shown in FIG. 18, for example.

[0055] The laser irradiation unit 130 may, for example, irradiate (draw) a visible laser pattern LP including the outline of a rut 310 onto the snow surface S, as shown in Fig. 16. The laser irradiation unit 130 may, for example, irradiate (draw) a visible laser pattern LP including the outline of a snow bank 320 onto the snow surface S, as shown in Fig. 17. In this way, a visible laser pattern LP including the outline of at least one of the rut 310 and the snow bank 320 is irradiated (drawn) onto the snow surface S.

[0056] On the other hand, if it is set that the estimation results are to be reflected in the screen display (step S110; N), the driving assistance unit 151 generates image data including the contours of the ruts 310 and the snow banks 320, for example, based on the estimation results (rut data and snow bank data) about the positions and shapes of the ruts 310 and the snow banks 320. The driving assistance unit 151 then superimposes, for example, a stereo image Ia (for example, an image obtained from the main camera 121) obtained when the visible laser pattern LP is irradiated (drawn) on the snow surface S and the image data including the contours of the ruts 310 and the snow banks 320. As a result, the driving assistance unit 151 generates, for example, a video signal for displaying an image including the visible laser pattern LP including the contours of the ruts 310 and the snow banks 320. The driving assistance unit 151 outputs the generated video signal to the display unit 180. Based on the video signal input from the driving assistance unit 151, the display unit 180 displays an image including a visible laser pattern LP including the outlines of the ruts 310 and snow banks 320 on the display screen 180A, for example, as shown in Figure 21 (step S112).

[0057] The display unit 180 may display on the display screen 180A, for example, an image including a visible laser pattern LP including the outline of the wheel rut 310, as shown in Fig. 19. The display unit 180 may display on the display screen 180A, for example, an image including a visible laser pattern LP including the outline of the snow bank 320, as shown in Fig. 20. In this way, an image including a visible laser pattern LP including the outline of at least one of the wheel rut 310 and the snow bank 320 is displayed on the display screen 180A.

[0058] [Effects] Next, the effects of vehicle 100 according to this embodiment will be described.

[0059] In this embodiment, a visible laser pattern LP is projected onto the snow surface S ahead of the vehicle 100. This allows the unevenness of the snow surface S to appear three-dimensionally by the visible laser pattern LP, making it easier for the driver to see the unevenness of the snow surface S. As a result, safe driving becomes possible even in a snowstorm or at twilight. In this embodiment, the shape of the snow surface S ahead of the vehicle 100 is further estimated based on a stereo image Ia (distance image Ib) of the snow surface S ahead of the vehicle 100 projected with the visible laser pattern LP. This makes it possible to control the projection of the visible laser pattern LP so that the pattern takes into account the estimated shape of the snow surface S ahead of the vehicle 100. This makes it easier to predict which lane the driver should drive in, making safe driving possible even in a snowstorm or at twilight.

[0060] Furthermore, in this embodiment, the position and shape of the ruts 310 or the position and shape of the snow bank 320 ahead of the vehicle 100 are estimated based on the stereo image Ia (distance image Ib). This makes it possible to control the irradiation of the visible laser pattern LP so that the visible laser pattern LP is a pattern that takes into account the estimated shape of the ruts 310 or snow bank 320 ahead of the vehicle 100. This makes it easier to predict which lane in the future the vehicle should travel in, allowing for safe driving even in a snowstorm or at dusk.

[0061] In this embodiment, the visible laser pattern LP may be a two-dimensional geometric pattern such as those shown in Figures 6 to 14. In this case, the visible laser pattern LP can make the unevenness of the snow surface S appear three-dimensional, making it easier for the driver to see the unevenness of the snow surface S. As a result, safe driving becomes possible even in a snowstorm or at dusk.

[0062] In this embodiment, the visible laser pattern LP may be a mesh pattern, for example, as shown in Figures 13 and 14. In this case, the visible laser pattern LP can make the unevenness of the snow surface S appear three-dimensional, making it easier for the driver to see the unevenness of the snow surface S. As a result, safe driving becomes possible even in a snowstorm or at dusk.

[0063] Furthermore, in this embodiment, if the visible laser pattern LP includes one or more straight lines (line segments) that diagonally intersect with the snow bank 320, the visible laser pattern LP can make the unevenness of the snow surface S appear three-dimensional. This makes it easier for the driver to see the unevenness of the snow surface S. As a result, safe driving becomes possible even in a snowstorm or at twilight.

[0064] 2. Second embodiment [Configuration] Next, a vehicle according to a second embodiment of the present disclosure will be described. Fig. 24 shows an example of functional blocks of a cruise control device 1000 mounted on a vehicle according to the second embodiment of the present disclosure. Fig. 25 shows an example of functional blocks of a control device 2000 provided in a network environment NW in which cruise control devices 1000 mounted on multiple vehicles are connected via wireless communication.

[0065] The control device 2000 is capable of sequentially integrating and updating the road map information transmitted from the cruise control devices 1000 of the respective vehicles, and transmitting the updated road map information to the respective vehicles. The control device 2000 includes, for example, a road map information integration ECU 270 and a transceiver 280.

[0066] The road map boundary information integration ECU 270 is capable of continuously updating road map information surrounding a vehicle on a road by integrating road map information collected from multiple vehicles via the transceiver 280. The road map information may be, for example, a dynamic map, and includes static information and quasi-static information that mainly constitute road information, and quasi-dynamic information and dynamic information that mainly constitute traffic information.

[0067] Static information that makes up road information includes information that needs to be updated within one month, such as roads, road structures, lane information, road surface information, and permanent traffic regulations. "Roads" include, for example, road locations and shapes, intersections, and road attributes (e.g., national roads, prefectural roads, city roads, private roads, priority roads, non-priority roads, general roads, and expressways). "Road structures" include, for example, traffic signs, traffic lights, convex mirrors, and pedestrian bridges.

[0068] The quasi-static information that constitutes the road information is made up of information that needs to be updated every hour, such as traffic regulation information due to road construction or events, wide-area weather information, and congestion forecasts.

[0069] The semi-dynamic information that constitutes traffic information is composed of information that requires updating within one minute, such as the actual traffic congestion situation at the time of observation, driving restrictions, temporary driving obstructions such as fallen objects and obstacles, actual accident conditions, and narrow-area weather information.

[0070] The dynamic information that constitutes the traffic information is composed of information that needs to be updated every second, such as information sent and exchanged between mobile units, information on currently displayed traffic signals, information on pedestrians and bicycles at intersections, information on vehicles traveling on roads, etc. Such road map information is maintained and updated periodically until the next information is received from each vehicle, and the updated road map information is transmitted to each vehicle as appropriate via the transceiver 280.

[0071] 24, the cruise control device 1000 includes a stereo camera 120, a laser irradiation unit 130, an outside air temperature sensor 140, a control unit 150, a memory unit 160, a control flag input unit 170, and a display unit 180. The cruise control device 1000 further includes, for example, a transceiver 210, a vehicle state quantity sensor 220, a GNSS receiver 230, a throttle actuator 240, an electric power steering motor 250, and a brake actuator 260, as shown in FIG.

[0072] The control unit 150 includes, for example, a driving assistance unit 151, a driving control unit 152, an engine control unit 153, a power steering control unit 154, and a brake control unit 155, as shown in FIG.

[0073] A throttle actuator 240 is connected to the output side of the engine control unit 153. The throttle actuator 240 opens and closes a throttle valve of an electronically controlled throttle provided in a throttle body of the engine. The engine control unit 153 is able to control the operation of the throttle actuator 240 by outputting a drive signal to the throttle actuator 240. The throttle actuator 240 opens and closes the throttle valve based on the drive signal from the engine control unit 153 to adjust the intake air flow rate, thereby generating a desired engine output.

[0074] An electric power steering motor 250 is connected to the output side of the power steering control unit 154. The electric power steering motor 250 applies steering torque to the steering mechanism using the rotational force of the motor. The power steering control unit 154 controls the operation of the electric power steering motor 250 by outputting a drive signal to the electric power steering motor 250. In autonomous driving, the electric power steering motor 250 is capable of executing lane keeping control, which keeps the vehicle traveling in the current lane, and lane change control, which moves the vehicle to an adjacent lane (lane change control for overtaking control, etc.), based on the drive signal from the power steering control unit 154.

[0075] A brake actuator 260 is connected to the output side of the brake control unit 155. The brake actuator 260 is capable of adjusting the brake hydraulic pressure supplied to the brake wheel cylinders provided on each wheel. The brake control unit 155 is capable of controlling the operation of the brake actuator 260 by outputting a drive signal to the brake actuator 260. Based on the drive signal from the brake control unit 155, the brake actuator 260 generates a braking force on each wheel using the brake wheel cylinder, thereby forcibly decelerating the vehicle.

[0076] The driving control unit 152 controls the vehicle according to, for example, a driving mode. Examples of driving modes include a manual driving mode and a driving control mode. The manual driving mode is a driving mode that requires the driver to maintain steering, and the host vehicle is driven according to driving operations such as steering, accelerator, and brake operations by the driver. The driving control mode is a driving mode that supports the driver in driving operations by the driver to increase the safety of pedestrians and other vehicles around the vehicle (host vehicle). In the driving control mode, the driving control unit 152 is capable of, for example, controlling the steering angle during automatic driving based on the estimation results obtained by the snow surface shape estimation unit 53. In the driving control mode, the driving control unit 152 is capable of, for example, automatically driving the vehicle (host vehicle) along ruts 310 formed in the snow 300 on the driving road surface 200, or automatically driving the vehicle (host vehicle) along snow banks 320 formed in the snow 300 on the driving road surface 200.

[0077] The driving control unit 152 includes, for example, a driving environment detection unit 55, a locator calculation unit 56, and a high-precision road map database 57, as shown in FIG.

[0078] The driving environment detection unit 55 is capable of determining lane markings that divide the road around the vehicle based on the distance image Ib received from the stereo camera 120. The driving environment detection unit 55 is further capable of determining, for example, the road curvature [1 / m] of the markings that divide the left and right sides of the road (driving lane) on which the vehicle is traveling, and the width between the left and right markings (vehicle width). The driving environment detection unit 55 is also capable of performing, for example, predetermined pattern matching on the distance image to detect lanes and three-dimensional objects such as structures that exist around the vehicle.

[0079] Here, the detection of a three-dimensional object by the driving environment detection unit 55 includes, for example, detecting the type of the three-dimensional object, the distance to the three-dimensional object, the speed of the three-dimensional object, the relative speed between the three-dimensional object and the vehicle (host vehicle), etc. Examples of three-dimensional objects to be detected include traffic lights, intersections, road signs, stop lines, other vehicles, pedestrians, and various buildings.

[0080] The locator calculation unit 56 estimates the position of the vehicle (host vehicle position) on a road map, and is capable of estimating the host vehicle position. A vehicle state quantity sensor 220 and a GNSS receiver 230, which are required to estimate the vehicle position (host vehicle position), are connected to the input side of the locator calculation unit 56. The vehicle state quantity sensor 220 is configured to include, for example, an acceleration sensor, a vehicle speed sensor, a gyro sensor, etc.

[0081] The high-precision road map database 57 is a large-capacity storage medium such as an HDD, and stores high-precision road map information (dynamic map). This high-precision road map information, like the road map information included in the road map information integration_ECU 270, mainly includes static information and quasi-static information constituting road information, and quasi-dynamic information and dynamic information constituting traffic information.

[0082] The locator calculation unit 56 includes, for example, a map information acquisition unit, a vehicle position estimation unit, and a driving environment recognition unit.

[0083] The vehicle position estimation unit is capable of acquiring the position coordinates of the vehicle (host vehicle) based on the positioning signal received by the GNSS receiver 230. The vehicle position estimation unit is also capable of estimating the host vehicle's position on a road map by map-matching the acquired position coordinates with route map information. The map information acquisition unit is capable of acquiring map information of a predetermined range including the vehicle (host vehicle) from map information stored in the high-precision road map database 57, based on the position coordinates of the vehicle (host vehicle) acquired by the vehicle position estimation unit.

[0084] In an environment where valid positioning signals from positioning satellites cannot be received due to reduced sensitivity of the GNSS receiver 230, such as when driving inside a tunnel, the vehicle position estimation unit can switch to autonomous navigation, which estimates the vehicle's position based on the vehicle speed detected by the vehicle speed sensor, the angular velocity detected by the gyro sensor, and the longitudinal acceleration detected by the acceleration sensor, and estimate the vehicle's position on a road map.

[0085] As described above, the vehicle position estimation unit estimates the position of the vehicle (host vehicle position) on a road map based on the positioning signal received by the GNSS receiver 230 or information detected by a gyro sensor, etc., and is then able to determine the road type, etc. of the road on which the vehicle (host vehicle) is traveling based on the estimated host vehicle position on the road map.

[0086] The driving environment recognition unit is capable of updating the road map information stored in the high-precision road map database 57 to the latest state using road map information acquired through external communication (roadside-to-vehicle communication and vehicle-to-vehicle communication) via the transceiver 210. This information update is performed not only for static information but also for quasi-static information, quasi-dynamic information, and dynamic information. As a result, the road map information is composed of road information and traffic information acquired through communication with outside the vehicle, and information on moving bodies such as vehicles traveling on roads is updated in approximately real time.

[0087] The driving environment recognition unit verifies road map information based on the driving environment information recognized by the driving environment detection unit 55, and is capable of updating the road map information stored in the high-precision road map database 57 to the latest state. This information update is performed not only on static information, but also on quasi-static information, quasi-dynamic information, and dynamic information. As a result, information on moving objects such as vehicles traveling on roads recognized by the driving environment detection unit 55 is updated in real time.

[0088] The road map information thus updated is then transmitted to the control device 2000 and vehicles around the vehicle (host vehicle) by road-to-vehicle communication and vehicle-to-vehicle communication via the transceiver 210. Furthermore, the driving environment recognition unit is capable of outputting, from the updated road map information, map information of a predetermined range including the host vehicle position estimated by the vehicle position estimation unit, together with the host vehicle position (vehicle position information) to the driving_ECU 21.

[0089] 24, the storage unit 160 stores, for example, drawing data 161, a control flag 162, and image data 163. The image data 163 is a distance image Ib obtained by the stereo camera 120 in a season when there is no snowfall.

[0090] In this embodiment, the snow surface shape estimation unit 53 is able to estimate the shape of the snow surface S ahead of the vehicle 100 by comparing a distance image Ib obtained from the stereo camera 120 (distance image Ib during snowfall) with a distance image Ib obtained by the stereo camera 120 in a season when there is no snowfall (distance image Ib during non-snowfall). The non-snowfall distance image Ib corresponds to a specific example of "reference image data" according to an embodiment of the present disclosure. When comparing the distance image Ib during snowfall with the distance image Ib during non-snowfall, the snow surface shape estimation unit 53 is able to use position information of the driving lane shown in these distance images Ib. That is, in this embodiment, the distance image Ib during snowfall and position information are stored in the storage unit 160 in association with each other, and the distance image Ib during non-snowfall and position information are stored in the storage unit 160 in association with each other.

[0091] The snow surface shape estimation unit 53 is capable of estimating the position and shape of the ruts 310 and the position and shape of the snow banks 320, for example, based on the difference between the position of the snow surface S obtained from the distance image Ib during snowfall and the position of the road surface 200 obtained from the distance image Ib during non-snowfall.

[0092] [Effects] Next, the effects of vehicle 100 according to this embodiment will be described.

[0093] In this embodiment, as in the first embodiment, a visible laser pattern LP is projected onto the snow surface S in front of the vehicle 100. This makes it possible for the unevenness of the snow surface S to appear three-dimensionally by the visible laser pattern LP, making it easier for the driver to see the unevenness of the snow surface S. As a result, safe driving becomes possible even in a snowstorm or at twilight. Furthermore, in this embodiment, as in the first embodiment, the shape of the snow surface S in front of the vehicle 100 is estimated based on a stereo image Ia (distance image Ib) of the snow surface S in front of the vehicle 100 projected with the visible laser pattern LP. This makes it possible to control the projection of the visible laser pattern LP so that the visible laser pattern LP becomes a pattern that takes into account the estimated shape of the snow surface S in front of the vehicle 100. This makes it easier to predict which lane the driver should drive in, making safe driving possible even in a snowstorm or at twilight.

[0094] Furthermore, in this embodiment, as in the first embodiment, the position and shape of the rut 310 or the position and shape of the snow bank 320 ahead of the vehicle 100 are estimated based on the stereo image Ia (distance image Ib). This makes it possible to control the irradiation of the visible laser pattern LP so that the visible laser pattern LP is a pattern that takes into account the estimated shape of the rut 310 or snow bank 320 ahead of the vehicle 100. This makes it easier to predict which lane in the future the vehicle should travel in, enabling safe driving even in a snowstorm or at dusk.

[0095] Furthermore, in this embodiment, the shape of the snow surface S ahead of the vehicle 100 is estimated by comparing the distance image Ib when it is snowing with the distance image Ib when it is not snowing. This makes it possible to control the irradiation of the visible laser pattern LP so that the visible laser pattern LP is a pattern that takes into account the estimated shape of the ruts 310 or snow banks 320 ahead of the vehicle 100. This makes it easier to predict which lane in the future the vehicle should travel in, allowing for safe driving even in a snowstorm or at twilight.

[0096] Furthermore, in this embodiment, when steering angle control during automatic driving is performed based on the estimation results obtained by the snow surface shape estimation unit 53, the outlook for which lane to drive in from here on out is automatically determined, making it possible to drive safely even in a snowstorm or at dusk.

[0097] In this embodiment, as in the first embodiment, the visible laser pattern LP may be a two-dimensional geometric pattern such as those shown in Figures 6 to 14. In this case, the visible laser pattern LP can make the unevenness of the snow surface S appear three-dimensional, making it easier for the driver to see the unevenness of the snow surface S. As a result, safe driving becomes possible even in a snowstorm or at dusk.

[0098] In this embodiment, similar to the first embodiment, the visible laser pattern LP may be a mesh pattern, for example, as shown in Figures 13 and 14. In this case, the visible laser pattern LP can make the unevenness of the snow surface S appear three-dimensional, making it easier for the driver to see the unevenness of the snow surface S. As a result, safe driving becomes possible even in a snowstorm or at dusk.

[0099] Furthermore, in this embodiment, if the visible laser pattern LP includes one or more straight lines (line segments) that diagonally intersect with the snow bank 320, the visible laser pattern LP can make the unevenness of the snow surface S appear three-dimensional. This makes it easier for the driver to see the unevenness of the snow surface S. As a result, safe driving becomes possible even in a snowstorm or at twilight.

[0100] Furthermore, in this embodiment, if the visible laser pattern LP includes one or more straight lines (line segments) that diagonally intersect with the snow bank 320, the visible laser pattern LP can make the unevenness of the snow surface S appear three-dimensional. This makes it easier for the driver to see the unevenness of the snow surface S. As a result, safe driving becomes possible even in a snowstorm or at twilight.

[0101] 3. Modifications of the Second Embodiment 3-1. Modification A In the second embodiment, the driving control unit 152 may include a snow-surface driving path control unit 58, as shown in FIG. 26 . The snow-surface driving path control unit 58 is capable of detecting a bending point of the visible laser pattern LP included in the stereo image Ia or the distance image Ib. The snow-surface driving path control unit 58 is capable of detecting a circular or elliptical area of ​​the visible laser pattern LP that is included from the detected bending point to the tip of the visible laser pattern LP. Based on the shape of the detected area, the snow-surface driving path control unit 58 is capable of estimating the height from the road surface of a three-dimensional object (a rut or a snow bank) that exists in that area. Based on the estimated height, the snow-surface driving path control unit 58 determines whether the vehicle 100 needs to avoid the three-dimensional object (a rut or a snow bank), and performs driving control (e.g., steering control and braking control) of the vehicle 100 based on the determination result.

[0102] Fig. 27 is a diagram for explaining a method of irradiating a visible laser pattern LP in a vehicle 100. Fig. 28(A) shows an example of a visible laser pattern LP when the angle θ1 between the line of sight EL of the driver DR and the laser beam La is smaller than a threshold value θth. Fig. 28(B) shows an example of a visible laser pattern LP when the angle θ1 between the line of sight EL of the driver DR and the laser beam La is equal to or greater than a threshold value θth. Figs. 27, 28(A), and 28(B) show examples of a linear visible laser pattern LP being irradiated onto a snow surface S.

[0103] The laser irradiation unit 130 is disposed inside one of the pair of headlights 110 of the vehicle 100 that is farther away from the roadside strip (snow bank 320). The laser irradiation unit 130 is disposed so that the angle θ1 between the laser beam La emitted from the laser irradiation unit 130 and the line of sight EL of the driver DR of the vehicle 100 is equal to or greater than a threshold value θth. At this time, the laser irradiation unit 130 is disposed so that the angle θ2 between the laser beam La (visible laser pattern LP) and the step 320A of the snow bank 320 is equal to or greater than a threshold value θth.

[0104] 28(A), for example, when the angle θ1 is less than the threshold value θth, the driver DR has difficulty visually recognizing the bends in the visible laser pattern LP irradiated onto the step 320A. On the other hand, when the angle θ1 is equal to or greater than the threshold value θth, for example, as shown in FIG. 28(B), the driver DR can easily visually recognize the bends in the visible laser pattern LP irradiated onto the step 320A. Therefore, when a straight visible laser pattern LP is irradiated onto the snow surface S, it is important that the laser irradiation unit 130 is positioned so that the angles θ1 and θ2 are equal to or greater than the threshold value θth.

[0105] Fig. 29 is a diagram for explaining a method of irradiating a visible laser pattern LP in the driving control device 1000. Fig. 30(A) shows an example of a visible laser pattern LP when the angle θ1 between the line of sight EL of the driver DR and the laser beam La is smaller than a threshold value θth. Fig. 30(B) shows an example of a visible laser pattern LP when the angle θ1 between the line of sight EL of the driver DR and the laser beam La is equal to or greater than a threshold value θth. Figs. 29, 30(A), and 30(B) show examples of a visible laser pattern LP of a two-dimensional geometric pattern or a mesh-like pattern being irradiated onto a snow surface S.

[0106] The laser irradiation unit 130 scans the snow surface S with the laser beam La to draw a visible laser pattern LP, which is a two-dimensional geometric pattern or a mesh-like pattern, on the snow surface S. Therefore, while the laser irradiation unit 130 is drawing the visible laser pattern LP, the angle θ1 between the driver DR's line of sight EL and the laser beam La may be, for example, less than the threshold value θth as shown in FIG. 30(A) or greater than the threshold value θth as shown in FIG. 30(B). On the other hand, the visible laser pattern LP includes one or more straight lines extending in a direction intersecting the ruts 310. Therefore, regardless of the drawing position of the visible laser pattern LP by the laser irradiation unit 130, the one or more straight lines included in the visible laser pattern LP and extending in a direction intersecting the ruts 310 can make the ruts 310 appear three-dimensional. Therefore, when a visible laser pattern LP having a two-dimensional geometric pattern or a mesh-like pattern is projected onto the snow surface S, the position of the laser projection unit 130 can be anywhere in the front part of the vehicle 100.

[0107] Figures 31, 32, 33, and 34 show an example of a method for calculating the height of the snow bank 320. Figures 31 and 33 illustrate an example in which the long side of the elliptical calculation line Lc is fitted to the visible laser pattern LP when the height of the snow bank 320 is relatively low. Figures 32 and 34 illustrate an example in which the long side of the elliptical calculation line Lc is fitted to the visible laser pattern LP when the height of the snow bank 320 is relatively high. Figures 31 and 32 also illustrate an example in which the long side of the elliptical calculation line Lc is fitted to the visible laser pattern LP. Figures 33 and 34 illustrate an example in which the short side of the elliptical calculation line Lc is fitted to the visible laser pattern LP.

[0108] The snow-surface traveling path control unit 58 is capable of detecting a bending point P1 of the visible laser pattern LP included in the stereo image Ia or the distance image Ib. The snow-surface traveling path control unit 58 is capable of detecting a circular or elliptical area α of the visible laser pattern LP that is included from the detected bending point P1 to the leading edge P2 of the visible laser pattern LP. Based on the shape of the detected area α, the snow-surface traveling path control unit 58 is capable of estimating a height 320H from the traveling road surface 200 of a three-dimensional object (a rut 310 or a snow bank 320) that exists in the area α. Based on the estimated height 320H, the snow-surface traveling path control unit 58 determines whether the vehicle 100 needs to avoid the three-dimensional object (the rut 310 or the snow bank 320), and is capable of performing traveling control (e.g., steering control and braking control) of the vehicle 100 based on the determination result.

[0109] Figures 35, 36, 37, and 38 show an example of a method for calculating the height of the snow bank 320. Figures 35 and 37 illustrate an example in which the long side of the elliptical calculation line Lc is fitted to a single line segment (hereinafter referred to as "horizontal line segment Lx") that extends in a direction intersecting the step 320A of the snow bank 320 and is included in the visible laser pattern LP when the height of the snow bank 320 is relatively low. Figures 36 and 38 illustrate an example in which the long side of the elliptical calculation line Lc is fitted to a single horizontal line segment Lx that is included in the visible laser pattern LP when the height of the snow bank 320 is relatively high. Also, Figures 35 and 36 illustrate an example in which the long side of the elliptical calculation line Lc is fitted to the visible laser pattern LP. Figures 37 and 38 illustrate an example in which the short side of the elliptical calculation line Lc is fitted to the visible laser pattern LP.

[0110] 35 to 38 illustrate an example in which, of the multiple horizontal line segments Lx included in the visible laser pattern LP and extending in a direction intersecting with the step 320A of the snow bank 320, the horizontal line segment Lx closest to the vehicle 100 is selected as the line segment for fitting. The line segment for fitting may be a line segment different from the horizontal line segment Lx closest to the vehicle 100 among the multiple horizontal line segments Lx extending in a direction intersecting with the step 320A of the snow bank 320.

[0111] The snow-surface running path control unit 58 is capable of selecting one horizontal line segment Lx included in the visible laser pattern LP included in the stereo image Ia or the distance image Ib. The snow-surface running path control unit 58 is capable of detecting a bending point P1 of the selected horizontal line segment Lx. The snow-surface running path control unit 58 is capable of detecting a circular or elliptical area α included in the selected horizontal line segment Lx from the detected bending point P1 to the tip P2 of the selected horizontal line segment Lx. Based on the shape of the detected area α, the snow-surface running path control unit 58 is capable of estimating the height 320H from the traveling road surface 200 of a three-dimensional object (a rut 310 or a snow bank 320) present in the area α.

[0112] The snow-surface traveling path control unit 58 is capable of fitting a circular or elliptical calculation line Lc to the region α of the selected horizontal line segment Lx. The snow-surface traveling path control unit 58 is capable of estimating the height of the snow bank 320 as a distance 320H between a vertex P1 of the fitted calculation line Lc in a direction perpendicular to the traveling road surface 200 and a reference line Lr (reference position) described below. The reference line Lr is derived from the shape of the selected horizontal line segment Lx from the bending point P1 to a predetermined point P3 closer to the vehicle 100 (for example, the end of the selected horizontal line segment Lx).

[0113] Figure 39 shows a state in which a step 320A of a snow bank 320 snakes relative to the travel road surface 200. The snow bank 320 is formed, for example, by snow removal by a snow removal vehicle. When a snow removal vehicle snakes along the travel road surface 200, the step 320A of the snow bank 320 formed by snow removal by the snow removal vehicle also snakes along the travel road surface 200, as shown in Figure 39. Note that Figure 39 also shows a step 320B formed by snow removal by the snow removal vehicle before the step 320A was formed.

[0114] When a snow bank 320 exists on the travel road surface 200, the vehicle 100 generally needs to avoid the snow bank 320. However, if the height of the snow bank 320 is low enough for the vehicle 100, the vehicle 100 can travel over the snow bank 320 without avoiding it. In this embodiment, if the height of the snow bank 320 on the travel road surface 200 is low enough for the vehicle 100, the vehicle 100 can travel over the snow bank 320 without avoiding it. The processing procedure for traveling on a snowy surface S will be described below.

[0115] 40 shows an example of a processing procedure for traveling on a snowy surface S. The snowy traveling path control unit 58 estimates the position, shape, and height of the snow bank 320 using the calculation method described above (step S113). Next, the snowy traveling path control unit 58 determines whether or not a snow bank 320 exists on the traveling road surface 200 (step S113). If the snowy traveling path control unit 58 determines that a snow bank 320 exists on the traveling road surface 200 (step S113; Y), it determines whether or not the snow bank 320 should be avoided (step S115).

[0116] For example, when the height 320H is equal to or greater than a predetermined threshold, the snow-surface running path control unit 58 determines that the vehicle 100 needs to avoid the snow bank 320. For example, when the height 320H is less than the predetermined threshold, the snow-surface running path control unit 58 determines that the vehicle 100 does not need to avoid the snow bank 320. If the snow-surface running path control unit 58 determines that the vehicle 100 needs to avoid the snow bank 320 (step S115; Y), it performs running control (e.g., steering control and braking control) to avoid the snow bank 320 (step S116). If the snow-surface running path control unit 58 determines that the vehicle 100 does not need to avoid the snow bank 320 (step S115; N), it performs running control (e.g., steering control and braking control) of the vehicle 100 so that the vehicle 100 runs along the running road surface 200 (step S117). In this manner, running control for running on the snow surface S is performed.

[0117] The snow-surface running path control unit 58 may also determine whether or not the vehicle 100 should travel along the ruts 310. In this case, the snow-surface running path control unit 58 estimates the position, shape, and height of the ruts 310 using a calculation method similar to the calculation method described above. Next, the snow-surface running path control unit 58 determines that the vehicle 100 should travel along the ruts 310, for example, when the depth of the ruts 310 is equal to or greater than a predetermined threshold. The snow-surface running path control unit 58 determines that the vehicle 100 does not need to travel along the ruts 310, for example, when the height 320H is less than a predetermined threshold. If the snow-surface running path control unit 58 determines that the vehicle 100 should travel along the ruts 310, the snow-surface running path control unit 58 performs running control (e.g., steering control and braking control) to run along the ruts 310. When it is determined that the vehicle 100 does not need to travel along the ruts 310, the snowy roadway control unit 58 performs travel control (e.g., steering control and braking control) of the vehicle 100 so that the vehicle 100 travels along the travel road surface 200. In this way, travel control for traveling on the snowy surface S is performed.

[0118] Next, the effects of the vehicle 100 according to this modification will be described.

[0119] In this modification, the laser irradiation unit 130 is disposed inside one of the pair of headlights 110 of the vehicle 100 that is farther away from the roadside strip (snow bank 320). This allows the visible laser pattern LP, or one or more straight lines (line segments) included in the visible laser pattern LP, to intersect diagonally with the snow bank 320, making it possible for the visible laser pattern LP to three-dimensionally highlight the unevenness of the snow surface S. This makes it easier for the driver to see the unevenness of the snow surface S. As a result, safe driving becomes possible even in a snowstorm or at twilight.

[0120] In this modification, a bending point P1 of the visible laser pattern LP included in the stereo image Ia or the distance image Ib is detected, and a circular or elliptical region α of the visible laser pattern LP, extending from the detected bending point P1 to the leading edge P2 of the visible laser pattern LP, is detected. Based on the shape of the detected region α, a height 320H of a three-dimensional object (a rut 310 or a snow bank 320) present in the region α from the road surface 200 is estimated. Based on the estimated height 320H, it is determined whether the vehicle 100 needs to avoid the three-dimensional object (the rut 310 or the snow bank 320). This allows for driving control (e.g., steering control and braking control) of the vehicle 100 to be performed based on the determination result. As a result, safe driving is possible even in a snowstorm or at twilight.

[0121] In this modification, a circular or elliptical calculation line Lc is fitted to the area α of the selected horizontal line segment Lx, and the distance 320H between the apex P1 of the fitted calculation line Lc in the direction perpendicular to the traveled road surface 200 and the reference line Lr is estimated as the height of the snow bank 320. This makes it possible to calculate the height of the snow bank 320 with a small amount of calculation.

[0122] In this modification, when it is determined that the vehicle 100 does not need to avoid the snow bank 320, travel control (e.g., steering control and braking control) of the vehicle 100 is performed so that the vehicle 100 travels along the travel road surface 200 without avoiding the snow bank 320. This makes it possible to prevent the vehicle 100 from traveling along the snow bank 320 when there is no need to avoid it.

[0123] [3-2. Modification B] In the second embodiment and its modifications, the laser irradiation unit 130 may be capable of irradiating (drawing) an infrared laser pattern on the snow surface S, for example, under control of the control unit 150. In this case, the stereo camera 120 may be configured with a camera capable of detecting light in the infrared range.

[0124] The visible laser pattern LP is easily visible to the driver when the external illuminance is low, such as during a snowstorm, twilight, or nighttime. However, the visible laser pattern LP is difficult for the driver to see when the external illuminance is high, such as during the daytime. Therefore, in this modification, the laser irradiation control unit 52 is capable of selecting irradiation of the visible laser pattern LP when the external illuminance is low, such as during a snowstorm, twilight, or nighttime (e.g., when the external illuminance is within a first range). At this time, the laser irradiation control unit 52 is capable of outputting, for example, a control signal to the laser irradiation unit 130 that selects irradiation of the visible laser pattern LP. Furthermore, the laser irradiation control unit 52 is capable of selecting irradiation of the infrared laser pattern when the external illuminance is high, such as during the daytime (e.g., when the external illuminance is within a second range that is lower than the first range). At this time, the laser irradiation control unit 52 is capable of outputting, for example, a control signal to the laser irradiation unit 130 that selects irradiation of the infrared laser pattern.

[0125] When the infrared laser pattern is projected onto the snow surface S, the driver of the vehicle 100 cannot see the infrared laser pattern projected onto the snow surface S. However, if the stereo camera 120 is configured with a camera capable of detecting light in the infrared range, the infrared laser pattern is included in the stereo image Ia obtained by the stereo camera 120 and the distance image Ib obtained from the stereo image Ia. Therefore, the position, shape, and height of the rut 310 and snow bank 320 ahead of the vehicle 100 can be estimated based on the stereo image Ia and the distance image Ib obtained from the stereo camera 120, and driving control (e.g., steering control and braking control) of the vehicle 100 can be performed based on the estimation results. Therefore, safe autonomous driving is possible even when external illuminance is high, such as during the day.

[0126] In this way, in this modification, either the visible laser pattern LP or the infrared laser pattern is selected depending on the external illuminance, thereby enabling safe automatic driving regardless of the external illuminance.

[0127] Although the present disclosure has been described above using embodiments, the present disclosure is not limited to these embodiments and various modifications are possible. The effects described in this specification are merely examples, and the effects of the present disclosure are not limited to the effects described in this specification. Therefore, other effects may be obtained with respect to the present disclosure.

[0128] Furthermore, the present disclosure may take the following forms. (1) A driving assistance device including a control unit capable of providing driving assistance, wherein the control unit is capable of controlling the emission of a visible laser pattern or an infrared laser pattern, acquiring image data of a snow surface in front of the vehicle onto which the visible laser pattern or the infrared laser pattern is emitted, and estimating the shape of the snow surface in front of the vehicle based on the acquired image data. (2) The driving assistance device described in (1), wherein the control unit is capable of estimating the position and shape of ruts in front of the vehicle or the position and shape of snow banks in front of the vehicle based on the image data. (3) The driving assistance device described in (1) or (2), wherein the control unit is capable of estimating the shape of the snow surface in front of the vehicle by comparing the image data with reference image data. (4) The driving assistance device described in any one of (1) to (3), wherein the control unit is capable of controlling the emission of the visible laser pattern or the infrared laser pattern so that the visible laser pattern or the infrared laser pattern is a pattern that takes into account the estimation result. (5) The driving assistance device according to any one of (1) to (4), wherein the control unit is capable of controlling the steering angle during autonomous driving based on the estimation result. (6) The driving assistance device according to any one of (1) to (5), wherein the visible laser pattern or the infrared laser pattern is a two-dimensional geometric pattern. (7) The driving assistance device according to any one of (1) to (5), wherein the visible laser pattern or the infrared laser pattern is a mesh-like pattern. (8) The driving assistance device according to (6) or (7), wherein the visible laser pattern or the infrared laser pattern includes one or more straight lines or curves that diagonally intersect with the traveling direction of the vehicle.(9) The driving assist device according to (8), wherein the control unit is capable of: detecting a bending point in the visible laser pattern or the infrared laser pattern included in the acquired image data; detecting a circular or elliptical area in the visible laser pattern or the infrared laser pattern included from the detected bending point to a tip of the visible laser pattern or the infrared laser pattern; estimating a height from the road surface of a three-dimensional object present in the area based on the shape of the area obtained by detection; and determining whether or not the vehicle needs to avoid the three-dimensional object based on the estimated height. (10) The driving assist device according to (9), wherein the control unit is capable of fitting a circle or an ellipse to the shape of the area and estimating, as the height, a distance between a vertex of the fitted circle or ellipse in a direction perpendicular to the road surface and a reference position derived from the shape of the visible laser pattern or the infrared laser pattern from the bending point to a predetermined point closer to the vehicle. (11) The driving assist device according to (9) or (10), wherein the control unit is capable of controlling the driving of the vehicle without avoiding the three-dimensional object when it is determined that the vehicle does not need to avoid the three-dimensional object. (12) The driving assist device according to any one of (1) to (11), wherein the control unit is capable of selecting irradiation of the visible laser pattern when external illuminance is within a first range, and selecting irradiation of the infrared laser pattern when external illuminance is within a second range lower than the first range. (13) A vehicle including a control unit capable of performing driving assistance, wherein the control unit is capable of controlling irradiation of the visible laser pattern or the infrared laser pattern, acquiring image data of the snow surface in front of the vehicle onto which the visible laser pattern or the infrared laser pattern is irradiated, and estimating the shape of the snow surface in front of the vehicle based on the acquired image data.(14) The vehicle according to (13), further comprising a left headlight and a right headlight provided on each side of the front of the vehicle, wherein the light source unit is located inside the headlight of the left headlight or the right headlight that is farther from the shoulder. (15) A driving assistance method comprising: irradiating a snow surface in front of the vehicle with a visible laser pattern or an infrared laser pattern, acquiring image data of the snow surface in front of the vehicle onto which the visible laser pattern or the infrared laser pattern is irradiated, and estimating the shape of the snow surface in front of the vehicle based on the acquired image data. (16) A driving assistance device comprising: a control unit capable of providing driving assistance, wherein the control unit is capable of controlling the irradiation of the visible laser pattern or the infrared laser pattern, and acquiring image data of the snow surface in front of the vehicle onto which the visible laser pattern or the infrared laser pattern is irradiated, and the visible laser pattern or the infrared laser pattern is a two-dimensional geometric pattern or a mesh pattern.

[0129] The control unit 150 shown in FIGS. 15, 24, and 26 may be implemented by circuitry including at least one semiconductor integrated circuit, such as at least one processor (e.g., a central processing unit (CPU)), at least one application-specific integrated circuit (ASIC), and / or at least one field-programmable gate array (FPGA). The at least one processor may be configured to perform all or a portion of the various functions of the control unit 150 shown in FIGS. 15, 24, and 26 by reading instructions from at least one non-transitory, tangible computer-readable medium. Such medium may take various forms, including, but not limited to, various magnetic media such as hard disks, various optical media such as CDs or DVDs, and various semiconductor memories (i.e., semiconductor circuits) such as volatile or nonvolatile memories. Volatile memories may include DRAM and SRAM. Non-volatile memories may include ROM and NVRAM. An ASIC is an integrated circuit (IC) specialized to perform all or a portion of the various functions of the control unit 150 shown in FIGS. 15, 24, and 26. An FPGA is an integrated circuit that is designed to be configurable after manufacture to perform all or part of the various functions of the control unit 150 shown in FIGS.

Claims

1. A control unit capable of performing driving assistance is provided, The control unit Controlling the laser irradiation unit so that it does not irradiate the entire snow surface in front of the vehicle, but only irradiates specific areas of the snow surface with a visible laser pattern or an infrared laser pattern; acquiring image data of the snow surface in front of the vehicle onto which the visible laser pattern or the infrared laser pattern is irradiated; estimating the shape of the snow surface ahead of the vehicle based on the acquired image data; It is possible to carry out Driving aids.

2. A control unit capable of providing driving assistance, The control unit controlling the irradiation of a visible or infrared laser pattern; acquiring image data of the snow surface in front of the vehicle onto which the visible laser pattern or the infrared laser pattern is irradiated; estimating the shape of the snow surface ahead of the vehicle based on the acquired image data; Detecting bending points of the visible laser pattern or the infrared laser pattern included in the acquired image data; Detecting a circular or elliptical area included in the visible laser pattern or the infrared laser pattern from the detected bending point to a tip of the visible laser pattern or the infrared laser pattern; estimating the height of a three-dimensional object present in the area from a road surface based on the shape of the area obtained by detection; determining whether or not the vehicle needs to avoid the three-dimensional object based on the estimated height; It is possible to execute The visible laser pattern or the infrared laser pattern is a two-dimensional geometric pattern or a mesh-like pattern, and includes one or more straight lines or curves that intersect obliquely with the traveling direction of the vehicle. Driving aids.

3. The control unit is capable of estimating the position and shape of ruts or snow banks ahead of the vehicle based on the image data. The driving assistance device according to claim 1 or 2.

4. The control unit is capable of estimating the shape of the snow surface ahead of the vehicle by comparing the image data with reference image data. The driving assistance device according to claim 1 or 2.

5. The control unit is capable of controlling the irradiation of the visible laser pattern or the infrared laser pattern so that the visible laser pattern or the infrared laser pattern becomes a pattern that takes into account the estimation result. The driving assistance device according to claim 1 or 2.

6. The control unit is capable of controlling the steering angle during automatic driving based on the estimation result. The driving assistance device according to claim 1 or 2.

7. The visible laser pattern or the infrared laser pattern is a two-dimensional geometric pattern or a mesh-like pattern. The driving assistance device according to claim 1 .

8. The visible laser pattern or the infrared laser pattern includes one or more straight lines or curves that intersect obliquely with the traveling direction of the vehicle. The driving assistance device according to claim 7.

9. The control unit Detecting bending points of the visible laser pattern or the infrared laser pattern included in the acquired image data; Detecting a circular or elliptical area included in the visible laser pattern or the infrared laser pattern from the detected bending point to a tip of the visible laser pattern or the infrared laser pattern; estimating the height of a three-dimensional object present in the area from a road surface based on the shape of the area obtained by detection; determining whether or not the vehicle needs to avoid the three-dimensional object based on the estimated height; It is possible to carry out The driving assistance device according to claim 8.

10. The control unit is capable of fitting a circle or an ellipse to the shape of the area, and estimating, as the height, the distance between the vertex of the fitted circle or ellipse in the direction perpendicular to the road surface and a reference position derived from the shape of the visible laser pattern or the infrared laser pattern from the bending point to a predetermined point closer to the vehicle. The driving assistance device according to claim 9.

11. When the control unit determines that the vehicle does not need to avoid the three-dimensional object, the control unit is capable of controlling the vehicle to travel without avoiding the three-dimensional object. The driving assistance device according to claim 9.

12. The control unit is capable of selecting irradiation of the visible laser pattern when the external illuminance is in a first range, and selecting irradiation of the infrared laser pattern when the external illuminance is in a second range lower than the first range. The driving assistance device according to claim 1 .

13. A control unit capable of performing driving assistance; a light source unit that is capable of irradiating only specific portions of the snow surface in front of the vehicle with a visible laser pattern or an infrared laser pattern, without irradiating the entire snow surface in front of the vehicle, according to control by the control unit; Equipped with The control unit controlling the light source unit to irradiate the visible laser pattern or the infrared laser pattern; acquiring image data of the snow surface in front of the vehicle onto which the visible laser pattern or the infrared laser pattern is irradiated; estimating the shape of the snow surface ahead of the vehicle based on the acquired image data; It is possible to carry out vehicle.

14. a left headlight and a right headlight provided on the left and right sides of a front portion of the vehicle, The light source unit is disposed inside one of the left and right headlights, whichever is farther from the side of the road.

14. The vehicle of claim 13.

15. The visible laser pattern or the infrared laser pattern generated by the laser irradiation unit is irradiated only at a specific location on the snow surface in front of the vehicle, without irradiating the entire snow surface in front of the vehicle. acquiring image data of the snow surface in front of the vehicle onto which the visible laser pattern or the infrared laser pattern is irradiated; estimating the shape of the snow surface ahead of the vehicle based on the acquired image data; Contains Driving assistance methods.

16. A control unit capable of performing driving assistance is provided, The control unit Controlling the laser irradiation unit so that it does not irradiate the entire snow surface in front of the vehicle, but only irradiates specific areas of the snow surface with a visible laser pattern or an infrared laser pattern; acquiring image data of the snow surface in front of the vehicle onto which the visible laser pattern or the infrared laser pattern is irradiated; It is possible to execute The visible laser pattern or the infrared laser pattern is a two-dimensional geometric pattern or a mesh-like pattern. Driving aids.