Driving assistance device, vehicle, and driving assistance method

Abstract

A driving assistance device according to one embodiment of the present disclosure is provided with a control unit capable of executing driving assistance. The control unit is capable of: deriving a road surface irradiation position on the basis of the position of a road surface shape estimation marker which was detected by a sensor and which is disposed at a location spaced apart from the road surface of a road; and controlling the irradiation of visible laser light on the basis of the derived irradiation position.

Description

Driving assistance device, vehicle, and driving assistance method

[0001] The present disclosure relates to a driving assistance device mounted on a vehicle, a vehicle equipped with such a driving assistance device, and a driving assistance method implemented in such a driving assistance device.

[0002] In snowy regions, poor visibility (e.g., whiteout) due to snowstorms and the like can make it difficult for drivers to accurately grasp the shape of the roadway (driving lane). In such cases, the driver may, for example, deviate from the roadway (driving lane) or get stuck in a groove at the edge of the roadway (driving lane). Technologies that enable safe driving even in snowstorms have been proposed, for example, in Patent Documents 1 and 2.

[0003] JP 2022-53409 A JP 2010-221909 A

[0004] A driving assistance device according to an embodiment of the present disclosure includes a control unit capable of providing driving assistance, the control unit being capable of deriving an irradiation position on the road surface based on the position of a road surface shape estimation sign located at a position away from the road surface, which is detected by a sensor, and controlling irradiation of visible laser light based on the derived irradiation position.

[0005] A vehicle according to an embodiment of the present disclosure includes a control unit capable of providing driving assistance, the control unit being capable of deriving an irradiation position on the road surface based on the position of a road surface shape estimation sign located at a position away from the road surface, which is detected by a sensor, and controlling irradiation of visible laser light based on the derived irradiation position.

[0006] A driving assistance method according to an embodiment of the present disclosure includes the following three steps: (A) deriving an irradiation position on a road based on the position of a road surface shape estimation sign located at a position away from the road surface, which is detected by a sensor; and (B) irradiating a visible laser beam based on the derived irradiation position.

[0007] The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification. The drawings illustrate one embodiment and, together with the description, serve to explain the principles of the disclosure.

[0008] FIG. 1 is a diagram illustrating an example of the appearance of a front portion of a vehicle equipped with a driving assistance device according to an embodiment of the present disclosure. FIG. 2 is a diagram illustrating another example of the appearance of the front portion of the vehicle of FIG. 1. FIG. 3 is a diagram illustrating an example of a view ahead of the vehicle when viewed from the driver's seat. FIG. 4 is a diagram illustrating an example of a state in which visibility is poor due to a snowstorm or the like. FIG. 5 is a diagram illustrating an example of a state in which a linear visible laser pattern is projected from the vehicle of FIG. 1 onto the edge of a roadway (driving lane) estimated from the position of an arrow feather. FIG. 6 is a diagram illustrating an example of a state in which a linear visible laser pattern is projected from the vehicle of FIG. 1 onto the edge of a roadway (driving lane) and near the center line estimated from the position of an arrow feather. FIG. 7 is a diagram illustrating an example of a state in which a curved visible laser pattern is projected from the vehicle of FIG. 1 onto the edge of a roadway (driving lane) estimated from the position of an arrow feather. FIG. 8 is a diagram illustrating an example of a state in which a wide visible laser pattern is projected from the vehicle of FIG. 1 onto the road edge of the roadway (driving lane) estimated from the position of the arrow feathers. FIG. 9 is a diagram illustrating an example of a state in which a mesh-shaped visible laser pattern is projected from the vehicle of FIG. 1 onto the road edge of the roadway (driving lane) estimated from the position of the arrow feathers. FIG. 10 is a diagram illustrating an example of a functional block of a cruise control device mounted on the vehicle of FIG. 1. FIG. 11 is a diagram illustrating a method for calculating the irradiation position of the visible laser pattern. FIG. 12 is a diagram illustrating an example of a procedure for projecting a visible laser pattern in the cruise control device of FIG. 10. FIG. 13 is a diagram illustrating an example of a state in which a linear visible laser pattern is projected from the vehicle of FIG. 1 onto the road edge of the roadway (driving lane) estimated from the position of a snow pole. FIG. 14 is a diagram illustrating a method for calculating the irradiation position of the visible laser pattern.

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0010] <1. Background> In snowy regions, poor visibility (e.g., whiteout) can occur due to snowstorms and the like, making it difficult for drivers to accurately grasp the shape of the roadway (driving lane). In such cases, there is a risk that the vehicle may deviate from the roadway (driving lane) or get stuck in a groove at the edge of the roadway (driving lane). Technologies that enable safe driving even in snowstorms have been proposed, for example, in Patent Documents 1 and 2.

[0011] In the invention described in Patent Document 1, traffic safety features, including guardrails and white lines, are detected by microwave radar, and an image (laser image) including the traffic safety features at the detected location is projected onto the road surface, snow, etc. ahead of the vehicle. However, the invention described in Patent Document 1 cannot handle traffic safety features that are difficult to detect with microwave radar. Furthermore, since the invention described in Patent Document 1 projects a laser image including the traffic safety feature at the location of the detected traffic safety feature, even if a laser image is projected onto a sign that indirectly indicates the edge of the roadway (driving lane), such as an arrow feather or a snow pole, the driver cannot directly recognize the edge of the roadway (driving lane). Therefore, the invention described in Patent Document 1 may, for example, cause the vehicle to deviate from the roadway (driving lane) or get stuck in a groove at the edge of the roadway (driving lane).

[0012] In the invention described in Patent Document 2, the position of the arrow feathers is detected using a stereo camera, and the road shape is estimated based on the detection results. As a result, it is possible to perform driving control and issue danger warnings based on the estimated road shape. However, since the invention described in Patent Document 2 is premised on automatic driving control, there is a risk that the vehicle may deviate from the roadway (driving lane) or get stuck in a groove at the edge of the roadway (driving lane) during manual driving.

[0013] As described above, conventional systems have been unable to cope with the problem of safely driving in a snowstorm when manually driving. As a result of extensive research, the inventors of the present application have concluded that it is desirable to provide a driving assistance device that enables safe driving in a snowstorm when manually driving, a vehicle equipped with such driving assistance device, and a driving assistance method executed in such a driving assistance device.

[0014] Some exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Note that the following description illustrates one specific example of the present disclosure and should not be construed as limiting the present disclosure. For example, each element, including numerical values, shapes, materials, parts, the position of each part, and the connection method of each part, is merely an example and should not be construed as limiting the present disclosure. Furthermore, in the following exemplary embodiments, components not described in independent claims based on the highest concept of the present disclosure are optional and may be provided as needed. The drawings are schematic and are not intended to be drawn to scale. Throughout this specification and the drawings, components having substantially the same function and configuration are designated by the same reference numerals, and redundant description will be omitted. Furthermore, components not directly related to one embodiment of the present disclosure are not shown in the drawings.

[0015] 2. Embodiment [Configuration] FIG. 1 illustrates an example of the appearance of a front portion of a vehicle 100 according to an embodiment of the present disclosure. The drive system of the vehicle 100 is not particularly limited, and the vehicle 100 can be driven, for example, by at least one of an engine and a motor. As shown in FIG. 1 , the vehicle 100 includes a pair of left and right headlights 110 and a front window FW at its front portion. A laser irradiation unit 130 is provided inside the headlight 110, and a stereo camera 120 including a main camera 121 and a sub-camera 122 is provided in the vehicle interior CR visible from the outside through the front window FW. The location of the laser irradiation unit 130 is not particularly limited as long as it is located at the front portion of the vehicle 100. For example, the laser irradiation unit 130 may be provided in the vehicle interior CR as shown in FIG. 2.

[0016] 1 and 2 illustrate an example of a vehicle 100 traveling on a roadway (travel lane) 200 in a snowy region. A non-roadway 300 is provided beside the roadway 200. The non-roadway 300 includes, for example, a roadside, a shoulder, or a sidewalk. The surface of the roadway 200 (travel surface 210) is covered with snow FS, and ruts, for example, have been formed in the snow FS. Therefore, the surface of the snow FS (snow surface S) has various irregularities formed by the ruts, etc.

[0017] Fig. 3 shows an example of a view ahead of a vehicle 100 as viewed from the driver's seat while the vehicle is traveling on a roadway 200 covered with snow FS. For ease of explanation, the snow FS is omitted from Fig. 3. The roadway 200 borders a non-roadway 300. A white line 230, for example, is provided at the point of the roadway 200 where it borders the non-roadway 300. The roadway 200 borders an oncoming lane via a center line 220, for example.

[0018] As shown in Figure 3, a plurality of arrow feathers 400, which are a type of road surface shape estimation marking that indicates the position of the road edge (white line 230) of the roadway 200, are installed at predetermined intervals above the travel road surface 210. In other words, each arrow feather 400 is positioned away from the road surface (travel road surface 210) of the roadway 200. The road edge (white line 230) of the roadway 200 is positioned directly below each arrow feather 400. Each arrow feather 400 is connected to a support pole 410 fixed to the non-roadway 300, and is supported by the support pole 410.

[0019] 4 shows an example of poor visibility caused by a snowstorm, etc. As shown in Fig. 4, poor visibility caused by a snowstorm, etc., can make it impossible to see not only the white lines 230 and center lines 220, but also the outline and irregularities of the roadway 200, and even the support poles 410. In this case, there is a risk that the vehicle 100 may deviate from the roadway 200 or get stuck in a groove at the edge of the roadway 200, for example.

[0020] Therefore, as shown in FIG. 5 , the vehicle 100 can use the laser irradiation unit 130 to irradiate the snow surface S with a straight (linear) visible laser pattern LP1 (visible laser light). The visible laser pattern LP1 corresponds to the pattern of the road edge (white line 230) of the roadway 200. This allows the vehicle 100 to indicate the position of the road edge (white line 230) of the roadway 200 using the visible laser pattern LP1, making it easier for the driver to see the position of the road edge (white line 230) of the roadway 200. FIG. 5 illustrates an example in which the visible laser pattern LP1 is irradiated onto the snow surface S at a position including a position directly below the arrow feather 400. In FIG. 5 , one end of the visible laser pattern LP1 is positioned on the snow surface S directly below the arrow feather 400, and the other end of the visible laser pattern LP1 is positioned on the snow surface S near the front wheel (left wheel) of the vehicle 100.

[0021] 6 , the vehicle 100 may be capable of using the laser irradiation unit 130 to irradiate the snow surface S with not only a linear visible laser pattern LP1 but also a visible laser pattern LP2 (visible laser light). In this case, the vehicle 100 may be capable of irradiating the visible laser pattern LP2 at a position shifted a predetermined distance Δd1 toward the oncoming lane from the irradiation position of the visible laser pattern LP1. The position shifted a predetermined distance Δd1 toward the oncoming lane from the irradiation position of the visible laser pattern LP1 is near the center line 220.

[0022] For example, as shown in Fig. 7 , the vehicle 100 may be capable of using the laser irradiation unit 130 to irradiate a curved visible laser pattern LP1 onto the snow surface S. The visible laser pattern LP1 corresponds to the pattern of the road edge (white line 230) of the roadway 200. Fig. 7 illustrates an example in which the visible laser pattern LP1 is irradiated onto the snow surface S at positions including a position directly below the arrow feather 400. In Fig. 7 , one end of the visible laser pattern LP1 is located on the snow surface S at a position directly below the arrow feather 400, and the other end of the visible laser pattern LP1 is located on the snow surface S near the front wheel (left wheel) of the vehicle 100.

[0023] For example, as shown in Fig. 8 , the vehicle 100 may be capable of using the laser irradiation unit 130 to irradiate the snow surface S with a visible laser pattern LP1 that is wider than that in the above example. The visible laser pattern LP1 is a pattern that corresponds to the pattern of the road edge (white line 230) of the roadway 200. Fig. 8 illustrates an example in which the visible laser pattern LP1 is irradiated onto the snow surface S at a position that includes a position directly below the arrow feather 400. In Fig. 8 , one end of the visible laser pattern LP1 is located on the snow surface S at a position directly below the arrow feather 400, and the other end of the visible laser pattern LP1 is located on the snow surface S near the front wheel (left wheel) of the vehicle 100.

[0024] For example, as shown in Fig. 9 , the vehicle 100 may be capable of using the laser irradiation unit 130 to irradiate the snow surface S with a visible laser pattern LP1 that is wider and has a mesh shape than the above example. The visible laser pattern LP1 is a pattern that corresponds to the pattern of the road edge (white lines 230) of the roadway 200. Fig. 9 illustrates an example in which the visible laser pattern LP1 is irradiated onto the snow surface S at positions including a position directly below the arrow feather 400. In Fig. 9 , one end of the visible laser pattern LP1 is located on the snow surface S at a position directly below the arrow feather 400, and the other end of the visible laser pattern LP1 is located on the snow surface S near the front wheel (left wheel) of the vehicle 100.

[0025] The following describes in detail a cruise control device 1000 mounted on a vehicle 100 that is capable of irradiating at least the visible laser pattern LP1 of the visible laser patterns LP1 and LP2 onto a snow surface S. The cruise control device 1000 corresponds to a specific example of a "driving assistance device" according to an embodiment of the present disclosure. Hereinafter, the visible laser pattern LP will be collectively referred to as the visible laser pattern LP1 and the visible laser pattern LP2.

[0026] FIG. 10 shows an example of functional blocks of the cruise control device 1000. The cruise control device 1000 is configured to be able to project a visible laser pattern LP onto a snow surface S. The cruise control device 1000 includes, for example, a stereo camera 120, a laser projection unit 130, a control unit 150, a storage unit 160, and a control flag input unit 170. The cruise control device 1000 may further include components other than those shown in FIG. 10. FIG. 10 illustrates an example of a portion of the configuration of the cruise control device 1000.

[0027] The stereo camera 120 is fixed, for example, to the upper center of the vehicle interior CR and includes, for example, a main camera 121 and a sub-camera 122. The main camera 121 and the sub-camera 122 are autonomous sensors that sense the real space ahead of the vehicle 100. The main camera 121 and the sub-camera 122 are, for example, arranged at symmetrical positions on either side of the center portion in the width direction of the vehicle 100, and capture stereo images of the area ahead of the vehicle 100 from different viewpoints. The stereo camera 120 is further capable of outputting, for example, a pair of stereo images Ia of the area ahead of the vehicle 100 captured by the main camera 121 and the sub-camera 122 to the control unit 150 described below. The stereo camera 120 is further capable of generating a distance image Ib calculated from the amount of displacement between the positions of corresponding objects based on the pair of stereo images Ia, and outputting the distance image Ib to the control unit 150 described below.

[0028] The laser irradiation unit 130 is provided, for example, inside the headlight 110 or inside the vehicle interior CR, as shown in Figure 1 or Figure 2. The laser irradiation unit 130 is capable of irradiating (drawing) a visible laser pattern LP on the snow surface S under the control of the control unit 150.

[0029] The laser irradiation unit 130 includes, for example, a laser emitter capable of emitting visible laser light, an emission control driver capable of controlling the emission of the laser emitter, an optical system capable of scanning the snow surface S with the laser light, and a scan control driver capable of controlling the scanning of the laser light by the optical system. The emission control driver is capable of controlling the emission of the laser emitter under control of the control unit 150. The scan control driver is capable of controlling the operation of the optical system under control of the control unit 150. The laser emitter includes, for example, a semiconductor laser that emits visible laser light. The optical system is configured to include, for example, a polygon mirror and an fθ lens. The polygon mirror reflects the visible laser light emitted from the laser emitter and is capable of scanning the reflected visible laser light on the snow surface S via the fθ lens.

[0030] The control unit 150 is capable of controlling the entire vehicle 100. The control unit 150 is, for example, a so-called ECU (Electronic Control Unit) and is configured to include, for example, one or more processors and one or more memories. The control unit 150 may be configured to include, for example, a CPU (Central Processing Unit). In this case, the control unit 150 may be capable of controlling the entire vehicle 100 by, for example, executing a program stored in the storage unit 160.

[0031] The control unit 150 has, for example, a driving assistance unit 151 as shown in Fig. 10. The driving assistance unit 151 is capable of assisting the driver in driving the vehicle 100. The driving assistance unit 151 has, for example, a laser irradiation control unit 52, an irradiation position determination unit 53, and a road edge margin estimation unit 54 as shown in Fig. 10.

[0032] The irradiation position determination unit 53 is capable of estimating the position (position A, FIG. 11 ) of the arrow feather 400 based on the distance image Ib obtained from the stereo camera 120 (sensor). The irradiation position determination unit 53 is capable of estimating the position (position A) of the arrow feather 400 based on, for example, the parallax image of the stereo camera 120. The irradiation position determination unit 53 is capable of calculating the distance from the stereo camera 120 to the arrow feather 400 based on, for example, the degree of overlap between the image obtained by the main camera 121 and the image obtained by the sub-camera 122, and calculating position A based on the calculated distance. The distance can be calculated, for example, from the number of pixels in the parallax image.

[0033] The irradiation position determination unit 53 is further capable of estimating the position (position B, FIG. 11 ) directly below the arrow feather 400 on the road surface of the roadway 200 (traveling road surface 210) based on the position (position A) of the arrow feather 400. The position directly below the arrow feather 400 on the road surface of the roadway 200 (traveling road surface 210) refers to the position of the intersection between the perpendicular to the arrow feather 400 and the road surface of the roadway 200 (traveling road surface 210). The irradiation position determination unit 53 is capable of estimating the position (position B) directly below the arrow feather 400 on the road surface of the roadway 200 (traveling road surface 210) based on, for example, parallax images from the stereo camera 120. For example, when a perpendicular line is drawn from position A to road surface 210 in the parallax image of stereo camera 120, irradiation position determination unit 53 can calculate the distance between position A and a position (position B) where the perpendicular line intersects with road surface 210, and calculate position B based on the calculated distance. The distance can be calculated from the number of pixels in the parallax image, for example.

[0034] The irradiation position determination unit 53 is further capable of calculating, based on position B, the position (position C, FIG. 11 ) of the end of the visible laser pattern LP1 to be irradiated onto the snow surface S on the vehicle 100 side. The irradiation position determination unit 53 is capable of calculating, for example, the parallax image of the stereo camera 120 and the mounting position of the stereo camera 120, the position (position C) of the end of the visible laser pattern LP1 to be irradiated onto the snow surface S on the vehicle 100 side. The irradiation position determination unit 53 is capable of calculating, for example, position information (position D) of a portion of the vehicle 100 on the non-roadway 300 side based on the parallax image of the stereo camera 120 and the mounting position of the stereo camera 120, and setting position C based on the calculated position D. The irradiation position determination unit 53 is capable of outputting position information (position B, position C) obtained based on the position (position A) of the arrow feather 400 to the laser irradiation control unit 52 and the roadside margin estimation unit 54.

[0035] The laser irradiation control unit 52 is capable of controlling the irradiation (drawing) of the visible laser pattern LP. The laser irradiation control unit 52 is capable of generating a drawing control signal required for the irradiation (drawing) of the visible laser pattern LP based on, for example, drawing data 161 (described later) in the storage unit 160 and position information (position B, position C), and outputting the generated drawing control signal to the laser irradiation unit 130.

[0036] For example, the drawing data 161 is assumed to include two-dimensional coordinate data of a straight-line visible laser pattern LP1. In this case, the laser irradiation control unit 52 can correct the two-dimensional coordinate data of the straight-line visible laser pattern LP1 so that one end of the straight-line visible laser pattern LP1 is at position B and the other end of the straight-line visible laser pattern LP1 is at position C.

[0037] For example, the drawing data 161 is assumed to include two-dimensional coordinate data of a curved visible laser pattern LP1. In this case, the laser irradiation control unit 52 can correct the two-dimensional coordinate data of the curved visible laser pattern LP1 so that one end of the curved visible laser pattern LP1 is at position B and the other end of the curved visible laser pattern LP1 is at position C.

[0038] For example, the drawing data 161 may include two-dimensional coordinate data of the visible laser patterns LP1 and LP2. In this case, the laser irradiation control unit 52 can correct the two-dimensional coordinate data of the visible laser pattern LP2 so that the position of the visible laser pattern LP2 is shifted by a predetermined distance Δd1 toward the oncoming lane from the irradiation position of the visible laser pattern LP1.

[0039] It is assumed that a danger flag is not input from the road edge margin estimation unit 54 to the laser irradiation control unit 52. In this case, the laser irradiation control unit 52 is capable of, for example, not correcting the data relating to the width of the visible laser pattern LP1 in the two-dimensional coordinate data of the visible laser pattern LP1. When a danger flag is not input from the road edge margin estimation unit 54 to the laser irradiation control unit 52, the laser irradiation control unit 52 is further capable of, for example, generating a color control signal for setting the emission color of the visible laser pattern LP1 to a color (e.g., green) indicating a low danger level, and outputting the color control signal to the laser irradiation unit 130.

[0040] Assume that a danger flag is input from the road edge margin estimation unit 54 to the laser irradiation control unit 52. At this time, the laser irradiation control unit 52 is capable of correcting the two-dimensional coordinate data of the visible laser pattern LP1 so that the width of the visible laser pattern LP1 corresponds to the danger flag. The laser irradiation control unit 52 is further capable of generating a color control signal that sets the emission color of the visible laser pattern LP1 to a color corresponding to the danger flag and outputting the color control signal to the laser irradiation unit 130.

[0041] For example, if the danger flag is a flag indicating a medium danger level, the laser irradiation control unit 52 can correct the two-dimensional coordinate data of the visible laser pattern LP1 so that the width of the visible laser pattern LP1 is twice as large as usual. For example, if the danger flag is a flag indicating a medium danger level, the laser irradiation control unit 52 can further generate a color control signal that sets the emission color of the visible laser pattern LP1 to a color indicating a medium danger level (for example, yellow), and output the color control signal to the laser irradiation unit 130.

[0042] For example, if the danger flag is a flag indicating a high danger level, the laser irradiation control unit 52 can correct the two-dimensional coordinate data of the visible laser pattern LP1 so that the width of the visible laser pattern LP1 becomes three times larger than normal. For example, if the danger flag is a flag indicating a high danger level, the laser irradiation control unit 52 can further generate a color control signal that sets the emission color of the visible laser pattern LP1 to a color indicating a high danger level (for example, red), and output the color control signal to the laser irradiation unit 130.

[0043] For example, as shown in FIG. 11 , the road edge margin estimation unit 54 is capable of estimating the distance (road edge margin Δd2) between the vehicle 100 and the road edge of the roadway 200. For example, as shown in FIG. 11 , the road edge margin estimation unit 54 is capable of estimating the distance (road edge margin Δd2) between the vehicle 100 and the road edge of the roadway 200 based on position information (position B, position C) input from the irradiation position determination unit 53 and position information (position D) of a portion of the vehicle 100 on the non-roadway 300 side. Position D is, for example, position information of a portion of the vehicle 100 that is captured in the stereo image Ia. For example, as shown in FIG. 11 , the road edge margin estimation unit 54 is capable of estimating the position (position E, FIG. 11 ) directly below position D on the road surface of the roadway 200 (traveling road surface 210) based on position D. The road edge margin estimating unit 54 is capable of estimating the road edge margin Δd2 based on the positions B, C, and E, for example.

[0044] Based on the estimated road edge margin Δd2, the road edge margin estimating unit 54 is able to determine the risk of the vehicle 100 deviating from the road edge of the roadway 200. For example, when the road edge margin Δd2 is equal to or greater than the threshold value th1, the road edge margin estimating unit 54 determines that there is a low risk of the vehicle 100 deviating from the road edge of the roadway 200, and is able to not generate a risk flag.

[0045] For example, when the road edge margin Δd2 is equal to or greater than the threshold value th2 and smaller than the threshold value th1, the road edge margin estimation unit 54 determines that there is a relatively high risk that the vehicle 100 will deviate from the road edge of the roadway 200, and is able to generate a risk flag indicating a medium risk level and output it to the laser irradiation control unit 52. For example, when the road edge margin Δd2 is smaller than the threshold value th3, the road edge margin estimation unit 54 determines that there is an extremely high risk that the vehicle 100 will deviate from the road edge of the roadway 200, and is able to generate a risk flag indicating a high risk level and output it to the laser irradiation control unit 52.

[0046] For example, if the road edge margin estimation unit 54 calculates the distance between the white line 230 and the tire edge (road edge margin Δd2) when there is no snow on the road surface and stores this in the memory unit 160, it is also possible to set position C based on position B and the road edge margin Δd2 read out from the memory unit 160.

[0047] The road-edge margin estimation unit 54 may be capable of estimating the positional relationship between the vehicle 100 and the road-edge of the roadway 200, instead of the distance between the vehicle 100 and the road-edge of the roadway 200 (road-edge margin Δd2). An example of the positional relationship is the positional relationship between a structure at the road-edge of the roadway 200 and the end of the vehicle 100 on the non-roadway 300 side. In this case, the road-edge margin estimation unit 54 may be capable of determining the risk of the vehicle 100 deviating from the road-edge of the roadway 200, based on the positional relationship between the vehicle 100 and the road-edge of the roadway 200.

[0048] The storage unit 160 stores, for example, drawing data 161 necessary for drawing at least the visible laser pattern LP1 of the visible laser patterns LP1 and LP2. The drawing data 161 includes, for example, two-dimensional coordinate data of at least the visible laser pattern LP1 of the visible laser patterns LP1 and LP2. The storage unit 160 further stores, for example, a control flag 162 input from the control flag input unit 170. The storage unit 160 may also store, for example, a program executed by the control unit 150. This program causes the control unit 150 to execute a series of procedures for controlling the entire vehicle 100. The storage unit 160 is configured, for example, by a random access memory (RAM), a read-only memory (ROM), an auxiliary storage device (such as a hard disk), etc.

[0049] The control flag input unit 170 is capable of receiving input of the control flag 162 from the driver. The control flag input unit 170 is, for example, a paddle shifter attached to the steering wheel. For example, when the driver simultaneously presses and holds the left and right paddle shifters, the control flag input unit 170 can store "1" as the control flag 162 in the storage unit 160. For example, when the driver simultaneously presses and holds the left and right paddle shifters again after previously storing "1" as the control flag 162 in the storage unit 160, the control flag input unit 170 can store "0" as the control flag 162 in the storage unit 160. For example, when the driver simultaneously presses and holds the left and right paddle shifters again after previously storing "0" as the control flag 162 in the storage unit 160, the control flag input unit 170 can store "1" as the control flag 162 in the storage unit 160.

[0050] When the control flag 162 is "1", it means that the mode is, for example, laser irradiation mode. When the control flag 162 is "0", it means that the mode is, for example, normal mode in which laser irradiation is not performed automatically. Note that the values ​​that the control flag 162 can take are not limited to those mentioned above.

[0051] [Operation] Next, the operation of the driving control device 1000 will be described with reference to Fig. 12. Fig. 12 is a diagram for explaining an example of a visible laser pattern irradiation procedure in the driving control device 1000.

[0052] The driving assistance unit 151 reads the control flag 162 from the storage unit 160 and determines whether the read control flag 162 is on ("1") (step S101). If the control flag 162 is on ("1") (step S101; Y), the driving assistance unit 151 determines whether there is an abnormality in the stereo image Ia obtained from the stereo camera 120 (step S102).

[0053] In step S102, the driving assistance unit 151 determines whether the view ahead of the vehicle 100 included in the stereo image Ia is poorly visible due to a snowstorm (e.g., a whiteout). If the view ahead of the vehicle 100 included in the stereo image Ia is poorly visible due to a snowstorm (step S102; N), the driving assistance unit 151 determines whether the arrow feathers 400 can be detected in the stereo image Ia (step S103). If the arrow feathers 400 can be detected in the stereo image Ia (step S103; Y), the driving assistance unit 151 calculates the irradiation position of at least the visible laser pattern LP1 of the visible laser patterns LP1 and LP2 (step S104). Based on the calculated irradiation position, the driving assistance unit 151 irradiates the snow surface S with at least the visible laser pattern LP1 of the visible laser patterns LP1 and LP2 (step S105).

[0054] At this time, the driving assistance unit 151 irradiates the snow surface S with a straight (linear) visible laser pattern LP1, for example, as shown in FIG. 5 . The driving assistance unit 151 irradiates the snow surface S with not only the straight visible laser pattern LP1 but also a straight visible laser pattern LP2, for example, as shown in FIG. 6 . The driving assistance unit 151 irradiates the snow surface S with a curved visible laser pattern LP1, for example, as shown in FIG. 7 . The driving assistance unit 151 irradiates the snow surface S with a visible laser pattern LP1 of a thickness corresponding to the danger flag, for example, as shown in FIG. 8 . The driving assistance unit 151 irradiates the snow surface S with a visible laser pattern LP1 of a color corresponding to the danger flag, for example. The driving assistance unit 151 irradiates the snow surface S with a mesh-shaped visible laser pattern LP1, for example, as shown in FIG. 9 . In this manner, the visible laser pattern LP is irradiated by the cruise control device 1000.

[0055] If the control flag 162 is off (“0”) (step S101; N), if the view ahead of the vehicle 100 included in the stereo image Ia is in good visibility (step S102; Y), or if the arrow feathers 400 cannot be detected in the stereo image Ia (step S103; N), the driving assistance unit 151 does not emit laser light.

[0056] [Effects] Next, the effects of vehicle 100 according to this embodiment will be described.

[0057] In this embodiment, the irradiation position of the traveling road surface 210 (or snow surface S) is derived based on the position of an arrow feather 400, which is a type of road surface shape estimation mark located at a position away from the road surface (traveling road surface 210) of the roadway 200, detected by a sensor (stereo camera 120). Then, the irradiation of the visible laser pattern LP is controlled based on the derived irradiation position. This allows the driver to travel along the roadway (traveling lane) 200 by relying on the visible laser pattern LP, even when traveling in a snowstorm. As a result, during manual driving, the vehicle can avoid, for example, deviating from the roadway (traveling lane) or getting stuck in a groove at the edge of the roadway (traveling lane). Therefore, manual driving can be performed safely in a snowstorm.

[0058] In this embodiment, the visible laser pattern LP may be a straight line (linear) or mesh-shaped visible laser light projected onto the snow surface S. In this case, the driver can rely on the visible laser pattern LP to travel along the roadway (driving lane) 200. As a result, when the vehicle is being manually driven, it is possible to avoid, for example, the vehicle deviating from the roadway (driving lane) or getting stuck in a groove at the edge of the roadway (driving lane). Therefore, the vehicle can be safely driven in a snowstorm when being manually driven.

[0059] In this embodiment, at least one of the irradiation shape, irradiation range, and irradiation color of the visible laser pattern LP may be controlled based on the positional relationship or distance between the irradiation position and the vehicle 100. The irradiation range is a concept that includes whether only the visible laser pattern LP1 is irradiated or whether the visible laser pattern LP2 is also irradiated. In this case, the visibility of the visible laser pattern LP to the driver can be improved, or the driver can be notified of danger.

[0060] 3. Modifications The present disclosure has been described above by giving embodiments, but the present disclosure is not limited to these embodiments and various modifications are possible.

[0061] In the above embodiment, instead of the arrow feathers 400, for example, as shown in Fig. 13, a plurality of snow poles 500, which are a type of road surface shape estimation marking that indicates the position of the road edge (white line 230) of the roadway 200, may be placed at positions away from the road surface (traveling road surface 210) of the roadway 200. Each snow pole 500 is placed, for example, on the roadside, shoulder, or sidewalk, and is fixed at a position a predetermined distance away from the road edge (white line 230) of the roadway 200.

[0062] The irradiation position determination unit 53 is capable of estimating the position of the snow pole 500 (position F, FIG. 14 ) based on the distance image Ib obtained from the stereo camera 120. The irradiation position determination unit 53 is capable of estimating the position of the snow pole 500 (position F) based on, for example, the parallax image of the stereo camera 120. The irradiation position determination unit 53 is capable of calculating the distance from the stereo camera 120 to the arrow feather 400 from, for example, the degree of overlap between the image obtained by the main camera 121 and the image obtained by the sub-camera 122, and calculating the position F based on the calculated distance. The distance can be calculated, for example, from the number of pixels in the parallax image.

[0063] The irradiation position determination unit 53 can further estimate the position (position G, FIG. 14 ) of the road edge (white line 230) of the roadway 200 based on the position (position F) of the snow pole 500. For example, the irradiation position determination unit 53 can determine the position (position G) of the road edge (white line 230) of the roadway 200 by shifting a predetermined distance Δd3 toward the roadway 200 from the location estimated to be the ground contact point of the snow pole 500.

[0064] The irradiation position determination unit 53 is further capable of calculating the position (position H, FIG. 14 ) of the end of the visible laser pattern LP1 to be irradiated onto the snow surface S on the vehicle 100 side based on the position G. The irradiation position determination unit 53 is capable of calculating the position (position H) of the end of the visible laser pattern LP1 to be irradiated onto the snow surface S on the vehicle 100 side based on, for example, the parallax image of the stereo camera 120 and the mounting position of the stereo camera 120. The irradiation position determination unit 53 is capable of outputting position information (position G, position H) obtained based on the position (position F) of the arrow feather 400 to the laser irradiation control unit 52 and the roadside margin estimation unit 54.

[0065] In this modification, the irradiation position of the travel road surface 210 (or snow surface S) is derived based on the position of a snow pole 500, a type of road surface shape estimation mark located at a distance from the road surface (travel road surface 210) of the roadway 200, detected by a sensor (stereo camera 120). Then, the irradiation of the visible laser pattern LP is controlled based on the derived irradiation position. This allows the driver to navigate the roadway (travel lane) 200 by relying on the visible laser pattern LP, even when traveling in a snowstorm. As a result, during manual driving, the vehicle can avoid, for example, deviating from the roadway (travel lane) or getting stuck in a groove at the edge of the roadway (travel lane). Therefore, manual driving can be performed safely in a snowstorm.

[0066] Although the present disclosure has been described above using embodiments, the present disclosure is not limited to these embodiments and various modifications are possible. The effects described in this specification are merely examples, and the effects of the present disclosure are not limited to the effects described in this specification. Therefore, other effects may be obtained with respect to the present disclosure.

[0067] Furthermore, the present disclosure may take the following forms. (1) A driving assistance device including a control unit capable of providing driving assistance, wherein the control unit is capable of: deriving an irradiation position on the road surface based on the position of a road surface shape estimation sign positioned at a position away from the road surface of a roadway, detected by a sensor; and controlling irradiation of visible laser light based on the derived irradiation position. (2) The driving assistance device described in (1), wherein the road surface shape estimation sign is an arrow feather installed above the road surface, and the control unit is capable of setting the irradiation position to a linear position including an intersection of a perpendicular line of the arrow feather and the road surface. (3) The driving assistance device described in (1), wherein the road surface shape estimation sign is a snow pole installed on the side of the road, a shoulder, or a sidewalk, and the control unit is capable of setting the irradiation position to a linear position including a point shifted a predetermined distance toward the roadway from a point estimated to be the contact point of the snow pole. (4) The driving assistance device according to any one of (1) to (3), wherein the control unit is capable of controlling the irradiation of the visible laser light so as to irradiate linear or mesh-shaped light. (5) The driving assistance device according to any one of (1) to (4), wherein the control unit is capable of controlling at least one of the irradiation shape, irradiation range, and irradiation color of the visible laser light based on the positional relationship or distance between the irradiation position and a vehicle whose driving is assisted by the driving assistance device. (6) A vehicle including a control unit capable of performing driving assistance, wherein the control unit is capable of deriving an irradiation position on the road surface based on the position of a road surface shape estimation sign located at a position away from the road surface, detected by a sensor, and controlling the irradiation of the visible laser light based on the derived irradiation position. (7) A driving assistance method including: deriving an irradiation position on the road based on the position of a road surface shape estimation sign located at a position away from the road surface, detected by a sensor, and irradiating the visible laser light based on the derived irradiation position.

[0068] The control unit 150 shown in FIG. 10 can be implemented by circuitry including at least one semiconductor integrated circuit, such as at least one processor (e.g., a central processing unit (CPU)), at least one application-specific integrated circuit (ASIC), and / or at least one field-programmable gate array (FPGA). The at least one processor can be configured to perform all or a portion of the various functions of the control unit 150 shown in FIG. 10 by reading instructions from at least one non-transitory, tangible computer-readable medium. Such medium can take various forms, including, but not limited to, various magnetic media such as hard disks, various optical media such as CDs or DVDs, and various semiconductor memories (i.e., semiconductor circuits) such as volatile or non-volatile memories. Volatile memory can include DRAM and SRAM. Non-volatile memory can include ROM and NVRAM. An ASIC is an integrated circuit (IC) specialized to perform all or a portion of the various functions of the control unit 150 shown in FIG. 10. An FPGA is an integrated circuit designed to be configurable after manufacture to perform all or part of the various functions of the control unit 150 shown in FIG.

Claims

1. A driving assistance device comprising a control unit capable of performing driving assistance, the control unit being capable of executing the following: deriving an irradiation position of the road surface based on a position of a road surface shape estimation sign located at a position away from the road surface of a roadway, which is detected by a sensor; and controlling the irradiation of visible laser light based on the derived irradiation position.

2. The driving assistance device according to claim 1, wherein the road surface shape estimation mark is an arrow feather installed above the road surface, and the control unit is capable of setting the irradiation position to a linear position including an intersection point between the perpendicular line of the arrow feather and the road surface.

3. The driving assistance device according to claim 1, wherein the road surface shape estimation mark is a snow pole installed on the roadside, shoulder, or sidewalk, and the control unit is capable of setting the illumination position to a linear position including a point shifted a predetermined distance toward the roadway from a point estimated to be the contact point of the snow pole.

4. The driving assistance device according to claim 1, wherein the control unit is capable of controlling the irradiation of the visible laser light so as to irradiate a linear or mesh-shaped light.

5. The driving assistance device according to claim 1, wherein the control unit is capable of controlling at least one of the irradiation shape, irradiation range and irradiation color of the visible laser light based on the positional relationship or distance between the irradiation position and a vehicle whose driving is assisted by the driving assistance device.

6. A vehicle comprising a control unit capable of performing driving assistance, the control unit being capable of deriving an irradiation position of the road surface based on a position of a road surface shape estimation sign located at a position away from the road surface, detected by a sensor, and controlling the irradiation of visible laser light based on the derived irradiation position.

7. A driving assistance method comprising: deriving an irradiation position on the road based on the position of a road surface shape estimation sign located at a position away from the road surface detected by a sensor; and irradiating a visible laser light based on the derived irradiation position.

Images