Driver assistance system, estimation device, and vehicle control system

Abstract

The present invention provides a driver assistance system and estimation device that can appropriately estimate lane boundaries even when the road surface is covered with snow, and a vehicle control system that controls the vehicle based on the lane boundaries estimated by the estimation device. [Solution] A driving support system 1 that estimates the boundary of the lane in which a vehicle is traveling and displays the boundary on a screen, comprising a display device 3, an external environment acquisition device 2 that acquires information for identifying the type and position of objects located around the vehicle, and an estimation device 4 that estimates the boundary based on the acquisition results of the external environment acquisition device and displays the estimated boundary on the display device, wherein the estimation device acquires the positions of specific structures 13, 13A, 13B whose positions relative to the boundary are determined, and road surface information indicating the shape of the road surface, estimates the boundary based on the positions of the specific structures and the road surface information, and displays the estimated boundary on the display device.

Description

Technical Field

[0001] The present invention relates to a driving support system that estimates and displays the lane boundary based on road signs and the like, an estimation device that estimates the lane boundary based on road signs and the like, and a vehicle control system that controls a vehicle based on the lane boundary estimated by the estimation device.

Background Art

[0002] In recent years, efforts have been actively made to provide access to a sustainable transport system that takes into account people in vulnerable positions among traffic participants. As one of such efforts, research and development on driving support technologies for further improving traffic safety and convenience have attracted attention.

[0003] For example, Patent Document 1 discloses an image processing device for performing image display using a head-up display on a windshield located in front of a driver. The image processing device analyzes a captured image of a front camera, detects the position of a dividing line in the captured image, and superimposes and displays display elements.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In areas where a large amount of snow accumulates in winter, when a vehicle is running, the dividing line (outer lane line) indicating the lane boundary may be covered with snow.

[0006] However, if the dividing line indicating the lane boundary is covered with snow, the dividing line cannot be photographed by the front camera, so that it becomes difficult to detect the lane boundary by the method described in Patent Document 1.

[0007] In view of the above background, the present invention aims to provide a driver assistance system that can appropriately estimate lane boundaries even when the road surface is covered with snow, an estimation device that can appropriately estimate lane boundaries, and a vehicle control system that controls the vehicle based on the lane boundaries estimated by the estimation device. Ultimately, the aim is to contribute to the development of a sustainable transportation system. [Means for solving the problem]

[0008] To solve the above problems, one aspect of the present invention provides a driving support system (1) that estimates the boundary of a lane in which a vehicle is traveling and displays the boundary on a screen, comprising: a display device (3) that displays on a screen; an external environment acquisition device (2) that acquires information for identifying the type and position of objects located around the vehicle; and an estimation device (4) that estimates the boundary based on the acquisition results of the external environment acquisition device and displays the estimated boundary on the display device, wherein the estimation device acquires the position of specific structures (13, 13A, 13B) whose positions relative to the boundary are determined, and road surface information indicating the shape of the road surface, and estimates the boundary based on the position of the specific structures and the road surface information and displays the estimated boundary on the display device.

[0009] According to this embodiment, lane boundaries are estimated based on specific structures and road surface information. Therefore, even when the road surface is covered with snow, a driver assistance system can be provided that can appropriately estimate and display lane boundaries.

[0010] In the above embodiment, preferably, the specific structure (13A) includes a vane (16) positioned relative to the boundary, and the estimation device identifies the specific structure as the vane based on the shape of the specific structure.

[0011] According to this embodiment, the location of signs for identifying lane boundaries can be appropriately obtained.

[0012] In the above embodiment, preferably, the estimation device estimates the boundary by obtaining a road surface plane that approximates the road surface with a plane based on the road surface information, obtaining a virtual line (L0) that passes through the position of the arrow feather (M0) and extends downward, and obtaining the intersection point (P0) of the road surface plane and the virtual line.

[0013] According to this embodiment, lane boundaries can be appropriately estimated by a simple method.

[0014] In the above embodiment, preferably, the estimation device identifies the type of the specific structure, obtains a road surface plane that approximates the road surface with a plane based on the road surface information, obtains virtual lines (L1, L2) that pass through the endpoints (M1, M2) on the lane side of the specific structure (13B) and extend downward, obtains the intersection points (P1, P2) of the road surface plane and the virtual lines, and estimates the boundary based on the obtained intersection points and the type.

[0015] According to this embodiment, lane boundaries can be appropriately estimated by a simple method.

[0016] In the above embodiment, preferably, the display device includes a head-up display (3A) that displays on the windshield (11), and the estimation device causes the estimated boundary to be displayed as a virtual image on the head-up display.

[0017] According to this embodiment, the estimated lane boundaries can be displayed in a way that is easy for the driver to understand.

[0018] To solve the above problems, one aspect of the present invention provides an estimation device (4) for estimating the boundary of a lane on which a vehicle is traveling, comprising: a structure recognition unit (32) that acquires the position of specific structures (13, 13A, 13B) whose positions relative to the boundary are defined; a road surface acquisition unit (33) that acquires road surface information indicating the shape of the road surface; and a lane boundary estimation unit (34) that estimates the boundary based on the position of the specific structures and the road surface information.

[0019] According to this aspect, based on the position of a specific structure and road surface information indicating the shape of the road surface, the boundary of the lane is estimated. Therefore, even when the road surface is covered with snow, an estimation device that can appropriately estimate the boundary of the lane can be provided.

[0020] In the above aspect, preferably, the structure recognition unit is connected to an external acquisition device (2) that acquires information related to an object located around the vehicle, and the structure recognition unit acquires the position of the specific structure and the road surface information based on the acquisition result of the external acquisition device.

[0021] According to this aspect, the position of the specific structure and the road surface information can be appropriately acquired.

[0022] In the above aspect, preferably, the specific structure (13A) includes a chevron (16) whose position is determined with respect to the boundary, and the structure recognition unit specifies that the specific structure is the chevron based on the shape of the specific structure.

[0023] According to this aspect, by using the chevron, the influence of snow accumulation on the estimation of the lane boundary can be suppressed.

[0024] In the above aspect, preferably, the lane boundary estimation unit acquires a road surface plane (S) that approximates the road surface as a plane based on the road surface information, acquires a virtual line that passes through the position of the chevron and extends downward, and estimates the boundary by acquiring the intersection point between the road surface plane and the virtual line.

[0025] According to this aspect, the boundary of the lane can be appropriately estimated by a simple method.

[0026] In the above aspect, preferably, the structure recognition unit identifies the type of the specific structure, the lane boundary estimation unit acquires a road surface plane approximating the road surface in a plane based on the road surface information, acquires a virtual line that passes through an end point on the lane side of the specific structure (13B) and extends downward, acquires an intersection point between the road surface plane and the virtual line, and estimates the boundary based on the acquired intersection point and the type.

[0027] According to this aspect, the boundary of the lane can be appropriately estimated by a simple method.

[0028] In the above aspect, there is provided a vehicle control system (51) including the above-described estimation device and a vehicle control device (54) that controls the running of the vehicle based on the boundary estimated by the estimation device.

[0029] According to this aspect, the vehicle is controlled based on the estimated lane boundary based on the specific structure and the road surface information. Even when the road surface is covered with snow, a vehicle control system that can appropriately estimate the lane boundary and control the vehicle can be provided.

Advantages of the Invention

[0030] According to the above aspects, an operation support system that can appropriately estimate the lane boundary even when the road surface is covered with snow, an estimation device that can appropriately estimate the lane boundary, and a vehicle control system that controls the vehicle based on the lane boundary estimated by the estimation device can be provided.

Brief Description of the Drawings

[0031] [Figure 1] Block diagram of the operation support system according to the present invention [Figure 2] View showing the scenery seen from the inside of the vehicle equipped with the operation support system [Figure 3] First embodiment showing the scenery seen from the vehicle interior and the display content of the display device superimposed on the scenery [Figure 4]Functional block diagram of the estimation device [Figure 5] Diagram illustrating the method for obtaining road surface curvature. [Figure 6] (A) An explanatory diagram illustrating how to obtain lane boundaries based on arrowheads, (B) snow walls, and (C) utility poles. [Figure 7] A second embodiment showing the scenery viewed from inside the vehicle and the display content of a display device superimposed on that scenery. [Figure 8] A second embodiment showing the scenery viewed from inside the vehicle and the display content of a display device superimposed on that scenery. [Figure 9] A third embodiment showing the scenery viewed from inside the vehicle and the display content of a display device superimposed on that scenery. [Figure 10] A fourth embodiment showing the scenery viewed from inside the vehicle and the display content of a display device superimposed on that scenery. [Figure 11] Block diagram of the vehicle control system according to the present invention [Modes for carrying out the invention]

[0032] Hereinafter, embodiments of the driver assistance system, estimation device, and vehicle control system according to the present invention will be described with reference to the drawings.

[0033] <<First Embodiment>> The driver assistance system according to the first embodiment is installed in a vehicle and estimates the lane boundaries in which the vehicle is traveling, and presents them to the driver. The driver assistance system particularly assists the driver's driving operations in environments where lane boundaries are difficult to see due to reasons such as snow.

[0034] As shown in Figure 1, the driver assistance system 1 includes an external environment acquisition device 2, a display device 3, and an estimation device 4.

[0035] The external environment acquisition device 2 is mounted on the vehicle and detects information (hereinafter referred to as "external environment information") for identifying the type and location of objects located around the vehicle, and outputs it to the estimation device 4.

[0036] The external environment acquisition device 2 includes one or more external environment sensors 6 that detect information related to the vehicle's surrounding environment. The external environment acquisition device 2 may also include two or more external environment sensors 6.

[0037] The external sensor 6 may be a LiDAR (Laser Imaging Detection and Ranging) that acquires the positions of objects located on both sides by irradiating with laser light or infrared light and detecting the reflected waves. The external sensor 6 may also be a radar 8 (millimeter-wave radar) that acquires the positions of objects located around the vehicle by irradiating electromagnetic waves such as millimeter waves around the vehicle and detecting the reflected waves. In addition, the external sensor 6 may be an on-board camera 9 that photographs the area around the vehicle, including the area in front of the vehicle.

[0038] The external environment acquisition device 2 may include a plurality of external environment sensors 6 of different types. For example, the external environment acquisition device 2 may include at least one lidar 7 and at least one radar 8. In addition, the external environment acquisition device 2 may include at least one radar 8, at least one lidar 7 and at least one on-board camera 9.

[0039] The external environment acquisition device 2 communicates with the estimation device 4 via wired or wireless connection and outputs the acquired external environment information to the estimation device 4.

[0040] In this embodiment, the external environment acquisition device 2 includes a lidar 7, a radar 8, and an on-board camera 9, and the external environment acquisition device 2 outputs the respective detection results to the estimation device 4.

[0041] As shown in Figure 2, the display device 3 notifies the driver by displaying various information on its screen. In this embodiment, the display device 3 includes a known augmented reality head-up display 3A (also known as AR-HUD) that displays information on the windshield 11.

[0042] When the display device 3 (specifically, the head-up display 3A) obtains information such as the position, shape, and color of the object to be displayed from the estimation device 4, it projects a virtual image corresponding to that object onto the windshield 11 in front of the driver's seat. In this way, the display device 3 overlays the virtual image A corresponding to the object to be displayed onto the real scenery visible through the windshield 11.

[0043] The estimation device 4 uses the external information acquired by the external information acquisition device 2 to estimate the lane boundaries on the road and displays the estimated lane boundaries on the display device 3.

[0044] Specifically, the estimation device 4 uses external information to acquire the shape (preferably, shape and position) of structures 12 installed on or around the roadway, and identifies the type of structure 12. From the identified structures 12, the estimation device 4 extracts various structures 12 (hereinafter referred to as "specific structures 13") whose positions relative to the lane boundaries are defined, and acquires the position of the specific structures 13. Subsequently, the estimation device 4 estimates the lane boundaries based on road surface information indicating the shape of the road surface, and the position and type of the specific structures 13.

[0045] Furthermore, the specified structure 13 refers to a structure 12 whose boundary location can be determined by considering the road surface shape from its position, and specifically, it may include a structure 12 whose horizontal position relative to the lane boundary is defined.

[0046] The specified structure 13 includes, for example, a first specified structure 13A that overlaps with the lane boundary when viewed from above, such as a fixed visual guidance post 14 (see Figures 2 and 3). The fixed visual guidance post 14 is installed in areas with heavy snowfall where the road surface may be covered with snow, in order to indicate the shoulder (i.e., the lane boundary).

[0047] As shown in Figure 3, the fixed-type traffic guidance post 14 includes a support column 15 and an arrowhead 16 (more specifically, an arrowhead-shaped sign) supported by the support column 15. The support column 15 has a vertical beam section 15A fixed to the ground outside the roadway at its lower end, and a horizontal beam section 15B extending from the upper end of the vertical beam section 15A toward the roadway side (lane side). The arrowhead 16 is positioned so that its tip points downward vertically above the lane boundary and is fixed to the horizontal beam section 15B. That is, the lane boundary is located vertically below the arrowhead 16. Therefore, the tip of the arrowhead 16 is positioned so that it overlaps with the lane boundary in a top view.

[0048] In addition, the specific structure 13 used by the estimation device 4 to estimate the lane boundary includes a second specific structure 13B located at a distance corresponding to the left-right direction relative to the lane boundary in a top view. The lane boundary is located at a point a predetermined distance away from the intersection of a virtual line extending downward from the outer endpoint of the second specific structure 13B and the road surface, moving towards the road side (forward in the vehicle width direction). This distance can be determined according to the type of the second specific structure 13B.

[0049] The second specified structure 13B may include snow walls 12A, utility poles 12B, guardrails, streetlights, and various buildings. Snow walls 12A refer to walls of snow that are piled up on both sides of a road after snow removal.

[0050] As shown in Figure 1, the estimation device 4 may be composed of an electronic control unit (ECU) consisting of one or more computers configured to perform various processes. The estimation device 4 includes a processor 21 such as a CPU or MPU, memory 22 such as ROM or RAM, and storage 23 such as an SSD or HDD.

[0051] The processor 21 executes a lane estimation program stored in memory 22 or storage 23, and uses the external information acquired by the external information acquisition device 2 to perform a lane estimation method to estimate the position of the lane boundaries, and then performs the lane estimation process. In this embodiment, the processor 21 determines whether or not there is snow on the road surface, and if there is snow on the roadside, it performs the lane estimation method and displays the estimated lane boundaries on the display device 3.

[0052] As shown in Figure 4, the estimation device 4 includes, as functional units, a snow accumulation state determination unit 31 that determines the presence or absence of snow on the left and right sides in the direction of travel, a structure recognition unit 32, a road surface acquisition unit 33, and a lane boundary estimation unit 34 for lane estimation processing, and a display processing unit 35 that controls the display on the display device 3.

[0053] The snow accumulation condition determination unit 31 may be configured by the processor 21 executing a predetermined program. Based on external information acquired by the external information acquisition device 2, the snow accumulation condition determination unit 31 determines whether or not there is snow accumulation on the left and right sides in the direction of travel.

[0054] When the vehicle is equipped with a lidar 7 that emits near-infrared light, the snow accumulation determination unit 31 may determine the presence or absence of snow on the left and right sides in the direction of travel by calculating the intensity of reflected light at wavelengths that are not easily absorbed by either water or ice, and wavelengths that are not easily absorbed by ice but are easily absorbed by water.

[0055] In addition, the snow accumulation condition determination unit 31 may analyze the images captured by the on-board camera 9 using a machine learning method based on a known learning model to determine whether or not there is snow accumulation on the left and right sides in the direction of travel.

[0056] The structure recognition unit 32 may be configured by the processor 21 executing a lane estimation program. The structure recognition unit 32 uses external information acquired by the external information acquisition device 2 to acquire the position and type of structure 12 located in front of the vehicle. The structure recognition unit 32 may also acquire the position and type of structure 12 located in front of the vehicle by acquiring multiple detection results from the lidar 7, radar 8, and on-board camera 9 and performing sensor fusion processing.

[0057] The structure recognition unit 32 recognizes structures 12, including specific structures 13 such as fixed-type visual guidance posts 14. The structure recognition unit 32 uses external information acquired by the external information acquisition device 2 to acquire the shape of the structure 12. Subsequently, the structure recognition unit 32 identifies the type of structure 12 based on the acquired shape and determines whether or not it falls under the category of specific structures 13. If the recognized structure 12 falls under the category of specific structures 13, the structure recognition unit 32 then acquires the type of specific structure 13 and the relative position of the specific structure 13 with respect to the vehicle. When the structure recognition unit 32 recognizes a fixed-type visual guidance post 14, the structure recognition unit 32 acquires the relative position of the tip of the arrow vane 16 with respect to the vehicle.

[0058] The structures 12 recognized by the structure recognition unit 32 may include various traffic signs other than the fixed traffic guidance poles 14. When a traffic sign is recognized, the structure recognition unit 32 may acquire the information indicated by the traffic sign. The structure recognition unit 32 may also constitute part of a traffic recognition system that realizes a so-called TSR (Traffic Sign Recognition) function, which recognizes road signs installed on the roadway and notifies the driver of the recognized information at an appropriate time.

[0059] In this embodiment, the structure recognition unit 32 is also configured to recognize a snow wall 12A as a specific structure 13. The structure recognition unit 32 may determine whether the detected structure 12 is made of snow and recognize the snow wall 12A based on a method similar to that of the snow accumulation condition determination unit 31 (for example, the intensity of reflected light).

[0060] The road surface acquisition unit 33 may be configured by the processor 21 executing a lane estimation program. The road surface acquisition unit 33 uses external information acquired by the external information acquisition device 2 to acquire road surface information indicating the shape of the road surface located in front of the vehicle.

[0061] The road surface acquisition unit 33 may be configured to acquire a road surface plane S, which approximates the shape of the road surface using a plane, as road surface information. In this case, the road surface acquisition unit 33 may acquire the coordinates (more specifically, relative coordinates with respect to the vehicle) of three or more points located on the road surface (T1, T2, T3 in Figure 3), and acquire the plane passing through those points as the road surface plane S.

[0062] The road surface acquisition unit 33 may also acquire the road surface plane S by using parameters such as the vehicle height, the mounting positions of the rider 7 and radar 8, which are pre-set at the time of factory shipment and stored in the storage 23 (or memory 22), as the normal vector for the vector pointing in the vertical direction, and the plane passing under the wheels.

[0063] The road surface acquisition unit 33 may also be configured to acquire a road surface curve U, which approximates the shape of the road surface using a curved surface, as road surface information. In that case, as shown in Figure 5, the road surface acquisition unit 33 acquires a point cloud VL located in front of the left wheel and a point cloud VR located in front of the right wheel at predetermined distances toward the front of the vehicle. Subsequently, the road surface acquisition unit 33 may acquire the road surface curve U by passing through points that are aligned left and right within the left and right point clouds VL and VR, assuming extrapolation lines W that extend outward to the left and right from the left and right point clouds VL and VR, and connecting them in the longitudinal direction.

[0064] When the snow accumulation condition determination unit 31 determines that there is snow on either the left or right side of the road surface in the direction of travel, and the structure recognition unit 32 recognizes the first specific structure 13A, the lane boundary estimation unit 34 estimates the position of the lane boundary line and obtains the coordinates of the lane boundary line based on the position of the first specific structure 13A and the road surface information obtained by the road surface acquisition unit 33.

[0065] Specifically, the lane boundary estimation unit 34 acquires the coordinates of points on the road surface plane S or road surface curve U in the relative direction corresponding to the first specific structure 13A as the coordinates of a point cloud indicating the lane boundary line (hereinafter simply referred to as the lane boundary line).

[0066] When the structure recognition unit 32 recognizes a structure 12 as a vane 16, it calculates a virtual line L0 that passes through the position M0 of the vane 16 (in this embodiment, the tip of the vane 16) and extends vertically downward, as shown in Figure 6(A). Subsequently, the lane boundary estimation unit 34 calculates the intersection point P0 between each virtual line L0 and the road surface plane S or the road surface curve U. When multiple vanes 16 are recognized by the structure recognition unit 32, the lane boundary estimation unit 34 obtains the estimated lane boundary line by connecting the calculated intersection points P0. In this way, by using vanes 16, the lane boundary can be obtained well and simply while suppressing the effects of snow accumulation.

[0067] At this time, the lane boundary estimation unit 34 obtains the coordinates of the lane boundary in a relative coordinate system with the vehicle's position as the origin. However, it is not limited to this embodiment, and when the vehicle is equipped with a GNSS receiver 36, the lane boundary estimation unit 34 may obtain coordinates (latitude, longitude) indicating the vehicle's position in the Earth coordinate system based on the reception results of the GNSS receiver 36, and use these to obtain the coordinates of the lane boundary in the Earth coordinate system.

[0068] In some cases, the structure recognition unit 32 may not recognize the first specific structure 13A, such as the arrowhead 16, but may recognize the snow wall 12A (an example of the second specific structure 13B). In this case, as shown in Figure 6(B), the lane boundary estimation unit 34 acquires the point on the roadside of the snow wall 12A that is located a predetermined distance forward from the vehicle and is closest to the road. Alternatively, the lane boundary estimation unit 34 may acquire the point on the snow wall 12A that is located a predetermined distance forward from the vehicle and is closest to the vehicle's centerline as the roadside endpoint M1 of the snow wall 12A. Next, the lane boundary estimation unit 34 calculates a virtual line L1 that extends downward, passing through the roadside endpoint M1 of the snow wall 12A. Subsequently, the lane boundary estimation unit 34 acquires the intersection point P1 between the virtual line L1 and the road surface plane S (or the road surface curve U) as a point indicating the lane boundary. Alternatively, the lane boundary estimation unit 34 may acquire a point indicating the lane boundary at a predetermined distance (hereinafter, first distance d1) from the intersection point P1 between the virtual line and the road surface plane S toward the roadway (forward side in the vehicle width direction). The first distance d1 is a predetermined value of 0m or more, and can be set to a distance of 1m or less, preferably 0.8m or less. The lane boundary estimation unit 34 acquires points indicating the lane boundary while changing the distance from the vehicle, and may acquire a line connecting the acquired points indicating the lane boundary as the estimated lane boundary line.

[0069] In some cases, the structure recognition unit 32 fails to recognize either the first specified structure 13A, such as the arrow vane 16, or the snow wall 12A, and instead detects the second specified structure 13B, excluding the snow wall 12A, such as the utility pole 12B, streetlights, guardrails, buildings, and other structures located near the roadway. In this case, as shown in Figure 6(C), the lane boundary estimation unit 34 acquires the point on the roadway side of the second specified structure 13B's wall surface that is located a predetermined distance forward from the vehicle and is at the end of the wall surface of the second specified structure 13B (the point closest to the road), as the roadway-side endpoint M2 of the second specified structure 13B. In this case, the lane boundary estimation unit 34 may also acquire the point on the wall surface of the second specified structure 13B that is located a predetermined distance forward from the vehicle and is closest to the vehicle's centerline as the roadway-side endpoint M2 of the second specified structure 13B. Furthermore, the lane boundary estimation unit 34 calculates a virtual line L2 that extends downward, passing through the roadway-side endpoint M2 of the second specific structure 13B. Subsequently, the lane boundary estimation unit 34 may acquire the intersection point P2 of the virtual line L2 and the road surface plane S (or road surface curve U) as a point indicating the lane boundary. Alternatively, the lane boundary estimation unit 34 may acquire a point Q indicating the lane boundary, which is a predetermined distance (hereinafter referred to as the second distance d2) moved from the intersection point P2 of the virtual line and the road surface plane S toward the roadway side in the vehicle width direction (the side approaching the centerline of the vehicle body). The second distance d2 is a predetermined value of 0m or more, and may be set to a distance of approximately 0.5m to 1.5m. The lane boundary estimation unit 34 may acquire points indicating the lane boundary while changing the distance from the vehicle, and acquire the line connecting the acquired points indicating the lane boundary as the estimated lane boundary line.

[0070] When the lane boundary estimation unit 34 acquires the lane boundary line, it outputs the acquired lane boundary line (more specifically, the coordinates of the point cloud indicating the lane boundary line) to the display processing unit 35.

[0071] When the display processing unit 35 acquires the lane boundary line, it causes the display device 3 to display a virtual image A to indicate the lane boundary line. Specifically, the display processing unit 35 uses the coordinates of the acquired point cloud indicating the lane boundary line to set a frame corresponding to the outer edge of the white line drawn on the road surface. Next, the display processing unit 35 assumes a virtual straight line connecting each point on the frame to the driver's eye position, and acquires the intersection point between this virtual straight line and the projection surface (display surface) of the windshield 11 of the display device 3. After that, the display processing unit 35 controls the display device 3 to color and display the area enclosed by that intersection point. As a result, the display device 3 displays the white line indicating the lane boundary line superimposed as a virtual image A on the real scenery visible through the windshield 11.

[0072] If a vehicle is equipped with a car navigation system (not shown), the display processing unit 35 may be configured to obtain the number of lanes and road width on which the vehicle is traveling from the map information stored in the car navigation system, and to fill in any missing lane boundaries and various road markings, as shown in Figure 7, and display the corresponding virtual image B.

[0073] In addition, the display processing unit 35 may color the area C outside the lane and display it on the display device 3, as shown in Figure 8. This allows the driver to easily recognize the area C outside the lane.

[0074] Furthermore, the display processing unit 35 may cause the display device 3 to display a virtual image D corresponding to the structure 12 recognized by the structure recognition unit 32, colored as shown in Figure 9. The color assigned to the virtual image may be determined based on the type of structure 12. For example, when the structure recognition unit 32 recognizes a snow wall 12A, a guardrail, etc., the display processing unit 35 may assign a different color to the corresponding virtual image D and display it on the display device 3.

[0075] As shown in Figure 10, the display processing unit 35 may display text 37 on the display device 3 indicating the recommended vehicle speed during snowfall (hereinafter referred to as the recommended vehicle speed) along with a virtual image A corresponding to the lane boundary line. When displaying the recommended vehicle speed, the display processing unit 35 should first refer to map information to obtain the speed limit of the road the vehicle is traveling on, and then substitute this into the following equation (1) to obtain the recommended vehicle speed.

[0076]

number

[0077] Note that in equation (1), v rec The recommended vehicle speed is v u The speed limit is μ s μ0 represents the average road surface friction coefficient when covered with snow, while μ0 represents the average road surface friction coefficient of dry asphalt.

[0078] The display processing unit 35 may, for example, obtain the vehicle speed from the vehicle speed sensor 38 used for VSA (Vehicle Stability Assist), and if the obtained vehicle speed is greater than the recommended vehicle speed, it may display a warning message 39 such as "Please slow down a little more" on the display device 3.

[0079] Next, we will explain the operation of the estimation device 4 configured in this way, and its effects.

[0080] When the snow accumulation condition determination unit 31 determines that there is snow on either the left or right side of the road surface in the direction of travel, and the structure recognition unit 32 recognizes a specific structure 13, the lane boundary estimation unit 34 estimates the position of the lane boundary based on the type and location of the specific structure 13 and road surface information indicating the shape of the road surface.

[0081] Once the lane boundary estimation unit 34 has completed estimating the position of the lane boundary, the display processing unit 35 displays the estimated lane boundary as a virtual image A on the head-up display 3A. As a result, the virtual image A, which represents the estimated lane boundary, is superimposed on the real scenery on the windshield 11.

[0082] The first specified structure 13A is positioned so as to overlap with the lane boundary when viewed from above. Therefore, the lane boundary estimation unit 34 can obtain a point on the road surface indicating the lane boundary by tracing downwards (vertically downwards) from the position of the first specified structure 13A. Thus, even when the road surface is covered with snow, a point indicating the lane boundary can be obtained using the relative direction from the first specified structure 13A and road surface information indicating the road surface shape.

[0083] The position of the second specified structure 13B relative to the lane boundary is determined from a top view. Therefore, the lane boundary estimation unit 34 can obtain a point on the road surface indicating the lane boundary by hanging down from the position of the second specified structure 13B and moving toward the roadway by a distance corresponding to its type. Thus, even when the road surface is covered with snow, a point indicating the lane boundary can be obtained using the type and position of the second specified structure 13B and road surface information indicating the road surface shape.

[0084] This enables the provision of an estimation device 4 that can appropriately estimate lane boundaries even when it is difficult to obtain the white lines indicating lane boundaries, specifically, for example, when the road surface is covered with snow, and a driving support system 1 that displays the lane boundaries estimated by the estimation device 4. This reduces anxiety during driving and improves the safety of the driver's driving when it is difficult to confirm lane boundaries due to snow. As a result, traffic safety and convenience will be further improved, and ultimately, the estimation device 4 and the driving support system 1 can each contribute to the development of a sustainable transportation system.

[0085] In this embodiment, the specified structure 13 (first specified structure 13A) includes arrow vanes 16 that overlap the lane boundaries when viewed from above. Since the arrow vanes 16 are positioned away from the road surface to minimize the effects of snow accumulation, using the arrow vanes 16 can reduce the influence of snow accumulation on lane boundary estimation.

[0086] The structure recognition unit 32 acquires the position of the specific structure 13 based on the results acquired by the external environment acquisition device 2. Furthermore, the road surface acquisition unit 33 acquires road surface information, including the road surface plane S and the road surface curve U, based on the results acquired by the external environment acquisition device 2. Therefore, since the external environment acquisition device 2 acquires the information necessary to acquire the position of the specific structure 13 and the road surface information, the configuration is simple.

[0087] The lane boundary estimation unit 34 obtains a road surface plane S that approximates the road surface with a plane based on road surface information, and obtains a virtual line L0 that passes through the position M0 of the arrow feather 16 and extends downward. Next, the lane boundary estimation unit 34 estimates the lane boundary by obtaining the intersection point P0 of the road surface plane S and the virtual line L0. Therefore, the lane boundary can be obtained appropriately and simply. Furthermore, when the lane boundary estimation unit 34 estimates the lane boundary using the road surface curve U, it can appropriately estimate the lane boundary even when the road surface is close to a curve, such as on a mountain road with sharp changes in incline and descent.

[0088] <<Second Embodiment>> The estimation device 4 according to the second embodiment constitutes part of the vehicle control system 51 to which the vehicle autonomously drives. Hereinafter, an embodiment of the vehicle control system 51 to which the estimation device 4 is applied will be described.

[0089] As shown in Figure 11, the vehicle control system 51 includes, in addition to the external environment acquisition device 2, display device 3, and estimation device 4 according to the first embodiment, a vehicle position identification device 52, a map information storage device 53, and a vehicle control device 54.

[0090] The vehicle positioning device 52 is a device for determining the position of the vehicle and includes a GPS receiver mounted on the vehicle. Based on the reception results from the GNSS receiver 36, the vehicle positioning device 52 acquires coordinates (latitude and longitude) indicating the position of the vehicle in the Earth coordinate system and outputs them to the vehicle control device 54.

[0091] The map information storage device 53 is equipped with storage 56. Storage 56 stores map information 56A for autonomous driving of the vehicle. The map information 56A stored in the map information storage device 53 preferably includes so-called high-precision maps, which include data such as the position information of each lane, the position of lane markings, and the gradient and curvature of the road.

[0092] The vehicle control device 54 may consist of an electronic control unit (ECU) comprising one or more computers configured to perform various processes. The vehicle control device 54 includes a processor 61 such as a CPU or MPU, memory 62 such as ROM or RAM, and storage 63 such as an SSD or HDD. The vehicle control device 54 (specifically the processor 61) controls the braking device 71, drive device 72 such as a motor, and steering device 73 provided in the vehicle based on the acquisition results from the external environment acquisition device 2, the map information stored in the map information storage device 53, and the lane boundaries estimated by the estimation device 4, thereby enabling the vehicle to drive autonomously.

[0093] The vehicle control device 54 may change the vehicle control based on the determination result by the snow condition determination unit 31. When the snow condition determination unit 31 determines that there is snow and the vehicle is turning, the vehicle control device 54 may control the braking device 71 to apply the brakes to the wheels more gradually in accordance with the curvature of the road, compared to when there is no snow. In addition, when there is snow, the vehicle control device 54 may control the steering device 73, braking device 71, and drive device 72 to prevent the vehicle from deviating from the lane markings by correcting the control amount and control timing of the steering device 73 compared to when there is no snow.

[0094] Next, the effects of the vehicle control system 51 configured in this way will be explained.

[0095] This makes it possible to provide a vehicle control system 51 that can appropriately estimate lane boundaries and control the vehicle even when the road surface (especially the lane markings) is difficult to see, specifically, for example, when the road is covered with snow.

[0096] This concludes the description of specific embodiments, but the present invention is not limited to the above embodiments or modifications and can be broadly modified and implemented.

[0097] In the above embodiment, when the snow accumulation condition determination unit 31 determines that there is snow on the left and right sides in the direction of travel, the lane boundary estimation unit 34 estimates the position of the lane boundary line and displays it on the display device 3. However, determining the snow accumulation condition is not essential. That is, the system may be configured to display the lane boundary line estimated by the lane boundary estimation unit 34 on the display device 3 regardless of whether there is snow or not. In addition, the estimation device 4 may be configured to estimate the lane boundary line and display it on the display device 3 when it is difficult to obtain the lane boundary line, such as when the white line indicating the lane boundary line is faded or when the lane boundary line is hidden by an object. This allows the driver to easily see the lane boundary line even when the lane boundary line or shoulder is difficult to see, thereby reducing the driver's anxiety during driving and improving the safety of the driver's driving.

[0098] Furthermore, when the estimation device 4 is used to estimate lane boundaries when the road surface is not covered with snow, the estimation device 4 (display processing unit 35) should stop displaying the recommended vehicle speed.

[0099] In the above embodiment, an example was described in which the snow accumulation condition determination unit 31 calculates the intensity of reflected light to determine whether or not there is snow accumulation on the left and right sides in the direction of travel. However, the determination of whether or not there is snow accumulation may also be made based on whether or not the lane boundary can be recognized by the external environment acquisition device 2. Specifically, the snow accumulation condition determination unit 31 may determine that there is snow accumulation when the lane boundary cannot be recognized by the external environment acquisition device 2, and determine that there is no snow accumulation when the lane boundary can be recognized by the external environment acquisition device 2.

[0100] In the above embodiment, an example was described in which the display device 3 is configured as a head-up display 3A, but the embodiment is not limited to this, and the display device 3 may be configured as a display 70 (see Figure 2) provided near the driver's seat. In that case, the estimation device 4 may display the estimated lane boundaries superimposed on the image captured by the on-board camera 9.

[0101] In the above embodiment, examples were described in which the estimation device 4 is applied to the driver assistance system 1 and the vehicle control system 51, but the present invention is not limited thereto. The estimation device 4 can also be applied to work machines that perform various tasks by estimating lane boundaries (for example, snowplows, etc.), various robots, aircraft, and guidance systems that guide pedestrians, etc. [Explanation of symbols]

[0102] 1: Driver assistance system 2: External world acquisition device 3:Display device 3A: Head-Up Display 4: Estimation device 13: Specific structures 13A: First-class specified structure (an example of a specified structure) 13B: Second-class designated structure (an example of a designated structure) 16: Arrow feather 31: Snow depth determination unit 32: Structure recognition section 33: Road surface acquisition part 34: Lane boundary estimation unit 35: Display Processing Unit 51: Vehicle control system 54: Vehicle control device M0: Position of the arrowhead М1, М2: End points L0: Virtual line L1, L2: Virtual lines P0: Intersection P1, P2: Intersection S: Road surface plane

Claims

1. A driver assistance system that estimates the lane boundary in which a vehicle is traveling and displays the boundary on a screen, A display device that performs screen display, An external environment acquisition device that acquires information for identifying the type and location of objects located around the vehicle, The system includes an estimation device that estimates the boundary based on the acquisition results of the external environment acquisition device and displays the estimated boundary on the display device, The estimation device acquires the position of a specific structure whose position relative to the boundary is determined based on the acquisition results of the external environment acquisition device, and road surface information indicating the shape of the road surface, estimates the boundary based on the position of the specific structure and the road surface information, and displays the estimated boundary on the display device.

2. The aforementioned specific structure includes arrowheads whose position is determined with respect to the boundary, The driving support system according to claim 1, wherein the estimation device identifies the specific structure as the arrow vane based on the shape of the specific structure.

3. The estimation device is, Based on the aforementioned road surface information, a road surface plane is obtained by approximating the road surface with a plane. Obtain a virtual line that passes through the position of the aforementioned arrowhead and extends downward, The driving assistance system according to claim 2, which estimates the boundary by obtaining the intersection point between the road surface plane and the virtual line.

4. The estimation device is, Identify the type of the aforementioned specific structure, Based on the aforementioned road surface information, a road surface plane is obtained by approximating the road surface with a plane. Obtain a virtual line that passes through the endpoint on the lane side of the specified structure and extends downward, The intersection point of the road surface plane and the virtual line is obtained, The driving assistance system according to claim 1, which estimates the boundary based on the acquired intersection and the type.

5. The display device includes a head-up display that displays on the windshield. The driving assistance system according to any one of claims 1 to 4, wherein the estimation device displays the estimated boundary as a virtual image on the head-up display.

6. An estimation device for estimating the lane boundaries in which a vehicle is traveling, A structure recognition unit that acquires the position of a specific structure whose position relative to the aforementioned boundary has been determined, A road surface acquisition unit that acquires road surface information indicating the shape of the road surface, An estimation device comprising a lane boundary estimation unit that estimates the boundary based on the location of the specified structure and the road surface information.

7. The structure recognition unit is connected to an external environment acquisition device that acquires information relating to objects located around the vehicle. The estimation device according to claim 6, wherein the structure recognition unit acquires the position of the specific structure and the road surface information based on the acquisition results of the external environment acquisition device.

8. The aforementioned specific structure includes arrowheads whose position is determined with respect to the boundary, The estimation device according to claim 6, wherein the structure recognition unit identifies the specific structure as the arrow feather based on the shape of the specific structure.

9. The lane boundary estimation unit is, Based on the aforementioned road surface information, a road surface plane is obtained by approximating the road surface with a plane. Obtain a virtual line that passes through the position of the aforementioned arrowhead and extends downward, The estimation device according to claim 8, which estimates the boundary by obtaining the intersection point of the road surface plane and the virtual line.

10. The structure recognition unit identifies the type of the specified structure, The lane boundary estimation unit is, Based on the aforementioned road surface information, a road surface plane is obtained by approximating the road surface with a plane. Obtain a virtual line that passes through the endpoint on the lane side of the specified structure and extends downward, The intersection point of the road surface plane and the virtual line is obtained, The estimation device according to claim 7, which estimates the boundary based on the acquired intersection and the type.

11. The estimation device according to any one of claims 6 to 10, A vehicle control system comprising: a vehicle control device that controls the movement of the vehicle based on the boundary estimated by the estimation device.

Images