Driving support device, driving support method, and storage medium

Abstract

This invention provides a driving support device, a driving support method, and a storage medium. The driving support device includes: a marker recognition unit that identifies road markers in the current lane and road markers in a first adjacent lane based on an image of the road surface captured by a vehicle, and determines the drivable direction of the current lane and the drivable direction of the first adjacent lane; a detection unit that detects lane changes by the vehicle; and an information management unit that stores first information indicating the drivable direction of the current lane and second information indicating the drivable direction of the first adjacent lane, determined by the marker recognition unit, in a storage unit, and updates the information stored in the storage unit when the detection unit detects a lane change to the first adjacent lane, such that the second information becomes the drivable direction of the current lane and the first information becomes the drivable direction of the second adjacent lane.

Description

Technical Field

[0001] This invention relates to a driving support device, a driving support method, and a storage medium. Background Technology

[0002] In vehicle control technologies for driving assistance or autonomous driving, accurate identification of road markings, road signs, and traffic lights, such as road markings or road dividing lines, becomes crucial. Previous technologies have focused on improving recognition accuracy when identifying road markings or signs based on images captured during driving, for example, by considering the driving environment, such as whether the vehicle is traveling on a highway or a regular road (see, for example, Japanese Patent Application Publication No. 2018-25898). Summary of the Invention

[0003] However, in the aforementioned prior art, for example, when a vehicle changes lanes by making a lane change, road markings need to be re-identified. Therefore, many vehicles may be traveling without recognizing road markings. Consequently, the aforementioned prior art may not be able to provide adequate driving assistance to occupants.

[0004] One of the objectives of this invention is to provide a driving support device, driving support method, and storage medium that can provide more appropriate driving support.

[0005] The driving support device of the first aspect of the present invention includes: a marking recognition unit that identifies road markings of the current lane and road markings of a first adjacent lane based on an image of the road surface captured by the vehicle, and determines the drivable direction of the current lane and the drivable direction of the first adjacent lane; a detection unit that detects lane changes of the vehicle; and an information management unit that stores first information indicating the drivable direction of the current lane and second information indicating the drivable direction of the first adjacent lane, determined by the marking recognition unit, in a storage unit, and updates the information stored in the storage unit when the detection unit detects a lane change to the first adjacent lane, such that the second information becomes the drivable direction of the current lane and the first information becomes the drivable direction of the second adjacent lane.

[0006] The second solution, based on the driving support device of the first solution, may also include the following: the marker recognition unit further identifies the travel direction of the second adjacent lane and determines the travel direction of the second adjacent lane; the information management unit further stores the third information indicating the travel direction of the second adjacent lane identified by the marker recognition unit in the storage unit; and invalidates the third information stored in the information management unit when the detection unit detects a lane change to the first adjacent lane.

[0007] The third option, based on the driving support device of the first or second option described above, may also be that, in the case where there is a lane where the road mark is not recognized by the mark recognition unit, the information management unit considers the drivable direction of the lane to be the same as the drivable direction of the lane based on the information stored in the storage unit immediately preceding it, and considers the road mark indicating the drivable direction of the lane to be recognized at the location where the road mark of the lane is not recognized.

[0008] The fourth scheme can also be based on the driving support device of any of the first to third schemes mentioned above, and further include: a determination unit that determines whether the condition that the vehicle has performed any of the following events is met, namely, making a left or right turn, passing through an intersection, or traveling a predetermined distance after the permissible direction of travel is identified by the mark recognition unit. If the determination unit determines that the condition is met, the information management unit invalidates the information indicating the permissible direction of travel stored in the storage unit.

[0009] The fifth option may also be based on the driving support device of any of the first to fourth options described above, and further include: a traffic light status recognition unit that recognizes the display status of traffic lights and determines the permissible direction of travel; and a reporting unit that reports to the occupants of the vehicle if the permissible direction of travel recognized by the traffic light status recognition unit is inconsistent with the permissible direction of travel of the lane based on the information stored in the storage unit.

[0010] The sixth embodiment may also be based on the driving support device of any of the first to fifth embodiments described above, and further include: a lane recognition unit that determines the current lane and the adjacent lane based on the road dividing lines recognized from the image; and an interpolation unit that, in the case where there are road dividing lines that have not been recognized, interpolates using road dividing lines estimated based on the vehicle's direction of travel.

[0011] The seventh option, based on the driving support device of the sixth option, may also involve the information management unit invalidating the information representing the drivable direction of the lane stored in the storage unit when the drivable direction of the lane determined based on the estimated road markings is consistent with the drivable direction of the adjacent lane based on the road markings identified from the new image.

[0012] In the driving support method of the eighth aspect of the present invention, the computer performs the following processing: identifying road markings of the current lane and road markings of a first adjacent lane based on an image of the road surface captured by the vehicle; determining the drivable direction of the current lane and the drivable direction of the first adjacent lane; detecting a lane change of the vehicle; and storing first information indicating the drivable direction of the current lane and second information indicating the drivable direction of the first adjacent lane in a storage unit; and updating the information stored in the storage unit when a lane change to the first adjacent lane is detected, such that the second information becomes the drivable direction of the current lane and the first information becomes the drivable direction of the second adjacent lane.

[0013] The ninth aspect of the present invention is a storage medium storing a program, wherein the program causes a computer to perform the following processing: identifying road markings of the current lane and road markings of a first adjacent lane based on an image of the road surface captured by a vehicle; determining the drivable direction of the current lane and the drivable direction of the first adjacent lane; detecting a lane change of the vehicle; and storing first information indicating the drivable direction of the current lane and second information indicating the drivable direction of the first adjacent lane in a storage unit; and, upon detecting a lane change to the first adjacent lane, updating the information stored in the storage unit such that the second information becomes the drivable direction of the current lane and the first information becomes the drivable direction of the second adjacent lane.

[0014] According to the first to ninth plans, more appropriate driving support can be provided. Attached Figure Description

[0015] Figure 1 This is a structural diagram of a vehicle system utilizing a driving support device according to an implementation method.

[0016] Figure 2 This is a functional structure diagram of the first control unit and the second control unit.

[0017] Figure 3 This diagram illustrates the driving support control performed by the identification unit.

[0018] Figure 4 This diagram illustrates the driving support control performed by the identification unit.

[0019] Figure 5 This diagram illustrates the processing of the identification unit in the first driving support mode.

[0020] Figure 6 This diagram illustrates the processing of the identification unit in the first driving support mode.

[0021] Figure 7This diagram illustrates the processing of the identification unit in the first driving support mode.

[0022] Figure 8 This diagram illustrates the processing of the identification unit in the first driving support mode.

[0023] Figure 9 This diagram illustrates the processing of the identification unit in the first driving support mode.

[0024] Figure 10 This is a flowchart illustrating an example of a processing flow performed by an autonomous driving control device.

[0025] Figure 11 This diagram illustrates the processing of the identification unit in the second driving support mode.

[0026] Figure 12 This diagram illustrates the processing of the identification unit in the second driving support mode.

[0027] Figure 13 This diagram illustrates the processing of the identification unit in the third driving support mode.

[0028] Figure 14 This diagram illustrates the processing of the identification unit in the fourth driving support mode.

[0029] Figure 15 This diagram illustrates the processing of the identification unit in the fourth driving support mode.

[0030] Figure 16 This diagram illustrates the processing of the identification unit in the fourth driving support mode.

[0031] Figure 17 This diagram illustrates the processing of the identification unit in the fourth driving support mode.

[0032] Figure 18 This is a diagram illustrating an example of the hardware structure of an automated driving control device according to an embodiment. Detailed Implementation

[0033] Hereinafter, embodiments of the driving support device, driving support method, and storage medium of the present invention will be described with reference to the accompanying drawings. The driving support device of the embodiments is mounted in an autonomous vehicle. The driving support device of the embodiments is a device that supports the driving control of the vehicle. Driving control, as referred to here, includes, for example, control that moves the vehicle to an appropriate lane according to a desired direction of travel, and control that appropriately starts and stops the vehicle based on the display status of the lane and traffic lights. The driving support device of the embodiments, for example, manages information related to lanes or road markings, and supports the driving control of the vehicle by providing information to other devices. For example, the driving support device of the embodiments appropriately provides information related to lanes or road markings to the MPU (Map Positioning Unit), which determines the recommended lane based on the driving path searched by the navigation system. Therefore, the driving support device of the embodiments enables reliable lane selection based on the MPU.

[0034] However, vehicles equipped with the driving support device according to embodiments of the present invention are not limited to autonomous vehicles, but can also be vehicles in which the driver manually operates the vehicle. It should be noted that in vehicles in which the driver manually operates the vehicle, the driving support device of the present invention appropriately provides information related to lanes or road markings to the HMI (Human Machine Interface) that provides information to the occupants. Thus, the driving support device according to embodiments of the present invention can, for example, cause the HMI to report to the occupants when the direction in which the vehicle can travel in the driving lane differs from the direction indicated by the turn signal.

[0035] [Overall Structure]

[0036] Figure 1 This is a structural diagram of a vehicle system 1 utilizing the driving support device of the first embodiment. The vehicle equipped with vehicle system 1 (hereinafter referred to as "this vehicle M") is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using electricity generated by a generator connected to the internal combustion engine or electricity discharged from a secondary battery or fuel cell.

[0037] Vehicle system 1 includes, for example, a camera 10, a radar device 12, a detector 14, an object recognition device 16, a communication device 20, an HMI 30, vehicle sensors 40, a navigation device 50, an MPU 60, driving controls 80, an automatic driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. It should be noted that the automatic driving control device 100 is an example of a "driving support device." These devices or equipment are interconnected via multiplexed communication lines such as CAN (Controller Area Network) communication lines, serial communication lines, or wireless communication networks. It should be noted that... Figure 1 The structure shown is just one example; a part of the structure can be omitted, and other structures can be added.

[0038] Camera 10 is, for example, a digital camera utilizing a solid-state imaging element such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). Camera 10 can be mounted at any location on the vehicle M. For example, when photographing the front of the vehicle M, camera 10 can be mounted on the upper part of the windshield or behind the rearview mirror inside the vehicle. Similarly, when photographing the rear of the vehicle M, camera 10 can be mounted on the upper part of the rearview window. Furthermore, when photographing the right or left side of the vehicle M, camera 10 can be mounted on the right or left side of the rearview mirror on the vehicle body or door. Camera 10 can periodically and repeatedly photograph the perimeter of the vehicle M. Camera 10 can also be a stereo camera.

[0039] Furthermore, camera 10 can photograph the road surface when the vehicle M is moving and when it is stationary. For example, camera 10 photographs the road surface in front of the vehicle M. Camera 10 photographs the road surface in a manner that includes, for example, the road markings of the lane in which the vehicle M is traveling (hereinafter referred to as "this lane") and other lanes existing parallel to this lane (hereinafter referred to as "other lanes"). The other lanes mentioned here include at least other lanes that are parallel to and adjacent to this lane (hereinafter referred to as "adjacent lanes").

[0040] Additionally, camera 10 can photograph traffic lights (vehicle traffic lights) when the vehicle M is moving and stationary. For example, camera 10 photographs the traffic lights from slightly above and in front of the vehicle M. Camera 10 photographs the traffic lights in such a way that the captured image includes the three lights and arrow lights of the traffic lights.

[0041] Radar device 12 radiates millimeter-wave and other radio waves to the periphery of the vehicle M, and detects the radio waves reflected by objects (reflected waves) to at least detect the position (distance and orientation) of the objects. Radar device 12 can be installed at any location on the vehicle M. Radar device 12 can also detect the position and speed of objects using FM-CW (Frequency Modulated Continuous Wave) method.

[0042] Detector 14 is a LIDAR (Light Detection and Ranging) system. Detector 14 illuminates light around the vehicle M and measures the scattered light. Based on the time from emitting light to receiving light, detector 14 detects the distance to the object. The illuminated light can be, for example, a pulsed laser. Detector 14 is installed at any location on the vehicle M.

[0043] The object recognition device 16 performs sensor fusion processing on some or all of the detection results from the camera 10, radar device 12, and detector 14 to identify the object's position, type, speed, etc. The object recognition device 16 outputs the recognition results to the autonomous driving control device 100. Alternatively, the object recognition device 16 can directly output the detection results from the camera 10, radar device 12, and detector 14 to the autonomous driving control device 100. The object recognition device 16 can also be omitted from the vehicle system 1.

[0044] Additionally, the object recognition device 16 identifies road markings and lane markings, for example, based on images captured by the camera 10. Furthermore, the object recognition device 16 identifies, for example, three-light fixtures and arrow lights, based on images captured by the camera 10.

[0045] The communication device 20 communicates with other vehicles in the vicinity of the vehicle M, for example, using cellular networks or Wi-Fi networks, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), etc., or communicates with various server devices via wireless base stations.

[0046] The HMI30 provides various information to the occupants (e.g., the driver) of the vehicle M and responds to input operations performed by the occupants. The HMI30 includes, for example, various display devices, speakers, buzzers, touch panels, switches, buttons, etc.

[0047] The vehicle sensors 40 include a speed sensor for detecting the speed of the vehicle M, an acceleration sensor for detecting acceleration, a yaw rate sensor for detecting angular velocity about a vertical axis, and an orientation sensor for detecting the direction or travel direction of the vehicle M. The information detected by the vehicle sensors 40 is output to, for example, the navigation device 50 or the automatic driving control device 100.

[0048] The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as an HDD (Hard Disk Drive) or flash memory. The GNSS receiver 51 determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M can also be determined or supplemented by INS (Inertial Navigation System) output from the vehicle sensors 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, buttons, etc. Part or all of the navigation HMI 52 can be integrated with the HMI 30 described above. The route determination unit 53, for example, refers to the first map information 54 to determine the route (hereinafter referred to as the map route) from the position of the vehicle M determined by the GNSS receiver 51 (or any input position) to the destination input by the occupant using the navigation HMI 52. The first map information 54 is, for example, information showing the shape of a road by representing road lines and nodes connected by those lines. The first map information 54 may also include road curvature, POI (Point of Interest) information, etc. The path on the map is output to the MPU 60. The navigation device 50 may also provide route guidance using the navigation HMI 52 based on the path on the map. The navigation device 50 may also be implemented through the functions of a terminal device such as a smartphone or tablet held by the occupant. The navigation device 50 may also send its current location and destination to the navigation server via the communication device 20, and obtain a path equivalent to the path on the map from the navigation server.

[0049] MPU 60 includes, for example, a lane recommendation unit 61 that stores second map information 62 in a storage device such as an HDD or flash memory. The lane recommendation unit 61 divides the path on the map provided by the navigation device 50 into multiple segments (e.g., every 100 [m] in the vehicle's direction of travel), refers to the second map information 62, and determines a recommended lane for each segment. The lane recommendation unit 61 makes a decision such as which lane to drive in from the left. If there are branching points in the path on the map, the lane recommendation unit 61 determines the recommended lane as one that allows the vehicle M to travel on a reasonable path to the branch destination.

[0050] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 may include, for example, information about the center of a lane or the boundaries of a lane. Furthermore, the second map information 62 may include road information, traffic restriction information, residential information (address / postal code), facility information, telephone number information, etc. The second map information 62 can be updated in real time by communicating with other devices via the communication device 20.

[0051] The driving control unit 80 includes, for example, an accelerator pedal, a brake pedal, a gear lever, a steering wheel, a custom steering wheel, a joystick, and other control components. Sensors are installed on the driving control unit 80 to detect the amount of operation or whether operation has occurred. The detection results are output to some or all of the following: the automatic driving control unit 100, the driving force output device 200, the braking device 210, and the steering device 220.

[0052] The automatic driving control device 100 (driving support device) includes, for example, a first control unit 120, a second control unit 160, a storage unit 170, and a notification control unit 180. The first control unit 120, the second control unit 160, and the notification control unit 180 are each implemented by executing programs (software) via hardware processors such as CPUs (Central Processing Units). Furthermore, some or all of these components can be implemented using hardware (including circuitry) such as LSIs (Large Scale Integration), ASICs (Application Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays), and GPUs (Graphics Processing Units), or through the coordinated use of software and hardware. The program can also be pre-stored in a storage device such as an HDD or flash memory (a storage device with a non-transitory storage medium) of the automatic driving control device 100, or stored in a removable storage medium such as a DVD or CD-ROM, and installed into the HDD or flash memory of the automatic driving control device 100 by assembling the storage medium (non-transitory storage medium) into a drive unit.

[0053] Figure 2This is a functional structure diagram of the first control unit 120 and the second control unit 160. The first control unit 120, for example, includes a recognition unit 130 and an action plan generation unit 140. The first control unit 120, for example, implements AI (Artificial Intelligence) based functions and functions based on pre-given models in parallel. For example, the function of "recognizing intersections" can be implemented by simultaneously performing intersection recognition based on deep learning and other methods, and recognition based on pre-given conditions (traffic lights with pattern matching, road markings, etc.), scoring both and comprehensively evaluating them. This ensures the reliability of autonomous driving.

[0054] The identification unit 130 identifies the environment surrounding the vehicle M. For example, based on information input from the camera 10, radar device 12, and detector 14 via the object identification device 16, the identification unit 130 identifies the position, speed, acceleration, and direction of travel of objects (e.g., surrounding vehicles or landmarks) located around the vehicle M. The position of an object is identified, for example, as its position on absolute coordinates with a representative point of the vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of an object can also be represented by the object's center of gravity or representative points such as the center or corners, or by the area it represents. In the case of a vehicle, the object's "state" can also include the object's acceleration, jerk, or "action state" (e.g., whether a lane change is in progress or whether a lane change is about to occur).

[0055] Additionally, the recognition unit 130 can identify, for example, its own lane and other lanes. For instance, the recognition unit 130 identifies the driving lane by comparing the pattern of road markings (e.g., the arrangement of solid and dashed lines) obtained from the second map information 62 with the pattern of road markings surrounding the vehicle M identified based on images captured by the camera 10. It should be noted that, not limited to road markings, the recognition unit 130 can also identify driving lanes by recognizing driving road boundaries (road boundaries), including road markings, shoulders, curbs, median strips, guardrails, etc. In this recognition, the position of the vehicle M obtained from the navigation device 50 or INS-based processing results can also be incorporated. Furthermore, the recognition unit 130 identifies temporary stop lines (hereinafter referred to as stop lines), traffic lights, obstacles, toll booths, and other road features.

[0056] When identifying a driving lane, the identification unit 130 identifies the position and attitude of the vehicle M relative to the driving lane. For example, the identification unit 130 may identify the deviation of the vehicle M's reference point from the center of the lane, and the angle formed by the vehicle M's direction of travel relative to the line connecting the center of the lane, as the relative position and attitude of the vehicle M relative to the driving lane. Alternatively, the identification unit 130 may also identify the position of the vehicle M's reference point relative to any side end (road dividing line or road boundary) of the driving lane, as the relative position of the vehicle M relative to the driving lane.

[0057] The identification unit 130 identifies information related to the location of surrounding vehicles based on images captured by the camera 10, images captured by the camera 10, congestion information around the vehicle M obtained by the navigation device 50, or location information obtained from the second map information 62.

[0058] It should be noted that the identification unit 130 can also obtain various information received from vehicles traveling around the vehicle M via vehicle-to-vehicle communication through the communication device 20, and identify the surroundings of the vehicle M based on this information. Furthermore, the identification unit 130 may include, for example, a lane identification unit 131, a marker identification unit 132, a marker information management unit 133 (an example of an "information management unit"), a lane change detection unit 134 (an example of a "detection unit"), a reset determination unit 135 (an example of a "determination unit"), a traffic light status identification unit 136, and a lane interpolation unit 137 (an example of an "interpolation unit"). The functions of the identification unit 130, including these components, will be described in detail below.

[0059] The action plan generation unit 140 generates a target trajectory for the future travel of vehicle M automatically (without driver intervention), ensuring that it travels within the recommended lane determined by the recommended lane determination unit 61 and is able to respond to the surrounding conditions of vehicle M. The target trajectory includes, for example, speed elements. For instance, the target trajectory is represented by a track consisting of locations (track points) that vehicle M should reach arranged side-by-side. Track points are locations that vehicle M should reach at predetermined travel distances (e.g., a few meters). In addition, target speed and target acceleration are generated as part of the target trajectory at predetermined sampling times (e.g., a few tenths of a second). Alternatively, track points can also be locations that vehicle M should reach at predetermined sampling times. In this case, the target speed and target acceleration information are represented by the intervals between track points.

[0060] When generating the target track, the action plan generation unit 140 can set events for automatic driving. These events include, for example, constant speed driving events, low-speed following events, lane change events, branching events, merging events, and takeover events. The action plan generation unit 140 generates the target track corresponding to the initiated events.

[0061] The second control unit 160 controls the driving force output device 200, the braking device 210 and the steering device 220, so that the vehicle M passes through the target track generated by the action plan generation unit 140 at a predetermined time.

[0062] The second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires information about the target track (track point) generated by the action plan generation unit 140 and stores it in a memory (not shown). The speed control unit 164 controls the driving force output device 200 or the braking device 210 based on the speed elements of the target track stored in the memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target track stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is achieved, for example, through a combination of feedforward control and feedback control. As an example, the steering control unit 166 combines feedforward control corresponding to the curvature of the road ahead of the vehicle M with feedback control based on deviation from the target track.

[0063] The notification control unit 180 notifies the occupants of prescribed information via the HMI 30. It should be noted that the notification control unit 180 and the HMI 30 are examples of a "reporting unit." Prescribed information may include, for example, information identified by the identification unit 130, the status of autonomous driving (driving support) performed by the autonomous driving control device 100, and other information related to vehicle control. Additionally, the prescribed information may also include information obtained by the navigation device 50, content stored on storage media such as television programs or DVDs (e.g., movies). Furthermore, the notification control unit 180 outputs information received by the HMI 30 to devices such as the communication device 20, the navigation device 50, and the first control unit 120.

[0064] The driving force output device 200 outputs driving force (torque) to the drive wheels for vehicle propulsion. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, as well as an ECU (Electronic Control Unit) that controls them. The ECU controls the above structure according to information input from the second control unit 160 or from the driving operation device 80.

[0065] The braking device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a braking ECU. The braking ECU controls the electric motor according to information input from the second control unit 160 or from the driving control unit 80, outputting braking torque corresponding to the braking operation to each wheel. The braking device 210 may include, as a backup, a mechanism for transmitting hydraulic pressure generated by the operation of the brake pedal included in the driving control unit 80 via a master hydraulic cylinder to the cylinder. It should be noted that the braking device 210 is not limited to the structure described above, and may also be an electronically controlled hydraulic braking device that controls the actuator according to information input from the second control unit 160 and transmits hydraulic pressure from the master hydraulic cylinder to the cylinder.

[0066] The steering system 220 may include a steering ECU and an electric motor.

[0067] An electric motor, for example, applies force to a rack and pinion mechanism to change the direction of the steering wheels. The steering ECU drives the electric motor to change the direction of the steering wheels according to information input from the second control unit 160 or from the driving control unit 80.

[0068] The functions of the identification unit 130 in the following embodiment and the content of the driving support control based on the identification results of the identification unit 130 will be described in detail below. The driving support control mode for each scenario will be described below.

[0069] <First Driver Support Control Mode>

[0070] Figure 3 and Figure 4 This diagram illustrates the driving control performed by the identification unit 130. Figure 3 The diagram shows road RD, lane center line CL, two lane boundary lines BL, lane outer line OL, road marking RMa at location A, and road marking RMc at location C. Additionally, in... Figure 3 The diagram shows vehicle M, which will pass through points A, B, and C. That is, Figure 3 This shows the situation where vehicle M travels along road RD while passing through points A, B, and C in that order.

[0071] The lane recognition unit 131 of the automatic driving control device 100 installed in the vehicle M identifies the current lane, the adjacent lane to the left of the current lane (hereinafter referred to as the "left lane"), and the adjacent lane to the right of the current lane (hereinafter referred to as the "right lane") based on information output from the object recognition device 16. The information output from the object recognition device 16 refers to the information representing the lane center line CL, lane boundary line BL, and lane outer line OL identified by the object recognition device 16 based on the image of the road surface captured by the camera 10 from the vehicle M.

[0072] The marking recognition unit 132 of the automatic driving control device 100 installed in the vehicle M identifies road markings displayed in the current lane and adjacent lanes based on information output from the object recognition device 16. Based on the identified road markings displayed in the current lane and adjacent lanes, the marking recognition unit 132 determines the permissible travel direction (hereinafter referred to as "travel direction") for both lanes. The information output from the object recognition device 16 refers to the information representing road markings RMa and RMc identified by the object recognition device 16 based on images of the road surface captured by the camera 10 from the vehicle M. For example... Figure 3 As shown, road markings RMa and RMc include the road markings for this lane and the road markings for adjacent lanes.

[0073] When vehicle M is about to pass point A, the marking recognition unit 132 obtains information representing road marking RMa and identifies the road markings of this lane and the adjacent lane (left lane).

[0074] Based on the recognition results, the marker recognition unit 132 determines that the permissible directions of travel in the current lane are straight ahead and right turn, and the permissible directions of travel in the left lane are straight ahead and left turn. The marker recognition unit 132 outputs information indicating the determination results to the marker information management unit 133. The marker information management unit 133 obtains the information output from the marker recognition unit 132. The marker information management unit 133 stores the obtained information as marker information 171 in the storage unit 170.

[0075] Figure 4 The marker information 171 stored in the storage unit 170 at the time points when the vehicle M passed various locations is shown in tabular form. For example... Figure 4 As shown in the marking information 171 at location A, the permitted directions of travel in the left lane are left turn and straight ahead, while the permitted directions of travel in this lane are straight ahead and right turn. It should be noted that, as... Figure 3 As shown, there is no right lane at location A; therefore, no right lane road markings were identified. Therefore, as... Figure 4As shown, in the marker information 171 at location A, the right lane's travel direction is marked as undetected. It should be noted that when a travel direction is marked as undetected, it is considered that travel is possible in all directions.

[0076] Location B is a point where a new right lane has been added to the right of this lane. When vehicle M is about to pass through location B, lane recognition unit 131 identifies the right lane. For example... Figure 3 As shown, the right lane road markings are still not displayed at location B, therefore, as Figure 4 As shown, the marker information management unit 133 does not update the marker information 171, and the right lane remains in a state where it can travel in all directions.

[0077] Next, as vehicle M is about to pass point C, the sign recognition unit 132 acquires information representing road sign RMc and identifies the road signs for its own lane and the adjacent lanes (left and right lanes). Based on the recognition results, the sign recognition unit 132 determines that the only permissible direction of travel for its own lane is straight ahead, the permissible directions of travel for the left lane are straight ahead and left turn, and the permissible directions of travel for the right lane are straight ahead and right turn. The sign recognition unit 132 outputs information indicating the determination results to the sign information management unit 133. The sign information management unit 133 acquires the information output from the sign recognition unit 132. The sign information management unit 133 stores the acquired information as sign information 171 in the storage unit 170.

[0078] like Figure 4 As shown, the marking information 171 at location C is updated based on the marking information 171 at location B, making the permissible directions of travel for the left lane both left turn and straight ahead, the permissible direction of travel for this lane only straight ahead, and the permissible direction of travel for the right lane only right turn. Thus, the marking information management unit 133 updates the marking information 171 stored in the storage unit 170 each time a new road marking is identified as the vehicle M travels.

[0079] In the first driving support mode, the automatic driving control device 100 provides driving support corresponding to the lane change when the vehicle M changes lanes. Figure 5 This diagram illustrates the processing of the identification unit 130 in the first driving support mode. (See diagram for example.) Figure 5 As shown, after passing point D marked with road sign RMd, vehicle M changes lanes to the left lane at point E.

[0080] When vehicle M is about to pass point D, the sign recognition unit 132 obtains information representing road sign RMd and identifies the road sign of this lane and the road sign of the adjacent lane (left lane).

[0081] It should be noted that there is no right lane at location D; therefore, no right lane road markings were identified. Based on the recognition result, the marking recognition unit 132 determines that the permissible directions of travel for this lane are straight ahead and right turn, and the permissible directions of travel for the left lane are straight ahead and left turn. The marking recognition unit 132 outputs information indicating the determination result to the marking information management unit 133. The marking information management unit 133 obtains the information output from the marking recognition unit 132. The marking information management unit 133 stores the obtained information as marking information 171 in the storage unit 170.

[0082] Figure 6 The marker information 171 stored in the storage unit 170 at the time points when the vehicle M passed various locations is shown in tabular form. For example... Figure 6 As shown in the marking information 171 at location D, the permitted directions of travel in the left lane are left turn and straight ahead, while the permitted directions of travel in this lane are straight ahead and right turn. It should be noted that, as... Figure 5 As shown, there is no right lane at location D; therefore, no road markings were identified. Therefore, as... Figure 6 As shown, in the marker information 171 at location D, the travel direction of the right lane is not detected. It should be noted that, as mentioned above, when the travel direction is not detected, it is considered that travel is possible in all directions.

[0083] Location E is the point where vehicle M changed lanes to the left lane. Lane change detection unit 134 detects the lane change of vehicle M. When lane change detection unit 134 detects a lane change, marker information management unit 133 replaces the marker information 171 stored in storage unit 170 by sliding it along each lane. Specifically, as... Figure 6 As shown, the tag information management unit 133 updates the tag information 171 stored in the storage unit 170, so that the lane of the vehicle M changing its destination (in Figure 5 The permissible direction of travel for vehicle M is the left lane (in the middle). Furthermore, the marker information management unit 133 updates the marker information 171 stored in the storage unit 170, causing the permissible direction of travel for vehicle M from the lane change source to the lane opposite to the direction in which the lane change occurred (in the middle lane). Figure 5 The permitted direction of travel is the right lane (middle lane). It should be noted that, as... Figure 5 As shown, there is no left lane in this lane after the lane change, therefore, as Figure 6 As shown, in the marking information 171 at location E, the drivable direction of the left lane is not detected. It should be noted that, as mentioned above, when the drivable direction is not detected, it is considered that travel is possible in all directions.

[0084] It should be noted that the marking information management unit 133 can also invalidate (or delete) the marking information 171 indicating the travel direction of lanes that are no longer adjacent due to lane changes. For example, it can be configured such that, in a three-lane road, if the vehicle M changes lanes from the central lane to one of the adjacent lanes (e.g., the left lane, the first adjacent lane), the marking information 171 indicating the travel direction of the other adjacent lane (e.g., the right lane, the second adjacent lane of the lane before the lane change) is invalidated (or deleted). This reduces the data storage capacity required by the storage unit 170 and reduces device costs, etc.

[0085] If the predetermined conditions are met, the reset determination unit 135 invalidates (or deletes) all the marker information 171 stored in the storage unit 170. The predetermined conditions referred to here are, for example, conditions that satisfy any one of the following events: the vehicle M makes a left or right turn, the vehicle M passes through an intersection, and the vehicle M travels a predetermined distance (e.g., a long distance such as 800 m) after recognizing the road markings. This is because satisfying the above conditions usually results in a situation where the currently identified traversable direction becomes meaningless.

[0086] When vehicle M makes a left or right turn, the road on which vehicle M is traveling changes, therefore, the currently stored marking information 171 is invalidated. Additionally, road markings that typically indicate the direction of travel indicate the direction of travel for each lane up to the next intersection; therefore, when vehicle M passes through an intersection, the currently stored marking information 171 is invalidated.

[0087] Furthermore, if the vehicle M travels a certain distance after recognizing a road mark without detecting any other road marks, there is a high probability that it will miss road marks displayed along the way or miss passing through intersections. Therefore, if the vehicle M travels a predetermined distance after recognizing a road mark without recognizing the next road mark, the currently stored mark information 171 will be invalidated.

[0088] Figure 7 This diagram illustrates the situation where vehicle M passes through an intersection. The road sign recognition unit 132 recognizes a road sign at location F and stores the sign information 171 in the storage unit 170. Afterwards, vehicle M passes through the intersection IS located at location G. Furthermore, Figure 8This diagram illustrates the scenario where the vehicle M travels a predetermined distance after recognizing a road mark. The mark recognition unit 132 recognizes the road mark at location H and stores the mark information 171 in the storage unit 170. Afterwards, the vehicle M arrives at location I, which is a predetermined distance away from location H. It should be noted that the reset determination unit 135 measures the distance traveled by the vehicle M after recognizing the road mark.

[0089] Figure 9 The marker information 171 stored in the storage unit 170 at the time points when the vehicle M passed various locations is illustrated in tabular form. Thus, after passing location G or location I, the reset determination unit 135 invalidates (or deletes) all the marker information 171 stored in the storage unit 170.

[0090] (Action of the automatic driving control device)

[0091] The following describes an example of the operation of the automatic driving control device 100. Figure 10 This is a flowchart illustrating the operation of an automatic driving control device 100 according to an embodiment of the present invention.

[0092] The lane recognition unit 131 identifies the current lane and adjacent lanes based on information output from the object recognition device 16 (step S010). Next, after the marker recognition unit 132 determines the permissible directions of travel for the current lane and adjacent lanes based on the road markers displayed in the identified lanes (step S020), the marker information management unit 133 stores the information indicating the determined permissible directions as marker information 171 in the storage unit 170. Furthermore, the reset determination unit 135 begins measuring the distance traveled by the vehicle M after the road markers were identified (step S030).

[0093] Next, if the lane change detection unit 134 detects a lane change of the vehicle M (step S040), the tag information management unit 133 will then... Figure 6 As shown, the marker information 171 stored in the storage unit 170 is replaced by sliding it along each lane (step S050). Next, when the marker recognition unit 132 determines the travelable direction of the current lane and the adjacent lane based on the road markers displayed in the recognized current lane and the adjacent lane respectively (step S060), the marker information management unit 133 updates the marker information 171 stored in the storage unit 170 according to the information indicating the determined travelable direction. In addition, the reset determination unit 135 resets the measurement of the distance traveled by the vehicle M after the road marker was recognized (step S070).

[0094] If the specified conditions are met (step S080), the reset determination unit 135 invalidates (or deletes) all the marker information 171 stored in the storage unit 170 (step S090). As described above, the specified conditions here refer to, for example, the conditions that satisfy any one of the following events: the vehicle M makes a left or right turn, the vehicle M passes through an intersection, and the vehicle M travels a specified distance after recognizing the road markings.

[0095] The automatic driving control device 100 repeats the actions following step S040 until the driving of the vehicle M ends (step S100). That concludes... Figure 10 The operation of the automatic driving control device 100 shown in the flowchart has ended.

[0096] As explained above, the automated driving control device 100 of this embodiment identifies road markings for its own lane and those of one adjacent lane based on images of the road surface captured by the vehicle M. The automated driving control device 100 determines the permissible direction of travel for its own lane and the permissible direction of travel for one adjacent lane. The automated driving control device 100 stores first information indicating the permissible direction of travel for its own lane and second information indicating the permissible direction of travel for one adjacent lane. When detecting a lane change to one adjacent lane, the automated driving control device 100 updates the information, making the second information the permissible direction of travel for its own lane and the first information the permissible direction of travel for the other adjacent lane. Furthermore, as explained above, the automated driving control device 100 can further identify and determine the permissible direction of travel for the other adjacent lane. In this case, the automated driving control device 100 also stores third information indicating the permissible direction of travel for the other adjacent lane in its storage unit. When detecting a lane change to one adjacent lane, the automated driving control device 100 invalidates the third information. Additionally, as explained above, the automated driving control device 100 may invalidate the stored information indicating the permissible direction of travel if it determines that the vehicle M has performed any of the following events: making a left or right turn, passing through an intersection, or traveling a predetermined distance after identifying the permissible direction of travel.

[0097] By having such a structure, the autonomous driving control unit 100 can use the second information indicating the drivable direction of the adjacent lane representing the destination of the lane change as information indicating the drivable direction of the current lane immediately after a lane change, thus enabling more appropriate driving support. For example, the autonomous driving control unit 100 can provide information indicating the drivable direction of each lane after the lane change to the MPU 60 at the time of the lane change.

[0098] <Second Driver Support Control Mode>

[0099] In the second driving support mode, if there is a situation where the lane marking recognition unit 132 cannot recognize the lane marking, such as when other vehicles are parked on the road marking (hereinafter referred to as "other vehicles"), the lane marking information management unit 133 will not update the lane marking information 171.

[0100] Figure 11 This diagram illustrates the processing of the identification unit 130 in the second driving support mode. Figure 11 The diagram shows road RD, lane center line CL, two lane boundary lines BL, lane outer lines OL, road marking RMj shown at location J, and road marking RMk shown at location K. Additionally, in Figure 11 The diagram shows the vehicle M, which will pass through points J and K, and other vehicles m1 to m3. That is, Figure 11 This illustrates the situation where vehicle M travels along road RD while passing through points J and K in that order.

[0101] like Figure 11 As shown, the following situation occurs: When vehicle M is about to pass point J, it identifies road sign RMj. Then, when it is about to pass point K, another vehicle m3 traveling in the left lane stops on road sign RMk displayed at point K. Therefore, the sign recognition unit 132 of vehicle M cannot recognize the road sign RMk for the left lane and cannot determine the permissible direction of travel for the left lane. In this case, for the left lane, the sign recognition unit 132 does not update the sign information 171 indicating the permissible direction of travel for the left lane based on the road sign RMj displayed at the immediately preceding (previous) point J, but keeps the sign information 171 unchanged. That is, for the left lane, the sign recognition unit 132 assumes that at point K, it also displays road signs indicating the permissible direction of travel for the left lane, i.e., the straight-ahead direction and the left-turn direction, based on the road sign RMj displayed at the previous point J. Therefore, the marking information 171 indicating the travelable direction of the left lane based on the road marking RMj is not invalidated when the vehicle M travels a predetermined distance from location J, but is maintained until the vehicle M travels a predetermined distance from location K.

[0102] It should be noted that when the sign recognition unit 132 recognizes a road sign for at least one lane, it assumes that road signs are also displayed in other lanes at the same location. This is because road signs for each lane are usually displayed in a concentrated location. Figure 12 The marker information 171 stored in the storage unit 170 at the time points when the vehicle M passed various locations is shown in tabular form. For example... Figure 12 As shown, the information indicating the drivable direction of the left lane determined at point J continues to be used as the drivable direction of the left lane at the next point K where no left lane has been identified by the road markings.

[0103] As explained above, the automated driving control device 100 of this embodiment does not update the information indicating the drivable direction of lanes where road markings have not been identified. By having this structure, the automated driving control device 100 can use information indicating the drivable direction based on previously identified road markings for lanes where road markings have not been identified, thus enabling more appropriate driving support. For example, when there are lanes where road markings have not been identified, the automated driving control device 100 can provide information indicating the previously identified drivable direction of that lane to the MPU 60.

[0104] <Third Driver Support Control Mode>

[0105] In the third driving support mode, if the drivable direction obtained based on the marker information 171 stored in the storage unit 170 is inconsistent with the drivable direction shown by the traffic light, the control unit 180 is notified to issue a warning report to the occupants of the vehicle M.

[0106] Figure 13 This diagram illustrates the processing of the identification unit 130 in the third driving support mode. Figure 13 The diagram shows the road RD, lane center line CL, lane boundary line BL, lane outer line OL, road marking RM1 at location L, traffic light S, and the vehicle M. Figure 13 This shows the situation where vehicle M is traveling towards an intersection with traffic lights S after passing point L.

[0107] like Figure 13 As shown, when vehicle M passes point L, the permissible directions of travel in this lane become both straight and left-turn directions in the marker information 171 stored in storage unit 170. However, as Figure 13 As shown, the traffic light S indicates a right turn. Therefore, if the direction of travel obtained from the marker information 171 stored in the storage unit 170 is inconsistent with the direction of travel indicated by the traffic light S, the control unit 180 is notified to control the HMI 30 to report to the occupants of the vehicle M. It should be noted that any reporting method can be used, such as sound-based reporting, image-based reporting, warning light-based reporting, or vibration-based reporting.

[0108] It should be noted that the notification control unit 180 may also have the following structure: if the permissible direction of travel obtained based on the marker information 171 stored in the storage unit 170 is partially consistent with the permissible direction of travel shown by the traffic indicator S, no report is made. For example, if the permissible direction of travel obtained based on the marker information 171 stored in the storage unit 170 is a straight-ahead direction and a left-turn direction, and the permissible direction of travel shown by the traffic indicator S is a straight-ahead direction and a right-turn direction, both are consistent in that the straight-ahead direction is the permissible direction of travel. In this case, the notification control unit 180 may also not make a report. This is because there is a possibility that the vehicle M is intended to travel in the straight-ahead direction and the vehicle M is traveling in the correct lane.

[0109] It should be noted that if the traffic indicator S does not have arrow lights or if the arrow lights are not illuminated, the control unit 180 can determine whether to report based on the display status of the three lights. For example, in this case, the control unit 180 can be notified that if the green or yellow lights of the traffic indicator S are illuminated, all directions are considered safe to travel, and if the red light is illuminated, all directions are considered not safe to travel.

[0110] As explained above, the automated driving control device 100 in this embodiment identifies the display status of the traffic light S and determines the permissible direction of travel. If the permissible direction of travel obtained based on the display status of the traffic light S is inconsistent with the permissible direction of travel for the current lane stored based on road markings, the automated driving control device 100 reports this to the occupants of the vehicle M. By having such a structure, for example, when the vehicle M is traveling in a lane inconsistent with the intended direction of travel, the automated driving control device 100 can report to the occupants using the HMI 30, thus enabling the occupants to identify surrounding risks in advance and provide more appropriate driving assistance.

[0111] <Fourth Driver Support Control Mode>

[0112] In the fourth driving support mode, when at least one road marking is not detected, the lane interpolation unit 137 interpolates the undetected road marking using an estimated (hypothetical) road marking. This allows lane recognition by the lane recognition unit 131. Furthermore, if the undetected road marking continues, the lane interpolation unit 137 appropriately corrects the estimated road marking based on newly identified road signs.

[0113] Figures 14-16 This diagram illustrates the processing of the identification unit 130 in the fourth driving support mode. Figure 14This illustrates a scenario where the left lane dividing line BL of the lane in which vehicle M is traveling is identified, but the right lane dividing line is not detected. In this case, the lane interpolation unit 137 determines the position of the right lane dividing line VLr, which is considered a hypothetical lane dividing line. The lane interpolation unit 137 determines the position of the right lane dividing line VLr, for example, based on the distance from the position of vehicle M to the position of the left lane dividing line BL. For example, the lane interpolation unit 137 defines the position of the right lane dividing line VLr as a distance that is separated to the right of vehicle M in the direction of travel by the same distance as the distance from the travel line of vehicle M to the position of the left lane dividing line BL. That is, the lane interpolation unit 137 considers, for example, that vehicle M is traveling in the center of the lane.

[0114] in addition, Figure 15 This illustrates a scenario where neither the left nor right lane markings of the lane in which vehicle M is traveling are detected. In this case, the lane interpolation unit 137 determines the positions of the left lane marking VL1 and the right lane marking VLr, which are assumed to be lane markings. For example, the lane interpolation unit 137 defines the positions of the left lane marking VL1 and the right lane marking VLr as being separated from the vehicle M's travel line by a predetermined distance (e.g., 2 [m]) on either side of the vehicle M's travel direction. That is, the lane interpolation unit 137 assumes, for example, that the width of the lane is 4 [m] (=2 [m]×2), and that vehicle M is traveling in the center of the lane.

[0115] Figure 16 The correction process for the estimated road markings is shown. Figure 16 The diagram shows road RD, lane center line CL, two lane boundary lines BL, lane outer line OL, road marking RMm at location M, and road marking RMn at location N. Here, the lane center line CL, the two lane boundary lines BL, and the lane outer line OL may be difficult to identify, for example, due to paint deterioration or snow accumulation.

[0116] In addition, Figure 16 The diagram shows the vehicle M that will pass through points M and N respectively. That is, Figure 16 This illustrates the scenario where vehicle M travels along road RD, passing points M and N in that order. Additionally, in... Figure 16 The diagram shows the left road dividing line VLl-1 and the right road dividing line VLr-1 estimated by the lane interpolation unit 137 when passing point M, and the left road dividing line VLl-2 and the right road dividing line VLr-2 estimated by the lane interpolation unit 137 when passing point N.

[0117] like Figure 16As shown, the lane recognition unit 131, based on the left road dividing line VL1-1 and the right road dividing line VLr-1 estimated by the lane interpolation unit 137 when the vehicle M is about to pass point M, identifies a lane that is tilted to the lower right of the map, which is different from the actual lane, as the current lane. Therefore, the marker recognition unit 132 mistakenly identifies road marker RMm as the road marker for this lane. However, next, the lane recognition unit 131, based on the left road dividing line VL1-2 and the right road dividing line VLr-2 estimated by the lane interpolation unit 137 when the vehicle M is about to pass point N, identifies a lane that is similar to the actual lane as the current lane. Therefore, the marker recognition unit 132 identifies road marker RMn as the road marker for the adjacent lane on the right.

[0118] At this time, if the travel direction indicated by the road marker RMm identified at location M is the same as the travel direction indicated by the road marker RMn identified at location N (that is, if the road marker RMm and the road marker RMn are the same road marker), the marker information management unit 133 considers the assumption of the left road dividing line VLl-1 and the right road dividing line VLr-1 at location M to be an incorrect assumption, updates the marker information 171 stored in the storage unit 170, and invalidates (removes) the information indicating the travel direction of this lane.

[0119] Figure 17 The marker information 171 stored in the storage unit 170 at the time points when the vehicle M passed various locations is shown in tabular form. For example... Figure 17 As shown, the marker information management unit 133 invalidates (or deletes) the information indicating the travel direction of this lane stored in the storage unit 170 at location M at location N, and updates the marker information 171 to make it undetectable. It should be noted that, as described above, when the travel direction is undetectable, it is considered that travel is possible in all directions. Furthermore, the marker information management unit 133 updates the marker information 171 based on the road marker RMn identified at location N, thereby updating the information indicating the travel direction of the right lane.

[0120] As explained above, the automated driving control device 100 in this embodiment determines the current lane and adjacent lanes based on road markings identified from an image. In the case of unidentified road markings, it interpolates the undetected road markings using hypothetical road markings inferred from the vehicle M's direction of travel. With this structure, the automated driving control device 100 can infer the current lane and adjacent lanes even when road markings cannot be correctly identified, and by using information indicating the possible direction of travel, it can provide more appropriate driving support. For example, even when road markings are not identified, the automated driving control device 100 can infer lanes based on hypothetical road markings and provide information indicating the possible direction of travel of the inferred lanes to the MPU 60.

[0121] Furthermore, when controlling the vehicle M based on hypothetical road markings, the automatic driving control device 100, if the permissible direction of travel for the current lane, determined based on the road markings estimated according to the vehicle M's direction of travel, coincides with the permissible direction of travel for adjacent lanes obtained based on road markings identified from a new image, invalidates the information indicating the permissible direction of travel for the current lane and updates the information indicating the permissible direction of travel for adjacent lanes. With this structure, the automatic driving control device 100 can further improve the lane estimation accuracy in accordance with the road markings identified as the vehicle M travels, and can correct the information indicating the permissible direction of travel for the current lane, thus enabling more appropriate driving support.

[0122] [Hardware Structure]

[0123] Figure 18This diagram illustrates an example of the hardware structure of the automated driving control device 100 according to an embodiment. As shown, the automated driving control device 100 is structured to interconnect a communication controller 100-1, a CPU 100-2, a RAM (Random Access Memory) 100-3 used as working memory, a ROM (Read Only Memory) 100-4 storing boot programs, a storage device 100-5 such as a flash memory or HDD (Hard Disk Drive), and a drive device 100-6 via an internal bus or dedicated communication line. The communication controller 100-1 communicates with components other than the automated driving control device 100. A program 100-5a for execution by the CPU 100-2 is stored in the storage device 100-5. This program is expanded in the RAM 100-3 via a DMA (Direct Memory Access) controller (not shown) and executed by the CPU 100-2. Thus, some or all of the first control unit 120, the second control unit 160, and the notification control unit 180 are realized.

[0124] The implementation methods described above can be performed as follows.

[0125] A driving support device includes a storage device storing programs and a hardware processor.

[0126] The driving support device is configured as follows:

[0127] Based on images of the road surface captured by the vehicle, the road markings of the current lane and the road markings of the adjacent lane on one side are identified, and the permissible direction of travel of the current lane and the adjacent lane on one side are determined.

[0128] Detect the vehicle's lane change.

[0129] The first information indicating the travel direction of the current lane and the second information indicating the travel direction of the adjacent lane of the same lane are stored in a storage unit. When a lane change to the adjacent lane of the same lane is detected, the information stored in the storage unit is updated so that the second information becomes the travel direction of the current lane and the first information becomes the travel direction of the adjacent lane of the other lane.

[0130] The above describes specific embodiments of the present invention, but the present invention is not limited to such embodiments in any way, and various modifications and substitutions can be made without departing from the spirit of the present invention.

Claims

1. A driving support device, wherein, The driving support device includes: The marking recognition unit identifies the road markings of the current lane and the road markings of the first adjacent lane based on images of the road surface captured by the vehicle, and determines the travel direction of the current lane and the travel direction of the first adjacent lane. The inspection department detects lane changes of the vehicle. as well as The information management unit stores first information indicating the travel direction of the current lane and second information indicating the travel direction of the first adjacent lane, determined by the mark recognition unit, in a storage unit. When the detection unit detects a lane change to the first adjacent lane, the information stored in the storage unit is updated so that the second information becomes the travel direction of the current lane and the first information becomes the travel direction of the second adjacent lane.

2. The driving support device according to claim 1, wherein, The marker recognition unit also identifies the travel direction of the second adjacent lane, and further determines the travel direction of the second adjacent lane. The information management unit also stores third information, which indicates the travel direction of the second adjacent lane as identified by the mark recognition unit, in the storage unit. If the detection unit detects a lane change to the first adjacent lane, the third information stored in the information management unit is invalidated.

3. The driving support device according to claim 1 or 2, wherein, If there is a lane where the road mark is not recognized by the mark recognition unit, the information management unit considers the traversable direction of the lane to be the same as the traversable direction of the lane based on the information stored in the storage unit immediately preceding it, and considers the road mark indicating the traversable direction of the lane to be recognized at the location where the road mark of the lane is not recognized.

4. The driving support device according to claim 1 or 2, wherein, The driving support device also includes: The determination unit determines whether the condition is met that the vehicle has performed any of the following events: making a left or right turn, passing through an intersection, or traveling a predetermined distance after the sign recognition unit has identified a permissible direction of travel. When the determination unit determines that the condition has been met, the information management unit invalidates the information representing the feasible direction of travel stored in the storage unit.

5. The driving support device according to claim 1 or 2, wherein, The driving support device also includes: A traffic light status recognition unit identifies the display status of traffic lights to determine the permissible direction of travel; and The reporting unit reports to the occupants of the vehicle if the permissible direction of travel identified by the traffic light status recognition unit is inconsistent with the permissible direction of travel for the lane based on the information stored in the storage unit.

6. The driving support device according to claim 1 or 2, wherein, The driving support device also includes: A lane recognition unit determines the current lane and the adjacent lane based on road markings identified from the image; and The interpolation unit interpolates the road markings that are not identified when they exist, using road markings that are estimated based on the direction of travel of the vehicle.

7. The driving support device according to claim 6, wherein, If the travel direction of the lane determined based on the estimated road markings is consistent with the travel direction of the adjacent lane based on the road markings identified from the new image, the information management unit invalidates the information representing the travel direction of the lane stored in the storage unit.

8. A driving support method, wherein, The computer performs the following processing: Based on images of the road surface captured by the vehicle, identify the road markings of the current lane and the road markings of the first adjacent lane, and determine the travel direction of the current lane and the travel direction of the first adjacent lane; Detect lane changes of the vehicle; as well as The first information indicating the travel direction of the current lane and the second information indicating the travel direction of the first adjacent lane are stored in a storage unit. When a lane change to the first adjacent lane is detected, the information stored in the storage unit is updated so that the second information becomes the travel direction of the current lane and the first information becomes the travel direction of the second adjacent lane.

9. A storage medium having a stored program, wherein, The program causes the computer to perform the following processes: Based on images of the road surface captured by the vehicle, identify the road markings of the current lane and the road markings of the first adjacent lane, and determine the travel direction of the current lane and the travel direction of the first adjacent lane; Detect lane changes of the vehicle; as well as The first information indicating the travel direction of the current lane and the second information indicating the travel direction of the first adjacent lane are stored in a storage unit. When a lane change to the first adjacent lane is detected, the information stored in the storage unit is updated so that the second information becomes the travel direction of the current lane and the first information becomes the travel direction of the second adjacent lane.

Images