Vehicle control device, vehicle control method, and storage medium

Abstract

A vehicle control device, a vehicle control method, and a storage medium that more accurately determine a travel lane of a vehicle even when a road on which the vehicle travels is a bidirectional travel section. The vehicle control device includes a driving control section that controls one or both of a steering and a speed of the vehicle based on a surrounding situation recognized by a recognition section; an acquisition section that acquires map information including lane information of a surrounding of the vehicle and reference information for determining a position of the vehicle; and a determination section that determines the travel lane of the vehicle in accordance with information of a road on which the vehicle travels, which is acquired from the map information based on the reference information, and a class of a road division line that divides one or more lanes of the surrounding of the vehicle, which is recognized by the recognition section. The determination section determines the travel lane without using information of a line type of at least a bidirectional travel division line of the recognition result of the surrounding of the vehicle, which is recognized by the recognition section, when it is determined based on at least one of the information of the road and the class of the road division line that the road is the bidirectional travel section.

Description

Technical Field

[0001] This invention relates to vehicle control devices, vehicle control methods, and storage media. Background Technology

[0002] In recent years, research on autonomous driving, which involves automatically controlling the vehicle's movement by recognizing its surroundings, has been progressing. Related technologies include: detecting information related to the driving environment by recognizing the state of road markings; determining control actions by inferring lane markings based on pre-stored lane types; and determining the vehicle's position using camera recognition results and map information (see, for example, Patent Documents 1-4).

[0003] Prior art literature

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2000-105898

[0006] Patent Document 2: Japanese Patent Application Publication No. 2010-221859

[0007] Patent Document 3: Japanese Patent Application Publication No. 10-300494

[0008] Patent Document 4: Japanese Patent Application Publication No. 2013-032953 Summary of the Invention

[0009] The problem that the invention aims to solve

[0010] However, in previous technologies, it was sometimes impossible to identify whether a road was two-way based on images captured by a camera. Therefore, even when road markings were successfully identified from the image, the vehicle's lane could not always be determined, or the lane might be incorrectly determined.

[0011] The present invention was made with this in mind, and one of its objectives is to provide a vehicle control device, vehicle control method, and storage medium that can more accurately determine the vehicle's lane even when the road in which the vehicle is traveling is a two-way traffic section.

[0012] Solution for solving the problem

[0013] The vehicle control device, vehicle control method, and storage medium of the present invention adopt the following structure.

[0014] (1): One aspect of the present invention relates to a vehicle control device, wherein the vehicle control device comprises: an identification unit that identifies the surrounding conditions of a vehicle; a driving control unit that controls one or both of the vehicle's steering and speed based on the surrounding conditions identified by the identification unit; an acquisition unit that acquires map information including lane information of the vehicle's surroundings and reference information for determining the vehicle's position; and a determination unit that determines the vehicle's driving lane based on information about the road the vehicle travels on, obtained from the map information based on the reference information, and the type of road dividing lines that divide one or more lanes around the vehicle, as identified by the identification unit, wherein the determination unit determines the driving lane without using information about the type of the dividing lines for two-way traffic from the identification results of the vehicle's surroundings identified by the identification unit, if it determines that the road is a two-way traffic section based on at least one of the road information and the type of the road dividing lines.

[0015] (2): Based on the above (1) scheme, when the determining unit obtains that the road is a two-way traffic section and has multiple parallel driving lanes based on the map information, it determines the driving lane without using the information on the type of the dividing line for two-way traffic in the identification results of the vehicle's surroundings identified by the identification unit.

[0016] (3): Based on the above (1) scheme, when the road is a two-way traffic section with one lane on one side, the determining unit determines that the vehicle's driving lane is a one-way lane when the category of the road dividing line on the left and right sides of the vehicle is solid line, dashed line, special line or shoulder.

[0017] (4): Based on the above (1) scheme, the determining unit determines that the vehicle is traveling in the right lane of the two lanes on one side when the road is a two-way traffic section with two lanes on one side, the road dividing line on the left side of the vehicle is a dashed line or a special line, and the dividing line that is located further away from the left side of the road dividing line when viewed from the vehicle is a solid line.

[0018] (5): Based on the above (1) scheme, when the determining unit determines that the vehicle's driving lane is the right lane of the three-lane or more lanes on one side of the road, and the road dividing line on the right side of the vehicle is a yellow lane line and the road dividing line on the left side of the vehicle is a dashed line or a special line, the determining unit determines that the vehicle's driving lane is the right lane of the three-lane or more lanes on one side.

[0019] (6): Based on the above (1) scheme, the driving control unit executes at least one of a plurality of driving modes including a first driving mode and a second driving mode that has a heavier task for the occupants of the vehicle compared with the first driving mode to drive the vehicle. The driving control unit executes the first driving mode when the determining unit determines the driving lane of the vehicle, and executes the second driving mode when the determining unit does not determine the driving lane of the vehicle.

[0020] (7): Based on the above (1) scheme, the driving control unit executes any one of multiple driving modes with different tasks for the occupants of the vehicle to drive the vehicle. If the determination unit fails to determine the driving lane of the vehicle, the driving mode being executed continues to drive the vehicle.

[0021] (8): One aspect of the present invention relates to a vehicle control method, wherein the vehicle control method causes a computer to perform the following processing: identifying the surrounding conditions of a vehicle; performing driving control based on the identified surrounding conditions to control one or both of the vehicle's steering and speed; obtaining map information containing lane information of the vehicle's surroundings and reference information for determining the vehicle's position; determining the vehicle's driving lane based on information about the road the vehicle is traveling on, obtained from the map information based on the reference information, and the identified categories of road dividing lines that divide one or more lanes around the vehicle; and determining the driving lane without using information about the type of the dividing lines for two-way traffic in the identification results of the vehicle's surroundings when the road is determined to be a two-way traffic section based on at least one of the road information and the category of the road dividing lines.

[0022] (9): One aspect of the present invention relates to a storage medium storing a program, wherein the program causes a computer to perform the following processing: identifying the surrounding conditions of a vehicle; performing driving control based on the identified surrounding conditions to control one or both of the vehicle's steering and speed; acquiring map information containing lane information of the vehicle's surroundings and reference information for determining the vehicle's position; determining the vehicle's driving lane based on information about the road the vehicle is traveling on, obtained from the map information based on the reference information, and the identified categories of road dividing lines that divide one or more lanes around the vehicle; and determining the driving lane without using information about the type of the dividing lines for two-way traffic in the identified results of the identification of the vehicle's surroundings, if the road is determined to be a two-way traffic section based on at least one of the road information and the category of the road dividing lines.

[0023] Invention Effects

[0024] According to the above schemes (1) to (9), even if the road where the vehicle travels is a two-way traffic section, the vehicle's driving lane can be determined more accurately. Attached Figure Description

[0025] Figure 1 This is a structural diagram of a vehicle system utilizing a vehicle control device according to an implementation method.

[0026] Figure 2 This is a functional structure diagram of the first control unit and the second control unit in the implementation method.

[0027] Figure 3 This is a diagram illustrating the relationship between driving modes, vehicle control states, and tasks.

[0028] Figure 4 This is a diagram representing an example of the contents of a decision table.

[0029] Figure 5 This is a diagram representing an example of the first decision pattern.

[0030] Figure 6 This is a diagram representing an example of the second decision pattern.

[0031] Figure 7 This is a diagram representing an example of the third decision pattern.

[0032] Figure 8 This is a diagram representing an example of the fourth decision pattern.

[0033] Figure 9 This is a diagram representing an example of the fifth decision pattern.

[0034] Figure 10This is a diagram representing an example of the sixth decision pattern.

[0035] Figure 11 This is a diagram representing an example of the seventh decision pattern.

[0036] Figure 12 This is a flowchart illustrating an example of the driving control process performed by an automatic driving control device. Detailed Implementation

[0037] Hereinafter, embodiments of the vehicle control device, vehicle control method, and storage medium of the present invention will be described with reference to the accompanying drawings. The following description applies to regulations governing left-hand traffic; however, in cases where right-hand traffic regulations apply, the left and right sides should be reversed.

[0038] [Overall Structure]

[0039] Figure 1 This is a structural diagram of vehicle system 1 utilizing the vehicle control device of the embodiment. The vehicle equipped with vehicle system 1 (hereinafter referred to as vehicle M) is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using electricity generated by a generator connected to the internal combustion engine, or electricity discharged from a secondary battery or fuel cell. Hereinafter, as an example, an embodiment of the vehicle control device applicable to an autonomous vehicle will be described. Autonomous driving, for example, refers to automatically controlling one or both of the steering and speed of vehicle M to perform driving control. Driving control of vehicle M may include various driving assistance systems such as ACC (Adaptive Cruise Control), ALC (Auto Lane Changing), and LKAS (Lane Keeping Assistance System). Autonomous vehicles can also be partially or fully controlled by manual driving by the occupant (driver).

[0040] Vehicle system 1 includes, for example, a camera (an example of an imaging unit) 10, a radar device 12, a LIDAR (Light Detection and Ranging) device 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, vehicle sensors 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driver monitoring camera 70, driving controls 80, an automatic driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are interconnected via CAN (Controller Area Network) communication lines, serial communication lines, wireless communication networks, etc. Figure 1 The structure shown is only one example; some parts of the structure may be omitted, and other structures may be added. Combining camera 10, radar device 12, and LIDAR 14 is an example of an "external sensor ES". The external sensor ES may also include other detection units (e.g., sonar) that identify the vehicle's surroundings, and may also include an object recognition device 16. HMI 30 is an example of an "output device". Autonomous driving control device 100 is an example of a "vehicle control device".

[0041] Camera 10 is, for example, a digital camera utilizing a solid-state imaging element such as CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor). Camera 10 can be mounted anywhere on the vehicle M. For example, when photographing the front of the vehicle M, camera 10 can be mounted on the upper part of the windshield, behind the rearview mirror inside the vehicle, etc. When photographing the rear of the vehicle M, camera 10 can be mounted on the upper part of the rear windshield, the tailgate, etc. When photographing the sides and rear sides of the vehicle M, camera 10 can be mounted on the rearview mirror on the door, etc. Camera 10 can periodically and repeatedly photograph the perimeter of the vehicle M. Camera 10 can also be a stereo camera. Regarding camera 10, multiple cameras (e.g., a first camera, a second camera) can be provided, multiple cameras can photograph in the same direction, and the first camera can normally take pictures, while the second camera, or both the first and second cameras, take pictures under specified conditions. The specified conditions include, for example, identifying road dividing lines (hereinafter referred to as dividing lines) that define the lanes (driving lanes) along the road on which vehicle M travels, based on images captured by a camera (hereinafter referred to as camera images). The dividing lines may include, for example, line segment information other than the dividing lines that define the lanes.

[0042] Radar device 12 radiates millimeter-wave or other radio waves around the vehicle M and detects the radio waves (reflected waves) reflected by objects to detect at least the position (distance and orientation) of the objects. Radar device 12 can be installed at any part of the vehicle M. Radar device 12 can also detect the position and speed of objects using FM-CW (Frequency Modulated Continuous Wave) method.

[0043] The LIDAR14 illuminates the periphery of vehicle M with light (or electromagnetic waves with wavelengths close to light) and measures the scattered light. The LIDAR14 determines the distance to the object based on the time from the emission of light to the reception of light. The illuminating light can be, for example, a pulsed laser. The LIDAR14 can be mounted at any location on vehicle M.

[0044] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the detections from the external sensors ES, including the camera 10, radar device 12, and LIDAR 14, to identify the position, type, speed, etc. of objects. The object recognition device 16 outputs the recognition results to the autonomous driving control device 100. Alternatively, the object recognition device 16 can directly output the detection results from the camera 10, radar device 12, and LIDAR 14 to the autonomous driving control device 100. The object recognition device 16 can also be omitted from the vehicle system 1.

[0045] The communication device 20 uses, for example, cellular networks, Wi-Fi networks, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), etc., to communicate with other vehicles in the vicinity of vehicle M, or communicates with various server devices via wireless base stations.

[0046] The HMI 30, controlled by the HMI control unit 170, provides various information to the occupants of the vehicle M and accepts input operations from the occupants. The HMI 30 includes, for example, various display devices, speakers, switches, microphones, buzzers, touch panels, buttons, etc. Various display devices include, for example, LCD (Liquid Crystal Display) and OLED (ElectroLuminescence) display devices. The display device is, for example, located near the front of the driver's seat (the seat closest to the steering wheel) in the dashboard, and positioned so that the occupant can visually recognize it from the gap in the steering wheel or over the steering wheel. The display device may also be located in the center of the dashboard. The display device may also be a HUD (Head-Up Display). A HUD projects an image onto a portion of the windshield in front of the driver's seat, allowing the occupant in the driver's seat to visually perceive the virtual image. The display device displays the image generated by the HMI control unit 170, described later. The HMI 30 may also include a driving switch for switching between automatic driving and occupant-based manual driving. Switches include, for example, turn indicator switches (direction indicators) 32. The turn signal switch 32 is, for example, located on the steering column or steering wheel. The turn signal switch 32 is, for example, an operating unit that receives instructions from an occupant regarding lane changes of the vehicle M.

[0047] Vehicle sensor 40 includes a vehicle speed sensor for detecting the speed of vehicle M, an acceleration sensor for detecting acceleration, a yaw rate sensor for detecting angular velocity about a vertical axis, and an orientation sensor for detecting the orientation of vehicle M. Vehicle sensor 40 may include a steering angle sensor for detecting the steering angle of vehicle M (which could be the angle of the steering wheels or the operating angle of the steering wheel). Vehicle sensor 40 may include a position sensor for obtaining the position of vehicle M. The position sensor may be, for example, a sensor that obtains position information (longitude and latitude information) from a GPS (Global Positioning System) device. Alternatively, the position sensor may be a sensor that obtains position information using a GNSS (Global Navigation Satellite System) receiver 51 of navigation device 50.

[0048] The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores the first map information 54 in a storage device such as an HDD (Hard Disk Drive) or flash memory. The GNSS receiver 51 determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M can also be determined or supplemented by an INS (Inertial Navigation System) utilizing the output of the vehicle sensors 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, buttons, etc. The navigation HMI 52 can also be partially or entirely shared with the aforementioned HMI 30. The route determination unit 53, for example, refers to the first map information 54 to determine the route (hereinafter referred to as the map path) from the position of the vehicle M determined by the GNSS receiver 51 (or any input position) to the destination input by the occupant using the navigation HMI 52.

[0049] The first map information 54 includes, for example, information related to lanes attached to a unit road (hereinafter referred to as lane information). Lane information includes, for example, nodes indicating the start and end of a road section, and road segments showing the road shape between nodes. Lane information may include the number of lanes (number of side-by-side lanes) in a defined section such as a unit road, the number of lanes added or removed, and the direction of lane addition or removal (indicating which side of the road is being added or removed relative to the direction of travel). Lane information may include, for example, information related to the type of dividing line (e.g., solid line, dashed line, shoulder dividing line) and color. The first map information 54 may also include information indicating whether a road is a two-way traffic section. A two-way traffic section refers, for example, a road section in which lanes for traffic in both directions are not divided by a central median, but by dividing lines, lines similar to dividing lines, or objects (e.g., poles, road buttons), and the road structure does not clearly separate lanes for different directions of traffic. The first map information 54 can also include distances, curvatures, road types (e.g., highways, general roads), and POI (Point of Interest) information for road sections. Paths on the map are output to MPU60.

[0050] The navigation device 50 can also provide route guidance using the navigation HMI 52 based on the path on the map. The navigation device 50 can be implemented, for example, through the functions of a terminal device such as a smartphone or tablet held by the occupant. The navigation device 50 can also send its current location and destination to the navigation server via the communication device 20, and obtain the path equivalent to the path on the map from the navigation server. The first map information 54 can also be stored in the storage unit 180 instead of being stored in the navigation device 50.

[0051] MPU60 includes, for example, a lane recommendation determination unit 61. The lane recommendation determination unit 61 divides the path on the map provided by the navigation device 50 into multiple blocks (e.g., in 100m increments along the vehicle's direction of travel), and determines a recommended lane for each block based on lane information from the first map information 54. The lane recommendation determination unit 61 can also determine recommended lanes for individual roads stored in the first map information 54. For example, the lane recommendation determination unit 61 makes a decision such as which lane to drive in from the left (or right). When the path on the map has branching points, the lane recommendation determination unit 61 determines recommended lanes so that the vehicle M can travel on a reasonable path to the branch destination.

[0052] The driver monitoring camera 70 is, for example, a digital camera utilizing a solid-state imaging element such as a CCD or CMOS. The driver monitoring camera 70 is installed anywhere in the vehicle M, for example, at a position and orientation that allows it to capture the head of the occupant (hereinafter referred to as the driver) seated in the driver's seat of the vehicle M from the front (facing the face). For example, the driver monitoring camera 70 is installed above a display device located in the center of the dashboard of the vehicle M.

[0053] The driving control unit 80, in addition to the steering wheel 82, includes, for example, an accelerator pedal, a brake pedal, a gear shift lever, and other control elements. Sensors are installed in the driving control unit 80 to detect the amount or presence of operation, and the detection results are output to some or all of the following: the automatic driving control unit 100, the driving force output device 200, the braking device 210, and the steering device 220. The steering wheel 82 is an example of a "control element that receives steering operations performed by the driver." The control element does not necessarily have to be ring-shaped; it can also be an irregularly shaped steering wheel, a lever, a button, etc. A steering wheel grip sensor 84 is installed in the steering wheel 82. The steering wheel grip sensor 84, implemented by a capacitance sensor or the like, outputs a signal that detects whether the driver is gripping (meaning in contact with the steering wheel 82 with applied force) to the automatic driving control unit 100.

[0054] The autonomous driving control device 100 includes, for example, a first control unit 120, a second control unit 160, an HMI control unit 170, and a storage unit 180. The first control unit 120, the second control unit 160, and the HMI control unit 170 are each implemented by executing programs (software) via hardware processors such as CPUs (Central Processing Units). Some or all of these components can also be implemented by hardware (including circuitry) such as LSIs (Large Scale Integration), ASICs (Application Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays), and GPUs (Graphics Processing Units), or through the coordinated use of software and hardware. The program can be pre-stored in a storage device such as an HDD or flash memory (a storage device with a non-transitory storage medium) of the autonomous driving control device 100, or stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the autonomous driving control device 100 by mounting the storage medium (a non-transitory storage medium) onto the drive unit. The action plan generation unit 140 and the second control unit 160 together constitute an example of a "driving control unit". The HMI control unit 170 is an example of an "output control unit".

[0055] The storage unit 180 may also be implemented using various storage devices described above, or SSD (Solid State Drive), EEPROM (Electrically Erasable Programmable Read Only Memory), ROM (Read Only Memory), or RAM (Random Access Memory). The storage unit 180 may store, for example, a decision table 182, a program, and various other information. The decision table 182 is, for example, a table referenced for determining the driving lane of vehicle M. Details regarding the decision table 182 will be described later. The first map information 54 may also be stored in the storage unit 180.

[0056] Figure 2 This is a functional structure diagram of the first control unit 120 and the second control unit 160 in the implementation embodiment. The first control unit 120 includes, for example, a recognition unit 130, an action plan generation unit 140, and a pattern determination unit 150. The first control unit 120 can, for example, implement AI (Artificial Intelligence) based functions and functions based on pre-given models in parallel. For example, the function of "recognizing intersections" can be achieved by "parallel execution of intersection recognition based on deep learning, etc., and recognition based on pre-given conditions (the existence of signals capable of pattern matching, road signs, etc.), and comprehensively evaluating both sides by scoring them." This ensures the reliability of autonomous driving.

[0057] The recognition unit 130 identifies the position, velocity, acceleration, and other states of objects surrounding the vehicle M based on information input from external sensors ES. The object's position is identified, for example, as its position on absolute coordinates with a representative point of the vehicle M (center of gravity, drive shaft center, etc.) as the origin, and is used for control. The object's position can also be represented by representative points such as its center of gravity or corners, or by a region. The object's "state" can also include its acceleration, jerk, or "action state" (e.g., whether a lane change is in progress or is about to occur).

[0058] The recognition unit 130 identifies, for example, the lane in which the vehicle M is traveling (driving lane). For instance, the recognition unit 130 identifies the left and right dividing lines of the vehicle M based on camera images captured by the camera 10, and identifies the driving lane based on the position of the identified dividing lines. The recognition unit 130 is not limited to identifying dividing lines; it can also identify objects (driving lane boundaries, road boundaries) that can determine the lane's position, including shoulders, curbs, median strips, guardrails, fences, walls, etc., thereby identifying the driving lane. In this recognition, the position of the vehicle M obtained from the navigation device 50 and the processing results from the INS may also be taken into consideration. The recognition unit 130 can also identify temporary stop lines, obstacles, red lights, toll booths, and other road phenomena.

[0059] When identifying a driving lane, the identification unit 130 identifies the position and posture of the vehicle M relative to the driving lane. For example, the identification unit 130 may identify the deviation of the reference point of the vehicle M from the center of the lane, and the angle formed by the direction of travel of the vehicle M relative to the line connecting the centers of the lanes, as the relative position and posture of the vehicle M relative to the driving lane. Alternatively, the identification unit 130 may identify the position of the reference point of the vehicle M relative to any side end (dividing line or road boundary) of the driving lane as the relative position of the vehicle M relative to the driving lane. The identification of the driving lane and the identification of the position and posture of the vehicle M relative to the driving lane performed by the identification unit 130 may be performed by the determination unit 153, which will be described later.

[0060] The action plan generation unit 140 generates a target trajectory for the vehicle M to travel automatically (independent of driver operation) in a manner that allows it to travel in the recommended lane determined by the recommended lane determination unit 61 and to be able to respond to the surrounding conditions of the vehicle M. The target trajectory may include speed elements. For example, the target trajectory may be represented as a track that sequentially arranges the locations (track points) that the vehicle M should reach. Track points are locations that the vehicle M should reach at predetermined travel distances (e.g., a few meters), but target speeds and target accelerations are generated as part of the target trajectory at predetermined sampling times (e.g., a few tenths of a second). Track points may also be locations that the vehicle M should reach at the specified sampling time. In this case, the target speed and target acceleration information are represented by the intervals between track points.

[0061] When generating a target track, the action plan generation unit 140 can set events (functions) for automatic driving. These events include constant speed driving events, low-speed following events, lane change events, branching events, merging events, and takeover events. The action plan generation unit 140 generates the target track corresponding to each activated event.

[0062] The mode determination unit 150 determines the driving mode of the vehicle M as any one of several driving modes (in other words, several modes with different degrees of automation) that correspond to different tasks assigned to the driver. The mode determination unit 150 includes, for example, a driver state determination unit 151, a first acquisition unit 152, a determination unit 153, and a mode change processing unit 154. Their individual functions will be described later. The first acquisition unit 152 is an example of an "acquisition unit".

[0063] Figure 3 This is a diagram illustrating an example of the relationship between driving mode, vehicle M's control state, and tasks. In Figure 3 In the example, the driving modes of vehicle M include, for instance, five modes: Mode A through Mode E. Figure 3 In this diagram, modes A and B are examples of "first driving modes," and modes C, D, and E are examples of "second driving modes." Driving modes can include modes other than A through E, as well as modes other than the first and second driving modes. Among modes A through E, regarding the degree of automation (control level) of the driving control of vehicle M, mode A has the highest level, decreasing in the order of mode B, mode C, and mode D, with mode E having the lowest. Conversely, regarding the tasks assigned to the driver (occupant), mode A is the lightest, decreasing in the order of mode B, mode C, and mode D, with mode E, which involves manual driving, being the most demanding. If modes B through E are present, it becomes a non-automatic driving control state, and therefore the automatic driving control device 100 performs its functions until the control involved in automatic driving ends and the system switches to driving assistance or manual driving. The following provides examples of the content of each mode.

[0064] In Mode A, the vehicle enters an autonomous driving state, and the driver is not responsible for monitoring the surroundings of the vehicle M or for holding the steering wheel 82 (hereinafter referred to as "steering wheel holding"). Surroundings monitoring includes at least monitoring the direction of travel of the vehicle M (e.g., ahead). "Ahead" refers to the space visible through the windshield as the direction of travel of the vehicle M. However, even in Mode A, the driver is required to have a body posture that allows for a rapid transition to manual driving based on the requirements of the system centered on the autonomous driving control unit 100. Autonomous driving, as referred to here, means that the steering and speed of the vehicle M are controlled independently of the driver's operation. Mode A is, for example, a driving mode that can be executed when conditions are met, such as the vehicle M traveling at a prescribed speed (e.g., around 50 km / h) or less on a dedicated motor vehicle road like a highway, and there is a vehicle following ahead; it is sometimes called TJP (Traffic Jam Pilot) mode. If these conditions are no longer met, the mode determination unit 150 changes the driving mode of the vehicle M to Mode B.

[0065] In Mode A, the occupant is able to perform secondary tasks. Secondary tasks are, for example, actions other than driving permitted by the occupant in the autonomous driving mode of vehicle M. Examples of secondary tasks include watching television, using the occupant's terminal device (e.g., smartphone, tablet) (e.g., making or sending / receiving calls or emails, using SNS (Social Networking Service), browsing the web, etc.), and eating.

[0066] In Mode B, the system enters a driver support state, assigning the driver the task of monitoring the surroundings of vehicle M (hereinafter referred to as perimeter monitoring), but not the task of controlling the steering wheel 82. For example, in Mode B, lane change instructions from the occupants are not accepted; instead, vehicle system 1 determines lane changes based on the route settings to the destination made by navigation device 50. Lane change refers to moving vehicle M from its current lane to an adjacent lane, and may also include lane changes based on branching or merging. In Modes A and B, the driving entity is vehicle system 1.

[0067] In Mode C, the system enters a driver support state, assigning tasks such as monitoring the surroundings and controlling the steering wheel 82 to the driver. For example, in Mode C, if the vehicle system 1 determines that a lane change for vehicle M is necessary, it inquires with the occupant via HMI 30. Upon receiving the occupant's consent to the lane change from HMI 30, the system executes the lane change as driver support. In Modes B and C, lane change control is performed by the main system.

[0068] Mode D is a driving mode in which at least one of the vehicle M's steering or acceleration / deceleration requires some degree of driver input. For example, in Mode D, driver assistance systems such as ACC (Adaptive Cruise Control) and LKAS (LaneKeeping Assist System) are implemented. In Mode D, when the driver receives an instruction to change lanes in the indicated direction via operation of the turn signal switch 32, lane changes are performed in the indicated direction. Lane changes in Mode D are based on the driver's intention. The driver's operation of the turn signal switch 32 is an example of driving input. Driving input in Mode D can include operations for controlling steering or acceleration / deceleration.

[0069] In Mode E, the vehicle M is in a manual driving state where steering, acceleration, and deceleration all require manual operation by the driver. In Modes D and E, the driver is responsible for monitoring the surroundings of the vehicle M. In Modes C through E, the driver is the primary driving agent.

[0070] If the task involved in the determined driving mode is not performed by the driver, the mode determination unit 150 changes the driving mode of the vehicle M to a driving mode with a heavier task.

[0071] For example, during the execution of Mode A, if the driver is in a body posture that prevents them from transitioning to manual driving as requested by the system (e.g., continuing to look out of the permitted area, or detecting signs of driving difficulty), the Mode Determination Unit 150, through the HMI Control Unit 170, executes control to urge the driver to transition to manual driving in Mode E using HMI 30. If, after a predetermined time has elapsed since the HMI Control Unit 170 executed the control urging the driver to transition to manual driving, and the Mode Determination Unit 150 presumes that the driver is not in a state of manual driving, it performs control such as gradually decelerating the vehicle M while moving it toward the target position (e.g., the curb) and stopping the automatic driving. After stopping the automatic driving, the vehicle M enters a state of Mode D or E, and can be started by the driver's manual operation. The same applies to "stopping the automatic driving" below.

[0072] In Mode B, when the driver is not monitoring the road ahead, the mode determination unit 150 uses the HMI 30 to urge the driver to monitor the road ahead. If the driver does not respond, the unit performs control such as gradually bringing the vehicle M to a stop towards the target position and stopping the automatic driving function. In Mode C, when the driver is not monitoring the road ahead or is not holding the steering wheel 82, the mode determination unit 150 uses the HMI 30 to urge the driver to monitor the road ahead and / or hold the steering wheel 82. If the driver does not respond, the unit performs control such as gradually bringing the vehicle M to a stop towards the target position and stopping the automatic driving function.

[0073] The driver status determination unit 151 determines whether the occupant (driver) is in a suitable driving state. For example, the driver status determination unit 151 monitors the driver's state for the aforementioned mode change and determines whether the driver's state is appropriate for the task. For example, the driver status determination unit 151 analyzes the images captured by the driver monitoring camera 70 and performs posture estimation processing to determine whether the driver's body posture cannot be shifted to manual driving according to the system's requirements. The driver status determination unit 151 analyzes the images captured by the driver monitoring camera 70 and performs gaze estimation processing to determine whether the driver is monitoring the surroundings (more specifically, the front) of the vehicle M. If it is determined that the state is not appropriate for the task for a specified period of time or more, the driver status determination unit 151 determines that the driver is not suitable for driving the task. If it is determined that the state is appropriate for the task, the driver status determination unit 151 determines that the driver is suitable for driving the task. The driver status determination unit 151 can also determine whether the occupant is capable of driving replacement.

[0074] The first acquisition unit 152 acquires first map information 54. The first acquisition unit 152 acquires reference information for determining the position of vehicle M. The reference information may be, for example, the position information of vehicle M detected by vehicle sensor 40, or a camera image captured by camera 10. The reference information may also include part or all of the recognition result recognized by recognition unit 130.

[0075] The determining unit 153, based on the position information of vehicle M included in the reference information and referring to the first map information 54, determines the driving lane of vehicle M in one or more lanes of the road in which vehicle M travels. The determining unit 153 determines the driving lane, for example, based on road information obtained from the first map information 54 and the type of dividing lines identified by the identification unit 130. If the determining unit 153 determines that the road is for two-way traffic based on at least one of the road information and the type of dividing lines, it uses special rules (prescribed mitigation conditions) to determine the driving lane. For example, the determining unit 153 determines the driving lane without using information on the type of dividing lines for two-way traffic from the identification results of the area surrounding vehicle M identified by the identification unit 130. Dividing lines for two-way traffic are, for example, dividing lines that divide lanes in the road that allow vehicle M to travel facing the ground, and are other than dividing lines that divide lanes that allow travel in the same direction as the direction of travel of vehicle M. Dividing lines for two-way traffic do not include dividing lines that simultaneously divide lanes for the direction of travel of vehicle M and lanes for two-way traffic. More specifically, the determining unit 153 uses, for example, the identified road markings excluding those for two-way traffic to determine the driving lane. The two-way traffic markings can also be used as markings containing certain line segments to determine the position of the vehicle M relative to the driving lane, etc. Hereinafter, the case of using markings excluding those for two-way traffic to determine the driving lane will be mainly explained.

[0076] The determination unit 153 determines the driving lane of vehicle M at a predetermined time. This predetermined time may be, for example, a predetermined period, the start of autonomous driving, the start of driving on a specific road such as a highway, or a time when road sections are switched in the first map information 54. The predetermined time may also be, for example, when the number of lanes in the map information does not match the number of lanes obtained from the camera image, and the current driving lane is reset and needs to be determined again; when the driving lane of vehicle M cannot be determined and there is no road change within a predetermined distance; or when the occupant initiates the autonomous driving operation. The predetermined time may also be, for example, when the number of lanes on the road traveled by vehicle M has increased or decreased, or will increase or decrease in the near future. Details regarding the function of the determination unit 153 will be described later.

[0077] The mode change processing unit 154 determines the driving mode of vehicle M based on the determination results of the driver status determination unit 151 and the determination unit 153. The mode change processing unit 154 may also decide to continue the current driving mode or switch to another mode. The mode change processing unit 154 performs various processes for changing the driving mode determined by the mode determination unit 150. For example, the mode change processing unit 154 may issue operating instructions to a driving support device (not shown), output information from the HMI control unit 170 to the HMI 30 to urge the driver to act, or instruct the generation of a target track corresponding to the driving mode based on the action plan generation unit 140.

[0078] The second control unit 160 controls the driving force output device 200, the braking device 210 and the steering device 220 so that the vehicle M passes through the target track generated by the action plan generation unit 140 at a predetermined time.

[0079] The second control unit 160 includes, for example, a second acquisition unit 162, a speed control unit 164, and a steering control unit 166. The second acquisition unit 162 acquires information about the target track (track point) generated by the action plan generation unit 140 and stores this information in a memory (not shown). The speed control unit 164 controls the driving force output device 200 or the braking device 210 based on the speed elements associated with the target track stored in the memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target track stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is achieved, for example, through a combination of feedforward control and feedback control. As an example, the steering control unit 166 combines feedforward control corresponding to the curvature of the road ahead of the vehicle M with feedback control based on deviation from the target track.

[0080] The HMI control unit 170 notifies the occupant of prescribed information via the HMI 30. This prescribed information includes, for example, information related to the state of vehicle M and information related to driving control, which is relevant to the movement of vehicle M. Information related to the state of vehicle M includes, for example, vehicle M's speed, engine speed, and gear position. Information related to driving control includes, for example, inquiries about lane change requests, whether a driving mode is being executed, information related to driving mode changes, information on occupant instructions required to switch driving modes (task requests made to the occupant), and information related to the status of driving control (e.g., the content of ongoing events). Prescribed information may also include information not related to the driving control of vehicle M, such as television programs or items stored on storage media like DVDs (e.g., movies). Prescribed information may include, for example, information related to vehicle M's current location, destination, and remaining fuel, information indicating whether the vehicle M's driving lane can be determined, remaining distance until driving mode switching, lane entry / exit direction, number of lanes added / removed, and number of lanes parallel to the driving lane (parallel driving lanes).

[0081] For example, the HMI control unit 170 can generate an image containing the aforementioned specified information and display the generated image on the display device of the HMI 30. It can also generate sound representing the specified information and output the generated sound from the speaker of the HMI 30. The HMI control unit 170 can also output the information received by the HMI 30 to the communication device 20, the navigation device 50, the first control unit 120, etc.

[0082] The driving force output device 200 outputs driving force (torque) for vehicle movement to the drive wheels. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, as well as an ECU (Electronic Control Unit) that controls them. The ECU controls the above structure according to information input from the second control unit 160 or from the driving operation unit 80.

[0083] The braking device 210 includes, for example, a brake caliper, a hydraulic cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the hydraulic cylinder, and a braking ECU. The braking ECU controls the electric motor according to information input from the second control unit 160 or from the driving operation unit 80, so that braking torque corresponding to the braking operation is output to each wheel. The braking device 210 may have a mechanism as a backup for transmitting the hydraulic pressure generated by the operation of the brake pedal included in the driving operation unit 80 via the master hydraulic cylinder to the hydraulic cylinder. The braking device 210 is not limited to the structure described above, and may also be an electronically controlled hydraulic braking device that controls the actuator according to information input from the second control unit 160, thereby transmitting the hydraulic pressure from the master hydraulic cylinder to the hydraulic cylinder.

[0084] The steering system 220 includes, for example, a steering ECU and an electric motor. The electric motor applies force to a rack and pinion mechanism to change the direction of the steering wheels. The steering ECU drives the electric motor to change the direction of the steering wheels according to information input from the second control unit 160 or from the driving operation unit 80.

[0085] [The functions of the determination unit and the content of driving control]

[0086] The following describes the detailed functions of the determination unit 153 and the driving control performed based on the processing content of the determination unit 153. An example is described below where the driver state determination unit 151 determines that the driver is appropriately performing a task assigned according to the driving mode, and the driving mode is determined based on the processing content processed by the determination unit 153. If the driver state determination unit 151 determines that the driver is not performing a task assigned according to the mode, the mode determination unit 150 decides to change to a mode corresponding to the task being performed by the driver, or decides to perform control to stop automatic driving.

[0087] First, the determination unit 153, for example, obtains information related to the dividing lines around the vehicle M identified by the recognition unit 130. Specifically, the recognition unit 130, for example, analyzes an image (camera image) containing the road on which the vehicle M travels, captured by the camera 10 (either one or both cameras if a first camera or a second camera is present). It extracts edge points with large brightness differences from adjacent pixels in the image and connects these edge points to identify dividing lines in the image plane and objects whose lane positions can be determined. The recognition unit 130 may also perform feature extraction, image information extraction based on image enhancement processing, etc., and identify dividing lines and the aforementioned objects by matching the extracted image information with a predefined pattern matching model. Based on the image analysis results, the recognition unit 130 can identify category information such as category (solid line, dashed line) and color for each dividing line, and can also identify the type of object. The recognition unit 130 can also identify the positional relationships between the identified dividing lines and objects, and the positional relationships (relative positions) between the dividing lines, objects, and the vehicle M. The recognition unit 130 can also recognize text displayed on road signs and text drawn on roads.

[0088] The dividing lines around the vehicle M identified by the identification unit 130 are, for example, the left and right dividing lines closest to the vehicle M, and the left and right dividing lines that are next closest to the vehicle M compared to the aforementioned left and right dividing lines. Hereinafter, the left dividing line closest to the vehicle M is referred to as the "first left dividing line", and the next closest dividing line (which is located further away from the first left dividing line when viewed from the vehicle M) is referred to as the "second left dividing line". The right dividing line closest to the vehicle M is referred to as the "first right dividing line", and the next closest dividing line (which is located further away from the first right dividing line when viewed from the vehicle M) is referred to as the "second right dividing line".

[0089] Based on the location information included in the reference information obtained by the first acquisition unit 152, the determination unit 153 obtains information about the road on which the vehicle M travels, referring to the first map information 54. The road information may be, for example, information related to the number of lanes. The road information may include information related to the number of lanes on one side, information indicating whether two-way traffic is possible, and information indicating whether the road on which the vehicle M travels is a deceleration zone.

[0090] Next, the determination unit 153 determines the driving lane of vehicle M based on information related to the road traveled by vehicle M obtained from the first map information 54 and information related to the category of the dividing lines around vehicle M identified by the identification unit 130. For example, if the road traveled by vehicle M is a two-way section with one lane on one side, and the category of the dividing lines on the left and right sides of vehicle M is solid line, dashed line, special line, or shoulder dividing line, the determination unit 153 determines that the driving lane of vehicle M is a one-way lane.

[0091] When the road on which vehicle M travels, as obtained from the first map information 54, is a two-way traffic section with multiple parallel lanes (two or more lanes capable of traveling in the same direction), the determination unit 153 uses the lane markings excluding the two-way traffic markings from the identification results of the area surrounding vehicle M identified by the identification unit 130. For example, if the road on which vehicle M travels is a two-way traffic section with two lanes on one side, and the lane marking on the left side of vehicle M is a dashed line or a special line, the determination unit 153 determines that the driving lane of vehicle M is the right-hand lane (second lane from the left) of a two-way traffic section with two lanes on one side. If the road on which vehicle M travels is a two-way traffic section with three or more lanes on one side, and the lane marking on the right side of vehicle M is a yellow lane line and the lane marking on the left side of vehicle M is a dashed line or a special line, the determination unit 153 determines that the driving lane of vehicle M is the right-hand lane of a three- or more lane section.

[0092] The determination unit 153 may also determine the driving lane of vehicle M by referring to the determination table 182 pre-stored in the storage unit 180 when determining the driving lane as described above, provided that the corresponding conditions are met.

[0093] Figure 4 This is a diagram illustrating an example of the contents of Decision Table 182. Decision Table 182 establishes a correspondence between the decision results and the decision conditions and dividing line conditions. Figure 4 In the example of decision table 182 shown, the condition ID is also included as identification information for identifying each decision condition. The decision start condition is the condition used by the determination unit 153 to determine the driving lane. "Always" means a specified period. The dividing line condition includes the line type conditions for the two dividing lines on the left and right sides of the vehicle M (left first dividing line, left second dividing line, right first dividing line, and right second dividing line). Figure 4 The "-" part included in the dividing line condition indicates that the dividing line can be any type of line (including undetected cases, and any type of line is acceptable). The determination result is the determination result of the lane (driving lane) in which vehicle M is traveling, provided that the conditions shown in the determination start condition and the dividing line condition are met.

[0094] The following is about... Figure 4 The corresponding determination result (determination of the driving lane) based on the combination of the determination start condition and the dividing line information is explained according to each determination mode (condition ID). The following explanation includes cases where the road traveled by vehicle M could not be determined to be a two-way traffic section through the first map information 54 (or a non-two-way traffic section could be determined) (determination modes 1-4), and cases where the road traveled by vehicle M could be determined to be a two-way traffic section (determination modes 5-7).

[0095] <First Judgment Mode>

[0096] Figure 5 This is a diagram representing an example of the first decision pattern. In Figure 5 In the example, vehicle M is traveling at speed VM in the direction of the extension of road RD1 (the X-axis direction in the diagram). Regarding... Figure 5 For example, dividing line RL11 corresponds to the second dividing line on the left, dividing line RL1V2 corresponds to the first dividing line on the left, dividing line RL13 corresponds to the first dividing line on the right, and dividing line RL14 corresponds to the second dividing line on the right. The same applies to subsequent attached diagrams.

[0097] When it is necessary to determine the driving lane of vehicle M, the determination unit 153 determines at a predetermined period (always) whether the lane division conditions of conditions IDs "C101" to "C104" are met. Here, in Figure 5In the example, the recognition unit 130 failed to detect (identify) the dividing lines RL11 and RL14, but was able to identify that the dividing lines RL12 and RL13 were solid lines. In this case, the determination unit 153, because it satisfies... Figure 4 The condition ID “C101” shown indicates that the lane for vehicle M is the end lane of a single lane (single lane). In the first determination mode, if at least one of the dividing lines RL11 and RL14 is not detected and is identified by the identification unit 130 as a shoulder (shoulder dividing line), the determination unit 153 also determines that the lane for vehicle M is the end lane of a single lane.

[0098] <Second Judgment Mode>

[0099] Figure 6 This is a diagram representing an example of the second decision pattern. In Figure 6 In the example, the recognition unit 130 recognizes dividing line RL11 as a shoulder (or roadside area), dividing line RL12 as a solid line, and dividing line RL13 as a dashed line. In the second determination mode, dividing line RL14 can be undetected or a state where some dividing lines can be detected. In this case, the determination unit 153 determines the dividing line as satisfying the condition... Figure 4 The dividing line condition shown in condition ID "C102" indicates that the driving lane of vehicle M is the left-hand lane of a multi-lane road.

[0100] <Third Judgment Mode>

[0101] Figure 7 This is a diagram representing an example of the third decision pattern. In Figure 7 In the example, the recognition unit 130 recognizes the dividing line RL11 as a solid line and the dividing lines RL12 and RL1V3 as dashed lines. In the third determination mode, the dividing line RL14 can be undetected or a state where some dividing lines can be detected. In this case, the determination unit 153 determines the dividing line because it satisfies the condition... Figure 4 The condition shown is the dividing line condition ID "C103". Therefore, it is determined that the driving lane of vehicle M is the second lane (second lane from the left) of a road with three or more lanes. The determination unit 153 can also determine that the condition "C103" is met based on the conditions of the third determination mode, the location information of vehicle M, and the first map information 54.

[0102] <Fourth Judgment Mode>

[0103] Figure 8 This is a diagram representing an example of the fourth decision pattern. In Figure 8In the example, the recognition unit 130 recognizes dividing lines RL11 and RL13 as solid lines and dividing line RL12 as a dashed line. Dividing line RL14 can be undetected or can detect a road shoulder. In this case, the determination unit 153 determines the road shoulder because it satisfies the condition... Figure 4 The dividing line condition shown in condition ID "C104" is therefore determined to be the second lane (second lane from the left) of a two-lane road for vehicle M. The determination unit 153 may also determine that condition "C104" is met based on the above conditions, by referring to the first map information 54 based on the location information of vehicle M, when road RD1 is a two-lane road.

[0104] <Fifth Judgment Mode>

[0105] Figure 9 This is a diagram illustrating an example of the fifth decision pattern. In Figure 9 In the example, based on map information (first map information 54), it is determined that the road RD1 where vehicle M travels is a two-way traffic section with one lane (one lane on each side). In this case, the determination unit 153 determines the condition because it satisfies the following conditions: Figure 4 The condition shown is "C105", so the next step is to determine whether the dividing line condition of "C105" is met. Here, in Figure 9 In the example, the recognition unit 130 identifies the dividing lines RL12 and RL13 as solid lines. In the fifth determination mode, the dividing lines RL11 and RL14 can be undetected or in a state where some dividing lines can be detected. In this case, the determination unit 153 determines that the condition satisfies the condition. Figure 4 The condition ID “C105” indicates that the vehicle M’s driving lane is the end lane of a single lane on one side of a two-way traffic section.

[0106] In the fifth determination mode, the determination unit 153 may replace the dividing lines RL12 and RL13 as solid lines, while at least one of them is a dashed line, a special line, or a shoulder (shoulder dividing line). A special line is, for example, a line whose existence can be identified from a camera image, but whose type (solid line, dashed line, etc.) cannot be identified. Special lines include, for example, lines that have partially disappeared due to wear, pollution, etc., and speed reduction dashed lines. Speed ​​reduction dashed lines are, for example, road markings used to make the road appear narrower to drivers of vehicles. In the fifth determination mode, if the road where vehicle M travels is a two-way section with one lane on one side, and at least one of the dividing lines on the left and right sides of vehicle M is a solid line, a dashed line, a special line, or a shoulder dividing line, the determination unit 153 determines that the lane in which vehicle M travels is the end lane of a one-way road.

[0107] Determining which lane vehicle M is traveling in is crucial, for example, in driving control during the first driving mode. Therefore, even if a single lane on one side is identified as a two-way traffic zone according to the first map information 54, it is necessary to ensure that the vehicle is actually traveling in that lane. Thus, by determining that vehicle M is traveling in the end lane of a single-sided lane based on the type of left and right dividing lines, as in the fifth determination mode, more appropriate driving control can be executed (continued).

[0108] In the fifth determination mode, the determination unit 153 can also be in the following manner: Figure 9 In the case where there are surrounding vehicles (other vehicles within a specified distance of vehicle M) m1 as shown, the identification unit 130 determines whether road RD1 is a two-way road or identifies a two-way lane based on the movement direction of the surrounding vehicles m1. Figure 9 In the example, the surrounding vehicle m1 is traveling in the opposite direction to vehicle M's direction of travel at a speed Vm1. For example, as... Figure 9 As shown, when a neighboring vehicle m1 traveling in the opposite direction to the travel direction of vehicle M is detected in an adjacent lane separated from the travel lane of vehicle M by a dividing line RL13, the determination unit 153 determines that the right adjacent lane of the travel lane of vehicle M is a lane for two-way traffic. Alternatively, the determination unit 153 may determine that the travel lane of vehicle M is the lane closest to a lane for two-way traffic instead of determining it as a lane for two-way traffic.

[0109] <Sixth Judgment Mode>

[0110] Figure 10 This is a diagram representing an example of the sixth decision pattern. In Figure 10 In the example, based on map information (first map information 54), it is determined that the road RD1 where vehicle M travels is a two-way traffic section with two lanes (two lanes on each side). In this case, the determination unit 153 determines the condition because it satisfies the following conditions: Figure 4 The condition shown is the starting condition for condition ID "C106", therefore, the next step is to determine whether the dividing line condition of condition ID "C106" is met. Here, Figure 10 In the example, the identification unit 130 identifies at least the dividing line RL11 as a solid line and the dividing line RL12 as a dashed line or a special line. In the sixth determination mode, the dividing line RL13 can be identified as a solid line, and the dividing line RL14 can be either undetected or in a state where some dividing lines can be detected. In this case, the determination unit 153 determines the dividing line as satisfying the condition. Figure 4 The condition shown is ID "C106", therefore it is determined that the driving lane of vehicle M is the right-hand lane (or the second lane from the left) of a two-lane road on one side of a two-way traffic section.

[0111] In the sixth determination mode, the dividing line RL13 can be replaced by a dashed line, a special line, a shoulder (shoulder dividing line), or a pole, instead of a solid line. Poles include, for example, lane separation markers installed on the road surface, or temporary (not permanent) road sign towers (warning cones, registered trademarks) placed according to construction sections, etc. In the sixth determination mode, when road RD1 is a two-way traffic section with multiple parallel lanes (two or more lanes capable of traveling in the same direction), the determining unit 153 uses the dividing line excluding the two-way traffic dividing line RL14 from the identification results of the area surrounding vehicle M identified by the identification unit 130 to determine the driving lane. Specifically, when road RD1 is a two-way traffic section with two lanes on one side, and the dividing line RL12 on the left side of vehicle M is a dashed line, the determining unit 153 determines that vehicle M's driving lane is the right-hand lane of the two-way traffic section, regardless of the type of the dividing line RL13 on the right side.

[0112] In the sixth determination mode, the determination unit 153 can also be in the following manner: Figure 10 In the presence of surrounding vehicles (other vehicles) m1 and m2 as shown, the identification unit 130 determines whether road RD1 is a two-way road or identifies the driving lane of vehicle M based on the movement direction of the surrounding vehicles m1 and m2. Figure 10 In the example, surrounding vehicle m1 is traveling in the opposite direction to vehicle M's direction of travel at a speed of Vm1, while surrounding vehicle m2 is traveling in the same direction as vehicle M's direction of travel at a speed of Vm2. In this case, the determination unit 153 determines that the right adjacent lane of vehicle M's travel lane is a two-way lane. Since surrounding vehicle m2 is traveling in the lane divided by dividing lines RL11 and RL12, the determination unit 153 determines that vehicle M's travel lane is the right-hand lane of a two-lane roadway on one side.

[0113] <Seventh Judgment Mode>

[0114] Figure 11 This is a diagram representing an example of the seventh decision pattern. In Figure 11 In the example, based on map information (first map information 54), it is determined that the road RD1 on which vehicle M travels is a two-way traffic section with 3 or more lanes (three lanes on one side). In this case, the determination unit 153 determines the route because it satisfies the following conditions: Figure 4 The condition shown is the starting condition for condition ID "C107", therefore, the next step is to determine whether the dividing line condition of condition ID "C107" is met. Here, Figure 11In the example, the recognition unit 130 recognizes the dividing line RL12 as a dashed line and the dividing line RL13 as a solid yellow line. In the seventh determination mode, the dividing lines RL11 and RL14 can be either undetected or in a state where some dividing lines can be detected. In this case, the determination unit 153 determines the dividing line because it satisfies the condition... Figure 4 The condition shown is ID "C107", therefore it is determined that the driving lane of vehicle M is the right-end driving lane of a single-sided lane with three or more lanes in a two-way traffic section.

[0115] In the seventh determination mode, the dividing line RL12 can also replace the dashed line as a special line. In the seventh determination mode, when road RD1 is a two-way traffic section and has multiple parallel lanes (the number of lanes that can travel in the same direction is 3 or more), the determination unit 153 uses the dividing line excluding the two-way traffic dividing line RL14 from the identification results of the area around vehicle M identified by the identification unit 130 to determine the driving lane. Specifically, when the road on which vehicle M travels is a two-way traffic section with three or more lanes on one side, and the dividing line on the right side of vehicle M is a yellow lane line and the dividing line on the left side of vehicle M is a dashed line, the determination unit 153 determines that the driving lane of vehicle M is the right-hand driving lane of a lane with three or more lanes on one side.

[0116] In the seventh determination mode, similar to the sixth determination mode described above, the determination unit 153 can also determine whether road RD1 is a two-way road or determine the driving lane of vehicle M based on the movement direction of surrounding vehicles m1 and m2 when there are surrounding vehicles (other vehicles) m1 and m2.

[0117] In the fifth to seventh determination modes described above, the determination unit 153 may determine whether road RD1 is a two-way traffic section based on the type of dividing lines identified from the camera image, instead of determining that road RD1 is a two-way traffic section from the information contained in the first map information 54 (or, in addition to determining that road RD1 is a two-way traffic section from the information contained in the first map information 54). In this case, for example, the determination unit 153 may identify road RD1 as a two-way traffic section if the dividing lines identified from the camera image contain a solid yellow line or if the dividing lines are in the shape of a two-way traffic section. The determination unit 153 may also determine that road RD1 is a two-way traffic section if the road information identified from the camera image contains a two-way traffic road sign.

[0118] By using the first to seventh determination modes described above to determine the driving lane of vehicle M, even road conditions that are difficult to identify from camera images can be determined more accurately. Therefore, even on two-way roads that are difficult to identify from camera images, mismatches between road information obtained from map information and road (driving lane) information identified from camera images can be suppressed. Therefore, the first driving mode can continue without switching from the first driving mode to the second driving mode. The determination unit 153 can also determine the driving lane of vehicle M using modes other than the first to seventh determination modes.

[0119] If the vehicle does not fall under any of the first to seventh determination modes (condition IDs “C101” to “C107”) mentioned above, the determination unit 153 cannot determine which of the more than one lanes included in road RD1 the vehicle is traveling in.

[0120] When the determination unit 153 determines the driving lane of vehicle M, the mode change processing unit 154 executes a first driving mode, performing driving controls such as keeping vehicle M within the determined driving lane, following the vehicle in front, or changing lanes in the direction of the destination. When the determination unit 153 does not determine the driving lane of vehicle M, the mode change processing unit 154 executes a second driving mode. Therefore, when it is impossible to determine which lane vehicle M is traveling in, automatic driving can be limited to ensure safer driving of vehicle M.

[0121] The mode change processing unit 154 can also continue driving the vehicle M in the current driving mode even if the determination unit 153 has not determined the driving lane of the vehicle M. In this way, by maintaining the current driving mode without switching when the driving lane of the vehicle M cannot be determined, more stable driving can be achieved.

[0122] When the driving lane of vehicle M cannot be determined, the decision to switch to a second driving mode or continue the current driving mode (either the first or second driving mode) can be based on factors such as the surrounding conditions of vehicle M and road information obtained from the first map information 54. For example, according to the recognition unit 130, if there are more than a certain number of other vehicles around vehicle M (within a predetermined distance), there is a possibility that the lane markings cannot be recognized due to the influence of these other vehicles. Therefore, when there are more than a certain number of other vehicles around vehicle M, the mode change processing unit 154 continues the current driving mode even if the driving lane of vehicle M cannot be determined, and switches to the second driving mode when the number of other vehicles is less than the predetermined number. The mode change processing unit 154 can also refer to the first map information 54 to obtain road information around vehicle M based on the location information of vehicle M, and continue the current driving mode when there is no increase or decrease in the number of lanes within a predetermined distance of vehicle M and the road curvature is small (below a threshold), and switch to the second driving mode when there is an increase or decrease in the number of lanes and the road curvature is large (greater than the threshold). Therefore, a more appropriate driving mode can be executed based on the surrounding conditions of vehicle M.

[0123] If the determination unit 153 fails to determine the driving lane of vehicle M, the HMI control unit 170 can also output the situation that the driving lane of vehicle M cannot be determined and notify the occupants of this situation. Therefore, the occupants can be notified of the status of vehicle M, and driving controls such as allowing the occupants to monitor the vehicle's surroundings and switching to manual driving as needed can be performed in accordance with the situation.

[0124] [Processing Flow]

[0125] Next, the process flow executed by the automated driving control device 100 of the embodiment will be described. Hereinafter, the process executed by the automated driving control device 100 will be mainly described focusing on the process of determining the driving lane of the vehicle M and the driving mode switching process based on the determined processing result. The processes in this flowchart can, for example, be repeatedly executed at predetermined times.

[0126] Figure 12 This is a flowchart illustrating an example of the driving control processing performed by the automatic driving control device 100. Figure 12 In the example, the recognition unit 130 recognizes the surrounding conditions of the vehicle M (step S100). Next, the mode determination unit 150 drives the vehicle in any of a plurality of preset driving modes based on the surrounding conditions, etc. (step S102).

[0127] Next, the pattern determination unit 150 acquires the first map information, the recognition result identified by the recognition unit 130, and reference information for determining the position of vehicle M (step S104), and performs processing to determine the driving lane of vehicle M based on the acquired information (step S106). Here, the determination unit 153 of the pattern determination unit 150 determines whether the determination conditions related to two-way traffic are met based on the acquired information (for example, the conditions of condition IDs "C105" to "C107" in the determination table 182 mentioned above) (step S108). If the conditions are met, the determination unit 153 uses special rules (prescribed easing conditions) to determine the driving lane, at least without using information on the type of dividing lines for two-way traffic. More specifically, the determination unit 153 uses dividing lines other than those for two-way traffic to determine the driving lane (step S110). If the conditions are not met in the processing of step S108, the determination unit 153 does not use special rules, but uses the identified dividing lines to determine the driving lane of vehicle M (step S112).

[0128] Next, the mode change processing unit 154 determines whether the driving mode of vehicle M is currently in the first driving mode (step S114). If it is determined that the first driving mode is in operation, the mode change processing unit 154 determines whether the determination unit 153 has determined the driving lane of vehicle M (step S116). If it is determined that the driving lane has been determined, the mode change processing unit 154 continues the first driving mode based on the determined driving lane (step S118); if it is determined that the driving lane has not been determined, it performs control to switch from the first driving mode to the second driving mode (step S120). Thus, the processing of this flowchart ends. If it is determined in step S114 that the first driving mode is not being executed, the processing of this flowchart ends.

[0129] [Variation Example]

[0130] The following describes several variations of this embodiment. For example, in the embodiment described above, if the determination unit 153 is no longer able to determine the driving lane of vehicle M, the driving control unit can also change lanes such as moving vehicle M to a lane located at the left or right end of the road. This allows for more accurate lane determination after a lane change. Alternatively, the driving control unit can immediately determine the driving lane of vehicle M via the determination unit 153 after moving vehicle M to a lane at the left or right end of the road. This allows for faster lane determination even when the driving lane cannot be determined.

[0131] In the above-described embodiments, it is also possible to use the first map information 54 and the image captured by the camera 10 to determine the driving lane when there is map information (second map information) with higher precision than the first map information 54 in addition to the first map information 54, and the second map information cannot be obtained.

[0132] Here, the second map information is, for example, a map information that defines road information for each lane in shorter intervals compared to the first map information 54. The second map information may also include, for example, information about the center of the lane or information about the lane boundaries. The second map information may also include road information, traffic restriction information, residential information (address, postal code), facility information, telephone number information, etc. The second map information can be updated at any time by communicating with other devices via the communication device 20. The second map information can be stored on a storage device such as the HDD or flash memory of the MPU 60, or stored in the storage unit 180.

[0133] For example, if the second map information exists in the MPU60 or the storage unit 180, the lane recommendation unit 61 will divide the path on the map provided by the navigation device 50 into multiple blocks and determine the recommended lane for each block by referring to the second map information. In this case, the lane recommendation unit 61 may also use the lane information contained in the second map information to determine which lane from the left the vehicle M should travel in.

[0134] Here, for example, if vehicle M is in the first driving mode and the second map information cannot be used due to data anomalies or anomalies in the updating of the second map information, the mode determination unit 150 determines the driving lane of vehicle M based on the information obtained from the first map information 54 as described above, so that the first driving mode continues. Therefore, even when the second map information cannot be used, a highly automated driving mode can be executed.

[0135] In this embodiment, the mode determination unit 150 can also determine which of the multiple modes included in the second driving mode to switch to when switching from the first driving mode to the second driving mode, based on the driving state and driving environment of the vehicle M. The driving state is, for example, the driver's state determined by the driver state determination unit 151. The driving environment includes, for example, the shape of the road surrounding the vehicle M, the number of lanes, the presence or absence of branches and merging, the number of other vehicles present in the vicinity, and their relative positions. For example, when the conditions for switching from the first driving mode to the second driving mode are met, if the number of lanes is three and the driving lane of the vehicle M cannot be determined, the mode determination unit 150 decides to switch to mode C; if it has four lanes, it decides to switch to mode D; and if it has five or more lanes, it decides to switch to mode E. Thus, the vehicle M can be driven in a more appropriate mode based on the driving state and driving environment. The dividing line conditions in the determination table 182 of the above embodiment can also be appropriately reread as other dividing lines according to the traffic regulations (road traffic laws) of the foreign country to which this embodiment applies.

[0136] According to the embodiment described above, the vehicle control device includes: an identification unit 130 that identifies the surrounding conditions of the vehicle M; a driving control unit (action plan generation unit 140, second control unit 160) that controls one or both of the steering and speed of the vehicle M based on the surrounding conditions identified by the identification unit 130; an acquisition unit (first acquisition unit) 152 that acquires map information including lane information of the surroundings of the vehicle M and reference information for determining the position of the vehicle M; and a determination unit 153 that determines the road information of the vehicle based on the reference information obtained from the map information. The identification unit 130 identifies the types of lane markings that divide the area around vehicle M into one or more lanes, and determines the driving lane of vehicle M using the lane markings themselves. If the identification unit 153 determines that the road is a two-way road based on at least one of the road information and the type of lane markings, it uses the lane markings themselves instead of the type of lane markings used for two-way traffic identified by the identification unit 130. This allows for more accurate lane determination even if the road is a two-way road. Therefore, the determined lane information can be used to continue the first driving mode or to adjust the degree of driving control in a more appropriate manner.

[0137] Specifically, in situations where there are no surrounding vehicles, it is sometimes impossible to determine whether a road is a two-way road based solely on camera images, and thus the lane for vehicle M cannot be determined. Therefore, in this implementation, when a road is determined to be a two-way road section based on map information, a special rule (a prescribed easing condition) is used to determine the driving lane (e.g., determining the driving lane for vehicle M based on the category of the dividing line corresponding to the number of lanes on one side). This improves the accuracy of lane determination and suppresses false determinations.

[0138] According to the implementation method, the driving lane is determined based on the recognition results of the two dividing lines on each side of the vehicle M (left first dividing line, left second dividing line, right first dividing line, and right second dividing line), thus improving the accuracy of determining the driving lane of the vehicle M without increasing the load on the recognition processing. According to the implementation method, even when high-precision map information is not included in the vehicle M, or when the included high-precision map cannot be used, a navigation map (first map information 54) like that used in the navigation device 50 can be used to more accurately determine the position (driving lane) of the vehicle M, allowing the execution of the first driving mode to continue. Therefore, it eliminates the need for real-time map information updates and management based on map servers, as is required with high-precision maps, thus reducing operating costs.

[0139] The implementation methods described above can be performed as follows.

[0140] A vehicle control device, wherein...

[0141] The vehicle control device includes:

[0142] Storage device, which stores a program; and

[0143] Hardware processor,

[0144] The hardware processor executes the program stored in the storage device to perform the following processing:

[0145] Identify the vehicle's surroundings;

[0146] Perform driving control based on the identified surrounding conditions, controlling one or both of the vehicle's steering and speed;

[0147] Obtain map information containing lane information around the vehicle, and reference information for determining the vehicle's position;

[0148] The vehicle's driving lane is determined based on information about the road the vehicle travels on, obtained from the map information based on the reference information, and the identified categories of road markings that divide one or more lanes around the vehicle.

[0149] If the road is determined to be a two-way traffic section based on at least one of the road information and the type of the road markings, the driving lane is determined without using the information on the type of the two-way traffic markings from the identification results of the vehicle's surroundings.

[0150] The above description illustrates specific embodiments of the present invention, but the present invention is not limited to such embodiments in any way, and various modifications and substitutions can be made without departing from the spirit of the present invention.

Claims

1. A vehicle control device, wherein, The vehicle control device includes: The identification unit identifies the vehicle's surroundings. A driving control unit that controls one or both of the vehicle's steering and speed based on the surrounding conditions identified by the recognition unit. The acquisition unit acquires map information including lane information around the vehicle, and reference information for determining the position of the vehicle; as well as The determining unit determines the vehicle's driving lane based on information about the road the vehicle travels on, obtained from the map information based on the reference information, and the categories of road markings that divide one or more lanes around the vehicle, as identified by the identification unit. When the determining unit determines that the road is a two-way traffic section based on at least one of the road information and the type of the road markings, it determines the driving lane without using the information on the type of the two-way traffic markings from the identification results of the vehicle's surroundings identified by the identification unit. When the determining unit determines that the road is a two-way traffic section with multiple parallel lanes based on the map information, it does not use the information on the type of the two-way traffic dividing line in the recognition result of the vehicle's surroundings identified by the recognition unit to determine the driving lane.

2. The vehicle control device according to claim 1, wherein, When the determining unit determines that the vehicle's driving lane is a single-lane roadway in the case that the road is a two-way traffic section with one lane on one side, and the road dividing lines on the left and right sides of the vehicle are solid lines, dashed lines, special lines, or shoulders, the determining unit determines that the vehicle's driving lane is a single-lane roadway.

3. The vehicle control device according to claim 1, wherein, The determining unit determines that the vehicle is traveling in the right lane of the two-lane road with two lanes on one side, and the road dividing line on the left side of the vehicle is a dashed line or a special line, and the dividing line that is located further away from the left side of the road dividing line when viewed from the vehicle is a solid line.

4. The vehicle control device according to claim 1, wherein, When the road is a two-way traffic section with three or more lanes on one side, and the road dividing line on the right side of the vehicle is a yellow lane line and the road dividing line on the left side of the vehicle is a dashed line or a special line, the determining unit determines that the vehicle's driving lane is the right lane of the three or more lanes on one side.

5. The vehicle control device according to claim 1, wherein, The driving control unit executes at least one of several driving modes, including a first driving mode and a second driving mode that imposes a heavier workload on the vehicle's occupants compared to the first driving mode, to drive the vehicle. The driving control unit executes the first driving mode when the determining unit determines the vehicle's driving lane. The driving control unit executes the second driving mode if the determining unit fails to determine the vehicle's driving lane.

6. The vehicle control device according to claim 1, wherein, The driving control unit executes any one of multiple driving modes with different tasks assigned to the occupants of the vehicle to drive the vehicle. If the determining unit fails to determine the driving lane of the vehicle, the driving mode being executed continues to drive the vehicle.

7. A vehicle control method, wherein, The vehicle control method causes the computer to perform the following processing: Identify the vehicle's surroundings; Perform driving control based on the identified surrounding conditions, controlling one or both of the vehicle's steering and speed; Obtain map information containing lane information around the vehicle, and reference information for determining the vehicle's position; The vehicle's driving lane is determined based on information about the road the vehicle travels on, obtained from the map information based on the reference information, and the categories of road dividing lines that divide one or more lanes around the vehicle. If, based on at least one of the road information and the type of the road markings, the road is determined to be a two-way traffic section, the driving lane is determined without using the information on the type of the two-way traffic markings in the identification results of the vehicle's surroundings; and When the road is determined to be a two-way traffic section with multiple parallel lanes based on the map information, the driving lane is determined without using the information on the type of the two-way traffic dividing line in the identification results of the identified vehicle's surroundings.

8. A storage medium storing a program, wherein, The program causes the computer to perform the following processing: Identify the vehicle's surroundings; Perform driving control based on the identified surrounding conditions, controlling one or both of the vehicle's steering and speed; Obtain map information containing lane information around the vehicle, and reference information for determining the vehicle's position; The vehicle's driving lane is determined based on information about the road the vehicle travels on, obtained from the map information based on the reference information, and the categories of road dividing lines that divide one or more lanes around the vehicle. If, based on at least one of the road information and the type of the road markings, the road is determined to be a two-way traffic section, the driving lane is determined without using information about the type of at least the two-way traffic markings from the identified results of the vehicle's surroundings; and When the road is determined to be a two-way traffic section with multiple parallel lanes based on the map information, the driving lane is determined without using the information on the type of the two-way traffic dividing line in the identification results of the identified vehicle's surroundings.

Images