Vehicle control device, vehicle control method, and program

Abstract

To improve a specification rate of a travel lane of a vehicle.SOLUTION: A vehicle control device according to an embodiment comprises: a recognition unit which recognizes a peripheral situation of a vehicle; a driving control unit which controls one or both of steering and speed of the vehicle on the basis of the peripheral situation recognized by the recognition unit; an acquisition unit which acquires map information including lane information of the periphery of the vehicle and reference information for specifying the position of the vehicle; and a specification unit which specifies a travel lane of the vehicle on the basis of information on a road on which the vehicle travels acquired from the map information on the basis of the reference information and the type of a section line that divides the one or more lanes in the periphery of the vehicle recognized by the recognition unit. The specification unit specifies the travel lane of the vehicle without using at least information on the type for the specific road section line in the section lines recognized by the recognition unit.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a program. [Background technology]

[0002] In recent years, there has been progress in research into autonomous driving, which recognizes the situation around the vehicle and automatically controls the traveling of the vehicle. In this context, there are technologies that recognize the state of marked lines on the road surface to detect information about the traveling environment, determine control content by estimating the lane markings that separate the current traveling lane from the line types stored in advance in a storage unit, and determine the position of the vehicle using the recognition results from a camera and map information (see, for example, Patent Documents 1 to 4). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2000-105898 [Patent Document 2] Japanese Patent Application Laid-Open No. 2010-221859 [Patent Document 3] Japanese Patent Application Publication No. 10-300494 [Patent Document 4] Japanese Patent Application Laid-Open No. 2013-032953 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technology, when there are certain road dividing lines that are difficult to recognize from images originally captured by a camera, it is sometimes impossible to identify the vehicle's lane or the lane is identified incorrectly.

[0005] The aspects of the present invention have been made in consideration of these circumstances, and one of their objectives is to provide a vehicle control device, a vehicle control method, and a program that can improve the rate at which a vehicle's driving lane is identified. [Means for solving the problem]

[0006] A vehicle control device, a vehicle control method, and a program according to the present invention employ the following configuration. (1): A vehicle control device according to one embodiment of the present invention includes a recognition unit that recognizes the surrounding conditions of a vehicle; a driving control unit that controls one or both of the steering and speed of the vehicle based on the surrounding conditions recognized by the recognition unit; an acquisition unit that acquires map information including lane information about the surroundings of the vehicle and reference information for identifying the position of the vehicle; and an identification unit that identifies the driving lane of the vehicle based on information about the road on which the vehicle is traveling, acquired from the map information based on the reference information, and the type of marking lines that demarcate each of one or more lanes around the vehicle recognized by the recognition unit, and the identification unit is a vehicle control device that identifies the driving lane of the vehicle for specific road markings among the marking lines recognized by the recognition unit, without using at least the type information.

[0007] (2) In the above aspect (1), the specific road-dividing line is a dashed deceleration line at least on either the left or right side of the vehicle.

[0008] (3): In the above aspect (1) or (2), when there is a possibility that one of the left and right dividing lines of the vehicle is a deceleration broken line, the identification unit identifies the lane in which the vehicle is traveling based on the type of at least the other dividing line.

[0009] (4): In the above aspect (1) or (2), when there is a possibility that one of the left and right dividing lines of the vehicle is a dashed deceleration line, the identification unit identifies the lane in which the vehicle is traveling based on the shoulder dividing line outside the vehicle's roadway.

[0010] (5): In any one of the above aspects (1) to (4), the identification unit identifies the vehicle's driving lane based on solid lines other than the road dividing lines on the left and right of the vehicle when there is a possibility that one or both of the road dividing lines on the left and right of the vehicle are of the type of a deceleration broken line.

[0011] (6): In any one of the above aspects (1) to (5), the driving control unit drives the vehicle by executing one of a plurality of driving modes including at least a first driving mode and a second driving mode that imposes a heavier task on the vehicle occupants than the first driving mode, and executes the first driving mode when the identification unit identifies the vehicle's driving lane, and executes the second driving mode when the identification unit does not identify the vehicle's driving lane.

[0012] (7): In any one of the above aspects (1) to (5), the driving control unit runs the vehicle by executing one of a plurality of driving modes that impose different tasks on the vehicle's occupants, and if the identification unit does not identify the vehicle's driving lane, the driving mode currently being executed is continued to run the vehicle.

[0013] (8): A vehicle control method according to one embodiment of the present invention is a vehicle control method in which a computer recognizes the surrounding conditions of a vehicle, performs driving control to control one or both of the steering and speed of the vehicle based on the recognized surrounding conditions, acquires map information including lane information around the vehicle and reference information for identifying the position of the vehicle, identifies the vehicle's driving lane based on information about the road on which the vehicle is traveling, acquired from the map information based on the reference information, and the type of marking lines that demarcate each of the recognized one or more lanes around the vehicle, and identifies the vehicle's driving lane for specific road markings among the recognized marking lines, without using at least the type information.

[0014] (9): A program according to one embodiment of the present invention causes a computer to recognize the surrounding conditions of a vehicle, execute driving control to control one or both of the steering and speed of the vehicle based on the recognized surrounding conditions, acquire map information including lane information about the surroundings of the vehicle and reference information for identifying the position of the vehicle, identify the vehicle's traveling lane based on information about the road on which the vehicle is traveling, acquired from the map information based on the reference information, and the type of marking lines that mark each of the recognized one or more lanes around the vehicle, and identify the vehicle's traveling lane for specific road markings among the recognized marking lines without using at least the type information. [Effects of the Invention]

[0015] According to the above aspects (1) to (9), it is possible to improve the rate at which the vehicle's travel lane is identified. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a configuration diagram of a vehicle system 1 that uses a vehicle control device according to an embodiment. [Figure 2] FIG. 2 is a functional configuration diagram of a first control unit 120 and a second control unit 160 according to the embodiment. [Figure 3] 1 is a diagram showing an example of the relationship between a driving mode, a control state of a vehicle M, and a task. [Figure 4] FIG. 10 is a diagram showing an example of the contents of a decision table 182. [Figure 5] FIG. 10 is a diagram showing an example of a first line type pattern. [Figure 6] FIG. 10 is a diagram showing an example of a second line type pattern. [Figure 7] FIG. 10 is a diagram showing an example of a third line type pattern. [Figure 8] FIG. 10 is a diagram showing an example of a fourth line type pattern. [Figure 9] FIG. 10 is a diagram showing an example of a fifth line type pattern. [Figure 10] FIG. 10 is a diagram showing an example of a sixth line type pattern. [Figure 11] 3 is a flowchart showing an example of the flow of a driving control process executed by the automatic driving control device 100. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, an embodiment of a vehicle control device, a vehicle control method, and a program according to the present invention will be described with reference to the drawings. Note that the following description will be given for a case where a law stipulates driving on the left side of the road, but if a law stipulates driving on the right side of the road, the left and right sides can be reversed.

[0018] [Overall configuration] FIG. 1 is a configuration diagram of a vehicle system 1 using a vehicle control device according to an embodiment. The vehicle (hereinafter, vehicle M) on which the vehicle system 1 is mounted may be, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source may be an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using power generated by a generator connected to the internal combustion engine, or discharge power from a secondary battery or a fuel cell. In the following, an embodiment in which the vehicle control device is applied to an autonomous vehicle will be described as an example. Autonomous driving refers to, for example, automatically controlling one or both of the steering and the speed of the vehicle M to perform driving control. The driving control of the vehicle M may include various driving assistance systems such as adaptive cruise control (ACC), auto lane changing (ALC), and lane keeping assistance system (LKAS). Some or all of the driving of an autonomous vehicle may be manually controlled by an occupant (driver).

[0019] The vehicle system 1 includes, for example, a camera (an example of an imaging unit) 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, vehicle sensors 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driver monitor camera 70, a driving operator 80, an automatic driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are connected to each other via multiple communication lines such as a CAN (Controller Area Network) communication line, serial communication lines, a wireless communication network, etc. Note that the configuration shown in FIG. 1 is merely an example, and some of the configuration may be omitted, or other components may be added. A combination of the camera 10, the radar device 12, and the LIDAR 14 is an example of an "external environment sensor ES." The external sensor ES may include other detection units (e.g., sonar) that recognize the surrounding conditions of the vehicle, or may include an object recognition device 16. The HMI 30 is an example of an "output device." The automatic driving control device 100 is an example of a "vehicle control device."

[0020] The camera 10 is a digital camera that uses a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to any location on the vehicle M. For example, when capturing an image in front of the vehicle M, the camera 10 is attached to the top of the front windshield or the back of the rearview mirror. When capturing an image behind the vehicle M, the camera 10 is attached to the top of the rear windshield or the back door. When capturing an image of the sides and rear of the vehicle M, the camera 10 is attached to a door mirror. The camera 10 periodically captures images of the surroundings of the vehicle M. The camera 10 may be a stereo camera. The camera 10 may be provided with multiple cameras (e.g., a first camera and a second camera), and the multiple cameras may capture images in the same direction. Alternatively, the first camera may normally capture images, and when a predetermined condition is met, the second camera or both the first and second cameras may capture images. The predetermined condition is, for example, when road dividing lines (hereinafter referred to as dividing lines) that divide lanes or the like included in the road on which the vehicle M is traveling are recognized from an image captured by a camera (hereinafter referred to as a camera image). The dividing lines may include, for example, line segment information other than dividing lines that divide lanes.

[0021] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by an object (reflected waves) to detect at least the position (distance and direction) of the object. The radar device 12 is attached to any location on the vehicle M. The radar device 12 may detect the position and speed of an object using an FM-CW (Frequency Modulated Continuous Wave) method.

[0022] The LIDAR 14 irradiates the surroundings of the vehicle M with light (or electromagnetic waves with wavelengths similar to light) and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time between light emission and light reception. The irradiated light is, for example, pulsed laser light. The LIDAR 14 may be attached to any location on the vehicle M.

[0023] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the camera 10, radar device 12, and LIDAR 14 included in the external sensors ES to recognize the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition results to the automatic driving control device 100. The object recognition device 16 may output the detection results from the camera 10, radar device 12, and LIDAR 14 directly to the automatic driving control device 100. The object recognition device 16 may be omitted from the vehicle system 1.

[0024] The communication device 20 communicates with other vehicles in the vicinity of the vehicle M, for example, using a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), etc., or communicates with various server devices via a wireless base station.

[0025] The HMI 30 presents various information to the occupants of the vehicle M under the control of the HMI control unit 170 and accepts input operations by the occupants. The HMI 30 includes, for example, various display devices, speakers, switches, a microphone, a buzzer, a touch panel, keys, etc. The various display devices are, for example, a liquid crystal display (LCD) or an organic electroluminescence (EL) display device. The display device is provided, for example, in the instrument panel near the front of the driver's seat (the seat closest to the steering wheel) and is installed in a position where the occupant can see it through the gap in the steering wheel or over the steering wheel. The display device may also be installed in the center of the instrument panel. The display device may also be a head-up display (HUD). The HUD projects an image onto a portion of the front windshield in front of the driver's seat, allowing the occupant sitting in the driver's seat to see a virtual image. The display device displays an image generated by the HMI control unit 170, which will be described later. The HMI 30 may also include a driving changeover switch that switches between automatic driving and manual driving by the occupant. The switch includes, for example, a turn signal switch (directional indicator) 32. The turn signal switch 32 is provided, for example, on the steering column or the steering wheel. The turn signal switch 32 is an example of an operation unit that receives, for example, an instruction from the occupant to change lanes of the vehicle M.

[0026] The vehicle sensors 40 include a vehicle speed sensor that detects the speed of the vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity around a vertical axis, and a direction sensor that detects the direction of the vehicle M. The vehicle sensors 40 may also include a steering angle sensor that detects the steering angle of the vehicle M (which may be the angle of the steering wheels or the operating angle of the steering wheel). The vehicle sensors 40 may also include a position sensor that acquires the position of the vehicle M. The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. The position sensor may also be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver 51 of the navigation device 50.

[0027] The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as a hard disk drive (HDD) or flash memory. The GNSS receiver 51 identifies the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be identified or supplemented by an inertial navigation system (INS) that uses the output of the vehicle sensors 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, etc. The navigation HMI 52 may share some or all of the components with the HMI 30 described above. The route determination unit 53 determines, for example, a route (hereinafter, a route on a map) from the position of the vehicle M identified by the GNSS receiver 51 (or any input position) to a destination input by the occupant using the navigation HMI 52, with reference to the first map information 54.

[0028] The first map information 54 includes, for example, information in which information about lanes is added for each road (hereinafter referred to as lane information). The lane information includes, for example, nodes indicating the start and end of a road section and links representing the road shape between the nodes. The lane information may also include the number of lanes (number of parallel lanes), the number of lanes to be increased or decreased, and the direction of lane increase or decrease (information indicating whether lanes are increased or decreased on the left or right side of the road's direction of travel) in a given section, such as a road section. The lane information may also include, for example, information about the type of lane marking (for example, solid line, dashed line, deceleration dashed line, shoulder line). Deceleration dashed lines are road markings that make the road appear narrower to vehicle drivers in road sections where there are frequent vehicle collisions, for example. Deceleration dashed lines are set along the shapes of lane boundaries, outer lines, and center lines, for example, in downhill sections of roads, sections entering curves, intersections with poor visibility, etc. The dashed deceleration lines make the road appear narrower to the driver, which is expected to have an effect of slowing down the vehicle M during manual driving. The first map information 54 may also include information such as the distance and curvature of the road section, the road type (e.g., expressway, general road), and POI (Point of Interest) information. The route on the map is output to the MPU 60.

[0029] The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be realized, for example, by the functions of a terminal device such as a smartphone or tablet device carried by the occupant. The navigation device 50 may transmit the current position and destination to a navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server. Note that the first map information 54 may be stored in the storage unit 180 instead of in the navigation device 50.

[0030] The MPU 60 includes, for example, a recommended lane determination unit 61. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, by dividing the route into 100 m intervals in the vehicle travel direction), and determines a recommended lane for each block from the lane information in the first map information 54. The recommended lane determination unit 61 may also determine a recommended lane for each road stored in the first map information 54. For example, the recommended lane determination unit 61 determines which lane from the left (or right) to use for travel. When a branch point is present on the route on the map, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel along a reasonable route to reach the branch point.

[0031] The driver monitor camera 70 is, for example, a digital camera that uses a solid-state imaging element such as a CCD or CMOS. The driver monitor camera 70 is attached to any location on the vehicle M in a position and orientation that allows it to capture an image of the head of an occupant (hereinafter, driver) seated in the driver's seat of the vehicle M from the front (in an orientation that captures an image of the face). For example, the driver monitor camera 70 is attached to the top of a display device provided in the center of the instrument panel of the vehicle M.

[0032] The driving operators 80 include, for example, a steering wheel 82, an accelerator pedal, a brake pedal, a shift lever, and other operators. The driving operators 80 are equipped with sensors that detect the amount of operation or the presence or absence of operation, and the detection results are output to the automatic driving control device 100 or some or all of the driving force output device 200, the brake device 210, and the steering device 220. The steering wheel 82 is an example of an "operator that accepts steering operation by the driver." The operator does not necessarily have to be annular and may be in the form of an irregular steering wheel, a joystick, a button, or the like. A steering grip sensor 84 is attached to the steering wheel 82. The steering grip sensor 84 is realized by a capacitance sensor or the like, and outputs a signal to the automatic driving control device 100 that can detect whether the driver is gripping the steering wheel 82 (meaning that the driver is in contact with the steering wheel in a state where force can be applied).

[0033] The autonomous driving control device 100 includes, for example, a first control unit 120, a second control unit 160, an HMI control unit 170, and a storage unit 180. The first control unit 120, the second control unit 160, and the HMI control unit 170 are each realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Furthermore, some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transitory storage medium) such as the HDD or flash memory of the automatic driving control device 100, or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the automatic driving control device 100 by inserting the storage medium (non-transitory storage medium) into a drive device. The combination of the action plan generation unit 140 and the second control unit 160 is an example of a "driving control unit." The HMI control unit 170 is an example of an "output control unit."

[0034] The storage unit 180 may be realized by the various storage devices described above, or a solid state drive (SSD), an electrically erasable programmable read only memory (EEPROM), a read only memory (ROM), or a random access memory (RAM). The storage unit 180 stores, for example, a determination table 182, a program, and various other information. The determination table 182 is, for example, a table that is referenced to identify the lane in which the vehicle M is traveling. The determination table 182 will be described later. The storage unit 180 may also store the first map information 54.

[0035] FIG. 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160 according to the embodiment. The first control unit 120 includes, for example, a recognition unit 130, an action plan generation unit 140, and a mode determination unit 150. The first control unit 120, for example, implements a function based on AI (Artificial Intelligence) and a function based on a predefined model in parallel. For example, the function of "recognizing intersections" may be implemented by executing, in parallel, intersection recognition using deep learning or the like and recognition based on predefined conditions (such as the presence of traffic lights and road markings that can be pattern-matched), and then scoring and comprehensively evaluating both. This ensures the reliability of autonomous driving.

[0036] The recognition unit 130 recognizes the position, speed, acceleration, and other states of objects around the vehicle M based on information input from the external sensors ES. The position of an object is recognized as a position on an absolute coordinate system with a representative point of the vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or may be represented by an area. The "state" of an object may include the acceleration or jerk of the object, or the "behavioral state" (for example, whether or not the object is changing lanes or is about to change lanes).

[0037] Furthermore, the recognition unit 130 recognizes, for example, the lane in which the vehicle M is traveling (driving lane). For example, the recognition unit 130 recognizes the dividing lines on the left and right sides of the vehicle M from the camera image captured by the camera 10, and recognizes the driving lane based on the positions of the recognized dividing lines. Note that the recognition unit 130 may recognize the driving lane by recognizing landmarks (road boundaries, road boundaries) that can identify the lane position, including not only dividing lines but also shoulders, curbs, medians, guardrails, fences, walls, etc. In this recognition, the position of the vehicle M acquired from the navigation device 50 and the processing results by the INS may be taken into consideration. The recognition unit 130 may also recognize stop lines, obstacles, red lights, toll booths, and other road phenomena.

[0038] When recognizing the driving lane, the recognition unit 130 recognizes the position and orientation of the vehicle M with respect to the driving lane. For example, the recognition unit 130 may recognize the deviation of the reference point of the vehicle M from the center of the lane and the angle it forms with a line connecting the centers of the lanes in the traveling direction of the vehicle M as the relative position and orientation of the vehicle M with respect to the driving lane. Alternatively, the recognition unit 130 may recognize the position of the reference point of the vehicle M with respect to one of the side edges of the driving lane (a lane marking or a road boundary) as the relative position of the vehicle M with respect to the driving lane. Note that the recognition of the driving lane by the recognition unit 130 and the recognition of the position and orientation of the vehicle M with respect to the driving lane may be performed by the identification unit 153, which will be described later.

[0039] The behavior plan generation unit 140 generates a target trajectory along which the vehicle M will automatically (without driver operation) travel in the future so that, in principle, the vehicle M will travel along the recommended lane determined by the recommended lane determination unit 61 and can also respond to the surrounding conditions of the vehicle M. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequence of points (trajectory points) that the vehicle M should reach. The trajectory points are points that the vehicle M should reach at every predetermined travel distance (for example, about several meters) along the road. Separately, target speeds and target accelerations are generated as part of the target trajectory for every predetermined sampling time (for example, about a few tenths of a second). Furthermore, the trajectory points may be positions that the vehicle M should reach at each predetermined sampling time. In this case, information on the target speed and target acceleration is expressed as the interval between trajectory points.

[0040] The behavior plan generation unit 140 may set an autonomous driving event (function) when generating a target trajectory. The autonomous driving events include a constant speed driving event, a low speed following driving event, a lane change event, a branching event, a merging event, a takeover event, etc. The behavior plan generation unit 140 generates a target trajectory according to the activated event.

[0041] The mode determination unit 150 determines the driving mode of the vehicle M to be one of a plurality of driving modes in which the tasks assigned to the driver are different (in other words, a plurality of modes in which the degree of automation is different). The mode determination unit 150 includes, for example, a driver state determination unit 151, a first acquisition unit 152, an identification unit 153, and a mode change processing unit 154. The individual functions of these units will be described later. The first acquisition unit 152 is an example of an "acquisition unit."

[0042] FIG. 3 is a diagram showing an example of the relationship between driving modes, control states of vehicle M, and tasks. In the example of FIG. 3, the driving modes of vehicle M include, for example, five modes, Mode A to Mode E. In FIG. 3, Modes A and B are examples of a "first driving mode," and Modes C, D, and E are examples of a "second driving mode." Note that there may be driving modes other than Modes A to E, and there may be driving modes other than the first driving mode and the second driving mode. Among Modes A to E, the control state, i.e., the degree of automation (control degree) of the driving control of vehicle M, is highest in Mode A, followed by Mode B, Mode C, and Mode D, with Mode E being the lowest. Conversely, the tasks imposed on the driver (occupant) are lightest in Mode A, followed by Mode B, Mode C, and Mode D, with Mode E, which involves manual driving, being the most severe. In modes B to E, the vehicle is in a non-autonomous driving control state, so the role of the automatic driving control device 100 is to terminate the control related to the automatic driving and transition to driving assistance or manual driving. The contents of each mode are exemplified below.

[0043] In Mode A, the vehicle M is in an autonomous driving state, and the driver is not required to monitor the surroundings of the vehicle M or grip the steering wheel 82 (hereinafter referred to as "steering grip"). Surroundings monitoring includes monitoring at least the direction of travel of the vehicle M (e.g., forward). "Forward" refers to the space in the direction of travel of the vehicle M as seen through the front windshield. However, even in Mode A, the driver is required to be in a position to quickly transition to manual driving in response to a request from a system centered on the automatic driving control device 100. Note that the term "autonomous driving" as used here refers to control of both the steering and speed of the vehicle M independently of the driver's operation. Mode A is a driving mode that can be implemented, for example, on a motorway such as an expressway, when certain conditions are met, such as the vehicle M traveling at a predetermined speed (e.g., approximately 50 km / h) or less and there is a vehicle ahead to be followed. Mode A is sometimes referred to as TJP (Traffic Jam Pilot) mode. If these conditions are no longer met, the mode determination unit 150 changes the driving mode of the vehicle M to Mode B.

[0044] Furthermore, while mode A is being executed, the occupant can perform a second task. The second task is, for example, an action other than driving that is permitted by the occupant while the vehicle M is being driven autonomously. The second task includes, for example, watching television, using a terminal device (e.g., a smartphone or tablet) carried by the occupant (e.g., making calls, sending and receiving emails, using a social networking service (SNS), browsing the web, etc.), eating, etc.

[0045] In mode B, the vehicle enters a driving assistance state, and the driver is tasked with monitoring the surroundings of the vehicle M (hereinafter referred to as periphery monitoring), but is not tasked with gripping the steering wheel 82. For example, in mode B, the vehicle system 1 determines whether or not to change lanes based on route settings to the destination set by the navigation device 50, without receiving a lane change instruction from the occupant. A lane change refers to moving the vehicle M from the lane in which the vehicle M is traveling to an adjacent lane adjacent to the lane in which the vehicle M is traveling, and may include a lane change due to a branch or merge. The main driver in modes A and B is the vehicle system 1.

[0046] In Mode C, the system enters a driving assistance state, and the driver is tasked with the tasks of monitoring the surroundings and holding the steering wheel 82. For example, in Mode C, if the vehicle system 1 determines that the vehicle M needs to change lanes, it queries the occupant via the HMI 30, and if the HMI 30 or the like accepts the occupant's approval for the lane change, driving assistance is provided to execute the lane change. Lane change control in Modes B and C is system-driven.

[0047] Mode D is a driving mode that requires some degree of driving operation by the driver with respect to at least one of steering and acceleration / deceleration of the vehicle M. For example, in Mode D, driving assistance such as ACC (Adaptive Cruise Control) and LKAS (Lane Keeping Assist System) is provided. Also, in Mode D, when an instruction to change lanes of the vehicle M is received by the driver operating the turn signal switch 32, driving assistance is provided to execute the lane change in the instructed direction. A lane change in Mode D is a lane change intentionally made by the driver. The driver's operation of the turn signal switch 32 is an example of a driving operation. Also, the driving operation in Mode D may include a driving operation for controlling steering or acceleration / deceleration.

[0048] In mode E, the vehicle M is in a manual driving state where the driver must perform both steering and acceleration / deceleration. In both modes D and E, the driver is naturally tasked with monitoring the surroundings of the vehicle M. In modes C to E, the driver is the main driver.

[0049] When the driver does not perform the task related to the determined driving mode, the mode determination unit 150 changes the driving mode of the vehicle M to a driving mode with a more severe task.

[0050] For example, if the driver is in a position where he or she cannot transition to manual driving in response to a request from the system while mode A is being executed (for example, if the driver continues to look away from the vehicle outside the permitted area or if a sign of driving difficulty is detected), the mode determination unit 150 causes the HMI control unit 170 to execute control using the HMI 30 to prompt the driver to transition to manual driving in mode E. Furthermore, if the driver does not respond within a predetermined time after the HMI control unit 170 has executed control to prompt the driver to transition to manual driving, or if the mode determination unit 150 estimates that the driver is not in a state to perform manual driving, the mode determination unit 150 performs control such as gradually decelerating the vehicle M while pulling it to a target position (for example, a shoulder of the road) and stopping the automatic driving. Furthermore, after the automatic driving is stopped, the vehicle M enters a state of mode D or E, and the vehicle M can be started by manual operation by the driver. The same applies hereinafter to "stopping automatic driving."

[0051] In mode B, if the driver is not monitoring the road ahead, the mode determination unit 150 uses the HMI 30 to prompt the driver to monitor the road ahead, and if the driver does not comply, the mode determination unit 150 controls the vehicle M to approach a target position and gradually stop, and the automatic driving is stopped. In mode C, if the driver is not monitoring the road ahead or is not gripping the steering wheel 82, the mode determination unit 150 uses the HMI 30 to prompt the driver to monitor the road ahead and / or grip the steering wheel 82, and if the driver does not comply, the mode determination unit 150 controls the vehicle M to approach a target position and gradually stop, and the automatic driving is stopped.

[0052] The driver state determination unit 151 determines whether the occupant (driver) is in a state suitable for driving. For example, the driver state determination unit 151 monitors the driver's state for the above-mentioned mode change and determines whether the driver's state is suitable for the task. For example, the driver state determination unit 151 analyzes images captured by the driver monitor camera 70 and performs posture estimation processing to determine whether the driver is in a position that prevents the driver from switching to manual driving in response to a request from the system. The driver state determination unit 151 also analyzes images captured by the driver monitor camera 70 and performs gaze estimation processing to determine whether the driver is monitoring the periphery (more specifically, the front) of the vehicle M. If the driver state determination unit 151 determines that the driver is not in a state suitable for the task for a predetermined time or longer, it determines that the driver is in an unsuitable state for driving the task. If the driver state determination unit 151 determines that the driver is in a state suitable for driving the task, it determines that the driver is in a suitable state for driving the task. The driver state determination unit 151 may also determine whether the occupant is in a state where they can take over driving.

[0053] The first acquisition unit 152 acquires first map information 54. Furthermore, the first acquisition unit 152 acquires reference information for identifying the position of the vehicle M. The reference information is, for example, position information of the vehicle M detected by the vehicle sensor 40 or a camera image captured by the camera 10. Furthermore, the reference information may include a part or all of the recognition result by the recognition unit 130.

[0054] The identification unit 153 refers to the first map information 54 based on the position information of the vehicle M included in the reference information, and identifies the driving lane of the vehicle M from among one or more lanes included in the road on which the vehicle M is traveling. Furthermore, the identification unit 153 identifies the driving lane based on, for example, road information acquired from the first map information 54 and the type of lane marking recognized by the recognition unit 130. Furthermore, when a specific road lane marking is included in at least one of the road information and the type of lane marking, the identification unit 153 identifies the driving lane for the specific road lane marking from the recognition results of the surroundings of the vehicle M recognized by the recognition unit 130, without using at least the type information.

[0055] The specific road dividing line is, for example, not a dividing line that divides the driving lanes on the road, but line segment information drawn or formed on the road along the dividing line that divides the driving lane of vehicle M. The specific road dividing line is line segment information in which the type of dividing line cannot be detected (recognized), but the existence of a line segment can be recognized. The specific road dividing line is, for example, a deceleration dashed line, but may be other line segment information. The deceleration dashed line may be mistakenly detected as a normal dashed line or solid line based on the recognition result of the recognition unit 130. Therefore, a false detection may result in a discrepancy between the number of lanes on the road obtained from the map information and the number of lanes acquired from the camera image, or the vehicle's driving lane may be incorrectly identified. Therefore, when the identification unit 153 determines that a specific road dividing line may exist based on the recognition patterns of multiple dividing lines existing around vehicle M, it identifies the driving lane of vehicle M as a possible dividing line without using information on the type of the specific road dividing line. Furthermore, the identification unit 153 may identify the driving lane of the vehicle M based on the line type of other dividing lines excluding the specific road dividing line. In the following example, the specific road dividing line is assumed to be a dashed deceleration line, and the driving lane of the vehicle M is further identified using dividing lines excluding the specific road dividing line.

[0056] The identification unit 153 also identifies the driving lane of the vehicle M at a predetermined timing. The predetermined timing may be, for example, a predetermined cycle, or may be when the execution of autonomous driving begins, when driving on a specific road such as an expressway begins, or when a road section changes in the first map information 54. The predetermined timing may be, for example, when the number of lanes in the map information and the number of lanes obtained from the camera image do not match, causing the current driving lane to be reset and the driving lane to need to be identified again, when the driving lane of the vehicle M cannot be identified and there is no change in the road within a predetermined distance, or when an operation to start autonomous driving is performed by the occupant. The predetermined timing may also be, for example, when the number of lanes on the road on which the vehicle M is traveling has increased or decreased, or will increase or decrease in the near future. Details of the function of the identification unit 153 will be described later.

[0057] The mode change processing unit 154 determines the driving mode of the vehicle M based on the determination result of the driver state determination unit 151, the identification result of the identification unit 153, etc. The mode change processing unit 154 may also determine to continue the driving mode currently being executed or to switch to another mode. The mode change processing unit 154 also performs various processes for changing to the driving mode determined by the mode determination unit 150. For example, the mode change processing unit 154 instructs a driving assistance device (not shown) to operate, causes the HMI control unit 170 to output information for prompting the driver to take action to the HMI 30, or instructs the action plan generation unit 140 to generate a target trajectory based on the action plan generation unit 140 according to the driving mode.

[0058] The second control unit 160 controls the traveling driving force output device 200, the braking device 210, and the steering device 220 so that the vehicle M passes through the target trajectory generated by the behavior plan generation unit 140 at the scheduled time.

[0059] The second control unit 160 includes, for example, a second acquisition unit 162, a speed control unit 164, and a steering control unit 166. The second acquisition unit 162 acquires information on the target trajectory (trajectory points) generated by the action plan generation unit 140 and stores it in a memory (not shown). The speed control unit 164 controls the driving force output device 200 or the brake device 210 based on the speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized by, for example, a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road ahead of the vehicle M and feedback control based on the deviation from the target trajectory.

[0060] The HMI control unit 170 notifies the occupant of predetermined information via the HMI 30. The predetermined information includes, for example, information related to the driving of the vehicle M, such as information about the state of the vehicle M and information about driving control. The information about the state of the vehicle M includes, for example, the speed of the vehicle M, engine speed, and shift position. The information about driving control includes, for example, an inquiry about whether to change lanes, whether a driving mode is active, information about changing the driving mode, information required for switching the driving mode (task request information for the occupant), and information about the driving control status (for example, the content of an event currently being executed). The predetermined information may also include information unrelated to driving control of the vehicle M, such as television programs, content (for example, movies) stored on a storage medium such as a DVD, etc. In addition, the specified information may include, for example, information regarding the current location and destination of vehicle M, information regarding the remaining fuel, information indicating whether the lane vehicle M is traveling in has been identified, the remaining distance until the driving mode is switched, the direction in which lanes are added or removed, the number of lanes added or removed, the number of lanes running parallel to the traveling lane (number of parallel lanes), etc.

[0061] For example, the HMI control unit 170 may generate an image including the predetermined information described above and display the generated image on the display device of the HMI 30, or may generate a sound indicating the predetermined information and output the generated sound from a speaker of the HMI 30. Furthermore, the HMI control unit 170 may output the information received by the HMI 30 to the communication device 20, the navigation device 50, the first control unit 120, etc.

[0062] The driving force output device 200 outputs a driving force (torque) for the vehicle to travel to the drive wheels. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, etc., and an ECU (Electronic Control Unit) that controls these. The ECU controls the above components according to information input from the second control unit 160 or information input from the driving operator 80.

[0063] Braking device 210 may include, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from second control unit 160 or information input from driving operation device 80, so that a brake torque corresponding to the braking operation is output to each wheel. Braking device 210 may include a backup mechanism that transmits hydraulic pressure generated by operation of a brake pedal included in driving operation device 80 to the cylinder via a master cylinder. Note that braking device 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic brake device that controls an actuator according to information input from second control unit 160 to transmit hydraulic pressure from a master cylinder to the cylinder.

[0064] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor applies a force to a rack and pinion mechanism to change the direction of the steered wheels. The steering ECU drives the electric motor in accordance with information input from the second control unit 160 or information input from the driving operator 80 to change the direction of the steered wheels.

[0065] [Specific part functions and operation control details] Below, the details of the function of the identification unit 153 and the contents of driving control based on the processing contents of the identification unit 153 will be described. Note that, below, it is assumed that the driver state determination unit 151 has determined that the driver is appropriately performing the tasks assigned according to the driving mode, and an example will be described in which the driving mode is determined based on the processing contents of the identification unit 153. Note that, when the driver state determination unit 151 has determined that the driver is not performing the tasks assigned according to the mode, the mode determination unit 150 determines to change to a mode according to the task being performed by the driver, or determines to perform control to stop automatic driving.

[0066] First, the identification unit 153 acquires information about the lane markings around the vehicle M recognized by the recognition unit 130, for example. Specifically, the recognition unit 130 analyzes an image (camera image) including the road on which the vehicle M is traveling, captured by the camera 10 (a first camera, or one or both cameras if a second camera is present), extracts edge points in the image that have a large difference in brightness from adjacent pixels, and recognizes landmarks that can identify lane markings or lane positions on the image plane by connecting the edge points. The recognition unit 130 may also extract image information from the image by feature extraction, image enhancement processing, or the like, and recognize the lane markings or landmarks by matching the extracted image information with a predefined pattern matching model or the like. The recognition unit 130 may also recognize type information, such as the type (solid line, dashed line) or color, for each lane marking based on the image analysis results, or may recognize the type of landmark. The recognition unit 130 may also recognize the positional relationship between the recognized lane markings and landmarks, and the positional relationship (relative position) between the lane markings and the vehicle M. The recognition unit 130 may also recognize characters displayed on road signs or characters drawn on roads.

[0067] The marking lines around vehicle M recognized by recognition unit 130 are, for example, the left and right marking lines closest to vehicle M and the left and right marking lines next closest to vehicle M after the left and right marking lines. Hereinafter, the closest marking line on the left side of vehicle M will be referred to as the "left side first marking line," and the next closest marking line (a marking line located further back from vehicle M than the left side first marking line) will be referred to as the "left side second marking line." In addition, the closest marking line on the right side of vehicle M will be referred to as the "right side first marking line," and the next closest marking line (a marking line located further back from vehicle M than the right side first marking line) will be referred to as the "right side second marking line."

[0068] The identification unit 153 refers to the first map information 54 based on the location information included in the reference information acquired by the first acquisition unit 152, and acquires information about the road on which the vehicle M is traveling. The road information is, for example, information about the number of lanes. The road information may also include information about the number of lanes on each side, information indicating whether or not there is two-way traffic, and information indicating whether or not the road on which the vehicle M is traveling is a deceleration zone.

[0069] Next, the identification unit 153 identifies the driving lane of the vehicle M based on information about the road on which the vehicle M is traveling, obtained from the first map information 54, and information about the type of lane markings around the vehicle M recognized by the recognition unit 130. Furthermore, for example, when a specific road marking that is difficult to recognize in a camera image is included, the identification unit 153 does not immediately determine that the driving lane of the vehicle M cannot be identified, but instead identifies the driving lane by adopting special rules (predetermined relaxed conditions). For example, when the types of lane marks recognized by the recognition unit 130 include a lane marking that may be a dashed deceleration line, the identification unit 153 identifies the driving lane of the vehicle M without using at least information about the line type of the lane marking.

[0070] More specifically, when it is determined that there is a possibility that the type of at least one of the lane markings on the left and right sides of the vehicle M is a deceleration dashed line, the identification unit 153 identifies the driving lane of the vehicle M, for example, using the lane markings excluding the lane marking determined to be a deceleration dashed line. Note that the deceleration dashed line portion may be used to identify the position of the vehicle M relative to the driving lane, assuming that some kind of line segment exists.

[0071] For example, when there is a possibility that one of the left and right dividing lines of vehicle M is a dashed deceleration line, the identification unit 153 identifies the driving lane of vehicle M based on the type of the other dividing line. Furthermore, when there is a possibility that one of the left and right dividing lines of vehicle M is a dashed deceleration line, the identification unit 153 may identify the driving lane of vehicle M based on a shoulder dividing line outside the road on which vehicle M is traveling. Furthermore, when there is a possibility that one or both of the left and right dividing lines of vehicle M is a dashed deceleration line, the identification unit 153 may identify the driving lane of vehicle M based on a solid line other than the road dividing lines on the left and right of vehicle M.

[0072] When performing the above-described identification, the identification unit 153 may refer to a determination table 182 stored in advance in the storage unit 180, and identify the driving lane of the vehicle M when the corresponding condition is satisfied.

[0073] FIG. 4 is a diagram showing an example of the contents of the determination table 182. The determination table 182 is information in which determination conditions and lane marking conditions are associated with determination results. In the example of the determination table 182 shown in FIG. 4, a condition ID is also included as identification information for identifying each determination condition. The determination start condition is a condition under which the identification unit 153 identifies the driving lane. "Always" includes the meaning of a predetermined period. Furthermore, deceleration display detection means that a deceleration display encouraging vehicle M to decelerate is displayed on the road on which vehicle M is traveling, or the recognition unit recognizes a road sign, signboard, or the like encouraging vehicle M to decelerate.

[0074] The lane marking conditions include conditions for the line types of two lane markings on each side of vehicle M. The "-" in the lane marking conditions in Figure 4 indicates that the lane markings can be of any type (including cases where the lane markings are not detected). The judgment result identifies the lane in which vehicle M is traveling when the judgment start conditions and the conditions in the lane marking conditions are met.

[0075] Below, we will explain the line types of the lane markings around vehicle M and how to identify the driving lane based on the line types, dividing them into several patterns.

[0076] <First line pattern> Fig. 5 is a diagram showing an example of a first line type pattern. In the example of Fig. 5, a vehicle M is traveling at a speed VM in the direction of extension of road RD1 (the X-axis direction in the figure). In the example of Fig. 5, the demarcation line RL11 corresponds to the left-side second demarcation line, the demarcation line RL12 corresponds to the left-side first demarcation line, the demarcation line RL13 corresponds to the right-side first demarcation line, and the demarcation line RL14 corresponds to the right-side second demarcation line. The same applies to the subsequent drawings.

[0077] In the example of Figure 5, it is assumed that the recognition unit 130 has not detected (recognized) the lane markings RL11 and RL14 (is undetected), but has recognized the lane markings RL12 and RL13 as solid lines. In this case, the identification unit 153 determines that the lane in which the vehicle M is traveling is an edge lane of a single lane (is a single lane) because the condition of condition ID "C001" shown in Figure 4 is satisfied. Similarly, the identification unit 153 determines that the lane in which the vehicle M is traveling is an edge lane of a single lane when the recognition unit 130 recognizes that at least one of the lane markings RL11 and RL is a shoulder instead of being undetected in the first line type pattern.

[0078] <Second line pattern> FIG. 6 is a diagram showing an example of a second line type pattern. In the example of FIG. 6, the recognition unit 130 recognizes that the lane marking RL11 is a road shoulder (or side strip), the lane marking RL12 is a solid line, and the lane marking RL13 is a dashed line. In the second line type pattern, the lane marking RL14 may be undetected or may be a detected lane. In this case, the identification unit 153 identifies the lane in which the vehicle M is traveling as the end lane (the leftmost lane) of a multi-lane road because the condition ID "C002" shown in FIG. 4 is satisfied.

[0079] <Third line pattern> FIG. 7 is a diagram illustrating an example of a third line type pattern. In the example of FIG. 7, the recognition unit 130 recognizes the character string "Slow Down" drawn on the road, and further recognizes that the lane marking RL11 is a road shoulder, the lane marking RL12 is a special line, and the lane marking RL13 is a dashed line. In the third line type pattern, the lane marking RL14 may be undetected or may be a detected lane marking. Special lines are lines that cannot be recognized as solid or dashed lines, such as dashed deceleration lines, deceleration marks, zebra stripes, and lines partially obscured by rubbing or dirt. In other words, special lines are lines that may be dashed deceleration lines. In this case, the identification unit 153 determines that the lane of the vehicle M is the end lane (the leftmost lane) of the dashed deceleration line section based on the types of the lane marks RL11 and RL13, excluding the lane marking RL12, because the condition ID "C003" shown in FIG. 4 is satisfied. Similarly, in the third line type pattern, when the demarcation line RL12 is a solid line or a dashed line and the demarcation line RL13 is a special line, the identification unit 153 may identify the driving lane of the vehicle M as the end lane of the deceleration dashed line section. Furthermore, instead of recognizing the character "slow down" drawn on the road, the identification unit 153 may recognize that the road is a deceleration section from the first map information 54. As described above, in the case of the third line type pattern, when there is a possibility that the type of one of the demarcation lines RL12, RL13 on the left and right of the vehicle M is a deceleration dashed line, the identification unit 153 identifies the driving lane of the vehicle M based on the type of the other demarcation line. In the case of the third line type pattern, the identification unit 153 identifies the driving lane of the vehicle M based on the position of the shoulder demarcation line (demarcation line RL11) outside the roadway on which the vehicle M is traveling, as shown in FIG. 7 .

[0080] <Fourth line pattern> FIG. 8 is a diagram illustrating an example of a fourth line type pattern. In the example of FIG. 8, the recognition unit 130 recognizes that the demarcation line RL11 is a solid line and the demarcation lines RL12 and RL13 are dashed lines. In the fourth line type pattern, the demarcation line RL14 may be undetected, or some demarcation line may be detected. In this case, the identification unit 153 determines that the lane of vehicle M is the second lane (the second lane from the left) of a road with three or more lanes on each side because the condition of condition ID "C004" shown in FIG. 4 is satisfied. In the fourth line type pattern, the identification unit 153 may refer to the first map information 54 based on the position information of vehicle M and determine that the condition of "C004" is satisfied if road RD1 is a road with three or more lanes on each side.

[0081] <5th line pattern> 9 is a diagram illustrating an example of a fifth line type pattern. In the example of FIG. 9, it is assumed that the recognition unit 130 recognizes that the demarcation lines RL11 and RL14 are solid lines and the demarcation lines RL12 and RL13 are dashed lines. In this case, the identification unit 153 identifies the traveling lane of the vehicle M as the second lane (the second lane from the left) of a road with two lanes on each side because the condition of the condition ID "C005" shown in FIG. 4 is satisfied. In addition to the above conditions, the identification unit 153 may refer to the first map information 54 based on the position information of the vehicle M and determine that the condition of "C005" is satisfied if the road RD1 is a road with two lanes on each side.

[0082] <6th line pattern> FIG. 10 is a diagram illustrating an example of a sixth line type pattern. In the example of FIG. 10, the recognition unit 130 recognizes the character string "Slow Down" drawn on the road, and further recognizes that the demarcation line RL11 is a solid line and the demarcation lines RL12 and RL13 are special lines. In the fourth line type pattern, the demarcation line RL14 may be undetected or may be detected as a demarcation line. In this case, the identification unit 153 determines that the lane of vehicle M is the second lane (the second lane from the left) of the deceleration dashed line section because the condition ID "C006" shown in FIG. 4 is satisfied. In addition to the above conditions, the identification unit 153 may refer to the first map information 54 based on the position information of vehicle M and determine that the condition "C006" is satisfied if road RD1 is a road in a deceleration section. Similarly, when at least one of the demarcation lines RL12 and RL13 is recognized as a dashed line in the sixth line type pattern, the identification unit 153 may identify the traveling lane of the vehicle M as the second lane (the second lane from the left) of the deceleration dashed line section. Also, instead of recognizing the character "slow down" drawn on the road, the identification unit 153 may recognize that the road is a deceleration section from the first map information 54. As described above, in the case of the sixth line type pattern, when there is a possibility that the type of the road demarcation lines of one or both of the left and right road demarcation lines of the vehicle M is a deceleration dashed line, the identification unit 153 identifies the traveling lane of the vehicle M based on the position of a solid line other than the left and right road demarcation lines of the vehicle M (RL11 in the example of FIG. 10 ).

[0083] In this way, by using the above-described first to sixth line type patterns to identify the driving lane of the vehicle M, it is possible to more accurately identify the driving lane even when, for example, there is a marking line that is difficult to recognize from a camera image. Therefore, even when there is a marking line that is difficult to recognize from a camera image, it is possible to continue the first driving mode without immediately switching from the first driving mode to the second driving mode.

[0084] In addition, if the above-mentioned first to sixth line type patterns (condition IDs "C001" to "C006") are not met, the identification unit 153 is unable to identify which lane the vehicle M is traveling in among one or more lanes included in road RD1.

[0085] When the identification unit 153 identifies the driving lane of the vehicle M, the mode change processing unit 154 executes the first driving mode, and performs driving control such as driving the vehicle M without deviating from the identified driving lane, following a vehicle ahead, or changing lanes to drive toward the destination. Furthermore, when the identification unit 153 does not identify the driving lane of the vehicle M, the mode change processing unit 154 executes the second driving mode. As a result, when it cannot be identified which lane on the road the vehicle M is traveling in, the autonomous driving is restricted, and the vehicle M can be driven more safely.

[0086] Furthermore, when the identification unit 153 does not identify the driving lane of the vehicle M, the mode change processing unit 154 may continue the driving mode currently being executed to drive the vehicle M. In this way, by maintaining the current driving mode without switching the driving mode in a situation where the driving lane of the vehicle M cannot be identified, more stable driving can be achieved.

[0087] Whether to switch to the second driving mode or continue the current driving mode (first driving mode or second driving mode) when the driving lane of vehicle M cannot be identified may be determined based on, for example, the surrounding conditions of vehicle M and road information obtained from the first map information 54. For example, if the recognition unit 130 determines that a predetermined number or more of other vehicles are present around vehicle M (within a predetermined distance), there is a possibility that lane markings cannot be recognized due to the influence of the other vehicles. Therefore, the mode change processing unit 154 may continue the current driving mode even if the driving lane of vehicle M cannot be identified if the number of other vehicles is less than the predetermined number around vehicle M, and may switch to the second driving mode if the number of other vehicles is less than the predetermined number. Furthermore, the mode change processing unit 154 may refer to the first map information 54 and acquire information about the roads around vehicle M based on the position information of vehicle M, and may continue the current driving mode if the number of lanes on the road within a predetermined distance from vehicle M does not increase or decrease or the curvature of the road is small (below a threshold), and may switch to the second driving mode if the number of lanes increases or decreases or the curvature of the road is large (greater than a threshold). This allows the vehicle M to operate in a more appropriate driving mode depending on the surrounding conditions.

[0088] Furthermore, if the identification unit 153 is unable to identify the driving lane of the vehicle M, the HMI control unit 170 may cause the HMI 30 to output information that the driving lane of the vehicle M has not been identified, thereby notifying the occupant. This allows the occupant to be notified of the situation of the vehicle M, and enables the occupant to monitor the periphery of the vehicle, switch to manual driving as necessary, or perform other driving control according to the situation.

[0089] [Processing flow] Next, a flow of processing executed by the automatic driving control device 100 of the embodiment will be described. Note that, of the processing executed by the automatic driving control device 100, the following mainly focuses on processing for identifying the driving lane of the vehicle M and processing for switching the driving mode based on the result of the identification processing. Furthermore, the processing of this flowchart may be executed repeatedly at a predetermined timing, for example.

[0090] Fig. 11 is a flowchart showing an example of the flow of a driving control process executed by the automatic driving control device 100. In the example of Fig. 11, the recognition unit 130 recognizes the surrounding conditions of the vehicle M (step S100). Next, the mode determination unit 150 causes the vehicle to travel in one of a plurality of driving modes set in advance based on the surrounding conditions, etc. (step S102).

[0091] Next, the mode determination unit 150 acquires the first map information, the recognition result by the recognition unit 130, and reference information for identifying the position of the vehicle M (step S104), and performs processing to identify the driving lane of the vehicle M based on the acquired information (step S106). For example, the identification unit 153 of the mode determination unit 150 determines whether at least one of the acquired information, the road information and the type of the lane marking, includes a specific road marking (step S108). If it is determined that a specific road marking is included, the identification unit 153 does not immediately determine that the driving lane of the vehicle M cannot be identified, but instead adopts a special rule (predetermined relaxation condition) to identify the driving lane without using at least the line type information of the lane marking. More specifically, the identification unit 153 identifies the driving lane using the lane markings excluding the specific road marking (step S110). Furthermore, if it is determined in the processing of step S108 that a specific road marking is not included, the identification unit 153 identifies the driving lane using the recognized lane markings without adopting the special rule (step S112).

[0092] Next, the mode change processing unit 154 determines whether the driving mode of the vehicle M is currently in the first driving mode (step S114). If it is determined that the first driving mode is currently being executed, the mode change processing unit 154 determines whether the identification unit 153 has identified the driving lane of the vehicle M (step S116). If it is determined that the driving lane has been identified, the mode change processing unit 154 continues the first driving mode based on the identified driving lane (step S118). If it is determined that the driving lane has not been identified, the mode change processing unit 154 performs control to switch from the first driving mode to the second driving mode (step S120). This ends the processing of this flowchart. Furthermore, if it is determined in the processing of step S114 that the first driving mode is not currently being executed, the processing of this flowchart ends.

[0093] [Variations] Below, several modified examples of this embodiment will be described. For example, in the above-described embodiment, when the identification unit 153 is unable to identify the driving lane of the vehicle M, the driving control unit may perform a lane change or the like to position the vehicle M in the leftmost or rightmost lane of the road on which the vehicle M is traveling. This makes it possible to identify the driving lane more accurately. Note that the driving control unit may cause the identification unit 153 to perform processing to identify the driving lane of the vehicle M after the vehicle M has been positioned in the leftmost or rightmost lane of the road. This makes it possible to identify the driving lane more quickly when the driving lane cannot be identified.

[0094] Furthermore, in the above-described embodiment, in addition to the first map information 54, if map information (second map information) with higher accuracy than the first map information 54 is held, and if a situation arises in which the second map information cannot be acquired, control may be performed to identify the driving lane using the first map information 54 or an image captured by the camera 10.

[0095] Here, the second map information is, for example, map information in which road information is defined for each lane over a shorter section than the first map information 54. The second map information may also include, for example, information on the center of lanes or information on lane boundaries. The second map information may also include road information, traffic regulation information, address information (address and postal code), facility information, telephone number information, etc. The second map information may be updated as needed by the communication device 20 communicating with another device. The second map information may be held in a storage device such as an HDD or flash memory of the MPU 60, or may be stored in the storage unit 180.

[0096] For example, when the MPU 60 or the storage unit 180 has second map information, the recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into multiple blocks and determines the recommended lane for each block by referring to the second map information. In this case, the recommended lane determination unit 61 may use the lane information included in the second map information to determine which lane from the left the vehicle M should travel in.

[0097] Here, for example, if the vehicle M is executing the first driving mode and the second map information is unavailable due to an abnormality in the data of the second map information or an abnormality during the update of the second map information, the mode determination unit 150 continues the first driving mode by identifying the driving lane of the vehicle M as described above based on information acquired from the first map information 54. This makes it possible to execute a driving mode with a high degree of automation even in a situation where the second map information is unavailable.

[0098] Furthermore, in the embodiment, when switching from the first driving mode to the second driving mode, the mode determination unit 150 may determine which mode to switch to from among multiple modes included in the second driving mode, depending on the driving state and driving environment of the vehicle M. The driving state is, for example, the state of the driver determined by the driver state determination unit 151. The driving environment is, for example, the shape of the road around the vehicle M, the number of lanes, the presence or absence of branching and merging, the number and relative positions of other vehicles in the vicinity, etc. For example, when the condition for switching from the first driving mode to the second driving mode is satisfied, the mode determination unit 150 determines to switch to mode C if the number of lanes when the driving lane of the vehicle M cannot be identified is three lanes, to switch to mode D if the number is four lanes, and to switch to mode E if the number is five lanes or more. This allows the vehicle M to travel in a more appropriate mode depending on the driving state and driving environment. Furthermore, in the above-described embodiment, the specific road dividing lines have been mainly described as being deceleration dashed lines, but instead of (or in addition to) the deceleration dashed lines, other lines equivalent to the above-described specific road dividing lines may be used, and the deceleration dashed lines may be appropriately interpreted as other lines depending on the traffic laws (Road Traffic Act) of the foreign country to which this embodiment applies.

[0099] According to the embodiment described above, the vehicle control device includes a recognition unit 130 that recognizes the surrounding conditions of the vehicle M, a driving control unit (action plan generation unit 140, second control unit 160) that controls one or both of the steering and the speed of the vehicle M based on the surrounding conditions recognized by the recognition unit 130, an acquisition unit (first acquisition unit) 152 that acquires map information including lane information around the vehicle M and reference information for identifying the position of the vehicle M, and an identification unit 153 that identifies the driving lane of the vehicle M based on information about the road on which the vehicle M is traveling, acquired from the map information based on the reference information, and the type of lane markings that demarcate one or more lanes around the vehicle M recognized by the recognition unit 130. The identification unit 153 identifies the driving lane of the vehicle M without using at least the type information for the specific road markings recognized by the recognition unit 130, thereby improving the identification rate of the vehicle's driving lane. Therefore, the first driving mode can be continued using information about the identified driving lane, or the degree of driving control can be changed in a more appropriate situation.

[0100] Specifically, according to the embodiment, even if a specific road dividing line that is difficult to recognize is included in the camera image, the vehicle M's traveling lane is not immediately determined to be unidentifiable. Instead, the traveling lane is identified by adopting special rules (predetermined relaxed conditions). This improves the identification rate of the traveling lane and reduces erroneous identification. For example, according to the embodiment, when the positions and types (line types) of multiple dividing lines obtained from the camera image meet predetermined determination conditions, the combination of types can identify which of multiple lanes (parallel lanes) the vehicle M is traveling in. Furthermore, according to the embodiment, the traveling lane can be more accurately identified, for example, by determining whether the vehicle M is traveling in an edge lane of the road based on the position of a solid line, or by identifying the traveling lane using shoulder dividing lines outside the traveling lane when both sides of the vehicle M are dashed lines.

[0101] Furthermore, according to the embodiment, the driving lane is identified based on the recognition results of two lane markings on each side of the vehicle M (first left lane marking, second left lane marking, first right lane marking, second right lane marking), so the identification rate of the driving lane can be improved without increasing the load of the recognition process. Furthermore, according to the embodiment, even if high-precision map information is not installed in the vehicle M, the position (driving lane) of the vehicle M can be identified more accurately using a navigation map (first map information 54) such as that used in the navigation device 50, and the first driving mode can be continued. This eliminates the need for frequent updating of map information or management by a map server, as is required with high-precision maps, thereby reducing operating costs.

[0102] The above-described embodiment can be expressed as follows. a storage device storing a program; a hardware processor; The hardware processor executes the program stored in the storage device, Recognizes the vehicle's surroundings, Execute driving control to control one or both of the steering and the speed of the vehicle based on the recognized surrounding conditions; acquiring map information including lane information around the vehicle and reference information for identifying the position of the vehicle; identifying a lane in which the vehicle is traveling based on information about the road on which the vehicle is traveling, obtained from the map information based on the reference information, and based on the type of demarcation line that demarcates each of the one or more lanes around the recognized vehicle; and identifying the travel lane of the vehicle for a specific road-dividing line among the recognized road-dividing lines without using at least the type information. Vehicle control device.

[0103] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0104] 1...vehicle system, 10...camera, 12...radar device, 14...LIDAR, 16...object recognition device, 20...communication device, 30...HMI, 32...turn signal switch, 40...vehicle sensor, 50...navigation device, 60...MPU, 70...driver monitor camera, 80...driving operator, 82...steering wheel, 84...steering grip sensor, 100...automatic driving control device, 120...first control unit, 130...recognition unit, 140...action plan generation unit, 150...mode determination unit, 151...driver state determination unit, 152...first acquisition unit, 153...identification unit, 154...mode change processing unit, 160...second control unit, 162...second acquisition unit, 164...speed control unit, 166...steering control unit, 170...HMI control unit, 180...memory unit, 200...driving force output device, 210...brake device, 220...steering device, M...vehicle

Claims

1. a recognition unit that recognizes the surrounding situation of the vehicle; a driving control unit that controls one or both of the steering and the speed of the vehicle based on the surrounding conditions recognized by the recognition unit; an acquisition unit that acquires map information including lane information around the vehicle and reference information for identifying the position of the vehicle; an identification unit that identifies a driving lane of the vehicle based on information about a road on which the vehicle is traveling, obtained from the map information based on the reference information, and based on a type of demarcation line that demarcates each of one or more lanes around the vehicle recognized by the recognition unit, the identification unit identifies a driving lane of the vehicle for a specific road-dividing line among the dividing lines recognized by the recognition unit without using at least the type information; the identification unit identifies the driving lane of the vehicle based on a shoulder marking outside the driving lane of the vehicle when there is a possibility that the type of one of the left and right markings of the vehicle is a dashed deceleration line; Vehicle control device.

2. a recognition unit that recognizes the surrounding situation of the vehicle; a driving control unit that controls one or both of the steering and the speed of the vehicle based on the surrounding conditions recognized by the recognition unit; an acquisition unit that acquires map information including lane information around the vehicle and reference information for identifying the position of the vehicle; an identification unit that identifies a driving lane of the vehicle based on information about a road on which the vehicle is traveling, obtained from the map information based on the reference information, and based on a type of demarcation line that demarcates each of one or more lanes around the vehicle recognized by the recognition unit, the identification unit identifies a driving lane of the vehicle for a specific road-dividing line among the dividing lines recognized by the recognition unit without using at least the type information; The operation control unit running the vehicle by executing one of a plurality of driving modes including at least a first driving mode and a second driving mode in which a task imposed on an occupant of the vehicle is heavier than that in the first driving mode; When the identification unit identifies the driving lane of the vehicle, the first driving mode is executed; When the identification unit does not identify the driving lane of the vehicle, the second driving mode is executed. Vehicle control device.

3. a recognition unit that recognizes the surrounding situation of the vehicle; a driving control unit that controls one or both of the steering and the speed of the vehicle based on the surrounding conditions recognized by the recognition unit; an acquisition unit that acquires map information including lane information around the vehicle and reference information for identifying the position of the vehicle; an identification unit that identifies a driving lane of the vehicle based on information about a road on which the vehicle is traveling, obtained from the map information based on the reference information, and based on a type of demarcation line that demarcates each of one or more lanes around the vehicle recognized by the recognition unit, the identification unit identifies a driving lane of the vehicle for a specific road-dividing line among the dividing lines recognized by the recognition unit without using at least the type information; the driving control unit runs the vehicle by executing one of a plurality of driving modes in which tasks are differently assigned to an occupant of the vehicle, and when the identification unit does not identify a driving lane of the vehicle, continues running the driving mode to run the vehicle. Vehicle control device.

4. The specific road dividing line is a dashed deceleration line on at least one of the left and right sides of the vehicle. The vehicle control device according to any one of claims 1 to 3.

5. the identification unit, when there is a possibility that one of the left and right lane markings of the vehicle is a type of a dashed deceleration line, identifies the driving lane of the vehicle based on the type of at least the other lane marking; The vehicle control device according to claim 2 or 3.

6. the identification unit identifies the driving lane of the vehicle based on solid lines other than the road dividing lines on the left and right of the vehicle when there is a possibility that the type of one or both of the road dividing lines on the left and right of the vehicle is a dashed deceleration line; The vehicle control device according to claim 2 or 3.

7. The computer Recognizes the vehicle's surroundings, Execute driving control to control one or both of the steering and the speed of the vehicle based on the recognized surrounding conditions; acquiring map information including lane information around the vehicle and reference information for identifying the position of the vehicle; identifying a lane in which the vehicle is traveling based on information about the road on which the vehicle is traveling, obtained from the map information based on the reference information, and based on the type of demarcation line that demarcates each of the one or more lanes around the recognized vehicle; identifying a driving lane of the vehicle for a specific road-dividing line among the recognized road-dividing lines without using at least the type information; When there is a possibility that one of the left and right lane markings of the vehicle is a type of a dashed deceleration line, the lane of travel of the vehicle is identified based on a shoulder lane marking outside the lane of travel of the vehicle. Vehicle control method.

8. The computer Recognizes the vehicle's surroundings, Execute driving control to control one or both of the steering and the speed of the vehicle based on the recognized surrounding conditions; acquiring map information including lane information around the vehicle and reference information for identifying the position of the vehicle; identifying a lane in which the vehicle is traveling based on information about the road on which the vehicle is traveling, obtained from the map information based on the reference information, and based on the type of demarcation line that demarcates each of the one or more lanes around the recognized vehicle; identifying a driving lane of the vehicle for a specific road-dividing line among the recognized road-dividing lines without using at least the type information; running the vehicle by executing one of a plurality of driving modes including at least a first driving mode and a second driving mode in which a task imposed on an occupant of the vehicle is heavier than that in the first driving mode; When the driving lane of the vehicle is identified, the first driving mode is executed; When the driving lane of the vehicle is not identified, the second driving mode is executed. Vehicle control method.

9. The computer Recognizes the vehicle's surroundings, Execute driving control to control one or both of the steering and the speed of the vehicle based on the recognized surrounding conditions; acquiring map information including lane information around the vehicle and reference information for identifying the position of the vehicle; identifying a lane in which the vehicle is traveling based on information about the road on which the vehicle is traveling, obtained from the map information based on the reference information, and based on the type of demarcation line that demarcates each of the one or more lanes around the recognized vehicle; identifying a driving lane of the vehicle for a specific road-dividing line among the recognized road-dividing lines without using at least the type information; running the vehicle by executing one of a plurality of driving modes in which tasks are assigned to the vehicle occupants differently, and, if the driving lane of the vehicle is not specified, continuing the running driving mode to run the vehicle; Vehicle control method.

10. On the computer, Recognize the vehicle's surroundings, Execute driving control to control one or both of the steering and the speed of the vehicle based on the recognized surrounding conditions; acquiring map information including lane information around the vehicle and reference information for identifying the position of the vehicle; Identifying the lane in which the vehicle is traveling based on information about the road on which the vehicle is traveling, which is acquired from the map information based on the reference information, and based on the type of demarcation line that demarcates each of the one or more lanes around the recognized vehicle; Among the recognized lane markings, a lane in which the vehicle is traveling is identified for a specific road marking without using at least the type information; When there is a possibility that one of the left and right lane markings of the vehicle is a dashed deceleration line, the lane of travel of the vehicle is identified based on a shoulder lane marking outside the lane of travel of the vehicle. program.

11. On the computer, Recognize the vehicle's surroundings, Execute driving control to control one or both of the steering and the speed of the vehicle based on the recognized surrounding conditions; acquiring map information including lane information around the vehicle and reference information for identifying the position of the vehicle; Identifying the lane in which the vehicle is traveling based on information about the road on which the vehicle is traveling, which is acquired from the map information based on the reference information, and based on the type of demarcation line that demarcates each of the one or more lanes around the recognized vehicle; Among the recognized lane markings, a lane in which the vehicle is traveling is identified for a specific road marking without using at least the type information; running the vehicle by executing one of a plurality of driving modes including at least a first driving mode and a second driving mode in which a task imposed on an occupant of the vehicle is heavier than that in the first driving mode; When the driving lane of the vehicle is identified, the first driving mode is executed; When the driving lane of the vehicle is not identified, the second driving mode is executed. program.

12. On the computer, Recognize the vehicle's surroundings, Execute driving control to control one or both of the steering and the speed of the vehicle based on the recognized surrounding conditions; acquiring map information including lane information around the vehicle and reference information for identifying the position of the vehicle; Identifying the lane in which the vehicle is traveling based on information about the road on which the vehicle is traveling, which is acquired from the map information based on the reference information, and based on the type of demarcation line that demarcates each of the one or more lanes around the recognized vehicle; Among the recognized lane markings, a lane in which the vehicle is traveling is identified for a specific road marking without using at least the type information; running the vehicle by executing one of a plurality of driving modes in which tasks are assigned to the vehicle occupants differently, and, if the driving lane of the vehicle is not specified, continuing the running driving mode to run the vehicle; program.

Images