Vehicle control device, vehicle control method and program

Abstract

To execute operation control with improved appropriateness.SOLUTION: A vehicle control device comprises: a first recognition section which recognizes a surrounding situation including a first lane marking partitioning a travel lane of an own vehicle and an adjacent vehicle based on output of a detection device; a second recognition section which recognizes a second lane marking partitioning a travel lane around the own vehicle from map information; an operation control section which executes operation control based on a recognition result; and a travel trajectory estimation section which estimates a travel trajectory of the adjacent vehicle. When the first lane marking deviates from the second lane marking, the vehicle control device: executes the operation control based on one of the first lane marking and the second partition line; estimates a direction in which the adjacent vehicle moves between an extension direction of the other of the first lane marking and the second lane marking and a longitudinal direction of a vehicle body of the adjacent vehicle; calculates time for the adjacent vehicle to deviate from the one of the first lane marking and the second lane marking when the adjacent vehicle moves along the estimated direction; and continues the operation control when the calculated time is equal to or larger than a threshold.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a program. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transportation systems that take into consideration vulnerable traffic participants have been gaining momentum. To achieve this, efforts are being focused on research and development into autonomous driving technology to further improve traffic safety and convenience. In this regard, a technology has been known that, when it is determined that there is a discrepancy between road dividing lines shown in camera images and road dividing lines shown in map information, controls the vehicle's driving mode based on the parallelism between the driving trajectories of other vehicles in the vicinity and the road dividing lines shown in camera images (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-148405 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional autonomous driving technology, there is still room for improvement in driving control that takes into account cases where the road dividing line shown in the camera image diverges from the road dividing line shown in the map information, changes in the direction of the road dividing line, or the position of other vehicles.

[0005] In order to solve the above-mentioned problems, one of the objects of the present application is to provide a vehicle control device, a vehicle control method, and a program that can execute more appropriate driving control in accordance with the road markings around the vehicle and the situation of other vehicles, thereby contributing to the development of a sustainable transportation system. [Means for solving the problem]

[0006] A vehicle control device, a vehicle control method, and a program according to the present invention employ the following configuration. (1): A vehicle control device according to one aspect of the present invention includes a first recognition unit that recognizes a surrounding situation including a first dividing line that divides the lane in which the host vehicle is traveling and an adjacent vehicle traveling in an adjacent lane adjacent to the lane in which the host vehicle is traveling, based on an output of a detection device that detects a surrounding situation of the host vehicle; a second recognition unit that recognizes a second dividing line that divides the lanes around the host vehicle from map information, based on position information of the host vehicle; a driving control unit that executes driving control to control at least the steering or speed of the host vehicle, based on the recognition results of the first recognition unit and the second recognition unit; and a driving trajectory estimation unit that estimates a driving trajectory of the adjacent vehicle, wherein the driving control unit A driving control device that, when a first dividing line and a second dividing line diverge, executes the driving control based on either the first dividing line or the second dividing line, and when the first dividing line and the second dividing line diverge, the driving trajectory estimation unit estimates that the direction in which the adjacent vehicle will move is a direction between the direction in which the other of the first dividing line and the second dividing line extends and the longitudinal direction of the adjacent vehicle's body, and when the adjacent vehicle moves along the estimated direction of movement, calculates the time it takes for the adjacent vehicle to deviate from either the first dividing line or the second dividing line, and the driving control unit continues the driving control if the time is greater than or equal to a threshold value.

[0007] (2): A vehicle control device according to another aspect of the present invention includes a first recognition unit that recognizes a surrounding situation including a first dividing line that divides the lane in which the host vehicle is traveling and an adjacent vehicle traveling in an adjacent lane adjacent to the lane in which the host vehicle is traveling, based on an output from a detection device that detects a surrounding situation of the host vehicle; a second recognition unit that recognizes a second dividing line that divides the lanes around the host vehicle from map information, based on position information of the host vehicle; a driving control unit that executes driving control to control at least the steering or speed of the host vehicle, based on the recognition results of the first recognition unit and the second recognition unit; and a driving trajectory estimation unit that estimates a driving trajectory of the adjacent vehicle, wherein the driving control unit When the first and second marking lines diverge, the driving control is executed based on either the first or second marking line; when the first and second marking lines diverge, the driving trajectory estimation unit estimates that the adjacent vehicle will move in a direction between the direction in which the other of the first and second marking lines extends and the longitudinal direction of the body of the adjacent vehicle; when the adjacent vehicle moves along the estimated direction of movement, the driving control unit calculates the time until the vehicle traveling under the driving control interferes with the adjacent vehicle; and the driving control unit continues the driving control if the time is greater than or equal to a threshold.

[0008] (3): In the above aspect (1) or (2), a judgment unit is further provided that judges that either the first dividing line or the second dividing line is an erroneous recognition when an obstacle is present on the lane in which the vehicle is traveling.

[0009] (4): In the above aspect (1) or (2), the device further includes a determination unit that determines that one of the first demarcation line and the second demarcation line is misrecognized when it is determined that the time is less than a first threshold value a first predetermined number of times or more, and that the other of the first demarcation line and the second demarcation line is misrecognized when the number of times that the time is less than a second threshold value that is greater than the first predetermined number of times or more.

[0010] (5): In the above aspect (1) or (2), the driving trajectory estimation unit estimates the direction in which the other of the first demarcation line and the second demarcation line extends and the longitudinal center position of the body of the adjacent vehicle as the direction in which the adjacent vehicle will travel.

[0011] (6): In the above aspect (1) or (2), when the driving trajectory estimation unit estimates that the adjacent vehicle will be driving in a direction between the direction in which the other of the first and second dividing lines extends and the longitudinal direction of the body of the adjacent vehicle, the driving trajectory estimation unit adjusts the direction between the first and second dividing lines depending on the amount of deviation between the first and second dividing lines.

[0012] (7): In the above aspect (1) or (2), when determining whether the first demarcation line and the second demarcation line diverge, the device further includes a judgment unit that determines whether the first demarcation line is misrecognized based on at least one of the amount of change in curvature of the first demarcation line and the angle between the first demarcation line and the second demarcation line, and makes the judgment of misrecognition if the direction of change in the amount of change in curvature and the direction of change in the angle are the same and the amount of change in curvature and the angle increase according to the distance from the vehicle.

[0013] (8) A vehicle control method according to one aspect of the present invention includes a computer that, based on the output of a detection device that detects the surrounding conditions of the host vehicle, recognizes the surrounding conditions, including a first dividing line that divides the lane in which the host vehicle is traveling and an adjacent vehicle traveling in an adjacent lane adjacent to the lane in which the host vehicle is traveling; recognizes a second dividing line that divides the lanes around the host vehicle from map information based on the position information of the host vehicle; performs driving control to control at least the steering or speed of the host vehicle based on the recognition result; estimates the traveling trajectory of the adjacent vehicle; and, when the first dividing line and the second dividing line diverge, performs the driving control based on either the first dividing line or the second dividing line; and, when the first dividing line and the second dividing line diverge, estimates the direction in which the adjacent vehicle will move to be between the direction in which the other of the first dividing line and the second dividing line extends and the longitudinal direction of the body of the adjacent vehicle. This is a vehicle control method that, when the adjacent vehicle moves along the estimated direction of movement, calculates the time it takes for the adjacent vehicle to deviate from either the first marking line or the second marking line, and continues the driving control if the time is greater than or equal to a threshold value.

[0014] (9) In another aspect of the present invention, a vehicle control method includes a computer that recognizes a surrounding situation including a first dividing line that divides the lane in which the host vehicle is traveling and an adjacent vehicle traveling in an adjacent lane adjacent to the lane in which the host vehicle is traveling, based on an output from a detection device that detects a surrounding situation of the host vehicle, recognizes a second dividing line that divides the lane in which the host vehicle is traveling from map information based on position information of the host vehicle, and performs driving control that controls at least the steering or speed of the host vehicle based on the recognition result, estimates a traveling trajectory of the adjacent vehicle, and calculates a driving trajectory of the first dividing line and the second dividing line. and when the first and second marking lines diverge, the driving control is executed based on either the first marking line or the second marking line, and when the first and second marking lines diverge, the direction in which the adjacent vehicle will move is estimated to be a direction between the direction in which the other of the first and second marking lines extends and the longitudinal direction of the body of the adjacent vehicle, and when the adjacent vehicle moves along the estimated direction of movement, the method calculates the time until the vehicle traveling under the driving control interferes with the adjacent vehicle, and continues the driving control if the time is equal to or greater than a threshold.

[0015] (10) A program according to one aspect of the present invention causes a computer to recognize a surrounding situation including a first dividing line that divides the lane in which the host vehicle is traveling and an adjacent vehicle traveling in an adjacent lane adjacent to the lane in which the host vehicle is traveling, based on an output from a detection device that detects the surrounding situation of the host vehicle; recognize a second dividing line that divides the lane in which the host vehicle is traveling from map information, based on position information of the host vehicle; execute driving control that controls at least the steering or speed of the host vehicle based on the recognition result; estimate a traveling trajectory of the adjacent vehicle; and determine whether the first dividing line and the second dividing line are aligned. The program executes the driving control based on either the first or second dividing line when they diverge, estimates the direction in which the adjacent vehicle will move between the direction in which the other of the first or second dividing line extends and the longitudinal direction of the adjacent vehicle's body when the adjacent vehicle moves along the estimated direction of movement, calculates the time it takes for the adjacent vehicle to deviate from either the first or second dividing line, and continues the driving control if the time is greater than or equal to a threshold.

[0016] (11): A program according to another aspect of the present invention causes a computer to recognize a surrounding situation including a first dividing line that divides the lane in which the host vehicle is traveling and an adjacent vehicle traveling in an adjacent lane adjacent to the lane in which the host vehicle is traveling, based on an output from a detection device that detects the surrounding situation of the host vehicle; recognize a second dividing line that divides the lane in which the host vehicle is traveling from map information based on position information of the host vehicle; and execute driving control that controls at least the steering or speed of the host vehicle based on the recognition result; estimate a traveling trajectory of the adjacent vehicle; and determine a relationship between the first dividing line and the second dividing line. When the first and second dividing lines diverge, the driving control is executed based on either the first or second dividing line; when the first and second dividing lines diverge, the direction in which the adjacent vehicle will move is estimated to be between the direction in which the other of the first and second dividing lines extends and the longitudinal direction of the adjacent vehicle's body; when the adjacent vehicle moves along the estimated direction of movement, the program calculates the time until the vehicle, traveling under the driving control, interferes with the adjacent vehicle; and continues the driving control if the time is greater than or equal to a threshold. [Effects of the Invention]

[0017] According to the above aspects (1) to (11), more appropriate driving control can be executed in accordance with the road dividing lines around the vehicle and the situation of other vehicles. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a configuration diagram of a vehicle system 1 including a vehicle control device according to an embodiment. [Figure 2] 2 is a functional configuration diagram of a first control unit 120 and a second control unit 160. FIG. [Figure 3] 1 is a diagram for explaining driving control of a vehicle M in a first scene. FIG. [Figure 4] 10A and 10B are diagrams for explaining deviation determination based on the amount of change in curvature when the lane is a curved road; [Figure 5]FIG. 10 is a diagram for explaining driving control of the host vehicle M in a second scene. [Figure 6] 4 is a flowchart showing an example of the flow of an operation control process in the first embodiment. [Figure 7] 10 is a flowchart showing an example of the flow of an operation control process in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0019] Hereinafter, with reference to the drawings, embodiments of a vehicle control device, a vehicle control method, and a program according to the present invention will be described. Hereinafter, an embodiment in which the vehicle control device is applied to an autonomous vehicle will be described as an example. Autonomous driving refers to automatically controlling one or both of the steering and speed of a vehicle to perform driving control. The above-mentioned driving control may include, for example, an adaptive cruise control system (ACC), a traffic jam pilot (TJP), a lane keeping assistance system (LKAS), an automated lane change (ALC), a collision mitigation brake system (CMBS), and the like. Furthermore, an autonomous vehicle may be manually controlled by a vehicle user (e.g., a passenger) (so-called manual driving). Although the following description will be given for a case in which left-hand traffic regulations apply, if right-hand traffic regulations apply, the terms left and right can be reversed.

[0020] [Overall configuration] 1 is a configuration diagram of a vehicle system 1 including a vehicle control device according to an embodiment. The vehicle (hereinafter referred to as host vehicle M) on which the vehicle system 1 is mounted is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination of these. The electric motor operates using power generated by a generator connected to the internal combustion engine, or discharged power from a battery (storage battery) such as a secondary battery or a fuel cell.

[0021] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) device 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, vehicle sensors 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driving operator 80, an automatic driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are connected to each other via multiplexed communication lines such as a CAN (Controller Area Network) communication line, serial communication lines, a wireless communication network, etc. Note that the configuration shown in FIG. 1 is merely an example, and some of the configuration may be omitted, or other configurations may be added. The combination of the camera 10, the radar device 12, the LIDAR 14, and the object recognition device 16 is an example of a "detection device DD." The HMI 30 is an example of an "output device." The automatic driving control device 100 is an example of a "vehicle control device."

[0022] The camera 10 is a digital camera that uses a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to any location of the host vehicle M in which the vehicle system 1 is installed. When capturing an image of the front, the camera 10 is attached to the top of the front windshield, the back of the rearview mirror, the front of the vehicle body, etc. When capturing an image of the rear, the camera 10 is attached to the top of the rear windshield, the back door, etc. When capturing an image of the side, the camera 10 is attached to a door mirror, etc. The camera 10 periodically and repeatedly captures images of the surroundings of the host vehicle M, for example. The camera 10 may be a stereo camera.

[0023] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by surrounding objects (reflected waves) to detect at least the position (distance and direction) of the objects. The radar device 12 is attached to any location on the vehicle M. The radar device 12 may detect the position and speed of the objects using an FM-CW (Frequency Modulated Continuous Wave) method.

[0024] The LIDAR 14 irradiates light around the vehicle M and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time between emitting and receiving the light. The irradiated light is, for example, a pulsed laser beam. The LIDAR 14 is attached to any location on the vehicle M.

[0025] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the camera 10, the radar device 12, and the LIDAR 14 to recognize the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition results to the automatic driving control device 100. Alternatively, the object recognition device 16 may output the detection results from the camera 10, the radar device 12, and the LIDAR 14 directly to the automatic driving control device 100. In that case, the object recognition device 16 may be omitted from the configuration of the vehicle system 1 (detection device DD).

[0026] The communication device 20 communicates with, for example, other vehicles in the vicinity of the vehicle M, terminal devices of users using the vehicle M, or various server devices, using networks such as a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), LAN (Local Area Network), WAN (Wide Area Network), or the Internet.

[0027] The HMI 30 outputs various information to the occupants of the vehicle M and accepts input operations by the occupants. The HMI 30 includes, for example, various display devices, speakers, buzzers, touch panels, switches, keys, microphones, and the like.

[0028] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the host vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the yaw rate (for example, the rotational angular velocity around a vertical axis passing through the center of gravity of the host vehicle M), and a direction sensor that detects the orientation of the host vehicle M. The vehicle sensor 40 may also be provided with a position sensor that detects the position of the vehicle. The position sensor is an example of a "position measurement unit." The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. The position sensor may also be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver 51 of the navigation device 50. The vehicle sensor 40 may derive the speed of the host vehicle M from the difference (i.e., distance) of the position information at a predetermined time in the position sensor. The results detected by the vehicle sensor 40 are output to the automatic driving control device 100.

[0029] The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as a hard disk drive (HDD) or a flash memory. The GNSS receiver 51 identifies the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be identified or supplemented by an inertial navigation system (INS) that uses the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, etc. The GNSS receiver 51 may be provided in the vehicle sensor 40. The navigation HMI 52 may share some or all of the components with the HMI 30 described above. The route determination unit 53 determines, for example, a route (hereinafter, a route on a map) from the position of the vehicle M identified by the GNSS receiver 51 (or an arbitrary input position) to a destination input by the occupant using the navigation HMI 52, with reference to the first map information 54. The first map information 54 is information in which road shapes are expressed by, for example, links indicating roads and nodes connected by the links. The first map information 54 may also include POI (Point Of Interest) information and the like. The route on the map is output to the MPU 60. The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may transmit the current position and destination to a navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server. The navigation device 50 outputs the determined route on the map to the MPU 60.

[0030] The MPU 60 includes, for example, a recommended lane determination unit 61, and stores second map information 62 in a storage device such as an HDD or flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, by dividing it into 100 m intervals in the vehicle travel direction), and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 determines, for example, which lane from the left the vehicle should travel in. When there is a branch point on the route on the map, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel on a reasonable route to the branch point.

[0031] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, the number of lanes, the type and shape of road dividing lines (hereinafter referred to as dividing lines), information on the center of lanes, and road boundary information. The second map information 62 may also include information on whether the road boundary includes a structure that prevents a vehicle from passing (including crossing or coming into contact with). Examples of such structures include guardrails, curbs, medians, and fences. The term "impassable" may also include the presence of a low level step that allows passage if unusual vehicle vibrations are tolerated. The second map information 62 may also include road shape information, traffic regulation information, address information (address and postal code), facility information, parking information, telephone number information, and the like. The road shape information includes, for example, the curvature (which may be referred to as the radius of curvature; the same applies below), width, and gradient of the road. The second map information 62 may be updated as needed by the communication device 20 communicating with an external device. The first map information 54 and the second map information 62 may be provided as an integrated piece of map information.

[0032] The driving operators 80 include, for example, a steering wheel, an accelerator pedal, and a brake pedal. The driving operators 80 may also include a shift lever, a variable steering wheel, a joystick, or other operators. Each operator of the driving operators 80 is equipped with an operation detection unit that detects, for example, the amount of operation of the operator by the occupant or whether or not the operator is operated. The operation detection unit detects, for example, the steering angle and steering torque of the steering wheel, the amount of depression of the accelerator pedal and the brake pedal, etc. The operation detection unit then outputs the detection results to the automatic driving control device 100 or one or both of the driving force output device 200, the brake device 210, and the steering device 220.

[0033] The automatic driving control device 100 executes various types of driving control associated with automatic driving for the host vehicle M. The automatic driving control device 100 includes, for example, a first control unit 120, a second control unit 160, an HMI control unit 180, and a storage unit 190. The first control unit 120, the second control unit 160, and the HMI control unit 180 are each realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Furthermore, some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or an SOC (System On Chip), or may be realized by a combination of software and hardware. The above-mentioned program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as an HDD or flash memory of the automatic driving control device 100, or may be stored in a removable storage medium such as a DVD, CD-ROM, or memory card, and installed in the storage device of the automatic driving control device 100 by inserting the storage medium (non-transitory storage medium) into a drive device, card slot, etc.

[0034] The storage unit 190 may be realized by the various storage devices described above, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a ROM (Read Only Memory), or a RAM (Random Access Memory). The storage unit 190 stores, for example, various types of information, programs, and the like in the embodiments. The storage unit 190 may also store map information (for example, the first map information 54 and the second map information 62).

[0035] FIG. 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 140. The first control unit 120, for example, implements functions based on AI (Artificial Intelligence) and functions based on a predefined model in parallel. For example, the function of "recognizing intersections" may be implemented by concurrently executing intersection recognition using deep learning or the like and recognition based on predefined conditions (such as the presence of traffic lights and road markings that can be pattern matched), and then scoring and comprehensively evaluating both. This ensures the reliability of autonomous driving. The first control unit 120 also executes control related to autonomous driving of the host vehicle M based on instructions from, for example, the MPU 60, the HMI control unit 180, or the like.

[0036] The recognition unit 130 recognizes the surrounding situation of the host vehicle M based on the recognition results of the detection device DD (information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16). For example, the recognition unit 130 recognizes the status of objects present around the host vehicle M (within a predetermined distance), such as their position, speed, and acceleration. The objects include, for example, other vehicles (surrounding vehicles), traffic participants (pedestrians, bicycles, etc.) traveling on the road, road structures, and other obstacles present in the vicinity. The road structures include, for example, road signs, traffic signals, railroad crossings, curbs, medians, guardrails, fences, etc. The position of the object is recognized as a position on an absolute coordinate system with a representative point of the host vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of the object may be represented by a representative point such as the center of gravity or a corner of the object, or by a represented area. The "state" of an object may include, for example, if the object is a moving object such as another vehicle, the acceleration or jerk of the moving object, or the "behavioral state" (for example, whether the other vehicle is changing lanes or is about to change lanes).

[0037] The recognition unit 130 includes, for example, a first recognition unit 132 and a second recognition unit 134. The functions of these units will be described in detail later.

[0038] The behavior plan generation unit 140 generates a behavior plan for driving the host vehicle M by autonomous driving based on the recognition results of the recognition unit 130, etc. For example, the behavior plan generation unit 140 generates a target trajectory for the host vehicle M to travel automatically (without driver operation) in the future, so that the host vehicle M can respond to the surrounding conditions of the host vehicle M, while essentially traveling in the recommended lane determined by the recommended lane determination unit 61, based on the recognition results by the recognition unit 130 and the surrounding road shapes based on the current position of the host vehicle M acquired from map information, etc. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequential arrangement of points (trajectory points) to be reached by the host vehicle M. The trajectory points are points to be reached by the host vehicle M at predetermined travel distances (e.g., on the order of several meters) along the road, and separately, target speeds and target accelerations for predetermined sampling times (e.g., on the order of a few tenths of a second) are generated as part of the target trajectory. Alternatively, the trajectory points may be positions to be reached by the host vehicle M at the sampling times for each predetermined sampling time. In this case, the target speed and target acceleration information are expressed as the interval between trajectory points.

[0039] The action plan generation unit 140 may set an autonomous driving event when generating the target trajectory. Examples of the event include a constant speed driving event in which the host vehicle M drives in the same lane at a constant speed, a following driving event in which the host vehicle M follows another vehicle that is within a predetermined distance (for example, within 100 m) ahead of the host vehicle M and is closest to the host vehicle M, a lane change event in which the host vehicle M changes lanes from the host vehicle's own lane to an adjacent lane, a branching event in which the host vehicle M branches off into a lane on the destination side at a road branching point, a merging event in which the host vehicle M merges into a main lane at a merging point, a takeover event in which the autonomous driving is terminated and the host vehicle M switches to manual driving, and so on. Examples of the event may also include an overtaking event in which the host vehicle M temporarily changes lanes to an adjacent lane, overtakes a leading vehicle in the adjacent lane, and then changes lanes back to the original lane, and an avoidance event in which the host vehicle M performs at least one of braking and steering to avoid an obstacle ahead of the host vehicle M.

[0040] Furthermore, the behavior plan generation unit 140 may change an event already determined for the current section to another event or set a new event for the current section, depending on the surrounding conditions of the host vehicle M recognized while the host vehicle M is traveling. Furthermore, the behavior plan generation unit 140 may change an event already set for the current section to another event or set a new event for the current section, depending on the operation of the occupant on the HMI 30. The behavior plan generation unit 140 generates a target trajectory according to the set event.

[0041] The behavior plan generating unit 140 also includes, for example, a determining unit 142 and an execution control unit 144. The functions of these units will be described in detail later.

[0042] The second control unit 160 controls the traveling driving force output device 200, the braking device 210, and the steering device 220 so that the host vehicle M passes through the target trajectory generated by the action plan generation unit 140 at the scheduled time.

[0043] The second control unit 160 includes, for example, a target trajectory acquisition unit 162, a speed control unit 164, and a steering control unit 166. The target trajectory acquisition unit 162 acquires information on the target trajectory (trajectory points) generated by the action plan generation unit 140 and stores it in a memory (not shown). The speed control unit 164 controls the driving force output device 200 or the brake device 210 based on a speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized by, for example, a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road ahead of the host vehicle M and feedback control based on the deviation from the target trajectory.

[0044] Returning to FIG. 1 , the HMI control unit 180 notifies the occupant of predetermined information via the HMI 30. The predetermined information includes, for example, information related to the traveling of the vehicle M, such as information related to the state of the vehicle M and information related to driving control. The information related to the state of the vehicle M includes, for example, the speed of the vehicle M, engine speed, and shift position. The information related to driving control includes, for example, whether or not driving control is being performed by autonomous driving, information inquiring about whether or not to start autonomous driving, information related to the driving control status by autonomous driving, information related to the automation level, and information prompting the occupant to drive when switching from autonomous driving to manual driving. The predetermined information may also include information unrelated to the traveling of the vehicle M, such as television programs and content (e.g., movies) stored on a storage medium such as a DVD. The predetermined information may also include, for example, information related to the current location and destination during autonomous driving, and the remaining amount of fuel in the vehicle M. The HMI control unit 180 may output the information received by the HMI 30 to the communication device 20, the navigation device 50, the first control unit 120, etc.

[0045] Furthermore, the HMI control unit 180 may cause the HMI 30 to output inquiry information for the occupant, processing results by the first control unit 120 and the second control unit 160, etc. Furthermore, the HMI control unit 180 may transmit various pieces of information to be output by the HMI 30 to a terminal device used by the user of the vehicle M via the communication device 20.

[0046] Traveling drive force output device 200 outputs a traveling drive force (torque) to the drive wheels for the vehicle to travel. Traveling drive force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, etc., and an ECU (Electronic Control Unit) that controls these. The ECU controls the above components according to information input from second control unit 160 or information input from the accelerator pedal of driving operator 80.

[0047] The braking device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or information input from the brake pedal of the driving operation device 80, so that a braking torque corresponding to the braking operation is output to each wheel. The braking device 210 may include a backup mechanism that transmits hydraulic pressure generated by operation of the brake pedal to the cylinder via a master cylinder. Note that the braking device 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic braking device that controls an actuator according to information input from the second control unit 160 to transmit hydraulic pressure from the master cylinder to the cylinder.

[0048] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor changes the direction of the steered wheels by, for example, applying a force to a rack and pinion mechanism. The steering ECU drives the electric motor to change the direction of the steered wheels in accordance with information input from the second control unit 160 or information input from the steering wheel of the driving operator 80.

[0049] [Recognition and Action Plan Generation] Next, the functions of the recognition unit 130 (first recognition unit 132, second recognition unit 134) and the action plan generation unit 140 (determination unit 142, execution control unit 144) will be described in detail. Note that, below, the contents of the driving control (travel control) of the host vehicle M in the embodiment will be mainly explained by dividing it into several scenes.

[0050] [Scene 1] FIG. 3 is a diagram illustrating driving control of the host vehicle M in a first scenario. The example in FIG. 3 shows lane markings CL1-CL3 recognized by the detection device DD and lane markings ML1-ML3 obtained from map information (e.g., second map information 62) based on the position information of the host vehicle M. In the map information, lane L1 is defined by lane markings ML1 and ML2, and lane L2 is defined by lane markings ML2 and ML3. Lanes L1 and L2 are lanes along which a vehicle can travel in the same direction (the X-axis direction in the figure). In the example in FIG. 3, lane markings CL1-CL3 are an example of a "first lane marking," and lane markings ML1-ML3 are an example of a "second lane marking." Also, in FIG. 3, the host vehicle M is traveling on lane L1 at a speed VM, and another vehicle m1 is traveling on lane L2, an adjacent lane to lane L1, at a speed Vm1. The other vehicle m1 is an adjacent vehicle (a vehicle traveling parallel to the host vehicle M). An adjacent vehicle is, for example, a vehicle traveling in an adjacent lane adjacent to the lane in which the host vehicle is traveling. Furthermore, the adjacent vehicle may be a vehicle that is present within a predetermined distance from the host vehicle M. In the example of FIG. 3, the host vehicle M is assumed to be performing predetermined driving control (e.g., LKAS) based on the surrounding conditions, instructions from the occupants, etc.

[0051] The first recognition unit 132 recognizes the surrounding conditions of the host vehicle M based on the output of the detection device DD, which detects the surrounding conditions (external world) of the host vehicle M. For example, the first recognition unit 132 recognizes left and right lane markings CL1 and CL2 that demarcate the lane (lane L1) of the host vehicle M based on an image captured by the camera 10 (hereinafter referred to as a camera image). The first recognition unit 132 may also recognize a lane marking CL3 that demarcates an adjacent lane (lane L2) adjacent to the driving lane. Hereinafter, the lane markings CL1 to CL3 may be referred to as "camera lane marks CL1 to CL3." For example, the first recognition unit 132 analyzes the camera image, extracts edge points in the image that have a large difference in brightness from adjacent pixels, and recognizes the camera lane marks CL1 to CL3 on the image plane by connecting the edge points. The first recognition unit 132 also converts the positions of the camera lane markings CL1-CL3 into a vehicle coordinate system (e.g., the XY plane coordinates in FIG. 3 ) based on the position of the representative point of the vehicle M. The first recognition unit 132 may also recognize, for example, the curvature of the camera lane markings CL1-CL3. The first recognition unit 132 may also recognize the amount of curvature change of the camera lane markings CL1-CL3. The amount of curvature change is, for example, the time rate of change in the curvature of the camera lane markings CL1-CL3 recognized by the camera 10 at a distance x [m] forward as viewed from the vehicle M. The first recognition unit 132 may also recognize the curvature or amount of curvature change of the lane markings CL1-CL3 by averaging the curvatures or amounts of curvature change of the camera lane markings CL1-CL3. The camera lane markings CL1-CL3 may be recognized or corrected based on the output of a detection device other than the camera 10 (e.g., the radar device 12 or the LIDAR 14).

[0052] The first recognition unit 132 also recognizes other vehicles present in the vicinity of the host vehicle M. In the example of FIG. 3, the first recognition unit 132 recognizes the other vehicle m1 traveling parallel to the host vehicle M in an adjacent lane. The first recognition unit 132 also recognizes the position (relative position with respect to the host vehicle M) and speed (relative speed with respect to the host vehicle M) of the other vehicle m1, as well as the traveling lane, body orientation, and traveling direction of the other vehicle m1. The first recognition unit 132 also recognizes the traveling position of the other vehicle m1 on each traveling lane. The first recognition unit 132 may also recognize traveling position information of the other vehicle m1. The traveling position information is, for example, a traveling trajectory based on the position of each representative point of the other vehicle m1 while traveling at a predetermined time.

[0053] The second recognition unit 134 recognizes lane markings around the host vehicle M from map information based on the position of the host vehicle M detected by, for example, the vehicle sensor 40 or the GNSS receiver 51. For example, the second recognition unit 134 refers to map information based on the position information of the host vehicle M and recognizes lane markings ML1 to ML3 that exist in the traveling direction of the host vehicle M or in directions in which the host vehicle M can travel. Hereinafter, the lane markings ML1 to ML3 may be referred to as "map lane markings ML1 to ML3."

[0054] Furthermore, the second recognition unit 134 may recognize map division lines ML1 and ML2, among the recognized map division lines ML1 to ML3, as division lines that divide the driving lane of the host vehicle M. Furthermore, the second recognition unit 134 recognizes the curvature or curvature change amount of each of the map division lines ML1 to ML3 from the second map information 62. Furthermore, the second recognition unit 134 may average the curvature or curvature change amount of each of the map division lines ML1 to ML3 to recognize the curvature or curvature change amount of the lane divided by the map division lines.

[0055] The determination unit 142 determines whether the camera lane lines CL1-CL3 recognized by the first recognition unit 132 deviate from the map lane lines ML1-ML3 recognized by the second recognition unit 134. For example, the determination unit 142 derives the degree of deviation between the lane lines CL1 and ML1 located closest to the left of the host vehicle M, the degree of deviation between the lane lines CL2 and ML2 located closest to the right of the host vehicle M, and the degree of deviation between the lane lines CL3 and ML3 on the adjacent lane side. If the derived degree of deviation is equal to or greater than a threshold, the determination unit 142 determines that the camera lane lines and the map lane lines deviate, and if the derived degree of deviation is less than the threshold, the determination unit 142 determines that there is no deviation. The determination of whether there is a deviation is repeatedly performed at a predetermined timing or cycle.

[0056] For example, the determination unit 142 superimposes the camera lane lines CL1, CL2, and CL3 and the map lane lines ML1, ML2, and ML3 on the plane of the vehicle coordinate system (XY plane) based on the position of the representative point of the vehicle M. When determining the lane lines to be compared (lane lines CL1 and ML1, lane lines CL2 and ML2, and lane lines CL3 and ML3), the determination unit 142 determines that the lane lines diverge if the degree of deviation of each lane line is equal to or greater than a threshold, and determines that the lane lines do not diverge if the degree of deviation is less than the threshold. The degree of deviation is, for example, the amount of deviation in the lateral position (e.g., the Y-axis direction in the figure). In the example of Figure 3, the deviation determination may be performed using the average value of the lateral position deviation D1 between the marking lines CL1 and ML1, the lateral position deviation D2 between the marking lines CL2 and ML2, and the lateral position deviation D3 between the marking lines CL3 and ML3, or the deviation determination may be performed using the maximum or minimum value of the deviations D1, D2, and D3.

[0057] Furthermore, the degree of deviation may be, for example, the degree (magnitude) of the angle formed by the two marking lines being compared, instead of (or in addition to) the above-mentioned lateral position deviation. In the example of Fig. 3, the average value of the angle θ1 formed by marking lines CL1 and ML1, the angle θ2 formed by marking lines CL2 and ML2, and the angle θ3 formed by marking lines CL3 and ML3 may be used, or the maximum or minimum value of the angles θ1, θ2, and θ3 may be used.

[0058] Furthermore, the degree of deviation may be the degree (magnitude) of difference in curvature change of the lane markings instead of (or in addition to) the above-mentioned lateral position deviation or angle formed by the lane markings. The curvature change is mainly used when the lane is a curved road. Figure 4 is a diagram for explaining deviation determination based on the curvature change when the lane is a curved road. The example of Figure 4 differs from the example of Figure 3 in that lanes L1 and L2 are curved roads and the camera lane markings and map lane markings each have a predetermined curvature.

[0059] In the example of FIG. 4, the first recognition unit 132 recognizes the amount of curvature change of each of the camera-based lane markings CL1-CL3 based on the output of the detection device DD. The second recognition unit 134 recognizes the amount of curvature change of each of the map-based lane markings ML1-ML3 based on map information. The determination unit 142 may use the average value of the difference in curvature change between lane markings CL1 and ML1, the difference in curvature change between lane markings CL2 and ML2, and the difference in curvature change between lane markings CL3 and ML3, or may use the maximum or minimum of these differences. The determination unit 142 may also use the difference between the average value of the curvature change of lane markings CL1-CL3 and the average value of the curvature change of lane markings ML1-ML3. The determination unit 142 may also use the difference between the curvature change of the lane markings (lanes L1, L2) recognized from the camera image and the curvature change of the lane markings recognized from the map information.

[0060] For example, when determining whether a camera lane marking deviates from a map lane marking, the determination unit 142 may determine whether the camera lane marking is erroneously recognized based on one or both of the curvature change amount of the camera lane marking detected by the recognition unit 130 and the angle between the camera lane marking and the map lane marking. In this case, the determination unit 142 determines whether the camera lane marking is erroneously recognized when, for example, the direction of change in the curvature change amount and the direction of change in the angle are the same and the curvature change amount and the angle increase with the distance from the vehicle M. In the embodiment, for example, even if the curvature change amount is equal to or greater than a first predetermined value and the angle is equal to or greater than a second predetermined value, if at least one of them is decreasing, the determination unit 142 does not determine that the camera lane marking is erroneously recognized, because erroneous recognition due to the influence of a curved road or the like may have converged. On the other hand, if both the curvature change amount and the angle increase in the same direction and reach or exceed a predetermined value, the determination unit 142 determines that the camera lane marking is erroneously recognized. This means that if both the curvature change and angle are increasing, it is judged to be an erroneous recognition, and cases where they are decreasing and tending to converge can be excluded, making it possible to more accurately determine erroneous recognition of camera dividing lines when driving through lane change sections such as curved roads.

[0061] The execution control unit 144 determines driving control for the host vehicle M based on the determination result by the determination unit 142, and executes the determined driving control. "Determining driving control" may include, for example, determining the content (type) of driving control and determining whether to execute (suppress) driving control. "Executing driving control" may include, for example, switching and executing the content of driving control, as well as continuing driving control that is already being executed. "Suppressing driving control" may include not only not executing driving control, but also lowering the automation level of driving control and terminating driving control. Furthermore, the driving control executed by the execution control unit 144 may include ACC, TJP, LKAS, ALC, CMBS, etc., and may also include various other driving controls for avoiding contact with surrounding vehicles. The execution control unit 144 generates a target trajectory for executing driving control and outputs the generated target trajectory to the second control unit 160.

[0062] Here, in the first scenario, the driving control executed by the execution control unit 144 includes at least a first driving control and a second driving control. The first driving control is, for example, driving control that executes at least steering control of the steering or speed of the host vehicle M based on a lane marking recognized by the first recognition unit 132 or the second recognition unit 134 (e.g., a lane marking where the camera lane marking and the map lane marking coincide (are not separated)). For example, the first driving control is driving control that causes the host vehicle M to travel so that a representative point of the host vehicle M passes through the center of a lane marked by lane markings. The second driving control is, for example, driving control that executes at least steering control of the steering or speed of the host vehicle M based on the camera lane marking recognized by the first recognition unit 132 and the traveling position information of another vehicle. For example, the second driving control is driving control that causes the host vehicle M to travel so that a representative point of the host vehicle M travels on a trajectory that follows the traveling trajectory of the other vehicle m1.

[0063] Furthermore, the driving control may include a third driving control in which at least steering control of the vehicle M is performed, either steering or speed, while giving priority to camera lane lines over map lane lines, and a fourth driving control in which at least steering control of the vehicle M is performed, either steering or speed, while giving priority to map lane lines over camera lane lines. Prioritizing camera lane lines over map lane lines means, for example, that processing is basically performed based on the camera lane lines, but temporarily switches to processing based on the map lane lines when, for example, the recognition accuracy of the camera lane lines falls below a threshold or they become unrecognizable. Prioritizing map lane lines over camera lane lines means, for example, that processing is basically performed based on the map lane lines, but temporarily switches to processing based on the camera lane lines when, for example, the map lane lines cannot be identified. The third driving control and the fourth driving control are driving controls performed, for example, when the camera lane lines and the map lane lines do not match (are separated).

[0064] Furthermore, the driving control may include multiple driving control with different automation levels (an example of the degree of automation). The automation levels may include, for example, a first level, a second level with a lower degree of automation of driving control than the first level, and a third level with a lower degree of automation of driving control than the second level. The automation levels may also include a fourth level (an example of a fourth control degree) with a lower degree of automation of driving control than the third level. Here, the automation level may be a level determined by standardized information, laws and regulations, or an index value set independently of such. Therefore, the types, contents, and number of automation levels are not limited to the following examples. A low degree of automation of driving control means, for example, a low automation rate in driving control and a large (heavy) task imposed on the driver. A low degree of automation of driving control means a low degree of control of the steering or acceleration / deceleration of the host vehicle M by the automatic driving control device 100 (a high degree of need for the driver to intervene in steering or acceleration / deceleration operations). Tasks assigned to the driver include, for example, monitoring the surroundings of the vehicle M and operating driving controls. Operating driving controls includes, for example, the driver gripping the steering wheel (hereinafter referred to as a hands-on state). Tasks assigned to the driver include, for example, tasks for the occupant (driver tasks) required to maintain the autonomous driving of the vehicle M. Therefore, if the occupant is unable to perform the assigned tasks, the automation level will be reduced. For example, the first level of driving control may include driving controls such as ACC, ALC, LKAS, and TJP. Furthermore, the second or third level of driving control may include driving controls such as ACC, ALC, and LKAS. The fourth level of driving control may include manual driving. Furthermore, the fourth level of driving control may include, for example, driving controls such as ACC. Of the first to fourth levels, the first level has the highest degree of automation of driving control, and the fourth level has the lowest degree of automation of driving control.

[0065] Furthermore, at the first level, no tasks are assigned to the occupant (the driver has the lightest tasks). At the second level, the task assigned to the occupant is, for example, monitoring the surroundings (particularly ahead) of the vehicle M. At the third level, the task assigned to the occupant includes, for example, monitoring the surroundings of the vehicle M as well as being in a hands-on state. At the fourth level, the task assigned to the occupant (e.g., the driver) is, for example, monitoring the surroundings of the vehicle M and being in a hands-on state, as well as operating the driving operator 80 to control the steering and speed of the vehicle M. In other words, at the fourth level, the occupant is ready to immediately take over driving, and the driver has the most severe tasks. The content of driving control and the tasks assigned to the occupant at each automation level are not limited to the examples described above. The automatic driving control device 100 executes driving control at one of the first to fourth levels based on the surrounding conditions of the vehicle M and the tasks being performed by the occupant.

[0066] For example, the execution control unit 144 executes the first driving control when the determination unit 142 determines that the camera lane markings and the map lane markings are not separated, and executes the second driving control when the determination unit 142 determines that the camera lane markings and the map lane markings are separated. In addition, the execution control unit 144 performs control such as terminating the driving control of the host vehicle M and switching to manual driving by the occupant.

[0067] The execution control unit 144 includes, for example, a traveling trajectory estimation unit 144A and a traveling control unit 144B. The traveling control unit 144B and the second control unit 160 are an example of a "driving control unit." The traveling trajectory estimation unit 144A estimates the future traveling trajectory of the other vehicle m1, which is an adjacent vehicle. For example, when it is determined that the camera lane markings and the map lane markings are separated, the traveling trajectory estimation unit 144A estimates that the other vehicle m1 will move in a direction between the extension direction of the other of the camera lane markings and the map lane markings (the map lane markings in the example of FIG. 3) and the longitudinal direction of the body of the other vehicle m1 (which may also be rephrased as the traveling direction of the other vehicle m1 or the extension direction of the traveling trajectory). Specifically, in the example of Figure 3, the driving trajectory estimation unit 144A estimates the future driving trajectory of the other vehicle m1 by setting the direction A3 between the direction A1 in which the map division lines ML1 to ML3 extend and the longitudinal direction A2 of the body of the other vehicle m1 as the direction in which the other vehicle m1 will move.

[0068] For example, the traveling trajectory estimation unit 144A sets direction A3 to the center position (intermediate position) between direction A1 and direction A2. By setting direction A3 to the center position, the current driving control can be continued without immediately determining that the camera lane markings or the map lane markings are incorrect. Also, if one of them is actually incorrect, the decision is not delayed too long.

[0069] The traveling trajectory estimation unit 144A may also set direction A3 based on the amount of deviation (degree of divergence) between the road lane markings and the map lane markings. For example, it estimates that direction A3, which is shifted by a predetermined angle toward direction A1 or direction A2, is the direction in which the other vehicle m1 is traveling, based on the amount of lateral deviation, the angle formed by the camera lane markings and the map lane markings, and the difference in the amount of curvature change between the two lane markings. In this way, the intermediate direction position is adjusted to a position other than the center position based on the amount of deviation, making it possible to strike a balance between continuing control in accordance with the surrounding conditions and determining that the lane markings are misrecognized.

[0070] Furthermore, when the other vehicle m1 moves along the estimated movement direction A3, the traveling trajectory estimation unit 144A calculates the time to line crossing (TTLC) for the other vehicle m1 to deviate from either a camera lane line or a map lane line (for example, camera lane line CL2). The other vehicle m1 deviating from a lane line may mean that the reference position (for example, the center, center of gravity, or end) of the other vehicle m1 crosses (passes) the lane line, or that the entire other vehicle m1 (the entire vehicle body) crosses the lane line. The lane departure time TTLC is calculated, for example, by dividing the distance to the lane line CL2 by the speed Vm1 of the other vehicle m1.

[0071] Furthermore, instead of (or in addition to) calculating the lane departure time TTLC, the traveling trajectory estimation unit 144A may calculate a time to collision (TTC) until the host vehicle M travels along either the camera lane marking or the map lane marking (for example, the camera lane marking) when the other vehicle m1 moves along the direction A3 when it is determined that the camera lane marking is separated from the map lane marking. The collision time TTC is calculated, for example, by dividing the relative distance between the host vehicle M and the other vehicle m1 by the relative speed between the host vehicle M and the other vehicle m1. In the example of FIG. 3, the collision time TTC is calculated when the host vehicle M travels along a lane defined by the camera lane marks CL1 and CL2 and the other vehicle m1 travels in the direction A3.

[0072] The driving control unit 144B controls the driving of the host vehicle M based on at least one of the lane departure time TTLC and the interference margin time TTC calculated by the driving trajectory estimation unit 144A. For example, when the lane departure time TTLC is equal to or greater than a threshold (TTLC threshold), the driving control unit 144B generates a target trajectory for continuing driving control based on either the camera lane markings or the map lane markings. Furthermore, when the interference margin time TTC is equal to or greater than a threshold (TTC threshold), the driving control unit 144B may generate a target trajectory for continuing driving control based on either the camera lane markings or the map lane markings. Furthermore, when at least one of the lane departure time TTLC and the interference margin time TTC (particularly the interference margin time TTC) becomes less than the corresponding threshold, the driving control unit 144B executes control such as suppressing (terminating or switching the level of) driving control of the host vehicle M.

[0073] For example, if the camera lane markings and the map lane markings are different, at least one of them will be determined to be incorrect. However, if the error is determined too early, the vehicle M's driving control may be immediately switched if the other vehicle m1 is actually correct, potentially affecting its behavior. Therefore, in the first scenario, as shown in FIG. 3 , instead of immediately determining that the other vehicle m1 is traveling in a direction A1 aligned with the map lane markings or in the longitudinal direction A2 of the vehicle body of the other vehicle m1, the vehicle M may estimate that the other vehicle m1 is traveling in an intermediate direction A3. This allows the lane departure time TTLC and the interference margin time TTC to be extended, rather than traveling in the direction A1 aligned with the map lane markings ML2. This makes it easier to continue the current driving control. The driving control unit 144B then performs control, such as switching driving control or continuing driving control, based on the determination result of whether the lane markings are deviating, or the comparison result of the lane departure time TTLC and the interference margin time TTC with thresholds.

[0074] The above-described control is particularly effective when traveling through a lane change section, such as a curved road as shown in Fig. 4, because the camera division lines and the map division lines are likely to diverge. Therefore, the execution control unit 144 may execute the above-described control when the lane on which the host vehicle M is traveling is a lane change section, such as a curved road. Lane change sections include, in addition to curved roads, sections where the number of lanes increases or decreases, sections where the shape of the lanes temporarily changes due to road construction, etc.

[0075] [Second Scene] 5 is a diagram for explaining driving control of the host vehicle M in the second scenario. The second scenario differs from the first scenario described above in that an obstacle OB1 is present ahead of the lane L1 in which the host vehicle M is traveling (in other words, on the planned travel path of the host vehicle M). The obstacle is an object that the vehicle needs to avoid and overtake to avoid contact with, and includes, for example, a stationary object such as a parked vehicle.

[0076] When an obstacle OB1 is present ahead of the host vehicle M, the behavior plan generation unit 140 generates a target trajectory K1 to avoid contact with the obstacle OB1, as shown in FIG. 5. The host vehicle M then travels along the target trajectory K1 so that the reference position (e.g., the center or center of gravity) of the host vehicle M travels along the target trajectory K1. In this case, the orientation of the host vehicle M is tilted relative to the extension direction of the map lane markings ML1 and ML2. Furthermore, the camera lane markings also change due to the change in orientation, which changes the values ​​of the lane departure time TTLC and the interference margin time TTC. Therefore, when an obstacle OB1 is present on the travel trajectory of the host vehicle M, the determination unit 142 determines that either the camera lane markings or the map lane markings (e.g., the camera lane markings) are erroneously recognized, regardless of the values ​​of the lane departure time TTLC and the interference margin time TTC. This allows the determination unit 142 to determine that the lane markings are erroneously recognized early on when the obstacle OB1 is present, regardless of the movement of the other vehicle m1, and to perform driving control based on the determination result.

[0077] Furthermore, the determination unit 142 may determine that either the camera lane marking line or the map lane marking line (for example, the camera lane marking line) is erroneously recognized when the lane departure time TTLC or the time to interference TTC is determined to be less than the corresponding threshold (first threshold) a first predetermined number of times or more. Furthermore, the determination unit 142 may determine that either the camera lane marking line or the map lane marking line (for example, the map lane marking line) is erroneously recognized when the number of times the lane departure time TTLC or the time to interference TTC is less than a corresponding second threshold that is greater than the first threshold is determined to be greater than a second predetermined number of times that is greater than the first predetermined number of times. Since the number of samples used to determine whether or not a recognition error has occurred varies depending on the magnitude of the lane departure time TTLC or the time to interference TTC, an appropriate determination can be made while suppressing erroneous determinations depending on the magnitude of the lane departure time TTLC or the time to interference TTC.

[0078] This can prevent frequent switching of driving control and make driving control more stable. Furthermore, the execution control unit 144 may terminate the continuation of driving control when a predetermined time has elapsed in a state in which it has been determined that the camera lane markings and the map lane markings are separated. This can prevent driving control from continuing for a long period of time in a state in which the camera lane markings and the map lane markings are separated. In this case, the driving control unit 144B may switch to manual driving or execute control to lower the automation level of driving control from the current level. Note that the execution control unit 144 may set a condition that the host vehicle M travels a predetermined distance or more instead of (or in addition to) the above-mentioned predetermined time having elapsed.

[0079] [Processing flow] The following describes the processing executed by the automatic driving control device 100 of the embodiment. Of the processing executed by the automatic driving control device 100, the following mainly focuses on the driving control processing based on the recognition status of lane markings, etc. At the start of the flow, it is assumed that the host vehicle M is currently executing predetermined driving control. The following describes several examples. The processing described below may be repeatedly executed at a predetermined timing or at a predetermined cycle, and may be repeatedly executed while driving control by the automatic driving control device 100 is being executed.

[0080] [First Example] Fig. 6 is a flowchart showing an example of the flow of the driving control process in the first embodiment. In the example of Fig. 6, the first recognition unit 132 recognizes the lane markings (camera lane markings) present around the host vehicle M based on the output of the detection device DD that detects the surrounding conditions of the host vehicle M (step S100). Next, the second recognition unit 134 refers to map information based on the position information of the host vehicle M and recognizes the lane markings (map lane markings) present around the host vehicle M from the map information (step S110). Next, the determination unit 142 compares the camera lane markings with the map lane markings (step S120) and determines whether the camera lane markings and the map lane markings diverge (step S130).

[0081] If it is determined that the camera lane markings and the map lane markings diverge, the determination unit 142 determines whether an adjacent vehicle (another vehicle) adjacent to the host vehicle M has been detected by the first recognition unit 132 (step S140). If it is determined that an adjacent vehicle is present, the travel trajectory estimation unit 144A estimates the direction of movement of the adjacent vehicle and the future travel trajectory of the adjacent vehicle based on the direction (step S150). Next, the travel trajectory estimation unit 144A calculates the lane departure time TTLC of the adjacent vehicle based on the future travel trajectory of the adjacent vehicle (step S160). Next, the driving control unit 144B determines whether the lane departure time TTLC is equal to or greater than a threshold (TTLC threshold) (step S170). If it is determined that the TTLC is equal to or greater than the threshold, the driving control unit 144B continues the driving control currently being performed based on either the camera lane markings or the map lane markings (step S180). In the process of step S180, if the driving control currently being executed is the first driving control or the second driving control, the driving control unit 144B switches to the third driving control or the fourth driving control and continues the driving control. Either the camera lane markings or the map lane markings may be, for example, a predetermined one, or may be one that is closer to the shape of the past driving trajectory of the adjacent vehicle, or may be one that changes less from the shape of the surrounding roads.

[0082] Furthermore, if it is determined in the process of step S170 that the lane departure time TTLC of the adjacent vehicle is not equal to or greater than the threshold, or if it is determined in the process of step S140 that there is no adjacent vehicle, the driving control unit 144B suppresses driving control (step S190). "Suppressing driving control" includes, for example, terminating driving control or lowering the current automation level. Furthermore, if it is determined in the process of step S130 that there is no deviation, the driving control currently being executed is executed (continued) based on the recognized lane markings (camera lane markings, map lane markings) (step S200). This ends the process of this flowchart.

[0083] [Second Example] Fig. 7 is a flowchart showing an example of the flow of the operation control process in the second embodiment. The process in the second embodiment differs from the process in steps S100 to S200 in the first embodiment shown in Fig. 6 in that steps S162 and S172 are included instead of steps S160 and S170. Therefore, the following description will mainly focus on the processes in steps S162 and S172.

[0084] In the process of step S150 shown in FIG. 7, after estimating the direction of movement of the adjacent vehicle and the traveling trajectory based on the direction, the traveling trajectory estimation unit 144A calculates the interference margin time TTC based on the estimated traveling trajectory and the traveling trajectory of the host vehicle M (step S162). Next, the traveling control unit 144B determines whether the calculated interference margin time TTC is equal to or greater than a threshold value (TTC threshold value) (step S172). If it is determined that the interference margin time TTC is equal to or greater than the threshold value, the process of step S180 is performed. Furthermore, if it is determined in the process of step S172 that the interference margin time TTC is not equal to or greater than the threshold value, the process of step S190 is performed.

[0085] [Variations] In the above-described embodiment, when there are multiple adjacent vehicles, the vehicle closest to the host vehicle M may be used to estimate the travel trajectory of the adjacent vehicle, calculate the lane departure time TTLC and the time to collision TTC based on the estimated travel trajectory, and perform judgment and driving control based on the calculation results. Note that if the closest adjacent vehicle is a specific vehicle (emergency vehicle) such as a police vehicle or a fire engine, it may behave differently from a normal vehicle (general vehicle), and therefore may be excluded from the judgment of the adjacent vehicle.

[0086] In addition, in an embodiment, both the lane departure time TTLC and the interference margin time TTC may be calculated, and each calculation result may be compared with a corresponding threshold value. If both are equal to or greater than the threshold value, driving control may be continued based on either the camera lane markings or the map lane markings. This allows for safer driving control.

[0087] Furthermore, the execution control unit 144 may select and make a determination between the lane departure time TTLC and the interference margin time TTC depending on the surrounding conditions such as the road shape and surrounding vehicles, etc. This allows for the implementation of appropriate driving control depending on the surrounding conditions.

[0088] In addition, in the embodiment, instead of determining whether the camera lane line and the map lane line diverge, it may be determined whether the camera lane line and the map lane line coincide with each other. Also, in the embodiment, when it is determined that the camera lane line and the map lane line diverge, one of the camera lane line and the map lane line is set as the camera lane line and the other is set as the map lane line, but the camera lane line and the map lane line may be reversed.

[0089] According to the above-described embodiment, the automatic driving control device 100 (an example of a vehicle control device) includes a first recognition unit 132 that recognizes the surrounding situation including camera dividing lines (an example of first dividing lines) that divide the lane in which the host vehicle M is traveling and adjacent vehicles traveling in adjacent lanes adjacent to the lane in which the host vehicle M is traveling, based on the output of a detection device DD that detects the surrounding situation of the host vehicle M; a second recognition unit 134 that recognizes map dividing lines (second dividing lines) that divide the lanes around the host vehicle M from map information, based on the position information of the host vehicle M; a driving control unit that executes driving control to control at least the steering of the host vehicle M, of the steering or speed, based on the recognition results of the first recognition unit 132 and the second recognition unit 134; and a driving trajectory estimation unit 144A that estimates the driving trajectory of the adjacent vehicle. When the camera lane markings recognized by the first recognition unit 132 deviate from the map lane markings, the control unit executes driving control based on either the camera lane markings or the map lane markings. When the camera lane markings deviate from the map lane markings, the driving trajectory estimation unit 144A estimates that the adjacent vehicle will move in a direction between the extension direction of the other of the camera lane markings or the map lane markings and the longitudinal direction of the adjacent vehicle's body. When the adjacent vehicle moves along the estimated direction of movement, the control unit calculates the time it takes for the adjacent vehicle to deviate from either the camera lane markings or the map lane markings. When the estimated time is equal to or greater than a threshold, the driving control unit continues driving control, thereby executing more appropriate driving control in accordance with the conditions of the road lane markings around the vehicle and other vehicles. This ultimately contributes to the development of a sustainable transportation system.

[0090] Furthermore, according to the above-described embodiment, in the automatic driving control device 100 (an example of a vehicle control device), when the camera lane markings and the map lane markings diverge, the driving trajectory estimation unit 144A estimates that the direction of movement of the adjacent vehicle is between the direction in which the other of the camera lane markings and the map lane markings extends and the longitudinal direction of the adjacent vehicle's body. If the adjacent vehicle moves along the estimated direction of movement, the driving control unit calculates the time until the host vehicle, which is traveling under the driving control, interferes with the adjacent vehicle. If the time is equal to or greater than a threshold, the driving control unit continues driving control, thereby performing more appropriate driving control in accordance with the conditions of the road lane markings and other vehicles around the vehicle. This can ultimately contribute to the development of a sustainable transportation system.

[0091] Furthermore, according to the embodiment, when the camera lane markings and the map lane markings diverge, more appropriate driving control can be performed by appropriately predicting future interference between the vehicle M and an adjacent vehicle (e.g., another vehicle traveling in an adjacent lane) based on the other vehicle and the lane shape. Furthermore, according to the embodiment, when the camera lane markings and the map lane markings diverge, the future driving trajectory of the adjacent vehicle can be estimated based on the direction in which one of the lane markings extends and the direction of the adjacent vehicle, and control (including avoidance control to avoid interference with other vehicles) can be performed to enable the vehicle M to continue the driving control it is currently performing based on the estimated trajectory.

[0092] Because the camera lane markings and the map lane markings deviate from each other, if the lane departure time TTLC or the time to interference TTC is less than a threshold when the host vehicle M is traveling along one of the lane markings, it is determined that the lane markings used as the basis for the host vehicle M's execution of driving control are incorrect. In this case, if it is determined to be incorrect too early, the driving control will be immediately switched if it turns out to be correct, which will affect the behavior of the host vehicle M. Therefore, in the embodiment, the direction (travel direction) of the adjacent vehicle is estimated based on the extension direction of the lane markings and the longitudinal direction of the body of the adjacent vehicle so as to increase the lane departure time TTLC or the time to interference TTC, making it easier for the host vehicle M to continue the driving control it is currently executing, thereby preventing the driving control from being switched immediately.

[0093] Furthermore, according to the embodiment, driving control can be executed (continued) using more appropriate information in lane change sections such as curved roads where the camera division lines and map division lines are likely to diverge.

[0094] The above-described embodiment can be expressed as follows. a storage medium for storing computer-readable instructions; a processor connected to the storage medium; The processor executes the computer-readable instructions to: Based on an output from a detection device that detects the surrounding conditions of the host vehicle, the system recognizes the surrounding conditions including a first dividing line that divides the lane in which the host vehicle is traveling and an adjacent vehicle traveling in an adjacent lane that is adjacent to the lane in which the host vehicle is traveling; recognizes second lane markings that demarcate lanes around the vehicle from map information based on the position information of the vehicle; Based on the recognition result, a driving control is performed to control at least the steering of the vehicle or the speed of the vehicle; Estimating a travel trajectory of the adjacent vehicle; When the first demarcation line and the second demarcation line diverge, the driving control is performed based on either the first demarcation line or the second demarcation line; When the first demarcation line and the second demarcation line diverge, the direction in which the adjacent vehicle moves is estimated to be a direction between the direction in which the other of the first demarcation line and the second demarcation line extends and the longitudinal direction of the body of the adjacent vehicle, calculating a time for the adjacent vehicle to deviate from either the first lane marking or the second lane marking when the adjacent vehicle moves along the estimated movement direction; If the time is equal to or greater than a threshold, the operation control is continued. Operation control device.

[0095] The above-described embodiment can be expressed as follows. a storage medium for storing computer-readable instructions; a processor connected to the storage medium; The processor executes the computer-readable instructions to: The computer Based on an output from a detection device that detects the surrounding conditions of the host vehicle, the system recognizes the surrounding conditions including a first dividing line that divides the lane in which the host vehicle is traveling and an adjacent vehicle traveling in an adjacent lane that is adjacent to the lane in which the host vehicle is traveling; recognizes second lane markings that demarcate lanes around the vehicle from map information based on the position information of the vehicle; Based on the recognition result, a driving control is performed to control at least the steering of the vehicle or the speed of the vehicle; Estimating a travel trajectory of the adjacent vehicle; When the first demarcation line and the second demarcation line diverge, the driving control is performed based on either the first demarcation line or the second demarcation line; When the first demarcation line and the second demarcation line diverge, the direction in which the adjacent vehicle moves is estimated to be a direction between the direction in which the other of the first demarcation line and the second demarcation line extends and the longitudinal direction of the body of the adjacent vehicle, calculating a time until interference occurs between the host vehicle traveling under the driving control and the adjacent vehicle when the adjacent vehicle moves along the estimated moving direction; If the time is equal to or greater than a threshold, the operation control is continued. Operation control device.

[0096] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0097] 1...vehicle system, 10...camera, 12...radar device, 14...LIDAR, 16...object recognition device, 20...communication device, 30...HMI, 40...vehicle sensor, 50...navigation device, 60...MPU, 80...driving operator, 100...automatic driving control device, 120...first control unit, 130...recognition unit, 132...first recognition unit, 134...second recognition unit, 140...action plan generation unit, 142...determination unit, 144...execution control unit, 144A...traveling trajectory estimation unit, 144B...traveling control unit, 160...second control unit, 162...target trajectory acquisition unit, 164...speed control unit, 166...steering control unit, 180...HMI control unit, 190...memory unit, 200...traveling drive force output device, 210...brake device, 220...steering device, M...host vehicle

Claims

1. a first recognition unit that recognizes a surrounding situation including a first dividing line that divides the lane in which the host vehicle is traveling and an adjacent vehicle traveling in an adjacent lane adjacent to the lane in which the host vehicle is traveling, based on an output of a detection device that detects a surrounding situation of the host vehicle; a second recognition unit that recognizes second lane markings that demarcate lanes around the host vehicle from map information based on the position information of the host vehicle; a driving control unit that executes driving control to control at least the steering of the vehicle based on the recognition results of the first recognition unit and the second recognition unit; and a travel trajectory estimation unit that estimates a travel trajectory of the adjacent vehicle, When the first demarcation line and the second demarcation line deviate from each other, the driving control unit executes the driving control based on either the first demarcation line or the second demarcation line; The traveling trajectory estimation unit When the first demarcation line and the second demarcation line diverge, the direction in which the adjacent vehicle moves is estimated to be a direction between the direction in which the other of the first demarcation line and the second demarcation line extends and the longitudinal direction of the body of the adjacent vehicle, calculating a time for the adjacent vehicle to deviate from either the first lane marking or the second lane marking when the adjacent vehicle moves along the estimated movement direction; The operation control unit continues the operation control when the time is equal to or greater than a threshold value. Operation control device.

2. a first recognition unit that recognizes a surrounding situation including a first dividing line that divides the lane in which the host vehicle is traveling and an adjacent vehicle traveling in an adjacent lane adjacent to the lane in which the host vehicle is traveling, based on an output of a detection device that detects a surrounding situation of the host vehicle; a second recognition unit that recognizes second lane markings that demarcate lanes around the host vehicle from map information based on the position information of the host vehicle; a driving control unit that executes driving control to control at least the steering of the vehicle based on the recognition results of the first recognition unit and the second recognition unit; and a travel trajectory estimation unit that estimates a travel trajectory of the adjacent vehicle, When the first demarcation line and the second demarcation line deviate from each other, the driving control unit executes the driving control based on either the first demarcation line or the second demarcation line; The traveling trajectory estimation unit When the first demarcation line and the second demarcation line diverge, it is estimated that the adjacent vehicle will move in a direction between the direction in which the other of the first demarcation line and the second demarcation line extends and the longitudinal direction of the body of the adjacent vehicle, calculating a time until interference occurs between the host vehicle traveling under the driving control and the adjacent vehicle when the adjacent vehicle moves along the estimated moving direction; The operation control unit continues the operation control when the time is equal to or greater than a threshold value. Operation control device.

3. a determination unit that determines that one of the first demarcation line and the second demarcation line is an erroneous recognition when an obstacle is present on the lane on which the host vehicle is traveling, The operation control device according to claim 1 or 2.

4. The system further includes a determination unit that determines that one of the first demarcation line and the second demarcation line is erroneously recognized when it is determined that the time is less than a first threshold value a first predetermined number of times or more, and that determines that the other of the first demarcation line and the second demarcation line is erroneously recognized when the number of times that the time is less than a second threshold value that is greater than the first predetermined number of times or more. The operation control device according to claim 1 or 2.

5. the traveling trajectory estimation unit estimates a direction in which the other of the first demarcation line and the second demarcation line extends and a center position of a longitudinal direction of a body of the adjacent vehicle as a traveling direction of the adjacent vehicle; The operation control device according to claim 1 or 2.

6. when it is estimated that the adjacent vehicle will travel in a direction between a direction in which the other of the first demarcation line and the second demarcation line extends and a longitudinal direction of a body of the adjacent vehicle, the traveling trajectory estimation unit adjusts the direction between the first demarcation line and the second demarcation line in accordance with an amount of deviation between the first demarcation line and the second demarcation line. The operation control device according to claim 1 or 2.

7. a determination unit that, when determining whether the first demarcation line and the second demarcation line diverge, determines whether the first demarcation line is erroneously recognized based on at least one of the amount of change in curvature of the first demarcation line and the angle formed by the first demarcation line and the second demarcation line, and determines whether the first demarcation line is erroneously recognized if the direction of change in the amount of change in curvature and the direction of change in the angle are the same and the amount of change in curvature and the angle increase according to the distance from the vehicle. The operation control device according to claim 1 or 2.

8. The computer Based on an output from a detection device that detects a surrounding situation of the host vehicle, the surrounding situation is recognized, including a first dividing line that divides the lane in which the host vehicle is traveling and an adjacent vehicle traveling in an adjacent lane that is adjacent to the lane in which the host vehicle is traveling; recognizes second lane markings that demarcate lanes around the vehicle from map information based on the position information of the vehicle; Based on the recognition result, a driving control is performed to control at least the steering of the vehicle or the speed of the vehicle; Estimating a travel trajectory of the adjacent vehicle; When the first demarcation line and the second demarcation line deviate from each other, the driving control is executed based on either the first demarcation line or the second demarcation line; When the first demarcation line and the second demarcation line diverge, the direction in which the adjacent vehicle moves is estimated to be a direction between the direction in which the other of the first demarcation line and the second demarcation line extends and the longitudinal direction of the body of the adjacent vehicle, calculating a time for the adjacent vehicle to deviate from either the first lane marking or the second lane marking when the adjacent vehicle moves along the estimated movement direction; If the time is equal to or greater than a threshold, the operation control is continued. Vehicle control method.

9. The computer Based on an output from a detection device that detects a surrounding situation of the host vehicle, the surrounding situation is recognized, including a first dividing line that divides the lane in which the host vehicle is traveling and an adjacent vehicle traveling in an adjacent lane that is adjacent to the lane in which the host vehicle is traveling; recognizes second lane markings that demarcate lanes around the vehicle from map information based on the position information of the vehicle; Based on the recognition result, a driving control is performed to control at least the steering of the vehicle or the speed of the vehicle; Estimating a travel trajectory of the adjacent vehicle; When the first demarcation line and the second demarcation line deviate from each other, the driving control is executed based on either the first demarcation line or the second demarcation line; When the first demarcation line and the second demarcation line diverge, the direction in which the adjacent vehicle moves is estimated to be a direction between the direction in which the other of the first demarcation line and the second demarcation line extends and the longitudinal direction of the body of the adjacent vehicle, calculating a time until interference occurs between the host vehicle traveling under the driving control and the adjacent vehicle when the adjacent vehicle moves along the estimated moving direction; If the time is equal to or greater than a threshold, the operation control is continued. Vehicle control method.

10. On the computer, Based on an output from a detection device that detects a surrounding situation of the host vehicle, the surrounding situation including a first dividing line that divides the lane in which the host vehicle is traveling and an adjacent vehicle traveling in an adjacent lane adjacent to the lane in which the host vehicle is traveling is recognized; a second dividing line that divides a lane around the vehicle from map information based on the position information of the vehicle; Execute driving control that controls at least the steering of the vehicle or the speed of the vehicle based on the recognized result; Estimating a travel trajectory of the adjacent vehicle; When the first demarcation line and the second demarcation line deviate from each other, the driving control is executed based on either the first demarcation line or the second demarcation line; When the first demarcation line and the second demarcation line diverge, the direction in which the adjacent vehicle will move is estimated to be a direction between a direction in which the other of the first demarcation line and the second demarcation line extends and a longitudinal direction of the body of the adjacent vehicle, calculating a time for the adjacent vehicle to deviate from either the first lane marking or the second lane marking when the adjacent vehicle moves along the estimated movement direction; If the time is equal to or greater than a threshold, the operational control is continued. program.

11. On the computer, Based on an output from a detection device that detects a surrounding situation of the host vehicle, the surrounding situation including a first dividing line that divides the lane in which the host vehicle is traveling and an adjacent vehicle traveling in an adjacent lane adjacent to the lane in which the host vehicle is traveling is recognized; a second dividing line that divides a lane around the vehicle from map information based on the position information of the vehicle; Execute driving control that controls at least the steering of the vehicle or the speed of the vehicle based on the recognized result; Estimating a travel trajectory of the adjacent vehicle; When the first demarcation line and the second demarcation line deviate from each other, the driving control is executed based on either the first demarcation line or the second demarcation line; When the first demarcation line and the second demarcation line diverge, the direction in which the adjacent vehicle will move is estimated to be a direction between a direction in which the other of the first demarcation line and the second demarcation line extends and a longitudinal direction of the body of the adjacent vehicle, calculating a time until interference occurs between the host vehicle traveling under the driving control and the adjacent vehicle when the adjacent vehicle moves along the estimated moving direction; If the time is equal to or greater than a threshold, the operational control is continued. program.

Images