Vehicle control device, vehicle control method, and storage medium

Abstract

A vehicle control device, a vehicle control method, and a storage medium that flexibly change the driving control of a vehicle even when the road division line recognized by a camera differs from the content of map information mounted on the vehicle. The vehicle control device includes: an acquisition unit that acquires a camera image that photographs the surrounding situation of the vehicle; a driving control unit that controls the steering and acceleration / deceleration of the vehicle based on the camera image and the map information without relying on the operation of the driver of the vehicle; a mode determination unit that determines the driving mode of the vehicle to be any one of a plurality of driving modes including a first driving mode and a second driving mode; a determination unit that determines whether an abnormality has occurred in the road division line shown by the camera image or the road division line shown by the map information, and determines whether a preceding vehicle exists in the vehicle in the case where it is determined that an abnormality has occurred; and a calculation unit that calculates the degree of freedom for the vehicle to continue driving in the second driving mode in the case where it is determined that a preceding vehicle exists.

Description

TECHNICAL FIELD

[0001] The present application relates to a vehicle control device, a vehicle control method, and a storage medium. BACKGROUND

[0002] In the past, a technology of controlling travel of a host vehicle based on a road division line recognized by a camera mounted on the vehicle is known. For example, in Japanese Patent Application Publication No. 2020-050086, a technology is described in which the host vehicle is caused to travel based on a recognized road division line, and in a case where a degree of recognition of the road division line does not satisfy a prescribed criterion, the host vehicle is caused to travel based on a trajectory of a preceding vehicle.

[0003] The technology described in Patent Document 1 is a technology of controlling travel of a host vehicle based on a road division line recognized by a camera and map information mounted on the host vehicle. However, in the past technology, in a case where the road division line recognized by the camera and the content of the map information mounted on the host vehicle are different, the driving control of the vehicle is not always changed flexibly. SUMMARY

[0004] The present application is completed in consideration of such a situation, and one of the objects thereof is to provide a vehicle control device, a vehicle control method, and a storage medium capable of flexibly changing the driving control of the vehicle even in a case where the road division line recognized by the camera and the content of the map information mounted on the host vehicle are different.

[0005] The vehicle control device, the vehicle control method, and the storage medium of the present application adopt the following structure.

[0006] (1): One aspect of the present invention relates to a vehicle control device, wherein the vehicle control device includes: an acquisition unit that acquires a camera image obtained by capturing a surrounding situation of a vehicle; a driving control unit that controls steering and acceleration / deceleration of the vehicle based on the camera image and map information, without depending on an operation of a driver of the vehicle; a mode determination unit that determines a driving mode of the vehicle to be any one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task imposed on the driver is lighter than in the first driving mode, at least a part of the plurality of driving modes including the second driving mode being controlled by the driving control unit, the mode determination unit changing the driving mode of the vehicle to a driving mode in which a task is heavier than in the determined driving mode in a case where the task related to the determined driving mode is not performed by the driver; a determination unit that determines whether an abnormality occurs in a road division line shown in the camera image or a road division line shown in the map information, determines whether a preceding vehicle exists in a case where it is determined that the abnormality occurs; and a calculation unit that calculates a margin for the vehicle to continue driving in the second driving mode based on a distance from the vehicle to the preceding vehicle, a distance from the preceding vehicle to a point at which the preceding vehicle crosses the road division line shown in the camera image or the road division line shown in the map information, and a speed of the vehicle in a case where it is determined that the preceding vehicle exists, the mode determination unit causing the vehicle to continue driving in the second driving mode for a period indicated by the margin.

[0007] (2): On the basis of the aspect of (1) above, in a case where it is determined by the determination unit as the abnormality that there is a deviation on both sides of the road division line shown in the camera image and the road division line shown in the map information, the mode determination unit causes the driving in the second driving mode to continue for a period indicated by the margin using the road division line shown in the camera image as a reference line, and changes the second driving mode to the first driving mode after the period elapses.

[0008] (3) : In the aspect of the above (1), in a case where the determination unit determines that there is a deviation on both sides of the road division line shown in the camera image and the road division line shown in the map information as the abnormality, the determination unit determines whether a state in which the road division lines on both sides shown in the camera image deviate at a same angle and a curvature change rate of threshold or more from the road division lines on both sides shown in the map information continues for a prescribed period, and the determination unit determines that the road division line shown in the camera image is misrecognized in a case where it is determined that the state continues for the prescribed period, and the mode determination unit changes the second driving mode to the first driving mode using the road division line shown in the map information in a case where it is determined by the determination unit that the road division line shown in the camera image is misrecognized.

[0009] (4) : In the aspect of the above (1), in a case where the determination unit determines that there is a deviation only on one side of the road division line shown in the camera image and the road division line shown in the map information as the abnormality, the mode determination unit continues the travel in the second driving mode using the road division line on which the deviation does not occur as a reference line.

[0010] (5) : In the aspect of the above (1), in a case where the determination unit determines that the deviation occurs due to disappearance of both sides of the road division line shown in the camera image as the abnormality, the mode determination unit continues the travel in the second driving mode using the road division line shown in the map information as a reference line for a period indicated by the margin, and changes the second driving mode to the manual driving stage via the first driving mode after the period elapses.

[0011] (6) : In the aspect of the above (1), in a case where the determination unit determines that one side of the road division line shown in the camera image disappears and the other side of the road division line shown in the camera image is misrecognized due to a deviation as the abnormality, the mode determination unit continues the travel in the second driving mode using the road division line shown in the map information as a reference line for a period indicated by the margin, and changes the second driving mode to the manual driving stage via the first driving mode after the period elapses.

[0012] (7) : In any one of the aspects of the above (1) to (6), the second driving mode is a driving mode in which the driver is not arranged a task of holding an operation member that accepts a steering operation of the vehicle, and the first driving mode is a driving mode in which the driver is arranged a task of holding only the operation member.

[0013] (8): Another aspect of the present application relates to a vehicle control method, wherein the vehicle control method causes a computer to perform the following processing: acquire a camera image obtained by capturing a surrounding situation of a vehicle; control steering and acceleration / deceleration of the vehicle independently of an operation of a driver of the vehicle, based on the camera image and map information; determine a driving mode of the vehicle to be any one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task imposed on the driver is lighter than in the first driving mode, at least a part of the plurality of driving modes including the second driving mode being performed by controlling the steering and acceleration / deceleration of the vehicle independently of the operation of the driver of the vehicle, change the driving mode of the vehicle to a driving mode in which a task is heavier than in the determined driving mode in a case where the task is not performed by the driver; determine whether or not a road division line shown in the camera image or a road division line shown in the map information has become abnormal, in a case where it is determined that the abnormality has occurred, determine whether or not there is a preceding vehicle in front of the vehicle; in a case where it is determined that there is the preceding vehicle, calculate a margin for the vehicle to continue driving in the second driving mode, based on a distance from the vehicle to the preceding vehicle, a distance from the preceding vehicle to a point at which the preceding vehicle crosses the road division line shown in the camera image or the road division line shown in the map information, and a speed of the vehicle; and cause the vehicle to continue driving in the second driving mode for a period indicated by the margin.

[0014] (9) : Another aspect of the present application relates to a storage medium storing a program, wherein the program causes a computer to perform the following processing: acquire a camera image obtained by capturing a surrounding situation of a vehicle; control steering and acceleration / deceleration of the vehicle based on the camera image and map information, without depending on an operation of a driver of the vehicle; determine a driving mode of the vehicle to be any one of a plurality of driving modes including a first driving mode and a second driving mode which is a driving mode in which a task imposed on the driver is lighter than the first driving mode, at least a part of the plurality of driving modes including the second driving mode being performed by controlling the steering and acceleration / deceleration of the vehicle without depending on the operation of the driver of the vehicle; change the driving mode of the vehicle to a driving mode in which a task is heavier than the determined driving mode, in a case where the task related to the determined driving mode is not performed by the driver; determine whether an abnormality occurs in a road division line shown in the camera image or a road division line shown in the map information, in a case where it is determined that the abnormality occurs, determine whether a preceding vehicle exists in front of the vehicle; in a case where it is determined that the preceding vehicle exists, calculate a margin for the vehicle to continue driving in the second driving mode, based on a distance from the vehicle to the preceding vehicle, a distance from the preceding vehicle to a point at which the preceding vehicle crosses the road division line shown in the camera image or the road division line shown in the map information, and a speed of the vehicle; and cause the vehicle to continue driving in the second driving mode for a period indicated by the margin.

[0015] According to (1) to (9), even in a case where a road division line recognized by a camera is different from the content of map information mounted on the host vehicle, the driving control of the vehicle can be flexibly changed. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a configuration diagram of a vehicle system using the vehicle control device of the embodiment.

[0017] Figure 2 is a functional configuration diagram of the first control section and the second control section.

[0018] Figure 3 is a diagram showing an example of a correspondence relationship between a driving mode, a control state of the host vehicle, and a task.

[0019] Figure 4 is a diagram showing an example of a scenario in which the operation of the vehicle control device of the embodiment is performed.

[0020] Figure 5 is a diagram showing another example of a scenario in which the operation of the vehicle control device of the embodiment is performed.

[0021] Figure 6 FIG. 2 is a diagram showing another example of a scenario in which the action of the vehicle control device of the embodiment is performed.

[0022] Figure 7 FIG. 2 is a diagram showing another example of a scenario in which the action of the vehicle control device of the embodiment is performed.

[0023] Figure 8 FIG. 2 is a diagram showing another example of a scenario in which the action of the vehicle control device of the embodiment is performed.

[0024] Figure 9 FIG. 3 is a flowchart showing an example of a flow of the action performed by the vehicle control device of the embodiment.

[0025] Figure 10 FIG. 4 is a flowchart showing an example of a flow of the action performed by the vehicle control device when the camera lane division line disappears. DETAILED DESCRIPTION

[0026] Hereinafter, the embodiments of the vehicle control device, the vehicle control method, and the storage medium of the present application will be described with reference to the accompanying drawings.

[0027] [Overall Structure]

[0028] Figure 1 FIG. 1 is a structural diagram of a vehicle system 1 that utilizes the vehicle control device of the embodiment. The vehicle on which the vehicle system 1 is mounted is, for example, a two-wheeled, three-wheeled, four-wheeled, or the like vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using the electric power generated by a generator coupled to the internal combustion engine, or the discharge electric power of a secondary battery or a fuel cell.

[0029] The vehicle system 1 is provided with, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driver monitoring camera 70, a driving operation member 80, an automatic driving control device 100, a travel drive power output device 200, a brake device 210, and a steering device 220. These devices and apparatuses are connected to each other through a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. Note that, Figure 1 The structure shown is only an example, and a part of the structure can be omitted, or another structure can be further added.

[0030] The camera 10 is, for example, a digital camera that uses a solid-state image pickup element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is mounted at an arbitrary position of a vehicle (hereinafter referred to as the host vehicle M) on which the vehicle system 1 is mounted. In the case of capturing an image of the front, the camera 10 is mounted on the upper portion of the front windshield glass, the back surface of the interior rearview mirror, or the like. The camera 10 repeatedly captures an image of the periphery of the host vehicle M periodically, for example. The camera 10 can also be a stereo camera.

[0031] The radar device 12 radiates millimeter waves or the like toward the periphery of the host vehicle M and detects a wave (reflected wave) reflected by an object to detect at least the position (distance and direction) of the object. The radar device 12 is mounted at an arbitrary position of the host vehicle M. The radar device 12 can also detect the position and speed of an object by an FM-CW (Frequency Modulated Continuous Wave) method.

[0032] The LIDAR 14 radiates light (or an electromagnetic wave having a wavelength close to light) toward the periphery of the host vehicle M and measures scattered light. The LIDAR 14 detects the distance to an object on the basis of the time from light emission to light reception. The radiated light is, for example, pulsed laser light. The LIDAR 14 is mounted at an arbitrary position of the host vehicle M.

[0033] The object recognition device 16 performs sensor fusion processing on the detection results detected by some or all of the camera 10, the radar device 12, and the LIDAR 14 to recognize the position, type, speed, and the like of an object. The object recognition device 16 outputs the recognition result to the automatic driving control device 100. The object recognition device 16 can output the detection results of the camera 10, the radar device 12, and the LIDAR 14 directly to the automatic driving control device 100. The object recognition device 16 can also be omitted from the vehicle system 1.

[0034] The communication device 20 communicates with other vehicles existing in the periphery of the host vehicle M or communicates with various server devices via a wireless base station, for example, by using a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), or the like.

[0035] The HMI 30 prompts various information to the occupant of the host vehicle M and accepts an input operation by the occupant. The HMI 30 includes various display devices, a speaker, a buzzer, a touch panel, switches, buttons, and the like.

[0036] The vehicle sensors 40 include a vehicle speed sensor that detects the speed of the host vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity about the vertical axis, a direction sensor that detects the orientation of the host vehicle M, and the like.

[0037] The navigation device 50 has, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a route decision section 53. The navigation device 50 holds first map information 54 in a storage device such as an HDD (Hard Disk Drive), a flash memory, or the like. The GNSS receiver 51 determines the position of the host vehicle M based on signals received from GNSS satellites. The position of the host vehicle M can also be determined or supplemented by an INS (Inertial Navigation System) that utilizes the outputs of the vehicle sensors 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, a button, and the like. The navigation HMI 52 can also be partially or wholly shared with the aforementioned HMI 30. The route decision section 53 decides a route (hereinafter referred to as an on-map route) from the position of the host vehicle M determined by the GNSS receiver 51 (or an arbitrary position input) to a destination input by an occupant using the navigation HMI 52, for example, with reference to the first map information 54. The first map information 54 is, for example, information that represents the shape of a road by road segments and nodes connected by the road segments. The first map information 54 can also include the curvature of the road, POI (Point Of Interest) information, and the like. The on-map route is output to the MPU 60. The navigation device 50 can also perform route guidance using the navigation HMI 52 based on the on-map route. The navigation device 50 can also be realized by the functions of a terminal device such as a smartphone, a tablet terminal, or the like held by an occupant. The navigation device 50 can also transmit the current position and the destination to a navigation server via the communication device 20 and acquire a route equivalent to the on-map route from the navigation server.

[0038] The MPU 60 includes, for example, a recommended lane decision section 61 and holds second map information 62 in a storage device such as an HDD, a flash memory, or the like. The recommended lane decision section 61 divides the on-map route provided from the navigation device 50 into a plurality of blocks (for example, divided every 100 [m] in the vehicle travel direction) and decides a recommended lane for each block with reference to the second map information 62. The recommended lane decision section 61 performs the decision to travel on which lane numbered from the left. The recommended lane decision section 61 decides the recommended lane so that the host vehicle M can travel on a reasonable route for traveling to a branched destination in the case where there is a branch site in the on-map route.

[0039] The second map information 62 is map information of higher precision than the first map information 54. The second map information 62 includes, for example, information of the center of a lane or information of the boundary of a lane, and the like. In addition, the second map information 62 can include road information, traffic restriction information, dwelling information (dwellings, postal codes), facility information, telephone number information, information of a prohibited section in which the mode A or the mode B described later is prohibited, and the like. The second map information 62 can be updated at any time by the communication device 20 communicating with other devices.

[0040] The driver monitoring camera 70 is, for example, a digital camera that uses a solid-state imaging element such as a CCD or a CMOS. The driver monitoring camera 70 is installed at an arbitrary position in the host vehicle M in a position and an orientation capable of capturing the head of an occupant (hereinafter referred to as a driver) seated on the driver's seat of the host vehicle M from the front (in an orientation to capture the face). For example, the driver monitoring camera 70 is installed on the upper portion of a display device provided in the central portion of the instrument panel of the host vehicle M.

[0041] The driver operation member 80 includes, for example, in addition to the steering wheel 82, an accelerator pedal, a brake pedal, a shift lever, and other operation members. A sensor that detects an operation amount or the presence or absence of an operation is installed on the driver operation member 80, and the detection result is output to the automatic driving control device 100 or to some or all of the travel driving force output device 200, the brake device 210, and the steering device 220. The steering wheel 82 is an example of an "operation member that accepts a steering operation by a driver". The operation member need not necessarily be ring-shaped, and can be in the form of a shaped steering wheel, a joystick, a button, or the like. A steering wheel grip sensor 84 is installed on the steering wheel 82. The steering wheel grip sensor 84 is implemented by an electrostatic capacity sensor or the like, and is used to output to the automatic driving control device 100 a signal that can detect whether or not the driver is gripping (contacting in a force-applied state) the steering wheel 82.

[0042] The autonomous driving control device 100, for example, has a first control section 120 and a second control section 160. The first control section 120 and the second control section 160 are each realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). In addition, a part or all of these components can also be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), and can also be realized by a cooperative operation of software and hardware. The program can be stored in advance in a storage device (a storage device having a non-transitory storage medium) such as an HDD, a flash memory, or the like of the autonomous driving control device 100, can be stored in a removable storage medium such as a DVD, a CD-ROM, or the like, and can be installed in the HDD, the flash memory, or the like of the autonomous driving control device 100 by mounting the storage medium (non-transitory storage medium) in a drive device. The autonomous driving control device 100 is an example of a "vehicle control device", and the action plan generation section 140 and the second control section 160 together are an example of a "driving control section".

[0043] Figure 2 is a functional configuration diagram of the first control section 120 and the second control section 160. The first control section 120, for example, has an identification section 130, an action plan generation section 140, and a mode determination section 150. The first control section 120, for example, realizes functions based on AI (Artificial Intelligence) and functions based on a model given in advance in parallel. For example, the function of "identifying an intersection" can be realized by "performing identification of an intersection based on deep learning or the like and identification based on a condition given in advance (presence of a signal, a road sign, or the like capable of pattern matching, and the like) in parallel, and comprehensively evaluating both by scoring". Thereby, the reliability of autonomous driving can be ensured.

[0044] The recognition unit 130 recognizes the position, speed, and acceleration, and the like of an object in the periphery of the host vehicle M, on the basis of information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16. The position of the object is recognized as a position on an absolute coordinate with a representative point (center of gravity, center of a drive shaft, or the like) of the host vehicle M as an origin, for example, and is used for control. The position of the object can be represented by a representative point such as a center of gravity or a corner of the object, or can be represented by an area. The "state" of the object can also include the acceleration, the jerk, or the "behavioral state" (for example, whether or not the lane change is being performed or is to be performed) of the object.

[0045] In addition, the recognition unit 130 recognizes a lane (traveling lane) in which the host vehicle M is traveling, for example. The recognition unit 130 recognizes the traveling lane by comparing a pattern of road division lines (for example, an arrangement of solid lines and broken lines) obtained from the second map information 62 with a pattern of road division lines in the periphery of the host vehicle M recognized on the basis of an image captured by the camera 10, for example. Note that the recognition unit 130 is not limited to recognizing road division lines, and can recognize a traveling road boundary (road boundary) including a road shoulder, a curb, a median, a guardrail, and the like, thereby recognizing the traveling lane. In this recognition, the position of the host vehicle M obtained from the navigation device 50 and a processing result processed by the INS can be taken into consideration. In addition, the recognition unit 130 recognizes a stop line, an obstacle, a red light, a tollgate, and other road phenomena.

[0046] The recognition unit 130 recognizes the position and posture of the host vehicle M with respect to the traveling lane when recognizing the traveling lane. The recognition unit 130 can recognize the deviation of a reference point of the host vehicle M from the center of the lane and the angle of the traveling direction of the host vehicle M with respect to a line connecting the center of the lane, for example, as the relative position and posture of the host vehicle M with respect to the traveling lane. Instead of this, the recognition unit 130 can recognize the position of a reference point of the host vehicle M with respect to an arbitrary side end portion (road division line or road boundary) of the traveling lane, or the like, as the relative position of the host vehicle M with respect to the traveling lane. The recognition unit 130 is an example of the "acquisition unit".

[0047] The action plan generation section 140 generates a target track along which the host vehicle M is to travel automatically (independently of the operation of the driver) in the future, in a manner that the host vehicle M travels on the recommended lane decided by the recommended lane decision section 61 in principle and that the host vehicle M can cope with the surrounding situation of the host vehicle M. The target track includes, for example, a speed element. The target track is expressed, for example, as a track in which points (track points) at which the host vehicle M should arrive are arranged in order. The track points are points at which the host vehicle M should arrive at every prescribed travel distance (for example, several [m]) along the route, and, in addition thereto, a target speed and a target acceleration are generated as a part of the target track at every prescribed sampling time (for example, several [sec]). In addition, the track points can be positions at which the host vehicle M should arrive at every prescribed sampling time. In this case, information of the target speed and the target acceleration is expressed by the interval of the track points.

[0048] The action plan generation section 140 can set an event of automatic driving when generating the target track. In the event of automatic driving, there are a constant speed driving event, a low speed following driving event, a lane change event, a branch event, a merging event, a takeover event, and the like. The action plan generation section 140 generates a target track corresponding to the event that is started.

[0049] The mode decision section 150 decides the driving mode of the host vehicle M as any one of a plurality of driving modes in which the task arranged to the driver is different. The mode decision section 150 has, for example, an abnormality determination section 152 and a margin calculation section 154. The individual functions thereof will be described later.

[0050] Figure 3 is a diagram showing an example of a correspondence relation between the driving mode, the control state of the host vehicle M, and the task. In the driving mode of the host vehicle M, there are, for example, five modes of Mode A to Mode E. As for the degree of automation of the driving control of the host vehicle M, Mode A is the highest, and the degree of automation is lowered in order of Mode B, Mode C, Mode D, and Mode E. On the contrary, as for the task arranged to the driver, Mode A is the lightest, and the degree of the task is increased in order of Mode B, Mode C, Mode D, and Mode E. Note that, under Mode D and Mode E, the control state becomes not automatic driving, and thus the automatic driving control device 100 functions before the control involved in the automatic driving is ended and shifted to driving support or manual driving. Hereinafter, the content of each driving mode will be exemplified.

[0051] In mode A, a state of automatic driving, neither the front monitoring nor the holding of the steering wheel 82 (steering wheel holding in the figure) is arranged to the driver. However, even in mode A, the driver is required to be in a physical condition to shift to manual driving promptly in accordance with a request from the system centered on the automatic driving control device 100. Note that the automatic driving herein means that the steering, acceleration and deceleration are controlled without depending on the operation of the driver. The front means the space in the direction of travel of the host vehicle M visually recognized via the front windshield. Mode A is a driving mode that can be executed, for example, in a case where the host vehicle M is traveling at a prescribed speed (e.g., 50 [km / h] or so) or less on an exclusive road for motor vehicles such as an expressway, and a preceding vehicle exists as a following object, and is sometimes referred to as TJP (Traffic Jam Pilot). In a case where the conditions are no longer satisfied, the mode determining portion 150 changes the driving mode of the host vehicle M to mode B.

[0052] In mode B, a state of driving support, the task of monitoring the front of the host vehicle M (hereinafter referred to as front monitoring) is arranged to the driver, but the task of holding the steering wheel 82 is not arranged. In mode C, a state of driving support, the tasks of front monitoring and holding the steering wheel 82 are arranged to the driver. Mode D is a driving mode in which a certain degree of driving operation by the driver is required with respect to at least one of the steering and the acceleration and deceleration of the host vehicle M. For example, in mode D, driving support such as ACC (Adaptive Cruise Control), LKAS (Lane Keeping Assist System) is performed. In mode E, a state of manual driving in which the steering and the acceleration and deceleration are required to be performed by the driver. In modes D and E, the task of monitoring the front of the host vehicle M is of course arranged to the driver.

[0053] The automatic driving control device 100 (and the driving support device (not shown)) performs automatic lane change corresponding to the driving mode. In the automatic lane change, there are an automatic lane change (1) based on a system request, and an automatic lane change (2) based on a driver request. In the automatic lane change (1), there are an automatic lane change for overtaking performed in a case where the speed of the preceding vehicle is less than or equal to a predetermined small reference value compared to the speed of the host vehicle, and an automatic lane change for traveling toward a destination (automatic lane change due to a change in the recommended lane). The automatic lane change (2) means that, in a case where a condition related to the speed, the positional relationship with the surrounding vehicles, or the like is satisfied, the host vehicle M is caused to perform lane change toward the operation direction when the driver operates the direction indicator.

[0054] The automatic driving control device 100 does not perform any of the automatic lane changes (1) and (2) in mode A. The automatic driving control device 100 performs any of the automatic lane changes (1) and (2) in modes B and C. The driving support device (not shown) does not perform the automatic lane change (1) but performs the automatic lane change (2) in mode D. In mode E, any of the automatic lane changes (1) and (2) is not performed.

[0055] The mode determining section 150 changes the driving mode of the host vehicle M to a driving mode in which the task is more intensive in a case where the task involved in the determined driving mode (hereinafter referred to as the current driving mode) is not performed by the driver.

[0056] For example, in a case where the driver is in a physical posture in which the driver cannot shift to manual driving in accordance with the request from the system in mode A (for example, a case where the driver continues to look out of the allowed area to the east and west, or a case where a precursor to driving difficulty is detected), the mode determining section 150 urges the driver to shift to manual driving using the HMI 30, and if the driver does not respond, performs control to gradually stop the host vehicle M by moving it toward the shoulder and to stop the automatic driving. After stopping the automatic driving, the host vehicle becomes in a state of mode D or E, and the host vehicle M can be started by manual operation by the driver. Hereinafter, the same applies to "stopping the automatic driving". In a case where the driver does not monitor the front in mode B, the mode determining section 150 urges the driver to monitor the front using the HMI 30, and if the driver does not respond, performs control to gradually stop the host vehicle M by moving it toward the shoulder and to stop the automatic driving. In a case where the driver does not monitor the front or does not hold the steering wheel 82 in mode C, the mode determining section 150 urges the driver to monitor the front and / or to hold the steering wheel 82 using the HMI 30, and if the driver does not respond, performs control to gradually stop the host vehicle M by moving it toward the shoulder and to stop the automatic driving.

[0057] The second control section 160 controls the travel driving force output device 200, the brake device 210, and the steering device 220 so that the host vehicle M passes through the target trajectory generated by the action plan generating section 140 at a predetermined timing.

[0058] Return Figure 2The second control unit 160 includes, for example, a retrieval unit 162, a speed control unit 164, and a steering control unit 166. The retrieval unit 162 retrieves information of the target trajectory (trajectory point) generated by the travel plan generation unit 140, and causes a memory (not shown) to store the information. The speed control unit 164 controls the travel drive force output device 200 or the brake device 210 on the basis of a speed component attached to the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 in accordance with a bending condition of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is implemented, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 performs feedforward control corresponding to the curvature of the road ahead of the host vehicle M and feedback control based on deviation from the target trajectory in combination.

[0059] The travel drive force output device 200 outputs a travel drive force (torque) for vehicle travel to the drive wheels. The travel drive force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, and the like, and an ECU (Electronic Control Unit) that controls them. The ECU controls the above-described structure in accordance with information input from the second control unit 160 or information input from the driving operation member 80.

[0060] The brake device 210 includes, for example, a brake caliper, a hydraulic cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the hydraulic cylinder, and a brake ECU. The brake ECU controls the electric motor in accordance with information input from the second control unit 160 or information input from the driving operation member 80 so that a brake torque corresponding to a brake operation is output to each wheel. The brake device 210 can include, as a backup, a mechanism that transmits hydraulic pressure generated by operation of a brake pedal included in the driving operation member 80 to the hydraulic cylinder via a master hydraulic cylinder. Note that the brake device 210 is not limited to the structure described above, and can be an electronically controlled hydraulic brake device that transmits hydraulic pressure of the master hydraulic cylinder to the hydraulic cylinder by controlling an actuator in accordance with information input from the second control unit 160.

[0061] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies force to a rack-and-pinion mechanism to change the orientation of the steered wheels. The steering ECU drives the electric motor in accordance with information input from the second control unit 160 or information input from the driving operation member 80 to change the orientation of the steered wheels.

[0062] [Operation of Vehicle Control Device]

[0063] Next, the operation of the vehicle control device of the embodiment will be described. In the following description, it is assumed that the host vehicle M is traveling in the driving mode of Mode B. Figure 4is a diagram showing an example of a scenario in which the operation of the vehicle control device according to the embodiment is performed. In Figure 4 the vehicle M is traveling on the lane LI, and a preceding vehicle Ml is traveling in front of the vehicle M. While the vehicle M is traveling on the lane LI, the recognition unit 130 recognizes the situation around the vehicle M, particularly the road division lines on both sides of the vehicle M, based on the image captured by the camera 10. Hereinafter, the road division line recognized based on the image captured by the camera 10 will be referred to as "camera road division line CL" (hereinafter, referred to as "camera road division line CL"), and the road division line recognized based on the second map information 62 will be referred to as "map road division line ML" (hereinafter, referred to as "map road division line ML").

[0064] The abnormality determination unit 152 determines whether the camera road division line CL or the map road division line ML has an abnormality, and determines whether the preceding vehicle Ml exists when it is determined that the abnormality exists. Here, the abnormality means that there is a deviation between the camera road division line CL and the map road division line ML, or the camera road division line CL disappears (is lost) due to a bad condition of the camera 10 or an external environment (backlight, etc.). The abnormality determination unit 152 determines that there is a deviation when the distance between the camera road division line CL and the map road division line ML is equal to or greater than a predetermined value, or the angle formed by the camera road division line CL and the map road division line ML is equal to or greater than a predetermined value, for example. The abnormality determination unit 152 determines whether the preceding vehicle Ml exists within a predetermined distance from the vehicle M when it is determined that the camera road division line CL or the map road division line ML has an abnormality.

[0065] The excess degree calculation unit 154 calculates the excess degree for the vehicle M to continue traveling in the driving mode of the mode B based on the distance from the vehicle M to the preceding vehicle Ml, the distance from the preceding vehicle Ml to the point at which the preceding vehicle Ml crosses the camera road division line CL or the map road division line ML, and the speed of the vehicle M when it is determined that the preceding vehicle Ml exists within the predetermined distance from the vehicle M. The mode determination unit 150 causes the vehicle M to continue traveling in the driving mode of the mode B in accordance with the excess degree calculated by the excess degree calculation unit 154.

[0066] More specifically, for example, in the case of the scenario shown in Figure 4 the abnormality determination unit 152 determines that the deviation occurs on both sides of the camera road division line CL and the map road division line ML (i.e., both the left road division line and the right road division line). Generally, it is assumed that the camera road division line CL has higher reliability than the map road division line ML, and thus the excess degree calculation unit 154 preferentially uses the camera road division line CL to calculate the excess degree in this case.

[0067] The excess degree calculating section 154 first determines the intersection IP between the extension line of the track of the preceding vehicle Ml and the camera road division line CL. The excess degree calculating section 154 then calculates the excess degree P by P = (Dl + D2) / V [sec] based on the distance Dl from the host vehicle M to the preceding vehicle Ml, the distance D2 from the preceding vehicle Ml to the line PL (a line passing through the intersection IP and perpendicular to the camera road division line CL), and the speed V of the host vehicle M. That is, the excess degree P represents the period during which there is a margin for the host vehicle M to continue the driving mode of the mode B at least until reaching the line PL.

[0068] The mode deciding section 150 uses the camera road division line CL as the reference line and causes the continuation of the travel in the driving mode of the mode B for the period shown by the excess degree P. The mode deciding section 150 changes the driving mode from the mode B to the mode C after the continuation of the travel in the driving mode of the mode B for the period shown by the excess degree P.

[0069] Figure 5 is a diagram representing another example of a scenario in which the action of the vehicle control device of the embodiment is performed. As with the scenario of Figure 4 , in the scenario of Figure 5 , the abnormality determining section 152 determines that a deviation has occurred on both sides of the camera road division line CL and the map road division line ML. At this time, the abnormality determining section 152 also determines whether or not the state in which the camera road division line CL on both sides deviates at the same angle and at a curvature change rate of the threshold value or more with respect to the map road division line ML continues for a prescribed period.

[0070] The mode deciding section 150 determines that the camera road division line CL is misrecognized in a case where it is determined that the state in which the camera road division line CL on both sides deviates at the same angle and at a curvature change rate of the threshold value or more with respect to the map road division line ML continues for the prescribed period. In this case, the mode deciding section 150 changes the driving mode of the mode B to the driving mode of the mode C with the map road division line ML as the reference line. At this time, the mode deciding section 150 can also cause the driving mode of the mode B to continue temporarily (for example, until reaching the line PL) with the map road division line ML as the reference line, but generally, the reliability of the map road division line ML is assumed to be lower than the reliability of the camera road division line CL, and thus the driving mode of the mode B with the map road division line ML as the reference line is caused to continue only for a short period as compared with the case of Figure 4

[0071] Figure 6 is a diagram representing another example of a scenario in which the action of the vehicle control device of the embodiment is performed. As with the scenario of Figure 6 , in the scenario of Figure 6 ​the left side) is generated. In this case, the mode decision unit 150 continues the travel in the driving mode of mode B with the road division line (i.e., the road division line on the right side) on the other side where no deviation is generated as a reference line. Figure 6

[0072] Figure 7 is a diagram representing another example of a scenario in which the action of the vehicle control device of the embodiment is performed. Figure 7 For example, it represents a situation in which both sides of the camera road division line CL are lost due to a bad condition of the camera 10 and only the map road division line ML can be normally acquired. At this time, the abnormality determination unit 152 determines that both sides of the camera road division line CL are lost, and the excess degree calculation unit 154 determines the intersection IP between the extension line of the track of the preceding vehicle Ml and the map road division line ML. Next, the excess degree calculation unit 154 calculates the excess degree P by P = (Dl + D2) / V [sec] based on the distance Dl from the host vehicle M to the preceding vehicle Ml, the distance D2 from the preceding vehicle Ml to the line PL (a line passing through the intersection IP and perpendicular to the camera road division line CL), and the speed V of the host vehicle M. The mode decision unit 150 continues the travel in the driving mode of mode B for a period indicated by the excess degree P using the map road division line ML as a reference line. The mode decision unit 150 stages the change of the driving mode from mode B to mode C, from mode C to mode D, or mode E after the period indicated by the excess degree P in which the travel in the driving mode of mode B is continued. This is because, as described above, generally, it is assumed that the reliability of the map road division line ML is lower than that of the camera road division line CL, and thus it is not desirable to continue the automatic driving with the map road division line ML as a reference line.

[0073] Figure 8 is a diagram representing another example of a scenario in which the action of the vehicle control device of the embodiment is performed. Figure 8 represents a situation in which one side (i.e., the camera road division line CL on the left side) of the camera road division line CL is lost and the other side of the camera road division line CL generates a misrecognition. The abnormality determination unit 152 determines that the camera road division line CL generates a misrecognition, for example, in a case where the camera road division line CL is discontinuously shifted laterally. In Figure 8 In this case, the camera road division line CL on the right side is discontinuously shifted to the right side by ΔY. Therefore, the abnormality determination unit 152 determines that the camera road division line CL on the right side generates a misrecognition.

[0074] In a case where it is determined by the abnormality determination unit 152 that one side of the camera road division line CL is lost and the other side of the camera road division line CL generates a misrecognition, the excess degree calculation unit 154 calculates the excess degree P based on the distance Dl from the host vehicle M to the preceding vehicle Ml, the distance D2 from the preceding vehicle Ml to the line PL (a line passing through the intersection IP and perpendicular to the camera road division line CL), and the speed V of the host vehicle M. The mode decision unit 150 continues the travel in the driving mode of mode B for a period indicated by the excess degree P using the map road division line ML as a reference line. Figure 7 ​In the same manner, the excess degree calculating section 154 determines the intersection IP between the extension line of the track of the preceding vehicle Ml and the map road division line ML, and calculates the excess degree P. The mode deciding section 150 uses the map road division line ML as the reference line, and continues the travel in the driving mode of the mode B for a period indicated by the excess degree P. The mode deciding section 150 stages the change of the driving mode from the mode B to the mode C, from the mode C to the mode D, or the mode E after the continuation of the travel in the driving mode of the mode B for the period indicated by the excess degree P.

[0075] Next, the flow of the operation performed by the vehicle control device of the embodiment will be described with reference to Figure 9 Figure 9 is a flowchart showing an example of the flow of the operation performed by the vehicle control device of the embodiment. The processing of the present flowchart is performed in a predetermined cycle during the travel of the host vehicle M in the driving mode of the mode B.

[0076] First, the mode deciding section 150 acquires the camera road division line CL and the map road division line ML via the recognition section 130 (step S100). Next, the abnormality determining section 152 determines whether at least one of the camera road division lines CL is missing (step S101).

[0077] In the case where it is determined that at least one of the camera road division lines CL is missing, the processing proceeds to the step S200 described later. On the other hand, in the case where it is not determined that at least one of the camera road division lines CL is missing, the abnormality determining section 152 next determines whether there is a deviation between the camera road division line CL and the map road division line ML (step S102).

[0078] In the case where it is determined that there is no deviation between the camera road division line CL and the map road division line ML, the processing returns to the step S100. On the other hand, in the case where it is determined that there is a deviation between the camera road division line CL and the map road division line ML, the abnormality determining section 152 next determines whether there is a preceding vehicle Ml in front of the host vehicle M (step S103). In the case where it is determined that there is no preceding vehicle Ml in front of the host vehicle M, the mode deciding section 150 changes the driving mode of the mode B to a driving mode of a higher task (the mode C, the mode D, or the mode E) (step S106).

[0079] ​On the other hand, in a case where it is determined that the preceding vehicle Ml exists in front of the host vehicle M, the margin calculation portion 154 calculates the margin P based on the distance from the host vehicle M to the preceding vehicle Ml, the distance from the preceding vehicle Ml to the point at which the preceding vehicle Ml crosses the camera road division line CL, and the speed of the host vehicle M (step S104). Next, the mode determination portion 150 causes the host vehicle M to travel for a period indicated by the calculated margin P in the driving mode of mode B (step S105). Thereby, the processing of the present flowchart ends.

[0080] Figure 10 is a flowchart of an example of a flow of actions performed by the vehicle control device when the camera road division line CL is lost. In Figure 9 In a case where it is determined in step S101 of the flowchart of Fig. 10 that at least one of the camera road division lines CL is lost, the abnormality determination portion 152 determines whether both sides of the camera road division line CL are lost (step S200). In a case where it is determined that both sides of the camera road division line CL are not lost, i.e., only one side of the camera road division line CL is lost, next, the abnormality determination portion 152 determines whether misrecognition has occurred due to deviation occurring in the camera road division line CL that is not lost (step S201).

[0081] In a case where it is determined that misrecognition has not occurred due to deviation occurring in the camera road division line CL that is not lost, the mode determination portion 150 continues the driving mode of mode B with the camera road division line CL that is not lost as the reference line (step S202). On the other hand, in a case where it is determined that both sides of the camera road division line CL are lost, or misrecognition has occurred due to deviation occurring in the camera road division line CL that is not lost, the abnormality determination portion 152 next determines whether the preceding vehicle Ml exists in front of the host vehicle M (step S203). In a case where it is determined that the preceding vehicle Ml does not exist in front of the host vehicle M, the mode determination portion 150 changes the driving mode of mode B to a driving mode (mode C, mode D, or mode E) having a heavier task (step S204).

[0082] On the other hand, in a case where it is determined that the preceding vehicle Ml exists in front of the host vehicle M, the margin calculation portion 154 calculates the margin P based on the distance from the host vehicle M to the preceding vehicle Ml, the distance from the preceding vehicle Ml to the point at which the preceding vehicle Ml crosses the map road division line ML, and the speed of the host vehicle M (step S205). Next, the mode determination portion 150 causes the host vehicle M to travel for a period indicated by the calculated margin P in the driving mode of mode B (step S206). Thereby, the processing of the present flowchart ends.

[0083] According to the embodiment as described above, in a case where a deviation occurs between the camera road division line and the map road division line and a preceding vehicle exists in front of the host vehicle, the degree of freedom for continuing the automatic driving is calculated using one of the camera road division line and the map road division line, the position and the speed of the host vehicle, and the position of the preceding vehicle, and the automatic driving of the host vehicle is continued in accordance with the calculated degree of freedom. Thus, even in a case where the road division line recognized by the camera is different from the content of the map information mounted on the host vehicle, the driving control of the vehicle can be flexibly changed.

[0084] The above-described embodiment can be expressed as follows.

[0085] A vehicle control device is configured to include:

[0086] a storage device in which a program is stored; and

[0087] a hardware processor,

[0088] the program stored in the storage device is executed by the hardware processor to perform the following processing:

[0089] acquire a camera image obtained by photographing a surrounding situation of a vehicle;

[0090] control steering and acceleration / deceleration of the vehicle based on the camera image and map information, without depending on an operation of a driver of the vehicle;

[0091] determine a driving mode of the vehicle to be any one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task arranged to the driver is lighter than the first driving mode, at least a part of the plurality of driving modes including the second driving mode being performed by controlling the steering and the acceleration / deceleration of the vehicle without depending on the operation of the driver of the vehicle, and in a case where a task related to the determined driving mode is not performed by the driver, change the driving mode of the vehicle to a driving mode in which a task is heavier;

[0092] determine whether or not there is a deviation between a road division line shown in the camera image and a road division line shown in the map information, and in a case where it is determined that there is the deviation, determine whether or not there is a preceding vehicle of the vehicle;

[0093] In a case where it is determined that the preceding vehicle exists, a margin for the vehicle to continue traveling in the second driving mode is calculated based on a distance from the vehicle to the preceding vehicle, a distance from the preceding vehicle to a point at which the preceding vehicle intersects a road division line shown in the camera image or a road division line shown in the map information, and a speed of the vehicle; and

[0094] Continuation of the vehicle traveling in the second driving mode is made for a period indicated by the margin.

[0095] The above describes specific embodiments of the present application using the embodiments, but the present application is by no means limited to such embodiments, and various modifications and substitutions can be made within the scope of the gist of the present application.

Claims

1. A vehicle control device, wherein the vehicle control device is provided with: an acquisition unit that acquires a camera image obtained by photographing a surrounding situation of a vehicle; a driving control unit that controls steering and acceleration / deceleration of the vehicle based on the camera image and map information, without depending on an operation of a driver of the vehicle; a mode determination unit that determines a driving mode of the vehicle to be any one of a plurality of driving modes including a first driving mode and a second driving mode that is a driving mode in which a task imposed on the driver is lighter than the first driving mode, at least a part of the plurality of driving modes including the second driving mode being controlled by the driving control unit, the mode determination unit changing the driving mode of the vehicle to a driving mode in which a task is heavier than the determined driving mode in a case where the task involved in the determined driving mode is not performed by the driver; a determination unit that determines whether or not an abnormality occurs in a road division line shown in the camera image or a road division line shown in the map information, determines whether or not there is a preceding vehicle of the vehicle in a case where it is determined that the abnormality occurs; and an arithmetic unit that, in a case where it is determined that there is the preceding vehicle, calculates a margin for the vehicle to continue driving in the second driving mode, based on a distance from the vehicle to the preceding vehicle, a distance from the preceding vehicle to a point at which the preceding vehicle crosses the road division line shown in the camera image or the road division line shown in the map information, and a speed of the vehicle, the mode determination unit causing the vehicle to continue driving in the second driving mode for a period indicated by the margin, the arithmetic unit calculating the margin by dividing a distance obtained by adding the distance from the vehicle to the preceding vehicle to the distance from the preceding vehicle to the point at which the preceding vehicle crosses the road division line shown in the camera image or the road division line shown in the map information, by the speed of the vehicle.

2. The vehicle control device according to claim 1, wherein in a case where it is determined by the determination unit that there is a deviation on both sides of the road division line shown in the camera image and the road division line shown in the map information as the abnormality, the mode determination unit causes driving in the second driving mode to continue for a period indicated by the margin with the road division line shown in the camera image serving as a reference line, and changes the second driving mode to the first driving mode after the period elapses.

3. The vehicle control device according to claim 1, wherein the determination unit, in a case where it is determined that there is a deviation on both sides of the road division line shown in the camera image and the road division line shown in the map information as the abnormality, determines whether or not a state in which the road division lines on both sides shown in the camera image deviate at the same angle and at a curvature change rate of a threshold value or more with respect to the road division lines on both sides shown in the map information continues for a prescribed period, ​ the determination unit determines that the road division line shown in the camera image is misrecognized, the mode determination unit changes the second driving mode to the first driving mode using the road division line shown in the map information, in a case where the determination unit determines that the road division line shown in the camera image is misrecognized.

4. The vehicle control device according to claim 1, wherein in a case where the determination unit determines that the deviation exists only on one side of the road division line shown in the camera image and the road division line shown in the map information as the abnormality, the mode determination unit continues the travel in the second driving mode using the road division line on which the deviation does not occur as a reference line.

5. The vehicle control device according to claim 1, wherein in a case where the determination unit determines that both sides of the road division line shown in the camera image disappear as the abnormality, the mode determination unit continues the travel in the second driving mode using the road division line shown in the map information as a reference line for a period indicated by the margin, and after the period elapses, changes the second driving mode to the manual driving stage via the first driving mode.

6. The vehicle control device according to claim 1, wherein in a case where the determination unit determines that one side of the road division line shown in the camera image disappears and the other side of the road division line shown in the camera image is misrecognized due to the deviation as the abnormality, the mode determination unit continues the travel in the second driving mode using the road division line shown in the map information as a reference line for a period indicated by the margin, and after the period elapses, changes the second driving mode to the manual driving stage via the first driving mode.

7. The vehicle control device according to any one of claims 1 to 6, wherein the second driving mode is a driving mode in which the driver is not arranged to hold an operation member that accepts a steering operation of the vehicle, the first driving mode is a driving mode in which the driver is arranged to hold at least only the operation member.

8. A vehicle control method, wherein the vehicle control method causes a computer to perform the following processing: acquire a camera image obtained by capturing a surrounding situation of a vehicle; control a steering and acceleration / deceleration of the vehicle based on the camera image and map information, without depending on an operation of a driver of the vehicle. determining a driving mode of the vehicle to be any one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task arranged to the driver is lighter than the first driving mode, at least a part of the plurality of driving modes including the second driving mode being performed by controlling steering and acceleration / deceleration of the vehicle without depending on an operation of the driver of the vehicle, changing the driving mode of the vehicle to a driving mode in which a task is heavier in a case where a task involved in the determined driving mode is not performed by the driver; determining whether an abnormality occurs in a road division line shown in the camera image or a road division line shown in the map information, and determining whether a preceding vehicle exists in the vehicle in a case where it is determined that the abnormality occurs; calculating a margin for the vehicle to continue driving in the second driving mode, based on a distance from the vehicle to the preceding vehicle, a distance from the preceding vehicle to a point at which the preceding vehicle crosses the road division line shown in the camera image or the road division line shown in the map information, and a speed of the vehicle in a case where it is determined that the preceding vehicle exists; continuing driving of the vehicle in the second driving mode for a period indicated by the margin, calculating the margin by dividing a distance obtained by adding the distance from the vehicle to the preceding vehicle to the distance from the preceding vehicle to the point at which the preceding vehicle crosses the road division line shown in the camera image or the road division line shown in the map information by the speed of the vehicle.

9. A storage medium storing a program, wherein the program causes a computer to perform the following processing: acquiring a camera image obtained by capturing a surrounding situation of a vehicle; controlling steering and acceleration / deceleration of the vehicle without depending on an operation of a driver of the vehicle, based on the camera image and map information; determining a driving mode of the vehicle to be any one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task arranged to the driver is lighter than the first driving mode, at least a part of the plurality of driving modes including the second driving mode being performed by controlling steering and acceleration / deceleration of the vehicle without depending on an operation of the driver of the vehicle, changing the driving mode of the vehicle to a driving mode in which a task is heavier in a case where a task involved in the determined driving mode is not performed by the driver; determining whether an abnormality occurs in a road division line shown in the camera image or a road division line shown in the map information, and determining whether a preceding vehicle exists in the vehicle in a case where it is determined that the abnormality occurs; In a case where it is determined that the preceding vehicle exists, a margin for the vehicle to continue traveling in the second driving mode is calculated based on a distance from the vehicle to the preceding vehicle, a distance from the preceding vehicle to a point at which the preceding vehicle intersects with a road division line shown in the camera image or a road division line shown in the map information, and a speed of the vehicle; the vehicle is caused to continue traveling in the second driving mode for a period indicated by the margin, the margin is calculated by dividing a distance obtained by adding the distance from the vehicle to the preceding vehicle to the distance from the preceding vehicle to the point at which the preceding vehicle intersects with the road division line shown in the camera image or the road division line shown in the map information by the speed of the vehicle.

10. A vehicle control device in which the vehicle control device includes: an acquisition unit that acquires a camera image obtained by capturing a surrounding situation of a vehicle; a driving control unit that controls steering and acceleration / deceleration of the vehicle based on the camera image and map information, without depending on an operation of a driver of the vehicle; a mode determination unit that determines a driving mode of the vehicle to be any one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task imposed on the driver is lighter than in the first driving mode, at least a part of the plurality of driving modes including the second driving mode being controlled by the driving control unit, the mode determination unit changing the driving mode of the vehicle to a driving mode in which a task is heavier than in the determined driving mode in a case where the task involved in the determined driving mode is not performed by the driver; a determination unit that determines whether an abnormality occurs in a road division line shown in the camera image or a road division line shown in the map information, and determines whether a preceding vehicle exists in a case where it is determined that the abnormality occurs; and a calculation unit that calculates a margin for the vehicle to continue traveling in the second driving mode based on a distance from the vehicle to the preceding vehicle, a distance from the preceding vehicle to a point at which the preceding vehicle intersects with the road division line shown in the camera image or the road division line shown in the map information, and a speed of the vehicle in a case where it is determined that the preceding vehicle exists, the mode determination unit causes the vehicle to continue traveling in the second driving mode for a period indicated by the margin, the calculation unit calculates the margin based on the distance from the preceding vehicle to the point at which the preceding vehicle intersects with the road division line shown in the map information in a case where it is determined by the determination unit that the abnormality involves disappearance of at least one of the road division lines shown in the camera image, and calculates the margin based on the distance from the preceding vehicle to the point at which the preceding vehicle intersects with the road division line shown in the camera image in a case where it is determined by the determination unit that the abnormality does not involve disappearance of at least one of the road division lines shown in the camera image.

Images