Vehicle control device, vehicle control method, and program

Abstract

To appropriately change driving control of a vehicle even in a case where a road lane marking is different from a content of map information installed in the own vehicle.SOLUTION: The present invention relates to a vehicle control device comprising: an acquisition section for acquiring a camera image picking up a peripheral situation of a vehicle; a driving control section for controlling steering and acceleration / deceleration of the vehicle without depending on operation of a driver of the vehicle, based on the camera image and map information; a mode determination section for determining, as a driving mode of the vehicle, one of a plurality of driving modes including a first driving mode and a second driving mode; a deviation determination section for determining the presence / absence of deviation between a road lane marking indicated in the camera image and a road lane marking indicated in the map information; and a parallelism calculation section for calculating parallelism between a trajectory of one or more other vehicles existing around the vehicle and the road lane marking indicated in the camera image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a program. [Background technology]

[0002] Conventionally, there is known a technique for controlling the driving of a vehicle based on road dividing lines recognized by a camera mounted on the vehicle. For example, Patent Document 1 describes a technique for driving a vehicle based on recognized road dividing lines, and if the recognition degree of the road dividing lines does not satisfy a predetermined standard, for driving the vehicle based on the trajectory of a preceding vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2020-050086 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 controls the driving of a vehicle based on road dividing lines recognized by a camera and map information installed in the vehicle. However, with the conventional technology, if the road dividing lines recognized by the camera differ from the content of the map information installed in the vehicle, the vehicle's driving control may not be changed appropriately.

[0005] The present invention has been made taking these circumstances into consideration, and one of its objectives is to provide a vehicle control device, a vehicle control method, and a program that can appropriately change the vehicle's driving control even if the road dividing lines recognized by the camera differ from the contents of the map information installed in the vehicle. [Means for solving the problem]

[0006] A vehicle control device, a vehicle control method, and a program according to the present invention employ the following configuration. (1) A vehicle control device according to one aspect of the present invention includes an acquisition unit that acquires camera images of a vehicle's surroundings, a driving control unit that controls the steering and acceleration / deceleration of the vehicle based on the camera images and map information without relying on the operation of the driver of the vehicle, and a driving control unit that determines a driving mode of the vehicle to one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task imposed on the driver is lighter than that of the first driving mode, at least some of the plurality of driving modes including the second driving mode being controlled by the driving control unit, and the task related to the determined driving mode being less difficult for the driver to perform. a mode determination unit that changes the driving mode of the vehicle to a driving mode with a heavier task if the driving mode is not executed by the camera image; a deviation determination unit that determines whether or not there is a deviation between the road dividing line shown in the camera image and the road dividing line shown in the map information; and a parallelism calculation unit that calculates the parallelism between the trajectories of one or more other vehicles present around the vehicle and the road dividing line shown in the camera image if it is determined that there is a deviation between the road dividing line shown in the camera image and the road dividing line shown in the map information, and the mode determination unit determines to continue the second driving mode if the calculated parallelism is equal to or greater than a first threshold.

[0007] (2): In the above aspect (1), the parallelism calculation unit calculates the parallelism when the number of one or more other vehicles present around the vehicle is equal to or greater than a second threshold value.

[0008] (3): In the above aspect (1) or (2), the parallelism calculation unit calculates the parallelism based on the trajectories of one or more other vehicles present around the vehicle, the road dividing lines shown in the camera image, and the road dividing lines shown in the map information.

[0009] (4): In the above aspect (3), the parallelism calculation unit calculates the parallelism based on the average value of the angle between each of the trajectories of one or more other vehicles present around the vehicle and the road dividing line shown in the camera image, and the average value of the angle between each of the trajectories of one or more other vehicles present around the vehicle and the road dividing line shown in the map information.

[0010] (5): In the above aspect (3), the parallelism calculation unit calculates the parallelism based on the median of the angle between each of the trajectories of one or more other vehicles present around the vehicle and the road dividing line shown in the camera image, and the median of the angle between each of the trajectories of one or more other vehicles present around the vehicle and the road dividing line shown in the map information.

[0011] (6): In any of the above aspects (1) to (5), the second driving mode is a driving mode in which the driver is not tasked with holding an operator that receives steering operations of the vehicle, and the first driving mode is a driving mode in which the driver is tasked with at least holding the operator.

[0012] (7): A vehicle control method according to another aspect of the present invention includes a computer acquiring a camera image capturing a surrounding situation of a vehicle, controlling steering and acceleration / deceleration of the vehicle based on the camera image and map information without relying on an operation by a driver of the vehicle, and determining a driving mode of the vehicle to one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task imposed on the driver is lighter than that of the first driving mode, and controlling steering and acceleration / deceleration of the vehicle without relying on an operation by the driver of the vehicle, and determining the determined driving mode. If the driver does not perform a task related to the mode, the driving mode of the vehicle is changed to a driving mode with a heavier task, and it is determined whether or not there is a discrepancy between the road dividing lines shown in the camera image and the road dividing lines shown in the map information. If it is determined that there is a discrepancy between the road dividing lines shown in the camera image and the road dividing lines shown in the map information, the parallelism between the trajectories of one or more other vehicles present around the vehicle and the road dividing lines shown in the camera image is calculated, and if the calculated parallelism is equal to or greater than a first threshold, it is determined to continue the second driving mode.

[0013] (8) A program according to another aspect of the present invention causes a computer to acquire a camera image capturing a surrounding situation of a vehicle, and controls the steering and acceleration / deceleration of the vehicle based on the camera image and map information without relying on an operation by a driver of the vehicle, and determines a driving mode of the vehicle to one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task imposed on the driver is lighter than that of the first driving mode, and the second driving mode is performed by controlling the steering and acceleration / deceleration of the vehicle without relying on an operation by the driver of the vehicle, and the determined driving mode If the driver does not perform a task related to the second driving mode, the driving mode of the vehicle is changed to a driving mode with a heavier task, and it is determined whether or not there is a discrepancy between the road dividing lines shown in the camera image and the road dividing lines shown in the map information. If it is determined that there is a discrepancy between the road dividing lines shown in the camera image and the road dividing lines shown in the map information, the parallelism between the trajectories of one or more other vehicles present around the vehicle and the road dividing lines shown in the camera image is calculated, and if the calculated parallelism is equal to or greater than a first threshold, it is determined to continue the second driving mode. [Effects of the Invention]

[0014] According to (1) to (8), even if the road dividing lines recognized by the camera differ from the contents of the map information installed in the vehicle, the driving control of the vehicle can be appropriately changed. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a configuration diagram of a vehicle system 1 that uses a vehicle control device according to an embodiment. [Figure 2] 2 is a functional configuration diagram of a first control unit 120 and a second control unit 160. FIG. [Figure 3] 10 is a diagram showing an example of the correspondence between the driving mode, the control state of the host vehicle M, and the task. FIG. [Figure 4] FIG. 2 is a diagram illustrating an example of a scene in which the operation of the vehicle control device according to the embodiment is performed. [Figure 5] 10 is a graph illustrating a method for determining an operation mode based on parallelism. [Figure 6] FIG. 10 is a diagram illustrating an example of another vehicle that is excluded from calculation targets for calculating parallelism. [Figure 7] 4 is a flowchart illustrating an example of a flow of an operation executed by the vehicle control device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of a vehicle control device, a vehicle control method, and a program according to the present invention will be described with reference to the drawings.

[0017] [Overall configuration] 1 is a configuration diagram of a vehicle system 1 that uses a vehicle control device according to an embodiment. The vehicle on which the vehicle system 1 is mounted may be, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source may be an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination of these. The electric motor operates using power generated by a generator connected to the internal combustion engine, or discharged power from a secondary battery or a fuel cell.

[0018] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, vehicle sensors 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driver monitor camera 70, a driving operator 80, an automatic driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are connected to each other via multiplexed communication lines such as a CAN (Controller Area Network) communication line, serial communication lines, a wireless communication network, etc. Note that the configuration shown in FIG. 1 is merely an example, and some of the configuration may be omitted, or other configurations may be added.

[0019] The camera 10 is, for example, a digital camera using a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to any location of a vehicle (hereinafter referred to as the host vehicle M) in which the vehicle system 1 is installed. When capturing an image of the front, the camera 10 is attached to the top of the front windshield, the back of the rearview mirror, or the like. The camera 10, for example, periodically and repeatedly captures images of the surroundings of the host vehicle M. The camera 10 may be a stereo camera.

[0020] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by an object (reflected waves) to detect at least the position (distance and direction) of the object. The radar device 12 is attached to any location on the vehicle M. The radar device 12 may detect the position and speed of an object using an FM-CW (Frequency Modulated Continuous Wave) method.

[0021] The LIDAR 14 irradiates the surroundings of the vehicle M with light (or electromagnetic waves with wavelengths similar to light) and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time between light emission and light reception. The irradiated light is, for example, pulsed laser light. The LIDAR 14 is attached to any location on the vehicle M.

[0022] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the camera 10, the radar device 12, and the LIDAR 14 to recognize the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition results to the automatic driving control device 100. The object recognition device 16 may output the detection results from the camera 10, the radar device 12, and the LIDAR 14 directly to the automatic driving control device 100. The object recognition device 16 may be omitted from the vehicle system 1.

[0023] The communication device 20 communicates with other vehicles in the vicinity of the vehicle M, for example, using a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), etc., or communicates with various server devices via a wireless base station.

[0024] The HMI 30 presents various information to the occupants of the vehicle M and accepts input operations by the occupants. The HMI 30 includes various display devices, a speaker, a buzzer, a touch panel, switches, keys, and the like.

[0025] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the host vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects angular velocity around a vertical axis, a direction sensor that detects the direction of the host vehicle M, and the like.

[0026] The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as a hard disk drive (HDD) or flash memory. The GNSS receiver 51 identifies the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be identified or supplemented by an inertial navigation system (INS) that uses the output of the vehicle sensors 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, etc. The navigation HMI 52 may share some or all of the components with the HMI 30 described above. The route determination unit 53 determines, for example, a route (hereinafter, a map route) from the position of the vehicle M identified by the GNSS receiver 51 (or any input position) to a destination input by the occupant using the navigation HMI 52, with reference to the first map information 54. The first map information 54 is information that represents road shapes using, for example, links indicating roads and nodes connected by the links. The first map information 54 may also include information such as road curvature and POI (Point of Interest) information. The route on the map is output to the MPU 60. The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be realized, for example, by the functions of a terminal device such as a smartphone or tablet device owned by the occupant. The navigation device 50 may transmit the current position and destination to a navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server.

[0027] The MPU 60 includes, for example, a recommended lane determination unit 61, and stores second map information 62 in a storage device such as an HDD or flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, by dividing it into 100 m intervals in the vehicle travel direction), and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 determines, for example, which lane from the left the vehicle should travel in. When there is a branch point on the route on the map, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel on a reasonable route to the branch point.

[0028] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on the centers of lanes or information on lane boundaries. The second map information 62 may also include road information, traffic regulation information, address information (address and postal code), facility information, telephone number information, information on prohibited sections where mode A or mode B, described below, is prohibited, and the like. The second map information 62 may be updated as needed by the communication device 20 communicating with another device.

[0029] The driver monitor camera 70 is, for example, a digital camera that uses a solid-state imaging element such as a CCD or CMOS. The driver monitor camera 70 is attached to any location on the vehicle M in a position and orientation that allows it to capture an image of the head of an occupant (hereinafter, driver) seated in the driver's seat of the vehicle M from the front (in an orientation that captures an image of the face). For example, the driver monitor camera 70 is attached to the top of a display device provided in the center of the instrument panel of the vehicle M.

[0030] The driving operators 80 include, for example, a steering wheel 82, an accelerator pedal, a brake pedal, a shift lever, and other operators. The driving operators 80 are equipped with sensors that detect the amount of operation or the presence or absence of operation, and the detection results are output to the automatic driving control device 100 or some or all of the driving force output device 200, the brake device 210, and the steering device 220. The steering wheel 82 is an example of an "operator that accepts steering operation by the driver." The operator does not necessarily have to be annular and may be in the form of an irregular steering wheel, a joystick, a button, or the like. A steering grip sensor 84 is attached to the steering wheel 82. The steering grip sensor 84 is realized by a capacitance sensor or the like, and outputs a signal to the automatic driving control device 100 that can detect whether the driver is gripping the steering wheel 82 (meaning that the driver is in contact with the steering wheel in a state where force can be applied).

[0031] The automatic driving control device 100 includes, for example, a first control unit 120 and a second control unit 160. The first control unit 120 and the second control unit 160 are each realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transitory storage medium) such as a HDD or flash memory of the automatic driving control device 100, or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the automatic driving control device 100 by inserting the storage medium (non-transitory storage medium) into a drive device. The automatic driving control device 100 is an example of a "vehicle control device", and the action plan generation unit 140 and the second control unit 160 together are an example of a "driving control unit".

[0032] FIG. 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130, an action plan generation unit 140, and a mode determination unit 150. The first control unit 120, for example, implements functions based on AI (Artificial Intelligence) and functions based on a predefined model in parallel. For example, the function of "recognizing intersections" may be implemented by executing intersection recognition using deep learning or the like and recognition based on predefined conditions (such as traffic lights and road signs that can be pattern-matched) in parallel, and then scoring and comprehensively evaluating both. This ensures the reliability of autonomous driving.

[0033] The recognition unit 130 recognizes the position, speed, acceleration, and other states of objects around the vehicle M based on information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16. The position of an object is recognized as a position on an absolute coordinate system with a representative point of the vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or may be represented by an area. The "state" of an object may include the acceleration or jerk of the object, or the "behavioral state" (for example, whether the object is changing lanes or is about to change lanes).

[0034] The recognition unit 130 also recognizes, for example, the lane in which the host vehicle M is traveling (driving lane). For example, the recognition unit 130 recognizes the driving lane by comparing the pattern of road dividing lines (e.g., an arrangement of solid lines and dashed lines) obtained from the second map information 62 with the pattern of road dividing lines around the host vehicle M recognized from an image captured by the camera 10. Note that the recognition unit 130 may recognize the driving lane by recognizing road boundaries (road boundaries) including not only road dividing lines but also road dividing lines, shoulders, curbs, medians, guardrails, etc. In this recognition, the position of the host vehicle M obtained from the navigation device 50 and the processing results by the INS may be taken into consideration. The recognition unit 130 also recognizes stop lines, obstacles, red lights, toll booths, and other road phenomena.

[0035] When recognizing the driving lane, the recognition unit 130 recognizes the position and attitude of the host vehicle M with respect to the driving lane. For example, the recognition unit 130 may recognize the deviation of the reference point of the host vehicle M from the center of the lane and the angle it forms with a line connecting the centers of the lanes in the traveling direction of the host vehicle M as the relative position and attitude of the host vehicle M with respect to the driving lane. Alternatively, the recognition unit 130 may recognize the position of the reference point of the host vehicle M with respect to either side edge of the driving lane (a road dividing line or a road boundary) as the relative position of the host vehicle M with respect to the driving lane.

[0036] The behavior plan generation unit 140 generates a target trajectory along which the host vehicle M will travel in the future automatically (without relying on the driver's operation) so that the host vehicle M will, in principle, travel along the recommended lane determined by the recommended lane determination unit 61 and can respond to the surrounding conditions of the host vehicle M. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequence of points (trajectory points) that the host vehicle M should reach. The trajectory points are points that the host vehicle M should reach at every predetermined travel distance (for example, about several meters) along the road. Separately, target speeds and target accelerations are generated as part of the target trajectory for every predetermined sampling time (for example, about a few tenths of a second). Furthermore, the trajectory points may be positions that the host vehicle M should reach at each sampling time for each predetermined sampling time. In this case, information on the target speed and target acceleration is expressed as the interval between trajectory points.

[0037] The behavior plan generation unit 140 may set an autonomous driving event when generating the target trajectory. The autonomous driving events include a constant speed driving event, a low-speed following driving event, a lane change event, a branching event, a merging event, a takeover event, etc. The behavior plan generation unit 140 generates a target trajectory according to the activated event.

[0038] The mode determination unit 150 determines the driving mode of the host vehicle M to be one of a plurality of driving modes that impose different tasks on the driver. The mode determination unit 150 includes, for example, a deviation determination unit 152 and a parallelism calculation unit 154. The functions of the deviation determination unit 152 and the parallelism calculation unit 154 will be described later.

[0039] FIG. 3 is a diagram showing an example of the correspondence between driving modes, control states of the host vehicle M, and tasks. The driving modes of the host vehicle M include, for example, five modes, Mode A to Mode E. The control state, i.e., the degree of automation of the driving control of the host vehicle M, is highest in Mode A, followed by Mode B, Mode C, and Mode D, with Mode E being the lowest. Conversely, the tasks imposed on the driver are lightest in Mode A, followed by Mode B, Mode C, and Mode D, with Mode E being the most severe. Note that modes D and E are non-autonomous driving control states, and therefore the autonomous driving control device 100 is responsible for terminating control related to autonomous driving and transitioning to driving assistance or manual driving. The contents of each driving mode are exemplified below.

[0040] In Mode A, the vehicle is in an autonomous driving state, and the driver is not required to monitor the road ahead or grip the steering wheel 82 (in the figure, gripping the steering wheel). However, even in Mode A, the driver is required to be in a position where he or she can quickly switch to manual driving in response to a request from a system centered on the automatic driving control device 100. Note that, as used herein, "automatic driving" refers to control of both steering and acceleration / deceleration without driver input. "Ahead" refers to the space in the direction of travel of the host vehicle M, as viewed through the front windshield. Mode A is a driving mode that can be implemented, for example, on a motorway such as an expressway, when certain conditions are met, such as the host vehicle M traveling at a predetermined speed (e.g., approximately 50 km / h) or less and there is a vehicle ahead to be followed, and is sometimes referred to as TJP (Traffic Jam Pilot). If these conditions are no longer met, the mode determination unit 150 changes the driving mode of the host vehicle M to Mode B.

[0041] In mode B, the vehicle is in a driving assistance state, and the driver is tasked with monitoring the area ahead of the vehicle M (hereinafter referred to as forward monitoring), but is not tasked with holding the steering wheel 82. In mode C, the vehicle is in a driving assistance state, and the driver is tasked with monitoring the area ahead and holding the steering wheel 82. Mode D is a driving mode that requires some degree of driver operation for at least one of steering and acceleration / deceleration of the vehicle M. For example, in mode D, driving assistance such as ACC (Adaptive Cruise Control) and LKAS (Lane Keeping Assist System) is provided. Mode E is a manual driving state in which the driver must perform both steering and acceleration / deceleration operations. In both mode D and mode E, the driver is naturally tasked with monitoring the area ahead of the vehicle M.

[0042] The automatic driving control device 100 (and the driving assistance device (not shown)) executes an automated lane change according to the driving mode. There are two types of automated lane changes: a system-requested automated lane change (1) and a driver-requested automated lane change (2). The automated lane change (1) is an automated lane change for overtaking, which is performed when the speed of the vehicle ahead is slower than the speed of the vehicle itself by a standard or more, and an automated lane change for proceeding toward the destination (an automated lane change due to a change in the recommended lane). The automated lane change (2) is a lane change in which, when conditions related to the speed and the positional relationship with surrounding vehicles are met and the driver operates the turn signal, the vehicle M changes lanes in the direction of the operation.

[0043] In mode A, the automatic driving control device 100 does not perform either automated lane change (1) or (2). In modes B and C, the automatic driving control device 100 performs either automated lane change (1) or (2). In mode D, the driving assistance device (not shown) does not perform automated lane change (1), but performs automated lane change (2). In mode E, neither automated lane change (1) nor (2) is performed.

[0044] When the driver does not perform a task related to the determined driving mode (hereinafter, the current driving mode), the mode determination unit 150 changes the driving mode of the vehicle M to a driving mode with a more severe task.

[0045] For example, in mode A, if the driver is in a position where he or she cannot switch to manual driving in response to a request from the system (for example, if the driver continues to look away from the road outside the allowable area or if a sign of driving difficulty is detected), the mode determination unit 150 uses the HMI 30 to prompt the driver to switch to manual driving, and if the driver does not comply, the mode determination unit 150 performs control such as pulling the host vehicle M to the shoulder of the road and gradually stopping it, and stopping the automatic driving. After the automatic driving is stopped, the host vehicle enters a state of mode D or E, and the host vehicle M can be started by manual operation by the driver. The same applies below to "stopping automatic driving." In mode B, if the driver is not monitoring the road ahead, the mode determination unit 150 performs control such as prompting the driver to monitor the road ahead using the HMI 30, and if the driver does not comply, pulling the host vehicle M to the shoulder of the road and gradually stopping it, and stopping the automatic driving. In mode C, if the driver is not monitoring the road ahead or is not gripping the steering wheel 82, the mode determination unit 150 uses the HMI 30 to prompt the driver to monitor the road ahead and / or grip the steering wheel 82, and if the driver does not comply, the mode determination unit 150 controls the vehicle M to move to the shoulder of the road and gradually stop, thereby terminating automatic driving.

[0046] The second control unit 160 controls the traveling driving force output device 200, the braking device 210, and the steering device 220 so that the host vehicle M passes through the target trajectory generated by the action plan generation unit 140 at the scheduled time.

[0047] Returning to FIG. 2, the second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires information on the target trajectory (trajectory points) generated by the action plan generation unit 140 and stores it in a memory (not shown). The speed control unit 164 controls the driving force output device 200 or the brake device 210 based on a speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized by, for example, a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road ahead of the host vehicle M and feedback control based on the deviation from the target trajectory.

[0048] The driving force output device 200 outputs a driving force (torque) for the vehicle to travel to the drive wheels. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, etc., and an ECU (Electronic Control Unit) that controls these. The ECU controls the above components according to information input from the second control unit 160 or information input from the driving operator 80.

[0049] Braking device 210 may include, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from second control unit 160 or information input from driving operation device 80, so that a brake torque corresponding to the braking operation is output to each wheel. Braking device 210 may include a backup mechanism that transmits hydraulic pressure generated by operation of a brake pedal included in driving operation device 80 to the cylinder via a master cylinder. Note that braking device 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic brake device that controls an actuator according to information input from second control unit 160 to transmit hydraulic pressure from a master cylinder to the cylinder.

[0050] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor applies a force to a rack and pinion mechanism to change the direction of the steered wheels. The steering ECU drives the electric motor in accordance with information input from the second control unit 160 or information input from the driving operator 80 to change the direction of the steered wheels.

[0051] [Vehicle control device operation] Next, the operation of the vehicle control device according to the embodiment will be described. In the following description, it is assumed that the host vehicle M is traveling in the driving mode of mode B. Fig. 4 is a diagram showing an example of a scene in which the operation of the vehicle control device according to the embodiment is executed. In Fig. 4, the host vehicle M is traveling on lane L1, and three other vehicles M1, M2, and M3 are traveling ahead of the host vehicle M.

[0052] While the host vehicle M is traveling in the lane L1, the recognition unit 130 recognizes the surrounding conditions of the host vehicle M, in particular the road dividing lines on both sides of the host vehicle M, based on images captured by the camera 10. Hereinafter, the road dividing lines recognized based on the images captured by the camera 10 will be represented by CL (hereinafter referred to as "camera road dividing lines CL"), and the road dividing lines recognized based on the second map information 62 will be represented by ML (hereinafter referred to as "map road dividing lines ML").

[0053] The deviation determination unit 152 determines whether or not there is a deviation (mismatch) between the camera road-dividing line CL and the map road-dividing line ML while the host vehicle M is traveling. Here, deviation means, for example, that the distance between the camera road-dividing line CL and the map road-dividing line ML is equal to or greater than a predetermined value, or that the angle between the camera road-dividing line CL and the map road-dividing line ML is equal to or greater than a predetermined value.

[0054] If it is determined that there is a deviation between the camera road-dividing line CL and the map road-dividing line ML, the parallelism calculation unit 154 determines whether there are a threshold or more of other vehicles within a predetermined distance from the host vehicle M. If it is determined that there are a threshold or more of other vehicles within a predetermined distance from the host vehicle M, the parallelism calculation unit 154 calculates the parallelism between the trajectories of these other vehicles and the camera road-dividing line CL using the method described below.

[0055] 4, for example, the parallelism calculation unit 154 first calculates the traveling trajectories T1, T2, and T3 for each of the other vehicles M1, M2, and M3. The parallelism calculation unit 154 can calculate the traveling trajectories T1, T2, and T3 by measuring the displacement of the positions of the other vehicles M1, M2, and M3 over a predetermined period of time based on, for example, camera images.

[0056] Next, the parallelism calculation unit 154 calculates the angles between the calculated traveling trajectories T1, T2, and T3 and the camera road dividing line CL and the map road dividing line ML. For example, in the case of FIG. 4, the parallelism calculation unit 154 calculates, for the other vehicle M1, the angle θc1 between the traveling trajectory T1 and the camera road dividing line CL1 (the camera road dividing line CL1 is illustrated by translating the camera road dividing line CL for the sake of convenience in explaining the calculation method. The same applies to CL2 and CL3 below), and the angle θm1 between the traveling trajectory T1 and the map road dividing line ML1 (the map road dividing line ML1 is illustrated by translating the map road dividing line ML for the sake of convenience in explaining the calculation method. The same applies to ML2 and ML3 below). Similarly, the parallelism calculation unit 154 calculates, for the other vehicle M2, the angle θc2 between the traveling trajectory T2 and the camera road dividing line CL2, and the angle θm2 between the traveling trajectory T2 and the map road dividing line ML2. Similarly, the parallelism calculation unit 154 calculates the angle θc3 between the travel trajectory T3 and the camera road dividing line CL3 and the angle θm2 between the travel trajectory T3 and the map road dividing line ML3 for the other vehicle M3. Note that at this time, the parallelism calculation unit 154 calculates the angle as a positive angle in the clockwise direction and a negative angle in the counterclockwise direction, for example, based on the travel trajectory of the other vehicle (this setting may be reversed).

[0057] Next, for the detected other vehicle, the parallelism calculation unit 154 calculates the average value of the angle between the calculated traveling trajectory T1 and the camera road dividing line CL1 and the angle between the traveling trajectory T1 and the map road dividing line ML1. More specifically, in the case of Figure 4, the parallelism calculation unit 154 calculates the average value of the angle between the traveling trajectory T1 and the camera road dividing line CL1 using θc_av = (|θc1| + |θc2| + |θc3|) / 3, and calculates the average value of the angle between the traveling trajectory T1 and the map road dividing line ML1 using θm_av = (|θm1| + |θm2| + |θm3|) / 3.

[0058] Next, the parallelism calculation unit 154 calculates the parallelism between the detected travel trajectory T of the other vehicle and the camera road-dividing line CL using θm_av-θc_av. In other words, the parallelism θm_av-θc_av can be considered an index value that indicates whether the other vehicle is traveling more parallel to the camera road-dividing line CL or the map road-dividing line ML. The larger the value of the parallelism θm_av-θc_av, the more parallel the other vehicle is traveling to the camera road-dividing line CL, and the smaller the value of the parallelism θm_av-θc_av, the more parallel the other vehicle is traveling to the map road-dividing line ML.

[0059] In the above example, the parallelism calculation unit 154 calculates the average value of the angle between the travel trajectory T1 and the camera road-dividing line CL1, and the average value of the angle between the travel trajectory T1 and the map road-dividing line ML1. However, the present invention is not limited to this configuration, and the parallelism calculation unit 154 may calculate the median value of the angle between the travel trajectory T1 and the camera road-dividing line CL1, and the median value of the angle between the travel trajectory T1 and the map road-dividing line ML1. This prevents a decrease in the accuracy of the calculation results due to a specific vehicle among the other vehicles following an abnormal travel trajectory.

[0060] FIG. 5 is a graph illustrating a method for determining a driving mode based on parallelism. As shown in FIG. 5, when the calculated parallelism θm_av−θc_av is equal to or greater than the threshold Th (the region indicated by the diagonal line R), the mode determination unit 150 determines that the reliability of the camera-based road-dividing line CL is higher than that of the map-based road-dividing line ML, and continues the driving mode in mode B, which uses the camera-based road-dividing line CL as the reference line. The threshold Th is set to a value greater than zero, taking into account a safety margin. On the other hand, when the calculated parallelism θm_av−θc_av is less than the threshold, the mode determination unit 150 changes the driving mode to a driving mode (mode C, mode D, or mode E) that is more difficult to carry out than mode B. This allows the vehicle's driving control to be appropriately changed even when the road-dividing line recognized by the camera differs from the map information installed in the vehicle.

[0061] In this embodiment, as described with reference to FIG. 5 , when the calculated parallelism θm_av−θc_av is equal to or greater than the threshold value Th, the mode determination unit 150 continues the driving mode of mode B using the camera road-dividing line CL as the reference line. However, the present invention is not limited to such a configuration. Even when the calculated parallelism θm_av−θc_av is equal to or greater than the threshold value Th, the mode determination unit 150 may change the driving mode from mode B to mode C after notifying an occupant of the host vehicle M. For example, when the deviation determination unit 152 determines that a deviation exists and the parallelism θm_av−θc_av is equal to or greater than the threshold value Th, the mode determination unit 150 may notify an occupant of the host vehicle M of information indicating the deviation and recommend a change to mode C to the occupant, or may change to mode C a certain period after the deviation determination.

[0062] In the above description, the parallelism calculation unit 154 calculates the parallelism between the trajectories of the other vehicles and the camera road-dividing line CL when it is determined that a threshold or more of other vehicles are present within a predetermined distance from the host vehicle M. At this time, the parallelism calculation unit 154 may exclude from the calculation other vehicles that are inappropriate for calculating parallelism, even if they are present within the predetermined distance from the host vehicle M.

[0063] FIG. 6 is a diagram showing an example of another vehicle excluded from calculation targets for calculating parallelism. FIG. 6 illustrates a situation in which, in addition to the host vehicle M, another vehicle M1 traveling in lane L1 and another vehicle M2 traveling in lane L2 adjacent to lane L1 are present. As shown in FIG. 6, when the calculated traveling trajectory M1 cannot be approximated by a straight line (more specifically, when the error between the traveling trajectory M1 and the approximated straight line is equal to or greater than a threshold), the parallelism calculation unit 154 excludes the other vehicle M1 corresponding to the traveling trajectory M1 from calculation targets for calculating parallelism. Furthermore, for example, the parallelism calculation unit 154 excludes the other vehicle M2 traveling in a lane L2 different from the lane L1 in which the host vehicle M is traveling from calculation targets for calculating parallelism. This can improve the accuracy of calculating parallelism.

[0064] Next, the flow of operations executed by the vehicle control device according to the embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the flow of operations executed by the vehicle control device according to the embodiment. The processing according to this flowchart is executed in a predetermined cycle while the host vehicle M is traveling in mode B driving mode using camera road-dividing lines CL.

[0065] First, the mode determination unit 150 acquires the camera road dividing lines CL and the map road dividing lines ML via the recognition unit 130 (step S100). Next, the deviation determination unit 152 determines whether or not there is a deviation between the acquired camera road dividing lines CL and the map road dividing lines ML (step S102).

[0066] Next, if it is determined that there is a deviation between the acquired camera road dividing lines CL and the map road dividing lines ML, the deviation determination unit 152 determines whether or not there are other vehicles within a predetermined distance from the host vehicle M in a number equal to or greater than a threshold (step S104).If it is determined that there are not other vehicles within a predetermined distance from the host vehicle M in a number equal to or greater than the threshold, the mode determination unit 150 changes the driving mode from mode B to mode C (step S106).

[0067] On the other hand, if it is determined that the number of other vehicles present within a predetermined distance from the host vehicle M is equal to or greater than a threshold, the parallelism calculation unit 154 calculates parallelism based on the travel trajectories of these other vehicles, the camera road-dividing lines CL, and the map road-dividing lines ML (step S108). Next, the mode determination unit 150 determines whether the calculated parallelism is equal to or greater than a threshold (step S110). If it is determined that the calculated parallelism is less than the threshold, the mode determination unit 150 changes the driving mode from mode B to mode C in step S106. On the other hand, if it is determined that the calculated parallelism is equal to or greater than the threshold, the mode determination unit 150 determines to continue mode B using the camera road-dividing lines CL (step S112). This ends the processing of this flowchart.

[0068] According to the present embodiment described above, when a deviation occurs between the camera-captured road-dividing lines and the map-based road-dividing lines and there are multiple other vehicles around the host vehicle, the parallelism is calculated based on the driving trajectories of the multiple other vehicles, the camera-captured road-dividing lines, and the map-based road-dividing lines, and the driving mode of the host vehicle is controlled according to the calculated parallelism. This makes it possible to appropriately change the vehicle's driving control even when the road-dividing lines recognized by the camera differ from the content of the map information installed in the host vehicle.

[0069] The above-described embodiment can be expressed as follows. a storage device storing a program; a hardware processor; The processor executes the computer-readable instructions to: Acquire camera images of the vehicle's surroundings, Controlling the steering and acceleration / deceleration of the vehicle based on the camera image and map information without relying on the operation of the driver of the vehicle; determining a driving mode of the vehicle to one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task imposed on the driver is lighter than that imposed on the driver in the first driving mode, and being performed by controlling the steering and acceleration / deceleration of the vehicle without relying on an operation by the driver of the vehicle, and changing the driving mode of the vehicle to a driving mode in which the task is heavier when the driver does not perform the task related to the determined driving mode; determining whether or not there is a discrepancy between the road dividing line shown in the camera image and the road dividing line shown in the map information; If it is determined that there is a discrepancy between the road dividing line shown in the camera image and the road dividing line shown in the map information, a parallelism between the trajectories of one or more other vehicles present around the vehicle and the road dividing line shown in the camera image is calculated; If the calculated parallelism is equal to or greater than a first threshold, it is determined to continue the second operation mode. The vehicle control device is configured as follows.

[0070] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0071] 10 Camera 12 Radar equipment 14 LIDAR 16 Object recognition device 100 Automatic driving control device 120 First Control Section 130 Recognition part 140 Action Plan Generation Unit 150 Mode determination unit 152 Deviation judgment unit 154 Parallelism calculation unit 160 Second Control Section

Claims

1. an acquisition unit that acquires a camera image of the surroundings of the vehicle; a driving control unit that controls the steering and acceleration / deceleration of the vehicle based on the camera image and map information without relying on an operation by a driver of the vehicle; a mode determination unit that determines a driving mode of the vehicle to be one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task imposed on the driver is lighter than that imposed on the driver in the first driving mode, at least some of the plurality of driving modes including the second driving mode being controlled by the driving control unit, and that changes the driving mode of the vehicle to a driving mode in which the task is heavier when the driver does not perform a task related to the determined driving mode; a deviation determination unit that determines whether or not there is a deviation between the road dividing line shown in the camera image and the road dividing line shown in the map information; a parallelism calculation unit that, when it is determined that there is a deviation between the road dividing line shown in the camera image and the road dividing line shown in the map information, calculates a parallelism between the trajectories of one or more other vehicles present around the vehicle and the road dividing line shown in the camera image, the mode determination unit determines to continue the second operation mode when the calculated parallelism is equal to or greater than a first threshold value. Vehicle control device.

2. the parallelism calculation unit calculates the parallelism when the number of one or more other vehicles present around the vehicle is equal to or greater than a second threshold value; The vehicle control device according to claim 1 .

3. the parallelism calculation unit calculates the parallelism based on trajectories of one or more other vehicles present around the vehicle, road dividing lines shown in the camera image, and road dividing lines shown in the map information; The vehicle control device according to claim 1 or 2.

4. the parallelism calculation unit calculates the parallelism based on an average value of an angle between each of the trajectories of one or more other vehicles present around the vehicle and a road dividing line shown in the camera image, and an average value of an angle between each of the trajectories of one or more other vehicles present around the vehicle and a road dividing line shown in the map information; The vehicle control device according to claim 3.

5. the parallelism calculation unit calculates the parallelism based on a median value of an angle between each of the trajectories of one or more other vehicles present around the vehicle and a road dividing line shown in the camera image, and a median value of an angle between each of the trajectories of one or more other vehicles present around the vehicle and a road dividing line shown in the map information; The vehicle control device according to claim 3.

6. the second driving mode is a driving mode in which the driver is not required to hold an operator that receives a steering operation of the vehicle; The first driving mode is a driving mode in which the driver is assigned at least a task of gripping the operating element. The vehicle control device according to any one of claims 1 to 5.

7. The computer Acquire camera images of the vehicle's surroundings, Controlling the steering and acceleration / deceleration of the vehicle based on the camera image and map information without relying on the operation of the driver of the vehicle; determining a driving mode of the vehicle to one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task imposed on the driver is lighter than that imposed on the driver in the first driving mode, and being performed by controlling the steering and acceleration / deceleration of the vehicle without relying on an operation by the driver of the vehicle, and changing the driving mode of the vehicle to a driving mode in which the task is heavier when the driver does not perform a task related to the determined driving mode; determining whether or not there is a discrepancy between the road dividing line shown in the camera image and the road dividing line shown in the map information; If it is determined that there is a discrepancy between the road dividing line shown in the camera image and the road dividing line shown in the map information, a parallelism between the trajectories of one or more other vehicles present around the vehicle and the road dividing line shown in the camera image is calculated; If the calculated parallelism is equal to or greater than a first threshold, it is determined to continue the second operation mode. Vehicle control method.

8. On the computer, Acquire a camera image capturing the surroundings of the vehicle, Controlling the steering and acceleration / deceleration of the vehicle based on the camera image and map information without relying on the operation of the driver of the vehicle; determining a driving mode of the vehicle to one of a plurality of driving modes including a first driving mode and a second driving mode, the second driving mode being a driving mode in which a task imposed on the driver is lighter than that imposed on the driver in the first driving mode, and being performed by controlling the steering and acceleration / deceleration of the vehicle without relying on an operation by the driver of the vehicle, and changing the driving mode of the vehicle to a driving mode in which the task is heavier when the driver does not perform a task related to the determined driving mode; determining whether or not there is a discrepancy between the road dividing line shown in the camera image and the road dividing line shown in the map information; If it is determined that there is a discrepancy between the road dividing line shown in the camera image and the road dividing line shown in the map information, a parallelism between the trajectories of one or more other vehicles present around the vehicle and the road dividing line shown in the camera image is calculated; When the calculated parallelism is equal to or greater than a first threshold value, it is determined to continue the second operation mode. program.

Images