Vehicle control device, vehicle control method, and storage medium

Abstract

The present application provides a vehicle control device, a vehicle control method, and a storage medium, which appropriately change the driving control of a vehicle even when a road division line recognized by a camera and the content of map information mounted on the vehicle are different. The vehicle control device includes: an acquisition unit that acquires a camera image obtained by photographing the surrounding situation of a vehicle; a driving control unit that controls the steering and acceleration / deceleration of the vehicle based on the camera image and map information, without depending on the operation of the driver of the vehicle; a mode determination unit that determines the driving mode of the vehicle to be any one of a plurality of driving modes including a first driving mode and a second driving mode; and a determination unit that determines whether there is a deviation between a road division line shown in the camera image and a road division line shown in the map information, and determines whether the vehicle is at a branch location shown in the map information.

Description

Technical Field

[0001] This invention relates to vehicle control devices, vehicle control methods, and storage media. Background Technology

[0002] Previously, it was known to control the driving of a vehicle based on road markings identified by a camera mounted on the vehicle. For example, Japanese Patent Application Publication No. 2020-050086 describes a technology that allows the vehicle to drive based on identified road markings, and, if the degree of road marking recognition does not meet a prescribed standard, allows the vehicle to drive based on the trajectory of a preceding vehicle.

[0003] The technology described in Japanese Patent Application Publication No. 2020-050086 controls the driving of a vehicle based on road markings identified by a camera and map information carried by the vehicle. However, in conventional technology, when the road markings identified by the camera differ from the map information carried by the vehicle, the driving control of the vehicle cannot always be properly adjusted. Summary of the Invention

[0004] The present invention was made in consideration of such circumstances, and one of its objectives is to provide a vehicle control device, vehicle control method and storage medium that can appropriately change the driving control of the vehicle even when the road markings recognized by the camera are different from the content of the map information carried by the vehicle.

[0005] The vehicle control device, vehicle control method, and storage medium of the present invention adopt the following structure.

[0006] (1): A vehicle control device according to one aspect of the present invention includes: an acquisition unit that acquires camera images obtained by photographing the surrounding conditions of the vehicle; a driving control unit that controls the steering and acceleration / deceleration of the vehicle independently of the driver's operation based on the camera images and map information; and a mode determination unit that determines the driving mode of the vehicle as any one of a plurality of driving modes including a first driving mode and a second driving mode, wherein the second driving mode is a driving mode in which the task assigned to the driver is less demanding than that in the first driving mode, and at least a portion of the plurality of driving modes including the second driving mode is controlled by the driving control unit, wherein the determined driving mode... If the task in question is not performed by the driver, the mode determination unit changes the vehicle's driving mode to a more demanding driving mode; and the determination unit determines whether there is a deviation between the road dividing lines shown in the camera image and the road dividing lines shown in the map information, and determines whether the vehicle is at a branch point shown in the map information. If the mode determination unit determines that there is a deviation between the road dividing lines shown in the camera image and the road dividing lines shown in the map information, and determines that the vehicle is at a branch point shown in the map information, it determines the vehicle's driving mode based on the branch direction of the branch point and the direction of the road dividing lines shown in the camera image.

[0007] (2): Based on the above (1) scheme, if the deviation between the branch direction at the branch location and the direction of the road dividing line shown in the camera image is above a threshold, the mode determination unit changes the second driving mode to the first driving mode and continues the first driving mode using the road dividing line shown in the camera image.

[0008] (3): Based on the above (1) scheme, if the deviation between the branch direction at the branch location and the direction of the road dividing line shown in the camera image is less than a threshold, the mode determination unit uses the road dividing line shown in the map information to continue the second driving mode.

[0009] (4): Based on any of the above schemes (1) to (3), the second driving mode is a driving mode in which the driver is not assigned the task of holding the control device, which accepts the steering operation of the vehicle, and the first driving mode is a driving mode in which the driver is assigned the task of holding the control device.

[0010] (5): Another aspect of the vehicle control method of the present invention involves a computer performing the following processing: acquiring camera images obtained by photographing the surrounding conditions of the vehicle; controlling the steering and acceleration / deceleration of the vehicle independently of the driver's operation based on the camera images and map information; determining the driving mode of the vehicle as any one of a plurality of driving modes including a first driving mode and a second driving mode, wherein the second driving mode is a driving mode with a lighter task assigned to the driver than the first driving mode, and at least a portion of the plurality of driving modes including the second driving mode controls the steering and acceleration / deceleration of the vehicle independently of the driver's operation. Acceleration and deceleration are performed. If the task involved in the determined driving mode is not performed by the driver, the driving mode of the vehicle is changed to a driving mode with a heavier task. It is determined whether there is a deviation between the road dividing line shown in the camera image and the road dividing line shown in the map information, and whether the vehicle is at a branch point shown in the map information. If it is determined that there is a deviation between the road dividing line shown in the camera image and the road dividing line shown in the map information, and it is determined that the vehicle is at a branch point shown in the map information, the driving mode of the vehicle is determined based on the branch direction of the branch point and the direction of the road dividing line shown in the camera image.

[0011] (6): In another embodiment of the present invention, the storage medium stores a program, wherein the program causes a computer to perform the following processing: acquiring camera images obtained by taking pictures of the vehicle's surroundings; based on the camera images and map information, controlling the vehicle's steering and acceleration / deceleration independently of the driver's operation; determining the vehicle's driving mode as any one of a plurality of driving modes including a first driving mode and a second driving mode, wherein the second driving mode is a driving mode with a lighter task assigned to the driver than the first driving mode, and at least a portion of the plurality of driving modes including the second driving mode controls the vehicle independently of the driver's operation. The system controls vehicle steering and acceleration / deceleration. If the task involved in the determined driving mode is not performed by the driver, the vehicle's driving mode is changed to a more demanding driving mode. It determines whether there is a deviation between the road markings shown in the camera image and the road markings shown in the map information, and whether the vehicle is at a branch point shown in the map information. If a deviation is found between the road markings shown in the camera image and the road markings shown in the map information, and the vehicle is at a branch point shown in the map information, the driving mode of the vehicle is determined based on the branch direction of the branch point and the direction of the road markings shown in the camera image.

[0012] According to (1) to (6), even if the road markings identified by the camera are different from the content of the map information carried by the vehicle, the driving control of the vehicle can be appropriately changed. Attached Figure Description

[0013] Figure 1 This is a structural diagram of a vehicle system utilizing a vehicle control device according to an implementation method.

[0014] Figure 2 This is a functional structure diagram of the first control unit and the second control unit.

[0015] Figure 3 This is a diagram illustrating an example of the correspondence between driving modes and the control states and tasks of the vehicle M.

[0016] Figure 4 This is a diagram illustrating an example of the operation of a vehicle control device implementing an embodiment.

[0017] Figure 5 This is another example of a scenario depicting the operation of a vehicle control device implementing an embodiment.

[0018] Figure 6 This is a flowchart illustrating an example of the flow of actions performed by the vehicle control device of the embodiment. Detailed Implementation

[0019] Hereinafter, embodiments of the vehicle control device, vehicle control method, and storage medium of the present invention will be described with reference to the accompanying drawings.

[0020] [Overall Structure]

[0021] Figure 1 This is a structural diagram of vehicle system 1 utilizing the vehicle control device of the embodiment. The vehicle equipped with vehicle system 1 is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using electricity generated by a generator connected to the internal combustion engine or electricity discharged from a secondary battery or fuel cell.

[0022] Vehicle system 1 includes, for example, a camera 10, a radar device 12, a LiDAR (Light Detection and Ranging) system 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, vehicle sensors 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driver monitoring camera 70, driving controls 80, an automatic driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are interconnected via CAN (Controller Area Network) communication lines, serial communication lines, wireless communication networks, etc. It should be noted that... Figure 1 The structure shown is just one example; you can omit part of the structure or add other structures.

[0023] Camera 10 is, for example, a digital camera utilizing a solid-state imaging element such as CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor). Camera 10 is mounted anywhere on the vehicle equipped with vehicle system 1 (hereinafter referred to as the vehicle M). When taking pictures of the front, camera 10 is mounted on the upper part of the windshield, the back of the rearview mirror inside the vehicle, etc. Camera 10, for example, periodically and repeatedly takes pictures of the surroundings of the vehicle M. Camera 10 can also be a stereo camera.

[0024] Radar device 12 radiates millimeter-wave or other radio waves around the vehicle M and detects the radio waves reflected by objects (reflected waves) to detect at least the position (distance and orientation) of the objects. Radar device 12 can be installed at any location on the vehicle M. Radar device 12 can also detect the position and speed of objects using FM-CW (Frequency Modulated Continuous Wave) method.

[0025] The LIDAR14 illuminates the periphery of the vehicle M with light (or electromagnetic waves with wavelengths close to light) and measures the scattered light. The LIDAR14 detects the distance to the object based on the time from the emission of light to the reception of light. The illuminating light can be, for example, a pulsed laser. The LIDAR14 can be mounted at any location on the vehicle M.

[0026] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the cameras 10, radar device 12, and LIDAR 14 to identify the position, type, speed, etc. of objects. The object recognition device 16 outputs the recognition results to the autonomous driving control device 100. Alternatively, the object recognition device 16 can directly output the detection results from the cameras 10, radar device 12, and LIDAR 14 to the autonomous driving control device 100. The object recognition device 16 can also be omitted from the vehicle system 1.

[0027] The communication device 20 communicates with other vehicles in the vicinity of the vehicle M, for example, using cellular networks, Wi-Fi networks, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), etc., or communicates with various server devices via wireless base stations.

[0028] The HMI30 provides various information to the occupants of vehicle M and accepts input operations performed by the occupants. The HMI30 includes various display devices, speakers, buzzers, touch panels, switches, buttons, etc.

[0029] The vehicle sensor 40 includes a vehicle speed sensor for detecting the speed of the vehicle M, an acceleration sensor for detecting acceleration, a yaw rate sensor for detecting angular velocity about the vertical axis, and an orientation sensor for detecting the orientation of the vehicle M.

[0030] The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as an HDD (Hard Disk Drive) or flash memory. The GNSS receiver 51 determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M can also be determined or supplemented using INS (Inertial Navigation System) output from the vehicle sensors 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, buttons, etc. The navigation HMI 52 can also be partially or entirely shared with the aforementioned HMI 30. The route determination unit 53, for example, refers to the first map information 54 to determine the route (hereinafter referred to as the map path) from the position of the vehicle M determined by the GNSS receiver 51 (or any input position) to the destination input by the occupant using the navigation HMI 52. The first map information 54 is, for example, information representing the shape of a road by indicating road segments and nodes connecting the road segments. The first map information 54 may also include road curvature, POI (Point of Interest) information, etc. The path on the map is output to the MPU 60. The navigation device 50 can also provide route guidance using the navigation HMI 52 based on the path on the map. The navigation device 50 can also be implemented, for example, through the functions of a terminal device such as a smartphone or tablet held by the occupant. The navigation device 50 can also send its current location and destination to the navigation server via the communication device 20, and obtain the path equivalent to the path on the map from the navigation server.

[0031] MPU 60 includes, for example, a lane recommendation unit 61, and stores second map information 62 in a storage device such as an HDD or flash memory. The lane recommendation unit 61 divides the path on the map provided by the navigation device 50 into multiple blocks (e.g., in 100m increments along the vehicle's direction of travel), and determines a recommended lane for each block by referring to the second map information 62. The lane recommendation unit 61 determines which lane to drive in from the left. When the path on the map branches off, the lane recommendation unit 61 determines a recommended lane so that the vehicle M can travel on a reasonable path to the branch destination.

[0032] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 may include, for example, information about the center of a lane or the boundaries of a lane. Furthermore, the second map information 62 may include road information, traffic restriction information, residential information (address, postal code), facility information, telephone number information, and information about prohibited areas in Mode A or Mode B (described later). The second map information 62 can be updated in real time by communicating with other devices via the communication device 20. In this embodiment, the second map information 62 is specifically designed to include the location information of lane branch points as source information.

[0033] The driver monitoring camera 70 is, for example, a digital camera utilizing a solid-state imaging element such as a CCD or CMOS. The driver monitoring camera 70 can be installed anywhere in the vehicle M in a position and orientation that allows it to capture the head of the occupant (hereinafter referred to as the driver) seated in the driver's seat of the vehicle M from the front (with the orientation for capturing the face). For example, the driver monitoring camera 70 is installed above a display device located in the center of the dashboard of the vehicle M.

[0034] The driving control unit 80 includes, for example, the accelerator pedal, brake pedal, gear shift lever, and other operating components in addition to the steering wheel 82. Sensors are installed in the driving control unit 80 to detect the amount or presence of operation, and the detection results are output to some or all of the following: the automatic driving control unit 100, the driving force output device 200, the braking device 210, and the steering device 220. The steering wheel 82 is an example of an operating component that receives steering operations performed by the driver. The operating component does not necessarily have to be ring-shaped; it can also be a non-circular steering gear, a lever, a button, etc. A steering wheel grip sensor 84 is installed in the steering wheel 82. The steering wheel grip sensor 84, implemented by a capacitance sensor or the like, outputs a signal that detects whether the driver is gripping (meaning in contact with the steering wheel 82 with applied force) to the automatic driving control unit 100.

[0035] The autonomous driving control device 100 includes, for example, a first control unit 120 and a second control unit 160. The first control unit 120 and the second control unit 160 are respectively implemented by executing programs (software) using hardware processors such as CPUs (Central Processing Units). Furthermore, some or all of these components can also be implemented using hardware (including circuitry) such as LSIs (Large Scale Integration), ASICs (Application Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays), and GPUs (Graphics Processing Units), or through the coordinated use of software and hardware. The program can be pre-stored in a storage device such as an HDD or flash memory (a storage device with a non-transitory storage medium) of the autonomous driving control device 100, or stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the autonomous driving control device 100 by mounting the storage medium (a non-transitory storage medium) to the drive unit. The automatic driving control device 100 is an example of a "vehicle control device", and the action plan generation unit 140 and the second control unit 160 together are an example of a "driving control unit".

[0036] Figure 2 This is a functional structure diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130, an action plan generation unit 140, and a pattern determination unit 150. The first control unit 120 can, for example, implement AI (Artificial Intelligence) based functions and functions based on pre-given models in parallel. For example, the function of "recognizing intersections" can be achieved by "parallel execution of intersection recognition based on deep learning, etc., and recognition based on pre-given conditions (the existence of signals that can match patterns, road signs, etc.), and comprehensively evaluating both sides by scoring them." This ensures the reliability of autonomous driving.

[0037] The recognition unit 130 identifies the position, speed, acceleration, and other states of objects surrounding the vehicle M based on information input from the camera 10, radar device 12, and LIDAR 14 via the object recognition device 16. The position of an object is identified, for example, as its position on absolute coordinates with a representative point of the vehicle M (center of gravity, drive shaft center, etc.) as the origin, and is used for control. The position of an object can also be represented by representative points such as its center of gravity or corners, or by a region. The "state" of an object can also include its acceleration, jerk, or "action state" (e.g., whether it is changing lanes or about to change lanes).

[0038] Additionally, the recognition unit 130 identifies, for example, the lane in which the vehicle M is traveling. For instance, the recognition unit 130 identifies the lane by comparing the pattern of road markings (e.g., the arrangement of solid and dashed lines) obtained from the second map information 62 with the pattern of road markings surrounding the vehicle M identified from the image captured by the camera 10. It should be noted that the recognition unit 130 is not limited to road markings; it can also identify the driving lane by recognizing road markings, driving boundaries including shoulders, curbs, median strips, guardrails, etc. In this recognition, the position of the vehicle M obtained from the navigation device 50 and the processing results from the INS may also be taken into consideration. Furthermore, the recognition unit 130 identifies temporary stop lines, obstacles, red lights, toll booths, and other road phenomena.

[0039] When identifying a driving lane, the identification unit 130 identifies the position and orientation of the vehicle M relative to the driving lane. For example, the identification unit 130 may identify the deviation of the vehicle M's reference point from the center of the lane, and the angle formed by the vehicle M's direction of travel relative to the line connecting the centers of the lanes, as the relative position and orientation of the vehicle M relative to the driving lane. Alternatively, the identification unit 130 may identify the position of the vehicle M's reference point relative to any side end (road dividing line or road boundary) of the driving lane as the relative position of the vehicle M relative to the driving lane.

[0040] The action plan generation unit 140 generates a target trajectory for the vehicle M to travel automatically (independent of driver operation) in the future, in a manner that allows it to travel in the recommended lane determined by the recommended lane determination unit 61 and to be able to cope with the surrounding conditions of the vehicle M. The target trajectory includes, for example, a speed element. For instance, the target trajectory is represented by a sequence of locations (track points) that the vehicle M should reach. Track points are locations that the vehicle M should reach at predetermined travel distances (e.g., several meters), but target speeds and target accelerations are generated as part of the target trajectory at predetermined sampling times (e.g., a few tenths of a second). Alternatively, track points can also be positions that the vehicle M should reach at the specified sampling time. In this case, the target speed and target acceleration information are represented by the intervals between track points.

[0041] When generating a target track, the action plan generation unit 140 can set events for automatic driving. These events include constant speed driving events, low-speed following events, lane change events, branching events, merging events, and takeover events. The action plan generation unit 140 generates the target track corresponding to the initiated events.

[0042] The mode determination unit 150 determines the driving mode of the vehicle M as any one of several driving modes with different tasks assigned to the driver. The mode determination unit 150 includes, for example, a determination unit 152. The function of the determination unit 152 will be described later.

[0043] Figure 3 This diagram illustrates an example of the correspondence between driving modes and the control states and tasks of the vehicle M. The vehicle M has five driving modes, for example, modes A through E. Regarding the control state, i.e., the degree of automation of the driving control of the vehicle M, mode A is the highest, followed by modes B, C, and D, which decrease sequentially, with mode E being the lowest. Conversely, regarding the tasks assigned to the driver, mode A is the lightest, followed by modes B, C, and D, which become increasingly more demanding, with mode E being the most demanding. It should be noted that modes D and E are not automatic driving control states; therefore, the automatic driving control device 100 is obligated to perform processing until the control involved in automatic driving is terminated and the system is transferred to driving assistance or manual driving. The following provides examples of the content of each driving mode.

[0044] In Mode A, the vehicle enters an autonomous driving state, and neither forward monitoring nor steering wheel control (as shown in the diagram) is assigned to the driver. However, even in Mode A, the driver is required to have a body posture that allows them to quickly transition to manual driving based on the requirements of the system centered on the autonomous driving control unit 100. It should be noted that autonomous driving here refers to steering, acceleration, and deceleration being controlled without driver input. "Forward" refers to the space in the direction of travel of the vehicle M as visually perceived through the windshield. Mode A is a driving mode that can be executed, for example, when the vehicle M is traveling at a speed below a prescribed speed (e.g., around 50 km / h) on a dedicated motor vehicle road such as a highway, and when there are following vehicles in front; it is sometimes called TJP (Traffic Jam Pilot). If these conditions are no longer met, the mode determination unit 150 changes the driving mode of the vehicle M to Mode B.

[0045] In Mode B, the system enters a driver support state, assigning the driver the task of monitoring the area ahead of the vehicle M (hereinafter referred to as forward monitoring), but not the task of controlling the steering wheel 82. In Mode C, the system also enters a driver support state, assigning the driver both the task of forward monitoring and the task of controlling the steering wheel 82. Mode D is a driving mode where at least one of the steering or acceleration / deceleration of the vehicle M requires some degree of driver intervention. For example, in Mode D, driver support functions such as ACC (Adaptive Cruise Control) and LKAS (Lane Keeping Assist System) are activated. In Mode E, the system enters a manual driving state where both steering and acceleration / deceleration require driver intervention. Both Modes D and E naturally assign the driver the task of monitoring the area ahead of the vehicle M.

[0046] The automatic driving control unit 100 (and driving support unit (not shown)) performs automatic lane changes corresponding to the driving mode. Automatic lane changes include automatic lane changes based on system requirements (1) and automatic lane changes based on driver requirements (2). Automatic lane changes (1) include automatic lane changes for overtaking when the speed of the preceding vehicle is more than a certain threshold lower than the vehicle's speed, and automatic lane changes for heading towards the destination (automatic lane changes performed due to a change in the recommended lane). Automatic lane changes (2) are automatic lane changes performed when the driver operates the direction indicator, causing the vehicle M to change lanes in the direction of operation, provided that conditions related to speed and position relative to surrounding vehicles are met.

[0047] In Mode A, the automatic driving control device 100 does not perform automatic lane change (1) and (2). In Modes B and C, the automatic driving control device 100 performs both automatic lane change (1) and (2). In Mode D, the driving assistance device (not shown) performs automatic lane change (2) instead of automatic lane change (1). In Mode E, neither automatic lane change (1) nor (2) is performed.

[0048] If the driver does not perform the task related to the determined driving mode (hereinafter referred to as the current driving mode), the mode determination unit 150 changes the driving mode of the vehicle M to a driving mode with a heavier task.

[0049] For example, in Mode A, if the driver's body posture prevents them from shifting to manual driving as requested by the system (e.g., continuously looking out of the permitted area, or detecting signs of driving difficulty), the mode determination unit 150 uses the HMI 30 to urge the driver to shift to manual driving. If the driver does not respond, the unit performs control such as gradually stopping the vehicle M by pulling it towards the curb and discontinuing automatic driving. After discontinuing automatic driving, the vehicle enters Mode D or E, and the driver can start the vehicle M manually. The same applies to "discontinuing automatic driving." In Mode B, if the driver is not monitoring the road ahead, the mode determination unit 150 uses the HMI 30 to urge the driver to monitor the road ahead. If the driver does not respond, the unit performs control such as gradually stopping the vehicle M by pulling it towards the curb and discontinuing automatic driving. In Mode C, if the driver is not monitoring the situation ahead or is not holding the steering wheel 82, the mode decision unit 150 uses HMI 30 to urge the driver to monitor the situation ahead and / or hold the steering wheel 82. If the driver does not respond, the system will gradually bring the vehicle M to a stop by moving it toward the curb and discontinue automatic driving.

[0050] The second control unit 160 controls the driving force output device 200, the braking device 210 and the steering device 220 so that the vehicle M passes through the target track generated by the action plan generation unit 140 at a predetermined time.

[0051] return Figure 2The second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires information about the target track (track point) generated by the action plan generation unit 140 and stores this information in a memory (not shown). The speed control unit 164 controls the driving force output device 200 or the braking device 210 based on the speed elements associated with the target track stored in the memory. The steering control unit 166 controls the steering device 220 based on the curvature of the target track stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is achieved, for example, through a combination of feedforward control and feedback control. As an example, the steering control unit 166 combines feedforward control corresponding to the curvature of the road ahead of the vehicle M with feedback control based on deviation from the target track.

[0052] The driving force output device 200 outputs driving force (torque) for vehicle movement to the drive wheels. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, as well as an ECU (Electronic Control Unit) that controls them. The ECU controls the above structure according to information input from the second control unit 160 or from the driving operation unit 80.

[0053] The braking device 210 includes, for example, a brake caliper, a hydraulic cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the hydraulic cylinder, and a braking ECU. The braking ECU controls the electric motor according to information input from the second control unit 160 or from the driving operation unit 80, so that braking torque corresponding to the braking operation is output to each wheel. The braking device 210 may have a backup mechanism for transmitting hydraulic pressure generated by the operation of the brake pedal included in the driving operation unit 80 via the master hydraulic cylinder to the hydraulic cylinder. It should be noted that the braking device 210 is not limited to the structure described above, and may also be an electronically controlled hydraulic braking device that transmits hydraulic pressure from the master hydraulic cylinder to the hydraulic cylinder by controlling the actuator according to information input from the second control unit 160.

[0054] The steering system 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies force to a rack and pinion mechanism to change the orientation of the steering wheels. The steering ECU drives the electric motor according to information input from the second control unit 160 or from the driving operation unit 80, thereby changing the orientation of the steering wheels.

[0055] [Action of vehicle control devices]

[0056] Next, the operation of the vehicle control device according to the embodiment will be explained. In the following description, it is assumed that the vehicle M is driving in driving mode B. Figure 4This diagram illustrates an example of a scenario depicting the operation of a vehicle control device implementing an embodiment. Figure 4 The text indicates that vehicle M is in lane L1 and driving in front of the branch point, and the driving mode of mode B is set to go straight.

[0057] While the vehicle M is traveling in lane L1, the recognition unit 130 identifies the surrounding conditions of the vehicle M based on images captured by the camera 10, particularly identifying the road markings on both sides of the vehicle M. Hereinafter, CL represents the road markings identified based on images captured by the camera 10 (hereinafter referred to as "camera road markings CL"), and ML represents the road markings identified based on the second map information 62 (hereinafter referred to as "map road markings ML"). Figure 4 In the image, the map road demarcation line ML deviates from the actual road demarcation line AL. This indicates a situation where, for example, due to construction or redrawing of the road demarcation lines, the map road demarcation line ML deviates from the actual road demarcation line AL.

[0058] The determination unit 152 determines, while the vehicle M is in motion, whether there is a deviation (mismatch) between the camera road dividing line CL and the map road dividing line ML. Here, deviation means, for example, that the distance between the camera road dividing line CL and the map road dividing line ML is greater than a predetermined value. The determination unit 152 also determines whether the vehicle M is at a branch location based on the branch location location information obtained from the second map information 62. More specifically, for example, the identification unit 130 determines whether the vehicle M is at a branch location by determining whether the vehicle M is within a predetermined distance from the branch location location information (e.g., GPS coordinates).

[0059] When the mode determination unit 150 determines that there is a deviation between the camera road dividing line CL and the map road dividing line ML, and determines that the vehicle M is at a branch point, it determines the driving mode of the vehicle M based on the branch direction of the branch point obtained from the second map information 62 and the direction of the camera road dividing line CL.

[0060] More specifically, for example, such as Figure 4As shown, the mode determination unit 150 calculates the deviation between the branching direction (the extension direction of the map road dividing line ML) of the branching location obtained from the second map information 62 and the direction of the camera road dividing line CL. If the calculated deviation is above a threshold, it is determined that the reliability of the camera road dividing line CL is higher than that of the map road dividing line ML. This is because, generally, when the vehicle M passes by a branching location, for example, due to the special painting conditions of the branching location, the frequency of misidentification of the identified camera road dividing line CL is higher than that of the main road towards the branch road ML. That is, if the identified camera road dividing line CL does not lean towards the branch road ML, it means that the reliability of the camera road dividing line CL is high.

[0061] Therefore, if the deviation between the branch direction of the branch location obtained from the second map information 62 and the direction of the camera road dividing line CL is greater than or equal to a threshold, the mode determination unit 150 changes the driving mode from mode B to mode C and continues driving in mode C using the camera road dividing line CL as a reference line. Alternatively, the mode determination unit 150 may not change the driving mode from mode B and continue driving in mode B using the camera road dividing line CL as a reference line. Thus, even if there is a mismatch between the camera road dividing line CL and the map road dividing line ML at the branch location, the driving of the vehicle M can be controlled using a more reliable road dividing line.

[0062] Figure 5 This is another example of a scenario depicting the operation of a vehicle control device implementing an embodiment. Figure 5 The scenario is as follows: the camera road dividing line CL is drawn towards the direction of the branch road ML. As a result, the camera road dividing line CL is drawn in a direction different from the lane L1 where the vehicle M is originally traveling. At this time, the determination unit 152 determines that the deviation between the branch direction of the branch location obtained from the second map information 62 and the direction of the camera road dividing line CL is less than a threshold, and the mode determination unit 150 uses the map road dividing line ML as a reference line to continue the driving mode of mode B. That is, when the deviation between the branch direction of the branch location obtained from the second map information 62 and the direction of the camera road dividing line CL is less than the threshold, this indicates that there is a possibility of misidentification that the camera road dividing line CL is drawn towards the direction of the branch road ML, so the map road dividing line ML is used to continue the driving mode of mode B. As a result, the vehicle M is not guided by the camera road dividing line CL which is drawn towards the direction of the branch road ML, and can travel straight along the map road dividing line ML in lane L1.

[0063] Alternatively, if the deviation between the direction of branch road ML and the direction of the camera road dividing line CL is less than a threshold, the mode determination unit 150 may report to the occupants of the vehicle M and then change the driving mode from mode B to mode C. For example, if the deviation between the direction of branch road ML and the direction of the camera road dividing line CL is less than a threshold, the mode determination unit 150 may also report information indicating the possibility of misidentification at branch road ML to the occupants of the vehicle M and recommend that the occupants change to mode C, or change to mode C a certain period after the deviation is determined.

[0064] Next, refer to Figure 6 This will be used to explain the flow of actions performed by the vehicle control device of the implementation method. Figure 6 This is a flowchart illustrating an example of the flow of actions performed by the vehicle control device according to the embodiment. The processes involved in this flowchart are executed in a prescribed cycle while the vehicle M is driving in mode B, which uses camera-based road markings CL.

[0065] First, the pattern determination unit 150 obtains the camera road dividing line CL and the map road dividing line ML via the recognition unit 130 (step S100). Next, the determination unit 152 determines whether there is a deviation between the obtained camera road dividing line CL and the map road dividing line ML (step S102).

[0066] Next, if the determination unit 152 determines that there is a deviation between the acquired camera road dividing line CL and the map road dividing line ML, it determines whether the vehicle M is at a branch point based on the second map information 62 (step S104). If it determines that there is no deviation between the acquired camera road dividing line CL and the map road dividing line ML, or if it determines that the vehicle M is not at a branch point, the mode determination unit 150 returns the process to step S100.

[0067] On the other hand, if it is determined that the vehicle M is at a branch point, the determination unit 152 then determines whether the deviation between the direction of the camera road dividing line CL and the branch direction of the branch point is greater than or equal to a threshold (step S106). If it is determined that the deviation between the direction of the camera road dividing line CL and the branch direction of the branch point is less than the threshold, the mode determination unit 150 decides to use the map road dividing line ML as a reference line to continue driving mode B (step S108). On the other hand, if it is determined that the deviation between the direction of the camera road dividing line CL and the branch direction of the branch point is greater than or equal to the threshold, the mode determination unit 150 changes the driving mode from mode B to mode C (step S110). At this time, the mode determination unit 150 temporarily discards the map road dividing line ML and decides to use the camera road dividing line CL as a reference line to continue driving mode C. Thus, the processing of this flowchart ends.

[0068] According to this embodiment as described above, when there is a deviation between the camera's road markings and the map's road markings, and the vehicle is at a branch point, the vehicle's driving mode is controlled based on the direction of the camera's road markings and the branching direction of the branch point. Therefore, even when the road markings recognized by the camera differ from the map information displayed on the vehicle, the vehicle's driving control can be appropriately adjusted.

[0069] The implementation methods described above can be performed as follows.

[0070] A vehicle control device comprising:

[0071] A storage device containing a program; and

[0072] Hardware processor,

[0073] The processor performs the following processing by executing computer-readable instructions:

[0074] Camera images obtained by photographing the surroundings of the vehicle;

[0075] Based on the camera images and map information, the vehicle's steering and acceleration / deceleration are controlled independently of the driver's operation.

[0076] The driving mode of the vehicle is determined to be any one of a plurality of driving modes, including a first driving mode and a second driving mode. The second driving mode is a driving mode with a lighter task assigned to the driver than the first driving mode. At least a portion of the plurality of driving modes, including the second driving mode, are controlled by steering and acceleration / deceleration of the vehicle independently of the driver's operation. If the task involved in the determined driving mode is not performed by the driver, the driving mode of the vehicle is changed to a driving mode with a heavier task.

[0077] Determine whether there is a deviation between the road dividing lines shown in the camera image and the road dividing lines shown in the map information, and determine whether the vehicle is at a branch point shown in the map information;

[0078] If it is determined that there is a deviation between the road dividing line shown in the camera image and the road dividing line shown in the map information, and it is determined that the vehicle is at a branch point shown in the map information, the driving mode of the vehicle is determined based on the branch direction of the branch point and the direction of the road dividing line shown in the camera image.

[0079] The above description illustrates specific embodiments of the present invention, but the present invention is not limited to such embodiments in any way, and various modifications and substitutions can be made without departing from the spirit of the present invention.

Claims

1. A vehicle control device, wherein, The vehicle control device includes: The acquisition unit acquires camera images obtained by photographing the surrounding environment of the vehicle. The driving control unit controls the steering and acceleration / deceleration of the vehicle based on the camera images and map information, without relying on the driver's operation. The mode determination unit determines the driving mode of the vehicle as any one of a plurality of driving modes, including a first driving mode and a second driving mode. The second driving mode is a driving mode where the task assigned to the driver is lighter than that of the first driving mode. At least a portion of the plurality of driving modes, including the second driving mode, is controlled by the driving control unit. If the task involved in the determined driving mode is not performed by the driver, the mode determination unit changes the driving mode of the vehicle to a driving mode with a heavier task. as well as The determination unit determines whether there is a deviation between the road dividing lines shown in the camera image and the road dividing lines shown in the map information, and determines whether the vehicle is at a branch point shown in the map information. When the mode determination unit determines that there is a deviation between the road dividing line shown in the camera image and the road dividing line shown in the map information, and determines that the vehicle is at a branch point shown in the map information, it determines the driving mode of the vehicle based on the branch direction of the branch point and the direction of the road dividing line shown in the camera image.

2. The vehicle control device according to claim 1, wherein, If the deviation between the branch direction at the branch location and the direction of the road dividing line shown in the camera image is above a threshold, the mode determination unit changes the second driving mode to the first driving mode and continues the first driving mode using the road dividing line shown in the camera image.

3. The vehicle control device according to claim 1, wherein, If the deviation between the branching direction at the branching point and the direction of the road dividing line shown in the camera image is less than a threshold, the mode determination unit uses the road dividing line shown in the map information to continue the second driving mode.

4. The vehicle control device according to any one of claims 1 to 3, wherein, The second driving mode is a driving mode in which the driver is not assigned the task of controlling the steering mechanism of the vehicle. The first driving mode is a driving mode in which the driver is assigned at least the task of holding the control components.

5. A vehicle control method, wherein, The vehicle control method causes the computer to perform the following processing: Camera images obtained by photographing the surroundings of the vehicle; Based on the camera images and map information, the vehicle's steering and acceleration / deceleration are controlled independently of the driver's operation. The driving mode of the vehicle is determined to be any one of a plurality of driving modes, including a first driving mode and a second driving mode. The second driving mode is a driving mode with a lighter task assigned to the driver than the first driving mode. At least a portion of the plurality of driving modes, including the second driving mode, are controlled by steering and acceleration / deceleration of the vehicle independently of the driver's operation. If the task involved in the determined driving mode is not performed by the driver, the driving mode of the vehicle is changed to a driving mode with a heavier task. Determine whether there is a deviation between the road dividing lines shown in the camera image and the road dividing lines shown in the map information, and determine whether the vehicle is at a branch point shown in the map information; If it is determined that there is a deviation between the road dividing line shown in the camera image and the road dividing line shown in the map information, and it is determined that the vehicle is at a branch point shown in the map information, the driving mode of the vehicle is determined based on the branch direction of the branch point and the direction of the road dividing line shown in the camera image.

6. A storage medium storing a program, wherein, The program causes the computer to perform the following processing: Camera images obtained by photographing the surroundings of the vehicle; Based on the camera images and map information, the vehicle's steering and acceleration / deceleration are controlled independently of the driver's operation. The driving mode of the vehicle is determined to be any one of a plurality of driving modes, including a first driving mode and a second driving mode. The second driving mode is a driving mode with a lighter task assigned to the driver than the first driving mode. At least a portion of the plurality of driving modes, including the second driving mode, are controlled by steering and acceleration / deceleration of the vehicle independently of the driver's operation. If the task involved in the determined driving mode is not performed by the driver, the driving mode of the vehicle is changed to a driving mode with a heavier task. Determine whether there is a deviation between the road dividing lines shown in the camera image and the road dividing lines shown in the map information, and determine whether the vehicle is at a branch point shown in the map information; If it is determined that there is a deviation between the road dividing line shown in the camera image and the road dividing line shown in the map information, and it is determined that the vehicle is at a branch point shown in the map information, the driving mode of the vehicle is determined based on the branch direction of the branch point and the direction of the road dividing line shown in the camera image.

Images