Vehicle control device, vehicle control method, and storage medium

Abstract

Even when the road markings detected by the camera differ from the map information displayed on the vehicle, the vehicle control device, vehicle control method, and storage medium can flexibly change the vehicle's driving control. The vehicle control device includes: an acquisition unit that acquires camera images of the vehicle's surroundings; a driving control unit that controls the vehicle's steering and acceleration / deceleration based on the camera images and map information, independent of the driver's operation; a mode determination unit that determines the vehicle's driving mode as one of several driving modes, including a first driving mode and a second driving mode; and a determination unit that determines whether there is a deviation between the road markings shown in the camera image and the road markings shown in the map information. If a deviation exists, the unit determines whether there is an overtaking vehicle. If a deviation exists, the unit controls the driving mode based on the degree of interference between the road markings shown in the camera image and the overtaking vehicle.

Description

Technical Field

[0001] This invention relates to vehicle control devices, vehicle control methods, and storage media. Background Technology

[0002] Previously, there were known technologies for controlling the driving of a vehicle based on road markings identified by a camera mounted on the vehicle. For example, Japanese Patent Application Publication No. 2020-050086 describes a technology that drives the vehicle based on identified road markings, and drives the vehicle based on the trajectory of a preceding vehicle when the degree of road marking recognition does not meet a prescribed standard.

[0003] The technology described in Patent Document 1 is based on controlling the driving of a vehicle using road markings identified by a camera and map information carried by the vehicle. However, in conventional technology, when the road markings identified by the camera differ from the map information carried by the vehicle, it is sometimes impossible to flexibly change the driving control of the vehicle. Summary of the Invention

[0004] This invention was made with the consideration of such circumstances in mind, and one of its objectives is to provide a vehicle control device, vehicle control method, and storage medium that allows for flexible changes to the driving control of the vehicle even when the road markings recognized by the camera differ from the content of the map information carried by the vehicle.

[0005] The vehicle control device, vehicle control method, and storage medium of the present invention adopt the following structure.

[0006] (1): One aspect of the present invention relates to a vehicle control device, wherein the vehicle control device comprises: an acquisition unit that acquires camera images of the vehicle's surroundings; a driving control unit that controls the vehicle's steering and acceleration / deceleration based on the camera images and map information, independent of the driver's operation; and a mode determination unit that determines the vehicle's driving mode as any one of a plurality of driving modes, including a first driving mode and a second driving mode, wherein the second driving mode is a driving mode with a lighter task assigned to the driver compared to the first driving mode, and at least a portion of the plurality of driving modes, including the second driving mode, is controlled by the driving control unit, wherein the task involved in the determined driving mode is not performed by the driver. In the case of driving, the mode determination unit changes the vehicle's driving mode to a more demanding driving mode; and the determination unit determines whether there is a deviation between the road markings shown in the camera image and the road markings shown in the map information. If the deviation is determined to exist, the determination unit determines whether there is a preceding vehicle. If the preceding vehicle is determined to exist and the degree of interference between the road markings shown in the camera image and the preceding vehicle is below a first threshold, the mode determination unit allows driving in the second driving mode using the road markings shown in the camera image to continue. If the preceding vehicle is not determined to exist, the mode determination unit changes the second driving mode to the first driving mode using the road markings shown in the camera image.

[0007] (2): Based on the above (1) scheme, the vehicle control device further includes a calculation unit, which calculates the margin for the vehicle to continue driving in the second driving mode based on the distance from the vehicle to the preceding vehicle, the distance from the preceding vehicle to the intersection of the preceding vehicle and the road dividing line shown in the camera image, and the speed of the vehicle, as the degree of interference.

[0008] (3): Based on the above (1) or (2) scheme, when the determination unit determines that there is the preceding vehicle and the degree of interference is below the first threshold, it determines whether the deviation between the road dividing line shown in the camera image and the road dividing line shown in the map information is below the second threshold. When the determination unit determines that the deviation is greater than the second threshold, the mode determination unit changes the second driving mode that uses the road dividing line shown in the camera image to the first driving mode that uses the road dividing line shown in the camera image.

[0009] (4): Based on the above (1) scheme, when the determination unit determines that there is the preceding vehicle and the degree of interference is greater than the first threshold, the mode determination unit changes the second driving mode that uses the road dividing line shown in the camera image to the first driving mode that uses the road dividing line shown in the camera image.

[0010] (5): Based on the above (1) scheme, if the determination unit determines that there is no preceding vehicle, it determines whether there are other vehicles crossing the road dividing lines shown in the camera image. If the determination unit determines that there are other vehicles, it determines whether the curvature change rate of the road dividing lines shown in the camera image is above a third threshold. If the determination unit determines that the curvature change rate is above the third threshold, the mode determination unit changes the second driving mode using the road dividing lines shown in the camera image to the second driving mode using the road dividing lines shown in the map information.

[0011] (6): Based on the above (1) scheme, the second driving mode is a driving mode in which the driver is not assigned the task of holding the control device to receive the steering operation of the vehicle, and the first driving mode is a driving mode in which the driver is assigned the task of holding the control device at least.

[0012] (7): Another aspect of the present invention relates to a vehicle control method, wherein the vehicle control method enables a computer to perform the following processing: acquiring camera images obtained by capturing images of the vehicle's surroundings; based on the camera images and map information, controlling the vehicle's steering and acceleration / deceleration independently of the driver's operation; determining the vehicle's driving mode as any one of a plurality of driving modes, including a first driving mode and a second driving mode, wherein the second driving mode is a driving mode with a lighter workload for the driver compared to the first driving mode, and at least a portion of the plurality of driving modes, including the second driving mode, are controlled independently of the driver's operation. The system controls the vehicle's steering and acceleration / deceleration. If the task involved in the determined driving mode is not performed by the driver, the driving mode is changed to a more demanding driving mode. It determines whether there is a deviation between the road markings shown in the camera image and the road markings shown in the map information. If a deviation is found, it determines whether there is a preceding vehicle. If the preceding vehicle is found to exist and the interference between the road markings shown in the camera image and the preceding vehicle is below a first threshold, the vehicle continues driving in the second driving mode. If no preceding vehicle is found, the second driving mode is changed back to the first driving mode.

[0013] (8): Another aspect of the present invention relates to a storage medium storing a program, wherein the program causes a computer to perform the following processing: acquiring camera images of the vehicle's surroundings; based on the camera images and map information, controlling the vehicle's steering and acceleration / deceleration independently of the driver's operation; determining the vehicle's driving mode as any one of a plurality of driving modes, including a first driving mode and a second driving mode, wherein the second driving mode is a driving mode with a lighter workload for the driver compared to the first driving mode, and at least a portion of the plurality of driving modes, including the second driving mode, is controlled independently of the driver's operation. The system controls the vehicle's steering and acceleration / deceleration. If the task involved in the determined driving mode is not performed by the driver, the driving mode is changed to a more demanding driving mode. It determines whether there is a deviation between the road markings shown in the camera image and the road markings shown in the map information. If a deviation is found, it determines whether there is a preceding vehicle. If the preceding vehicle is found to exist and the interference between the road markings shown in the camera image and the preceding vehicle is below a first threshold, the vehicle continues driving in the second driving mode. If no preceding vehicle is found, the second driving mode is changed back to the first driving mode.

[0014] According to (1) to (8), even if the road markings identified by the camera are different from the content of the map information carried by the vehicle, the driving control of the vehicle can be flexibly changed. Attached Figure Description

[0015] Figure 1 This is a structural diagram of a vehicle system utilizing a vehicle control device according to an implementation method.

[0016] Figure 2 This is a functional structure diagram of the first control unit and the second control unit.

[0017] Figure 3 This is a diagram illustrating an example of the correspondence between driving modes, the control state of the vehicle M, and tasks.

[0018] Figure 4 This is a diagram illustrating an example of a scenario where the operation of the vehicle control device according to the embodiment is performed.

[0019] Figure 5 This is an example of a scenario where a vehicle is traveling along the wrong camera lane markings.

[0020] Figure 6This diagram illustrates the process performed when the deviation determination unit determines that there is no preceding vehicle.

[0021] Figure 7 This is a flowchart illustrating an example of the flow of actions performed by the vehicle control device of the embodiment.

[0022] Figure 8 This is a flowchart illustrating an example of the flow of actions performed by the vehicle control device of the embodiment. Detailed Implementation

[0023] Hereinafter, embodiments of the vehicle control device, vehicle control method and program of the present invention will be described with reference to the accompanying drawings.

[0024] [Overall Structure]

[0025] Figure 1 This is a structural diagram of vehicle system 1 utilizing the vehicle control device of the embodiment. The vehicle equipped with vehicle system 1 is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using electricity generated by a generator connected to the internal combustion engine, or electricity discharged from a secondary battery or fuel cell.

[0026] Vehicle system 1 includes, for example, a camera 10, a radar device 12, a LiDAR (Light Detection and Ranging) system 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, vehicle sensors 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driver monitoring camera 70, driving controls 80, an automatic driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are interconnected via CAN (Controller Area Network) communication lines, serial communication lines, wireless communication networks, etc. It should be noted that... Figure 1 The structure shown is just one example; you can omit part of the structure or add other structures.

[0027] Camera 10 is, for example, a digital camera utilizing a solid-state imaging element such as CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor). Camera 10 is mounted anywhere on the vehicle equipped with vehicle system 1 (hereinafter referred to as the vehicle M). When taking pictures of the front, camera 10 is mounted on the upper part of the windshield, behind the rearview mirror inside the vehicle, etc. Camera 10, for example, periodically and repeatedly takes pictures of the surroundings of the vehicle M. Camera 10 can also be a stereo camera.

[0028] Radar device 12 radiates millimeter-wave or other radio waves around the vehicle M and detects the radio waves reflected by objects (reflected waves) to detect at least the position (distance and orientation) of the objects. Radar device 12 can be installed at any location on the vehicle M. Radar device 12 can also detect the position and speed of objects using FM-CW (Frequency Modulated Continuous Wave) method.

[0029] The LIDAR14 illuminates the periphery of the vehicle M with light (or electromagnetic waves with wavelengths close to light) and measures the scattered light. The LIDAR14 determines the distance to the object based on the time from the emission of light to the reception of light. The illuminating light can be, for example, a pulsed laser. The LIDAR14 can be mounted at any location on the vehicle M.

[0030] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the cameras 10, radar device 12, and LIDAR 14 to identify the position, type, speed, etc. of objects. The object recognition device 16 outputs the recognition results to the autonomous driving control device 100. Alternatively, the object recognition device 16 can directly output the detection results from the cameras 10, radar device 12, and LIDAR 14 to the autonomous driving control device 100. The object recognition device 16 can also be omitted from the vehicle system 1.

[0031] The communication device 20 may use, for example, cellular networks, Wi-Fi networks, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), etc., to communicate with other vehicles in the vicinity of the vehicle M, or communicate with various server devices via wireless base stations.

[0032] The HMI30 provides various information to the occupants of vehicle M and accepts input operations performed by the occupants. The HMI30 includes various display devices, speakers, buzzers, touch panels, switches, buttons, etc.

[0033] The vehicle sensor 40 includes a vehicle speed sensor for detecting the speed of the vehicle M, an acceleration sensor for detecting acceleration, a yaw rate sensor for detecting angular velocity about a vertical axis, and an orientation sensor for detecting the orientation of the vehicle M.

[0034] The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as an HDD (Hard Disk Drive) or flash memory. The GNSS receiver 51 determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M can also be determined or supplemented by an INS (Inertial Navigation System) utilizing the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, buttons, etc. The navigation HMI 52 can also be partially or entirely shared with the aforementioned HMI 30. The route determination unit 53, for example, refers to the first map information 54 to determine the route (hereinafter referred to as the map path) from the position of the vehicle M determined by the GNSS receiver 51 (or any input position) to the destination input by the occupant using the navigation HMI 52. The first map information 54 is, for example, information representing the shape of a road by indicating road segments and nodes connecting the road segments. The first map information 54 may also include road curvature, POI (Point of Interest) information, etc. The path on the map is output to the MPU 60. The navigation device 50 can also provide route guidance using the navigation HMI 52 based on the path on the map. The navigation device 50 can also be implemented, for example, through the functions of a terminal device such as a smartphone or tablet held by the occupant. The navigation device 50 can also send its current location and destination to the navigation server via the communication device 20, and obtain the path equivalent to the path on the map from the navigation server.

[0035] MPU 60 includes, for example, a lane recommendation unit 61, and stores second map information 62 in a storage device such as an HDD or flash memory. The lane recommendation unit 61 divides the path on the map provided by the navigation device 50 into multiple blocks (e.g., every 100m along the vehicle's direction of travel), and determines a recommended lane for each block based on the second map information 62. The lane recommendation unit 61 determines which lane to drive in from the left. When the path on the map branches off, the lane recommendation unit 61 determines a recommended lane so that the vehicle M can travel on a reasonable path to the branch destination.

[0036] The second map information 62 is more precise than the first map information 54. The second map information 62 may include, for example, information about the center of a lane or the boundaries of a lane. Additionally, the second map information 62 may include road information, traffic restriction information, residential information (address, postal code), facility information, telephone number information, and information about prohibited areas in Mode A or Mode B (described later). The second map information 62 can be updated in real time by communicating with other devices via the communication device 20.

[0037] The driver monitoring camera 70 is, for example, a digital camera utilizing a solid-state imaging element such as a CCD or CMOS. The driver monitoring camera 70 is installed anywhere in the vehicle M, positioned and facing the driver's seat occupant (hereinafter referred to as the driver) to capture their head from the front (with an orientation for capturing their face). For example, the driver monitoring camera 70 is installed above a display device located in the center of the dashboard of the vehicle M.

[0038] The driving control unit 80, in addition to the steering wheel 82, includes, for example, an accelerator pedal, a brake pedal, a gear shift lever, and other control elements. Sensors are installed on the driving control unit 80 to detect the amount or presence of operation, and the detection results are output to some or all of the following: the automatic driving control unit 100, the driving force output device 200, the braking device 210, and the steering device 220. The steering wheel 82 is an example of a "control element that receives steering operations performed by the driver." The control element does not necessarily have to be ring-shaped; it can also be a non-circular steering gear, a lever, a button, etc. A steering wheel grip sensor 84 is installed on the steering wheel 82. The steering wheel grip sensor 84, implemented by a capacitance sensor or the like, is used to output a signal to the automatic driving control unit 100 that detects whether the driver is gripping (meaning in contact with the steering wheel 82 with applied force).

[0039] The autonomous driving control device 100 includes, for example, a first control unit 120 and a second control unit 160. The first control unit 120 and the second control unit 160 are respectively implemented by executing programs (software) using hardware processors such as CPUs (Central Processing Units). Furthermore, some or all of these components can also be implemented by hardware (including circuitry) such as LSIs (Large Scale Integration), ASICs (Application Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays), and GPUs (Graphics Processing Units), or through the coordinated use of software and hardware. The program can be pre-stored in a storage device such as an HDD or flash memory (a storage device with a non-transitory storage medium) of the autonomous driving control device 100, or it can be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the autonomous driving control device 100 by mounting the storage medium (a non-transitory storage medium) to the drive unit. The automatic driving control device 100 is an example of a "vehicle control device", and the action plan generation unit 140 and the second control unit 160 together are an example of a "driving control unit".

[0040] Figure 2 This is a functional structure diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130, an action plan generation unit 140, and a pattern determination unit 150. The first control unit 120 can, for example, implement AI (Artificial Intelligence) based functions and functions based on pre-given models in parallel. For example, the function of "recognizing intersections" can be achieved by "parallel execution of intersection recognition based on deep learning, etc., and recognition based on pre-given conditions (the existence of signals and road signs capable of pattern matching), and comprehensively evaluating both sides by scoring them." This ensures the reliability of autonomous driving.

[0041] The recognition unit 130 identifies the position, speed, acceleration, and other states of objects surrounding the vehicle M based on information input from the camera 10, radar device 12, and LIDAR 14 via the object recognition device 16. The position of an object is identified, for example, as its position on absolute coordinates with a representative point of the vehicle M (center of gravity, drive shaft center, etc.) as the origin, and is used for control. The position of an object can be represented by representative points such as its center of gravity or corners, or by a region. The "state" of an object can also include its acceleration, jerk, or "action state" (e.g., whether it is changing lanes or intends to change lanes).

[0042] Additionally, the recognition unit 130 identifies, for example, the lane in which the vehicle M is traveling. For instance, the recognition unit 130 identifies the lane by comparing the pattern of road markings (e.g., an arrangement of solid and dashed lines) obtained from the second map information 62 with the pattern of road markings surrounding the vehicle M identified from images captured by the camera 10. It should be noted that the recognition unit 130 is not limited to recognizing road markings; it can also identify road boundaries (road boundaries), including shoulders, curbs, median strips, guardrails, etc., thereby identifying the lane. In this recognition, the position of the vehicle M obtained from the navigation device 50 and the processing results from the INS may also be taken into consideration. Furthermore, the recognition unit 130 identifies temporary stop lines, obstacles, red lights, toll booths, and other road phenomena.

[0043] When identifying a driving lane, the identification unit 130 identifies the position and orientation of the vehicle M relative to the driving lane. For example, the identification unit 130 can identify the deviation of the vehicle M's reference point from the center of the lane, and the angle formed by the vehicle M's direction of travel relative to the line connecting the centers of the lanes, to determine the relative position and orientation of the vehicle M relative to the driving lane. Alternatively, the identification unit 130 can identify the position of the vehicle M's reference point relative to any side end (road dividing line or road boundary) of the driving lane to determine the relative position of the vehicle M relative to the driving lane. The identification unit 130 is an example of an "acquisition unit".

[0044] The action plan generation unit 140 generates a target trajectory for the future travel of vehicle M, in a manner that allows it to travel in the recommended lane determined by the recommended lane determination unit 61 and is able to cope with the surrounding conditions of vehicle M. The target trajectory may include, for example, a speed element. For instance, the target trajectory may be represented as a track where the locations (track points) that vehicle M should reach are arranged sequentially. Track points are locations that vehicle M should reach at predetermined travel distances (e.g., a few meters), but target speeds and target accelerations are generated as part of the target trajectory at predetermined sampling times (e.g., a few tenths of a second). Alternatively, track points may be positions that vehicle M should reach at predetermined sampling times. In this case, the target speed and target acceleration information are represented by the intervals between track points.

[0045] When generating a target track, the action plan generation unit 140 can set events for automatic driving. These events include constant speed driving events, low-speed following events, lane change events, branching events, merging events, and takeover events. The action plan generation unit 140 generates target tracks corresponding to the initiated events.

[0046] The mode determination unit 150 determines the driving mode of the vehicle M as any one of several driving modes with different tasks assigned to the driver. The mode determination unit 150 includes, for example, a deviation determination unit 152 and a redundancy calculation unit 154. Their individual functions will be described later.

[0047] Figure 3 This diagram illustrates an example of the correspondence between driving modes, the control state of the vehicle M, and tasks. The driving modes of the vehicle M include, for example, five modes: Mode A through Mode E. Regarding the control state, i.e., the degree of automation of the driving control of the vehicle M, Mode A is the highest, followed by Modes B, C, and D in that order, with Mode E being the lowest. Conversely, regarding the tasks assigned to the driver, Mode A is the lightest, followed by Modes B, C, and D in that order, becoming more demanding, with Mode E being the most demanding. It should be noted that in Modes D and E, the control state is not automatic driving; therefore, the automatic driving control device 100 performs its function before ending the control involved in automatic driving and transitioning to driving assistance or manual driving. The following provides examples of the content of each driving mode.

[0048] In Mode A, the vehicle enters an autonomous driving state, and neither forward monitoring nor steering wheel control (as shown in the diagram) is assigned to the driver. However, even in Mode A, the driver is required to have a body posture that allows them to quickly transition to manual driving based on the requirements of the system centered on the autonomous driving control unit 100. It should be noted that autonomous driving here refers to steering, acceleration, and deceleration being controlled without the driver's input. "Forward" refers to the space visible through the windshield as the direction of travel of the vehicle M. Mode A is, for example, a driving mode that can be executed when the vehicle M is traveling at a prescribed speed (e.g., around 50 km / h) or less on a dedicated motor vehicle road such as a highway, and when there is a vehicle following ahead; it is sometimes referred to as TJP (Traffic Jam Pilot). If these conditions are no longer met, the mode determination unit 150 changes the driving mode of the vehicle M to Mode B.

[0049] In Mode B, the system is in a driver support state, assigning the driver the task of monitoring the front of the vehicle M (hereinafter referred to as forward monitoring), but not the task of holding the steering wheel 82. In Mode C, the system is in a driver support state, assigning the driver the tasks of forward monitoring and holding the steering wheel 82. Mode D is a driving mode that requires some degree of driver operation regarding at least one of the steering or acceleration / deceleration of the vehicle M. For example, in Mode D, driver support functions such as ACC (Adaptive Cruise Control) and LKAS (Lane Keeping Assist System) are used. In Mode E, the system is in a manual driving state where both steering and acceleration / deceleration require driver operation. In both Modes D and E, the driver is naturally assigned the task of monitoring the front of the vehicle M.

[0050] The automatic driving control unit 100 (and driving support unit (not shown)) performs automatic lane changes corresponding to the driving mode. Automatic lane changes include automatic lane changes based on system requirements (1) and automatic lane changes based on driver requirements (2). Automatic lane changes (1) include automatic lane changes for overtaking when the speed of the preceding vehicle is more than a certain threshold compared to the speed of the current vehicle, and automatic lane changes for traveling towards the destination (automatic lane changes caused by changes in the recommended lane). Automatic lane changes (2) refer to changing the vehicle M towards the operating direction when the driver operates the direction indicator, provided that conditions related to speed and positional relationship with surrounding vehicles are met.

[0051] In Mode A, the automatic driving control device 100 does not perform any of the automatic lane changes in automatic lane change (1) and (2). In Modes B and C, the automatic driving control device 100 performs any of the automatic lane changes in automatic lane change (1) and (2). In Mode D, the driving assistance device (not shown) performs automatic lane change (2) instead of automatic lane change (1). In Mode E, it does not perform any of the automatic lane changes in automatic lane change (1) and (2).

[0052] If the task involved in the determined driving mode (hereinafter referred to as the current driving mode) is not performed by the driver, the mode determination unit 150 changes the driving mode of the vehicle M to a driving mode with a heavier task.

[0053] For example, in Mode A, if the driver is in a posture that prevents them from switching to manual driving as requested by the system (e.g., continuing to look out of the permitted area, or detecting signs of driving difficulty), the mode determination unit 150 uses the HMI 30 to urge the driver to switch to manual driving. If the driver does not respond, it performs control such as gradually stopping the vehicle M by pulling it towards the curb, or stopping the automatic driving. After stopping the automatic driving, the vehicle enters Mode D or E, and can be started manually by the driver. The same applies to stopping the automatic driving. In Mode B, if the driver is not monitoring the road ahead, the mode determination unit 150 uses the HMI 30 to urge the driver to monitor the road ahead. If the driver does not respond, it performs control such as gradually stopping the vehicle M by pulling it towards the curb, or stopping the automatic driving. In Mode C, if the driver is not monitoring the situation ahead or is not holding the steering wheel 82, the mode decision unit 150 uses HMI 30 to urge the driver to monitor the situation ahead and / or hold the steering wheel 82. If the driver does not respond, the system will perform controls such as gradually stopping the vehicle M by moving it toward the curb or stopping the automatic driving.

[0054] The second control unit 160 controls the driving force output device 200, the braking device 210 and the steering device 220 so that the vehicle M passes through the target track generated by the action plan generation unit 140 at a predetermined time.

[0055] return Figure 2The second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires information about the target track (track point) generated by the action plan generation unit 140 and stores this information in a memory (not shown). The speed control unit 164 controls the driving force output device 200 or the braking device 210 based on the speed elements associated with the target track stored in the memory. The steering control unit 166 controls the steering device 220 based on the curvature of the target track stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is achieved, for example, through a combination of feedforward control and feedback control. As an example, the steering control unit 166 combines feedforward control corresponding to the curvature of the road ahead of the vehicle M with feedback control based on deviation from the target track.

[0056] The driving force output device 200 outputs driving force (torque) for vehicle movement to the drive wheels. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, as well as an ECU (Electronic Control Unit) that controls them. The ECU controls the above-mentioned structure according to information input from the second control unit 160 or from the driving operation device 80.

[0057] The braking device 210 includes, for example, a brake caliper, a hydraulic cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the hydraulic cylinder, and a braking ECU. The braking ECU controls the electric motor according to information input from the second control unit 160 or from the driving operation unit 80, so that braking torque corresponding to the braking operation is output to each wheel. The braking device 210 may have a mechanism for transmitting hydraulic pressure generated by the operation of the brake pedal included in the driving operation unit 80 to the hydraulic cylinder via the master hydraulic cylinder as a backup. It should be noted that the braking device 210 is not limited to the structure described above, and may also be an electronically controlled hydraulic braking device that transmits hydraulic pressure from the master hydraulic cylinder to the hydraulic cylinder by controlling the actuator according to information input from the second control unit 160.

[0058] The steering system 220 includes, for example, a steering ECU and an electric motor. The electric motor applies force to a rack and pinion mechanism to change the direction of the steering wheels. The steering ECU drives the electric motor to change the direction of the steering wheels according to information input from the second control unit 160 or from the driving operation unit 80.

[0059] [Action of vehicle control devices]

[0060] Next, the operation of the vehicle control device according to the embodiment will be explained. Figure 4 This diagram illustrates an example of a scenario where the operation of the vehicle control device according to the embodiment is performed. Figure 4 In this scenario, vehicle M is traveling in lane L1, with another vehicle M1 traveling ahead of it. While vehicle M is traveling in lane L1, the recognition unit 130 identifies the surrounding conditions of vehicle M, particularly the road markings on both sides of vehicle M, based on images captured by camera 10. Hereinafter, the road markings identified based on images captured by camera 10 will be represented by CL (hereinafter referred to as "camera road markings CL"), and the road markings identified based on second map information 62 will be represented by ML (hereinafter referred to as "map road markings ML"). In the following description, it is assumed that vehicle M is traveling in mode B, which uses camera road markings CL (as reference lines).

[0061] The deviation determination unit 152 determines whether there is a deviation between the camera road dividing line CL and the map road dividing line ML. Here, deviation refers to, for example, a distance between the camera road dividing line CL and the map road dividing line ML that is greater than or equal to a predetermined value, or an angle formed by the camera road dividing line CL and the map road dividing line ML that is greater than or equal to a predetermined value.

[0062] If the deviation determination unit 152 determines that there is a deviation between the camera road demarcation line CL and the map road demarcation line ML, it then determines whether a preceding vehicle M1 exists within the range of the camera road demarcation line CL. Generally, it is assumed that if the identified camera road demarcation line CL is correct, the preceding vehicle M1 will be traveling within the range of the camera road demarcation line CL; therefore, this determination process can be considered a process for confirming the reliability of the camera road demarcation line CL. Figure 4 In the situation shown, the deviation determination unit 152 determines that the vehicle is traveling within the area of ​​the camera road dividing line CL.

[0063] If it is determined that there is a preceding vehicle M1 within the range of the camera road dividing line CL, the redundancy calculation unit 154 calculates the redundancy for driving mode B of vehicle M based on the distance from the vehicle M to the preceding vehicle M1, the distance from the preceding vehicle M1 to the point where the preceding vehicle M1 intersects with the camera road dividing line CL, and the speed of the vehicle M.

[0064] More specifically, the redundancy calculation unit 154 first determines the intersection point IP between the extension of the track of the leading vehicle M1 and the camera road dividing line CL. The redundancy calculation unit 154 then calculates the redundancy P based on the distance D1 from the current vehicle M to the leading vehicle M1, the distance D2 from the leading vehicle M1 to line PL (the line passing through intersection point IP and perpendicular to the camera road dividing line CL), and the speed V of the current vehicle M, using the formula P = (D1 + D2) / V [sec]. That is, the higher the value of the redundancy P, the more parallel the leading vehicle M1 is to the camera road dividing line CL, and the higher the reliability of the camera road dividing line CL. In other words, the redundancy P is an example of the "degree of interference" between the camera road dividing line CL and the leading vehicle M1. Alternatively, whether the trajectory of the camera road dividing line CL intersects with that of the leading vehicle M1 can be calculated as the degree of interference between the camera road dividing line CL and the leading vehicle M1.

[0065] The deviation determination unit 152 then determines whether the calculated margin P is above or below a first threshold (i.e., whether the degree of interference is below the first threshold). If it is determined that the calculated margin P is less than the first threshold, this means that the reliability of the camera road dividing line CL is relatively low. Therefore, the mode determination unit 150 changes the driving mode of mode B, which uses the camera road dividing line CL, to the driving mode of mode C, which uses the camera road dividing line CL.

[0066] If the deviation determination unit 152 determines that the calculated redundancy P is above the first threshold, it then determines whether the degree of deviation between the camera road dividing line CL and the map road dividing line ML is within the second threshold. Even if the preceding vehicle M1 is within the range of the camera road dividing line CL and the redundancy P is above the first threshold, the preceding vehicle M1 may sometimes travel along the wrong camera road dividing line CL. Therefore, this determination process is used to more reliably evaluate the reliability of the camera road dividing line CL.

[0067] Figure 5 This is an example of a scenario where the preceding vehicle M1 is traveling along the incorrect camera road demarcation line CL. Figure 5 In this context, it is assumed that the road demarcation lines ML on the map correspond to the actual roads. Figure 5 In the scenario shown, the leading vehicle M1 exists within the camera road demarcation line CL and is traveling approximately parallel to it. Therefore, the margin P is calculated to exceed the first threshold. However, in reality, the driving path along the camera road demarcation line CL differs from the actual road. Therefore, relying on the camera road demarcation line CL to continue driving mode B is problematic.

[0068] Therefore, even if the preceding vehicle M1 is within the range of the camera road dividing line CL and the margin P is above the first threshold, the deviation determination unit 152 then determines whether the deviation between the camera road dividing line CL and the map road dividing line ML is below the second threshold. For example, the deviation determination unit 152 determines whether the angle θ formed by the camera road dividing line CL and the map road dividing line ML is below the second threshold.

[0069] If the deviation determination unit 152 determines that the deviation between the camera road dividing line CL and the map road dividing line ML is below a second threshold, the mode determination unit 150 continues driving mode B, which uses the camera road dividing line CL. On the other hand, if the deviation between the camera road dividing line CL and the map road dividing line ML is determined to be greater than the second threshold, the mode determination unit 150 changes the driving mode B, which uses the camera road dividing line CL, to driving mode C, which uses the camera road dividing line CL. In this way, even if the preceding vehicle M1 is driving along an incorrect camera road dividing line CL, the reliability of the camera road dividing line CL can be evaluated more reliably by checking the deviation between the camera road dividing line CL and the map road dividing line ML.

[0070] Figure 6 This diagram illustrates the process performed when the deviation determination unit 152 determines that there is no preceding vehicle M1. If the deviation determination unit 152 determines that there is no preceding vehicle M1 within the camera road dividing line CL, it then determines whether there are other vehicles cutting into the camera road dividing line CL. This is because, generally, when other vehicles are crossing the camera road dividing line CL, the reliability of the camera road dividing line CL tends to be relatively lower due to interference caused by these other vehicles.

[0071] If the deviation determination unit 152 determines that there are no other vehicles crossing the camera road dividing line CL, it means that it is unclear which of the two road dividing lines, the camera road dividing line CL and the map road dividing line ML, is more reliable. Therefore, the mode determination unit 150 changes the driving mode from mode B, which uses the camera road dividing line CL, to mode C, which also uses the camera road dividing line CL.

[0072] On the other hand, if it is determined that there are other vehicles crossing the camera road demarcation line CL, the deviation determination unit 152 then determines whether the rate of curvature change of the camera road demarcation line CL is above a third threshold. If it is determined that the rate of curvature change of the camera road demarcation line CL is above the third threshold, this means that the reliability of the map road demarcation line ML is higher than that of the camera road demarcation line CL. Therefore, the mode determination unit 150 changes the driving mode of mode B using the camera road demarcation line CL to the driving mode of mode B using the map road demarcation line ML. On the other hand, if it is determined that the rate of curvature change of the camera road demarcation line CL is less than the third threshold, this means that it is unclear which of the camera road demarcation line CL and the map road demarcation line ML is more reliable. Therefore, the mode determination unit 150 changes the driving mode of mode B using the camera road demarcation line CL to the driving mode of mode C using the camera road demarcation line CL.

[0073] Next, refer to Figure 7 and Figure 8 The flow of actions performed by the vehicle control device according to the implementation method will be explained. Figure 7 This is a flowchart illustrating an example of the flow of actions performed by the vehicle control device according to the embodiment. The processes involved in this flowchart are executed in a prescribed cycle while the vehicle M is traveling in mode B, which uses camera-based road markings CL.

[0074] First, the pattern determination unit 150 obtains the camera road dividing line CL and the map road dividing line ML via the recognition unit 130 (step S100). Next, the deviation determination unit 152 determines whether there is a deviation between the obtained camera road dividing line CL and the map road dividing line ML (step S101).

[0075] Next, if the deviation determination unit 152 determines that there is a deviation between the acquired camera road demarcation line CL and the map road demarcation line ML, it determines whether there is a preceding vehicle M1 within the range of the camera road demarcation line CL (step S102). If it determines that there is no preceding vehicle M1 within the range of the camera road demarcation line CL, the deviation determination unit 152 advances the process to S201.

[0076] On the other hand, if it is determined that there is a preceding vehicle M1 within the area of ​​the camera road dividing line CL, the redundancy calculation unit 154 calculates the redundancy P based on the distance from the current vehicle M to the preceding vehicle M1, the distance from the preceding vehicle M1 to the point where the preceding vehicle M1 intersects with the camera road dividing line CL, and the speed of the current vehicle M (step S103). Next, the deviation determination unit 152 determines whether the calculated redundancy P is above the first threshold (step S104).

[0077] If the calculated margin P is determined to be less than a first threshold, the mode determination unit 150 changes the driving mode of mode B, which uses the camera road dividing line CL, to the driving mode of mode C, which uses the camera road dividing line CL (step S105). On the other hand, if the calculated margin P is determined to be greater than or equal to the first threshold, the deviation determination unit 152 then determines whether the deviation between the camera road dividing line CL and the map road dividing line ML is within a second threshold (step S106). If the deviation between the camera road dividing line CL and the map road dividing line ML is determined to be greater than the second threshold, the mode determination unit 150 performs the processing in step S105. On the other hand, if the deviation between the camera road dividing line CL and the map road dividing line ML is determined to be less than or equal to the second threshold, the mode determination unit 150 continues the driving mode of mode B, which uses the camera road dividing line CL (step S107). Thus, the processing of this flowchart ends.

[0078] Figure 8 This is a flowchart illustrating an example of the flow of actions performed by the vehicle control device of the embodiment. Figure 8 The flowchart shown illustrates the processing in... Figure 7 The flowchart shown indicates that the process is executed if there is no preceding vehicle M1 within the area defined by the camera's road dividing line CL.

[0079] First, the deviation determination unit 152 determines whether any other vehicle is crossing the camera road dividing line CL (step S201). If it is determined that no other vehicle is crossing the camera road dividing line CL, the mode determination unit 150 changes the driving mode from mode B (which uses the camera road dividing line CL) to mode C (which uses the camera road dividing line CL) (step S202). On the other hand, if it is determined that other vehicles are crossing the camera road dividing line CL, the deviation determination unit 152 determines whether the rate of curvature change of the camera road dividing line CL is above a third threshold (step S203).

[0080] If the curvature change rate of the camera road dividing line CL is determined to be less than the third threshold, the mode determination unit 150 performs the processing in step S202. On the other hand, if the curvature change rate of the camera road dividing line CL is determined to be greater than or equal to the third threshold, the mode determination unit 150 changes the driving mode using mode B that uses camera road dividing line CL to driving mode B that uses map road dividing line ML (step S204). Thus, the processing of this flowchart ends.

[0081] It should be noted that in the above flowchart processing, in step S102, if it is determined that there is no preceding vehicle M1 within the range of the camera road dividing line CL, then based on the presence of other vehicles crossing the camera road dividing line CL and the rate of curvature change of the camera road dividing line CL, it is determined whether to execute the driving mode using mode B which uses map road dividing line ML, or the driving mode using mode C which uses camera road dividing line CL. However, the present invention is not limited to that structure. It is also possible that, in the absence of preceding vehicle M1, it is unclear which of the camera road dividing line CL and map road dividing line ML has higher reliability, and the driving mode using mode C which uses camera road dividing line CL is changed.

[0082] According to this embodiment as described above, when a deviation occurs between the camera's road markings and the map's road markings, and a preceding vehicle exists within the area of ​​the camera's road markings, the degree of interference between the preceding vehicle and the camera's road markings is determined, and the vehicle's autonomous driving mode is controlled based on the determined degree of interference. Therefore, even when the road markings recognized by the camera differ from the map information carried by the vehicle, the vehicle's driving control can be flexibly adjusted.

[0083] The implementation methods described above can be performed as follows.

[0084] A vehicle control device, wherein...

[0085] The vehicle control device includes:

[0086] A storage device containing a program; and

[0087] Hardware processor,

[0088] The hardware processor executes the program stored in the storage device to perform the following processing:

[0089] Obtain camera images of the vehicle's surroundings;

[0090] Based on the camera images and map information, the vehicle's steering and acceleration / deceleration are controlled independently of the driver's operation.

[0091] The driving mode of the vehicle is determined to be any one of a plurality of driving modes, including a first driving mode and a second driving mode. The second driving mode is a driving mode with a lighter task assigned to the driver compared to the first driving mode. At least some of the plurality of driving modes, including the second driving mode, are controlled by steering and acceleration / deceleration of the vehicle independently of the driver's operation. If the task involved in the determined driving mode is not performed by the driver, the driving mode of the vehicle is changed to a driving mode with a heavier task.

[0092] Determine whether there is a deviation between the road markings shown in the camera image and the road markings shown in the map information; if the deviation is determined to exist, determine whether there is a preceding vehicle; and

[0093] If it is determined that the preceding vehicle is present and the degree of interference between the road markings shown in the camera image and the preceding vehicle is below a first threshold, driving in the second driving mode using the road markings shown in the camera image continues. If it is not determined that the preceding vehicle is present, the second driving mode is changed to the first driving mode using the road markings shown in the camera image.

[0094] The above description illustrates specific embodiments of the present invention, but the present invention is not limited to such embodiments in any way, and various modifications and substitutions can be made without departing from the spirit of the present invention.

Claims

1. A vehicle control device, wherein, The vehicle control device includes: The acquisition unit acquires camera images of the vehicle's surroundings. The driving control unit controls the steering and acceleration / deceleration of the vehicle based on the camera images and map information, without relying on the driver's operation. The mode determination unit determines the driving mode of the vehicle as any one of a plurality of driving modes, including a first driving mode and a second driving mode. The second driving mode is a driving mode with a lighter task assigned to the driver compared to the first driving mode. At least a portion of the plurality of driving modes, including the second driving mode, is controlled by the driving control unit. If the task involved in the determined driving mode is not performed by the driver, the mode determination unit changes the driving mode of the vehicle to a driving mode with a heavier task. as well as The determination unit determines whether there is a deviation between the road markings shown in the camera image and the road markings shown in the map information. If the deviation is determined to exist, it determines whether there is a preceding vehicle. If it is determined that the preceding vehicle is present and the degree of interference between the road markings shown in the camera image and the preceding vehicle is below a first threshold, the mode determination unit allows driving in the second driving mode using the road markings shown in the camera image to continue. If it is not determined that the preceding vehicle is present, the mode determination unit changes the second driving mode to the first driving mode using the road markings shown in the camera image.

2. The vehicle control device according to claim 1, wherein, The vehicle control device further includes a calculation unit that calculates, based on the distance from the vehicle to the preceding vehicle, the distance from the preceding vehicle to the point where the preceding vehicle intersects with the road dividing line shown in the camera image, and the speed of the vehicle, a margin for the vehicle to continue driving in the second driving mode, as the degree of interference.

3. The vehicle control device according to claim 1 or 2, wherein, If the determination unit determines that the preceding vehicle exists and the degree of interference is below the first threshold, it determines whether the deviation between the road dividing lines shown in the camera image and the road dividing lines shown in the map information is below the second threshold. If the determination unit determines that the deviation is greater than the second threshold, the mode determination unit changes the second driving mode, which uses the road markings shown in the camera image, to the first driving mode, which uses the road markings shown in the camera image.

4. The vehicle control device according to claim 1, wherein, When the determination unit determines that the preceding vehicle exists and the degree of interference is greater than the first threshold, the mode determination unit changes the second driving mode that uses the road markings shown in the camera image to the first driving mode that uses the road markings shown in the camera image.

5. The vehicle control device according to any one of claims 1 to 4, wherein, If the determination unit determines that the preceding vehicle does not exist, it determines whether there are other vehicles crossing the road markings shown in the camera image. If it determines that there are other vehicles, it determines whether the rate of curvature change of the road markings shown in the camera image is above a third threshold. If the determination unit determines that the curvature change rate is above the third threshold, the mode determination unit changes the second driving mode that uses the road dividing lines shown in the camera image to the second driving mode that uses the road dividing lines shown in the map information.

6. The vehicle control device according to claim 1, wherein, The second driving mode is a driving mode in which the driver is not assigned the task of controlling the steering components of the vehicle. The first driving mode is a driving mode in which the driver is assigned at least the task of holding the control components.

7. A vehicle control method, wherein, The vehicle control method causes the computer to perform the following processing: Obtain camera images of the vehicle's surroundings; Based on the camera images and map information, the vehicle's steering and acceleration / deceleration can be controlled independently of the driver's operation. The driving mode of the vehicle is determined to be any one of a plurality of driving modes, including a first driving mode and a second driving mode. The second driving mode is a driving mode with a lighter task assigned to the driver compared to the first driving mode. At least some of the plurality of driving modes, including the second driving mode, are controlled by steering and acceleration / deceleration of the vehicle independently of the driver's operation. If the task involved in the determined driving mode is not performed by the driver, the driving mode of the vehicle is changed to a driving mode with a heavier task. Determine whether there is a deviation between the road dividing lines shown in the camera image and the road dividing lines shown in the map information; if the deviation is determined to exist, determine whether the vehicle has a preceding vehicle. If it is determined that the preceding vehicle is present and the degree of interference between the road markings shown in the camera image and the preceding vehicle is below a first threshold, driving in the second driving mode using the road markings shown in the camera image continues. If it is not determined that the preceding vehicle is present, the second driving mode is changed to the first driving mode using the road markings shown in the camera image.

8. A storage medium storing a program, wherein, The program causes the computer to perform the following processing: Obtain camera images of the vehicle's surroundings; Based on the camera images and map information, the vehicle's steering and acceleration / deceleration are controlled independently of the driver's operation. The driving mode of the vehicle is determined to be any one of a plurality of driving modes, including a first driving mode and a second driving mode. The second driving mode is a driving mode with a lighter task assigned to the driver compared to the first driving mode. At least some of the plurality of driving modes, including the second driving mode, are controlled by steering and acceleration / deceleration of the vehicle independently of the driver's operation. If the task involved in the determined driving mode is not performed by the driver, the driving mode of the vehicle is changed to a driving mode with a heavier task. Determine whether there is a deviation between the road dividing lines shown in the camera image and the road dividing lines shown in the map information; if the deviation is determined to exist, determine whether the vehicle has a preceding vehicle. If it is determined that the preceding vehicle is present and the degree of interference between the road markings shown in the camera image and the preceding vehicle is below a first threshold, driving in the second driving mode using the road markings shown in the camera image continues. If it is not determined that the preceding vehicle is present, the second driving mode is changed to the first driving mode using the road markings shown in the camera image.

Images