Vehicle control device, vehicle control method and storage medium

Abstract

A vehicle control device, vehicle control method, and storage medium capable of performing more appropriate driving controls based on the recognition status of road markings. The vehicle control device includes: a first recognition unit that identifies a first road marking dividing the vehicle's driving lane based on the output of a detection device that detects the surrounding conditions of the vehicle; a second recognition unit that identifies a second road marking dividing the driving lane using a different means than the first recognition unit; a comparison unit that compares the first road marking with the second road marking; and a determination unit that, when a difference is found between the first road marking and the second road marking in the comparison result obtained by the comparison unit, determines any one of a plurality of misidentification determinations, including determining that the first recognition unit has misidentified the vehicle, and determining that one or both of the first and second recognition units have misidentified the vehicle.

Description

Technical Field

[0001] This invention relates to vehicle control devices, vehicle control methods, and storage media. Background Technology

[0002] In recent years, research on automatically controlling vehicle movement has made continuous progress. Relatedly, an invention of a vehicle movement support device has been disclosed that, upon detecting a left-right difference in shape between a left-side and right-side lane marking on the road, infers the lane marking in the vehicle's direction of travel based on past lane marking information, determines the road's lane conditions based on the shape difference, and controls the vehicle's movement based on the lane conditions (e.g., Japanese Patent No. 6790187). Summary of the Invention

[0003] In autonomous driving, sometimes the following control is performed: a first road marking line identified from an image captured by an onboard camera is compared with a second road marking line identified from map information. If they match, autonomous driving continues; if they do not match, autonomous driving terminates. In this case, sometimes it is not possible to perform appropriate driving control based on the road marking line recognition status.

[0004] The present invention was made in consideration of such circumstances, and its purpose is to provide a vehicle control device, vehicle control method and storage medium capable of performing more appropriate driving control based on the identification status of road markings.

[0005] The vehicle control device, vehicle control method, and storage medium of the present invention adopt the following structure.

[0006] (1): A vehicle control device according to one aspect of the present invention includes: a first identification unit that identifies road markings dividing the driving lane of the vehicle based on the output of a detection device that detects the surrounding conditions of the vehicle; a second identification unit that identifies the road markings dividing the driving lane by a different means than the first identification unit; a comparison unit that compares a first road marking identified by the first identification unit with a second road marking identified by the second identification unit; and a determination unit that, when a difference is found between the first road marking and the second road marking in the comparison result of the comparison unit, determines any one of a plurality of misidentification determinations, including a case where the first identification unit misidentifies, and a case where one or both of the first identification unit and the second identification unit misidentify.

[0007] (2): In the above (1) scheme, the vehicle control device further includes a driving control unit, which controls at least one of the acceleration, deceleration and steering of the vehicle, and the driving control unit executes any one of a plurality of driving modes with different tasks assigned to the occupants of the vehicle based on the determination result determined by the determination unit.

[0008] (3): In the above-mentioned (1) scheme, the vehicle control device further includes an output control unit, which, based on the determination result determined by the determination unit, causes the output device to output information or warnings related to the state of the vehicle and reports to the occupants of the vehicle.

[0009] (4): In the above (2) scheme, the multiple driving modes include a first driving mode and a second driving mode that is more demanding on the occupants than the first driving mode. When the first driving mode is being executed and the determination unit determines that the first identification unit has misidentified the first driving mode, the driving control unit continues the first driving mode based on the second road dividing line.

[0010] (5): In the above (4) scheme, when the driving control unit is executing the first driving mode and the determination unit determines that one or both of the first identification unit and the second identification unit have misidentified the vehicle, the driving control unit changes the driving mode of the vehicle from the first driving mode to the second driving mode.

[0011] (6): In the above (1) scheme, the determination unit makes a determination of any one of the multiple misidentification determinations based on the determination conditions based on the curvature change of the first road dividing line and the angle formed by the first road dividing line and the second road dividing line.

[0012] (7): In the above (6) scheme, the determination unit changes the determination conditions based on the surrounding conditions of the vehicle.

[0013] (8): In the above (7) scheme, when there is a branch, convergence, tunnel entrance or exit in the direction of travel of the vehicle, or when the vehicle in front is changing lanes or driving in a winding manner, the determination unit changes the first determination condition and the second determination condition in a way that is easy to determine as a misidentification by the first identification unit.

[0014] (9): In the above (7) scheme, if there is a curve entrance or exit in the direction of travel of the vehicle, the determination unit changes the determination condition in a way that suppresses the determination that the first identification unit misidentifies.

[0015] (10): A vehicle control method according to one aspect of the present invention causes a computer of a vehicle control device to perform the following processing: based on the output of a detection device that detects the surrounding conditions of the vehicle, identify a first road dividing line that divides the driving lane of the vehicle; identify a second road dividing line that divides the driving lane using a different means than the means by which the first road dividing line was identified; compare the identified first road dividing line with the second road dividing line; and, if a difference is found between the first road dividing line and the second road dividing line in the comparison result, determine any one of a plurality of misidentification determinations, the plurality of misidentification determinations including determining that the first road dividing line is a misidentified dividing line, and determining that one or both of the first road dividing line and the second road dividing line are misidentified dividing lines.

[0016] (11): A storage medium of one aspect of the present invention stores a program, wherein the program causes a computer of a vehicle control device to perform the following processing: based on the output of a detection device that detects the surrounding conditions of the vehicle, identify a first road dividing line that divides the driving lane of the vehicle; identify a second road dividing line that divides the driving lane using a different means than the means by which the first road dividing line was identified; compare the identified first road dividing line with the second road dividing line; and, if a difference is found between the first road dividing line and the second road dividing line in the comparison result, determine any one of a plurality of misidentification determinations, the plurality of misidentification determinations including determining that the first road dividing line is a misidentified dividing line, and determining that one or both of the first road dividing line and the second road dividing line are misidentified dividing lines.

[0017] According to the schemes (1) to (11) above, more appropriate driving control can be performed based on the recognition status of road markings. Attached Figure Description

[0018] Figure 1 It is a structural diagram of a vehicle system, including the vehicle control device of the implementation method.

[0019] Figure 2 This is a functional structure diagram of the first control unit and the second control unit.

[0020] Figure 3 This is a diagram illustrating an example of the correspondence between driving modes, the control state of vehicle M, and tasks.

[0021] Figure 4 This diagram illustrates the processing procedures of the first identification unit, the second identification unit, the comparison unit, and the misidentification determination unit.

[0022] Figure 5 It is a graph used to illustrate the degree of deviation of the amount of curvature change.

[0023] Figure 6 This is a diagram used to illustrate the peeling angle.

[0024] Figure 7 This diagram illustrates the misidentification criteria using curvature variation and peeling angle.

[0025] Figure 8 It is a diagram used to illustrate the change of area based on the surrounding conditions of vehicle M.

[0026] Figure 9 This diagram illustrates the changes made to suppress situations where the first identification unit is determined to be a misidentification.

[0027] Figure 10 This is a flowchart illustrating an example of the processing flow performed by an autonomous driving control device.

[0028] Figure 11 This is a flowchart illustrating an example of the processing flow for step S106. Detailed Implementation

[0029] Hereinafter, embodiments of the vehicle control device, vehicle control method, and storage medium of the present invention will be described with reference to the accompanying drawings. An example of the vehicle control device being applied to an autonomous vehicle will be described below. Autonomous driving, for example, automatically controls one or both of the vehicle's steering and speed to perform driving control. Driving control may include various driving controls such as LKAS (Lane Keeping Assistance System), ALC (Auto Lane Changing), ACC (Adaptive Cruise Control System), and CMBS (Collision Mitigation Brake System). Driving control may also include driver assistance controls such as ADAS (Advanced Driver Assistance System). Autonomous vehicles can also be controlled by manual driving by the occupant (driver).

[0030] [Overall Structure]

[0031] Figure 1This is a structural diagram of vehicle system 1, including the vehicle control device of the embodiment. The vehicle equipped with vehicle system 1 (hereinafter referred to as vehicle M) is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using electricity generated by a generator connected to the internal combustion engine, or electricity discharged from a battery such as a secondary battery or a fuel cell.

[0032] Vehicle system 1 includes, for example, a camera 10, a radar device 12, a LiDAR (Light Detection and Ranging) device 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, vehicle sensors 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driver monitoring camera 70, driving controls 80, an automatic driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are interconnected through multiple communication lines such as CAN (Controller Area Network) communication lines, serial communication lines, and wireless communication networks. Figure 1 The structure shown is merely an example; some parts of the structure may be omitted, and other structures may be added. The automatic driving control device 100 is an example of a "vehicle control device." Combining the camera 10, radar device 12, and LIDAR 14 object recognition device 16 is an example of a "detection device DD." The HMI 30 is an example of an "output device."

[0033] Camera 10 is, for example, a digital camera utilizing a solid-state imaging element such as CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor). Camera 10 is mounted anywhere on the vehicle M equipped with vehicle system 1. When shooting forward, camera 10 is mounted on the upper part of the windshield, behind the interior rearview mirror, or at the front of the vehicle body. When shooting backward, camera 10 is mounted on the upper part of the rear windshield or on the tailgate. When shooting to the side, camera 10 is mounted on the side mirror on the door. Camera 10 periodically and repeatedly captures images of the surrounding area of ​​vehicle M. Camera 10 can also be a stereo camera.

[0034] Radar device 12 radiates millimeter-wave or other radio waves around the vehicle M and detects the radio waves (reflected waves) reflected by surrounding objects to at least detect the position (distance and orientation) of the objects. Radar device 12 can be installed at any part of the vehicle M. Radar device 12 can also detect the position and speed of objects using FM-CW (Frequency Modulated Continuous Wave) method.

[0035] The LIDAR14 illuminates the perimeter of vehicle M and measures the scattered light. The LIDAR14 determines the distance to the object based on the time from emission to reception. The illuminating light can be, for example, a pulsed laser. The LIDAR14 can be mounted at any location on vehicle M.

[0036] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the cameras 10, radar device 12, and LIDAR 14 to identify the position, type, speed, etc., of objects surrounding the vehicle M. Objects include, for example, other vehicles (e.g., surrounding vehicles within a specified distance of the vehicle M), pedestrians, bicycles, road structures, etc. Road structures include, for example, road signs, traffic signals, intersections, curbs, median strips, guardrails, fences, etc. Road structures may also include, for example, road markings drawn or affixed to the road surface (hereinafter simply referred to as "marking lines"), pedestrian crossings, bicycle crossings, temporary stop lines, etc. The object recognition device 16 outputs the recognition results to the automatic driving control device 100. The object recognition device 16 can also directly output the detection results from the cameras 10, radar device 12, and LIDAR 14 to the automatic driving control device 100. In this case, the object recognition device 16 can also be omitted from the structure of the vehicle system 1 (specifically, the detection device DD). The object recognition device 16 may also be included in the automatic driving control device 100.

[0037] The communication device 20 uses networks such as cellular networks, Wi-Fi networks, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), LAN (Local Area Network), WAN (Wi-Fi Area Network), and the Internet to communicate with other vehicles existing around vehicle M, terminal devices of users of vehicle M, or various server devices.

[0038] The HMI30 outputs various information to the occupants of vehicle M and accepts input operations performed by the occupants. The HMI30 includes, for example, various display devices, speakers, buzzers, touch panels, switches, buttons, microphones, etc.

[0039] Vehicle sensor 40 includes a vehicle speed sensor for detecting the speed of vehicle M, an acceleration sensor for detecting acceleration, a yaw rate sensor for detecting yaw rate (e.g., rotational angular velocity about a vertical axis passing through the center of gravity of vehicle M), and an orientation sensor for detecting the orientation of vehicle M. Vehicle sensor 40 may also include a position sensor for detecting the position of vehicle M. The position sensor may be a sensor that obtains position information (longitude and latitude information) from a GPS (Global Positioning System) device. Alternatively, the position sensor may be a sensor that obtains position information using a GNSS (Global Navigation Satellite System) receiver 51 of navigation device 50. The results detected by vehicle sensor 40 are output to automatic driving control device 100.

[0040] The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as an HDD (Hard Disk Drive) or flash memory. The GNSS receiver 51 determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M can also be determined or supplemented using the INS (Inertial Navigation System) output from the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, buttons, etc. The GNSS receiver 51 may also be installed on the vehicle sensor 40. The navigation HMI 52 may also be partially or entirely shared with the aforementioned HMI 30. The route determination unit 53, for example, refers to the first map information 54 to determine the route (hereinafter referred to as the map path) from the position of the vehicle M determined by the GNSS receiver 51 (or any input position) to the destination input by the occupant using the navigation HMI 52. The first map information 54 is, for example, information showing the road shape by representing road segments of a defined section and nodes connecting the road segments. The first map information 54 may also include POI (Point of Interest) information, etc. The path on the map is output to the MPU 60. The navigation device 50 can also provide route guidance using the navigation HMI 52 based on the path on the map. The navigation device 50 can also send its current location and destination to the navigation server via the communication device 20, and obtain a path equivalent to the path on the map from the navigation server. The navigation device 50 outputs the determined path on the map to the MPU 60.

[0041] MPU 60 includes, for example, a lane recommendation unit 61, and stores second map information 62 in a storage device such as an HDD or flash memory. The lane recommendation unit 61 divides the path on the map provided by the navigation device 50 into multiple blocks (e.g., every 100 [m] in the vehicle's direction of travel), and determines a recommended lane for each block by referring to the second map information 62. The lane recommendation unit 61, for example, determines which lane to drive in from the left. Lanes are defined by dividing lines. When the path on the map branches, the lane recommendation unit 61 determines a recommended lane so that the vehicle M can travel on a reasonable path to the branch destination.

[0042] The second map information 62 is map information with higher precision than the first map information 54. The second map information 62 includes, for example, road shapes and information related to road structures. Road shapes, as more detailed than those in the first map information 54, include, for example, branching, merging, tunnels (entrances, exits), curves (entrances, exits), the radius of curvature (or curvature) of roads or dividing lines, the amount of curvature change, the number of lanes, width, and gradient. This information can also be stored in the first map information 54. Information related to road structures can include the type, location, orientation relative to the road's extension direction, size, shape, and color of the road structure. Among the categories of road structures, for example, dividing lines can be set as one category, or lane markings, curbs, and medians to which dividing lines belong can be set as different categories. The categories of dividing lines can also include, for example, dividing lines indicating lane changes are permitted and dividing lines indicating lane changes are not permitted. The categories of dividing lines can be set for each road segment or each lane interval, or multiple categories can be set within a single road segment.

[0043] The second map information 62 may include location information (latitude and longitude) of roads and buildings, residential information (address, postal code), facility information, etc. The second map information 62 can communicate with external devices via the communication device 20 and be updated in real time. The first map information 54 and the second map information 62 can also be set up as a single map information unit. The map information (first map information 54 and second map information 62) can also be stored in the storage unit 190.

[0044] The driver monitoring camera 70 is, for example, a digital camera utilizing a solid-state imaging element such as a CCD or CMOS sensor. The driver monitoring camera 70 can be installed anywhere in the vehicle M, positioned and facing forward (facing the face), to capture images of the head of the driver, passenger, and other occupants in the rear seats. For example, it can be installed above a display device located in the center of the dashboard, above the windshield, or in the rearview mirror. The driver monitoring camera 70 periodically and repeatedly captures images including those inside the vehicle.

[0045] The driving control unit 80 includes, for example, a steering wheel 82, an accelerator pedal, a brake pedal, a gear shift lever, and other operating components. Sensors are installed in the driving control unit 80 to detect the amount or presence of operation, and the detection results are output to some or all of the following: the automatic driving control unit 100, the driving force output device 200, the braking device 210, and the steering device 220. The steering wheel 82 is an example of an operating component that receives steering operations performed by the driver. The operating component does not necessarily have to be ring-shaped; it can also be an irregularly shaped steering wheel, a lever, a button, etc. A steering wheel grip sensor 84 is installed in the steering wheel 82. The steering wheel grip sensor 84, implemented by a capacitance sensor or the like, outputs a signal to the automatic driving control unit 100 that detects whether the driver is gripping the steering wheel 82 (meaning contacting it with applied force).

[0046] The autonomous driving control device 100 includes, for example, a first control unit 120, a second control unit 160, an HMI control unit 180, and a storage unit 190. The first control unit 120, the second control unit 160, and the HMI control unit 180 are each implemented by executing programs (software) via hardware processors such as CPUs (Central Processing Units). Some or all of these components can also be implemented using hardware (including circuitry) such as LSIs (Large Scale Integration), ASICs (Application Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays), and GPUs (Graphics Processing Units), or through the coordinated use of software and hardware. The aforementioned programs can be pre-stored in a storage device such as an HDD or flash memory (a storage device with a non-transitory storage medium) of the autonomous driving control device 100, or stored in a removable storage medium such as a DVD, CD-ROM, or memory card, and installed in the storage device of the autonomous driving control device 100 by mounting the storage medium (a non-transitory storage medium) to a drive unit, card slot, etc. The action plan generation unit 140 and the second control unit 160 together constitute an example of a "driving control unit". The HMI control unit 180 is an example of an "output control unit".

[0047] The storage unit 190 can also be implemented using the various storage devices described above, or EEPROM (Electrically Erasable Programmable Read Only Memory), ROM (Read Only Memory), or RAM (Random Access Memory). The storage unit 190 stores, for example, information, programs, and other various types of information required for executing various controls in the implementation method. Map information (first map information 54, second map information 62) may also be stored in the storage unit 190.

[0048] Figure 2This is a functional structure diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130, an action plan generation unit 140, and a pattern determination unit 150. The first control unit 120, for example, performs AI (Artificial Intelligence) based functions and functions based on pre-given models in parallel. For example, the function of "recognizing intersections" can be achieved by "parallel execution of intersection recognition based on deep learning, etc., and recognition based on pre-given conditions (the existence of signals that can be pattern-matched, road signs, etc.), and comprehensively evaluating both." This ensures the reliability of autonomous driving. The first control unit 120, for example, performs controls related to the autonomous driving of the vehicle M based on instructions from the MPU 60, HMI control unit 180, etc.

[0049] The identification unit 130 identifies the surrounding conditions of the vehicle M based on the identification results of the detection device DD (information input from the camera 10, radar device 12, and LIDAR 14 via the object recognition device 16). For example, the identification unit 130 identifies the type, position, speed, acceleration, and other states of the vehicle M and objects existing around the vehicle M. The object type can be, for example, a vehicle or a pedestrian, or a type used for identification on a per-vehicle basis. The object position is identified, for example, as the position on an absolute coordinate system (hereinafter referred to as the vehicle coordinate system) with a representative point of the vehicle M (center of gravity, drive shaft center, etc.) as the origin, and is used for control. The object position can also be represented by representative points such as the object's center of gravity, corners, or front end in the direction of travel, or by the area it represents. The speed can also include, for example, the speed of the vehicle M and other vehicles relative to the direction of travel (longitudinal) of the lane (hereinafter referred to as longitudinal speed), and the lateral speed of the vehicle M and other vehicles relative to the lane (hereinafter referred to as lateral speed). The "state" of an object, for example, in the case of a moving object such as another vehicle, may also include the object's acceleration, jerk, or "action state" (e.g., whether a lane change is in progress or about to occur). The identification unit 130 includes, for example, a first identification unit 132 and a second identification unit 134. Details of these functions will be described later.

[0050] The action plan generation unit 140 generates an action plan for driving the vehicle M through driving control, such as autonomous driving, based on the recognition results of the recognition unit 130. For example, the action plan generation unit 140 generates a target trajectory for the vehicle M to travel automatically (independent of driver operation) in the future, based on the recognition results obtained from the recognition unit 130 or from map information, including the surrounding road shape and lane markings, in order to cope with the surrounding conditions of the vehicle M. The target trajectory may include speed elements, for example. For example, the target trajectory is represented by a sequence of locations (track points) that the vehicle M should reach. Track points are locations that the vehicle M should reach every predetermined travel distance (e.g., a few meters) along the route, while target speeds (and target accelerations) are generated as part of the target trajectory at predetermined sampling times (e.g., a few tenths of a second). Track points may also be locations that the vehicle M should reach at the sampling time at predetermined sampling times. In this case, the target velocity (and target acceleration) information is represented by the intervals of the orbital points.

[0051] When generating a target track, the action plan generation unit 140 can set events for automatic driving. These events include, for example, a constant-speed driving event that causes vehicle M to travel at a constant speed in the same lane; a following event that causes vehicle M to follow another vehicle (hereinafter referred to as the "vehicle in front") that is within a specified distance (e.g., within 100 m) ahead of vehicle M; a lane-changing event that causes vehicle M to change lanes from its current lane to an adjacent lane; a branching event that causes vehicle M to branch off to the destination lane at a road junction; a merging event that causes vehicle M to merge onto a main road at a merging point; and a takeover event that terminates automatic driving and switches to manual driving. The action plan generation unit 140 generates a target track corresponding to the initiated events.

[0052] The mode determination unit 150 determines the driving mode of the vehicle M as any one of several driving modes that differ in the tasks assigned to the driver (an example of a passenger). The mode determination unit 150 includes, for example, a comparison unit 152, a misidentification determination unit 154, a driver status determination unit 156, and a mode change processing unit 158. The misidentification determination unit 154 is an example of a "determination unit." Details of their functions will be described later.

[0053] Figure 3This diagram illustrates an example of the correspondence between driving modes, the control states of vehicle M, and tasks. Vehicle M has, for example, five driving modes: Mode A through Mode E. Among these five modes, regarding the degree of automation of the driving control of vehicle M, Mode A is the highest, followed by Modes B, C, and D, decreasing sequentially, with Mode E being the lowest. Conversely, regarding the tasks assigned to the driver, Mode A is the lightest, followed by Modes B, C, and D, increasing sequentially, with Mode E being the most demanding. In Modes D and E, the control states are not automated driving; therefore, the automated driving control device 100 performs its function before ending the control involved in automated driving and transitioning to driving assistance or manual driving. The contents of each driving mode are illustrated below.

[0054] In Mode A, the vehicle enters an autonomous driving state, where neither the surrounding monitoring of vehicle M nor the handling of the steering wheel 82 (shown as steering wheel handling in the diagram) is directed to the driver. Surrounding monitoring includes at least monitoring of the area in front of vehicle M. However, even in Mode A, the driver is required to be able to quickly switch to manual driving posture according to requests from the system centered on the autonomous driving control unit 100. Autonomous driving, as used here, means that neither steering nor acceleration / deceleration is controlled by the driver. "In front" refers to the space in the direction of travel of vehicle M, visually confirmed through the windshield. Mode A is, for example, a driving mode that can be executed when vehicle M is traveling at a speed below a prescribed speed (e.g., around 50 km / h) on a dedicated motor vehicle road such as a highway, and when conditions such as the presence of a following vehicle are met; it is sometimes referred to as TJP (Traffic Jam Pilot). If these conditions are no longer met, the mode determination unit 150 changes the driving mode of vehicle M to Mode B.

[0055] In Mode B, a driving support mode is established, assigning the driver the task of monitoring the front of vehicle M (hereinafter referred to as forward monitoring), but not the task of holding the steering wheel 82. In Mode C, a driving support mode is established, assigning the driver the tasks of forward monitoring and holding the steering wheel 82. Mode D is a driving mode that requires some degree of driving operation by the driver regarding at least one of the steering and acceleration / deceleration of vehicle M. For example, in Modes C and D, driving support functions such as ACC and LKAS are performed. ACC is a function that maintains a constant distance between vehicle M and the preceding vehicle while allowing vehicle M to follow the preceding vehicle. LKAS is a function that supports lane keeping of vehicle M by keeping vehicle M near the center of the driving lane. In Mode E, a manual driving mode is established, requiring driving operation by the driver for both steering and acceleration / deceleration, and driving support functions such as ACC and LKAS are not performed. Modes D and E naturally assign the driver the task of monitoring the front of vehicle M. In the implementation, for example, when Mode A is the "first driving mode," Modes B to E are examples of "second driving modes." When Mode B is the "first driving mode", Modes C through E become examples of the "second driving mode". That is, the second driving mode is a task with a higher level of difficulty for the driver compared to the first driving mode.

[0056] If the driver fails to perform the task involved in the determined driving mode, the mode determination unit 150 changes the driving mode of vehicle M to a more demanding driving mode. For example, in mode A, if the driver is unable to switch to manual driving posture as requested by the system (e.g., continuing to look out of the permitted area, or detecting signs of driving difficulty), the mode determination unit 150 uses the HMI 30 to urge the driver to switch to manual driving. If the driver does not respond, the mode determination unit 150 performs control such as pulling vehicle M towards the curb, gradually stopping it, and discontinuing automatic driving. After discontinuing automatic driving, the vehicle enters mode D or E, and can be started manually by the driver. The same applies to "discontinuing automatic driving." In mode B, if the driver is not monitoring the road ahead, the mode determination unit 150 uses the HMI 30 to urge the driver to monitor the road ahead. If the driver does not respond, the mode determination unit 150 performs control such as pulling vehicle M towards the curb, gradually stopping it, and discontinuing automatic driving. In Mode C, if the driver is not monitoring the situation ahead or is not holding the steering wheel 82, the mode decision unit 150 uses HMI 30 to urge the driver to monitor the situation ahead and / or urge the driver to hold the steering wheel 82. If the driver does not respond, the system will control the vehicle M to move toward the curb and gradually stop and then discontinue the automatic driving.

[0057] The second control unit 160 controls the driving force output device 200, the braking device 210, and the steering device 220 to ensure that the vehicle M passes through the target track generated by the action plan generation unit 140 at a predetermined time. The second control unit 160 includes, for example, a target track acquisition unit 162, a speed control unit 164, and a steering control unit 166. The target track acquisition unit 162 acquires information about the target track (track point) generated by the action plan generation unit 140 and stores this information in a memory (not shown). The speed control unit 164 controls the driving force output device 200 or the braking device 210 based on the speed elements associated with the target track stored in the memory. The steering control unit 166 controls the steering device 220 based on the curvature of the target track stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is achieved, for example, through a combination of feedforward control and feedback control. As an example, the steering control unit 166 combines feedforward control corresponding to the radius of curvature (or curvature) of the road ahead of the vehicle M with feedback control based on deviations from the target track.

[0058] HMI control unit 180 notifies the occupant of prescribed information via HMI 30. Prescribed information includes, for example, information related to the state of vehicle M, information related to driving control, and other information relevant to the movement of vehicle M. Information related to the state of vehicle M includes, for example, the vehicle M's speed, engine speed, and gear position. Information related to driving control includes, for example, whether driving control is being performed by automatic driving, a message asking whether to start automatic driving, the status of driving control performed by automatic driving (e.g., the driving mode being executed, the content of the event), and information related to switching driving modes. Prescribed information may also include items stored on storage media (e.g., movies), such as television programs or DVDs, that are unrelated to the driving control of vehicle M. Prescribed information may also include, for example, information related to the vehicle M's current location, destination, and remaining fuel.

[0059] For example, the HMI control unit 180 can generate an image containing the aforementioned specified information and display the generated image on the display device of the HMI 30. It can also generate sound representing the specified information and output the generated sound from the speaker of the HMI 30. The HMI control unit 180 can also output information received by the HMI 30 to the communication device 20, navigation device 50, first control unit 120, etc. The HMI control unit 180 can also send various information output by the HMI 30 to a terminal device used by the occupants of the vehicle M via the communication device 20. The terminal device may be, for example, a smartphone or tablet.

[0060] The driving force output device 200 outputs driving force (torque) for the vehicle M to drive to the drive wheels. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, as well as an ECU (Electronic Control Unit) that controls them. The ECU controls the above structure according to information input from the second control unit 160 or information input from the accelerator pedal of the driving operation unit 80.

[0061] The braking device 210 includes, for example, a brake caliper, a hydraulic cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the hydraulic cylinder, and a braking ECU. The braking ECU controls the electric motor according to information input from the second control unit 160 or from the brake pedal of the driving control unit 80, so that braking torque corresponding to the braking operation is output to each wheel. The braking device 210 may have a backup mechanism for transmitting the hydraulic pressure generated by the operation of the brake pedal to the hydraulic cylinder via the master hydraulic cylinder. The braking device 210 is not limited to the structure described above, and may also be an electronically controlled hydraulic braking device that controls the actuator according to information input from the second control unit 160 and transmits the hydraulic pressure from the master hydraulic cylinder to the hydraulic cylinder.

[0062] The steering system 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies force to a rack and pinion mechanism to change the direction of the steering wheels. The steering ECU drives the electric motor to change the direction of the steering wheels according to information input from the second control unit 160 or from the steering wheel of the driving operation unit 80.

[0063] [Identification Department, Pattern Decision Department]

[0064] The following describes the details of the functions included in the recognition unit 130 and the pattern determination unit 150. Figure 4 This diagram illustrates the processing procedures of the first identification unit 132, the second identification unit 134, the comparison unit 152, and the misidentification determination unit 154. Figure 4 The example shows lane L1, which allows travel in the same direction (the X-axis in the diagram). Lane L1 is divided by dividing lines LL and RL. Lane L1 could be, for example, a highway, a dedicated motor vehicle lane, or other arterial roads with priority for vehicles. Lanes L2 and L3 follow the same pattern. Figure 4 In the example, let's say vehicle M is traveling at speed VM along the extension of lane L1.

[0065] The first recognition unit 132 identifies, for example, the left and right dividing lines LL1 and RL1 that divide the driving lane (lane L1) of the vehicle M based on the output of the detection device DD. The dividing lines L11 and RL1 are examples of "first road dividing lines". For example, the first recognition unit 132 analyzes an image captured by the camera 10, extracts edge points with large brightness differences between adjacent pixels, and connects these edge points to identify the dividing lines LL1 and RL1 in the image plane. The first recognition unit 132 transforms the positions of the dividing lines LL1 and RL1, based on the position information of a representative point (e.g., center of gravity or center) of the vehicle M ((X1, Y1) in the figure), into vehicle coordinates (e.g., XY plane coordinates in the figure). The first recognition unit 132 may also identify, for example, the radius of curvature or curvature of the dividing lines LL1 and RL1. The first recognition unit 132 may also identify the amount of curvature change of the dividing lines LL1 and RL1. The curvature change is, for example, the rate of change of the curvature R of the dividing lines LL1 and RL1 in front of the vehicle M when viewed from the vehicle M. The first recognition unit 132 may also average the curvature radius, curvature or curvature change of the dividing lines LL1 and RL1 to recognize the curvature radius, curvature or curvature change of the road (lane L1).

[0066] The second identification unit 134 identifies the dividing lines LL2 and RL2 that divide the driving lane L1 of vehicle M using a different means than the first identification unit 132. The dividing lines LL2 and RL2 are examples of "second road dividing lines." "Different means" include at least one of the following: different devices for identifying the dividing lines, different methods, and different input information. For example, the second identification unit 134 identifies the dividing lines LL2 and RL2 that divide the driving lane L1 of vehicle M from map information based on the location of vehicle M. The map information mentioned above can be second map information 62, newly downloaded map information from an external device, or information obtained by combining these map information. For example, the second identification unit 134 obtains the location information of vehicle M (X1, Y1 in the figure) from vehicle sensor 40 and navigation device 50, and based on the obtained location information, refers to second map information 62 to identify the dividing lines LL2 and RL2 that divide lane L1 at the location of vehicle M from the second map information 62. The second identification unit 134 identifies the radius of curvature, curvature, or curvature change of each of the road dividing lines LL2 and RL2 from the second map information 62. The second identification unit 134 may also average the radius of curvature, curvature, or curvature change of each of the road dividing lines LL2 and RL2 to identify the radius of curvature, curvature, or curvature change of the road (lane L1). Hereinafter, dividing lines LL1 and RL1 represent the dividing lines identified by the first identification unit 132, and dividing lines LL2 and RL2 represent the dividing lines identified by the second identification unit 134.

[0067] The comparison unit 152 compares the recognition result (first road dividing line) identified by the first recognition unit 132 with the recognition result (second road dividing line) identified by the second recognition unit 134. For example, the comparison unit 152 compares the positions of dividing line LL1 and dividing line LL2 based on the position (X1, Y1) of vehicle M. The comparison unit 152 similarly compares the positions of dividing line RL1 and dividing line RL2. The comparison unit 152 can also compare the amount of curvature change and the extension direction of the dividing lines in dividing lines LL1 and LL2, and dividing lines RL1 and RL2.

[0068] When a difference arises between the identification result (first road dividing line) identified by the first identification unit 132 and the identification result (second road dividing line) identified by the second identification unit 134 in the comparison results compared by the comparison unit 152, the misidentification determination unit 154 performs a determination of any one of multiple misidentification determinations. These multiple misidentification determinations include determining that the first identification unit 132 has misidentified the road, and determining that one or both of the first identification unit 132 and the second identification unit 134 have misidentified the road. A difference is, for example, when the magnitude of the difference is greater than or equal to a predetermined value (threshold). The magnitude of the difference refers, for example, to the degree of deviation described later. Multiple misidentification determinations may also include determining that the second identification unit 134 has misidentified the road. The term "determined as misidentified" can also be replaced with "determined whether a misidentification has occurred." "Determined that the first identification unit 132 has misidentified the road" can also be replaced with "determined that the first road dividing line is a dividing line misidentified by the first identification unit 132." In addition, "determining that one or both of the first identification unit 132 and the second identification unit 134 are misidentified" can also be referred to as "determining that one or both of the first road dividing line and the second road dividing line are misidentified dividing lines".

[0069] For example, the comparison unit 152 overlaps dividing lines LL1 and LL2 on the vehicle coordinate system plane (XY plane) with the position (X1, Y1) of the representative point of vehicle M as a reference. Similarly, the comparison unit 152 also overlaps dividing lines RL1 and RL2 with the position (X1, Y1) of the representative point of vehicle M as a reference. The misidentification determination unit 154 determines whether the position of the overlapping dividing line LL1 matches the position of the dividing line LL2. In addition, the misidentification determination unit 154 also determines whether the positions of dividing lines RL1 and RL2 match in the same way.

[0070] For example, when the misidentification determination unit 154 uses dividing lines LL1 and LL2 for determination, it determines a match if the deviation of each dividing line is less than a threshold, and determines a mismatch (a discrepancy has occurred) if the deviation is greater than the threshold. The deviation can be, for example, a deviation in the lateral position (Y-axis direction in the figure) (e.g., the offset W1 between dividing lines LL1 and LL2 in the figure), a difference in the longitudinal position (distance in the X-axis direction), or a combination thereof. The deviation can also be the difference in the amount of curvature change between dividing lines LL1 and LL2, or the angle formed by dividing lines LL1 and LL2 (hereinafter referred to as the peel angle).

[0071] For example, if the comparison lines are determined to match, the misidentification determination unit 154 determines that the first identification unit 132 and the second identification unit 134 have not misidentified (in other words, the first road dividing line and the second road dividing line can be correctly identified). If the comparison lines are determined to not match (a difference has occurred), the misidentification determination unit 154 determines that one or both of the first identification unit 132 and the second identification unit 134 have misidentified. If the comparison lines are determined to not match, the misidentification determination unit 154 derives the degree of deviation of the curvature change and the peeling angle, and uses the derived values ​​to perform a more detailed misidentification determination.

[0072] Figure 5 This is a graph used to illustrate the degree of deviation of the curvature change. In Figure 5 In the example, let's assume that vehicle M is traveling at speed VM on lane L2, which is a curve. For example, the first recognition unit 132 derives the curvature change of the dividing line LL1 based on the analysis results of the image captured by camera 10. For example, let's assume that the position at X[m] ahead of vehicle M on the dividing line LL1 obtained from the image captured by camera 10 is represented by the polynomial (Z(X)) in equation (1) shown below.

[0073] Z(X) = C3X 3 +C2X 2 +C1X+C0···(1)

[0074] C0~C3 represent the specified coefficients. When the curvature change of the dividing line LL1 is obtained, the first identification unit 132 first differentiates the polynomial of equation (1) twice with X, and derives the curvature R [rad / m] shown in equation (2).

[0075]

[0076] Next, the first identification unit 132 differentiates equation (2) at time t and derives the time change of curvature R at the front X [m] as shown in equation (3) [rad / m / sec].

[0077]

[0078] Moreover, the first identification unit 132 in such Figure 5 If the position of the representative point (e.g., the center of gravity) of the vehicle M is predetermined as (X1, Y1), the curvature change rate of the dividing line LL1 is derived by substituting X1 into X in (1) to (3) above. The first recognition unit 132 derives the curvature change rate of the dividing line RL1 in the same way.

[0079] The second identification unit 134 identifies the curvature change rates of the dividing lines LL2 and RL2 based on the location information of the vehicle M and with reference to the map information (second map information 62).

[0080] The misidentification determination unit 154 compares the degree of deviation of the curvature change rates of the dividing lines LL1 and LL2. In this case, the misidentification determination unit 154 obtains the degree to which the dividing line LL1 deviates from the dividing line LL2 as a reference. For example, the misidentification determination unit 154 derives the absolute value of the value obtained by subtracting the curvature change rate of the dividing line LL1 from the curvature change rate of the dividing line LL2 based on the position (X1, Y1) of the vehicle M as the degree of deviation of the curvature change rate. The misidentification determination unit 154 uses the curvature change rates of the dividing lines RL1 and RL2 to derive the aforementioned degree of deviation of the curvature change rate. The derivation of the aforementioned degree of deviation can also be performed by the comparison unit 152.

[0081] Furthermore, if the deviation of one or both of the deviations in the rate of change of curvature between dividing lines LL1 and LL2 and between dividing lines RL1 and RL2 exceeds a predetermined value, the misidentification determination unit 154 determines that the first identification unit 132 has misidentified. The misidentification determination unit 154 may also calculate the average value of the deviations between dividing lines LL1 and LL2 and between dividing lines RL1 and RL2, and determine that the first identification unit 132 has misidentified if the calculated average value exceeds a predetermined value.

[0082] The misidentification determination unit 154 can also determine whether the first identification unit 132 has misidentified based on the peeling angle between the dividing lines LL1 and LL2. Figure 6 This is a diagram used to illustrate the peeling angle. In Figure 6In the example, suppose vehicle M is traveling at speed VM in lane L3. The misidentification determination unit 154, when vehicle M is at a predetermined position (X1, Y1), derives the angle between dividing lines LL1 and LL2 as the peeling angle θL. The misidentification determination unit 154 also derives the angle between dividing lines RL1 and RL2 as the peeling angle θR. The peeling angle θL is the offset of dividing line LL1 from dividing line LL2, and the peeling angle θR is the offset of dividing line RL1 from dividing line RL2. The derivation of the peeling angles described above can also be performed by the comparison unit 152.

[0083] Furthermore, if one or both of the peeling angles θL and θR are above a predetermined angle, the misidentification determination unit 154 determines that the first identification unit 132 has misidentified the image. The misidentification determination unit 154 may also use only one of the peeling angles θL and θR to determine that the first identification unit 132 has misidentified the image, or it may use the average angle of the peeling angles θR and θL to determine that the dividing line has misidentified the image.

[0084] For example, the dividing lines that are misidentified based on images captured by camera 10 often differ significantly from the actual dividing lines due to factors such as road shape and surrounding vehicles. Therefore, when the deviation in the rate of curvature change is large or the stripping angle is large, it is determined that the first recognition unit 132 has misidentified the line, thereby enabling a more appropriate misidentification determination.

[0085] The misidentification determination unit 154 can also use both curvature change and peeling angle to determine misidentification. Figure 7 This is a diagram used to illustrate the misidentification criteria when using curvature variation and peeling angle. Figure 7 The vertical axis represents the curvature change of the first road dividing line identified by the first identification unit 132, and the horizontal axis represents the peeling angle of the first road dividing line. Figure 7 In the example, three regions AR1 to AR3 are defined in the relationship between the curvature change and the peeling angle. Region AR1 is an example of the "first region", region AR2 is an example of the "second region", and region AR3 is an example of the "third region".

[0086] For example, region AR1 is a region where the stripping angle is less than a specified angle θa, and it is determined that neither the first recognition unit 132 nor the second recognition unit 134 has misidentified the dividing line. Region AR2 is determined, based on a first determination condition (first misidentification determination condition), to be a camera misidentification region where only the first recognition unit 132 has misidentified the image. The first determination condition is, for example... Figure 7As shown, this is a region where the peeling angle is θa or higher and the curvature change is Aa or higher. Furthermore, the first determination condition may also include: the case where the region from the peeling angle of θb to θc is above the boundary where the curvature change is set to decrease with increasing angle (the curvature change rate is a value above the boundary line); or the case where the peeling angle is θc or higher regardless of the curvature change. Region AR3 is determined based on the second determination condition (second misidentification determination condition) to be a region where it is impossible to determine which of the first identification unit 132 and the second identification unit 134 misidentified, but it is determined to be a misidentification by one or both of the first identification unit 132 and the second identification unit 134. The second determination condition includes, for example... Figure 7 As shown, the peeling angle is within the range of θa to θb, and the curvature change is less than Aa. Furthermore, the second criterion may also include cases where, in the interval from the peeling angle above θb to θc, the curvature change is lower than the boundary where the curvature change is set to decrease with increasing angle (the rate of curvature change is less than the value of the boundary line). The first and second criterion are examples of "criteria".

[0087] For example, if the values ​​of curvature change and peeling angle are both within region AR1, the misidentification determination unit 154 determines that the first identification unit 132 and the second identification unit 134 have not misidentified (correctly identified). If the values ​​of curvature change and peeling angle are within region AR2, the misidentification determination unit 154 determines that the first identification unit 132 has misidentified. If the values ​​of curvature change and peeling angle are within region AR3, the misidentification determination unit 154 determines that one or both of the first identification unit 132 and the second identification unit 134 have misidentified. In this way, misidentification can be determined in more detail based on the values ​​of curvature change rate and peeling angle.

[0088] The driver state determination unit 156 monitors the driver's state in response to the changes in the aforementioned modes and determines whether the driver's state is appropriate for the task. For example, the driver state determination unit 156 analyzes the images captured by the driver monitoring camera 70 to perform posture estimation processing and determines whether the driver's body posture cannot be shifted to manual driving according to the system's requirements. The driver state determination unit 156 analyzes the images captured by the driver monitoring camera 70 to perform gaze estimation processing and determines whether the driver is monitoring the surroundings (front, etc.).

[0089] The mode change processing unit 158 ​​performs various processing for mode changes, for example, based on the determination results made by the misidentification determination unit 154 and the determination results made by the driver status determination unit 156. For example, based on the driver's status (surrounding monitoring status) determined by the driver status determination unit 156, if the status is not suitable for the current mode, the mode change processing unit 158 ​​instructs the action plan generation unit 140 to generate a target track for shoulder stopping, or gives working instructions to the driving support device (not shown), or controls the HMI 30 to urge the driver to act.

[0090] The mode change processing unit 158 ​​changes the mode based on the determination result made by the misidentification determination unit 154. For example, if the misidentification determination unit 154 determines that neither the first identification unit 132 nor the second identification unit 134 has made a misidentification, the mode change processing unit 158 ​​performs autonomous driving or driving support according to the determination result made by the current driver state determination unit 156, the surrounding conditions, etc.

[0091] In the execution of the first driving mode (e.g., mode A), the mode change processing unit 158 ​​determines by the misidentification determination unit 154 that the first identification unit 132 has misidentified the vehicle (the values ​​of curvature change and peel angle exist). Figure 7 In the case of the area AR2 shown, mode A continues using the dividing line identified from the map information. In this way, even if the dividing line identified by the image captured by the camera 10 does not match the dividing line identified from the map information, when it is determined that only the first identification unit 132 has misidentified, driving control continues based on the dividing line identified from the map information, thereby suppressing excessive switching from the first driving mode to the second driving mode.

[0092] The mode change processing unit 158 ​​can also terminate the continuation of the first driving mode if the misidentification determination unit 154 determines that the first identification unit 132 has continued in a state of misidentification for a predetermined time. As a result, safer driving control is possible.

[0093] The mode change processing unit 158, during the execution of the first driving mode, determines by the misidentification determination unit 154 that one or both of the first identification unit 132 and the second identification unit 134 have misidentified the road markings (the curvature change and the stripping angle exist). Figure 7In the case of the location of area AR3 shown, the mode is changed from the first driving mode to the second driving mode (e.g., mode B). The mode change processing unit 158 ​​can also change to any one of modes C to E based on the surrounding conditions of the vehicle M and the determination result of the driver state determination unit 156, instead of changing from mode A to mode B. When changing from mode A to mode E, the mode change processing unit 158 ​​can switch between modes B, C, and E in stages, or it can switch directly from mode A to mode E.

[0094] Based on the control content controlled by the first control unit 120 and the second control unit 160, the HMI control unit 180 outputs information related to the state of the vehicle M or a prescribed warning to the HMI 30 to report to the occupants of the vehicle M. For example, based on the determination result made by the misidentification determination unit 154, the HMI control unit 180 outputs the driving mode and other driving status of the vehicle M, or a warning indicating that a misidentification has occurred. The HMI control unit 180 may also, if the misidentification determination unit 154 determines that the state of the first identification unit 132 has remained unchanged and the first driving mode continues, display information on the HMI 30's display device or output information from the HMI 30 by sound or other means indicating that the first driving mode has ended (or that the system will switch to the second driving mode after a predetermined time) after the current state has continued for a predetermined time (advance notification). Thus, the occupants can be notified in advance of the possibility of switching from the first driving mode to the second driving mode, allowing them to prepare for the task ahead of time. If a reporting device for issuing warnings or other reports is provided within the vehicle system 1, the HMI control unit 180 may also control the operation of the reporting device instead of the HMI 30 outputting (or otherwise). In this case, the reporting device becomes an example of an "output device".

[0095] <Variation Example>

[0096] For example, the misidentification determination unit 154 can also modify the above-mentioned [measures] based on the surrounding conditions of the vehicle M. Figure 7 At least one of the regions (reference regions) AR1 to AR3 shown (one or both of the first and second determination conditions). Figure 8This diagram illustrates how regions AR1 to AR3 are changed based on the surrounding conditions of vehicle M. For example, if the shape of the road on which vehicle M is traveling includes branches or merges in the direction of travel (forward) of vehicle M, the first identification unit 132 is highly likely to misidentify the first road dividing line. Therefore, the misidentification determination unit 154, for example, changes the first determination condition and the second determination condition in a way that makes it easy to determine that the first identification unit 132 has misidentified the road if branches or merges in the direction of travel of vehicle M. Specifically, the misidentification determination unit 154, based on the location information of vehicle M and referring to map information, determines the first determination condition if, on the road on which vehicle M is traveling, in the direction of travel of vehicle M, and within a predetermined distance from the current position of vehicle M, there are predetermined road shapes such as branches or merges. Figure 8 As shown, compared with the reference regions AR1 to AR3, region AR2 is increased, and region AR3 is changed to the reduced region (AR2#, AR3#). The misidentification determination unit 154 sets regions AR2# and AR3#, for example, by changing the parameter Aa of the curvature change included in the first determination condition and the second determination condition to Ab, which is smaller than Aa.

[0097] Therefore, near branches and convergences, use Figure 8 The regions AR1, AR2#, and AR3# shown are used for misidentification, making it easy to determine that the first identification unit 132 has misidentified the vehicle. In the case where the first identification unit 132 has misidentified the vehicle, the current driving control continues based on the dividing lines identified from the map information, thus enabling more appropriate driving control to be executed.

[0098] If the navigation device 50 has a pre-set path to the destination, and the path to the destination is a side lane instead of a main road, then the task assigned to the driver needs to be as demanding as manual driving. Therefore, it is also possible that the misidentification determination unit 154 does not change the aforementioned areas AR2 and AR3 even if there is a branch or the like in the direction of travel of the vehicle M, when the destination is a side lane.

[0099] For example, if there is a tunnel entrance or exit in the direction of travel of vehicle M, the misidentification determination unit 154 may also change the region AR2, which is the reference area described above, and decrease the region AR3, since the first identification unit 132 is more likely to misidentify the dividing line due to changes in brightness. The misidentification determination unit 154 may also change the regions AR2 and AR3 as described above, since the dividing line is more likely to be misidentified by the first identification unit 132 due to the obstruction of the dividing line by the preceding vehicle, when the identification unit 130 identifies vehicle M as a vehicle changing lanes or driving in a winding manner.

[0100] The misidentification determination unit 154 can also adjust the increase in region AR2 and the decrease in region AR1 based on the surrounding conditions of vehicle M. For example, compared to a merging situation, the misidentification determination unit 154 increases the increase in region AR2 (or decreases the decrease in region AR3) in a branching situation, and compared to a tunnel exit situation, it increases the increase in region AR2 (or decreases the decrease in region AR3) in a tunnel entrance situation. By adjusting each region according to the surrounding conditions in this way, more appropriate misidentification determinations can be made.

[0101] When the surrounding conditions (driving lane) of vehicle M are near the entrance or exit of a curve, the curvature change of the dividing line identified by the image captured by camera 10 increases. However, due to factors such as the offset between the dividing line identified from the map information based on the position information of vehicle M and the dividing line ahead, the angle (separation angle) between the first and second road dividing lines may increase for a corresponding period of time (short time). Therefore, the misidentification determination unit 154 may also change the first and second determination conditions to suppress the misidentification by the first identification unit 132 when there is an entrance or exit of a curve in the direction of travel of vehicle M. Specifically, the misidentification determination unit 154 changes the size of regions AR1 to AR3 to suppress the misidentification by the first identification unit 132 when there is an entrance or exit of a curve in the direction of travel of vehicle M and within a predetermined distance from the current position of vehicle M, based on the position information of vehicle M and referring to the map information.

[0102] Figure 9 This diagram illustrates the modification of regions AR1 to AR3 to suppress situations where the first identification unit 132 determines a misidentification. Figure 9 In the example, the misidentification determination unit 154 sets the region AR1, which was not determined to have a misidentification, to the enlarged region AR1##, and sets the regions AR2 and AR3, which were determined to have a misidentification, to the reduced regions AR2## and AR3##. For example, the misidentification determination unit 154 sets regions AR1## to AR3## by changing the parameter θa of the peeling angle included in the first and second determination conditions to θa##, which is larger than θa (here, θa## < θb). Thus, if there is an entrance or exit to a curve in the direction of travel of vehicle M on the road where vehicle M is traveling, the region AR1## to AR3## is changed. Figure 9 By determining the area as shown, misidentification can be made, thereby suppressing the situation where one or both of the first identification unit 132 and the second identification unit 134 are determined to be misidentified.

[0103] Furthermore, the misidentification determination unit 154 can also be like Figure 9The parameter Aa, representing the curvature change, is changed to Ac, which is larger than Aa, thereby increasing the change in region AR3. This further suppresses the possibility of misidentification by the first recognition unit 132.

[0104] The misidentification determination unit 154 can also change the size of the reference area AR1 to AR3 based on the weather around the vehicle M (e.g., heavy rain, blizzard) and the time period of travel (e.g., the time period when the dividing line contained in the camera image is easily misidentified due to the influence of shadows formed on the road surface, sunlight, etc.).

[0105] [Processing Flow]

[0106] Next, the process flow executed by the automatic driving control device 100 of the embodiment will be described. Figure 10 This is a flowchart illustrating an example of the processing flow executed by the automatic driving control unit 100. Hereinafter, the explanation will focus on the process executed by the automatic driving control unit 100, which involves switching the driving control of the vehicle M based on the recognition results of the dividing lines recognized by the first recognition unit 132 and the second recognition unit 134. Figure 10 At the beginning of the flowchart, it is assumed that vehicle M is performing driving control through a first driving mode (e.g., mode A). In the following processing, in the determination result determined by the driver state determination unit 156, it is assumed that the driver's state is a state appropriate to the executed mode or the switched mode (i.e., a state in which no mode switching has occurred based on the determination result of the driver state determination unit 156). Figure 10 The process shown can also be repeated at specified times.

[0107] exist Figure 10 In the example, the first identification unit 132 identifies the lane dividing line that divides the vehicle M's lane based on the output of the detection device DD (step S100). Next, the second identification unit 134 identifies the lane dividing line that divides the vehicle M's lane based on the vehicle M's position information obtained from the vehicle sensor 40 and the GNSS receiver 51, and with reference to map information (step S102). Steps S100 and S102 can also be performed in reverse order or in parallel. Next, the comparison unit 152 compares the dividing line identified by the first identification unit 132 with the dividing line identified by the second identification unit 134 (step S104). Next, the misidentification determination unit 154 determines whether the dividing line identified by the first identification unit 132 and the second identification unit 134 is a misidentification based on the comparison result obtained by the comparison unit 152 (step S106). Details of the process in step S106 will be described later.

[0108] The misidentification determination unit 154 determines whether one or both of the first identification unit 132 and the second identification unit 134 have misidentified the road markings (step S108). If a misidentification is determined, the misidentification determination unit 154 determines whether only the first identification unit 132 has misidentified the road markings (step S110). If only the first identification unit 132 has misidentified the road markings, the mode change processing unit 158 ​​continues the current driving mode (step S112). If, during the processing in step S108, it is determined that neither the first identification unit 132 nor the second identification unit 134 has misidentified the road markings, the processing in step S112 is also performed.

[0109] If, in step S108, it is not determined that only the first identification unit 132 misidentified the vehicle, the mode change processing unit 158 ​​performs control to change the driving mode of the vehicle M from the first driving mode to the second driving mode (step S114). "Not determined that only the first identification unit 132 misidentified the vehicle" means, for example, that it is impossible to determine which of the first identification unit 132 and the second identification unit 134 misidentified the vehicle, but it is determined that one or both of the first identification unit 132 and the second identification unit 134 misidentified the vehicle. Therefore, the processing in this flowchart ends.

[0110] Figure 11 This is a flowchart illustrating an example of the processing flow for step S106. Figure 11 In the example, the misidentification determination unit 154 obtains the rate of change of curvature of the dividing line identified by the first identification unit 132 (step S106A). Next, the misidentification determination unit 154 obtains the angle (stripping angle) between the first road dividing line identified by the first identification unit 132 and the second road dividing line identified by the second identification unit 134 (step S106B).

[0111] Next, the misidentification determination unit 154 acquires the surrounding conditions of the vehicle M identified by the identification unit 130 (step S106C), and sets the first to third regions (regions AR1 to AR3) based on the acquired surrounding conditions (step S106D). Next, the misidentification determination unit 154 determines which of the first to third regions it belongs to based on the rate of curvature change and the peeling angle (step S106E). Next, based on the determined region, the misidentification determination unit 154 determines whether the first identification unit 132 misidentified the vehicle, or whether it is impossible to determine which of the first identification unit 132 and the second identification unit 134 misidentified the vehicle, but one or both of the first identification unit 132 and the second identification unit 134 misidentified the vehicle (step S106F). The processing of this flowchart ends here.

[0112] According to the embodiments described above, the vehicle control device includes: a first identification unit 132 that identifies road markings dividing the driving lanes of vehicle M based on the output of a detection device DD that detects the surrounding conditions of vehicle M; a second identification unit 134 that identifies road markings dividing the driving lanes using a different means than the first identification unit; a comparison unit that compares the first road marking identified by the first identification unit 132 with the second road marking identified by the second identification unit 134; and a misidentification determination unit 154 (an example of a determination unit) that, when a difference is found between the first road marking and the second road marking in the comparison results obtained by the comparison unit, determines any one of a plurality of misidentification determinations, including determining that the first identification unit has misidentified the vehicle, and determining that one or both of the first and second identification units have misidentified the vehicle. This allows for more appropriate driving control based on the identification status of the road markings.

[0113] Specifically, according to the implementation method, when the first road marking and the second road marking do not match, a misidentification determination is made based on a first determination condition that the first road marking is incorrect, and a second determination condition that one or both of the first and second road markings are incorrect but it cannot be determined which one is incorrect. Therefore, even if a misidentification is determined, when it is clear that the first road marking is incorrect, the driving mode with a high degree of automation can continue using map information. Furthermore, according to the implementation method, even if a misidentification is determined, the driving mode with a high degree of automation (lighter tasks assigned to the occupants) can continue based on map information, thus suppressing the reduction in the level of useless driving modes.

[0114] The implementation methods described above can be performed as follows.

[0115] A vehicle control device comprising:

[0116] Storage device, which stores a program; and

[0117] Hardware processor,

[0118] The hardware processor executes the program stored in the storage device to perform the following processing:

[0119] Based on the output of the detection device that detects the surrounding conditions of the vehicle, the first road dividing line that divides the vehicle's driving lane is identified.

[0120] The second road dividing the driving lanes is identified using a different means than the means by which the first road dividing line was identified.

[0121] Compare the identified first road dividing line with the second road dividing line;

[0122] If the comparison results show a difference between the first road dividing line and the second road dividing line, a determination is made on any one of a plurality of misidentification determinations. The plurality of misidentification determinations include determining that the first road dividing line is a misidentified dividing line, and determining that one or both of the first road dividing line and the second road dividing line are misidentified dividing lines.

[0123] The above description illustrates specific embodiments of the present invention, but the present invention is not limited to such embodiments in any way, and various modifications and substitutions can be made without departing from the spirit of the present invention.

Claims

1. A vehicle control device, wherein, The vehicle control device includes: The first identification unit identifies the road markings that divide the vehicle's driving lane based on the output of a detection device that detects the surrounding conditions of the vehicle. The second identification unit identifies the road markings that divide the driving lanes using a different means than the first identification unit. The comparison unit compares the first road dividing line identified by the first identification unit with the second road dividing line identified by the second identification unit; as well as The determination unit, when a difference is found between the first road dividing line and the second road dividing line in the comparison result obtained by the comparison unit, performs a determination of any one of a plurality of misidentification determinations. The plurality of misidentification determinations include determining that the first identification unit has misidentified the road, and determining that one or both of the first identification unit and the second identification unit have misidentified the road. The determination unit sets a first region, a second region, and a third region based on the surrounding conditions of the vehicle. The first region is the region where neither the first identification unit nor the second identification unit has made a misidentification. The second region is the region where the first identification unit has made a misidentification. The third region is the region where it is impossible to determine which of the first identification unit and the second identification unit made a misidentification, but it is determined that one or both of the first identification unit and the second identification unit made a misidentification. The determination unit determines which of the three regions (first region, second region, and third region) the misidentification belongs to based on the curvature change of the first road dividing line and the angle between the first road dividing line and the second road dividing line.

2. The vehicle control device according to claim 1, wherein, The vehicle control device further includes a driving control unit that controls at least one of the vehicle's acceleration, deceleration, and steering. The driving control unit executes any one of multiple driving modes with different tasks assigned to the occupants of the vehicle, based on the determination result determined by the determination unit.

3. The vehicle control device according to claim 1, wherein, The vehicle control device further includes an output control unit, which, based on the determination result determined by the determination unit, causes the output device to output information or warnings related to the state of the vehicle to report to the occupants of the vehicle.

4. The vehicle control device according to claim 2, wherein, The multiple driving modes include a first driving mode and a second driving mode that assigns a greater workload to the occupants compared to the first driving mode. If the driving control unit is executing the first driving mode and the determination unit determines that the first identification unit has misidentified the first driving mode, the driving control unit continues the first driving mode based on the second road dividing line.

5. The vehicle control device according to claim 4, wherein, When the driving control unit is executing the first driving mode and the determination unit determines that one or both of the first identification unit and the second identification unit have misidentified the vehicle, the driving control unit changes the driving mode of the vehicle from the first driving mode to the second driving mode.

6. The vehicle control device according to claim 1, wherein, The determination unit changes the determination conditions based on the curvature change of the first road dividing line and the angle between the first road dividing line and the second road dividing line, according to the surrounding conditions of the vehicle.

7. The vehicle control device according to claim 6, wherein, When the determination unit finds a branch, a merging, a tunnel entrance or exit in the direction of the vehicle's travel, or when the vehicle in front is changing lanes or driving in a winding manner, it changes the determination conditions in a way that makes it easy to determine that the first identification unit has misidentified the vehicle.

8. The vehicle control device according to claim 6, wherein, If the determination unit finds an entrance or exit to a curve in the direction of travel of the vehicle, it changes the determination condition in a way that suppresses the determination that the first identification unit has misidentified the vehicle.

9. A vehicle control method, wherein, The vehicle control method causes the computer of the vehicle control device to perform the following processing: Based on the output of the detection device that detects the surrounding conditions of the vehicle, the first road dividing line that divides the vehicle's driving lane is identified. The second road dividing the driving lanes is identified using a different means than the means by which the first road dividing line was identified. Compare the identified first road dividing line with the second road dividing line; If the first road dividing line and the second road dividing line differ in the comparison results, a determination is made on any one of a plurality of misidentification determinations. The plurality of misidentification determinations include determining that the first road dividing line is a misidentified dividing line, and determining that one or both of the first road dividing line and the second road dividing line are misidentified dividing lines. Based on the surrounding conditions of the vehicle, a first area, a second area, and a third area are defined. The first area is the area where both the first road dividing line and the second road dividing line are determined not to be misidentified. The second area is the area where the first road dividing line is determined to be misidentified. The third area is the area where it is impossible to determine which of the first and second road dividing lines is misidentified, but it is determined that one or both of the first and second road dividing lines are misidentified. Based on the curvature change of the first road dividing line and the angle between the first road dividing line and the second road dividing line, it is determined which of the three regions (first region, second region, and third region) the misidentification judgment belongs to.

10. A storage medium storing a program, wherein, The program causes the computer of the vehicle control unit to perform the following processing: Based on the output of the detection device that detects the surrounding conditions of the vehicle, the first road dividing line that divides the vehicle's driving lane is identified. The second road dividing the driving lanes is identified using a different means than the means by which the first road dividing line was identified. Compare the identified first road dividing line with the second road dividing line; If the first road dividing line and the second road dividing line differ in the comparison results, a determination is made on any one of a plurality of misidentification determinations. The plurality of misidentification determinations include determining that the first road dividing line is a misidentified dividing line, and determining that one or both of the first road dividing line and the second road dividing line are misidentified dividing lines. Based on the surrounding conditions of the vehicle, a first area, a second area, and a third area are defined. The first area is the area where both the first road dividing line and the second road dividing line are determined not to be misidentified. The second area is the area where the first road dividing line is determined to be misidentified. The third area is the area where it is impossible to determine which of the first and second road dividing lines is misidentified, but it is determined that one or both of the first and second road dividing lines are misidentified. Based on the curvature change of the first road dividing line and the angle between the first road dividing line and the second road dividing line, it is determined which of the three regions (first region, second region, and third region) the misidentification judgment belongs to.

Images