Vehicle control device, vehicle control method, and program

Abstract

To implement more appropriate driving control. [Solution] The vehicle control device of the embodiment includes: a first recognition unit that recognizes the left and right device lane markings as seen from the vehicle based on the output of a detection device; a second recognition unit that recognizes map lane markings that demarcate the vehicle's driving lane from map information; a comparison unit that, at a first distance in the direction of travel as seen from the vehicle, compares the device lane markings that demarcate the left side of the driving lane with the map lane markings, and compares the device lane markings that demarcate the right side of the driving lane with the map lane markings, and if the results of the comparison of the left lane markings and the comparison of the right lane markings satisfy a specific condition, performs a comparison of the left lane markings and the right lane markings at a second distance shorter than the first distance; and a driving control unit that controls the vehicle based on the comparison results by the comparison unit, wherein the specific condition includes that, at the first distance, the degree of deviation of either the left lane marking comparison or the right lane marking comparison is less than a threshold.

Description

Technical Field

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a program.

Background Art

[0002] In recent years, efforts to provide access to sustainable transportation systems that also consider vulnerable people among traffic participants have become active. To achieve this goal, efforts are focused on research and development that further improves traffic safety and convenience through research and development related to autonomous driving technology. In relation to this, conventionally, when it is determined that there is a discrepancy between the road marking shown in a camera image and the road marking shown in map information, and it is determined that the vehicle is at a branching point indicated in the map information, a technology is known that determines the driving mode of the vehicle based on the branching direction of the branching point and the direction of the road marking shown in the camera image (see, for example, Patent Document 1).

Prior Art Literature

Patent Literature

[0003]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0004] By the way, in conventional autonomous driving technology, it cannot be said that sufficient studies have been conducted on methods for comparing road markings recognized by detection devices such as cameras with road markings indicated in map information, and there has still been room for consideration. Therefore, there has been a problem that there is a possibility that appropriate driving control is not performed based on the recognition result of road markings.

[0005] One of the objectives of this application is to provide a vehicle control device, a vehicle control method, and a program that can perform more appropriate driving control based on the recognition results of road lane markings, in order to solve the above-mentioned problems. Ultimately, this will contribute to the development of a sustainable transportation system. [Means for solving the problem]

[0006] The vehicle control device, vehicle control method, and program according to this invention employ the following configuration. (1) A vehicle control device according to one aspect of the present invention includes: a first recognition unit that recognizes the left and right lane markings as seen from the vehicle that demarcate the vehicle's driving lane as device lane markings based on the output of a detection device that detects the surrounding conditions of the vehicle; a second recognition unit that recognizes the lane markings that demarcate the vehicle's driving lane as map lane markings based on the vehicle's position information from map information; a left lane marking comparison that compares the device lane markings that demarcate the left side of the driving lane with the map lane markings at a first distance in the direction of travel as seen from the vehicle; and a right lane marking... A vehicle control device comprising: a comparison unit that compares the device lane lines with the map lane lines and compares the right lane lines, and if the results of the comparison of the left lane lines and the right lane lines satisfy a specific condition, performs a comparison of the left lane lines and the right lane lines at a second distance shorter than the first distance; and a driving control unit that controls the vehicle based on the comparison results by the comparison unit, wherein the specific condition includes that at the first distance, the degree of deviation of either the comparison of the left lane lines or the comparison of the right lane lines is less than a threshold.

[0007] (2) In the embodiment of (1) above, the control of the vehicle includes driving control by a first driving mode and a second driving mode in which the degree of driving assistance is lower or the task of the vehicle occupant is greater than that of the first driving mode, and the driving control unit continues the first driving mode when the first driving mode is being executed and the degree of deviation of either the left lane marking and the right lane marking at the first distance is less than a threshold.

[0008] (3) In the embodiment of (2) above, the vehicle further comprises a correction unit that corrects the map lane lines based on the device lane lines, wherein the correction unit corrects the map lane lines based on the device lane lines at the second distance if the degree of deviation of both the left lane lines and the right lane lines at the second distance is less than a threshold, and the driving control unit controls the vehicle based on the map lane lines corrected by the correction unit.

[0009] (4) In the embodiment of (3) above, if the degree of deviation of at least one of the left lane markings and the right lane markings in the second distance is greater than or equal to a threshold, the correction unit corrects the map markings based on the device markings in the first distance for the lane markings on the side of the left and right lane markings where the degree of deviation is less than the threshold in the first distance, and the driving control unit controls the vehicle based on the map markings corrected by the correction unit.

[0010] (5) In the embodiment of (2) above, the driving control unit changes the control of the vehicle from the first driving mode to the second driving mode if the first driving mode is being executed and the degree of deviation of at least one of the comparisons of the left lane markings and the right lane markings at the second distance is greater than or equal to a threshold.

[0011] (6) In the embodiment of (2) above, the system further includes a correction unit that corrects the map grid lines based on the device grid lines, wherein if the degree of deviation of only one of the left grid lines and the right grid lines in the comparison at the second distance is less than a threshold, the correction unit corrects the map grid lines based on the device grid lines at the second distance for the grid line on the side of the left and right grid lines whose degree of deviation at the first distance was less than a threshold, and the driving control unit controls the vehicle based on the map grid lines corrected by the correction unit.

[0012] (7): In the embodiment of (6) above, the correction unit corrects the map division lines based on the device division lines at the second distance when the division line whose degree of deviation at the second distance was less than the threshold and the division line whose degree of deviation at the first distance was less than the threshold are on the same side of the left and right division lines.

[0013] (8) In the embodiment of (2) above, the driving control unit performs the control when a specific shape exists in the driving lane in the direction of travel of the vehicle, and the specific shape includes at least one of a branch road, a merging road, an increase or decrease in the number of lanes, an increase or decrease in the lane width, and an intersection.

[0014] (9): In the embodiment of (8) above, the operation control unit performs the comparison at the second distance if the degree of deviation is greater than or equal to a threshold value based on the comparison result by the comparison unit at the first distance for the section line on the opposite side of the left and right section lines from the side on which the specific shape exists.

[0015] (10): In the embodiment of (1) above, the control of the vehicle includes at least one of the following: steering control of the vehicle, speed control, and display control relating to the driving lane or the lane markings that demarcate the driving lane.

[0016] (11): A vehicle control method according to another aspect of the present invention is a vehicle control method in which a computer recognizes the left and right lane markings that demarcate the vehicle's driving lane as seen from the vehicle, based on the output of a detection device that detects the surrounding conditions of the vehicle, as device lane markings, recognizes the lane markings that demarcate the vehicle's driving lane from map information based on the vehicle's position information, performs a left lane marking comparison by comparing the device lane marking the left side of the driving lane with the map lane markings at a first distance in the direction of travel as seen from the vehicle, and performs a right lane marking comparison by comparing the device lane marking the right side of the driving lane with the map lane markings, and if the results of the left lane marking comparison and the right lane marking comparison satisfy a specific condition, performs a left lane marking comparison and the right lane marking comparison at a second distance shorter than the first distance, and controls the vehicle based on the results of the comparison, wherein the specific condition is that at the first distance, the degree of deviation of either the left lane marking comparison or the right lane marking comparison is less than a threshold.

[0017] (12): A program according to another aspect of the present invention causes a computer to recognize the left and right lane markings that demarcate the vehicle's driving lane as seen from the vehicle, based on the output of a detection device that detects the surrounding conditions of the vehicle, as device lane markings; recognize the lane markings that demarcate the vehicle's driving lane as map lane markings from map information based on the vehicle's position information; perform a left lane marking comparison by comparing the device lane marking the left side of the driving lane with the map lane markings at a first distance in the direction of travel as seen from the vehicle, and perform a right lane marking comparison by comparing the device lane marking the right side of the driving lane with the map lane markings; if the results of the left lane marking comparison and the right lane marking comparison satisfy a specific condition, perform a left lane marking comparison and the right lane marking comparison at a second distance shorter than the first distance, and control the vehicle based on the comparison results, wherein the specific condition includes that at the first distance, the degree of deviation of either the left lane marking comparison or the right lane marking comparison is less than a threshold. [Effects of the Invention]

[0018] According to the aspects of (1) to (12) above, more appropriate driving control can be executed based on the recognition result of road marking lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] [Figure 1] FIG. 1 is a configuration diagram of a vehicle system 1 including a vehicle control device according to an embodiment. [Figure 2] FIG. 2 is a functional configuration diagram of a first control unit 120 and a second control unit 160. [Figure 3] FIG. 3 is a diagram for explaining driving control of a vehicle M in a first scenario. [Figure 4] FIG. 4 is a diagram for explaining driving control of the vehicle M in a second scenario. [Figure 5] FIG. 5 is a diagram for explaining driving control of the vehicle M in a third scenario. [Figure 6] FIG. 6 is a diagram for explaining driving control of the vehicle M in a fourth scenario. [Figure 7] FIG. 7 is a flowchart showing an example of a flow of driving control processing in a first embodiment. [Figure 8] FIG. 8 is a flowchart showing an example of a flow of driving control processing in a second embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of a flow of driving control processing in a third embodiment. MODES FOR CARRYING OUT THE INVENTION

[0020] The following describes embodiments of the vehicle control device, vehicle control method, and program of the present invention with reference to the drawings. The following describes embodiments in which the vehicle control device is applied to an autonomous vehicle. Autonomous driving refers to performing driving control by automatically controlling, for example, the steering or speed of the vehicle, or both. The above-mentioned driving control may include various driving control systems such as ACC (Adaptive Cruise Control System), LKAS (Lane Keeping Assistance System), ALC (Automated Lane Change), TJP (Traffic Jam Pilot), and CMBS (Collision Mitigation Brake System). In addition, autonomous vehicles may also perform driving control by manual operation by the vehicle user (e.g., occupants) (so-called manual driving). The vehicle control device in the embodiments may be applied not only to vehicles, but also to mobile bodies such as ships that can move on the ground, such as hovercraft, flying vehicles that can travel on roads, and stand-up vehicles with power units.

[0021] [Overall structure] Figure 1 is a diagram showing the configuration of a vehicle system 1 including a vehicle control device according to an embodiment. The vehicle on which the vehicle system 1 is installed (hereinafter referred to as vehicle M) is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle or a micromobility, and its power source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using power generated by a generator connected to the internal combustion engine, or power discharged from a battery (storage battery) such as a secondary battery or a fuel cell.

[0022] Vehicle system 1 includes, for example, a camera 10, a radar device 12, a LiDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driver control unit 80, an automatic driving control device 100, a driving force output device 200, a brake device 210, and a steering device 220. These devices and equipment are connected to each other by multiplex communication lines such as CAN (Controller Area Network) communication lines, serial communication lines, wireless communication networks, etc. Note that the configuration shown in Figure 1 is merely an example, and some of the configuration may be omitted, or other configurations may be added. A combination of camera 10, radar device 12, LiDAR 14, and object recognition device 16 is an example of a "detection device DD". HMI 30 is an example of an "output device". Automatic driving control device 100 is an example of a "vehicle control device".

[0023] Camera 10 is a digital camera that uses a solid-state image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). Camera 10 is mounted at any location on the vehicle M on which the vehicle system 1 is installed. When imaging the front, camera 10 is mounted on the top of the front windshield, behind the rearview mirror, or on the front of the vehicle body. When imaging the rear, camera 10 is mounted on the top of the rear windshield or on the tailgate. When imaging the side, camera 10 is mounted on the door mirror or the like. Camera 10 periodically and repeatedly images the area around vehicle M. Camera 10 may also be a stereo camera.

[0024] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by surrounding objects (reflected waves) to determine at least the position (distance and direction) of an object. The radar device 12 can be mounted at any location on the vehicle M. The radar device 12 may also detect the position and velocity of an object using the FM-CW (Frequency Modulated Continuous Wave) method.

[0025] The LIDAR 14 illuminates the area around the vehicle M with light and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time from emission to reception. The emitted light is, for example, pulsed laser light. The LIDAR 14 can be mounted at any location on the vehicle M.

[0026] The object recognition device 16 performs sensor fusion processing on some or all of the detection results from the camera 10, radar device 12, and LIDAR 14 included in the detection device DD to recognize the position, type, speed, etc., of an object. The object recognition device 16 outputs the recognition results to the automatic driving control device 100. Alternatively, the object recognition device 16 may output the detection results from the camera 10, radar device 12, and LIDAR 14 directly to the automatic driving control device 100. In that case, the object recognition device 16 may be omitted from the configuration of the vehicle system 1 (detection device DD).

[0027] The communication device 20 communicates with other vehicles in the vicinity of vehicle M, terminal devices of users using vehicle M, or various server devices, for example, by utilizing networks such as cellular networks, Wi-Fi networks, Bluetooth®, DSRC (Dedicated Short Range Communication), LAN (Local Area Network), WAN (Wide Area Network), and the Internet.

[0028] The HMI30 outputs various information to the occupants of vehicle M (including the driver) and accepts input operations from the occupants. The HMI30 includes, for example, a display unit and a speaker. The display unit is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display device. The display unit displays various images (including video) in the embodiment. The display unit may be configured integrally with the input unit as a touch panel. The speaker outputs predetermined sounds (e.g., alarm sounds or message sounds). In addition to the display unit and speaker, the HMI30 may also include (or replace) a microphone, buzzer, touch panel, switches, keys, etc. Switches include switches that execute or terminate predetermined driving controls (e.g., ACC or LKAS) that can be executed by the driving control unit described later, and switches that approve (permit) or reject driving control recommendations (suggestions) from the system (vehicle system 1). Switches may also include switches for operating the turn signals (turn signal switches).

[0029] The vehicle sensor 40 includes a vehicle speed sensor for detecting the speed of the vehicle M, an acceleration sensor for detecting acceleration, a yaw rate sensor for detecting yaw rate (for example, the angular velocity of rotation around the vertical axis passing through the center of gravity of the vehicle M), and an orientation sensor for detecting the orientation of the vehicle M. The vehicle sensor 40 may also be provided with a position sensor for detecting the position of the vehicle. The position sensor is an example of a "position measurement unit". The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. Alternatively, the position sensor may be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver 51 of the navigation device 50. The vehicle sensor 40 may derive the speed of the vehicle M from the difference (i.e., distance) of position information at a predetermined time in the position sensor. The results detected by the vehicle sensor 40 are output to the automatic driving control device 100.

[0030] The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as an HDD (Hard Disk Drive) or flash memory. The GNSS receiver 51 determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be determined or supplemented by an INS (Inertial Navigation System) that utilizes the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, speaker, touch panel, keys, etc. The GNSS receiver 51 may be provided on the vehicle sensor 40. The navigation HMI 52 may be partially or completely shared with the HMI 30 described above. The route determination unit 53 determines, for example, a route (hereinafter referred to as a route on the map) from the position of the vehicle M determined by the GNSS receiver 51 (or any input position) to the destination input by the occupant using the navigation HMI 52, by referring to the first map information 54. The first map information 54 is information in which the shape of a road is represented by links indicating roads (an example of a travel route) and nodes connected by those links. The first map information 54 may also include POI (Point of Interest) information, etc. The route on the map is output to the MPU 60. The navigation device 50 may perform route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may transmit its current location and destination to the navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server. The navigation device 50 outputs the determined route on the map to the MPU 60.

[0031] The MPU 60 includes, for example, a recommended lane determination unit 61 and stores second map information 62 in a storage device such as an HDD or flash memory. The recommended lane determination unit 61 divides the map route provided by the navigation device 50 into multiple blocks (for example, every 100m with respect to the vehicle's direction of travel) and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 makes decisions such as which lane from the left the vehicle should travel in. If there is a branching point in the map route, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel along a reasonable route to proceed to the branching point.

[0032] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, the number of lanes (number of travel routes), the type and shape of road markings (hereinafter referred to as markings), information on the center of the lanes, or information on road boundaries. The second map information 62 may also include information on whether the road boundary is a boundary (physical boundary) that includes a structure that vehicles cannot pass through (including crossing or contact). A physical boundary is, for example, a guardrail, curb, median strip, fence, etc. The second map information 62 may also include road shape information, traffic regulation information, address information (address and postal code), facility information, parking information, telephone number information, etc. Road shape information is, for example, lane width (width), gradient, branching, merging, intersection, road curvature (may be rephrased as radius of curvature; the same applies hereinafter), amount of change in curvature, etc. The second map information 62 may be updated as needed by the communication device 20 communicating with an external device. The first map information 54 and the second map information 62 may be provided together as map information. The map information may also be stored in the storage unit 190.

[0033] The driver control unit 80 includes, for example, a steering wheel, an accelerator pedal, and a brake pedal. The driver control unit 80 may also include a shift lever, a modified steering wheel, a joystick, or other controls. Each control of the driver control unit 80 is equipped with an operation detection unit that detects, for example, the amount of operation performed by the occupant on the control unit or whether or not an operation has been performed. The operation detection unit detects, for example, the steering angle and steering torque of the steering wheel, the amount of depression of the accelerator pedal and brake pedal, etc. The operation detection unit then outputs the detection result to the automatic driving control device 100, or to one or both of the driving force output device 200, the brake device 210, and the steering device 220.

[0034] The automatic driving control device 100 performs various driving controls belonging to automatic driving on the vehicle M. The automatic driving control device 100 includes, for example, a first control unit 120, a second control unit 160, an HMI control unit 180, and a storage unit 190. The first control unit 120, the second control unit 160, and the HMI control unit 180 are each realized by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Furthermore, some or all of these components may be realized by hardware (including circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), and SOC (System On Chip), or by the cooperation of software and hardware. The above-mentioned program may be stored in advance in a storage device (a storage device equipped with a non-transient storage medium) such as the HDD or flash memory of the automatic driving control device 100, or it may be stored in a removable storage medium such as a DVD, CD-ROM, or memory card, and installed in the storage device of the automatic driving control device 100 when the storage medium (non-transient storage medium) is inserted into a drive device or card slot.

[0035] The storage unit 190 may be implemented using the various storage devices described above, or an EEPROM (Electrically Erasable Programmable Read Only Memory), ROM (Read Only Memory), or RAM (Random Access Memory), etc. The storage unit 190 may store, for example, various information and programs as described in the embodiment. The storage unit 190 may also store map information (for example, first map information 54 and second map information 62).

[0036] Figure 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 140. The first control unit 120 implements, for example, functions using AI (Artificial Intelligence) and functions using a pre-defined model in parallel. For example, the function of "recognizing intersections" may be implemented by simultaneously performing intersection recognition using deep learning and recognition based on pre-defined conditions (such as pattern-matchable signals and road markings), scoring both, and comprehensively evaluating them. This ensures the reliability of autonomous driving. The first control unit 120 also performs control related to the autonomous driving of the vehicle M based on instructions from, for example, the MPU 60 and the HMI control unit 180.

[0037] The recognition unit 130 recognizes the surrounding conditions of vehicle M based on the recognition results of the detection device DD (information input from camera 10, radar device 12, and LIDAR 14 via object recognition device 16). For example, the recognition unit 130 recognizes the position, speed, acceleration, and other states of objects present around vehicle M (within a predetermined distance). Objects include other vehicles, pedestrians, bicycles, and other traffic participants, as well as physical boundaries that demarcate roads (travel paths). The position of an object is recognized, for example, as a position on an absolute coordinate system with a representative point of vehicle M (such as the center of gravity or the center of the drive axis) as the origin, and is used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or by a represented region. The "state" of an object may include, for example, the acceleration or jerk of a moving object, or its "action state" (for example, whether the other vehicle is changing lanes or is about to change lanes), if the object is a moving object such as another vehicle.

[0038] Furthermore, the recognition unit 130 recognizes, for example, stop lines, obstacles, red lights, toll booths, other road events, road markings (speed limits), and road signs indicating speed limits. The recognition unit 130 also includes, for example, a first recognition unit 132 and a second recognition unit 134. Details of these functions will be described later.

[0039] The action plan generation unit 140 generates an action plan for driving vehicle M by autonomous driving based on the recognition results of the recognition unit 130, etc. For example, the action plan generation unit 140, in principle, drives in the recommended lane determined by the recommended lane determination unit 61, and further generates a target trajectory for vehicle M to travel automatically (without driver operation) in the future, based on the recognition results of the recognition unit 130 and the surrounding road shape based on the current position of vehicle M obtained from map information, etc., so as to respond to the surrounding conditions of vehicle M. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequence of points (trajectory points) that vehicle M should reach. The trajectory points are points that vehicle M should reach at predetermined driving distances (e.g., a few meters) along the road, and separately, target speed and target acceleration at predetermined sampling times (e.g., a few tenths of a second) are generated as part of the target trajectory. Alternatively, the trajectory points may be the positions that vehicle M should reach at the sampling time for each predetermined sampling time. In this case, information about the target velocity and target acceleration is represented by the interval between trajectory points.

[0040] The action plan generation unit 140 may set automated driving events when generating a target trajectory. These events include, for example, a constant speed driving event that drives vehicle M in the same lane at a constant speed, a follow driving event corresponding to ACC that causes vehicle M to follow the nearest vehicle that is within a predetermined distance (for example, within 100 [m]) in front of vehicle M, a lane keeping driving event corresponding to LKAS that causes vehicle M to drive in the center of the driving lane, a lane change event corresponding to ALC that causes vehicle M to change lanes from its own lane to an adjacent lane, a branching event that branches vehicle M to the destination lane at a road branching point, a merging event that causes vehicle M to merge onto the main road at a merging point, and a takeover event to end automated driving and switch to manual driving. Furthermore, events may include, for example, an overtaking event in which vehicle M temporarily changes lanes to an adjacent lane, overtakes a preceding vehicle in the adjacent lane, and then changes lanes back to its original lane, and an avoidance event in which vehicle M brakes and / or steers to avoid an obstacle in front of vehicle M.

[0041] Furthermore, the action plan generation unit 140 may, for example, change an event already determined for the current section to another event, or set a new event for the current section, depending on the surrounding conditions of vehicle M recognized while vehicle M is in motion. Also, the action plan generation unit 140 may change an event already set for the current section to another event, or set a new event for the current section, depending on the occupant's operation on the HMI 30. The action plan generation unit 140 generates a target trajectory according to the set event.

[0042] Furthermore, the action plan generation unit 140 includes, for example, a comparison unit 142, a driving control unit 144, and a correction unit 146. The driving control unit 144 is an example of a "driving control unit." Details of these functions will be described later.

[0043] The second control unit 160 controls the driving force output device 200, the braking device 210, and the steering device 220 so that the vehicle M passes through the target trajectory generated by the action plan generation unit 140 at the scheduled time.

[0044] The second control unit 160 includes, for example, a target trajectory acquisition unit 162, a speed control unit 164, and a steering control unit 166. The target trajectory acquisition unit 162 acquires information on the target trajectory (trajectory points) generated by the action plan generation unit 140 and stores it in memory (not shown). The speed control unit 164 controls the driving force output device 200 or the brake device 210 based on the speed elements associated with the target trajectory stored in memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in memory. The processing of the speed control unit 164 and the steering control unit 166 is realized, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 performs a combination of feedforward control according to the curvature of the road in front of the vehicle M and feedback control based on the deviation from the target trajectory.

[0045] Returning to Figure 1, the HMI control unit 180 notifies the occupant of predetermined information via the HMI 30. The predetermined information includes, for example, information related to the driving of vehicle M, such as information regarding the status of vehicle M and information regarding driving control. Information regarding the status of vehicle M includes, for example, the speed of vehicle M, engine speed, and shift position. Information regarding driving control includes, for example, information inquiring whether or not to perform driving control by automatic driving, whether or not to start automatic driving, information regarding the status of driving control by automatic driving, information regarding the driving mode, and information prompting the occupant to take action when switching from automatic driving to manual driving. The predetermined information may also include information regarding the surrounding conditions recognized by the detection device DD. The predetermined information may also include information unrelated to the driving of vehicle M, such as content stored on a storage medium such as a television program or DVD (e.g., a movie). The predetermined information may also include, for example, the current location and destination in automatic driving, and information regarding the remaining fuel level of vehicle M. The HMI control unit 180 may output the information received by the HMI 30 to the communication device 20, the navigation device 50, the first control unit 120, etc.

[0046] Furthermore, the HMI control unit 180 may output to the HMI 30 information such as inquiry information for the occupants and processing results from the first control unit 120 and the second control unit 160. In addition, the HMI control unit 180 may transmit various information to be output to the HMI 30 to terminal devices used by the occupants of the vehicle M via the communication device 20.

[0047] The driving force output device 200 outputs driving force (torque) to the drive wheels for the vehicle to move. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, and an ECU (Electronic Control Unit) that controls them. The ECU controls the above configuration according to information input from the second control unit 160 or information input from the accelerator pedal of the driver control unit 80.

[0048] The brake system 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or from the brake pedal of the driver control unit 80, so that brake torque corresponding to the braking operation is output to each wheel. The brake system 210 may be equipped with a backup mechanism that transmits the hydraulic pressure generated by the operation of the brake pedal to the cylinder via a master cylinder. The brake system 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic brake system that controls an actuator according to information input from the second control unit 160 to transmit hydraulic pressure from the master cylinder to the cylinder.

[0049] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies force to a rack and pinion mechanism to change the direction of the steering wheels. The steering ECU drives the electric motor to change the direction of the steering wheels according to information input from the second control unit 160 or information input from the steering wheel of the driver control unit 80.

[0050] [Recognition unit and action plan generation unit] Next, we will explain in detail the functions of the recognition unit 130 (mainly the first recognition unit 132 and the second recognition unit 134) and the action plan generation unit 140 (mainly the comparison unit 142, the driving control unit 144, and the correction unit 146). In the following, we will explain by dividing the explanation into several scenarios in which the recognition result by the recognition unit 130 and the content of the driving control based on that recognition result differ.

[0051] [Scene 1] Figure 3 is a diagram illustrating the driving control of vehicle M in the first scenario. In the example in Figure 3, device lane markings CL1 and CL2 recognized by the detection device DD, and map lane markings ML1 and ML2 obtained from map information (e.g., second map information 62) based on the position information of vehicle M are shown. For example, in the map information, lane L1 is demarcated by map lane markings ML1 and ML2. Lane L1 is a lane that can be traveled in the direction of extension (X-axis direction in the figure). Furthermore, when device lane markings CL1 and CL2 are not distinguished, they may simply be referred to as "device lane marking CL," and when map lane markings ML1 and ML2 are not distinguished, they may simply be referred to as "map lane marking ML." Also, in the first scenario shown in Figure 3, vehicle M is traveling on lane L1 at speed VM. Hereafter, lane L1 may be referred to as "driving lane L1" as needed.

[0052] In the first scenario, the first recognition unit 132 recognizes the surrounding conditions (external environment) of vehicle M based on the output of the detection device DD that detects the surrounding conditions (external environment) of vehicle M. For example, the first recognition unit 132 recognizes the left and right lane markings that demarcate the vehicle M's driving lane (lane L1) as seen from vehicle M as device lane markings CL1 and CL2, based on the image captured by the camera 10 (hereinafter referred to as the camera image). In the example in Figure 3, the device lane markings CL1 and CL2 in front of vehicle M are shown, but the side and rear device lane markings CL1 and CL2 may also be recognized.

[0053] For example, the first recognition unit 132 analyzes the camera image, extracts edge points with large brightness differences from adjacent pixels in the image, and connects these edge points to recognize device demarcation lines CL1 and CL2 in the image plane. The first recognition unit 132 also converts the positions of the device demarcation lines CL1 and CL2, based on the position of a representative point of the vehicle M, into the vehicle coordinate system (for example, the XY plane coordinates in Figure 3).

[0054] Furthermore, the first recognition unit 132 may recognize, for example, the curvature of the device lane markings CL1 and CL2. The first recognition unit 132 may also recognize the amount of change in curvature of the device lane markings CL1 and CL2. The amount of change in curvature is, for example, the rate of change over time of the curvature of the device lane markings CL1 and CL2 recognized by the camera 10 at a distance x [m] in front of the vehicle M. The first recognition unit 132 may also recognize the curvature or amount of change in curvature of the lane demarcated by the device lane markings CL1 and CL2 by averaging the respective curvature or amount of change in curvature of the device lane markings CL1 and CL2. The device lane markings CL1 and CL2 may be recognized or corrected based on the output of detection devices other than the camera 10 (for example, radar device 12, LIDAR 14).

[0055] The second recognition unit 134 recognizes lane markings around vehicle M from map information based on the position of vehicle M detected by, for example, vehicle sensor 40 or GNSS receiver 51. For example, the second recognition unit 134 refers to map information based on the position information of vehicle M and recognizes the left and right lane markings that demarcate vehicle M's driving lane L1 as map markings ML1 and ML2.

[0056] The second recognition unit 134 may also recognize the curvature or the amount of change in curvature of the map lane lines ML1 and ML2 from the second map information 62. Alternatively, the second recognition unit 134 may average the curvature or the amount of change in curvature of the map lane lines ML1 and ML2 to recognize the curvature or the amount of change in curvature of the driving lane L1.

[0057] The comparison unit 142 compares the device lane markings CL1 and CL2 recognized by the first recognition unit 132 with the map lane markings ML1 and ML2 recognized by the second recognition unit 134. For example, the comparison unit 142 first performs the above comparison at a distant location in the direction of travel as seen from the vehicle M, and then performs the above comparison at a nearby location based on the comparison result. Specifically, the comparison unit 142 performs a comparison of the left lane markings by comparing the device lane marking CL1 that marks the left side of the driving lane L1 with the map lane marking ML1, and a comparison of the right lane markings by comparing the device lane marking CL2 that marks the right side of the driving lane L1 with the map lane marking ML2, at a first distance D1 in the direction of travel (forward) from the current position of the vehicle M (point P0 in the figure). Furthermore, the comparison at the first distance D1 may also refer to the comparison over the section from the second distance D2 to the first distance D1 (the section from point P2 to point P1), as described later.

[0058] For example, the comparison unit 142 compares at least one of the following as a comparison of the left-side lane lines at a first distance D1: the position, extension direction (angle), curvature, and change in curvature between the device lane line CL1 and the map lane line ML1. For example, the comparison unit 142 superimposes the device lane line CL1 and the map lane line ML1 on the vehicle coordinate system plane (XY plane) with respect to the position of a representative point of the vehicle M, compares the lateral position (lane width direction, Y-axis direction in the figure) of the device lane line CL1 and the map lane line ML1, and obtains the displacement amount W11. Alternatively, the comparison unit 142 may compare the respective extension directions of the device lane line CL1 and the map lane line ML1 and obtain the displacement angle θ1 of the lane lines. Alternatively, the comparison unit 142 may compare the curvature or change in curvature of the device lane line CL1 and the map lane line ML1 and obtain the degree (magnitude) of the difference. The comparison unit 142 may then obtain a degree of deviation based on the amount of displacement W11, the displacement angle θ1, and the degree of difference in curvature or the amount of change in curvature. In this case, the larger the amount of displacement W11, the larger the displacement angle θ1, and the greater the degree of difference, the larger the degree of deviation.

[0059] Furthermore, the comparison unit 142, similar to the comparison of the left-side lane lines, compares the right-side lane lines at a first distance D1 by comparing at least one of the following: the position (amount of displacement W21 in the figure), the direction of extension (angle), the curvature, and the amount of change in curvature between the device lane line CL2 and the map lane line ML2, and obtains the degree of discrepancy.

[0060] Furthermore, the comparison unit 142 performs a comparison of the left lane markings and the right lane markings at a second distance D2 that is shorter than the first distance D1, if the results of the comparison of the left lane markings and the right lane markings described above satisfy specific conditions. The specific conditions include, for example, that as a result of the comparison of the left lane markings and the right lane markings at the first distance D1, the degree of deviation of either one is less than a threshold. In other words, the specific conditions are met when only one of the left lane markings or the right lane markings has a degree of deviation less than a threshold (the degree of deviation of the other is greater than or equal to the threshold). Here, the comparison at the second distance D2 is a comparison near point P2 that is a distance of the second distance D2 in the direction of travel from the vehicle M, and may include a predetermined tolerance range. Alternatively, the comparison at the second distance D2 may be a comparison over the section from the current position of the vehicle M to the second distance D2 (the section from point P0 to point P2).

[0061] In the example shown in Figure 3, the degree of deviation of the left lane line is greater than or equal to the threshold, and the degree of deviation of the right lane line is less than the threshold, thus satisfying the specific conditions. In this case, the comparison unit 142 performs a comparison of the left lane line and the right lane line (for example, a comparison of the deviation amounts W12 and W22 in the figure, the deviation angle, curvature, the amount of change in curvature, etc.) at a second distance D2 which is shorter than the first distance D1, in the same manner as described above. The first distance D1 and the second distance D2 may be fixed distances, or they may be variable distances depending on the speed VM of the vehicle M and the road shape (for example, the presence or absence of branches or merges, curvature, the amount of change in curvature, etc.). The first distance D1 may also be the performance limit distance of the detection device DD (recognizable limit distance). The second distance D2 may also be the distance obtained by subtracting the fixed distance from the first distance D1.

[0062] The driving control unit 144 controls the vehicle M (driving control) based on the recognition results of the first recognition unit 132 and the second recognition unit 134, the comparison results of the comparison unit 142, etc. For example, the driving control unit 144 determines the driving control for the vehicle M based on the above recognition results and comparison results, etc., and generates a target trajectory based on the determined driving control. "Determining driving control" may include, for example, determining the content (type) of the driving control, or deciding whether or not to execute (suppress) the driving control. Also, "executing driving control" may include, for example, switching and executing the content of the driving control, as well as continuing the driving control that is already being executed. "Suppressing driving control" may include not only not executing driving control, but also lowering the mode (automation level) of the driving control.

[0063] Here, the driving control includes a first driving mode and a second driving mode in which the degree of driving assistance is lower than that of the first driving mode, or in which the tasks of the vehicle M occupants are greater than those of the first driving mode. A lower degree of driving assistance means, for example, a low automation rate in the driving control. A low automation rate means, for example, a low degree to which the automatic driving control device 100 controls the steering or speed of the vehicle M (a high degree to which the driver needs to intervene in steering or acceleration / deceleration operations). A greater tasks for the occupants includes, for example, a large number of tasks assigned to the occupants or tasks that are heavy. Tasks include, for example, monitoring the surroundings of the vehicle M or the occupants operating the driving controls 80. Operating the driving controls includes, for example, the state in which the driver holds the steering wheel (hereinafter referred to as the hands-on state). The driving control may also include a third driving mode, etc., in which the degree of driving assistance is lower than that of the second driving mode, or in which the tasks of the vehicle M occupants are greater than those of the second driving mode. Furthermore, the driving mode in which the degree of driver assistance is lowest or the task of the vehicle M occupant is greatest may be the manual driving mode (a mode in which no driving control is performed).

[0064] For example, in the first driving mode, the occupant has no (or minimal) tasks, and driving control (e.g., ACC, LKAS, ALC, TJP, CMBS, etc.) is permitted when, for example, the occupant of vehicle M is not holding the steering wheel (hereinafter referred to as the hands-off state). In the second driving mode, tasks assigned to the occupant may include, for example, monitoring the surroundings of vehicle M, as well as being in a hands-on state.

[0065] For example, the driving control unit 144, while executing a first driving mode (for example, driving control in a hands-off state), will continue the first driving mode if, as a result of comparing the left lane markings and the right lane markings at a first distance D1, the degree of deviation of both lane markings is below a threshold. For example, when the driving control unit 144 is executing LKAS which drives in the center of the driving lane L1 as the first driving mode, it generates a target trajectory for driving in the center of the driving lane L1 based on device lane markings CL1, CL2 or map lane markings ML1, ML2, and causes the second control unit 160 to execute control to drive along the generated target trajectory K1.

[0066] In this embodiment, the map lane markings ML may be corrected by the correction unit 146 before the driving control unit 144 generates the target trajectory K1. For example, as described above, if the degree of deviation of both lane markings at the first distance D1 is both below a threshold, the correction unit 146 corrects the position of map lane marking ML1 to match the position of device lane marking CL1 in the section from the vehicle M to the first distance D1 (point P1), and corrects the position of map lane marking ML2 to match the position of device lane marking CL2. Position correction may be, for example, correction of the amount of deviation in the lateral direction (lane width direction) of each lane marking, correction of the deviation angle, or correction of curvature or the amount of change in curvature. Correcting to match means correcting the positions of the lane markings so that they are aligned (overlapped), and correcting the degree of deviation to be within an acceptable range smaller than the threshold. In addition, instead of correcting each map lane marking ML1 and ML2, the correction unit 146 may correct the position of lane L1, or correct the position of the entire map information. The driving control unit 144 can generate a more appropriate target track K1 by using the corrected map section lines ML1 and ML2, and can then achieve more appropriate driving control using this target track K1.

[0067] Furthermore, the driving control unit 144 may, while the second driving mode is in operation, perform control to switch from the second driving mode to the first driving mode if, as a result of comparing the left lane markings and the right lane markings at the first distance D1, the degree of deviation of both lane markings is below a threshold. Also, when performing switching control, the driving control unit 144 may output information to the HMI 30 asking the occupant whether or not switching is possible or whether they wish to switch, and switch the driving mode when information indicating permission to switch is received from the occupant. Also, the driving control unit 144 may switch from the second driving mode to the first driving mode when the degree of deviation of the left and right lane markings falls below a threshold after the HMI 30 has given an instruction to switch to the first driving mode. Furthermore, the driving control unit 144 may suppress driving control by the first driving mode if, as a result of comparing the left lane markings and the right lane markings at the first distance D1, the degree of deviation of both lane markings is above a threshold.

[0068] Furthermore, the driving control unit 144 may, for example, continue the first driving mode even if, as shown in Figure 3, the comparison of the left lane markings and the right lane markings at a first distance D1 satisfies specific conditions while the first driving mode is in operation. If the degree of deviation is less than a threshold for even one of them, for example, the driving control unit 144 can set the other lane marking at a position offset by the same distance on the other side, based on the lateral (lane width direction) distance between that lane marking and the vehicle M, and allow the vehicle M to drive. Therefore, even in such cases, the hands-off state can be maintained, improving the continuity of driving control. Note that, while the second driving mode is in operation, if the above-mentioned specific conditions are met, the driving control unit 144 may choose not to switch to the first driving mode, or it may decide whether or not to switch based on the comparison result at a second distance D2.

[0069] For example, as shown in Figure 3, if the comparison of the left lane markings and the right lane markings at a first distance D1 satisfies specific conditions, and the degree of deviation between the left and right lane markings in the comparison of the left lane markings and the right lane markings at a second distance D2 is less than a threshold, the correction unit 146 corrects the position of map lane marking ML1 to match the position of device lane marking CL1 in the section from the current position of vehicle M to the second distance D2, and corrects the position of map lane marking ML2 to match the position of device lane marking CL2. Then, the driving control unit 144 controls vehicle M (for example, driving control using a first driving mode such as LKAS) based on the map lane markings ML1 and ML2 corrected by the correction unit 146.

[0070] For example, recognition errors are more likely to occur at a long distance from vehicle M (first distance D1) compared to a short distance (second distance D2). Therefore, in this embodiment, as shown in Figure 3, even if the degree of deviation between the device lane line CL and the map lane line ML at a long distance from vehicle M is greater than or equal to a threshold, if the degree of deviation between the device lane line CL and the map lane line ML at a short distance from vehicle M is less than a threshold, the positions of the map lane lines ML1 and ML2 at the short distance can be corrected to recognize the left and right lane lines more appropriately and perform driving control based on the recognition result. Note that in the comparison between the degree of deviation and the threshold described above, the threshold for the comparison at the first distance D1 and the threshold for the comparison at the second distance D2 may be different values.

[0071] [Scene 2] Next, the second scenario will be explained. Figure 4 is a diagram illustrating the driving control of vehicle M in the second scenario. The second scenario differs from the first scenario in that, at the second distance D2, the degree of deviation between the device lane line CL2 and the map lane line ML2 (for example, the degree of deviation based on the amount of deviation W22, the deviation angle θ2, etc.) is above a threshold. In other words, in the second scenario, at the first distance D1, the degree of deviation between the device lane line CL1 and the map lane line ML1 is above a threshold, and at the second distance D2, the degree of deviation between the device lane line CL2 and the map lane line ML2 is above a threshold.

[0072] In the second scenario, as shown in Figure 4, if the degree of deviation of either the left lane marking and the right lane marking in the comparison of the left lane markings and the right lane markings at the second distance D2 is greater than or equal to a threshold, and the degree of deviation of the lane marking on the same side as the lane marking with a deviation greater than or equal to the threshold is less than the threshold at the first distance D1, then the map lane marking ML2 is corrected based on the device lane marking CL2 at the first distance D1. In this case, the correction unit 146 corrects the position of the map lane marking ML2 in the section from point P2 to point P1 so that it matches the position of the device lane marking CL2 in the same section. This correction also corrects the position of the map lane marking ML2 in the section from the current position of the vehicle M to the second distance D2 (the section from point P0 to point P2).

[0073] Furthermore, in the example shown in Figure 4, the degree of deviation in the comparison of the left-hand lane lines at the second distance D2 is below the threshold. Therefore, the correction unit 146 may correct the position of the map lane line ML1 in the section from the current position of the vehicle M to the second distance D2 (the section from point P0 to point P2) so that it matches the position of the device lane line CL1 in the same section. This allows the target trajectory K1 to be generated based on the corrected map lane line ML when the first driving mode is in operation in the second scenario.

[0074] Furthermore, if the first driving mode is in operation and the degree of deviation between the left and right lane markings at the second distance D2 is greater than or equal to a threshold, the correction unit 146 does not need to correct the lane markings. Even in this case, the driving control unit 144 can generate a target track K1 based on, for example, the device lane marking CL, and continue the first driving mode based on the generated target track K1. Alternatively, instead of performing correction, if the degree of deviation at the first distance D1 is less than a threshold, the correction unit 146 may correct the position of the map lane marking ML at the first distance D1 based on the position of the device lane marking CL at the first distance D1, and generate a target track K1 based on the corrected map lane markings.

[0075] Then, the driving control unit 144 controls the vehicle M based on the map lane markings ML corrected by the correction unit 146. In this way, even if a situation like the second scene occurs while the first driving mode is being executed, the first driving mode (hands-off state) can be continued, improving the continuity of driving control.

[0076] In this embodiment, in situations like the second scenario, the driving control unit 144 may switch to the second driving mode (hands-on state) instead of continuing the first driving mode (hands-off state). For example, the driving control unit 144 changes the control of the vehicle M from the first driving mode to the second driving mode if, while the first driving mode is in operation, at a second distance D2, the degree of deviation of at least one of the left lane markings and the right lane markings is greater than or equal to a threshold. For example, if the degree of deviation of the lane markings is greater than or equal to a threshold at close range where recognition accuracy is high, safety can be further improved by switching to the hands-on state even if the degree of deviation is less than the threshold at long distances.

[0077] In the second scenario, whether to continue in the first driving mode (hands-off state) or switch to the second driving mode (hands-on state) may be set based on, for example, the magnitude of the deviation, or based on the surrounding conditions of the vehicle M recognized by the first recognition unit 132. For example, even if the deviation at the second distance D2 is greater than or equal to a threshold, if it is less than a limit value, the first driving mode is continued after correction by the correction unit 146, and if the deviation is greater than or equal to a limit value, the system switches to the second driving mode. In addition, if there are more than a predetermined number of other vehicles around the vehicle M (within a predetermined distance), or if the road shape is difficult to recognize (for example, a curved road), the system switches to the second driving mode, and if there are fewer than a predetermined number of other vehicles or if the road shape is easy to recognize, the system continues in the first driving mode. In this way, more appropriate driving control can be performed according to the recognition status and surrounding conditions.

[0078] [Scene 3] Next, the third scenario will be explained. Figure 5 is a diagram illustrating the driving control of vehicle M in the third scenario. The example in Figure 5 differs from the example in Figure 3 in that, at the second distance D2, the degree of deviation between device lane line CL1 and map lane line ML1 (for example, the degree of deviation based on the amount of deviation W12, the deviation angle θ3, etc.) is greater than or equal to the threshold. In other words, in the third scenario, at both the first distance D1 and the second distance D2, the degree of deviation between device lane line CL1 and map lane line ML1 is greater than or equal to the threshold, and the degree of deviation between device lane line CL2 and map lane line ML2 is less than the threshold. In other words, the third scenario is when, at the second distance D2, the degree of deviation of only one of the left and right boundary lines is below the threshold, and the boundary line on the side where the degree of deviation at the second distance D2 was below the threshold is the same as the boundary line on the side where the degree of deviation at the first distance D1 was below the threshold.

[0079] In this case, the correction unit 146 corrects the map boundary line ML2 based on the device boundary line CL2 at the second distance D2 for the boundary line on the side where the degree of deviation between the left and right boundary lines was below the threshold. Since recognition accuracy is higher at close range than at long range, correcting the map boundary line ML2 based on the device boundary line CL2 at the second distance D2 allows for more accurate correction.

[0080] The driving control unit 144 then controls the vehicle M based on the map lane markings ML2 corrected by the correction unit 146. In this case, the driving control unit 144, for example, obtains the lateral distance from the vehicle M to the map lane markings ML2, sets the left lane markings at a position lateral to the left of the vehicle M by that distance, generates a target trajectory K1 so that the vehicle travels in the center of the lane divided by the left and right lane markings, and causes the second control unit 160 to execute driving control to drive the vehicle M along the generated target trajectory K1. As shown in the third scene, when the degree of deviation of one lane marking is below a threshold at both the first distance D1 and the second distance D2, the correction can be made with higher accuracy by performing a correction according to the position of the lane marking at the second distance D2, and more appropriate driving control can be achieved by generating a target trajectory K1 based on the corrected lane markings.

[0081] In the third scenario, the correction unit 146 may, if the degree of deviation of only one of the left and right grid lines in the comparison of the left grid lines and the right grid lines at the second distance D2 is less than the threshold, perform control to correct the map grid lines based on the device grid line at the second distance D2 of the grid line on the side where the degree of deviation at the first distance D1 was less than the threshold.

[0082] [Scene 4] Next, we will explain the fourth scenario. Figure 6 is a diagram illustrating the driving control of vehicle M in the fourth scenario. The fourth scenario shows a situation in which a specific road shape exists near the first distance D1. A specific road shape is a shape in which the position and orientation (extension direction) of the lane markings are expected to change, and includes at least one of the following: branch roads, merging roads, increases or decreases in the number of lanes, increases or decreases in lane width, and intersections (including T-junctions, etc.). In the example in Figure 6, a branch road is shown as an example of a specific road shape. In a branch road, in addition to the lane markings that mark the driving lane L1, there are also lane markings that mark the branch road. In the example in Figure 6, in addition to map lane markings ML1 and ML2, map lane marking ML3, which is one of the lane markings that marks the branch road, is shown.

[0083] In the fourth scenario, the comparison unit 142, based on the recognition results of the first recognition unit 132 or the second recognition unit 134, if a specific road shape exists in the driving lane L1 in the direction of travel of the vehicle M, performs a comparison at a first distance D1 between the lane marking on the side where the specific road shape exists (left lane marking in the figure) and the lane marking on the opposite side (right lane marking in the figure). If the degree of deviation in the comparison result is greater than or equal to a threshold, a comparison of the left and right lane markings is performed at a second distance D2. In the example of Figure 6, a comparison of the lane markings at the second distance D2 is performed, and the degree of deviation of both the left and right lane markings is less than the threshold. In this case, the correction unit 146 corrects the map lane markings ML1 and ML2 based on the device lane markings CL1 and CL2 at the second distance D2. The driving control unit 144 then generates a target trajectory K1 based on the corrected map lane markings ML1 and ML2. As a result, for example, if the first driving mode is in operation, the first driving mode can be continued for at least the section up to the second distance D2.

[0084] As shown in the fourth scenario, if there are specific road shapes such as branches or merges near the first distance D1, the degree of discrepancy between the device lane markings CL and the map lane markings ML is likely to exceed a threshold. Therefore, in such cases, by comparing the lane markings in the section up to the second distance D2, if the degree of discrepancy is below the threshold, the driving control in that section can be maintained. Thus, the control described above can maintain the continuity of driving control in specific road shapes.

[0085] Furthermore, since the control in the first to fourth scenarios described above is repeatedly executed at a predetermined cycle, after the vehicle M has traveled to the second distance D2, comparisons based on the first distance D1 and the second distance D2, and control based on the comparison results, are performed based on that position.

[0086] Furthermore, the driving control performed in the first to fourth scenarios described above may include not only the continuation of the first driving mode and switching between the first and second driving modes, but also the control of starting and ending predetermined driving controls (e.g., LKAS, ACC, etc.). In addition, the driving control performed may be at least one of the steering control and speed control of the vehicle M, or it may be a display control in which the HMI control unit 180 displays information such as the vehicle M's driving lane and the lane markings that demarcate the driving lane (e.g., lane marking recognition results and comparison results) on the display unit of the HMI 30, or it may include both of these.

[0087] [Processing flow] The following describes the processes performed by the automatic driving control device 100 of this embodiment. The following description will focus primarily on the driving control processes based on the recognition results of lane markings, etc., among the processes performed by the automatic driving control device 100. The processes described below may be repeatedly executed at predetermined timings or predetermined cycles. Furthermore, the driving control processes in this embodiment will be described in several examples.

[0088] [Operation control process in the first embodiment] Figure 7 is a flowchart showing an example of the flow of the driving control process in the first embodiment. In the example in Figure 7, the first recognition unit 132 recognizes the surrounding conditions, including lane markings (device lane markings CL) present around the vehicle M, based on the output of the detection device DD which detects the surrounding conditions of the vehicle M (step S100). Next, the second recognition unit 134 refers to map information based on the location information of the vehicle M and recognizes lane markings (map lane markings ML) present around the vehicle M from the map information (step S110).

[0089] Next, the driving control unit 144 determines whether or not to start a predetermined driving control (step S120). The predetermined driving control is a driving control that controls the steering, speed, etc., of the vehicle M based on the recognized lane markings, and includes, for example, ACC and LKAS. The predetermined driving control may be started, for example, when an instruction to execute the predetermined driving control is received by the occupant of the vehicle M through operation of the HMI 30, or when the surrounding conditions recognized by the first recognition unit 132 satisfy the conditions for starting the predetermined driving control. If it is determined that the predetermined driving control should be started, the driving control unit 144 determines whether or not the hands-off conditions are met (step S130). The hands-off conditions are the conditions for executing driving control in a hands-off state (first driving mode), and for example, in the case of LKAS, this includes, but is not limited to, that at least one of the left and right lane markings is recognized, or that the contact margin time (TTC) with surrounding obstacles (other vehicles, etc.) is longer than a predetermined time. The contact margin time (TTC) can be derived, for example, by dividing the relative distance between the vehicle M and the obstacle by the relative velocity.

[0090] If it is determined that the hands-off condition is met, the driving control unit 144 starts a first driving mode in which the hands-off state is possible (step S140). In the process of step S140, for example, when LKAS is executed as the first driving mode, the driving control unit 144 generates a target trajectory for the vehicle M to pass through the center of the driving lane L1 which is demarcated by the recognized left and right device lane lines CL or map lane lines ML, and causes the second control unit 160 to perform steering control and speed control of the vehicle M so that the vehicle M travels along the generated target trajectory.

[0091] Next, the comparison unit 142 compares the lane markings (device lane marking CL and map lane marking ML) on both the left and right sides of the vehicle M at a first distance D1 (step S150) and determines whether the degree of deviation is below a threshold on only one side (step S160). If it is determined that the degree of deviation is below a threshold on only one side, the driving control unit 144 continues driving control in the first driving mode (hands-off state) (step S170). The comparison unit 142 also compares the lane markings at a second distance D2 which is shorter than the first distance D1 (closer to the vehicle M) (step S180) and determines whether the degree of deviation is below a threshold for the lane markings on both sides (step S190). If it is determined that the degree of deviation is below a threshold for the lane markings on both sides, the correction unit 146 corrects each of the left and right map lane markings ML to match the positions of the corresponding left and right device lane markings CL (step S200). Next, the driving control unit 144 continues the driving control in the first driving mode (hands-off state) based on the corrected map lane markings (step S210).

[0092] Furthermore, in the process of step S190, if it is determined that the degree of deviation between the lane lines on both sides is not below a threshold, the operation control in the first operating mode (hands-off state) is continued based on the device lane line CL, for example, without correcting the lane lines (step S210).

[0093] Furthermore, in the process of step S160, if it is determined that the degree of deviation is not below the threshold on only one side, the comparison unit 142 determines whether the degree of deviation is below the threshold at both lane lines at the first distance D1 (step S220). If it is determined that the degree of deviation is below the threshold at both lane lines, the processes of steps S200 and S210 are executed. If it is determined that the degree of deviation is not below the threshold at both lane lines, the driving control unit 144 performs control to switch from the first driving mode to the second driving mode, which is a hands-on state (step S230). This completes the process of this flowchart.

[0094] Furthermore, if it is determined in step S120 that the predetermined operation control will not be started, or if it is determined in step S130 that the hands-on conditions are not met, the process of this flowchart will terminate.

[0095] [Operation control processing in the second embodiment] Figure 8 is a flowchart showing an example of the operation control process flow in the second embodiment. The process shown in Figure 8 differs from the operation control process in the first embodiment shown in Figure 7, which consists of steps S100 to S230, in that it includes step S192 instead of step S190. Therefore, the following explanation will mainly focus on the process in step S192, and the other explanations will be omitted.

[0096] After the processing in step S180 in Figure 8, the comparison unit 142 determines whether the degree of deviation between the lane lines on both sides is less than a threshold (step S192). If it is determined that the degree of deviation is less than a threshold, it proceeds to the processing in steps S200 and S210. On the other hand, if it is determined that the degree of deviation is not less than a threshold, the driving control unit 144 proceeds to the processing in step S230 and performs control to switch to a second driving mode that is hands-on. The control according to the second embodiment enables safer driving control when the lane lines on both the left and right sides do not match at the second distance D2.

[0097] [Operation control processing in the third embodiment] Figure 9 is a flowchart showing an example of the operation control process in the third embodiment. The process shown in Figure 9 differs from the process from step S160 onwards in the operation control process of the first embodiment shown in Figure 7, which consists of steps S100 to S230. Therefore, the following explanation will mainly focus on the process from step S160 onwards, and other explanations will be omitted. Note that Figure 9 shows a simplified representation of the process from steps S100 to S150. Also, in the process from step S160 onwards, processes that are the same as in the first embodiment will be indicated by the same step number.

[0098] After the processing in step S150 shown in Figure 9, the comparison unit 142 determines whether the degree of deviation of the lane markings at the first distance D1 is below the threshold on only one side in the comparison result (step S160). If it is determined that the degree of deviation is below the threshold on only one side, the driving control unit 144 continues driving control in the first driving mode (hands-off state) (step S170). The comparison unit 142 also compares the lane markings at the second distance D2, which is shorter than the first distance D1 (closer to vehicle M) (step S180), and determines whether the degree of deviation is below the threshold on both sides of the lane markings (step S300). If it is determined that the degree of deviation is not below the threshold on both sides of the lane markings, the comparison unit 142 determines whether the degree of deviation is below the threshold on only one side at the second distance D2 (step S310). If it is determined that the degree of deviation on only one side is not below the threshold, then since neither side matches at the second distance D2, the driving control unit 144 switches from the first driving mode to the second driving mode and performs hands-on driving control (step S320).

[0099] Furthermore, in the process of step S310, if it is determined that the degree of deviation is less than the threshold on only one side of the second distance D2, the driving control unit 144 determines whether the lane marking on the side where the degree of deviation is less than the threshold at the second distance D2 is on the same side as the lane marking where the degree of deviation is less than the threshold at the first distance D1 (step S330). If it is determined that they are not on the same side, the correction unit 146 performs lane marking correction based on the position of each lane marking at the first distance D1 (step S340). Also, in the process of step S330, if it is determined that they are on the same side, or in the process of step S300, if it is determined that the degree of deviation is less than the threshold at both lane markings, the correction unit 146 performs lane marking correction based on the position of each lane marking at the second distance D2 (step S350). Then, after the process of step S340 or step S350, the driving control of the first driving mode (hands-off state) is continued (step S360). This concludes this flowchart.

[0100] Furthermore, if it is determined in step S160 that only one side does not match, the comparison unit 142 determines whether the degree of deviation between the boundary lines on both sides at the first distance D1 is less than a threshold (step S370). If it is determined that the degree of deviation between the boundary lines on both sides is less than a threshold, the processing from step S340 onward is performed. If it is determined that the degree of deviation between the boundary lines on both sides is not less than a threshold, the processing in step S320 is executed.

[0101] According to the third embodiment, based on the comparison results of the lane markings at the first distance D1 and the second distance D2, the content of the correction control can be made different depending on whether the lane marking whose deviation is below a threshold is on the left or right side. Therefore, lane markings can be corrected more appropriately, and driving control can be realized using the corrected lane markings.

[0102] According to the above-described embodiment, in the automatic driving control device 100 (an example of a vehicle control device), a first recognition unit 132 recognizes the left and right lane markings as seen from vehicle M that demarcate the vehicle M's driving lane as device lane markings based on the output of a detection device DD that detects the surrounding conditions of vehicle M, a second recognition unit 134 recognizes the lane markings that demarcate the vehicle's driving lane as map lane markings based on the vehicle M's position information from map information, and a left lane marking comparison that compares the device lane marking the left side of the driving lane with the map lane markings at a first distance in the direction of travel as seen from vehicle M, and the device lane markings that demarcate the right side of the driving lane The system includes a comparison unit 142 that compares the right-hand lane markings with the map lane markings, and if the results of the comparison between the left-hand lane markings and the right-hand lane markings satisfy specific conditions, it performs a comparison between the left-hand lane markings and the right-hand lane markings at a second distance shorter than the first distance, and a driving control unit (an example of a driving control unit) that performs driving control of the vehicle based on the comparison results by the comparison unit 142, wherein the specific conditions include the fact that at the first distance, the degree of deviation of either the left-hand lane marking comparison or the right-hand lane marking comparison is less than a threshold, thereby enabling more appropriate driving control based on the road marking recognition results. This can ultimately contribute to the development of a sustainable transportation system.

[0103] For example, according to the embodiment, in a comparison (deviation determination) of the left and right device lane markings and map lane markings at a long distance (first distance) from the vehicle M, if the degree of deviation of either one exceeds a threshold, a comparison is made between the left and right device lane markings and map lane markings at a short distance (second distance), and the map lane markings are corrected based on the result, thereby enabling more appropriate recognition of lane markings and driving lanes. Therefore, the accuracy of estimating the vehicle's position can be improved. Furthermore, according to the embodiment, during the execution of the first driving mode (hands-off state), if the degree of deviation of either the left and right device lane markings and map lane markings at a long distance (first distance) exceeds a threshold, the first driving mode is continued, thereby improving the continuity (stability) of driving control.

[0104] The embodiments described above can be expressed as follows. A storage medium that stores computer-readable instructions, A processor connected to the storage medium, The processor executes the computer-readable instructions to: Based on the output of a detection device that detects the surrounding conditions of the vehicle, the left and right lane markings that demarcate the vehicle's driving lane, as seen from the vehicle's perspective, are recognized as device lane markings. Based on the vehicle's location information, the lane markings that demarcate the vehicle's driving lane are recognized from the map information as map lane markings. At a first distance in the direction of travel as viewed from the vehicle, a comparison of the left-side lane markings between the device markings that demarcate the left side of the driving lane and the map markings is performed, and a comparison of the right-side lane markings between the device markings that demarcate the right side of the driving lane and the map markings is performed. If the comparison of the left-hand lane lines and the right-hand lane lines satisfies certain conditions, the comparison of the left-hand lane lines and the right-hand lane lines is performed at a second distance shorter than the first distance. Based on the comparison results, the operation control of the vehicle is performed. The aforementioned specific condition includes, at the first distance, that the degree of deviation of either the left-side lane comparison or the right-side lane comparison is less than a threshold. Vehicle control system.

[0105] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of symbols]

[0106] 1...Vehicle system, 10...Camera, 12...Radar device, 14...LIDAR, 16...Object recognition device, 20...Communication device, 30...HMI, 40...Vehicle sensor, 50...Navigation device, 60...MPU, 80...Driver control unit, 100...Automatic driving control device, 120...First control unit, 130...Recognition unit, 132...First recognition unit, 134...Second recognition unit, 140...Action plan generation unit, 142...Comparison unit, 144...Driving control unit, 146...Correction unit, 160...Second control unit, 162...Target trajectory acquisition unit, 164...Speed ​​control unit, 166...Steering control unit, 180...HMI control unit, 190...Storage unit, 200...Driving force output device, 210...Brake device, 220...Steering device, M...Vehicle

Claims

1. A first recognition unit recognizes the left and right lane markings as seen from the vehicle, which demarcate the vehicle's driving lane, as device lane markings, based on the output of a detection device that detects the surrounding conditions of the vehicle. A second recognition unit recognizes the lane markings that demarcate the vehicle's driving lane from map information based on the vehicle's location information, A comparison unit performs a comparison of the left lane markings, comparing the device markings that demarcate the left side of the driving lane with the map markings, at a first distance in the direction of travel as viewed from the vehicle, and a comparison of the right lane markings, comparing the device markings that demarcate the right side of the driving lane with the map markings. If the results of the comparison of the left lane markings and the comparison of the right lane markings satisfy specific conditions, the comparison unit performs the comparison of the left lane markings and the comparison of the right lane markings at a second distance shorter than the first distance. The system includes an operation control unit that controls the vehicle based on the comparison results from the comparison unit, The aforementioned specific condition includes, at the first distance, that the degree of deviation of either the left-side lane line comparison or the right-side lane line comparison is less than a threshold. Vehicle control device.

2. The control of the vehicle includes driving control in a first driving mode and a second driving mode in which the degree of driving assistance is lower than in the first driving mode or the task of the vehicle occupant is greater. The operation control unit continues the first operation mode if it is executing the first operation mode and the degree of deviation of either the left lane markings or the right lane markings at the first distance is less than a threshold. The vehicle control device according to claim 1.

3. The system further includes a correction unit that corrects the map division lines based on the device division lines, The correction unit corrects the map grid lines based on the device grid lines at the second distance if the degree of deviation in both the comparison of the left grid lines and the right grid lines at the second distance is less than a threshold. The operation control unit controls the vehicle based on the map lane lines corrected by the correction unit. The vehicle control device according to claim 2.

4. The correction unit, if the degree of deviation of at least one of the left lane markings and the right lane markings at the second distance is greater than or equal to a threshold, corrects the map markings for the lane marking on the side of the driving lane where the degree of deviation at the first distance is less than the threshold, based on the device markings at the first distance. The operation control unit controls the vehicle based on the map lane lines corrected by the correction unit. The vehicle control device according to claim 3.

5. The operation control unit changes the control of the vehicle from the first operation mode to the second operation mode if the first operation mode is being executed and the degree of deviation of at least one of the left lane markings and the right lane markings in the second distance is greater than or equal to a threshold. The vehicle control device according to claim 2.

6. The system further includes a correction unit that corrects the map division lines based on the device division lines, The correction unit, if the degree of deviation of only one of the left and right section lines in the comparison of the left section line and the right section line at the second distance is less than the threshold, corrects the map section line on the section line on the side where the degree of deviation at the first distance was less than the threshold, based on the device section line at the second distance. The operation control unit controls the vehicle based on the map lane lines corrected by the correction unit. The vehicle control device according to claim 2.

7. The correction unit corrects the map boundary lines based on the device boundary lines at the second distance when the boundary line whose deviation at the second distance is less than the threshold and the boundary line whose deviation at the first distance is less than the threshold are on the same side of the left and right boundary lines. The vehicle control device according to claim 6.

8. The aforementioned driving control unit performs the control when a specific shape exists in the driving lane in the direction of travel of the vehicle. The aforementioned specific shape includes at least one of the following: a branching road, a merging road, an increase or decrease in the number of lanes, an increase or decrease in lane width, and an intersection. The vehicle control device according to claim 1.

9. The operation control unit performs the comparison at the second distance if, based on the comparison result by the comparison unit at the first distance, the degree of deviation of the left and right partition lines on the side opposite to the side where the specific shape exists is greater than or equal to a threshold. The vehicle control device according to claim 8.

10. The control of the vehicle includes at least one of the following: steering control of the vehicle, speed control, and display control relating to the driving lane or the lane markings that demarcate the driving lane. The vehicle control device according to claim 1.

11. Computers Based on the output of a detection device that detects the surrounding conditions of the vehicle, the left and right lane markings that demarcate the vehicle's driving lane, as seen from the vehicle's perspective, are recognized as device lane markings. Based on the vehicle's location information, the lane markings that demarcate the vehicle's driving lane are recognized from the map information as map lane markings. At a first distance in the direction of travel as viewed from the vehicle, a comparison of the left-side lane markings between the device markings that demarcate the left side of the driving lane and the map markings is performed, and a comparison of the right-side lane markings between the device markings that demarcate the right side of the driving lane and the map markings is performed. If the comparison of the left-hand lane line and the right-hand lane line satisfies specific conditions, the comparison of the left-hand lane line and the right-hand lane line is performed at a second distance shorter than the first distance. Based on the comparison results, the vehicle is controlled. The aforementioned specific condition includes, at the first distance, that the degree of deviation of either the left-side lane line comparison or the right-side lane line comparison is less than a threshold. Vehicle control method.

12. On the computer, Based on the output of a detection device that detects the surrounding conditions of the vehicle, the device recognizes the left and right lane markings that demarcate the vehicle's driving lane as seen from the vehicle's perspective as device lane markings. Based on the vehicle's location information, the lane markings that demarcate the vehicle's driving lane are recognized as map lane markings from the map information. At a first distance in the direction of travel as viewed from the vehicle, a comparison of the left-side lane markings between the device markings that demarcate the left side of the driving lane and the map markings is performed, and a comparison of the right-side lane markings between the device markings that demarcate the right side of the driving lane and the map markings is performed. If the results of the comparison of the left-side lane lines and the right-side lane lines satisfy specific conditions, the comparison of the left-side lane lines and the right-side lane lines is performed at a second distance shorter than the first distance. Based on the comparison results, the vehicle is controlled. The aforementioned specific condition includes, at the first distance, that the degree of deviation of either the left-side lane line comparison or the right-side lane line comparison is less than a threshold. program.

Images