Driving control device, driving control method, and program

Abstract

To perform more appropriate driving control. [Solution] The driving control device of the embodiment includes: a first recognition unit that recognizes the surrounding conditions including a first lane marking that demarcates the vehicle's driving lane based on the output of a detection device that detects the surrounding conditions of the vehicle; a second recognition unit that recognizes a second lane marking that demarcates the lanes around the vehicle from map information; a determination unit that determines whether the first lane marking and the second lane marking coincide; a selection unit that selects either the first lane marking or the second lane marking if it is determined that they do not coincide; and a driving control unit that executes driving control of the vehicle based on the selected lane marking. The selection unit compares the degree of deviation between the center line of the lane demarcated by the first lane marking on either side of the vehicle and the center line of the lane demarcated by the second lane marking on either side of the vehicle if either of the second lane markings on either side of the vehicle is interrupted in the direction of travel of the vehicle, and selects the first lane marking if the degree of deviation is greater than or equal to a threshold.

Description

Technical Field

[0001] The present invention relates to a travel control device, a travel control method, and a program. Background Art

[0002] In recent years, initiatives to provide access to sustainable transport systems that also take into consideration vulnerable road users among traffic participants have become active. To achieve this goal, efforts are focused on research and development that further improves traffic safety and convenience through research and development related to autonomous driving technology. In relation to this, there is conventionally known a technology in which the lateral position of the own vehicle is determined using at least one of a lateral position of the vehicle based on satellite information and map information, and a lateral position of the vehicle based on camera information, and a target steering angle is calculated based on a curvature of a driving lane at the position of the own vehicle and the determined lateral position of the own vehicle (see, for example, Patent Document 1). Prior Art Literature Patent Literature

[0003] Patent Document 1 Japanese Patent No. 6415629 Summary of the Invention Problems to be Solved by the Invention

[0004] Incidentally, in conventional autonomous driving technologies, since various road conditions exist in actual environments, there has still been room for study on travel control in a case where one of a lane marking obtained from map information and a lane marking obtained from an output of a detection device such as a camera becomes unrecognizable partway through. Therefore, there has been a problem that appropriate travel control may not be executed depending on road conditions.

[0005] One of the objectives of this application is to provide a vehicle control device, a vehicle control method, and a program that can perform more appropriate vehicle control based on road conditions, in order to solve the above-mentioned problems. Ultimately, this will contribute to the development of a sustainable transportation system. [Means for solving the problem]

[0006] The travel control device, travel control method, and program according to this invention employ the following configuration. (1) A driving control device according to one aspect of the present invention includes: a first recognition unit that recognizes the surrounding conditions including a first lane marking that demarcates the driving lane of the vehicle based on the output of a detection device that detects the surrounding conditions of the vehicle; a second recognition unit that recognizes a second lane marking that demarcates the lanes around the vehicle from map information based on the vehicle's position information; a determination unit that determines whether the first lane marking and the second lane marking coincide; and, if the determination unit determines that the first lane marking and the second lane marking do not coincide, the device determines that either the first lane marking or the second lane marking is The driving control device comprises a selection unit and a driving control unit that performs driving control of the vehicle based on the lane markings selected by the selection unit, wherein the selection unit compares the degree of deviation between the center line of the lane marked by the first lane markings on the left and right of the vehicle and the center line of the lane marked by the second lane markings on the left and right of the vehicle, when either of the second lane markings that mark the lane markings of the vehicle that mark the lane markings of the vehicle is interrupted in the direction of travel of the vehicle, and selects the first lane marking if the degree of deviation is greater than or equal to a threshold.

[0007] (2) In the embodiment of (1) above, the map information stores information about the center line of the lane demarcated by the second lane markings on both sides of the vehicle, even beyond the point where one of the second lane markings on both sides of the vehicle is interrupted.

[0008] (3) In the embodiment of (1) above, the vehicle's driving lane is the main lane in the map information, and it continues beyond the point where either of the second lane markings on the left or right of the vehicle is interrupted in the direction of travel of the vehicle.

[0009] (4): In the embodiment of (1) above, the vehicle's driving lane is a lane in which the map information does not include information regarding branching lanes, merging lanes, or lane changes.

[0010] (5) In the embodiment of (1) above, the selection unit compares the degree of deviation between the center line of the lane demarcated by the first lane markings located on the left and right sides of the vehicle and the center line of the lane demarcated by the second lane markings located on the left and right sides of the vehicle, within the detection range of the detection device by the first recognition unit.

[0011] (6) In the embodiment of (1) above, the selection unit compares the degree of deviation between the center lines of the lanes defined by the first lane markings that are on the left and right of the vehicle and the center lines of the lanes defined by the second lane markings that are on the left and right of the vehicle, within the range in which the center lines of the lanes defined by the second lane markings are not interrupted.

[0012] (7): In the embodiment of (1) above, the selection unit compares the degree of deviation between the center line of the lane demarcated by the first lane markings on the left and right of the vehicle and the center line of the lane demarcated by the second lane markings on the left and right of the vehicle, when the driving lane has connection points with a merging lane and connection points with a branching lane.

[0013] (8) In the embodiment of (7) above, the selection unit compares the degree of deviation between the center line of the lane demarcated by the first lane markings on the left and right of the vehicle and the center line of the lane demarcated by the second lane markings on the left and right of the vehicle when a connection point with a merging lane exists in the direction of travel of the vehicle in the driving lane and a connection point with a branching lane exists within a predetermined distance ahead of the connection point as seen from the vehicle.

[0014] (9): A driving control method according to one aspect of the present invention is a driving control method in which a computer recognizes the surrounding conditions including a first lane marking that demarcates the driving lane of the vehicle based on the output of a detection device that detects the surrounding conditions of the vehicle, recognizes a second lane marking that demarcates the lanes around the vehicle from map information based on the vehicle's position information, determines whether the first lane marking and the second lane marking coincide, and if it is determined that the first lane marking and the second lane marking do not coincide, selects one of the first lane marking and the second lane marking, executes driving control of the vehicle based on the selected lane marking, and if either of the second lane markings that demarcate the driving lane of the vehicle and are located on the left and right of the vehicle is interrupted in the direction of travel of the vehicle, compares the degree of deviation between the center line of the lane demarcated by the first lane marking located on the left and right of the vehicle and the center line of the lane demarcated by the second lane marking located on the left and right of the vehicle, and selects the first lane marking if the degree of deviation is greater than or equal to a threshold.

[0015] (10): A program according to one aspect of the present invention causes a computer to recognize the surrounding conditions, including a first lane marking that demarcates the vehicle's driving lane, based on the output of a detection device that detects the surrounding conditions of a vehicle; to recognize a second lane marking that demarcates the lanes around the vehicle from map information based on the vehicle's position information; to determine whether the first lane marking and the second lane marking coincide; if it is determined that the first lane marking and the second lane marking do not coincide, to select one of the first lane marking and the second lane marking; to execute driving control of the vehicle based on the selected lane marking; if either of the second lane markings that demarcate the vehicle's driving lane and are located to the left and right of the vehicle is interrupted in the direction of travel of the vehicle, to compare the degree of deviation between the center line of the lane demarcated by the first lane marking located to the left and right of the vehicle and the center line of the lane demarcated by the second lane marking located to the left and right of the vehicle; and if the degree of deviation is greater than or equal to a threshold, to select the first lane marking. [Effects of the Invention]

[0016] According to the embodiments described in (1) to (10) above, more appropriate driving control can be performed based on road conditions. [Brief explanation of the drawing]

[0017] [Figure 1] This is a configuration diagram of a vehicle system 1 including a driving control device according to an embodiment. [Figure 2] This is a functional configuration diagram of the first control unit 120 and the second control unit 160. [Figure 3] This is a diagram illustrating the driving control of vehicle M in the embodiment. [Figure 4] This figure shows an example of a scenario in which either the device boundary line CL or the map boundary line ML is selected by the selection unit 144. [Figure 5] This flowchart shows an example of the processing flow performed by the automatic driving control device 100. [Modes for carrying out the invention]

[0018] The following describes embodiments of the driving control device, driving control method, and program of the present invention with reference to the drawings. As an example, an embodiment in which the driving control device is applied to an autonomous vehicle will be described. Autonomous driving refers to performing driving control by automatically controlling, for example, the steering or speed of the vehicle, or both. The aforementioned driving control may include, for example, ACC (Adaptive Cruise Control System), TJP (Traffic Jam Pilot), LKAS (Lane Keeping Assistance System), ALC (Automated Lane Change), and CMBS (Collision Mitigation Brake System). Furthermore, autonomous vehicles may also perform driving control by manual operation by the vehicle user (e.g., occupants) (so-called manual driving). Also, the following describes the case where left-hand traffic regulations apply; however, if right-hand traffic regulations apply, simply reverse the left and right directions.

[0019] [Overall Configuration] Fig. 1 is a configuration diagram of a vehicle system 1 including a travel control device according to an embodiment. A vehicle on which the vehicle system 1 is mounted (hereinafter referred to as vehicle M) is, for example, a two-wheeled, three-wheeled, four-wheeled or other vehicle, and the driving source thereof is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination of the foregoing. The electric motor operates using electric power generated by a generator connected to the internal combustion engine, or discharge power from a battery (storage battery) such as a secondary battery or a fuel cell.

[0020] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driving operator 80, an automatic driving control device 100, a travel driving force output device 200, a brake device 210, and a steering device 220. These devices and instruments are connected to each other via a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. Note that the configuration shown in Fig. 1 is merely an example, a part of the configuration may be omitted, or another configuration may be additionally added. A combination of the camera 10, the radar device 12, the LIDAR 14, and the object recognition device 16 is an example of the "detection device DD". The HMI 30 is an example of an "output device". The automatic driving control device 100 is an example of a "travel control device".

[0021] The camera 10 is, for example, a digital camera using a solid-state image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to any location of a vehicle M on which the vehicle system 1 is mounted. When imaging the front of the vehicle, the camera 10 is attached to the upper part of the front windshield, the back surface of the room mirror, the front head of the vehicle body, or the like. When imaging the rear of the vehicle, the camera 10 is attached to the upper part of the rear windshield, the back door, or the like. When imaging the side of the vehicle, the camera 10 is attached to a door mirror or the like. For example, the camera 10 periodically and repeatedly images the surroundings of the vehicle M. The camera 10 may be a stereo camera.

[0022] The radar device 12 radiates radio waves such as millimeter waves to the surroundings of the vehicle M, detects radio waves (reflected waves) reflected by surrounding objects, and detects at least the position (distance and orientation) of the object. The radar device 12 is attached to any location of the vehicle M. The radar device 12 may detect the position and velocity of an object by the FM-CW (Frequency Modulated Continuous Wave) method.

[0023] The LIDAR 14 irradiates light to the surroundings of the vehicle M and measures scattered light. The LIDAR 14 detects the distance to a target based on the time from light emission to light reception. The irradiated light is, for example, pulsed laser light. The LIDAR 14 is attached to any location of the vehicle M.

[0024] The object recognition device 16 performs sensor fusion processing on detection results obtained by some or all of the camera 10, the radar device 12, and the LIDAR 14, and recognizes the position, type, velocity, and the like of an object. The object recognition device 16 outputs the recognition result to an automatic driving control device 100. In addition, the object recognition device 16 may output the detection results of the camera 10, the radar device 12, and the LIDAR 14 to the automatic driving control device 100 as they are. In this case, the object recognition device 16 may be omitted from the configuration of the vehicle system 1 (detection device DD).

[0025] The communication device 20 communicates with other vehicles in the vicinity of vehicle M, terminal devices of users using vehicle M, or various server devices, for example, by utilizing networks such as cellular networks, Wi-Fi networks, Bluetooth®, DSRC (Dedicated Short Range Communication), LAN (Local Area Network), WAN (Wide Area Network), and the Internet.

[0026] The HMI30 outputs various information to the occupants of vehicle M and accepts input operations from the occupants. The HMI30 includes, for example, various display devices, speakers, buzzers, touch panels, switches, keys, microphones, etc.

[0027] The vehicle sensor 40 includes a vehicle speed sensor for detecting the speed of the vehicle M, an acceleration sensor for detecting acceleration, a yaw rate sensor for detecting yaw rate (for example, the angular velocity of rotation around the vertical axis passing through the center of gravity of the vehicle M), and an orientation sensor for detecting the orientation of the vehicle M. The vehicle sensor 40 may also be provided with a position sensor for detecting the position of the vehicle. The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. Alternatively, the position sensor may be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver 51 of the navigation device 50. The vehicle sensor 40 may derive the speed of the vehicle M from the difference (i.e., distance) of position information at a predetermined time from the position sensor. The results detected by the vehicle sensor 40 are output to the automatic driving control device 100.

[0028] The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as an HDD (Hard Disk Drive) or flash memory. The GNSS receiver 51 determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be determined or supplemented by an INS (Inertial Navigation System) that utilizes the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, speaker, touch panel, keys, etc. The GNSS receiver 51 may be provided on the vehicle sensor 40. The navigation HMI 52 may be partially or completely shared with the HMI 30 described above. The route determination unit 53 determines, for example, a route (hereinafter referred to as a route on the map) from the position of the vehicle M determined by the GNSS receiver 51 (or any input position) to the destination input by the occupant using the navigation HMI 52, by referring to the first map information 54. The first map information 54 is, for example, information in which the shape of a road is represented by links indicating roads and nodes connected by those links. The first map information 54 may also include POI (Point of Interest) information, etc. The route on the map is output to the MPU 60. The navigation device 50 may perform route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may transmit its current location and destination to the navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server. The navigation device 50 outputs the determined route on the map to the MPU 60.

[0029] The MPU 60 includes, for example, a recommended lane determination unit 61 and stores second map information 62 in a storage device such as an HDD or flash memory. The recommended lane determination unit 61 divides the map route provided by the navigation device 50 into multiple blocks (for example, every 100m with respect to the vehicle's direction of travel) and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 makes decisions such as which lane from the left the vehicle should travel in. If there is a branching point in the map route, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel along a reasonable route to proceed to the branching point.

[0030] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, the number of lanes, the type and shape of road markings (hereinafter referred to as markings), information on the center of the lanes, or information on road boundaries. The second map information 62 may also include information on whether the road boundary includes a structure that a vehicle M cannot pass through (including crossing or contacting). Structures include, for example, guardrails, curbs, median strips, fences, etc. "Not passable" may include the existence of a low step that can be passed through if the vehicle is willing to tolerate vibrations that would not normally occur. The second map information 62 may also include road shape information, traffic regulation information, address information (address and postal code), facility information, parking information, telephone number information, etc. Road shape information includes, for example, the curvature of the road (which may be rephrased as radius of curvature; the same applies hereinafter), the amount of change in curvature at predetermined distances, width, gradient, etc.

[0031] Furthermore, the second map information 62 may include information about the center line of the lane. The center line is the center line between the left and right lane markings. This center line may exist in lanes where either of the left or right lane markings is interrupted. In addition, the second map information 62 may also store information about the center line of the lanes demarcated by the lane markings, even beyond the point where the lane markings are interrupted (the section beyond the point where they are interrupted). Furthermore, the second map information 62 may also store information about road branching points, merging points, lane additions and subtractions, etc. At branching points, it may include information about branching lanes and the main road, and at merging points, it may include information about merging lanes and the main road.

[0032] The second map information 62 may be updated as needed by the communication device 20 communicating with an external device. The first map information 54 and the second map information 62 may be provided together as map information. The map information may also be stored in the storage unit 190.

[0033] The driver control unit 80 includes, for example, a steering wheel, an accelerator pedal, and a brake pedal. The driver control unit 80 may also include a shift lever, a modified steering wheel, a joystick, or other controls. Each control of the driver control unit 80 is equipped with an operation detection unit that detects, for example, the amount of operation performed by the occupant on the control unit or whether or not an operation has been performed. The operation detection unit detects, for example, the steering angle and steering torque of the steering wheel, the amount of depression of the accelerator pedal and brake pedal, etc. The operation detection unit then outputs the detection result to the automatic driving control device 100, or to one or both of the driving force output device 200, the brake device 210, and the steering device 220.

[0034] The automatic driving control device 100 performs various driving controls belonging to automatic driving on the vehicle M. The automatic driving control device 100 includes, for example, a first control unit 120, a second control unit 160, an HMI control unit 180, and a storage unit 190. The first control unit 120, the second control unit 160, and the HMI control unit 180 are each realized by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Furthermore, some or all of these components may be realized by hardware (including circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), and SOC (System On Chip), or by the cooperation of software and hardware. The above-mentioned program may be stored in advance in a storage device (a storage device equipped with a non-transient storage medium) such as the HDD or flash memory of the automatic driving control device 100, or it may be stored in a removable storage medium such as a DVD, CD-ROM, or memory card, and installed in the storage device of the automatic driving control device 100 when the storage medium (non-transient storage medium) is inserted into a drive device or card slot.

[0035] The storage unit 190 may be implemented using the various storage devices described above, or an EEPROM (Electrically Erasable Programmable Read Only Memory), ROM (Read Only Memory), or RAM (Random Access Memory), etc. The storage unit 190 may store, for example, various information and programs as described in the embodiment. The storage unit 190 may also store map information (for example, first map information 54 and second map information 62).

[0036] Figure 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 140. The first control unit 120 implements, for example, functions using AI (Artificial Intelligence) and functions using a pre-defined model in parallel. For example, the function of "recognizing intersections" may be implemented by simultaneously performing intersection recognition using deep learning and recognition based on pre-defined conditions (such as pattern-matchable signals and road markings), scoring both, and comprehensively evaluating them. This ensures the reliability of autonomous driving. The first control unit 120 also performs control related to the autonomous driving of the vehicle M based on instructions from, for example, the MPU 60 and the HMI control unit 180.

[0037] The recognition unit 130 recognizes the surrounding conditions of vehicle M based on the recognition results of the detection device DD (information input from camera 10, radar device 12, and LIDAR 14 via object recognition device 16). For example, the recognition unit 130 recognizes the position, speed, acceleration, and other states of objects present around vehicle M (within a predetermined distance). These objects include, for example, other vehicles (surrounding vehicles), traffic participants (pedestrians, cyclists, etc.) using the road, road structures, and other obstacles present in the surrounding area. Road structures include, for example, road signs, traffic lights, level crossings, curbs, median strips, guardrails, fences, etc. The position of an object is recognized as a position on an absolute coordinate system with a representative point of vehicle M (such as the center of gravity or the center of the drive axis) as the origin, and is used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or it may be represented by a represented area. The "state" of an object may include, for example, the acceleration or jerk of a moving object such as another vehicle, or its "action state" (for example, whether or not the other vehicle is changing lanes or is about to change lanes).

[0038] Furthermore, the recognition unit 130 includes, for example, a first recognition unit 132 and a second recognition unit 134. Details of these functions will be described later.

[0039] The action plan generation unit 140 generates an action plan for driving vehicle M by autonomous driving based on the recognition results of the recognition unit 130, etc. For example, the action plan generation unit 140, in principle, drives in the recommended lane determined by the recommended lane determination unit 61, and further generates a target trajectory for vehicle M to travel automatically (without driver operation) in the future, based on the recognition results of the recognition unit 130 and the surrounding road shape based on the current position of vehicle M obtained from map information, etc., so as to respond to the surrounding conditions of vehicle M. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequence of points (trajectory points) that vehicle M should reach. The trajectory points are points that vehicle M should reach at predetermined driving distances (e.g., a few meters) along the road, and separately, target speed and target acceleration at predetermined sampling times (e.g., a few tenths of a second) are generated as part of the target trajectory. Alternatively, the trajectory points may be the positions that vehicle M should reach at the sampling time for each predetermined sampling time. In this case, information about the target velocity and target acceleration is represented by the interval between trajectory points.

[0040] The action plan generation unit 140 may set automated driving events when generating a target trajectory. Events may include, for example, a constant speed driving event in which vehicle M drives in the same lane at a constant speed, a follow driving event in which vehicle M follows the nearest vehicle that is within a predetermined distance (for example, within 100 [m]) in front of vehicle M, a lane change event in which vehicle M changes lanes from its own lane to an adjacent lane, a branching event in which vehicle M branches off to the destination lane at a road branching point, a merging event in which vehicle M merges onto the main road at a merging point, and a takeover event to end automated driving and switch to manual driving. In addition, events may include, for example, an overtaking event in which vehicle M changes lanes to an adjacent lane, overtakes a preceding vehicle in the adjacent lane, and then changes lanes back to the original lane, and an avoidance event in which vehicle M brakes and / or steers to avoid an obstacle in front of vehicle M.

[0041] Furthermore, the action plan generation unit 140 may, for example, change an event already determined for the current section to another event, or set a new event for the current section, depending on the surrounding conditions of vehicle M recognized while vehicle M is in motion. Also, the action plan generation unit 140 may change an event already set for the current section to another event, or set a new event for the current section, depending on the occupant's operation on the HMI 30. The action plan generation unit 140 generates a target trajectory according to the set event.

[0042] Furthermore, the action plan generation unit 140 includes, for example, a determination unit 142, a selection unit 144, and an execution control unit 146. The execution control unit 146 and the second control unit 160 are examples of a "driving control unit." Details of these functions will be described later.

[0043] The second control unit 160 controls the driving force output device 200, the braking device 210, and the steering device 220 so that the vehicle M passes through the target trajectory generated by the action plan generation unit 140 at the scheduled time.

[0044] The second control unit 160 includes, for example, a target trajectory acquisition unit 162, a speed control unit 164, and a steering control unit 166. The target trajectory acquisition unit 162 acquires information on the target trajectory (trajectory points) generated by the action plan generation unit 140 and stores it in memory (not shown). The speed control unit 164 controls the driving force output device 200 or the brake device 210 based on the speed elements associated with the target trajectory stored in memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in memory. The processing of the speed control unit 164 and the steering control unit 166 is realized, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 performs a combination of feedforward control according to the curvature of the road in front of the vehicle M and feedback control based on the deviation from the target trajectory.

[0045] Returning to Figure 1, the HMI control unit 180 notifies the occupant of predetermined information via the HMI 30. The predetermined information includes, for example, information related to the driving of vehicle M, such as information regarding the status of vehicle M and information regarding driving control. Information regarding the status of vehicle M includes, for example, the speed of vehicle M, engine speed, and shift position. Information regarding driving control includes, for example, information inquiring whether or not to perform driving control by automatic driving, whether or not to start automatic driving, information regarding the status of driving control by automatic driving, information regarding the automation level, and information prompting the occupant to take action when switching from automatic driving to manual driving. The predetermined information may also include information unrelated to the driving of vehicle M, such as content stored on a storage medium such as a television program or DVD (for example, a movie). The predetermined information may also include, for example, the current location and destination in automatic driving, and information regarding the remaining fuel level of vehicle M. The HMI control unit 180 may output the information received by the HMI 30 to the communication device 20, the navigation device 50, the first control unit 120, etc.

[0046] Furthermore, the HMI control unit 180 may output to the HMI 30 information such as inquiry information for the occupants and processing results from the first control unit 120 and the second control unit 160. In addition, the HMI control unit 180 may transmit various information to be output to the HMI 30 to a terminal device used by a user of the vehicle M via the communication device 20.

[0047] The driving force output device 200 outputs driving force (torque) to the drive wheels for the vehicle to move. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, and an ECU (Electronic Control Unit) that controls them. The ECU controls the above configuration according to information input from the second control unit 160 or information input from the accelerator pedal of the driver control unit 80.

[0048] The brake system 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or from the brake pedal of the driver control unit 80, so that brake torque corresponding to the braking operation is output to each wheel. The brake system 210 may be equipped with a backup mechanism that transmits the hydraulic pressure generated by the operation of the brake pedal to the cylinder via a master cylinder. The brake system 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic brake system that controls an actuator according to information input from the second control unit 160 to transmit hydraulic pressure from the master cylinder to the cylinder.

[0049] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies force to a rack and pinion mechanism to change the direction of the steering wheels. The steering ECU drives the electric motor to change the direction of the steering wheels according to information input from the second control unit 160 or information input from the steering wheel of the driver control unit 80.

[0050] [Recognition unit and action plan generation unit] Next, we will explain in detail the functions of the recognition unit 130 (first recognition unit 132, second recognition unit 134) and the action plan generation unit 140 (determination unit 142, selection unit 144, execution control unit 146).

[0051] Figure 3 is a diagram illustrating the driving control of vehicle M in the embodiment. In the example of Figure 3, device lane markings CL1 and CL2 recognized by the detection device DD, map lane markings ML1 and ML2 obtained from map information (e.g., second map information 62) based on the position information of vehicle M, and actual lane markings RL1 and RL2 drawn on the road (runway) are shown. For example, lane L1 is demarcated by actual lane markings RL1 and RL2. Lane L1 shown in Figure 3 is a lane that can be traveled in the extending direction (X-axis direction in the figure). Hereinafter, when device lane markings CL1 and CL2 are not distinguished, they will simply be referred to as "device lane marking CL", when map lane markings ML1 and ML2 are not distinguished, they will simply be referred to as "map lane marking ML", and when actual lane markings RL1 and RL2 are not distinguished, they will simply be referred to as "actual lane marking RL". Also, in the scene shown in Figure 3, vehicle M is traveling on lane L1 at speed VM. Additionally, the lane L1 in which vehicle M travels may be referred to as "driving lane L1" as needed. Device lane markings CL are an example of "first lane markings," and map lane markings ML are an example of "second lane markings."

[0052] In the example shown in Figure 3, the first recognition unit 132 recognizes the surrounding conditions (external environment) of vehicle M based on the output of the detection device DD that detects the surrounding conditions (external environment) of vehicle M. For example, the first recognition unit 132 recognizes the left and right lane markings (left and right lane markings as seen from vehicle M) that demarcate the vehicle M's driving lane L1 as device lane markings CL1 and CL2, based on the image captured by the camera 10 (hereinafter referred to as the camera image). In the example shown in Figure 3, the device lane markings CL1 and CL2 in front of vehicle M are shown, but the device lane markings CL1 and CL2 on the sides and rear of vehicle M may also be recognized.

[0053] For example, the first recognition unit 132 analyzes the camera image, extracts edge points with large brightness differences from adjacent pixels in the image, and connects these edge points to recognize device demarcation lines CL1 and CL2 in the image plane. The first recognition unit 132 also converts the positions of the device demarcation lines CL1 and CL2, based on the position of a representative point of the vehicle M, into the vehicle coordinate system (for example, the XY plane coordinates in Figure 3).

[0054] Furthermore, the first recognition unit 132 may recognize, for example, the curvature of the device lane markings CL1 and CL2. The first recognition unit 132 may also recognize the amount of change in curvature of the device lane markings CL1 and CL2. The first recognition unit 132 may also recognize the curvature or amount of change in curvature of the lane demarcated by the device lane markings CL1 and CL2 by averaging the respective curvature or amount of change in curvature of the device lane markings CL1 and CL2. Furthermore, the first recognition unit 132 may recognize the device lane markings CL1 and CL2 based on the output of a detection device DD other than the camera 10 (for example, a radar device 12, a LiDAR 14), and may correct the device lane markings CL1 and CL2 obtained from the camera image. Furthermore, the first recognition unit 132 may recognize the center line of the lane L1 demarcated by the device lane markings CL1 and CL2.

[0055] The second recognition unit 134 recognizes the lane markings around vehicle M from map information (first map information 54, second map information 62) based on the position of vehicle M detected by, for example, vehicle sensor 40 or GNSS receiver 51. For example, the second recognition unit 134 refers to map information based on the position information of vehicle M and recognizes the left and right lane markings that demarcate vehicle M's driving lane L1 as map markings ML1 and ML2.

[0056] Furthermore, the second recognition unit 134 may recognize the curvature or the amount of change in curvature of each of the map lane markings ML1 and ML2 from the map information. Alternatively, the second recognition unit 134 may average the curvature or the amount of change in curvature of each of the map lane markings ML1 and ML2 to recognize the curvature or the amount of change in curvature of the driving lane L1. Additionally, the second recognition unit 134 may acquire information about the center line of the lane L1 demarcated by the map lane markings ML1 and ML2 from the map information. Furthermore, the second recognition unit 134 may recognize the interrupted (missing) portions of the map lane markings ML1 and ML2 on the left and right sides of the vehicle M.

[0057] The determination unit 142 determines whether the device lane marking CL recognized by the first recognition unit 132 matches the map lane marking ML recognized by the second recognition unit 134. Matching may include a predetermined tolerance range of error. For example, the determination unit 142 determines whether the device lane marking CL1 and the map lane marking ML1 that demarcate the left side of the driving lane L1 as seen from the vehicle M match in the section (target range of determination) from the vehicle M's current position (point P0 in the figure) to a position (point P1 in the figure) located a distance (target distance of determination) D1 in the direction of travel (forward), and also determines whether the device lane marking CL2 and the map lane marking ML2 that demarcate the right side of the driving lane L1 match. Here, distance D1 may be, for example, the distance to the furthest point of the device lane markings CL1 and CL2 that can be recognized by the detection device DD by the first recognition unit 132, or it may be a distance corresponding to the road shape or the speed VM of the vehicle M, or it may be a fixed distance.

[0058] For example, the determination unit 142 compares at least one of the following: position, extension direction (angle), curvature, and amount of curvature change between the device lane line CL1 and the map lane line ML1 within the determination target range based on distance D1. For example, the determination unit 142 superimposes the device lane line CL1 and the map lane line ML1 on the vehicle coordinate system plane (XY plane) with reference to the position of a representative point of the vehicle M, compares the lateral position (lane width direction, Y axis direction in the figure) of the device lane line CL1 and the map lane line ML1, and obtains the displacement amount W1. The displacement amount W1 may be the maximum displacement within the determination target range, or it may be the average value.

[0059] Furthermore, the determination unit 142 may compare the extension directions of the device demarcation line CL1 and the map demarcation line ML1 to obtain the deviation angle θ1 of the demarcation line. The deviation angle θ1 may be the maximum deviation angle in the range to be determined, or it may be the average angle. The determination unit 142 may also compare the curvature or the amount of change in curvature of the device demarcation line CL1 and the map demarcation line ML1 to obtain the degree (magnitude) of the difference. Then, the determination unit 142 may obtain the degree of deviation based on the deviation amount W1, the deviation angle θ1, and the degree of difference in curvature or the amount of change in curvature. In this case, for example, the larger the deviation amount W1, the larger the deviation angle θ1, and the larger the degree of difference in curvature (amount of change in curvature), the greater the degree of deviation.

[0060] Furthermore, the determination unit 142, similar to the comparison of the left lane markings, compares at least one of the following between the device lane marking CL2 and the map lane marking ML2: position (amount of displacement W2 in the figure), displacement angle, curvature, and amount of curvature change, or obtains the degree of deviation. Alternatively, the determination unit 142 may comprehensively judge the comparison results of the left lane markings and the right lane markings of the vehicle M (in other words, the driving lane L1) to obtain the degree of deviation between the device lane marking CL and the map lane marking ML.

[0061] For example, the determination unit 142 determines that the device boundary line CL and the map boundary line ML match if both the degree of deviation in the left boundary line and the degree of deviation in the right boundary line are below a threshold, and determines that the device boundary line CL and the map boundary line ML do not match if at least one of them is above the threshold.

[0062] The selection unit 144 selects at least one of the device lane lines CL and map lane lines ML for use in the vehicle M's driving control, based on the determination result by the determination unit 142. For example, if the determination unit 142 determines that the device lane line CL and map lane line ML match, the selection unit 144 selects one or both of the predetermined lane lines from the device lane line CL and map lane line ML. Also, if the determination unit 142 determines that the device lane line CL and map lane line ML do not match, and predetermined conditions are met, the selection unit 144 selects either the device lane line CL or map lane line ML. Details of the function of the selection unit 144 will be described later. Note that if the determination unit 142 determines that the device lane line CL and map lane line ML do not match, and predetermined conditions are not met, the selection unit 144 does not have to select any lane lines.

[0063] The execution control unit 146 executes vehicle M control (driving control) based on the recognition results of the first recognition unit 132 and the second recognition unit 134, the determination results of the judgment unit, the selection results of the selection unit 144, etc. For example, the execution control unit 146 determines the driving control for vehicle M based on the above recognition results, judgment results, selection results, etc., and generates a target trajectory based on the determined driving control. "Determining driving control" may include, for example, determining the content (type) of the driving control or deciding whether or not to execute (suppress) the driving control. Also, "executing driving control" may include, for example, switching and executing the content of the driving control, as well as continuing the driving control that is already being executed. "Suppressing driving control" may include not only not executing driving control, but also lowering the mode (automation level) of the driving control.

[0064] Here, the driving control includes a first driving mode and a second driving mode in which the degree of driver assistance is lower than that of the first driving mode, or in which the tasks of the vehicle M occupants are greater than those of the first driving mode. A lower degree of driver assistance means, for example, a low automation rate in the driving control. A low automation rate means, for example, a low degree to which the automatic driving control device 100 controls the steering or speed of the vehicle M (a high degree to which the driver needs to intervene in steering or acceleration / deceleration operations). A greater tasks for the occupants includes, for example, a large number of tasks assigned to the occupants or tasks that are heavy. Tasks include, for example, monitoring the surroundings of the vehicle M or the occupants operating the driver control device 80. Operating the driver control device 80 includes, for example, the state in which the driver holds the steering wheel (hereinafter referred to as the hands-on state). The driving control may also include a third driving mode, etc., in which the degree of driver assistance is lower than that of the second driving mode, or in which the tasks of the vehicle M occupants are greater than those of the second driving mode. Furthermore, the driving mode in which the degree of driver assistance is lowest or the task of the vehicle M occupant is greatest may be the fully manual driving mode (a mode in which no driving control is performed).

[0065] For example, in the first driving mode, the occupant has no (or minimal) tasks and driving control (e.g., ACC, LKAS, ALC, TJP, CMBS, etc.) is permitted when the occupant of vehicle M is not holding the steering wheel (hereinafter referred to as the hands-off state). In the second driving mode, tasks assigned to the occupant may include, for example, monitoring the surroundings of vehicle M while also being in the hands-on state.

[0066] For example, if the execution control unit 146 determines that the device lane markings CL and the map lane markings ML match while the first driving mode (e.g., driving control in a hands-off state) is being executed, it continues the first driving mode. For example, when the execution control unit 146 is executing LKAS, which drives in the center of the driving lane L1, as the first driving mode, it generates a target trajectory (e.g., target trajectory K1 shown in Figure 3) for driving in the center of lane L1 based on one or both of the device lane markings CL1 and CL2 and the map lane markings ML1 and ML2 selected by the selection unit 144, and causes the second control unit 160 to execute control to drive along the generated target trajectory K1. For example, if the execution control unit 146 selects either the device lane markings CL or the map lane markings ML, it generates the target trajectory K1 based on the left and right positions of the selected lane markings. Furthermore, if the execution control unit 146 obtains information about the center line of lane L1 from the map information, it generates a target trajectory K1 such that the center of gravity (or center) of vehicle M travels along that center line. Also, if both device lane lines CL and map lane lines ML are selected by the selection unit 144, the execution control unit 146 moves (corrects) the position of the other lane line (e.g., map lane line ML) to match the position of one lane line (e.g., device lane line CL), and generates a target trajectory K1 using the information of the moved lane line.

[0067] Furthermore, the execution control unit 146 may, while executing the second driving mode (for example, driving in a hands-on state), perform control to switch from the second driving mode to the first driving mode if it determines that the device lane markings CL and the map lane markings ML match within the range to be determined. In this case, when performing the switching control, the execution control unit 146 may output information to the HMI 30 inquiring whether or not the occupant can switch, or information suggesting a switch, and perform the driving mode switch when information indicating permission to switch is received from the occupant. Alternatively, the execution control unit 146 may switch from the second driving mode to the first driving mode at the timing when it determines that the device lane markings CL and the map lane markings ML match after the HMI 30 has given an instruction to switch to the first driving mode.

[0068] Furthermore, if the execution control unit 146 determines that the device demarcation line CL and the map demarcation line ML in the range to be determined do not match, and if predetermined conditions are not met, it may suppress the driving control by the first driving mode. For example, if the execution control unit 146 determines that the device demarcation line CL and the map demarcation line ML do not match while the first driving mode is being executed, it executes driving control to switch from the first driving mode to the second driving mode. Also, if the execution control unit 146 determines that the device demarcation line CL and the map demarcation line ML do not match while the second driving mode is being executed, it may control the system so that it cannot switch back to the first driving mode.

[0069] Furthermore, if the execution control unit 146 determines during the execution of the first driving mode that the device demarcation line CL and the map demarcation line ML in the determination target range do not match, and if predetermined conditions are met, it will continue the first driving mode based on either the device demarcation line CL or the map demarcation line ML selected by the selection unit 144. Also, if the execution control unit 146 is running the second driving mode and the same conditions as described above are met, it may perform control to switch from the second driving mode to the first driving mode based on either the device demarcation line CL or the map demarcation line ML selected by the selection unit 144.

[0070] Next, the "predetermined conditions" mentioned above and the selection of lane markings by the selection unit 144 will be explained. Figure 4 is a diagram showing an example of a scenario in which the selection unit 144 selects either the device lane marking CL or the map lane marking ML. In the example in Figure 4, a road shape including lanes L11 to L13 is shown. Lane L11 is demarcated by actual lane markings RL11 and RL12, lane L12 is demarcated by actual lane markings RL13 and RL14, and lane L13 is demarcated by actual lane markings RL15 and RL16. Also, in the example in Figure 4, lane L12 is a merging lane that merges with lane L11, which is the main lane, and lane L13 is a diverging lane that separates from lane L11. In the following, lane L11 may be referred to as "driving lane L11," lane L12 as "merging lane L12," and lane L13 as "diverging lane L13." Also, in the example in Figure 4, it is assumed that vehicle M is traveling on lane L11 in the direction of extension (X-axis direction in the figure) at a speed VM.

[0071] Furthermore, in the example shown in Figure 4, the first recognition unit 132 recognizes the device lane markings CL11 and CL12 corresponding to the actual lane markings RL11 and RL12 that demarcate lane L11, and the second recognition unit 134 recognizes the map lane markings ML11 and ML12 of lane L11. Also in the example shown in Figure 4, the first recognition unit 132 recognizes the center line (hereinafter referred to as the device center line) CC of lane L11 demarcated by the device lane markings CL11 and CL12, and the second recognition unit 134 recognizes the center line (hereinafter referred to as the map center line) CM of lane L11 demarcated by the map lane markings ML11 and ML12 from the map information. Note that the map information also stores the map lane markings that demarcate lanes L12 and L13, but for the sake of explanation, Figure 4 shows the map lane markings ML11 and ML12 that demarcate lane L11 on which vehicle M is traveling. Furthermore, in the case of the map lane marking ML11, the portion that connects to the merging lane L12 may remain as is, since a vehicle M traveling in the main lane (lane L11) cannot travel in the merging lane L12. Also, in the example in Figure 4, the map information does not include information regarding branching, merging, or lane changes.

[0072] For example, on roads with branching or merging lanes, or roads where the number of lanes increases or decreases, the map lane markings ML may be interrupted (disappear) in some cases, particularly at points where multiple lanes connect or where the number of lanes changes. As a result, the map lane markings ML in the interrupted portions cannot be recognized. Therefore, as shown in Figure 4, when vehicle M is traveling in the main lane (lane L11), the degree of discrepancy between the device lane markings CL and the map lane markings ML exceeds a threshold, resulting in a state where the lane markings do not match, and thus the first driving mode may not be able to be executed (or continued).

[0073] In the example in Figure 4, the merging lane L12 connects to the driving lane L11 of vehicle M, and the connection point with the branching lane L13 (point Pb in Figure 3) is located at a distance D2 ahead of vehicle M (in the direction of travel) from the connection point (point Pa in Figure 3). In such road conditions, where there are merges or branches in the adjacent lanes (lanes L12 and L13) to the driving lane L11 of vehicle M, the center lines of the merging lane L12 and branching lane L13 overlap with the center line of lane L11. Because the map information does not distinguish and store this overlapping information, the map lane marking ML12 on the right side of the main lane (lane L11) may be interrupted at branching or merging points. In the example in Figure 4, the map lane marking ML12 on the opposite side of the map lane marking ML11 where lane L11 connects to lanes L12 and L13 is interrupted beyond the location of point Pb.

[0074] Furthermore, a portion of the map lane marking ML11 is connected to lane L11 and extends along the branching lane L13 (more specifically, the actual lane marking RL15) that vehicle M can proceed through. For example, if the actual lane marking RL11 is faded at the connection point, the recognition accuracy by camera 10 decreases, making it easier for misconnection to the branching side to occur. Therefore, conventionally, the road shape may be perceived as a merging lane L12 merging with the main lane (lane L11), and that main lane extending towards the branching lane L13, potentially leading to the misidentification that the branching lane L13 is the lane vehicle M should be traveling in. Consequently, conventionally, in the aforementioned road conditions, vehicle M's behavior may become unstable, or deceleration control (curve deceleration) may malfunction in order to travel on the branching lane L13. Moreover, in road shapes like that shown in Figure 4, the device lane marking CL and the map lane marking ML are determined not to match, which may prevent the first driving mode from being executed (continued).

[0075] Therefore, in this embodiment, when the device lane line CL and the map lane line do not match, and the predetermined conditions shown in Figure 4 are met, the selection unit 144 selects one of the lane lines from the device lane line CL and the map lane line ML, and the vehicle M is driven based on the selected lane line. Here, the predetermined conditions include, for example, the following conditions (1) to (4), but are not limited to these, and some of the conditions may not be included. (1) The map lines ML11 and ML12 in the left-right direction (lateral direction) at the current location of vehicle M (point P0 shown in Figure 4) must be recognizable (assuming they are recognizable in the example in Figure 4). (2) The area where the map lane line ML is interrupted, but the map center line CM is not interrupted, is greater than the specified range (in the example in Figure 4, the map lane line ML12 on the right is interrupted at point Pb, but the map center line CM is connected without interruption, although it is incorrectly connected to the left branch lane L13). (3) No information regarding branching lanes, merging lanes, or lane changes is stored in the map information for the driving lane L11 of vehicle M (in the example in Figure 4, the above conditions are assumed not to be stored). (4) The detection range of the detection device DD such as camera 10 (for example, the range to be judged) does not coincide with the device boundary line CL and the map boundary line ML (in the example in Figure 4, they do not coincide).

[0076] For example, in a road shape that satisfies the conditions (1) to (4) described above, it is difficult to determine whether the map information is incorrect or not when the map lane markings ML are interrupted in front of the vehicle M (detection of map discrepancies is difficult). Therefore, the selection unit 144 determines whether the map information is incorrect or not based on the degree of discrepancy between the map center line CM and the device center line CC when the above conditions are met, and if it is determined to be incorrect, it allows the device lane markings CL to be treated as correct lane markings. This makes it possible to suppress the vehicle M's behavior from becoming unstable when it tries to drive in a direction other than the road direction (for example, the direction of extension of the branching lane L13) when driving in the direction of the road (for example, the direction of extension of lane L11) by LKAS control, etc. (in other words, it makes the behavior of the vehicle M more stable).

[0077] For example, if either of the map lane markings ML11 and ML12 that define the vehicle M's driving lane L11 is interrupted in the direction of vehicle M's travel, the selection unit 144 compares the degree of deviation between the lane center line (device center line CC) defined by the device lane markings CL11 and CL12 that are located to the left and right of vehicle M (map center line CM). The selection unit 144 then selects the device lane markings CL11 and CL12 if the degree of deviation is greater than or equal to a threshold. This allows the system to determine whether the device lane marking CL is correct based on the map center line CM when one of the map lane markings ML11 or ML12 is interrupted. Therefore, a more appropriate determination can be made than simply selecting the device lane marking CL because the map lane marking ML is interrupted, and more appropriate driving control can be performed using the information of the lane markings selected by the determination.

[0078] In the example in Figure 4, the selection unit 144 derives the angle between the device center line CC and the map center line CM as the deviation angle θα, determines whether the degree of deviation corresponding to the derived deviation angle θα is greater than or equal to a threshold, and selects the device lane line CL if it is greater than or equal to the threshold. The map center line CM to be compared may, for example, be the map center line CM that extends to the diverging lane L13 side, or the connection point between the device center line CC and the map center line CM (the point from which the deviation angle θα is derived) may be within a predetermined distance from the connection point Pa or Pb, or the distance D2 may be within a predetermined distance range. The predetermined distance range may be, for example, a fixed distance range, or a distance range in which the merging lane L12 and the diverging lane L13 are included within the detection range of the detection device DD.

[0079] Furthermore, if the degree of discrepancy between the device center line CC and the map center line CM is less than a threshold, the selection unit 144 may select either the device center line CL or the map center line ML, which are predetermined, if a portion of the map division line ML12 is interrupted as shown in Figure 4, or if the discrepancy between the center lines is small.

[0080] Furthermore, as shown in Figure 4, if one of the left or right map grid lines is not interrupted, and the device grid line CL and the map grid line ML do not match, the selection unit 144 may not select either grid line and may suppress the execution (continuation) of the first driving mode (driving control).

[0081] In this embodiment, when the above-described condition (2) is met, the selection unit 144 compares the degree of discrepancy between the device center line CC and the map center line CM, and since the map center line CM is not interrupted, it can determine with greater accuracy whether the device demarcation line CL is correct or not, and can select a more appropriate demarcation line.

[0082] In addition, in this embodiment, the selection unit 144 may use the condition that the lane in which the vehicle M is traveling is a main lane in the map information, and that either of the map lane markings ML11 and ML12, which are located to the left and right of the vehicle M, continues beyond the point where it is interrupted in the direction of travel of the vehicle M (i.e., it continues as a main lane even after the point where it is interrupted Pb). In the example in Figure 4, although the map lane marking ML12 is interrupted from point Pb to point Pc, the main lane continues beyond point Pc because the map lane marking ML12 (and map lane marking ML11, map center line CM) are present. The processing of the selection unit 144 described above is based on the premise that the lane L11 of the vehicle M is a main lane, so by including this condition, it is possible to determine with greater accuracy whether the device lane marking CL is a correct lane marking or not.

[0083] Furthermore, the selection unit 144 may, for example, compare the degree of deviation between the device center line CC and the map center line CM within the detection range of the detection device DD by the first recognition unit 132. This makes it possible to determine with greater accuracy whether the device boundary line CL is a correct boundary line without having to perform processing such as drawing a virtual line outside the detection range for comparison.

[0084] Furthermore, the selection unit 144 may compare the degree of deviation between the map center line CM and the device center line CC within the range where the map center line CM is not interrupted. By using the range where the map center line CM is not interrupted in this way, it is possible to determine with greater accuracy whether or not the device lane line CL is a correct lane line.

[0085] Furthermore, as shown in Figure 4, the selection unit 144 may compare the degree of deviation between the device center line CC and the map center line CM when there is a connection point (end of merging point) with a merging lane L12 in the vehicle M's driving lane L11, and there is a connection point (start of branching point) with a branching lane L13 after that. For example, the selection unit 144 may compare the degree of deviation between the device center line CC and the map center line CM when there is a connection point (end of merging point) with a merging lane L12 in the direction of vehicle M's travel in the vehicle M's driving lane L11, and there is a connection point (start of branching point) with a branching lane L13 within a predetermined distance beyond the connection point (end of merging point) from the vehicle M. In this way, in road conditions where merging and branching exist within a predetermined distance, there is a high possibility that the map lane markings ML are stored in a fragmented manner in the map information. Therefore, by comparing the degree of deviation between the device center line CC and the map center line CM in such situations, it is possible to determine with greater accuracy whether the device lane markings CL are correct lane markings.

[0086] In addition, in this embodiment, the HMI control unit 180 may output to the HMI 30 information regarding the execution status of the driving control, as well as information regarding the lane lines selected by the selection unit 144 as described above, and information regarding the reason for the selection of those lane lines.

[0087] [Differentiation] In the embodiments described above, the case where a branching lane follows a merging lane was explained, but the same processing may be applied to sections where the number of lanes increases or decreases (for example, a section where the number of lanes decreases and then increases within a predetermined distance) and other road shapes (for example, a branching lane after a decrease in lanes). Furthermore, in the embodiments described above, the same processing may be applied not only when the map lane marking on the opposite side from the connecting side of the merging or branching lane is interrupted, but also when the map lane marking on the connecting side is interrupted. In addition, in the embodiments described above, the same processing may be applied to road shapes where, in addition to the merging lane and branching lane being connected on one side of the left and right map lane markings, the merging lane is connected on one side and the branching lane is connected on the other side.

[0088] Furthermore, in the embodiment, if the map information includes information regarding branching, merging, or lane changes, the device lane marking CL and the map lane marking ML are determined to match, taking into account the changes (differences) in the lane markings (or road shape) corresponding to each piece of information. In addition, the selection unit 144 may select the map lane marking ML as the one that is interrupted due to branching, merging, or lane changes, if the map information contains information regarding branching, merging, or lane changes.

[0089] Furthermore, in the embodiment, if both of the left and right map section lines are interrupted, the execution of the first driving mode may be suppressed, as the device section line CL and the map section line ML do not match.

[0090] Furthermore, in the embodiment described above, it was determined whether the device boundary line CL and the map boundary line ML matched, but instead of determining whether they matched, it is also possible to determine whether they deviated from each other. In this case, the determination unit 142 determines that there is a deviation if the degree of deviation is greater than or equal to a threshold, and determines that there is no deviation if the degree of deviation is less than a threshold.

[0091] [Processing flow] The following describes the processes performed by the automatic driving control device 100 of this embodiment. Figure 5 is a flowchart showing an example of the flow of processes performed by the automatic driving control device 100. In the following description, the processes performed by the automatic driving control device 100 will mainly focus on the driving control processes based on the recognition status of lane markings, etc. The processes shown below may be repeatedly performed at predetermined timings or predetermined cycles, and may be repeatedly performed while driving control by the automatic driving control device 100 is being performed.

[0092] In the example shown in Figure 5, the first recognition unit 132 recognizes the lane markings (device lane markings) present around the vehicle M based on the output of the detection device DD which has detected the surrounding conditions of the vehicle M (step S100). Next, the second recognition unit 134 refers to map information based on the location information of the vehicle M and recognizes the lane markings (map lane markings) present around the vehicle M from the map information (step S110). Next, the determination unit 142 compares the device lane markings and the map lane markings (step S120) and determines whether the device lane markings and the map lane markings match (step S130).

[0093] If it is determined that the device lane markings and the map lane markings do not match, the selection unit 144 determines whether one of the map lane markings on the left or right of the vehicle M's driving lane (the map lane markings in the direction of travel of the vehicle M) is interrupted (step S140). If it is determined that one of the left or right map lane markings is interrupted, the selection unit 144 obtains the device center line and the map center line (step S150) and compares the obtained center lines (step S160). Next, the selection unit 144 determines whether the degree of deviation between the device center line and the map center line is greater than or equal to a threshold as a result of the comparison (step S170). If it is determined that the degree of deviation is greater than or equal to a threshold, the selection unit 144 selects the device lane markings (step S180). Next, the execution control unit 146 generates a target trajectory based on the selected lane markings (device lane markings), and instructs the second control unit 160 to execute (or continue if already executing) driving control (for example, driving control using the first driving mode) based on the generated target trajectory (step S190).

[0094] Furthermore, in the process of step S170, if it is determined that the degree of deviation between the device center line CC and the map center line CM is not above a threshold, the selection unit 144 selects either the device section line CL or the map section line ML (step S200). Next, the execution control unit 146 generates a target trajectory based on the selected section line and causes the second control unit 160 to execute (or continue if already executing) driving control (for example, driving control using the first driving mode) based on the generated target trajectory (step S210).

[0095] Furthermore, in the process of step S140, if it is determined that one of the left or right map section lines is not interrupted (either both left and right map section lines are not interrupted, or both are interrupted), the selection unit 144 does not select a section line, and the execution control unit 146 suppresses the execution of driving control (step S220). Alternatively, in the process of step S140, if it is determined that one of the left or right map section lines is not interrupted, the selection unit 144 may, instead of proceeding to step S220, compare the device section line and the map section line, and proceed to step S180 if the degree of deviation is greater than or equal to a threshold, and proceed to step S200 if it is less than the threshold.

[0096] Furthermore, if it is determined in step S130 that the device demarcation line and the map demarcation line match, the selection unit 144 selects at least one of the device demarcation line and the map demarcation line (step S230). Next, the execution control unit 146 generates a target trajectory based on the selected demarcation line and causes the second control unit 160 to execute driving control based on the generated target trajectory (step S240). This completes the processing of this flowchart.

[0097] According to the above-described embodiment, the automatic driving control device 100 (an example of a driving control device) includes: a first recognition unit 132 that recognizes the surrounding situation including device lane markings (an example of first lane markings) that demarcate the driving lane of the vehicle M based on the output of a detection device DD that detects the surrounding situation of the vehicle M; a second recognition unit 134 that recognizes map lane markings (an example of second lane markings) that demarcate the lanes around the vehicle M from map information based on the position information of the vehicle M; a determination unit 142 that determines whether the device lane markings and the map lane markings match; and, if the determination unit 142 determines that the device lane markings and the map lane markings do not match, either of the device lane markings or the map lane markings... The system includes a selection unit 144 for selecting a lane marking, and a driving control unit (execution control unit 146, second control unit 160) that executes driving control of the vehicle M based on the lane marking selected by the selection unit 144. The selection unit 144 compares the degree of deviation between the center line of the lane marked by the device lane marking on both sides of the vehicle M and the center line of the lane marked by the map lane marking on both sides of the vehicle M if either of the map lane markings that mark the lane markings of the vehicle M is interrupted in the direction of travel of the vehicle M. If the degree of deviation is greater than or equal to a threshold, the system selects the device lane marking, thereby enabling more appropriate driving control based on road conditions. This can ultimately contribute to the development of a sustainable transportation system.

[0098] Furthermore, according to the embodiment, for example, when the driving lane of vehicle M is a main lane and the adjacent lane is a branching lane, if the map lane markings on the left and right corresponding to the driving lane, on the side opposite to the side connecting to the branching lane, are interrupted near the connection point, it is possible to accurately determine whether the device lane markings are correct or not by comparing the degree of deviation between the center line of the lane marked by the left and right map lane markings and the center line of the lane marked by the left and right device lane markings. In addition, according to the embodiment, when the map lane markings are interrupted, it is possible to more accurately determine whether the map information is incorrect (map discrepancy detection) by comparing the center lines as described above.

[0099] The embodiments described above can be expressed as follows. A storage medium that stores computer-readable instructions, A processor connected to the storage medium, The processor executes the computer-readable instructions to: Based on the output of a detection device that detects the surrounding conditions of a vehicle, the system recognizes the surrounding conditions, including a first lane marking that demarcates the vehicle's lane. Based on the location information of the vehicle, a second lane marking that demarcates the lanes around the vehicle is recognized from the map information. Determine whether the first lane line and the second lane line coincide. If it is determined that the first boundary line and the second boundary line do not coincide, select either the first boundary line or the second boundary line. Based on the selected lane markings, the vehicle's driving control is performed. If either of the second lane markings, which are located to the left and right of the vehicle and define the vehicle's lane, is interrupted in the direction of travel of the vehicle, the degree of deviation between the center line of the lane defined by the first lane markings located to the left and right of the vehicle and the center line of the lane defined by the second lane markings located to the left and right of the vehicle is compared, and if the degree of deviation is greater than or equal to a threshold, the first lane marking is selected. Driving control device.

[0100] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of Symbols]

[0101] 1...Vehicle system, 10...Camera, 12...Radar device, 14...LIDAR, 16...Object recognition device, 20...Communication device, 30...HMI, 40...Vehicle sensor, 50...Navigation device, 60...MPU, 80...Driver control unit, 100...Automatic driving control device, 120...First control unit, 130...Recognition unit, 132...First recognition unit, 134...Second recognition unit, 140...Action plan generation unit, 142...Determination unit, 144...Selection unit, 146...Execution control unit, 160...Second control unit, 162...Target trajectory acquisition unit, 164...Speed ​​control unit, 166...Steering control unit, 180...HMI control unit, 190...Storage unit, 200...Driving force output device, 210...Brake device, 220...Steering device, M...Vehicle

Claims

1. A first recognition unit recognizes the surrounding conditions, including a first lane marking that demarcates the vehicle's driving lane, based on the output of a detection device that detects the surrounding conditions of the vehicle. A second recognition unit recognizes a second lane marking that demarcates the lanes around the vehicle from map information based on the vehicle's location information, A determination unit that determines whether the first dividing line and the second dividing line coincide, If the determination unit determines that the first boundary line and the second boundary line do not coincide, the selection unit selects one of the boundary lines, The vehicle comprises a driving control unit that performs driving control of the vehicle based on the lane lines selected by the selection unit, The selection unit, when either of the second lane markings located to the left and right of the vehicle that demarcate the vehicle's driving lane is interrupted in the direction of travel of the vehicle, compares the degree of deviation between the center line of the lane demarcated by the first lane markings located to the left and right of the vehicle and the center line of the lane demarcated by the second lane markings located to the left and right of the vehicle, and selects the first lane marking if the degree of deviation is greater than or equal to a threshold. Driving control device.

2. The aforementioned map information includes information about the center line of the lane demarcated by the second lane markings on both sides of the vehicle, even beyond the point where one of the second lane markings on both sides of the vehicle is interrupted. The driving control device according to claim 1.

3. The lane in which the vehicle is traveling is the main lane in the map information, and it continues beyond the point where either of the second lane markings located to the left or right of the vehicle is interrupted in the direction of travel of the vehicle. The driving control device according to claim 1.

4. The lane in which the vehicle is traveling is a lane in which the map information does not include information regarding branching lanes, merging lanes, or lane changes. The driving control device according to claim 1.

5. The selection unit compares the degree of deviation between the center line of the lane demarcated by the first lane markings located on the left and right sides of the vehicle and the center line of the lane demarcated by the second lane markings located on the left and right sides of the vehicle, within the detection range of the detection device by the first recognition unit. The driving control device according to claim 1.

6. The selection unit compares the degree of deviation between the center lines of the lanes defined by the first lane markings, which are located to the left and right of the vehicle, and the center lines of the lanes defined by the second lane markings, which are located to the left and right of the vehicle, within the range where the center lines of the lanes defined by the second lane markings are not interrupted. The driving control device according to claim 1.

7. The selection unit compares the degree of deviation between the center line of the lane demarcated by the first lane markings on the left and right of the vehicle and the center line of the lane demarcated by the second lane markings on the left and right of the vehicle, when the driving lane has connection points with merging lanes and connection points with diverging lanes. The driving control device according to claim 1.

8. The selection unit compares the degree of deviation between the center line of the lane demarcated by the first lane markings on either side of the vehicle and the center line of the lane demarcated by the second lane markings on either side of the vehicle, when a connection point with a merging lane exists in the direction of travel of the vehicle in the driving lane, and a connection point with a branching lane exists within a predetermined distance ahead of the connection point as seen from the vehicle. The driving control device according to claim 7.

9. Computers Based on the output of a detection device that detects the surrounding conditions of a vehicle, the system recognizes the surrounding conditions, including a first lane marking that demarcates the vehicle's lane. Based on the location information of the vehicle, a second lane marking that demarcates the lanes around the vehicle is recognized from the map information. Determine whether the first lane line and the second lane line coincide. If it is determined that the first boundary line and the second boundary line do not coincide, select either the first boundary line or the second boundary line. Based on the selected lane markings, the vehicle's driving control is performed. If either of the second lane markings, which are located to the left and right of the vehicle and define the vehicle's lane, is interrupted in the direction of travel of the vehicle, the degree of deviation between the center line of the lane defined by the first lane markings located to the left and right of the vehicle and the center line of the lane defined by the second lane markings located to the left and right of the vehicle is compared, and if the degree of deviation is greater than or equal to a threshold, the first lane marking is selected. A method for controlling vehicle movement.

10. On the computer, Based on the output of a detection device that detects the surrounding conditions of a vehicle, the system recognizes the surrounding conditions, including a first lane marking that demarcates the vehicle's driving lane. Based on the location information of the vehicle, the map information is used to recognize the second lane markings that demarcate the lanes around the vehicle. Determine whether the first lane line and the second lane line coincide. If it is determined that the first boundary line and the second boundary line do not coincide, the system will prompt the user to select either the first boundary line or the second boundary line. The vehicle's driving control is performed based on the selected lane markings. If either of the second lane markings, which are located to the left and right of the vehicle and define the vehicle's lane, is interrupted in the direction of travel of the vehicle, the degree of deviation between the center line of the lane defined by the first lane markings located to the left and right of the vehicle and the center line of the lane defined by the second lane markings located to the left and right of the vehicle is compared, and if the degree of deviation is greater than or equal to a threshold, the first lane marking is selected. program.

Images