Determination device, determination method and program

Abstract

To determine correctness of a road lane marking with improved appropriateness based on the road lane marking around an own vehicle and a situation of a surrounding vehicle.SOLUTION: A determination device comprises: a first recognition section which recognizes a surrounding situation including a first lane marking partitioning a travel lane of an own vehicle and a surrounding vehicle around the own vehicle based on output of a detection device detecting the surrounding situation of the own vehicle; a second recognition section which recognizes a second lane marking partitioning a travel lane around the own vehicle from map information based on positional information of the own vehicle; and a determination section which determines whether the first lane marking and the second lane marking deviate from each other. The determination section differentiates a form indicating that the first lane marking deviates from a form indicating that the second lane marking deviates depending on whether or not the first recognition section recognizes the surrounding vehicle.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a determination device, a determination method, and a program. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transportation systems that take into consideration vulnerable traffic participants have been gaining momentum. To achieve this, efforts are being focused on research and development into autonomous driving technology to further improve traffic safety and convenience. In this regard, a technology has been known that, when it is determined that there is a discrepancy between road markings shown in a camera image (camera markings) and road markings shown in map information (map markings), controls the vehicle's driving mode based on the parallelism between the driving trajectories of other vehicles in the vicinity and the camera markings (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-148405 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional automated driving technology, when the driving trajectory of an adjacent vehicle traveling parallel to the host vehicle in an adjacent lane to the host vehicle is used, the timing at which the trajectory changes is delayed compared to the driving trajectory of the vehicle ahead of the host vehicle, which can lead to a delay in determining whether the camera lane markings are correct. Furthermore, conventionally, the camera lane markings may not be recognizable at long distances, making it impossible to compare the camera lane markings with the trajectory of other vehicles at long distances, which can also lead to a delay in determining whether the camera lane markings are correct.

[0005] In order to solve the above-mentioned problems, one of the objects of the present application is to provide a determination device, a determination method, and a program that can more appropriately determine whether a lane marking is correct or not, depending on the conditions of the road markings around the vehicle and surrounding vehicles, thereby contributing to the development of a sustainable transportation system. [Means for solving the problem]

[0006] The determination device, the determination method, and the program according to the present invention employ the following configuration. (1): A determination device according to one embodiment of the present invention includes a first recognition unit that recognizes the surrounding conditions, including a first dividing line that divides the lane in which the vehicle is traveling and surrounding vehicles present around the vehicle, based on the output of a detection device that detects the surrounding conditions of the vehicle; a second recognition unit that recognizes a second dividing line that divides the lanes around the vehicle from map information based on the position information of the vehicle; and a determination unit that determines whether the first dividing line and the second dividing line diverge, wherein the determination unit changes the manner in which it determines whether the first dividing line and the second dividing line diverge depending on whether the surrounding vehicle is recognized by the first recognition unit and whether the surrounding vehicle is not recognized.

[0007] (2): In the above aspect (1), the surrounding vehicles include adjacent vehicles that are traveling in an adjacent lane adjacent to the lane in which the host vehicle is traveling and that are located within a predetermined distance from the host vehicle, and the determination unit changes the manner of the determination when the surrounding vehicles include an adjacent vehicle.

[0008] (3): In the aspect (1) above, the judgment unit makes it easier to determine that the first dividing line and the second dividing line are separated when the surrounding vehicle is recognized than when the surrounding vehicle is not recognized.

[0009] (4): In the above aspect (2), when the surrounding vehicle is recognized by the first recognition unit, the judgment unit determines whether the first demarcation line and the second demarcation line diverge based on the driving trajectory of the recognized surrounding vehicle that is other than the adjacent vehicle and is located ahead of the adjacent vehicle in the direction of travel, and the second demarcation line.

[0010] (5): In the above aspect (4), the judgment unit sets a virtual first dividing line from the driving trajectory of the surrounding vehicle and judges whether or not the set virtual first dividing line and the second dividing line diverge.

[0011] (6): In the aspect (2) above, when the surrounding vehicle is recognized by the first recognition unit, the judgment unit determines whether or not the first dividing line and the second dividing line diverge at a position less than a predetermined distance from the host vehicle, and determines whether or not the second dividing line diverges from the driving trajectory of a surrounding vehicle other than an adjacent vehicle and located ahead of the adjacent vehicle at a position greater than a predetermined distance from the host vehicle.

[0012] (7) In the aspect (5) above, the determination unit determines whether or not the imaginary first demarcation line and the imaginary second demarcation line deviate from each other at a position that is a predetermined distance or more from the vehicle.

[0013] (8): In the above aspect (2), the judgment unit resets the judgment result of whether or not the first marking line and the second marking line will diverge when the first recognition unit no longer recognizes the adjacent vehicle after using the surrounding vehicle to determine whether or not the first marking line and the second marking line will diverge.

[0014] (9): In the above aspect (4), the judgment unit acquires a driving trajectory of the surrounding vehicle whose deviation angle from the extension direction of the second dividing line is equal to or greater than a predetermined angle, and judges whether the acquired driving trajectory deviates from the second dividing line.

[0015] (10): In the aspect (4) above, the judgment unit acquires the deviation direction of the driving trajectories of the surrounding vehicles relative to the extension direction of the second dividing line, and judges whether the driving trajectory with the greater number of identical deviation directions deviates from the second dividing line.

[0016] (11): In the aspect (4) above, the judgment unit acquires the deviation direction of the driving trajectory of the surrounding vehicle relative to the extension direction of the second dividing line, and if the number of driving trajectories with the same deviation direction is the same in multiple different directions, the judgment unit does not judge whether the driving trajectory of the surrounding vehicle deviates from the second dividing line.

[0017] (12): A determination method according to one embodiment of the present invention is a determination method in which a computer recognizes the surrounding conditions, including a first dividing line that divides the lane in which the vehicle is traveling and surrounding vehicles present around the vehicle, based on the output of a detection device that detects the surrounding conditions of the vehicle, recognizes a second dividing line that divides the lanes around the vehicle from map information based on the position information of the vehicle, determines whether the first dividing line and the second dividing line diverge, and determines whether the first dividing line and the second dividing line diverge in a manner different from a case in which the surrounding vehicles are recognized and a case in which the surrounding vehicles are not recognized.

[0018] (13): A program according to one embodiment of the present invention causes a computer to recognize the surrounding conditions, including a first dividing line that divides the lane in which the vehicle is traveling and surrounding vehicles present around the vehicle, based on the output of a detection device that detects the surrounding conditions of the vehicle; recognize a second dividing line that divides the lanes around the vehicle from map information based on the position information of the vehicle; determine whether the first dividing line and the second dividing line diverge; and determine whether the first dividing line and the second dividing line diverge in a manner that differs depending on whether the surrounding vehicle is recognized and whether the surrounding vehicle is not recognized. [Effects of the Invention]

[0019] According to the above aspects (1) to (13), it is possible to more appropriately determine whether a lane marking is correct or not, depending on the conditions of the road lane markings around the vehicle and surrounding vehicles. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a configuration diagram of a vehicle system 1 including a vehicle control device according to an embodiment. [Figure 2] 2 is a functional configuration diagram of a first control unit 120 and a second control unit 160. FIG. [Figure 3] 10A and 10B are diagrams for explaining the determination process and driving control of the host vehicle M in a first scene. [Figure 4] FIG. 10 is a diagram for explaining the determination content using the virtual camera lane markings. [Figure 5] 10A and 10B are diagrams for explaining the determination process and driving control of the host vehicle M in a second scene. [Figure 6] 3 is a flowchart illustrating an example of processing executed by the automatic driving control device 100 according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0021] Hereinafter, with reference to the drawings, embodiments of a determination device, a determination method, and a program according to the present invention will be described. Hereinafter, as an example, an embodiment will be described in which a vehicle control device including a determination device that determines whether a road dividing line (or lane) that divides the lane on which the vehicle is traveling is a correct dividing line (or lane) is applied to an autonomous vehicle. Autonomous driving refers to, for example, automatically controlling one or both of the steering and speed of the vehicle to perform driving control. The above-mentioned driving control may include, for example, an adaptive cruise control system (ACC), a traffic jam pilot (TJP), a lane keeping assistance system (LKAS), an automated lane change (ALC), a collision mitigation brake system (CMBS), and the like. Furthermore, autonomous vehicles may be manually controlled by a vehicle user (e.g., a passenger) (so-called manual driving). Although the following description will be given for a case where a law stipulates driving on the left side of the road, if a law stipulates driving on the right side of the road, the terms left and right may be reversed.

[0022] [Overall configuration] 1 is a configuration diagram of a vehicle system 1 including a vehicle control device according to an embodiment. The vehicle (hereinafter referred to as host vehicle M) on which the vehicle system 1 is mounted is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination of these. The electric motor operates using power generated by a generator connected to the internal combustion engine, or discharged power from a battery (storage battery) such as a secondary battery or a fuel cell.

[0023] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) device 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, vehicle sensors 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driving operator 80, an automatic driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are connected to each other via multiplexed communication lines such as a CAN (Controller Area Network) communication line, serial communication lines, a wireless communication network, etc. Note that the configuration shown in FIG. 1 is merely an example, and some of the configuration may be omitted, or other configurations may be added. The combination of the camera 10, the radar device 12, the LIDAR 14, and the object recognition device 16 is an example of a "detection device DD." The HMI 30 is an example of an "output device." The automatic driving control device 100 is an example of a "vehicle control device."

[0024] The camera 10 is a digital camera that uses a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to any location of the host vehicle M in which the vehicle system 1 is installed. When capturing an image of the front, the camera 10 is attached to the top of the front windshield, the back of the rearview mirror, the front of the vehicle body, etc. When capturing an image of the rear, the camera 10 is attached to the top of the rear windshield, the back door, etc. When capturing an image of the side, the camera 10 is attached to a door mirror, etc. The camera 10 periodically and repeatedly captures images of the surroundings of the host vehicle M, for example. The camera 10 may be a stereo camera.

[0025] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by surrounding objects (reflected waves) to detect at least the position (distance and direction) of the objects. The radar device 12 is attached to any location on the vehicle M. The radar device 12 may detect the position and speed of the objects using an FM-CW (Frequency Modulated Continuous Wave) method.

[0026] The LIDAR 14 irradiates light around the vehicle M and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time between emitting and receiving the light. The irradiated light is, for example, a pulsed laser beam. The LIDAR 14 is attached to any location on the vehicle M.

[0027] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the camera 10, the radar device 12, and the LIDAR 14 to recognize the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition results to the automatic driving control device 100. Alternatively, the object recognition device 16 may output the detection results from the camera 10, the radar device 12, and the LIDAR 14 directly to the automatic driving control device 100. In that case, the object recognition device 16 may be omitted from the configuration of the vehicle system 1 (detection device DD).

[0028] The communication device 20 communicates with, for example, other vehicles in the vicinity of the vehicle M, terminal devices of users using the vehicle M, or various server devices, using networks such as a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), LAN (Local Area Network), WAN (Wide Area Network), or the Internet.

[0029] The HMI 30 outputs various information to the occupants of the vehicle M and accepts input operations by the occupants. The HMI 30 includes, for example, various display devices, speakers, buzzers, touch panels, switches, keys, microphones, and the like.

[0030] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the host vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the yaw rate (for example, the rotational angular velocity around a vertical axis passing through the center of gravity of the host vehicle M), and a direction sensor that detects the orientation of the host vehicle M. The vehicle sensor 40 may also be provided with a position sensor that detects the position of the vehicle. The position sensor is an example of a "position measurement unit." The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. The position sensor may also be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver 51 of the navigation device 50. The vehicle sensor 40 may derive the speed of the host vehicle M from the difference (i.e., distance) of the position information at a predetermined time in the position sensor. The results detected by the vehicle sensor 40 are output to the automatic driving control device 100.

[0031] The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as a hard disk drive (HDD) or a flash memory. The GNSS receiver 51 identifies the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be identified or supplemented by an inertial navigation system (INS) that uses the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, etc. The GNSS receiver 51 may be provided in the vehicle sensor 40. The navigation HMI 52 may share some or all of the components with the HMI 30 described above. The route determination unit 53 determines, for example, a route (hereinafter, a route on a map) from the position of the vehicle M identified by the GNSS receiver 51 (or an arbitrary input position) to a destination input by the occupant using the navigation HMI 52, with reference to the first map information 54. The first map information 54 is information in which road shapes are expressed by, for example, links indicating roads and nodes connected by the links. The first map information 54 may also include POI (Point Of Interest) information and the like. The route on the map is output to the MPU 60. The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may transmit the current position and destination to a navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server. The navigation device 50 outputs the determined route on the map to the MPU 60.

[0032] The MPU 60 includes, for example, a recommended lane determination unit 61, and stores second map information 62 in a storage device such as an HDD or flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, by dividing it into 100 m intervals in the vehicle travel direction), and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 determines, for example, which lane from the left the vehicle should travel in. When there is a branch point on the route on the map, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel on a reasonable route to the branch point.

[0033] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, the number of lanes, the type and shape of road dividing lines (hereinafter referred to as dividing lines), information on the center of lanes, and road boundary information. The second map information 62 may also include information on whether the road boundary includes a structure that prevents a vehicle from passing (including crossing or coming into contact with). Examples of such structures include guardrails, curbs, medians, and fences. The term "impassable" may include the presence of a low level step that allows passage if unusual vehicle vibrations are tolerated. The second map information 62 may also include road shape information, traffic regulation information, address information (address and postal code), facility information, parking information, telephone number information, and the like. The road shape information includes, for example, the curvature (which may be interpreted as the radius of curvature; the same applies below), width, and gradient of the road. The second map information 62 may be updated as needed by the communication device 20 communicating with an external device. The first map information 54 and the second map information 62 may be provided as an integrated piece of map information.

[0034] The driving operators 80 include, for example, a steering wheel, an accelerator pedal, and a brake pedal. The driving operators 80 may also include a shift lever, a variable steering wheel, a joystick, or other operators. Each operator of the driving operators 80 is equipped with an operation detection unit that detects, for example, the amount of operation of the operator by the occupant or whether or not the operator is operated. The operation detection unit detects, for example, the steering angle and steering torque of the steering wheel, the amount of depression of the accelerator pedal and the brake pedal, etc. The operation detection unit then outputs the detection results to the automatic driving control device 100 or one or both of the driving force output device 200, the brake device 210, and the steering device 220.

[0035] The automatic driving control device 100 executes various types of driving control associated with automatic driving for the host vehicle M. The automatic driving control device 100 includes, for example, a first control unit 120, a second control unit 160, an HMI control unit 180, and a storage unit 190. The first control unit 120, the second control unit 160, and the HMI control unit 180 are each realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Furthermore, some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or an SOC (System On Chip), or may be realized by a combination of software and hardware. The above-mentioned program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as an HDD or flash memory of the automatic driving control device 100, or may be stored in a removable storage medium such as a DVD, CD-ROM, or memory card, and installed in the storage device of the automatic driving control device 100 by inserting the storage medium (non-transitory storage medium) into a drive device, card slot, etc.

[0036] The storage unit 190 may be realized by the various storage devices described above, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a ROM (Read Only Memory), or a RAM (Random Access Memory). The storage unit 190 stores, for example, various types of information, programs, and the like in the embodiments. The storage unit 190 may also store map information (for example, the first map information 54 and the second map information 62).

[0037] FIG. 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 140. The first control unit 120, for example, implements functions based on AI (Artificial Intelligence) and functions based on a predefined model in parallel. For example, the function of "recognizing intersections" may be implemented by concurrently executing intersection recognition using deep learning or the like and recognition based on predefined conditions (such as the presence of traffic lights and road markings that can be pattern matched), and then scoring and comprehensively evaluating both. This ensures the reliability of autonomous driving. The first control unit 120 also executes control related to autonomous driving of the host vehicle M based on instructions from, for example, the MPU 60, the HMI control unit 180, or the like.

[0038] The recognition unit 130 recognizes the surrounding situation of the host vehicle M based on the recognition results of the detection device DD (information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16). For example, the recognition unit 130 recognizes the status of objects present around the host vehicle M (within a predetermined distance), such as their position, speed, and acceleration. The objects include, for example, other vehicles (surrounding vehicles), traffic participants (pedestrians, bicycles, etc.) traveling on the road, road structures, and other obstacles present in the vicinity. The road structures include, for example, road signs, traffic signals, railroad crossings, curbs, medians, guardrails, fences, etc. The position of the object is recognized as a position on an absolute coordinate system with a representative point of the host vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of the object may be represented by a representative point such as the center of gravity or a corner of the object, or by a represented area. The "state" of an object may include, for example, if the object is a moving object such as another vehicle, the acceleration or jerk of the moving object, or the "behavioral state" (for example, whether the other vehicle is changing lanes or is about to change lanes).

[0039] The recognition unit 130 includes, for example, a first recognition unit 132 and a second recognition unit 134. The functions of these units will be described in detail later.

[0040] The behavior plan generation unit 140 generates a behavior plan for driving the host vehicle M by autonomous driving based on the recognition results of the recognition unit 130, etc. For example, the behavior plan generation unit 140 generates a target trajectory for the host vehicle M to travel automatically (without driver operation) in the future, so that the host vehicle M can respond to the surrounding conditions of the host vehicle M, while essentially traveling in the recommended lane determined by the recommended lane determination unit 61, based on the recognition results by the recognition unit 130 and the surrounding road shapes based on the current position of the host vehicle M acquired from map information, etc. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequential arrangement of points (trajectory points) to be reached by the host vehicle M. The trajectory points are points to be reached by the host vehicle M at predetermined travel distances (e.g., on the order of several meters) along the road, and separately, target speeds and target accelerations for predetermined sampling times (e.g., on the order of a few tenths of a second) are generated as part of the target trajectory. Alternatively, the trajectory points may be positions to be reached by the host vehicle M at the sampling times for each predetermined sampling time. In this case, the target speed and target acceleration information are expressed as the interval between trajectory points.

[0041] The action plan generation unit 140 may set an autonomous driving event when generating the target trajectory. Examples of the event include a constant speed driving event in which the host vehicle M drives in the same lane at a constant speed, a following driving event in which the host vehicle M follows another vehicle that is within a predetermined distance (for example, within 100 m) ahead of the host vehicle M and is closest to the host vehicle M, a lane change event in which the host vehicle M changes lanes from the host vehicle's own lane to an adjacent lane, a branching event in which the host vehicle M branches off into a lane on the destination side at a road branching point, a merging event in which the host vehicle M merges into a main lane at a merging point, a takeover event in which the autonomous driving is terminated and the host vehicle M switches to manual driving, and so on. Examples of the event may also include an overtaking event in which the host vehicle M temporarily changes lanes to an adjacent lane, overtakes a leading vehicle in the adjacent lane, and then changes lanes back to the original lane, and an avoidance event in which the host vehicle M performs at least one of braking and steering to avoid an obstacle ahead of the host vehicle M.

[0042] Furthermore, the behavior plan generation unit 140 may change an event already determined for the current section to another event or set a new event for the current section, depending on the surrounding conditions of the host vehicle M recognized while the host vehicle M is traveling. Furthermore, the behavior plan generation unit 140 may change an event already set for the current section to another event or set a new event for the current section, depending on the operation of the occupant on the HMI 30. The behavior plan generation unit 140 generates a target trajectory according to the set event.

[0043] The behavior plan generation unit 140 also includes, for example, a determination unit 142 and an execution control unit 144. Details of these functions will be described later. For example, the recognition unit 130 and the determination unit 142 are an example of a "determination device." The execution control unit 144 and the second control unit 160 are an example of a "driving control unit."

[0044] The second control unit 160 controls the traveling driving force output device 200, the braking device 210, and the steering device 220 so that the host vehicle M passes through the target trajectory generated by the action plan generation unit 140 at the scheduled time.

[0045] The second control unit 160 includes, for example, a target trajectory acquisition unit 162, a speed control unit 164, and a steering control unit 166. The target trajectory acquisition unit 162 acquires information on the target trajectory (trajectory points) generated by the action plan generation unit 140 and stores it in a memory (not shown). The speed control unit 164 controls the driving force output device 200 or the brake device 210 based on a speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized by, for example, a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road ahead of the host vehicle M and feedback control based on the deviation from the target trajectory.

[0046] Returning to FIG. 1 , the HMI control unit 180 notifies the occupant of predetermined information via the HMI 30. The predetermined information includes, for example, information related to the traveling of the vehicle M, such as information related to the state of the vehicle M and information related to driving control. The information related to the state of the vehicle M includes, for example, the speed of the vehicle M, engine speed, and shift position. The information related to driving control includes, for example, whether or not driving control is being performed by autonomous driving, information inquiring about whether or not to start autonomous driving, information related to the driving control status by autonomous driving, information related to the automation level, and information prompting the occupant to drive when switching from autonomous driving to manual driving. The predetermined information may also include information unrelated to the traveling of the vehicle M, such as television programs and content (e.g., movies) stored on a storage medium such as a DVD. The predetermined information may also include, for example, information related to the current location and destination during autonomous driving, and the remaining amount of fuel in the vehicle M. The HMI control unit 180 may output the information received by the HMI 30 to the communication device 20, the navigation device 50, the first control unit 120, etc.

[0047] Furthermore, the HMI control unit 180 may cause the HMI 30 to output inquiry information for the occupant, processing results by the first control unit 120 and the second control unit 160, etc. Furthermore, the HMI control unit 180 may transmit various pieces of information to be output by the HMI 30 to a terminal device used by the user of the vehicle M via the communication device 20.

[0048] Traveling drive force output device 200 outputs a traveling drive force (torque) to the drive wheels for the vehicle to travel. Traveling drive force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, etc., and an ECU (Electronic Control Unit) that controls these. The ECU controls the above components according to information input from second control unit 160 or information input from the accelerator pedal of driving operator 80.

[0049] The braking device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or information input from the brake pedal of the driving operation device 80, so that a braking torque corresponding to the braking operation is output to each wheel. The braking device 210 may include a backup mechanism that transmits hydraulic pressure generated by operation of the brake pedal to the cylinder via a master cylinder. Note that the braking device 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic braking device that controls an actuator according to information input from the second control unit 160 to transmit hydraulic pressure from the master cylinder to the cylinder.

[0050] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor changes the direction of the steered wheels by, for example, applying a force to a rack and pinion mechanism. The steering ECU drives the electric motor to change the direction of the steered wheels in accordance with information input from the second control unit 160 or information input from the steering wheel of the driving operator 80.

[0051] [Recognition and Action Plan Generation] Next, the functions of the recognition unit 130 (first recognition unit 132, second recognition unit 134) and the behavior plan generation unit 140 (determination unit 142, execution control unit 144) will be described in detail. Note that, below, the contents of the driving control (travel control) based on the determination process and determination control in the embodiment will be mainly explained by dividing them into several scenes.

[0052] [Scene 1] FIG. 3 is a diagram illustrating the determination process and driving control of the host vehicle M in a first scenario. The example in FIG. 3 shows lane markings CL1-CL3 recognized by the detection device DD and lane markings ML1-ML3 obtained from map information (e.g., second map information 62) based on the position information of the host vehicle M. In the map information, lane L1 is defined by lane markings ML1 and ML2, and lane L2 is defined by lane markings ML2 and ML3. Lanes L1 and L2 are lanes along which a vehicle can travel in the same direction (the X-axis direction in the figure). In the example in FIG. 3, lane markings CL1-CL3 are an example of a "first lane marking," and lane markings ML1-ML3 are an example of a "second lane marking." Also, in FIG. 3, the host vehicle M is traveling on lane L1 at a speed VM, and another vehicle m1 is traveling on lane L2, an adjacent lane to lane L1, at a speed Vm1. The other vehicle m1 is an adjacent vehicle to the host vehicle M. An adjacent vehicle is, for example, a vehicle traveling (running parallel) in an adjacent lane adjacent to the lane in which the host vehicle is traveling. Furthermore, the adjacent vehicle may be a vehicle located within a predetermined distance from the host vehicle M. In the example of FIG. 3, the host vehicle M is assumed to be performing predetermined driving control (e.g., LKAS) based on surrounding conditions, instructions from the occupants, etc. In the example of FIG. 3, in front of the host vehicle M (and the other vehicle m1), another vehicle m2 is traveling at a speed Vm2, and another vehicle m3 is traveling at a speed Vm3.

[0053] The first recognition unit 132 recognizes the surrounding conditions of the host vehicle M based on the output of the detection device DD, which detects the surrounding conditions (external world) of the host vehicle M. For example, the first recognition unit 132 recognizes left and right lane markings CL1 and CL2 that demarcate the lane (lane L1) of the host vehicle M based on an image captured by the camera 10 (hereinafter referred to as a camera image). The first recognition unit 132 may also recognize a lane marking CL3 that demarcates an adjacent lane (lane L2) adjacent to the driving lane. Hereinafter, the lane markings CL1 to CL3 may be referred to as "camera lane marks CL1 to CL3." For example, the first recognition unit 132 analyzes the camera image, extracts edge points in the image that have a large difference in brightness from adjacent pixels, and recognizes the camera lane marks CL1 to CL3 on the image plane by connecting the edge points. The first recognition unit 132 also converts the positions of the camera lane markings CL1-CL3 into a vehicle coordinate system (e.g., the XY plane coordinates in FIG. 3 ) based on the position of the representative point of the vehicle M. The first recognition unit 132 may also recognize, for example, the curvature of the camera lane markings CL1-CL3. The first recognition unit 132 may also recognize the amount of curvature change of the camera lane markings CL1-CL3. The amount of curvature change is, for example, the time rate of change in the curvature of the camera lane markings CL1-CL3 recognized by the camera 10 at a distance x [m] forward as viewed from the vehicle M. The first recognition unit 132 may also recognize the curvature or amount of curvature change of the lane markings CL1-CL3 by averaging the curvatures or amounts of curvature change of the camera lane markings CL1-CL3. The camera lane markings CL1-CL3 may be recognized or corrected based on the output of a detection device other than the camera 10 (e.g., the radar device 12 or the LIDAR 14).

[0054] The first recognition unit 132 also recognizes other vehicles (neighboring vehicles) present around (within a predetermined distance from) the host vehicle M. In the example of FIG. 3, the first recognition unit 132 recognizes the other vehicle (adjacent vehicle) m1 traveling parallel to the host vehicle M in an adjacent lane and the other vehicles (leading vehicles) m2 and m3 traveling ahead of the host vehicle M, based on the output of the detection device DD that detects the surrounding conditions of the host vehicle M. The first recognition unit 132 also recognizes the position (relative position with respect to the host vehicle M) and speed (relative speed with respect to the host vehicle M) of each of the other vehicles m1 to m3, as well as the traveling lane, vehicle orientation, and traveling direction of each of the other vehicles m1 to m3. The first recognition unit 132 may also recognize traveling position information of the other vehicles m1 to m3. The traveling position information is, for example, traveling trajectories K1 to K3 based on the respective reference positions (e.g., centers or centers of gravity) of the other vehicles m1 to m3 at a predetermined time.

[0055] The second recognition unit 134 recognizes, for example, from map information based on the position of the host vehicle M detected by the vehicle sensor 40 or the GNSS receiver 51, the marking lines that demarcate the lanes around the host vehicle M. For example, the second recognition unit 134 refers to map information based on the position information of the host vehicle M, and recognizes marking lines ML1 to ML3 that exist in the traveling direction of the host vehicle M or in directions in which the host vehicle M can travel. Hereinafter, the marking lines ML1 to ML3 may be referred to as "map marking lines ML1 to ML3."

[0056] Furthermore, the second recognition unit 134 may recognize, among the recognized map division lines ML1 to ML3, map division lines ML1 and ML2 as division lines that divide the lane L1 on which the host vehicle M is traveling. Furthermore, the second recognition unit 134 recognizes the curvature or curvature change amount of each of the map division lines ML1 to ML3 from the second map information 62. Furthermore, the second recognition unit 134 may average the curvature or curvature change amount of each of the map division lines ML1 to ML3 to recognize the curvature or curvature change amount of the lane divided by the map division lines.

[0057] The determination unit 142 determines whether the camera lane lines CL1-CL3 recognized by the first recognition unit 132 deviate from the map lane lines ML1-ML3 recognized by the second recognition unit 134. For example, the determination unit 142 derives the degree of deviation between the lane lines CL1 and ML1 located closest to the left of the host vehicle M, the degree of deviation between the lane lines CL2 and ML2 located closest to the right of the host vehicle M, and the degree of deviation between the lane lines CL3 and ML3 on the adjacent lane side. If the derived degree of deviation is equal to or greater than a threshold, the determination unit 142 determines that the camera lane lines and the map lane lines deviate, and if the derived degree of deviation is less than the threshold, the determination unit 142 determines that there is no deviation. The deviation determination described above is performed repeatedly at a predetermined timing or period.

[0058] For example, the determination unit 142 superimposes the camera lane lines CL1, CL2, and CL3 and the map lane lines ML1, ML2, and ML3 on the plane of the vehicle coordinate system (XY plane) based on the position of the representative point of the vehicle M. When determining the lane lines to be compared (lane lines CL1 and ML1, lane lines CL2 and ML2, and lane lines CL3 and ML3), the determination unit 142 determines that the lane lines diverge if the degree of deviation of each lane line is equal to or greater than a threshold, and determines that the lane lines do not diverge if the degree of deviation is less than the threshold. The degree of deviation is, for example, the amount of deviation in the lateral position (e.g., the Y-axis direction in the figure). In the example of Figure 3, the deviation determination may be performed using the average value of the lateral position deviation D1 between the marking lines CL1 and ML1, the lateral position deviation D2 between the marking lines CL2 and ML2, and the lateral position deviation D3 between the marking lines CL3 and ML3, or the deviation determination may be performed using the maximum or minimum value of the deviations D1, D2, and D3.

[0059] Furthermore, the degree of deviation may be, for example, the degree (magnitude) of the angle formed by the two marking lines being compared, instead of (or in addition to) the above-mentioned lateral position deviation. In the example of Fig. 3, the average value of the angle θ1 formed by marking lines CL1 and ML1, the angle θ2 formed by marking lines CL2 and ML2, and the angle θ3 formed by marking lines CL3 and ML3 may be used, or the maximum or minimum value of the angles θ1, θ2, and θ3 may be used.

[0060] Furthermore, the degree of deviation may be the degree (magnitude) of difference in curvature change of the lane markings, instead of (or in addition to) the lateral position deviation or the angle formed by the lane markings. The curvature change is mainly used when the lane is a curved road. The determination unit 142 may use the average value of the difference in curvature change between lane markings CL1 and ML1, the difference in curvature change between lane markings CL2 and ML2, and the difference in curvature change between lane markings CL3 and ML3, or may use the maximum or minimum of these differences. The determination unit 142 may also use the difference between the average value of the curvature change of lane markings CL1 to CL3 and the average value of the curvature change of lane markings ML1 to ML3. The determination unit 142 may also use the difference between the curvature change of the lane markings (lanes L1 and L2) recognized from the camera image and the curvature change of the lane markings recognized from map information.

[0061] In the embodiment, the determination unit 142 changes the manner of determining whether or not there is a deviation between the camera lane markings and the map lane markings depending on whether or not a nearby vehicle is recognized by the first recognition unit 132. "Varying the manner of determination" means, for example, varying the threshold value used in the deviation determination, varying the cycle of the determination process, or changing various other determination conditions. Note that the determination unit 142 may change the manner of the above-described determination when the recognized nearby vehicles include an adjacent vehicle. When an adjacent vehicle is present, changing the manner of determination can enable earlier suppression of interference (contact) between the host vehicle M and the adjacent vehicle, thereby realizing more appropriate deviation determination and driving control based on the determination result.

[0062] Furthermore, the determination unit 142 may determine whether or not a marking line (e.g., a camera marking line) that defines the lane on which the vehicle M is traveling is a correct marking line, based on the determination result of whether or not there is a deviation. Note that instead of (or in addition to) determining whether or not the marking line is correct, the determination unit 142 may determine whether or not the lane defined by the marking line is correct. If it is determined that the marking line is correct, the execution control unit 144 generates a target trajectory so that the vehicle M travels along the correct marking line, and the second control unit 160 executes driving control (travel control) of the vehicle M based on the target trajectory.

[0063] For example, when the first recognition unit 132 recognizes surrounding vehicles and the recognized surrounding vehicles include an adjacent vehicle, the determination unit 142 performs a deviation determination based on the travel path of a surrounding vehicle other than the adjacent vehicle and located ahead (in front of the vehicle M in the traveling direction) of the adjacent vehicle (or the vehicle M) and the map lane markings. When an adjacent vehicle is recognized, the determination unit 142 compares the travel path of the surrounding vehicle other than the adjacent vehicle (for example, the vehicle ahead) with the map lane markings, thereby making it possible to perform a deviation determination even when distant camera lane markings cannot be recognized. This makes it possible to execute driving control early on to prevent interference between the vehicle M and the adjacent vehicle based on a prediction that the adjacent vehicle will also travel along the travel path of the vehicle ahead, thereby more reliably suppressing interference.

[0064] For example, as in the scene shown in Fig. 3, when the recognition range (the range where the recognition accuracy is equal to or greater than a threshold) of the camera lane markings CL1 to CL3 ahead of the host vehicle M by the first recognition unit 132 is point P1 shown in Fig. 3, the host vehicle M cannot recognize the camera lane markings beyond that (farther than point P1 as seen from the host vehicle M). In this case, of the travel trajectories of the surrounding vehicles recognized by the first recognition unit 132 (travel trajectories K1 to K3 of the other vehicles m1 to m3 shown in Fig. 3), the determination unit 142 treats the travel trajectories K2 and K3 of the other vehicles m2 and m3 that are located farther than point P1 as equivalent to the camera lane markings, compares them with the map lane markings ML, and determines whether there is a deviation. In determining whether or not there is a deviation, the determination unit 142 derives the degree of deviation based on, for example, the angles θa and θb formed by the extension direction of the map division lines ML1 to ML3 and the extension direction of the driving trajectories K2 and K3 (which may be read as "the deviation angle of the driving trajectories K2 and K3 from the extension direction of the map division lines ML1 to ML3").The determination unit 142 determines that the division lines deviate if at least one of the derived degrees of deviation or the average value of the degrees of deviation is equal to or greater than a threshold, and determines that there is no deviation if it is less than the threshold.In addition, the determination unit 142 may determine that the camera division lines CL1 and CL2 are correct division lines if it determines that there is a deviation under the above conditions.

[0065] The conditions for determining that the camera lane markings CL1 and CL2 are correct may include, for example, whether or not there is any deviation between the travel path K1 of the adjacent vehicle m1 and the camera lane markings CL1 and CL2. The point at which the determination unit 142 can determine the deviation further than point P1 may be, for example, point P2 based on the end (far end) of the travel path of the surrounding vehicle (in the example of FIG. 3, the travel path K2 of the adjacent vehicle m2) that is located farthest from the subject vehicle M (or the adjacent vehicle m1) recognized by the subject vehicle M, or a point a predetermined distance away from point P1. By limiting the range in this way, erroneous determinations at long distances can be suppressed.

[0066] Alternatively, the determination unit 142 may set virtual camera lane lines (virtual camera lane lines, virtual first lane lines) using the travel trajectories K2 and K3 of the other vehicles m2 and m3 instead of the travel trajectories K2 and K3 of the other vehicles m2 and m3, and compare the set virtual camera lane lines with map lane lines to determine whether there is a discrepancy or whether the camera lane lines are correct. FIG. 4 is a diagram for explaining the determination content using the virtual camera lane lines. FIG. 4 shows a scene similar to the first scene shown in FIG. 3. In the example of FIG. 4, the determination unit 142 sets virtual camera lane lines VCL1 to VCL3 extending from the end (point P1) of the camera lane lines CL1 to CL3 recognized by the first recognition unit 132 in parallel to the extension direction of the travel trajectories K2 and K3 of the other vehicles m2 and m3 located farther away than point P1. The lengths of the virtual camera lane lines VCL1 to VCL3 are set based on, for example, the position of the travel trajectory of the preceding vehicle. When point P1 is the starting point of the virtual camera lane markings VCL1-VCL3, the end point may be point P2 based on the end (far end) of the travel path K2 of another vehicle m2 located farthest from the subject vehicle M (or another vehicle m1) recognized by the subject vehicle M, or point P3 obtained by adding a predetermined length. Furthermore, the length of the virtual camera lane markings VCL1-VCL3 may be a predetermined fixed length. By limiting the range in this way, erroneous determinations at long distances can be suppressed.

[0067] The determination unit 142 then regards the set virtual camera lane lines VCL1-VCL3 as camera lane lines and compares them with the map lane lines ML1-ML3 to determine whether there is a deviation. In the example of Fig. 4, the determination unit 142 obtains the angles θα, θβ, and θγ formed by the extension directions of the map lane lines ML1-ML3 and the extension directions of the virtual camera lane lines VCL1-VCL3, derives the degree of deviation using the obtained angles θα, θβ, and θγ, and performs deviation determination based on the derived degree of deviation and a threshold. The degree of deviation is derived, for example, based on the average, maximum, or minimum value of the angles θα, θβ, and θγ.

[0068] This makes it possible to compare the camera lane markings, which are virtually set based on the travel trajectories of surrounding vehicles, with the map lane markings to perform deviation determination, etc., even if the camera lane markings cannot be directly recognized. Therefore, it is possible to prevent the camera lane markings from being immediately determined to be incorrect because they cannot be recognized, and it is possible to continue driving control to cause the host vehicle M to travel along either the camera lane markings or the map lane markings.

[0069] Furthermore, when the vehicle is close to the host vehicle M (for example, less than a predetermined distance), the determination unit 142 performs a deviation determination between the camera lane markings and the road lane markings, even if a peripheral vehicle (for example, an adjacent vehicle) is present in the close distance. On the other hand, when the vehicle is far from the host vehicle M (for example, equal to or greater than a predetermined distance), the determination unit 142 performs a deviation determination based on the travel trajectory of a peripheral vehicle that is present at a long distance ahead of the adjacent vehicle (or the host vehicle M) and the map lane markings, as shown in FIG. 3. The predetermined distance may be a fixed distance determined in advance, or may be a variable distance depending on the speed VM of the host vehicle M, the road shape, the recognition range of the camera lane markings, and the like. When the vehicle is far from the host vehicle M, the determination unit 142 may perform a deviation determination based on the virtual camera lane markings and the map lane markings, as shown in FIG. 4. This allows for smooth switching of the determination conditions between a close distance and a long distance.

[0070] Furthermore, when the surrounding vehicles include an adjacent vehicle, the determination unit 142 may reset the deviation determination result if the surrounding vehicles no longer include an adjacent vehicle (the first recognition unit 132 no longer recognizes the adjacent vehicle) after determining that there is no deviation between the camera lane markings and the map lane markings based on the deviation determination based on the above-described method shown in FIGS. 3 and 4. The method shown in FIGS. 3 and 4 aims to perform deviation determination and correct / incorrect determination of lane markings early in order to suppress interference between the host vehicle M and an adjacent vehicle (another vehicle m1). Therefore, if there is no adjacent vehicle, the deviation determination result is reset. In this case, the determination unit 142 performs a new deviation determination based on the current situation, or performs a deviation determination from the map lane markings within the range in which the camera lane markings can be recognized. This allows for more appropriate determination processing to be performed according to the surrounding situation.

[0071] Furthermore, in the embodiment, when a nearby vehicle is recognized by the first recognition unit 132, the determination unit 142 reduces the threshold value used for the deviation determination (the threshold value compared with the deviation degree) to make it easier to determine that the lane markings have deviated compared to when a nearby vehicle is not recognized. As a result, when a nearby vehicle is present, the result of the deviation determination for the lane markings can be obtained earlier, and the driving control currently being executed by the host vehicle M can be switched. Therefore, interference (contact) between the host vehicle M and the nearby vehicles can be more appropriately suppressed. Furthermore, when a nearby vehicle is recognized, the determination unit 142 may perform the deviation determination more frequently compared to when a nearby vehicle is not recognized. This enables determination under conditions that are closer to the current condition and allows the driving control currently being executed by the host vehicle M to be switched earlier depending on the condition, thereby more appropriately suppressing interference between the host vehicle M and the nearby vehicles. In addition, the above phrase "when surrounding vehicles are recognized" may be read as "when the surrounding vehicles include adjacent vehicles," and "when surrounding vehicles are not recognized" may be read as "when the surrounding vehicles do not include adjacent vehicles (or when there are no adjacent vehicles)."

[0072] In addition, when a surrounding vehicle is not recognized, the determination unit 142 does not perform deviation determination using the driving trajectory of the surrounding vehicle as shown in Figures 3 and 4, but instead performs deviation determination from the map lane markings within the range in which the camera lane markings can be recognized.

[0073] The execution control unit 144 determines driving control for the host vehicle M based on the determination result by the determination unit 142, and executes the determined driving control. "Determining driving control" may include, for example, determining the content (type) of driving control and determining whether to execute (suppress) driving control. "Executing driving control" may include, for example, switching the content of driving control and executing it, as well as continuing driving control that is already being executed. "Suppressing driving control" may include not only not executing (or terminating) driving control, but also lowering the automation level of driving control. Furthermore, the driving control executed by the execution control unit 144 may include ACC, TJP, LKAS, ALC, CMBS, etc., and may also include various other driving controls for avoiding contact with surrounding vehicles. The execution control unit 144 generates a target trajectory for executing driving control and outputs the generated target trajectory to the second control unit 160.

[0074] Here, in the first scenario, the driving control executed by the execution control unit 144 includes at least a first driving control and a second driving control. The first driving control is, for example, driving control that executes at least steering control of the steering or speed of the host vehicle M based on a lane marking recognized by the first recognition unit 132 or the second recognition unit 134 (e.g., a portion of the lane marking that does not deviate from the camera lane marking and the map lane marking). For example, the first driving control is driving control that causes the host vehicle M to travel so that a representative point of the host vehicle M passes through the center of a lane marked by lane markings. The second driving control is, for example, driving control that executes at least steering control of the steering or speed of the host vehicle M based on the camera lane marking recognized by the first recognition unit 132 and the traveling position information of another vehicle. The second driving control is, for example, driving control that causes the host vehicle M to travel so that a representative point of the host vehicle M travels on a trajectory that follows the traveling trajectory of the other vehicle m1.

[0075] Furthermore, the driving control may include a third driving control in which at least steering control of the steering or speed of the host vehicle M is performed while giving priority to camera lane lines over map lane lines, and a fourth driving control in which at least steering control of the steering or speed of the host vehicle M is performed while giving priority to map lane lines over camera lane lines. Prioritizing camera lane lines over map lane lines means, for example, that processing is basically performed based on the camera lane lines, but temporarily switches to processing based on the map lane lines when, for example, the recognition accuracy of the camera lane lines falls below a threshold or they become unrecognizable. Prioritizing map lane lines over camera lane lines means, for example, that processing is basically performed based on the map lane lines, but temporarily switches to processing based on the camera lane lines when, for example, the map lane lines cannot be identified. The third driving control and the fourth driving control are driving controls performed, for example, when the camera lane lines and the map lane lines are separated from each other.

[0076] Furthermore, the driving control may include multiple driving control levels based on automation levels (an example of the degree of automation). The automation levels may include, for example, a first level, a second level with a lower degree of automation of driving control than the first level, and a third level with a lower degree of automation of driving control than the second level. The automation levels may also include a fourth level (an example of a fourth control level) with a lower degree of automation of driving control than the third level. Here, the automation level may be a level determined by standardized information, laws and regulations, or an index value set independently of such. Therefore, the types, contents, and number of automation levels are not limited to the following examples. A low degree of automation of driving control means, for example, a low automation rate in driving control and a large (heavy) task imposed on the driver. A low degree of automation of driving control means a low degree of control of the steering or acceleration / deceleration of the host vehicle M by the automatic driving control device 100 (a high degree of need for the driver to intervene in steering or acceleration / deceleration operations). Tasks assigned to the driver include, for example, monitoring the surroundings of the vehicle M and operating driving controls. Operating driving controls includes, for example, the driver gripping the steering wheel (hereinafter referred to as a hands-on state). Tasks assigned to the driver include, for example, tasks for the occupant (driver tasks) required to maintain the autonomous driving of the vehicle M. Therefore, if the occupant is unable to perform the assigned tasks, the automation level will be reduced. For example, the first level of driving control may include driving controls such as ACC, ALC, LKAS, and TJP. Furthermore, the second or third level of driving control may include driving controls such as ACC, ALC, and LKAS. The fourth level of driving control may include manual driving. Furthermore, the fourth level of driving control may include, for example, driving controls such as ACC. Of the first to fourth levels, the first level has the highest degree of automation of driving control, and the fourth level has the lowest degree of automation of driving control.

[0077] Furthermore, at the first level, no tasks are assigned to the occupant (the driver has the lightest tasks). At the second level, the task assigned to the occupant is, for example, monitoring the surroundings (particularly ahead) of the vehicle M. At the third level, the task assigned to the occupant includes, for example, monitoring the surroundings of the vehicle M as well as being in a hands-on state. At the fourth level, the task assigned to the occupant (e.g., the driver) is, for example, monitoring the surroundings of the vehicle M and being in a hands-on state, as well as operating the driving operator 80 to control the steering and speed of the vehicle M. In other words, at the fourth level, the occupant is ready to immediately take over driving, and the driver has the most severe tasks. The content of driving control and the tasks assigned to the occupant at each automation level are not limited to the examples described above. The automatic driving control device 100 executes driving control at one of the first to fourth levels based on the surrounding conditions of the vehicle M and the tasks being performed by the occupant. At least some of the first to fourth levels may be associated with the above-mentioned first to fourth operational controls, for example.

[0078] For example, the execution control unit 144 executes a first driving control when the determination unit 142 determines that the camera lane markings and the map lane markings are not separated, and executes one of the second to fourth driving controls depending on the situation when the determination unit 142 determines that the camera lane markings and the map lane markings are separated. Furthermore, the execution control unit 144 may execute control such as terminating driving control of the vehicle M and switching to manual driving by the occupant based on the determination result. Furthermore, the execution control unit 144 may switch the automation level corresponding to the driving control based on the determination result. In this case, for example, a first or second level driving control is executed when it is determined that the camera lane markings and the map lane markings are not separated, and a third or fourth level driving control is executed depending on the situation when it is determined that there is no separation.

[0079] [Second Scene] FIG. 5 is a diagram illustrating the determination process and driving control of the host vehicle M in a second scenario. The example in FIG. 5 differs from the example in FIG. 3 in that in addition to the other vehicles m1 to m3, there is also another vehicle (neighboring vehicle) m4 traveling ahead of the host vehicle M at a speed Vm4. In the second scenario, the first recognition unit 132 recognizes the positions, speeds, lane, vehicle orientation, traveling direction, and traveling trajectories K1 to K4 of the other vehicles m1 to m4 present around the host vehicle M. The determination unit 142 performs deviation determination using map division lines ML1 to ML3 and the traveling trajectories K1 to K4 of the other vehicles m1 to m4. In this case, the determination unit 142 performs deviation determination using, for example, traveling trajectories whose deviation angle of the traveling trajectories K1 to K4 from the extension direction of the map division lines ML1 to ML3 is equal to or greater than a predetermined angle. The predetermined angle may be a fixed angle or a variable angle depending on the road shape, etc.

[0080] 5, among other vehicles m1 to m4, the deviation angles θa to θc of other vehicles m2 to m4 are equal to or greater than a predetermined angle, so deviation determination is performed on the travel trajectories K2 to K4 of other vehicles m2 to m4 as target travel trajectories. Other vehicles whose travel trajectories have deviation angles from the map division lines that are less than a predetermined angle are expected to be unlikely to approach the host vehicle M, so processing efficiency can be improved by excluding travel trajectories whose deviation angles are less than the predetermined angle from the targets of deviation determination.

[0081] In the second scenario, the determination unit 142 may acquire the deviation directions of the travel trajectories K1 to K4 relative to the extension direction of the map division lines, compare the acquired deviation directions, and use the travel trajectory with the most common deviation direction to determine the deviation from the map division lines. In the example of FIG. 5, the deviation directions of the travel trajectories K2 and K3 are to the right relative to the extension direction of the map division lines ML1 to ML3, and the deviation direction of the travel trajectory K4 is to the left. Therefore, the determination unit 142 performs the deviation determination from the map division lines ML1 to ML3 using the travel trajectories K2 and K3 that deviate to the right, which is the most common direction. This makes it possible to exclude other vehicles that deviate in the opposite direction to avoid an obstacle or change lanes, thereby enabling more appropriate deviation determination based on the travel trajectories and the map division lines.

[0082] The determination unit 142 may not perform deviation determination between the travel trajectories and the map division lines when the number of travel trajectories with the same deviation direction is the same in multiple different directions. For example, when the number of travel trajectories deviating to the right of the extension direction of the map division lines is the same as the number of travel trajectories deviating to the left, the determination unit 142 does not perform deviation determination using the travel trajectories. When there are the same number of travel trajectories deviating to the left and right, it is difficult to determine which is correct. Therefore, by controlling not to perform deviation determination using the travel trajectories when there are the same number, erroneous determination can be suppressed. The processing in the second scenario may be performed, for example, only when the surrounding vehicles recognized by the first recognition unit 132 include both a leading vehicle and an adjacent vehicle.

[0083] [Processing flow] The following describes the processing executed by the automatic driving control device 100 of the embodiment. FIG. 6 is a flowchart showing an example of the processing executed by the automatic driving control device 100 of the embodiment. The following mainly describes the processing executed by the automatic driving control device 100, focusing on the processing for determining the deviation between the map lane markings and the camera lane markings and the driving control processing based on the determination results. At the start of the flow, it is assumed that the host vehicle M is performing predetermined driving control based on the surrounding conditions, instructions from the occupants, etc. The processing described below may be repeatedly executed at predetermined timing or at predetermined intervals, and may be repeatedly executed while the automatic driving by the automatic driving control device 100 is being performed.

[0084] 6, the first recognition unit 132 recognizes the lane markings (camera lane markings) present around the host vehicle M based on the output of the detection device DD that detects the surrounding conditions of the host vehicle M (step S100). Next, the first recognition unit 132 recognizes the surrounding vehicles present around the host vehicle M (step S110). Next, the second recognition unit 134 refers to map information based on the position information of the host vehicle M, and recognizes the lane markings (map lane markings) present around the host vehicle M from the map information (step S120).

[0085] Next, the determination unit 142 determines whether a nearby vehicle has been recognized by the first recognition unit 132 (step S130). If it is determined that a nearby vehicle has been recognized, the determination unit 142 compares the camera lane lines with the map lane lines based on a first condition (step S140). If it is determined by the processing of step S130 that a nearby vehicle has not been recognized, the determination unit 142 compares the camera lane lines with the map lane lines based on a second condition different from the first condition (step S150). In other words, the determination unit 142 uses different methods for determining whether the camera lane lines and the map lane lines diverge depending on whether a nearby vehicle has been recognized and whether it has not been recognized.

[0086] Next, the determination unit 142 determines whether the camera lane markings and the map lane markings diverge from each other through the processing of step S140 or S150 (step S160). If it is determined that there is a divergence, the execution control unit 144 suppresses driving control of the host vehicle M (step S170). Suppressing driving control includes, for example, switching the currently running driving control (e.g., switching from the first driving control to the third driving control or the fourth driving control), terminating (or not starting) the currently running driving control, or lowering the automation level of the driving control. Furthermore, if it is determined in the processing of step S160 that there is no divergence, the execution control unit 144 executes driving control (or continues the currently running driving control) based on at least one of the camera lane markings recognized by the first recognition unit 132 and the map lane markings recognized by the second recognition unit 134 (step S180). This ends the processing of this flowchart.

[0087] According to the above-described embodiment, the judgment device (recognition unit 130, judgment unit 142) is equipped with a first recognition unit 132 that recognizes the surrounding situation including camera dividing lines (an example of first dividing lines) that divide the lane in which the vehicle M is traveling and surrounding vehicles present around the vehicle M based on the output of a detection device that detects the surrounding situation of the vehicle M, a second recognition unit 134 that recognizes map dividing lines (second dividing lines) that divide the lanes around the vehicle M from map information based on the position information of the vehicle M, and a judgment unit 142 that judges whether the camera dividing lines and the map dividing lines diverge.The judgment unit 142 differs in the manner of judging whether the camera dividing lines and the map dividing lines diverge when a surrounding vehicle is recognized by the first recognition unit 132 from when a surrounding vehicle is not recognized, thereby being able to more appropriately judge whether the dividing lines are correct or not depending on the situation of the road dividing lines and surrounding vehicles around the vehicle. Therefore, according to the embodiment, it is possible to further improve the continuity of operation control, which in turn can contribute to the development of a sustainable transportation system.

[0088] Furthermore, according to the embodiment, for example, when an adjacent vehicle is present around the host vehicle M, it is possible to quickly determine the deviation from the map lane markings and the accuracy of the camera lane markings using the travel trajectories of the surrounding vehicles traveling ahead of the host vehicle M or the adjacent vehicle. Therefore, it is possible to prevent interference (contact) between the host vehicle M and the adjacent vehicle, and more appropriate driving control can be performed.

[0089] The above-described embodiment can be expressed as follows. a storage medium for storing computer-readable instructions; a processor connected to the storage medium; The processor executes the computer-readable instructions to: Based on an output from a detection device that detects a surrounding situation of the host vehicle, the surrounding situation including a first dividing line that divides the lane in which the host vehicle is traveling and surrounding vehicles that exist around the host vehicle is recognized; Recognizing second lane markings that demarcate lanes around the vehicle from map information based on the position information of the vehicle; determining whether the first demarcation line and the second demarcation line deviate from each other; The manner of determining whether or not the first marking line and the second marking line deviate from each other is changed depending on whether the surrounding vehicle is recognized or not. Judgment device.

[0090] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0091] 1...vehicle system, 10...camera, 12...radar device, 14...LIDAR, 16...object recognition device, 20...communication device, 30...HMI, 40...vehicle sensor, 50...navigation device, 60...MPU, 80...driving operator, 100...automatic driving control device, 120...first control unit, 130...recognition unit, 132...first recognition unit, 134...second recognition unit, 140...action plan generation unit, 142...determination unit, 144...execution control unit, 160...second control unit, 162...target trajectory acquisition unit, 164...speed control unit, 166...steering control unit, 180...HMI control unit, 190...memory unit, 200...driving force output device, 210...brake device, 220...steering device, M...host vehicle

Claims

1. a first recognition unit that recognizes a surrounding situation including a first dividing line that divides the lane of the host vehicle and surrounding vehicles that exist around the host vehicle based on an output of a detection device that detects a surrounding situation of the host vehicle; a second recognition unit that recognizes second lane markings that demarcate lanes around the host vehicle from map information based on the position information of the host vehicle; a determination unit that determines whether the first demarcation line and the second demarcation line deviate from each other, The determination unit determines whether the first marking line and the second marking line deviate from each other in a manner different from a manner in which the first recognition unit recognizes the surrounding vehicle and a manner in which the first recognition unit does not recognize the surrounding vehicle. Judgment device.

2. the surrounding vehicles include adjacent vehicles that travel in an adjacent lane adjacent to the lane in which the host vehicle is traveling and that are present within a predetermined distance from the host vehicle; The determination unit changes the manner of the determination when the surrounding vehicles include an adjacent vehicle. The determination device according to claim 1 .

3. The determination unit is configured to more easily determine that the first marking line and the second marking line are separated when the nearby vehicle is recognized than when the nearby vehicle is not recognized. The determination device according to claim 1 .

4. When the first recognition unit recognizes the surrounding vehicle, the determination unit determines whether or not the first demarcation line and the second demarcation line diverge based on a travel path of the recognized surrounding vehicle, the surrounding vehicle being other than the adjacent vehicle and located ahead of the adjacent vehicle in the traveling direction, and the second demarcation line. The determination device according to claim 2 .

5. the determination unit sets a virtual first demarcation line based on a travel path of the nearby vehicle, and determines whether or not the set virtual first demarcation line and the second demarcation line deviate from each other. The determination device according to claim 4 .

6. When the first recognition unit recognizes the nearby vehicle, the determination unit: determining whether or not the first demarcation line and the second demarcation line deviate from each other at a position less than a predetermined distance from the host vehicle; At a position that is a predetermined distance or more from the host vehicle, it is determined whether or not a travel path of a nearby vehicle that is other than an adjacent vehicle among the nearby vehicles and is located ahead of the adjacent vehicle deviates from the second lane marking. The determination device according to claim 2 .

7. the determination unit determines whether or not the virtual first demarcation line and the second demarcation line deviate from each other at a position that is a predetermined distance or more from the host vehicle. The determination device according to claim 5 .

8. the determination unit resets a determination result of whether or not the first marking line and the second marking line will diverge when the first recognition unit no longer recognizes the adjacent vehicle after determining whether or not the first marking line and the second marking line will diverge using the nearby vehicle; The determination device according to claim 2 .

9. the determination unit acquires a travel locus of the nearby vehicle in which a deviation angle of the travel locus of the nearby vehicle from an extension direction of the second lane marking is equal to or greater than a predetermined angle, and determines whether the acquired travel locus deviates from the second lane marking. The determination device according to claim 4 .

10. the determination unit acquires deviation directions of the travel trajectories of the nearby vehicles with respect to an extension direction of the second lane marking, and determines whether or not the second lane marking deviates from a travel trajectory with a greater number of identical deviation directions. The determination device according to claim 4 .

11. the determination unit acquires a deviation direction of the travel locus of the nearby vehicle with respect to the extension direction of the second lane marking, and when the number of travel loci with the same deviation direction is the same in a plurality of different directions, does not determine whether the travel locus of the nearby vehicle and the second lane marking deviate from each other. The determination device according to claim 4 .

12. The computer Based on an output from a detection device that detects a surrounding situation of the host vehicle, the surrounding situation including a first dividing line that divides the lane in which the host vehicle is traveling and surrounding vehicles that exist around the host vehicle is recognized; recognizes second lane markings that demarcate lanes around the vehicle from map information based on the position information of the vehicle; determining whether the first demarcation line and the second demarcation line deviate from each other; The manner of determining whether or not the first marking line and the second marking line deviate from each other is changed depending on whether the surrounding vehicle is recognized or not. Judgment method.

13. On the computer, Based on an output from a detection device that detects a surrounding situation of the host vehicle, a surrounding situation including a first dividing line that divides the travel lane of the host vehicle and surrounding vehicles that exist around the host vehicle is recognized; a second dividing line that divides a lane around the vehicle from map information based on the position information of the vehicle; determining whether or not the first demarcation line and the second demarcation line deviate from each other; The manner of determining whether or not the first marking line and the second marking line deviate from each other is changed depending on whether the surrounding vehicle is recognized or not. program.

Images