Mobile object control device, mobile object control method, and program

Abstract

To implement further appropriate movement control.SOLUTION: A moving body controller of an embodiment includes: a movement control unit that implements movement control of a moving body on the basis of at least one of a first mark line that defines a movement path on which the moving body recognized by a detection device moves, and a second mark line that defines a movement path around the moving body recognized from map information; an adjustment unit that, when a recognizing condition of the first mark line and the second mark line satisfies a predetermined requirement, adjusts or shifts the positions of the first mark line and the second mark line so that the positions of the mark lines coincide with each other; and a speed determination unit that determines whether a moving speed of moving the first mark line or the second mark line is equal to or faster than a predetermined speed. When the speed determination unit determines that the moving speed is equal to or faster than the predetermined speed, if there is an obstacle in an extension direction of the first mark line, at least one of restriction of travel control and notification concerning the restriction is carried out. When it is determined that the moving speed is equal to or faster than the predetermined speed, if there is no obstacle in the extension direction of the first mark line, the travel control is continued.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a mobile object control device, a mobile object control method, and a program. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transportation systems that take into consideration vulnerable traffic participants have become more active. To achieve this, efforts are being focused on research and development to further improve traffic safety and convenience through research and development of autonomous driving technology. In this regard, there are known technologies that, when only one of the white lines on the left and right of the vehicle is detected, estimate the position of the other white line using the lane width estimated from the vehicle and surrounding vehicles, and technologies that discontinue lane-keeping driving when no white lines can be detected (see, for example, Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-148893 [Patent Document 2] Japanese Patent Application Laid-Open No. 2001-023094 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional autonomous driving technology, it cannot be said that sufficient consideration has been given to restricting or continuing the movement control of a moving object based only on lane marking information on one side, and there is still room for further study.

[0005] In order to solve the above-mentioned problems, one of the objects of the present application is to provide a mobile object control device, a mobile object control method, and a program that can execute more appropriate movement control in accordance with the surrounding conditions of the mobile object, thereby contributing to the development of a sustainable transportation system. [Means for solving the problem]

[0006] A mobile object control device, a mobile object control method, and a program according to the present invention employ the following configuration. (1): A mobile object control device according to one aspect of the present invention includes a first recognition unit that recognizes a surrounding situation including a first division line that defines a path along which the mobile object moves and obstacles present around the mobile object based on an output from a detection device that detects the surrounding situation of the mobile object; a second recognition unit that recognizes a second division line that defines a path along which the mobile object moves from map information based on position information of the mobile object; a movement control unit that controls the movement of the mobile object based on at least one of the first division line and the second division line; and a movement control unit that controls the movement of the mobile object based on the first division line and the second division line when a recognition state of the first division line and the second division line that define a path along which the mobile object is located satisfies a predetermined condition. This mobile body control device comprises an adjustment unit that moves and adjusts the position of the first demarcation line or the second demarcation line so that the positions of the line and the second demarcation line are aligned, and a speed determination unit that determines whether the movement speed when the adjustment unit moves the first demarcation line or the second demarcation line is equal to or greater than a predetermined speed, wherein the movement control unit performs at least one of restricting the movement control and notifying regarding the restriction when the speed determination unit determines that the movement speed is equal to or greater than the predetermined speed and an obstacle exists in the direction of extension of the first demarcation line, and continues the movement control when the movement speed is determined to be equal to or greater than the predetermined speed and no obstacle exists in the direction of extension of the first demarcation line.

[0007] (2): In the above aspect (1), when the first and second dividing lines are recognized on one side of the left and right dividing lines that divide the travel path as seen from the moving body, and only the second dividing line is recognized on the other side, the adjustment unit moves the position of the second dividing line so that the angle formed by the extension directions of the first and second dividing lines on the one side is within a predetermined angle.

[0008] (3): In the above aspect (2), the adjustment by the adjustment unit rotates the second demarcation line, and the movement speed is a rotation speed based on the rotation of the second demarcation line.

[0009] (4): In the above aspect (3), the moving speed is an estimated angular velocity of the moving body's own position estimated based on the rotation of the second dividing line, and the speed determination unit determines that the moving speed is equal to or greater than a predetermined speed when the degree of deviation between the estimated angular velocity and the measured angular velocity measured by a sensor that measures the angular velocity of the moving body is equal to or greater than a threshold value.

[0010] (5): In the above aspect (2), the obstacle includes another moving body present in the vicinity of the moving body, and the movement control unit performs at least one of restricting the movement control and notifying the user about the restriction when the future position of the other moving body is in the direction of extension of the first dividing line on the one side.

[0011] (6): In the above aspect (1), when the movement speed is less than a predetermined speed, the movement control unit continues the movement control regardless of whether the obstacle is present or not.

[0012] (7) A mobile object control method according to another aspect of the present invention includes a computer that recognizes a surrounding situation including a first dividing line that defines a path along which the mobile object moves and obstacles present around the mobile object based on an output from a detection device that detects the surrounding situation of the mobile object, recognizes a second dividing line that defines the path along which the mobile object moves from map information based on position information of the mobile object, performs movement control of the mobile object based on at least one of the first dividing line and the second dividing line, and performs the movement control of the mobile object when the recognition state of the first dividing line and the second dividing line that define the path along which the mobile object is located satisfies a predetermined condition. A moving object control method comprising: moving and adjusting the position of the first or second demarcation line so that the positions of the first and second demarcation lines are aligned based on the position of the moving object; determining whether the movement speed when moving the first or second demarcation line is equal to or greater than a predetermined speed; if it is determined that the movement speed is equal to or greater than the predetermined speed and an obstacle exists in the direction of extension of the first demarcation line, performing at least one of restricting the movement control and notifying the user regarding the restriction; and if it is determined that the movement speed is equal to or greater than the predetermined speed and no obstacle exists in the direction of extension of the first demarcation line, continuing the movement control.

[0013] (8) A program according to another aspect of the present invention causes a computer to recognize a surrounding situation including a first dividing line that divides a path along which a moving body moves and an obstacle present around the moving body based on an output from a detection device that detects the surrounding situation of the moving body, recognize a second dividing line that divides the path along which the moving body moves from map information based on position information of the moving body, perform movement control of the moving body based on at least one of the first dividing line and the second dividing line, and when a recognition state of the first dividing line and the second dividing line that divides the path along which the moving body is located satisfies a predetermined condition, perform the movement control of the moving body. The program moves and adjusts the position of the first or second demarcation line so that the positions of the first and second demarcation lines are aligned based on the position of a moving object, determines whether the movement speed when moving the first or second demarcation line is greater than or equal to a predetermined speed, and if it is determined that the movement speed is greater than or equal to the predetermined speed and an obstacle exists in the direction of extension of the first demarcation line, performs at least one of restricting the movement control and notifying regarding the restriction, and if it is determined that the movement speed is greater than or equal to the predetermined speed and no obstacle exists in the direction of extension of the first demarcation line, continues the movement control. [Effects of the Invention]

[0014] According to the above aspects (1) to (8), more appropriate movement control can be executed in accordance with the surrounding conditions of the moving object. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a configuration diagram of a vehicle system 1 including a mobile object control device according to an embodiment. [Figure 2] 2 is a functional configuration diagram of a first control unit 120 and a second control unit 160. FIG. [Figure 3] 1 is a diagram for explaining driving control of a vehicle M in a first scene. FIG. [Figure 4] FIG. 10 is a diagram for explaining driving control of the host vehicle M in a second scene. [Figure 5]FIG. 10 is a diagram showing an example of the result of moving the map division lines ML1 and ML2 so as to align them with the extension direction of the camera division line CL. [Figure 6] FIG. 10 is a diagram for explaining driving control of the host vehicle M in a third scene. [Figure 7] FIG. 10 is a diagram for explaining driving control of the host vehicle M in a fourth scene. [Figure 8] FIG. 10 is a diagram for explaining an angular velocity deviation. [Figure 9] 4 is a flowchart illustrating an example of a flow of an operation control process in the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, with reference to the drawings, embodiments of a mobile object control device, a mobile object control method, and a program according to the present invention will be described. Hereinafter, a vehicle will be used as an example of a mobile object, and an embodiment in which the mobile object control device is applied to an autonomous vehicle will be described. Autonomous driving refers to automatically controlling one or both of the steering and speed of a vehicle to perform driving control. The above-mentioned driving control may include various driving controls, such as Automated Lane Change (ALC), Lane Keeping Assistance System (LKAS), Adaptive Cruise Control System (ACC), Traffic Jam Pilot (TJP), and Collision Mitigation Brake System (CMBS). Furthermore, an autonomous vehicle may be manually controlled by a user (e.g., a passenger) of the vehicle (so-called manual driving). Examples of mobile objects include, in addition to vehicles, ships capable of traveling on defined routes such as hovercrafts, aircraft capable of traveling on roads, and stand-up vehicles with power units.

[0017] [Overall configuration] 1 is a configuration diagram of a vehicle system 1 including a mobile object control device according to an embodiment. The vehicle (hereinafter referred to as host vehicle M) on which the vehicle system 1 is mounted is, for example, a two-wheeled, three-wheeled, four-wheeled, or micromobility vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination of these. The electric motor operates using power generated by a generator connected to the internal combustion engine, or discharged power from a battery (storage battery) such as a secondary battery or a fuel cell.

[0018] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a light detection and ranging (LIDAR) device 14, an object recognition device 16, a communication device 20, a human machine interface (HMI) device 30, vehicle sensors 40, a navigation device 50, a map positioning unit (MPU) device 60, a driving operation control device 80, an automatic driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are connected to each other via multiplexed communication lines such as a controller area network (CAN) communication line, serial communication lines, a wireless communication network, etc. Note that the configuration shown in FIG. 1 is merely an example, and some of the configuration may be omitted, or other configurations may be added. The combination of the camera 10, the radar device 12, the LIDAR device 14, and the object recognition device 16 is an example of a "detection device DD." The HMI device 30 is an example of an "output device." The automatic driving control device 100 is an example of a "mobile object control device."

[0019] The camera 10 is a digital camera that uses a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to any location of the host vehicle M in which the vehicle system 1 is installed. When capturing an image of the front, the camera 10 is attached to the top of the front windshield, the back of the rearview mirror, the front of the vehicle body, etc. When capturing an image of the rear, the camera 10 is attached to the top of the rear windshield, the back door, etc. When capturing an image of the side, the camera 10 is attached to a door mirror, etc. The camera 10 periodically and repeatedly captures images of the surroundings of the host vehicle M, for example. The camera 10 may be a stereo camera.

[0020] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by surrounding objects (reflected waves) to detect at least the position (distance and direction) of the objects. The radar device 12 is attached to any location on the vehicle M. The radar device 12 may detect the position and speed of the objects using an FM-CW (Frequency Modulated Continuous Wave) method.

[0021] The LIDAR 14 irradiates light around the vehicle M and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time between emitting and receiving the light. The irradiated light is, for example, a pulsed laser beam. The LIDAR 14 is attached to any location on the vehicle M.

[0022] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the camera 10, the radar device 12, and the LIDAR 14 to recognize the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition results to the automatic driving control device 100. Alternatively, the object recognition device 16 may output the detection results from the camera 10, the radar device 12, and the LIDAR 14 directly to the automatic driving control device 100. In that case, the object recognition device 16 may be omitted from the configuration of the vehicle system 1 (detection device DD).

[0023] The communication device 20 communicates with, for example, other vehicles in the vicinity of the vehicle M, terminal devices of users using the vehicle M, or various server devices, using networks such as a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), LAN (Local Area Network), WAN (Wide Area Network), or the Internet.

[0024] The HMI 30 outputs various information to the occupants of the vehicle M and accepts input operations by the occupants. The HMI 30 includes, for example, various display devices, speakers, buzzers, touch panels, switches, keys, microphones, and the like.

[0025] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the host vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the yaw rate (for example, the rotational angular velocity around a vertical axis passing through the center of gravity of the host vehicle M), and a direction sensor that detects the orientation of the host vehicle M. The vehicle sensor 40 may also be provided with a position sensor that detects the position of the host vehicle M. The position sensor is an example of a "position measurement unit." The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. The position sensor may also be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver 51 of the navigation device 50. The vehicle sensor 40 may derive the speed of the host vehicle M from the difference (i.e., distance) of the position information at a predetermined time in the position sensor. The results detected by the vehicle sensor 40 are output to the automatic driving control device 100.

[0026] The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as a hard disk drive (HDD) or a flash memory. The GNSS receiver 51 identifies the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be identified or supplemented by an inertial navigation system (INS) that uses the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, etc. The GNSS receiver 51 may be provided in the vehicle sensor 40. The navigation HMI 52 may share some or all of the components with the HMI 30 described above. The route determination unit 53 determines, for example, a route (hereinafter, a route on a map) from the position of the vehicle M identified by the GNSS receiver 51 (or an arbitrary input position) to a destination input by the occupant using the navigation HMI 52, with reference to the first map information 54. The first map information 54 is information that represents road shapes using, for example, links that indicate roads (an example of a travel route) and nodes connected by the links. The first map information 54 may also include POI (Point Of Interest) information and the like. The route on the map is output to the MPU 60. The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may transmit the current position and destination to a navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server. The navigation device 50 outputs the determined route on the map to the MPU 60.

[0027] The MPU 60 includes, for example, a recommended lane determination unit 61, and stores second map information 62 in a storage device such as an HDD or flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, by dividing it into 100 m intervals in the vehicle travel direction), and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 determines, for example, which lane from the left the vehicle should travel in. When there is a branch point on the route on the map, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel on a reasonable route to the branch point.

[0028] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, the number of lanes (number of travel routes), the type and shape of road dividing lines (hereinafter referred to as dividing lines), information on the center of lanes, and road boundary information. The second map information 62 may also include information on whether the road boundary is a boundary (physical boundary) that includes a structure that prevents a vehicle from passing (including crossing or contacting). Examples of physical boundaries include guardrails, curbs, medians, fences, etc. The term "impassable" may include the presence of a low level step that allows passage if unusual vehicle vibrations are tolerated. The second map information 62 may also include road shape information, traffic regulation information, address information (address and postal code), facility information, parking information, telephone number information, etc. Road shape information includes, for example, the curvature (which may be referred to as the radius of curvature; the same applies below), width, gradient, etc. The second map information 62 may be updated as needed by the communication device 20 communicating with an external device. The first map information 54 and the second map information 62 may be provided as an integrated piece of map information. The map information may also be stored in the storage unit 190.

[0029] The driving operators 80 include, for example, a steering wheel, an accelerator pedal, and a brake pedal. The driving operators 80 may also include a shift lever, a variable steering wheel, a joystick, or other operators. Each operator of the driving operators 80 is equipped with an operation detection unit that detects, for example, the amount of operation of the operator by the occupant or whether or not the operator is operated. The operation detection unit detects, for example, the steering angle and steering torque of the steering wheel, the amount of depression of the accelerator pedal and the brake pedal, etc. The operation detection unit then outputs the detection results to the automatic driving control device 100 or one or both of the driving force output device 200, the brake device 210, and the steering device 220.

[0030] The automatic driving control device 100 executes various types of driving control associated with automatic driving for the host vehicle M. The automatic driving control device 100 includes, for example, a first control unit 120, a second control unit 160, an HMI control unit 180, and a storage unit 190. The first control unit 120, the second control unit 160, and the HMI control unit 180 are each realized by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Furthermore, some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or an SOC (System On Chip), or may be realized by a combination of software and hardware. The above-mentioned program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as an HDD or flash memory of the automatic driving control device 100, or may be stored in a removable storage medium such as a DVD, CD-ROM, or memory card, and installed in the storage device of the automatic driving control device 100 by inserting the storage medium (non-transitory storage medium) into a drive device, card slot, etc.

[0031] The storage unit 190 may be realized by the various storage devices described above, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a ROM (Read Only Memory), or a RAM (Random Access Memory). The storage unit 190 stores, for example, various types of information, programs, and the like in the embodiments. The storage unit 190 may also store map information (for example, the first map information 54 and the second map information 62).

[0032] FIG. 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 140. The first control unit 120, for example, implements functions based on AI (Artificial Intelligence) and functions based on a predefined model in parallel. For example, the function of "recognizing intersections" may be implemented by concurrently executing intersection recognition using deep learning or the like and recognition based on predefined conditions (such as the presence of traffic lights and road markings that can be pattern matched), and then scoring and comprehensively evaluating both. This ensures the reliability of autonomous driving. The first control unit 120 also executes control related to autonomous driving of the host vehicle M based on instructions from, for example, the MPU 60, the HMI control unit 180, or the like.

[0033] The recognition unit 130 recognizes the surrounding situation of the host vehicle M based on the recognition results of the detection device DD (information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16). For example, the recognition unit 130 recognizes the status of objects present around the host vehicle M (within a predetermined distance), such as their position, speed, and acceleration. Objects include other vehicles (an example of other moving objects), traffic participants such as pedestrians and bicycles, and physical boundaries that define roads (travel paths). The position of an object is recognized as a position on an absolute coordinate system with a representative point of the host vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or by a represented area. For example, when the object is a moving object such as another vehicle, the "state" of the object may include the acceleration or jerk of the moving object, or the "behavioral state" (for example, whether the other vehicle is changing lanes or about to change lanes).

[0034] The recognition unit 130 also recognizes, for example, stop lines, red lights, toll booths, other road events, markings (speed limits) on the road, and road signs indicating speed limits. The recognition unit 130 also includes, for example, a first recognition unit 132 and a second recognition unit 134. Details of these functions will be described later.

[0035] The behavior plan generation unit 140 generates a behavior plan for driving the host vehicle M by autonomous driving based on the recognition results of the recognition unit 130, etc. For example, the behavior plan generation unit 140 generates a target trajectory for the host vehicle M to travel automatically (without driver operation) in the future, so that the host vehicle M can respond to the surrounding conditions of the host vehicle M, while essentially traveling in the recommended lane determined by the recommended lane determination unit 61, based on the recognition results by the recognition unit 130 and the surrounding road shapes based on the current position of the host vehicle M acquired from map information, etc. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequential arrangement of points (trajectory points) to be reached by the host vehicle M. The trajectory points are points to be reached by the host vehicle M at predetermined travel distances (e.g., on the order of several meters) along the road, and separately, target speeds and target accelerations for predetermined sampling times (e.g., on the order of a few tenths of a second) are generated as part of the target trajectory. Alternatively, the trajectory points may be positions to be reached by the host vehicle M at the sampling times for each predetermined sampling time. In this case, the target speed and target acceleration information are expressed as the interval between trajectory points.

[0036] The action plan generation unit 140 may set an autonomous driving event when generating the target trajectory. Examples of the event include a constant speed driving event in which the host vehicle M drives in the same lane at a constant speed, a following driving event in which the host vehicle M follows another vehicle that is within a predetermined distance (for example, within 100 m) ahead of the host vehicle M and is closest to the host vehicle M, a lane change event in which the host vehicle M changes lanes from the host vehicle's own lane to an adjacent lane, a branching event in which the host vehicle M branches off into a lane on the destination side at a road branching point, a merging event in which the host vehicle M merges into a main lane at a merging point, a takeover event in which the autonomous driving is terminated and the host vehicle M switches to manual driving, and so on. Examples of the event may also include an overtaking event in which the host vehicle M temporarily changes lanes to an adjacent lane, overtakes a leading vehicle in the adjacent lane, and then changes lanes back to the original lane, and an avoidance event in which the host vehicle M performs at least one of braking and steering to avoid an obstacle ahead of the host vehicle M.

[0037] Furthermore, the behavior plan generation unit 140 may change an event already determined for the current section to another event or set a new event for the current section, depending on the surrounding conditions of the host vehicle M recognized while the host vehicle M is traveling. Furthermore, the behavior plan generation unit 140 may change an event already set for the current section to another event or set a new event for the current section, depending on the operation of the occupant on the HMI 30. The behavior plan generation unit 140 generates a target trajectory according to the set event.

[0038] The behavior plan generation unit 140 also includes, for example, a deviation determination unit 142, an adjustment unit 144, a speed determination unit 146, and a travel control unit 148. The travel control unit 148, the second control unit 160, and the HMI control unit 180 are examples of a "mobility control unit." Details of these functions will be described later.

[0039] The second control unit 160 controls the traveling driving force output device 200, the braking device 210, and the steering device 220 so that the host vehicle M passes through the target trajectory generated by the action plan generation unit 140 at the scheduled time.

[0040] The second control unit 160 includes, for example, a target trajectory acquisition unit 162, a speed control unit 164, and a steering control unit 166. The target trajectory acquisition unit 162 acquires information on the target trajectory (trajectory points) generated by the action plan generation unit 140 and stores it in a memory (not shown). The speed control unit 164 controls the driving force output device 200 or the brake device 210 based on a speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized by, for example, a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road ahead of the host vehicle M and feedback control based on the deviation from the target trajectory.

[0041] Returning to FIG. 1 , the HMI control unit 180 notifies (announces) predetermined information to the occupant via the HMI 30. The predetermined information includes, for example, information related to the traveling of the host vehicle M, such as information related to the state of the host vehicle M and information related to limitations on driving control (cruising control). The information related to the state of the host vehicle M includes, for example, the speed of the host vehicle M, engine speed, and shift position. The information related to driving control includes, for example, whether or not driving control is being performed by autonomous driving, information inquiring about whether or not to start autonomous driving, information related to the driving control status by autonomous driving, information related to the automation level, and information prompting the occupant to drive when switching from autonomous driving to manual driving. The predetermined information may also include information related to the surrounding situation recognized by the detection device DD. The predetermined information may also include information unrelated to the traveling of the host vehicle M, such as television programs, content (e.g., movies) stored on a storage medium such as a DVD, and the like. The predetermined information may also include, for example, information related to the current location and destination during autonomous driving, and the remaining amount of fuel in the host vehicle M. The HMI control unit 180 may output the information received by the HMI 30 to the communication device 20, the navigation device 50, the first control unit 120, and the like.

[0042] The HMI control unit 180 may also cause the HMI 30 to output inquiry information for the occupant, processing results by the first control unit 120 and the second control unit 160, etc. The HMI control unit 180 may also transmit various pieces of information to be output by the HMI 30 to a terminal device used by the occupant of the vehicle M via the communication device 20.

[0043] Traveling drive force output device 200 outputs a traveling drive force (torque) to the drive wheels for the vehicle to travel. Traveling drive force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, etc., and an ECU (Electronic Control Unit) that controls these. The ECU controls the above components according to information input from second control unit 160 or information input from the accelerator pedal of driving operator 80.

[0044] The braking device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or information input from the brake pedal of the driving operation device 80, so that a braking torque corresponding to the braking operation is output to each wheel. The braking device 210 may include a backup mechanism that transmits hydraulic pressure generated by operation of the brake pedal to the cylinder via a master cylinder. Note that the braking device 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic braking device that controls an actuator according to information input from the second control unit 160 to transmit hydraulic pressure from the master cylinder to the cylinder.

[0045] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor changes the direction of the steered wheels by, for example, applying a force to a rack and pinion mechanism. The steering ECU drives the electric motor to change the direction of the steered wheels in accordance with information input from the second control unit 160 or information input from the steering wheel of the driving operator 80.

[0046] [Recognition and Action Plan Generation] Next, the functions of the recognition unit 130 (mainly the first recognition unit 132 and the second recognition unit 134) and the behavior plan generation unit 140 (mainly the deviation determination unit 142, the adjustment unit 144, the speed determination unit 146, and the traveling control unit 148) will be described in detail. Note that, below, the contents of the driving control (travel control) of the host vehicle M using the functions of the recognition unit 130 and the behavior plan generation unit will be described in several scenes.

[0047] [Scene 1] FIG. 3 is a diagram illustrating driving control of the host vehicle M in a first scene. The example of FIG. 3 shows lane markings CL1 and CL2 recognized by the detection device DD and lane markings ML1 and ML2 obtained from map information (e.g., second map information 62) based on the position information of the host vehicle M. In the map information, lane L1 is demarcated by lane markings ML1 and ML2. In the example of FIG. 3, lane markings CL1 and CL2 are an example of a "first lane marking," and lane markings ML1 and ML2 are an example of a "second lane marking." In the following, lane markings CL1 and CL2 may be referred to as "camera lane markings CL1 and CL2," and lane markings ML1 and ML2 may be referred to as "map lane markings ML1 and ML2." In addition, when the camera lane markings CL1 and CL2 are not distinguished from each other, they may be simply referred to as "camera lane markings CL." When the map lane markings ML1 and ML2 are not distinguished from each other, they may be simply referred to as "map lane markings ML."

[0048] The first recognition unit 132 recognizes the surrounding conditions of the host vehicle M based on the output of the detection device DD that detects the surrounding conditions (external world) of the host vehicle M. For example, the first recognition unit 132 recognizes left and right camera dividing lines CL1 and CL2 that divide the travel lane (travel path, lane L1) of the host vehicle M based on an image captured by the camera 10 (hereinafter referred to as a camera image). Furthermore, when an adjacent lane adjacent to lane L1 exists, the first recognition unit 132 may recognize the camera dividing line that divides the adjacent lane.

[0049] For example, the first recognition unit 132 analyzes the camera image, extracts edge points in the image that have a large difference in brightness from adjacent pixels, and recognizes the camera lane marking lines CL1 and CL2 on the image plane by connecting the edge points. Furthermore, the first recognition unit 132 converts the positions of the camera lane marking lines CL1 and CL2 into a vehicle coordinate system (for example, the XY plane coordinates in FIG. 3) based on the position of the representative point of the host vehicle M.

[0050] The first recognition unit 132 may also recognize, for example, the curvature of the camera lane markings CL1 and CL2. The camera lane markings CL1 and CL2 may be recognized or corrected based on the output of a detection device other than the camera 10 (for example, the radar device 12 or the LIDAR 14). The first recognition unit 132 may also recognize the amount of curvature change of the camera lane markings CL1 and CL2. The amount of curvature change is, for example, the time rate of change in the curvature of the camera lane markings CL1 and CL2 recognized by the camera 10 at a distance x [m] forward as viewed from the host vehicle M. The first recognition unit 132 may also recognize the curvature or amount of curvature change of the lane markings CL1 and CL2 by averaging the curvatures or amounts of curvature change of the camera lane markings CL1 and CL2. The camera lane markings CL1 and CL2 may be recognized or corrected based on the output of a detection device other than the camera 10 (for example, the radar device 12 or the LIDAR 14).

[0051] The first recognition unit 132 also recognizes other objects (for example, physical boundaries, other vehicles (an example of other moving objects), etc.) present in the vicinity of the host vehicle M. The first recognition unit 132 may also recognize, among the recognized objects, an object with which the host vehicle M may come into contact in future travel as an obstacle. For example, when a contact margin value based on the position and speed VM of the host vehicle M and the position and speed of the object is less than a threshold, the first recognition unit 132 determines that there is a possibility of contact and recognizes the object as an obstacle. The contact margin value is, for example, a time to collision (TTC), which is derived by dividing the relative distance between the host vehicle M and the other vehicle by the relative speed. The first recognition unit 132 may also recognize an object as an obstacle when it is recognized that the future position (position after a predetermined time) of the object is located on the driving lane of the host vehicle M based on the position and speed of the object.

[0052] The second recognition unit 134 recognizes lane markings around the host vehicle M from map information based on the position of the host vehicle M detected by, for example, the vehicle sensor 40 or the GNSS receiver 51. For example, the second recognition unit 134 refers to map information based on the position information of the host vehicle M and recognizes map marking lines ML1 and ML2 that exist in the traveling direction of the host vehicle M or in directions in which the host vehicle M can travel.

[0053] The second recognition unit 134 also recognizes map division lines ML1 and ML2 as division lines that divide lane L1. If an adjacent lane exists next to lane L1, the second recognition unit 134 may also recognize a division line that divides the adjacent lane. The second recognition unit 134 also recognizes the curvature or curvature change amount of each of the map division lines ML1 and ML2 from the second map information 62. The second recognition unit 134 may also average the curvature or curvature change amount of each of the map division lines ML1 and ML2 to recognize the curvature or curvature change amount of the lane divided by the map division lines.

[0054] The deviation determination unit 142 determines whether there is a deviation between the camera lane lines CL1 and CL2 recognized by the first recognition unit 132 and the map lane lines ML1 and ML2 recognized by the second recognition unit 134. For example, the deviation determination unit 142 derives the degree of deviation between the lane lines CL1 and ML1 located closest to the left of the host vehicle M, and the degree of deviation between the lane lines CL2 and ML2 located closest to the right of the host vehicle M. If the derived degree of deviation is equal to or greater than a threshold, the deviation determination unit 142 determines that there is a deviation between the camera lane lines CL and the map lane lines ML, and if the degree of deviation is less than the threshold, the deviation determination unit 142 determines that there is no deviation. The determination of whether there is a deviation may be performed repeatedly at a predetermined timing or at a predetermined interval.

[0055] For example, the deviation determination unit 142 superimposes the camera lane lines CL1 and CL2 and the map lane lines ML1 and ML2 on the plane of the vehicle coordinate system (XY plane) based on the position of the representative point of the vehicle M. When determining the lane lines to be compared (lane lines CL1 and ML1, and lane lines CL2 and ML2), the deviation determination unit 142 determines that the lane lines are deviating if the deviation degree of at least one lane line is equal to or greater than a threshold, and determines that the lane lines are not deviating if all lane lines are less than the threshold. The deviation degree is, for example, the degree of deviation (deviation distance, travel lane width direction deviation) in the road width direction (travel lane width direction, lateral direction, Y-axis direction in the figure). In the example of Figure 3, the deviation determination may be performed using the average value of the lateral position deviation D1 between the marking lines CL1 and ML1 and the lateral position deviation D2 between the marking lines CL2 and ML2, or the deviation determination may be performed using the maximum or minimum value of the deviation amounts D1 and D2.

[0056] Furthermore, the degree of deviation may be, for example, the degree of the angle (deviation angle) formed by the two marking lines being compared, instead of (or in addition to) the amount of lateral positional deviation described above. In the example of Fig. 3, the average value of the angle θ1 formed by marking lines CL1 and ML1 and the angle θ2 formed by marking lines CL2 and ML2 may be used, or the maximum or minimum value of the angles θ1 and θ2 may be used.

[0057] Furthermore, the degree of deviation may be the degree (magnitude) of difference in curvature change between the lane markings, instead of (or in addition to) the lateral position deviation or the angle formed by the lane markings. The curvature change is mainly used when the lane is a curved road. For example, the deviation determination unit 142 may use the average value of the difference in curvature change between lane markings CL1 and ML1 and the difference in curvature change between lane markings CL2 and ML2, or may use the maximum or minimum value of the difference. The deviation determination unit 142 may also use the difference between the average value of the curvature change between lane markings CL1 and CL2 and the average value of the curvature change between lane markings ML1 and ML2. The difference between the curvature change of lane markings L1 recognized from the camera image and the curvature change of lane markings recognized from map information may also be used.

[0058] When determining whether a camera lane marking deviates from a map lane marking, the deviation determination unit 142 may determine whether the camera lane marking is erroneously recognized based on one or both of the amount of curvature change of the camera lane marking detected by the recognition unit 130 and the angle between the camera lane marking and the map lane marking. In this case, the deviation determination unit 142 determines whether the camera lane marking is erroneously recognized when, for example, the direction of change in the amount of curvature and the direction of change in the angle are the same and the amount of curvature change and the angle increase with the distance from the host vehicle M. This allows for more accurate determination of erroneous recognition of a camera lane marking when traveling through a lane change section such as a curved road.

[0059] When the recognition state of the camera lane lines CL and map lane lines ML that demarcate the lane L1 in which the host vehicle M is located by the recognition units (first recognition unit 132, second recognition unit 134) satisfies a predetermined condition, the adjustment unit 144 adjusts the positions of the camera lane lines CL or map lane lines ML by moving them so that the positions of the camera lane lines CL and map lane lines ML match (including a predetermined error range in addition to perfect match) relative to the host vehicle M. The speed determination unit 146 determines whether the moving speed when the adjustment unit 144 moves the camera lane lines CL or map lane lines ML is equal to or greater than a predetermined speed. Specific examples of the functions of the adjustment unit 144 and the speed determination unit 146 will be described later.

[0060] The driving control unit 148 determines driving control (driving control) for the vehicle M based on the recognition results of the first recognition unit 132 and the second recognition unit 134 and the processing results of the deviation determination unit 142, the adjustment unit 144, and the speed determination unit 146, and generates a target trajectory based on the determined driving control. "Determining driving control" may include, for example, determining the content (type) of driving control and determining whether to execute (limit) driving control. Furthermore, "executing driving control" may include, for example, continuing driving control that is already being executed, in addition to switching and executing the content of driving control. Controlling driving control may include not only not executing driving control, but also lowering the automation level of driving control.

[0061] Here, the driving control executed by the driving control unit 148 includes at least a first driving control and a second driving control. The first driving control is, for example, driving control that executes at least steering control of the steering or speed of the host vehicle M based on at least one of the camera lane markings CL and the map lane markings ML. For example, in the case of ALC control, the first driving control generates a driving trajectory for the host vehicle M to change lanes from a driving lane (e.g., lane L1) to a lane of a destination lane (destination), and drives the host vehicle M so that a representative point of the host vehicle M travels on a trajectory along the generated driving trajectory. In addition, in the case of LKAS control, the first driving control drives the host vehicle M so that a representative point of the host vehicle M passes through the center of the lane marked by the lane markings. Note that, in the first driving control, for example, if the camera recognition accuracy is below a threshold, driving control may be performed with priority given to the map lane markings ML, and if the map information is old (e.g., the map update date is earlier than a predetermined date and time), driving control may be performed with priority given to the camera lane markings CL.

[0062] The second driving control is, for example, driving control that executes at least steering control of the steering or speed of the host vehicle M based on an object (e.g., a physical boundary, another vehicle) recognized by the first recognition unit 132. The second driving control, for example, identifies the position of a lane based on the physical boundary or the position of another vehicle, and drives the host vehicle M so that a representative point of the host vehicle M runs in the center of the identified lane. In addition, the second driving control drives the host vehicle M so that a representative point of the host vehicle M runs on a trajectory that follows the running trajectory of the other vehicle.

[0063] Furthermore, the driving control may include a plurality of driving control functions with different automation levels (an example of the degree of automation). The automation levels may include, for example, a first level, a second level with a lower degree of automation of driving control than the first level, and a third level with a lower degree of automation of driving control than the second level. The automation levels may also include a fourth level with a lower degree of automation of driving control than the third level. Here, the automation level may be a level determined by standardized information, laws and regulations, or an index value set independently. Therefore, the types, contents, and number of automation levels are not limited to the following examples. A low degree of automation of driving control means, for example, a low automation rate in driving control and a large (heavy) task assigned to the driver. A low degree of automation of driving control means a low degree of control of the steering or acceleration / deceleration of the host vehicle M by the automatic driving control device 100 (a high degree of need for the driver to intervene in steering or acceleration / deceleration operations). Tasks assigned to the driver include, for example, monitoring the surroundings of the host vehicle M and operating driving controls. The operation of the driving controls includes, for example, the driver gripping the steering wheel (hereinafter referred to as a hands-on state). The tasks assigned to the driver are, for example, tasks for the occupant (driver tasks) necessary to maintain the autonomous driving of the host vehicle M. Therefore, if the occupant is unable to perform the assigned tasks, the automation level will be reduced. For example, the first level of driving control may include driving controls such as ALC, LKAS, ACC, TJP, and CMBS. Furthermore, the second or third level of driving control may include driving controls such as ALC, LKAS, ACC, and CMBS. The fourth level of driving control may include manual driving. Furthermore, the fourth level of driving control may include driving controls such as ACC and CMBS. Of the first to fourth levels, the first level has the highest degree of automation of driving control, and the fourth level has the lowest degree of automation of driving control.

[0064] Furthermore, at the first level, no tasks are assigned to the occupant (the tasks assigned to the driver are the lightest), and therefore, for example, driving control is permitted when the driver of the host vehicle M is not gripping the steering wheel (hereinafter referred to as a hands-off state). At the second level, a task assigned to the driver is, for example, monitoring the surroundings (particularly the front) of the host vehicle M. At the third level, a task assigned to the driver is, for example, monitoring the surroundings of the host vehicle M and being in a hands-on state. At the fourth level, a task assigned to the driver is, for example, monitoring the surroundings of the host vehicle M and being in a hands-on state, as well as operating the driving operator 80 to control the steering and speed of the host vehicle M. In other words, at the fourth level, the occupant can immediately take over driving, and the tasks assigned to the driver are the most severe. The content of driving control and the tasks assigned to the occupant at each automation level are not limited to the examples described above. The automatic driving control device 100 executes driving control at one of first to fourth levels based on the surrounding conditions of the vehicle M and the task being performed by the occupant.

[0065] For example, the driving control unit 148 executes the first driving control when the deviation determination unit 142 determines that the camera lane markings CL and the map lane markings ML do not deviate from each other, and executes the second driving control when the deviation determination unit 142 determines that the camera lane markings CL and the map lane markings ML deviate from each other. Furthermore, the driving control unit 148 may, for example, execute control to switch from the first driving control to the second driving control when a predetermined condition is satisfied, or may execute control to terminate driving control of the vehicle M and switch to manual driving by the occupant. Furthermore, the driving control unit 148 may switch the automation level depending on the surrounding circumstances and the type of driving control.

[0066] Furthermore, during execution of driving control in a state in which the camera lane markings CL and the map lane markings ML are not deviated from each other, a deviation amount (a deviation of a predetermined deviation distance or deviation angle or less) that is not determined to be deviated from the camera lane markings CL and the map lane markings ML may occur due to changes in the behavior of the vehicle M while it is moving or changes in the road. In such a case, the adjustment unit 144, for example, moves one of the lane markings (e.g., the map lane marking) so that the position of the camera lane markings CL and the position of the map lane markings ML are aligned (within a predetermined error range). This movement includes movement in the lateral direction of the travel path (travel path width direction) and rotational movement that changes the angle with respect to the reference direction. Furthermore, the adjustment unit 144 increases the travel speed as the deviation amount increases. This allows the lane markings to be aligned in a short time and driving control to be continued.

[0067] In addition, when the driving control unit 148 changes the content of driving control or switches the automation level to restrict driving control, it may cause the HMI control unit 180 to notify the occupants of information (images or audio) regarding the restriction of driving control.

[0068] [Second Scene] Next, driving control of the host vehicle M in the second scene will be described. Fig. 4 is a diagram for explaining driving control of the host vehicle M in the second scene. In Fig. 4, the host vehicle M is traveling at a speed VM in a lane L1 defined by map division lines ML1 and ML2. Furthermore, at a time prior to the second scene, the deviation determination unit 142 determined that there was no deviation between the left and right camera division lines CL of the host vehicle M and the map division lines ML, and driving control such as LKAS based on the map division lines ML1 and ML2 was being executed by the first driving control.

[0069] Here, the recognition state of the left and right lane markings of lane L1 as viewed from vehicle M in the second scene is assumed to be a state in which camera marking CL1 and map marking ML1 are recognized on one side (the left side in the figure), and only map marking ML2 is recognized on the other side (the right side in the figure). In other words, in the second scene, camera marking CL2 on the right side as viewed from vehicle M is not recognized (recognition lost) by the first recognition unit 132 due to poor recognition accuracy or rubbing of the lane markings. The second scene also shows a situation in which there is a deviation between the recognized camera marking CL1 and map marking ML1. The deviation amount (deviation angle θ3) is assumed to be small enough that it is not determined that there is a deviation. The second scene described above is an example of a case in which the recognition state of camera marking CL and map marking ML satisfies a predetermined condition.

[0070] In this case, the adjustment unit 144, for example, moves the map division line ML1 to match the position of the camera division line CL1, and the driving control unit 148 continues driving the vehicle based on the lane divided by the moved map division line ML1.

[0071] Specifically, first, the adjustment unit 144 moves the position of the map demarcation line ML1 so that the angle θ3 formed by the extension directions of the camera demarcation line CL1 and the map demarcation line ML1 on the left side as viewed from the vehicle M is within a predetermined angle. Specifically, the adjustment unit 144 rotates the map demarcation line ML1 to the left in the drawing so that the angle θ3 is within the predetermined angle.

[0072] The adjustment unit 144 also rotates and moves the map division line ML2 on the right side of the lane L1 in the same manner as the map division line ML1. FIG. 5 is a diagram showing an example of the result of moving the map division lines ML1 and ML2 to align them with the extension direction of the camera division line CL. The adjustment unit 144 moves the map division line ML2 to match the moved position of the map division line ML1 so that the map division lines ML1 and ML2 are positioned parallel to each other. The adjustment unit 144 may, for example, move the map division lines ML1 and ML2 simultaneously. This makes it possible to adjust the position while maintaining the lane width of the lane L1. The adjustment unit 144 also acquires the movement speed when moving the map division line ML1. The movement speed in the second scene is the rotational angular velocity when the map division line ML1 is rotated.

[0073] The speed determination unit 146 determines whether the movement speed (rotational angular velocity) when the map division line ML1 is moved by the adjustment unit 144 is equal to or greater than a predetermined speed. When the speed determination unit 146 determines that the movement speed is not equal to or greater than the predetermined speed (is less than the predetermined speed), the traveling control unit 148 continues the traveling control currently being performed by the host vehicle M regardless of whether there are obstacles in the vicinity. Furthermore, when the movement speed is equal to or greater than the predetermined speed, the host vehicle M travels differently depending on whether there are obstacles in the vicinity of the host vehicle M or not, as shown in the third and fourth scenes described below.

[0074] [Scene 3] FIG. 6 is a diagram illustrating driving control of the host vehicle M in the third scenario. The third scenario differs from the second scenario in that other vehicles m1 to m3 are present around the host vehicle M. The following description focuses on the differences from the second scenario, and omits a description of the same content as in the second scenario. The same applies to the fourth scenario, which will be described later. The third scenario illustrates a state in which the map division lines ML1 and ML2 have been moved to match the position of the camera division line CL1, and the orientation of the host vehicle M has been changed to match the positions (extension direction) of the moved map division lines ML1 and ML2. In the third scenario, the first recognition unit 132 recognizes the other vehicles m1 and m2 as obstacles based on the position and speed MV of the host vehicle M and the respective positions and speeds Vm1 to Vm3 of the other vehicles m1 to m3. Note that "when other vehicles m1 and m2 are recognized as obstacles" may include, for example, not only when they are recognized as obstacles at present, but also when they are recognized as obstacles in the near future (e.g., within a few seconds). For example, other vehicles m1 and m2 are recognized as obstacles when the future positions of other vehicles m1 and m2 based on the positions and velocities Vm1 and Vm2 of other vehicles m1 and m2 are in the extension direction of camera lane marking CL1, the extension direction of map lane marking ML1 that has been moved in accordance with camera lane marking CL1, or the extension direction of the lane defined by the moved map lane marks ML1 and ML2. Furthermore, other vehicles m1 and m2 may be recognized as obstacles when the predicted future trajectories of other vehicles m1 and m2 interfere with the predicted future trajectory of host vehicle M.

[0075] In a third scene, when the movement speed for aligning the position of the map division line ML1 with the position of the camera division line CL1 is equal to or greater than a predetermined speed, and when an obstacle (in the example of Figure 6, other vehicles m1, m2) is present around the vehicle M, the driving control unit 148 performs at least one of restricting driving control (driving control) and notifying the driver of the restriction on driving control.

[0076] Limiting driving control includes terminating (canceling) driving control currently being performed or lowering the automation level (reducing the degree of automation of driving control). Notifications regarding driving control limitations are executed by the HMI control unit 180 in response to instructions from the driving control unit 148 to the HMI control unit 180. Notifications regarding driving control may include, for example, a notification of the content of the driving control to be limited (or changed), as well as a notification that an obstacle is present in the direction of travel (forward) of the vehicle M and a notification prompting the occupant to change their state according to the automation level (e.g., prompting them to take a hands-on state). Notifications may also include at least one of notification by image display and notification by audio (alarm) output. For example, the driving control unit 148 may first limit driving control or issue the above notification, and then issue the other notification a predetermined time later, or may issue both simultaneously. Furthermore, the driving control unit 148 may issue one or both of the driving control limitations and the above notification depending on road conditions and the speed and amount of movement of lane markings. Furthermore, the presence or absence of notification and the type of notification (image only, audio only, or both) may be set by the occupant of the vehicle M. This allows for more appropriate timing of driving control restrictions and notifications regarding restrictions when an obstacle is present immediately after the direction of travel of the vehicle M changes due to the movement of the map division lines ML1 and ML2. Therefore, more appropriate movement control can be executed according to the surrounding conditions of the vehicle M.

[0077] [Scene 4] FIG. 7 is a diagram for explaining driving control of the host vehicle M in the fourth scene. The fourth scene differs from the second scene in that an object OB1 is present in the extension direction of the camera lane marking CL1 (or the map lane marking ML1 after it has been moved). The object OB1 is, for example, a stationary object that is recognized by the first recognition unit 132 as having a size, material, or shape that prevents the host vehicle M from passing over (and therefore must be avoided by) the object OB1. In the fourth scene, the object OB1 is recognized as an obstacle by the first recognition unit 132.

[0078] In the fourth scene, when the movement speed for aligning the position of the map division line ML1 with the position of the camera division line CL1 is equal to or greater than a predetermined speed, and an obstacle (in the example of Figure 6, object OB1) is present in the extension direction of the camera division line CL1 (or the map division line ML1 after it has been moved), the driving control unit 148 performs at least one of restricting driving control and notifying the driver regarding the restriction on driving control, as in the third scene.

[0079] Furthermore, as in the third and fourth scenes, when the movement speed for aligning the position of the map division line ML1 with the position of the camera division line CL1 is equal to or greater than a predetermined speed and no obstacles are present around the host vehicle M, the cruise control unit 148 at least does not restrict the driving control and continues the ongoing driving control. In this situation, when the movement speed of the map division line ML1 exceeds the predetermined speed, a temporary error (angular velocity deviation) greater than or equal to a threshold value may occur between the rotational angular velocity of the host vehicle M estimated based on the movement of the map division line ML1 (estimated angular velocity) and the rotational angular velocity actually measured by the vehicle sensor 40 (measured angular velocity, yaw rate). This error may cause the host vehicle M to behave unsteadily, and therefore a notification such as an alarm may be issued even when no obstacles are present. In other words, the cruise control unit 148 performs control to suppress notification when no obstacles are present, but may issue a notification while continuing driving control when the movement speed for aligning the position of the map division line ML1 with the position of the camera division line CL1 is equal to or greater than a predetermined speed. Reducing notifications includes not providing notifications or delaying the timing of notifications. By delaying the timing of notifications (over time), the error becomes smaller, so it may be possible to not provide notifications at all. This makes it possible to reduce excessive notifications to passengers.

[0080] As shown in the second to fourth scenes described above, in this embodiment, when the camera lane markings CL1 and the map lane markings ML1 are aligned (matched) on one side of the lane L1 according to the surrounding conditions of the vehicle M, if a misalignment occurs between the angle formed by the extension directions of the camera lane markings CL1 and the map lane markings ML1, the map lane markings ML1 are rotated to match the position of the camera lane markings CL1, thereby continuing the ongoing driving control (cruising control). In this case, if the rotation speed of the map lane markings is equal to or greater than a predetermined speed, the ongoing driving control is continued only if there are no obstacles in the vicinity. Furthermore, if the rotation speed is equal to or less than the predetermined speed, the driving control is continued regardless of the presence or absence of obstacles. This makes it possible to prevent the immediate restriction of driving control or notification to the occupant due to a misalignment between the camera lane markings CL1 and the map lane markings ML1. This allows for more appropriate driving control to be achieved according to the surrounding conditions.

[0081] [Variations] In the above-described embodiment, the movement speed when the map division line ML1 is rotated to match the position of the camera division line CL1 is used, but the movement speed when the movement is performed to align the position with a lateral positional deviation may also be used. Also, in the above-described embodiment, instead of moving the map division line ML1, the movement speed when the movement is performed to align the camera division line CL1 with the map division line ML1 may also be used.

[0082] In addition, in an embodiment, instead of the rotational angular velocity of the moving lane line, the estimated self-position of the vehicle M in response to the movement of the lane line (orientation, angular velocity based on the direction of the vehicle before the lane line moved) may be used, or the rotational angular velocity (measured angular velocity) measured by the vehicle sensor 40 of the vehicle M may be used, or the error (angular velocity deviation) between the estimated angular velocity and the measured angular velocity may be used.

[0083] FIG. 8 is a diagram for explaining angular velocity deviation. In the example of FIG. 8, the shape of the host vehicle M is shown in a simplified form. For example, as shown in the second to fourth scenes described above, assume that the map division lines ML1 and ML2 are rotated leftward in the figure to align with the position of the camera division line CL1. In this case, in order to make the host vehicle M travel along the map division lines ML1 and ML2, the orientation of the host vehicle M also rotates leftward. Here, in the example of FIG. 8, the angular velocity ωdr(τ) indicates the angular velocity (measured angular velocity) measured by an internal sensor (vehicle sensor 40) provided on the host vehicle M, and the angular velocity ωlm(τ) indicates the self-position angular velocity (estimated angular velocity) estimated based on the movement of the map division line ML.

[0084] For example, the adjustment unit 144 calculates the angular velocity deviation Δω (= ωlm(τ) - angular velocity ωdr(τ)) between the angular velocity ωdr(τ) (extension direction θdr(τ)) and the angular velocity ωlm(τ) (extension direction θlm(τ)) in accordance with the movement of the map division lines ML1 and ML2 from the extension direction θlm(τ-1) before the rotational movement to the extension direction θlm(τ) after the rotational movement. The speed determination unit 146 regards the angular velocity ωdr(τ), the angular velocity ωlm(τ), or the angular velocity deviation Δω as the movement speed and performs the determination in the same manner as described above.

[0085] For example, when estimating the orientation of the vehicle M after aligning the camera lane line CL with the map lane line ML, the angular velocity ωlm(τ) based on the rotational angular velocity of the map lane line ML and the angular velocity ωdr(τ) based on the measurement results of the vehicle sensor 40 are not the same. Therefore, complementary filter processing or the like is performed to estimate the orientation of the vehicle M relative to the lane line. The rotation angle of the map lane line ML is smaller than the magnitude of the angular difference (gap) between the camera lane line CL and the map lane line ML. In the embodiment, the larger the gap, the more the map lane line ML rotates, and therefore the greater the rotation speed of the map lane line ML. However, the weights of the angular velocities ωlm(τ) and ωdr(τ) may be changed by changing the time constant of the complementary filter based on surrounding conditions such as road shape. This allows the value of the angular velocity deviation Δω to be adjusted depending on the situation, even if the gap size is the same.

[0086] Furthermore, in the embodiment, if a deviation occurs in the angle between the camera lane line and the map lane line, an error occurs between the estimated vehicle position and orientation and the measured angle, and this error may cause the vehicle M to behave erratically. Therefore, for example, if the angular velocity deviation Δω is equal to or greater than a threshold, driving control may be continued based on the movement trajectory (travel history) of the vehicle M before the deviation occurred in the angle between the camera lane line and the map lane line. This suppresses an increase in the error between the estimated angular velocity and the measured angular velocity, thereby suppressing erratic behavior of the vehicle M.

[0087] In addition, as a modified example of the embodiment, instead of determining whether the camera lane lines CL and the map lane lines ML diverge, it may be determined whether the camera lane lines CL and the map lane lines ML coincide with each other. In addition to the above-described driving control, in the embodiment, at least one of the steering and speed of the host vehicle M may be controlled to avoid contact with an object recognized by the recognition unit 130.

[0088] [Processing flow] The following describes the processing executed by the automatic driving control device 100 of the embodiment. Of the processing executed by the automatic driving control device 100, the following mainly describes the driving control (cruising control) processing based on the recognition status of lane markings, etc. It is assumed that when the flow starts, the host vehicle M is currently performing a predetermined driving control (for example, LKAS control in a first driving state (for example, the driver's hands are off)). The processing described below may be repeatedly performed at a predetermined timing or at a predetermined cycle (for example, while the driving control by the automatic driving control device 100 is being performed).

[0089] 9 is a flowchart showing an example of the flow of a driving control process in an embodiment. In the example of FIG. 9, the first recognition unit 132 recognizes the surrounding situation including lane markings (camera lane markings CL) present around the host vehicle M based on the output of the detection device DD that detects the surrounding situation of the host vehicle M (step S100). In the processing of step S100, for example, objects present around the host vehicle M (for example, physical boundaries, other vehicles, etc.) may be recognized, and obstacles among the objects may also be recognized.

[0090] Next, the second recognition unit 134 refers to map information based on the position information of the host vehicle M, and recognizes from the map information lane lines (map lane lines ML) that exist around the host vehicle M (step S110). Next, the driving control unit 148 determines whether the recognition state of the camera lane lines CL and the map lane lines ML satisfies a predetermined condition (step S120).

[0091] If it is determined in the processing of step S120 that the predetermined condition is satisfied, the adjustment unit 144 moves the camera lane line CL and the map lane line ML so that their positions are aligned with each other, based on the position of the host vehicle M (step S130). Note that in the processing of step S130, a process is performed to move the position of the map lane line ML so that it is aligned with the position of the camera lane line CL.

[0092] Next, the speed determination unit 146 determines whether the travel speed during movement is equal to or greater than a predetermined speed (step S140). If it is determined that the travel speed is equal to or greater than the predetermined speed, it determines whether an obstacle exists in the direction of extension of the camera lane marking CL (step S150). Note that the processing of step S150 may include determining whether an obstacle will exist in the direction of extension of the camera lane marking CL in the near future. Furthermore, in the processing of step S150, instead of the camera lane marking CL, a map lane marking ML or a lane defined by any lane marking may be used. If it is determined that an obstacle exists, the driving control unit 148 performs at least one of restricting driving control of the host vehicle M and issuing a notification regarding the restriction of driving control (step S160).

[0093] If the process of step S140 determines that the travel speed during movement is not equal to or greater than a predetermined speed, or if the process of step S150 determines that no obstacles exist in the direction of extension of the camera lane lines CL, the travel control unit 148 continues travel control of the host vehicle M (step S170). Note that the process of step S170 may suppress notification as necessary. Also, if the process of step S120 determines that the recognition state of the camera lane lines CL and the map lane lines ML does not satisfy a predetermined condition (for example, in the first scene), the travel control unit 148 executes travel control based on the surrounding situation (step S180). This ends the process of this flowchart.

[0094] According to the above-described embodiment, the automatic driving control device 100 (an example of a mobile body control device) includes a first recognition unit 132 that recognizes the surrounding situation including camera demarcation lines (an example of first demarcation lines) that demarcate the travel path on which the host vehicle M (an example of the host vehicle M) is traveling and obstacles present around the host vehicle M based on the output of a detection device DD that detects the surrounding situation of the host vehicle M; a second recognition unit 134 that recognizes map demarcation lines (an example of second demarcation lines) that demarcate the travel path on which the host vehicle M is traveling from map information based on position information of the host vehicle M; a movement control unit that controls the movement of the host vehicle M based on at least one of the camera demarcation lines and the map demarcation lines; and when the recognition state of the camera demarcation lines and the map demarcation lines that demarcate the travel path on which the host vehicle M is traveling satisfies a predetermined condition, a movement control unit that controls the movement of the host vehicle M based on at least one of the camera demarcation lines and the map demarcation lines. The system includes an adjustment unit 144 that moves and adjusts the position of the camera lane line or the map lane line so that the positions of the camera lane line and the map lane line are aligned based on the position, and a speed determination unit 146 that determines whether the movement speed when the adjustment unit 144 moves the camera lane line or the map lane line is equal to or greater than a predetermined speed, and the movement control unit at least one of restricting the movement control and notifying the user about the restriction if the speed determination unit 146 determines that the movement speed is equal to or greater than the predetermined speed and an obstacle exists in the direction of extension of the camera lane line, and continuing the movement control if the movement speed is determined to be equal to or greater than the predetermined speed and no obstacle exists in the direction of extension of the camera lane line, thereby enabling more appropriate movement control to be performed in accordance with the surrounding conditions of the vehicle M. This can ultimately contribute to the development of a sustainable transportation system.

[0095] Specifically, according to the embodiment, excessive cruise control restrictions and notifications can be suppressed when adjusting the position of map lane lines relative to the position of one of the camera lane lines, and risks can be reduced by issuing restrictions or notifications early if an obstacle is present. Furthermore, according to the embodiment, the appearance of the map lane lines changes in conjunction with the behavior of the host vehicle M, and the changes in appearance include not only lateral movement but also angle changes due to the turning of the host vehicle M. Therefore, by using the rotation speed, the situation can be determined more accurately. Furthermore, according to the embodiment, even if there is no other vehicle (an example of another moving object) in the direction of travel of the host vehicle M, risks can be reduced by issuing cruise control restrictions or notifications if there will be one in the future.

[0096] The above-described embodiment can be expressed as follows. a storage medium for storing computer-readable instructions; a processor connected to the storage medium; The processor executes the computer-readable instructions to: Based on an output from a detection device that detects the surrounding conditions of the moving body, the surrounding conditions including a first dividing line that divides a path along which the moving body will move and obstacles that exist around the moving body are recognized; Recognizing a second dividing line that divides a travel path around the moving object from map information based on the position information of the moving object; performing movement control of the moving body based on at least one of the first demarcation line and the second demarcation line; When the recognition state of the first demarcation line and the second demarcation line that demarcate the travel path on which the moving body is present satisfies a predetermined condition, the position of the first demarcation line or the second demarcation line is moved and adjusted so that the positions of the first demarcation line and the second demarcation line are aligned based on the position of the moving body; determining whether the moving speed when moving the first demarcation line or the second demarcation line is equal to or greater than a predetermined speed; When the movement speed is determined to be equal to or greater than a predetermined speed and an obstacle exists in the direction of extension of the first division line, at least one of restricting the movement control and notifying the user of the restriction is performed; If the movement speed is determined to be equal to or greater than a predetermined speed and no obstacle exists in the direction in which the first demarcation line extends, the movement control is continued. Mobile control device.

[0097] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0098] 1...vehicle system, 10...camera, 12...radar device, 14...LIDAR, 16...object recognition device, 20...communication device, 30...HMI, 40...vehicle sensor, 50...navigation device, 60...MPU, 80...driving operator, 100...automatic driving control device, 120...first control unit, 130...recognition unit, 132...first recognition unit, 134...second recognition unit, 140...action plan generation unit, 142...deviation determination unit, 144...adjustment unit, 146...speed determination unit, 148...travel control unit, 160...second control unit, 162...target trajectory acquisition unit, 164...speed control unit, 166...steering control unit, 180...HMI control unit, 190...memory unit, 200...traveling drive force output device, 210...brake device, 220...steering device, M...host vehicle

Claims

1. a first recognition unit that recognizes a surrounding situation including a first division line that divides a path along which the moving object moves and obstacles that exist around the moving object based on an output of a detection device that detects a surrounding situation of the moving object; a second recognition unit that recognizes a second division line that divides a travel path around the moving object from map information based on position information of the moving object; a movement control unit that controls the movement of the moving object based on at least one of the first demarcation line and the second demarcation line; an adjustment unit that, when a recognition state of the first demarcation line and the second demarcation line that demarcate a travel path on which the moving body is present satisfies a predetermined condition, moves and adjusts the position of the first demarcation line or the second demarcation line so that the positions of the first demarcation line and the second demarcation line are aligned with each other based on the position of the moving body; a speed determination unit that determines whether a moving speed when the adjustment unit moves the first demarcation line or the second demarcation line is equal to or greater than a predetermined speed, The movement control unit When the speed determination unit determines that the movement speed is equal to or greater than a predetermined speed and an obstacle exists in the extension direction of the first division line, at least one of restricting the movement control and notifying the user of the restriction is performed; If the movement speed is determined to be equal to or greater than a predetermined speed and no obstacle is present in the direction in which the first demarcation line extends, the movement control is continued. Mobile control device.

2. the adjustment unit, when the first and second demarcation lines are recognized on one side of the left and right demarcation lines that divide the travel path as viewed from the moving body, and only the second demarcation line is recognized on the other side, moves the position of the second demarcation line so that an angle formed by the extension directions of the first and second demarcation lines on the one side is within a predetermined angle; The mobile object control device according to claim 1 .

3. The adjustment by the adjustment unit rotates the second demarcation line, The moving speed is a rotation speed based on the rotation of the second division line. The mobile object control device according to claim 2 .

4. the moving speed is an estimated angular velocity of a self-position of the moving object estimated based on the rotation of the second lane marking; the speed determination unit determines that the moving speed is equal to or greater than a predetermined speed when a degree of deviation between the estimated angular speed and a measured angular speed measured by a sensor that measures the angular speed of the moving object is equal to or greater than a threshold value; The mobile object control device according to claim 3 .

5. the obstacle includes another moving body existing around the moving body, the movement control unit performs at least one of restricting the movement control and notifying the user of the restriction when a future position of the other moving object is in a direction in which the first division line on the one side extends. The mobile object control device according to claim 2 .

6. the movement control unit continues the movement control when the movement speed is less than a predetermined speed, regardless of the presence or absence of the obstacle. The mobile object control device according to claim 1 .

7. The computer Based on an output from a detection device that detects the surrounding conditions of the moving body, the surrounding conditions including a first dividing line that divides a path along which the moving body will move and obstacles that exist around the moving body are recognized; Recognizing a second dividing line that divides a travel path around the moving object from map information based on the position information of the moving object; performing movement control of the moving body based on at least one of the first demarcation line and the second demarcation line; When the recognition state of the first demarcation line and the second demarcation line that demarcate the travel path on which the moving body is present satisfies a predetermined condition, the position of the first demarcation line or the second demarcation line is moved and adjusted so that the positions of the first demarcation line and the second demarcation line are aligned with each other based on the position of the moving body; determining whether a moving speed when moving the first demarcation line or the second demarcation line is equal to or greater than a predetermined speed; When the movement speed is determined to be equal to or greater than a predetermined speed and an obstacle exists in the direction in which the first division line extends, at least one of restricting the movement control and notifying the user of the restriction is performed; If the movement speed is determined to be equal to or greater than a predetermined speed and no obstacle is present in the direction in which the first demarcation line extends, the movement control is continued. A mobile object control method.

8. On the computer, Based on an output from a detection device that detects the surrounding conditions of the moving body, a surrounding condition including a first dividing line that divides a path along which the moving body moves and an obstacle that exists around the moving body is recognized; a second division line that divides a travel path around the moving object from map information based on the position information of the moving object; performing movement control of the moving body based on at least one of the first demarcation line and the second demarcation line; When the recognition state of the first demarcation line and the second demarcation line that demarcate the travel path on which the moving body is present satisfies a predetermined condition, the position of the first demarcation line or the second demarcation line is moved and adjusted so that the positions of the first demarcation line and the second demarcation line are aligned with each other based on the position of the moving body; determining whether a moving speed when moving the first demarcation line or the second demarcation line is equal to or greater than a predetermined speed; When the movement speed is determined to be equal to or greater than a predetermined speed and an obstacle exists in the direction in which the first division line extends, at least one of a restriction on the movement control and a notification regarding the restriction is performed; When the moving speed is determined to be equal to or greater than a predetermined speed and no obstacle is present in the direction in which the first demarcation line extends, the moving control is continued. program.

Images