Mobile object control device, mobile object control method, and program

Abstract

To execute more appropriate movement control.SOLUTION: A mobile body control device comprises: a first recognition section for recognizing a peripheral situation which includes an object in the periphery of a mobile body and a first compartment line for partitioning a travel path on which the mobile body travels, on the basis of an output of a detection device having detected the peripheral situation of the mobile body; a second recognition section for recognizing a second compartment line for partitioning a travel path in the periphery of the mobile body from map information on the basis of positional information of the mobile body; a determination section for determining deviation between the first compartment line and the second compartment line; and a movement control section for controlling movement of the mobile body on the basis of at least either the first compartment line or the second compartment line. In cases where the first compartment line and the second compartment line exist on one side to the right or left when viewed from the mobile body, and the mobile body changes courses to the one side, and the first compartment line and the second compartment line of the one side deviate during the course change, movement control of the mobile body is performed according to the object.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a mobile object control device, a mobile object control method, and a program. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transportation systems that take into consideration vulnerable traffic participants have been gaining momentum. To achieve this, efforts are being focused on research and development into autonomous driving technology to further improve traffic safety and convenience. In this regard, a technology is known that detects road surface patterns formed by unevenness on the road ahead of a lane change, and corrects the lane change path if the vehicle interferes with the road surface pattern on the lane change path (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6294928 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional autonomous driving technology, it cannot be said that sufficient consideration has been given to controlling the movement of a moving object when changing lanes based on the results of comparing road dividing lines recognized by cameras, etc. with road dividing lines shown in map information, and there is room for further consideration.

[0005] In order to solve the above-mentioned problems, one of the objects of the present application is to provide a mobile object control device, a mobile object control method, and a program that can execute more appropriate movement control in accordance with the surrounding conditions of the mobile object when changing course, thereby contributing to the development of a sustainable transportation system. [Means for solving the problem]

[0006] A mobile object control device, a mobile object control method, and a program according to the present invention employ the following configuration. (1): A mobile body control device according to one embodiment of the present invention includes a first recognition unit that recognizes the surrounding situation, including objects around the mobile body and a first dividing line that divides the path of travel along which the mobile body moves, based on the output of a detection device that detects the surrounding situation of the mobile body; a second recognition unit that recognizes second dividing lines that divide the path of travel along which the mobile body moves from map information based on the position information of the mobile body; a determination unit that determines the deviation between the first dividing line and the second dividing line; and a mobile control unit that controls the movement of the mobile body based on at least one of the first dividing line and the second dividing line, wherein the mobile control unit is a mobile body control device that controls the movement of the mobile body in accordance with the object when the first dividing line and the second dividing line are present on one side, either left or right, of the mobile body when the mobile body changes course to the one side, and when the first dividing line and the second dividing line on the one side diverge during the course of the course change.

[0007] (2): In the above aspect (1), the movement control unit controls the movement of the moving body in accordance with the physical boundary existing on the one side when the surrounding conditions of the moving body satisfy a first condition, and the first condition is that the deviation angle between the first dividing line and the second dividing line on the one side is equal to or greater than a threshold value, and the first dividing line on the one side deviates from the second dividing line in a direction opposite to the direction in which the moving body changes course.

[0008] (3): In the above aspect (2), in addition to the first condition, the movement control unit controls the movement of the moving body in accordance with the physical boundary present on one side when the movement path is not a branching path or a merging path, the movement path is not a section where the movement path increases or decreases, the curvature of the movement path is less than a threshold, and the speed of the moving body is equal to or greater than a predetermined speed.

[0009] (4): In the above aspect (1), when the surrounding conditions of the moving body satisfy a second condition, the movement control unit performs movement control in accordance with at least one of a first preceding moving body that was present ahead of the moving body on the path of travel before the moving body changed course, and a second preceding moving body that is present ahead of the moving body on the path of travel to the destination of the path change, and the second condition is that the movement speed when the position of one of the first and second dividing lines is adjusted from a state in which the positions of the first and second dividing lines on one side in the path width direction are different to one of the positions of the first and second dividing lines is greater than or equal to a predetermined speed, and the deviation distance of the positions of the first and second dividing lines on one side in the path width direction is greater than or equal to a threshold value.

[0010] (5): In the above aspect (4), in addition to the second condition, the movement control unit performs movement control according to at least one of the first preceding moving body and the second preceding moving body when the movement path is not a branching path or a merging path, the movement path is not a section where the movement path increases or decreases, the curvature of the movement path is less than a threshold value, and the speed of the moving body is equal to or greater than a predetermined speed.

[0011] (6): Another aspect of the mobile body control method of the present invention is a mobile body control method in which a computer recognizes the surrounding situation including objects around the mobile body and a first dividing line that divides the path along which the mobile body moves, based on the output of a detection device that detects the surrounding situation of the mobile body; recognizes a second dividing line that divides the path along which the mobile body moves from map information based on the position information of the mobile body; determines the deviation between the first dividing line and the second dividing line; controls the movement of the mobile body based on at least one of the first dividing line and the second dividing line; and controls the movement of the mobile body in accordance with the object when the first dividing line and the second dividing line are present on one side or the left or right of the mobile body, the mobile body changes course to the one side, and when the first dividing line and the second dividing line on the one side diverge during the course of the course change.

[0012] (7): Another aspect of the present invention provides a program that causes a computer to recognize the surrounding conditions, including objects around a moving body and a first dividing line that divides the path along which the moving body moves, based on the output of a detection device that detects the surrounding conditions of the moving body; recognize a second dividing line that divides the path along which the moving body moves from map information based on the position information of the moving body; determine the deviation between the first dividing line and the second dividing line; control the movement of the moving body based on at least one of the first dividing line and the second dividing line; and control the movement of the moving body in accordance with the object when the first dividing line and the second dividing line are present on one side or the left or right of the moving body and the moving body changes course to the one side, and when the first dividing line and the second dividing line on the one side diverge during the course of the course change. [Effects of the Invention]

[0013] According to the above aspects (1) to (7), more appropriate movement control can be executed in accordance with the surrounding conditions of the moving object when changing course. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a configuration diagram of a vehicle system 1 including a mobile object control device according to an embodiment. [Figure 2] 2 is a functional configuration diagram of a first control unit 120 and a second control unit 160. FIG. [Figure 3] 1 is a diagram for explaining driving control of a vehicle M in a first scene. FIG. [Figure 4] FIG. 10 is a diagram for explaining driving control of the host vehicle M in a second scene. [Figure 5] FIG. 10 is a diagram for explaining driving control of the host vehicle M in a third scene. [Figure 6] FIG. 10 is a diagram for explaining driving control of the host vehicle M in a fourth scene. [Figure 7] 4 is a flowchart illustrating an example of a flow of an operation control process in the embodiment. [Figure 8]4 is a flowchart showing a first embodiment of a driving control process. [Figure 9] 10 is a flowchart showing a second embodiment of the driving control process. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, with reference to the drawings, embodiments of a mobile object control device, a mobile object control method, and a program according to the present invention will be described. Hereinafter, a vehicle will be used as an example of a mobile object, and an embodiment in which the mobile object control device is applied to an autonomous vehicle will be described. Autonomous driving refers to automatically controlling one or both of the steering and speed of a vehicle to perform driving control. The above-mentioned driving control may include various driving controls, such as Automated Lane Change (ALC), Lane Keeping Assistance System (LKAS), Adaptive Cruise Control System (ACC), Traffic Jam Pilot (TJP), and Collision Mitigation Brake System (CMBS). Furthermore, an autonomous vehicle may be manually controlled by a user (e.g., a passenger) of the vehicle (so-called manual driving). In addition to vehicles, mobile objects may include, for example, ships capable of moving on land such as hovercrafts, aircraft capable of traveling on roads, and stand-up vehicles with power units.

[0016] [Overall configuration] 1 is a configuration diagram of a vehicle system 1 including a mobile object control device according to an embodiment. The vehicle (hereinafter referred to as host vehicle M) on which the vehicle system 1 is mounted is, for example, a two-wheeled, three-wheeled, four-wheeled, or micromobility vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination of these. The electric motor operates using power generated by a generator connected to the internal combustion engine, or discharged power from a battery (storage battery) such as a secondary battery or a fuel cell.

[0017] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a light detection and ranging (LIDAR) device 14, an object recognition device 16, a communication device 20, a human machine interface (HMI) device 30, vehicle sensors 40, a navigation device 50, a map positioning unit (MPU) device 60, a driving operation control device 80, an automatic driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are connected to each other via multiplexed communication lines such as a controller area network (CAN) communication line, serial communication lines, a wireless communication network, etc. Note that the configuration shown in FIG. 1 is merely an example, and some of the configuration may be omitted, or other configurations may be added. The combination of the camera 10, the radar device 12, the LIDAR device 14, and the object recognition device 16 is an example of a "detection device DD." The HMI device 30 is an example of an "output device." The automatic driving control device 100 is an example of a "mobile object control device."

[0018] The camera 10 is a digital camera that uses a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to any location of the host vehicle M in which the vehicle system 1 is installed. When capturing an image of the front, the camera 10 is attached to the top of the front windshield, the back of the rearview mirror, the front of the vehicle body, etc. When capturing an image of the rear, the camera 10 is attached to the top of the rear windshield, the back door, etc. When capturing an image of the side, the camera 10 is attached to a door mirror, etc. The camera 10 periodically and repeatedly captures images of the surroundings of the host vehicle M, for example. The camera 10 may be a stereo camera.

[0019] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by surrounding objects (reflected waves) to detect at least the position (distance and direction) of the objects. The radar device 12 is attached to any location on the vehicle M. The radar device 12 may detect the position and speed of the objects using an FM-CW (Frequency Modulated Continuous Wave) method.

[0020] The LIDAR 14 irradiates light around the vehicle M and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time between emitting and receiving the light. The irradiated light is, for example, a pulsed laser beam. The LIDAR 14 is attached to any location on the vehicle M.

[0021] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the camera 10, the radar device 12, and the LIDAR 14 to recognize the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition results to the automatic driving control device 100. Alternatively, the object recognition device 16 may output the detection results from the camera 10, the radar device 12, and the LIDAR 14 directly to the automatic driving control device 100. In that case, the object recognition device 16 may be omitted from the configuration of the vehicle system 1 (detection device DD).

[0022] The communication device 20 communicates with, for example, other vehicles in the vicinity of the vehicle M, terminal devices of users using the vehicle M, or various server devices, using networks such as a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), LAN (Local Area Network), WAN (Wide Area Network), or the Internet.

[0023] The HMI 30 outputs various information to the occupants of the vehicle M and accepts input operations by the occupants. The HMI 30 includes, for example, various display devices, speakers, buzzers, touch panels, switches, keys, microphones, and the like.

[0024] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the host vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the yaw rate (for example, the rotational angular velocity around a vertical axis passing through the center of gravity of the host vehicle M), and a direction sensor that detects the orientation of the host vehicle M. The vehicle sensor 40 may also be provided with a position sensor that detects the position of the host vehicle M. The position sensor is an example of a "position measurement unit." The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. The position sensor may also be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver 51 of the navigation device 50. The vehicle sensor 40 may derive the speed of the host vehicle M from the difference (i.e., distance) of the position information at a predetermined time in the position sensor. The results detected by the vehicle sensor 40 are output to the automatic driving control device 100.

[0025] The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as a hard disk drive (HDD) or a flash memory. The GNSS receiver 51 identifies the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be identified or supplemented by an inertial navigation system (INS) that uses the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, etc. The GNSS receiver 51 may be provided in the vehicle sensor 40. The navigation HMI 52 may share some or all of the components with the HMI 30 described above. The route determination unit 53 determines, for example, a route (hereinafter, a route on a map) from the position of the vehicle M identified by the GNSS receiver 51 (or an arbitrary input position) to a destination input by the occupant using the navigation HMI 52, with reference to the first map information 54. The first map information 54 is information that represents road shapes using, for example, links that indicate roads (an example of a travel route) and nodes connected by the links. The first map information 54 may also include POI (Point Of Interest) information and the like. The route on the map is output to the MPU 60. The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may transmit the current position and destination to a navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server. The navigation device 50 outputs the determined route on the map to the MPU 60.

[0026] The MPU 60 includes, for example, a recommended lane determination unit 61, and stores second map information 62 in a storage device such as an HDD or flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, by dividing it into 100 m intervals in the vehicle travel direction), and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 determines, for example, which lane from the left the vehicle should travel in. When there is a branch point on the route on the map, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel on a reasonable route to the branch point.

[0027] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, the number of lanes (number of travel routes), the type and shape of road dividing lines (hereinafter referred to as dividing lines), information on the center of lanes, and road boundary information. The second map information 62 may also include information on whether the road boundary is a boundary (physical boundary) that includes a structure that prevents a vehicle from passing (including crossing or contacting). Examples of physical boundaries include guardrails, curbs, medians, fences, etc. The term "impassable" may include the presence of a low level step that allows passage if unusual vehicle vibrations are tolerated. The second map information 62 may also include road shape information, traffic regulation information, address information (address and postal code), facility information, parking information, telephone number information, etc. Road shape information includes, for example, the curvature (which may be referred to as the radius of curvature; the same applies below), width, gradient, etc. The second map information 62 may be updated as needed by the communication device 20 communicating with an external device. The first map information 54 and the second map information 62 may be provided as an integrated piece of map information. The map information may also be stored in the storage unit 190.

[0028] The driving operators 80 include, for example, a steering wheel, an accelerator pedal, and a brake pedal. The driving operators 80 may also include a shift lever, a variable steering wheel, a joystick, or other operators. Each operator of the driving operators 80 is equipped with an operation detection unit that detects, for example, the amount of operation of the operator by the occupant or whether or not the operator is operated. The operation detection unit detects, for example, the steering angle and steering torque of the steering wheel, the amount of depression of the accelerator pedal and the brake pedal, etc. The operation detection unit then outputs the detection results to the automatic driving control device 100 or one or both of the driving force output device 200, the brake device 210, and the steering device 220.

[0029] The automatic driving control device 100 executes various types of driving control associated with automatic driving for the host vehicle M. The automatic driving control device 100 includes, for example, a first control unit 120, a second control unit 160, an HMI control unit 180, and a storage unit 190. The first control unit 120, the second control unit 160, and the HMI control unit 180 are each realized by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Furthermore, some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or an SOC (System On Chip), or may be realized by a combination of software and hardware. The above-mentioned program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as an HDD or flash memory of the automatic driving control device 100, or may be stored in a removable storage medium such as a DVD, CD-ROM, or memory card, and installed in the storage device of the automatic driving control device 100 by inserting the storage medium (non-transitory storage medium) into a drive device, card slot, etc.

[0030] The storage unit 190 may be realized by the various storage devices described above, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a ROM (Read Only Memory), or a RAM (Random Access Memory). The storage unit 190 stores, for example, various types of information, programs, and the like in the embodiments. The storage unit 190 may also store map information (for example, the first map information 54 and the second map information 62).

[0031] FIG. 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 140. The first control unit 120, for example, implements functions based on AI (Artificial Intelligence) and functions based on a predefined model in parallel. For example, the function of "recognizing intersections" may be implemented by concurrently executing intersection recognition using deep learning or the like and recognition based on predefined conditions (such as the presence of traffic lights and road markings that can be pattern matched), and then scoring and comprehensively evaluating both. This ensures the reliability of autonomous driving. The first control unit 120 also executes control related to autonomous driving of the host vehicle M based on instructions from, for example, the MPU 60, the HMI control unit 180, or the like.

[0032] The recognition unit 130 recognizes the surrounding situation of the host vehicle M based on the recognition results of the detection device DD (information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16). For example, the recognition unit 130 recognizes the status of objects present around the host vehicle M (within a predetermined distance), such as their position, speed, and acceleration. Objects include traffic participants such as other vehicles, pedestrians, and bicycles, as well as physical boundaries that separate roads (travel paths). The position of an object is recognized as a position on an absolute coordinate system with a representative point of the host vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or by a represented area. For example, if the object is a moving object such as another vehicle, the "state" of the object may include the acceleration or jerk of the moving object, or the "behavioral state" (for example, whether the other vehicle is changing lanes or about to change lanes).

[0033] The recognition unit 130 also recognizes, for example, stop lines, obstacles, red lights, toll booths, other road phenomena, markings (speed limits) posted on the road, and road signs indicating speed limits. The recognition unit 130 also includes, for example, a first recognition unit 132 and a second recognition unit 134. Details of these functions will be described later.

[0034] The behavior plan generation unit 140 generates a behavior plan for driving the host vehicle M by autonomous driving based on the recognition results of the recognition unit 130, etc. For example, the behavior plan generation unit 140 generates a target trajectory for the host vehicle M to travel automatically (without driver operation) in the future, so that the host vehicle M can respond to the surrounding conditions of the host vehicle M, while essentially traveling in the recommended lane determined by the recommended lane determination unit 61, based on the recognition results by the recognition unit 130 and the surrounding road shapes based on the current position of the host vehicle M acquired from map information, etc. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequential arrangement of points (trajectory points) to be reached by the host vehicle M. The trajectory points are points to be reached by the host vehicle M at predetermined travel distances (e.g., on the order of several meters) along the road, and separately, target speeds and target accelerations for predetermined sampling times (e.g., on the order of a few tenths of a second) are generated as part of the target trajectory. Alternatively, the trajectory points may be positions to be reached by the host vehicle M at the sampling times for each predetermined sampling time. In this case, the target speed and target acceleration information are expressed as the interval between trajectory points.

[0035] The action plan generation unit 140 may set an autonomous driving event when generating the target trajectory. Examples of the event include a constant speed driving event in which the host vehicle M drives in the same lane at a constant speed, a following driving event in which the host vehicle M follows another vehicle that is within a predetermined distance (for example, within 100 m) ahead of the host vehicle M and is closest to the host vehicle M, a lane change event in which the host vehicle M changes lanes from the host vehicle's own lane to an adjacent lane, a branching event in which the host vehicle M branches off into a lane on the destination side at a road branching point, a merging event in which the host vehicle M merges into a main lane at a merging point, a takeover event in which the autonomous driving is terminated and the host vehicle M switches to manual driving, and so on. Examples of the event may also include an overtaking event in which the host vehicle M temporarily changes lanes to an adjacent lane, overtakes a leading vehicle in the adjacent lane, and then changes lanes back to the original lane, and an avoidance event in which the host vehicle M performs at least one of braking and steering to avoid an obstacle ahead of the host vehicle M.

[0036] Furthermore, the behavior plan generation unit 140 may change an event already determined for the current section to another event or set a new event for the current section, depending on the surrounding conditions of the host vehicle M recognized while the host vehicle M is traveling. Furthermore, the behavior plan generation unit 140 may change an event already set for the current section to another event or set a new event for the current section, depending on the operation of the occupant on the HMI 30. The behavior plan generation unit 140 generates a target trajectory according to the set event.

[0037] The behavior plan generation unit 140 also includes, for example, a determination unit 142 and a travel control unit 144. Details of these functions will be described later. The travel control unit 144 and the second control unit 160 are an example of a "mobility control unit."

[0038] The second control unit 160 controls the traveling driving force output device 200, the braking device 210, and the steering device 220 so that the host vehicle M passes through the target trajectory generated by the action plan generation unit 140 at the scheduled time.

[0039] The second control unit 160 includes, for example, a target trajectory acquisition unit 162, a speed control unit 164, and a steering control unit 166. The target trajectory acquisition unit 162 acquires information on the target trajectory (trajectory points) generated by the action plan generation unit 140 and stores it in a memory (not shown). The speed control unit 164 controls the driving force output device 200 or the brake device 210 based on a speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized by, for example, a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road ahead of the host vehicle M and feedback control based on the deviation from the target trajectory.

[0040] Returning to FIG. 1 , the HMI control unit 180 notifies the occupant of predetermined information via the HMI 30. The predetermined information includes, for example, information related to the traveling of the vehicle M, such as information related to the state of the vehicle M and information related to driving control. The information related to the state of the vehicle M includes, for example, the speed of the vehicle M, engine speed, and shift position. The information related to driving control includes, for example, information inquiring whether or not driving control is being performed by autonomous driving, information regarding the driving control status by autonomous driving, information regarding the automation level, and information prompting the occupant to drive when switching from autonomous driving to manual driving. The predetermined information may also include information related to the surrounding conditions recognized by the detection device DD. The predetermined information may also include information unrelated to the traveling of the vehicle M, such as television programs, content (e.g., movies) stored on a storage medium such as a DVD, and the like. The predetermined information may also include, for example, information regarding the current location and destination during autonomous driving, and the remaining fuel level of the vehicle M. The HMI control unit 180 may output the information received by the HMI 30 to the communication device 20, the navigation device 50, the first control unit 120, and the like.

[0041] The HMI control unit 180 may also cause the HMI 30 to output inquiry information for the occupant, processing results by the first control unit 120 and the second control unit 160, etc. The HMI control unit 180 may also transmit various pieces of information to be output by the HMI 30 to a terminal device used by the occupant of the vehicle M via the communication device 20.

[0042] Traveling drive force output device 200 outputs a traveling drive force (torque) to the drive wheels for the vehicle to travel. Traveling drive force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, etc., and an ECU (Electronic Control Unit) that controls these. The ECU controls the above components according to information input from second control unit 160 or information input from the accelerator pedal of driving operator 80.

[0043] The braking device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or information input from the brake pedal of the driving operation device 80, so that a braking torque corresponding to the braking operation is output to each wheel. The braking device 210 may include a backup mechanism that transmits hydraulic pressure generated by operation of the brake pedal to the cylinder via a master cylinder. Note that the braking device 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic braking device that controls an actuator according to information input from the second control unit 160 to transmit hydraulic pressure from the master cylinder to the cylinder.

[0044] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor changes the direction of the steered wheels by, for example, applying a force to a rack and pinion mechanism. The steering ECU drives the electric motor to change the direction of the steered wheels in accordance with information input from the second control unit 160 or information input from the steering wheel of the driving operator 80.

[0045] [Recognition and Action Plan Generation] Next, the details of the functions of the recognition unit 130 (mainly the first recognition unit 132 and the second recognition unit 134) and the behavior plan generation unit 140 (mainly the determination unit 142 and the driving control unit 144) will be described. Note that, below, the contents of the driving control (driving control) of the host vehicle M using the functions of the recognition unit 130 and the behavior plan generation unit will be described in several scenes.

[0046] [Scene 1] FIG. 3 is a diagram illustrating driving control of the host vehicle M in a first scene. The example of FIG. 3 shows lane markings CL1 to CL3 recognized by the detection device DD and lane markings ML1 to ML3 obtained from map information (e.g., second map information 62) based on the position information of the host vehicle M. In the map information, lane L1 is defined by lane markings ML1 and ML2, and lane L2 is defined by lane markings ML2 and ML3. Lanes L1 and L2 are lanes on which a vehicle can travel in the same direction (the X-axis direction in the figure). In the example of FIG. 3, lane markings CL1 to CL3 are an example of a "first lane marking," and lane markings ML1 to ML3 are an example of a "second lane marking." In the following description, lane markings CL1 to CL3 may be referred to as "camera lane markings CL1 to CL3," and lane markings ML1 to ML3 may be referred to as "map lane markings ML1 to ML3." Furthermore, when the camera division lines CL1 to CL3 are not distinguished from one another, they may simply be referred to as "camera division lines CL," and when the map division lines ML1 to ML3 are not distinguished from one another, they may simply be referred to as "map division lines ML."

[0047] In the example of FIG. 3, when viewed from lane L1 in the travelable direction (the X-axis direction in the figure), a physical boundary (e.g., a fence) OB1 is located to the left of lane L1, and when viewed from lane L2 in the travelable direction, a physical boundary (e.g., a fence) OB2 is located to the right of lane L2. In the example of FIG. 3, the physical boundaries OB1 and OB2 are located along the extension direction of lanes L1 and L2 (more specifically, the physical boundary OB1 is located along lane marking CL1, and the physical boundary OB2 is located along lane marking CL3). In addition, the first scene shown in FIG. 3 illustrates a situation in which a host vehicle M traveling on lane L1 at a speed VM changes lanes to lane L2 using ALC. In the example of FIG. 3, time progresses in the order of time T1 and T2. In addition, the host vehicle M(T*) and speed VM(T*) indicate the position and speed of the host vehicle M at time T*. The same applies to the explanation of the following figures.

[0048] The first scenario illustrates a situation in which, for example, the host vehicle M is in the middle of performing lane change (an example of a course change) control using ALC in a construction zone or the like (before the lane change is completed), and in particular, the camera lane line CL ahead of the host vehicle M deviates from the map lane line ML while at least a portion of the host vehicle M is passing over (straddling) the lane line. In this case, the camera lane line CL corresponding to the lane line temporarily drawn on the road surface to avoid the construction site is correct, so if the host vehicle M determines a route based on the incorrect map lane line ML, there is a possibility that the host vehicle M will deviate from its lane or enter the construction site. Therefore, in the embodiment, driving control (movement control) is performed according to the first scenario.

[0049] The first recognition unit 132 recognizes the surrounding conditions of the host vehicle M based on the output of the detection device DD that detects the surrounding conditions (external world) of the host vehicle M. For example, the first recognition unit 132 recognizes left and right camera division lines CL1 and CL2 that divide the driving lane (lane L1) of the host vehicle M based on an image captured by the camera 10 (hereinafter referred to as a camera image). The first recognition unit 132 may also recognize a camera division line CL3 that divides an adjacent lane (lane L2) adjacent to the driving lane.

[0050] For example, the first recognition unit 132 analyzes the camera image, extracts edge points in the image that have a large difference in brightness from adjacent pixels, and recognizes the camera lane marking lines CL1 to CL3 on the image plane by connecting the edge points. Furthermore, the first recognition unit 132 converts the positions of the camera lane marking lines CL1 to CL3 into a vehicle coordinate system (for example, the XY plane coordinates in FIG. 3) based on the position of the representative point of the host vehicle M.

[0051] The first recognition unit 132 may also recognize, for example, the curvature of the camera lane markings CL1 to CL3. The camera lane markings CL1 to CL3 may be recognized or corrected based on the output of a detection device other than the camera 10 (for example, the radar device 12 or the LIDAR 14). The first recognition unit 132 may also recognize the amount of curvature change of the camera lane markings CL1 to CL3. The amount of curvature change is, for example, the time rate of change in the curvature of the camera lane markings CL1 to CL3 recognized by the camera 10 at a distance x [m] forward as viewed from the host vehicle M. The first recognition unit 132 may also recognize the curvature or amount of curvature change of the lane markings CL1 to CL3 by averaging the curvatures or amounts of curvature change of the camera lane markings CL1 to CL3. The camera lane markings CL1 to CL3 may be recognized or corrected based on the output of a detection device other than the camera 10 (for example, the radar device 12 or the LIDAR 14).

[0052] Furthermore, in the first scene, the first recognition unit 132 recognizes physical boundaries OB1 and OB2 that exist around the host vehicle M. Furthermore, in the first scene, if another vehicle exists around the host vehicle M, the first recognition unit 132 may recognize the other vehicle.

[0053] The second recognition unit 134 recognizes lane markings around the host vehicle M from map information based on the position of the host vehicle M detected by, for example, the vehicle sensor 40 or the GNSS receiver 51. For example, the second recognition unit 134 refers to map information based on the position information of the host vehicle M, and recognizes map marking lines ML1 to ML3 that exist in the traveling direction of the host vehicle M or in directions in which the host vehicle M can travel.

[0054] Furthermore, of the recognized map division lines ML1 to ML3, the second recognition unit 134 may recognize map division lines ML1 and ML2 as division lines that define the lane in which the host vehicle M is traveling, and may recognize map division lines ML2 and ML3 as division lines that define adjacent lanes. Furthermore, the second recognition unit 134 recognizes the curvature or curvature change amount of each of the map division lines ML1 to ML3 from the second map information 62. Furthermore, the second recognition unit 134 may recognize the curvature or curvature change amount of the lanes defined by the map division lines by averaging the curvature or curvature change amount of each of the map division lines ML1 to ML3.

[0055] The determination unit 142 determines whether the camera lane lines CL1-CL3 recognized by the first recognition unit 132 deviate from the map lane lines ML1-ML3 recognized by the second recognition unit 134. For example, the determination unit 142 derives the degree of deviation between the lane lines CL1 and ML1 located closest to the left of the host vehicle M, the degree of deviation between the lane lines CL2 and ML2 located closest to the right of the host vehicle M, and the degree of deviation between the lane lines CL3 and ML3 on the adjacent lane side. If the derived degree of deviation is equal to or greater than a threshold, the determination unit 142 determines that the camera lane lines and the map lane lines deviate, and if the degree of deviation is less than the threshold, the determination unit 142 determines that there is no deviation. The determination of whether there is a deviation may be performed repeatedly at a predetermined timing or periodically.

[0056] For example, the determination unit 142 superimposes the camera lane lines CL1, CL2, and CL3 and the map lane lines ML1, ML2, and ML3 on the plane of the vehicle coordinate system (XY plane) based on the position of the representative point of the host vehicle M. When determining the lane lines to be compared (lane lines CL1 and ML1, lane lines CL2 and ML2, and lane lines CL3 and ML3), the determination unit 142 determines that the lane lines are deviating if the deviation degree of at least one lane line is equal to or greater than a threshold, and determines that the lane lines are not deviating if all lane lines are less than the threshold. The deviation degree is, for example, the degree of deviation (deviation distance, travel lane width direction deviation) in the road width direction (travel lane width direction, lateral direction, Y-axis direction in the figure). In the example of Figure 3, the deviation determination may be performed using the average value of the lateral position deviation D1 between the marking lines CL1 and ML1, the lateral position deviation D2 between the marking lines CL2 and ML2, and the lateral position deviation D3 between the marking lines CL3 and ML3, or the deviation determination may be performed using the maximum or minimum value of the deviations D1, D2, and D3.

[0057] Furthermore, the degree of deviation may be, for example, the degree of the angle (deviation angle) formed by the two compared marking lines instead of (or in addition to) the above-mentioned lateral position deviation amount. In the example of Fig. 3, the average value of the angle θ1 formed by marking lines CL1 and ML1, the angle θ2 formed by marking lines CL2 and ML2, and the angle θ3 formed by marking lines CL3 and ML3 may be used, or the maximum or minimum value of the angles θ1, θ2, and θ3 may be used.

[0058] Furthermore, the degree of deviation may be the degree (magnitude) of difference in curvature change of the lane markings, instead of (or in addition to) the lateral position deviation or the angle formed by the lane markings. The curvature change is mainly used when the lane is a curved road. For example, the determination unit 142 may use the average value of the difference in curvature change between lane markings CL1 and ML1, the difference in curvature change between lane markings CL2 and ML2, and the difference in curvature change between lane markings CL3 and ML3, or may use the maximum or minimum of these differences. The determination unit 142 may also use the difference between the average value of the curvature change of lane markings CL1 to CL3 and the average value of the curvature change of lane markings ML1 to ML3. The determination unit 142 may also use the difference between the curvature change of the lane markings (lanes L1 and L2) recognized from the camera image and the curvature change of the lane markings recognized from map information.

[0059] For example, when determining whether a camera lane marking deviates from a map lane marking, the determination unit 142 may determine whether the camera lane marking is erroneously recognized based on one or both of the amount of change in curvature of the camera lane marking detected by the recognition unit 130 and the angle between the camera lane marking and the map lane marking. In this case, the determination unit 142 determines whether the camera lane marking is erroneously recognized when, for example, the direction of change in curvature and the direction of change in angle are the same and the amount of change in curvature and the angle increase with the distance from the vehicle M. This allows for more accurate determination of erroneous recognition of a camera lane marking when traveling through a lane change section such as a curved road.

[0060] The driving control unit 144 determines driving control for the host vehicle M based on the recognition results of the first recognition unit 132 and the second recognition unit 134 and the determination result by the determination unit 142, and generates a target trajectory based on the determined driving control. "Determining driving control" may include, for example, determining the content (type) of driving control, or determining whether to execute (suppress) driving control. Furthermore, "executing driving control" may include, for example, continuing driving control that is already being executed, in addition to switching and executing the content of driving control. "Suppressing driving control" may include not only not executing driving control, but also lowering the automation level of driving control.

[0061] Here, the driving control executed by the driving control unit 144 includes at least a first driving control and a second driving control. The first driving control is, for example, driving control that executes at least steering control of the steering or speed of the host vehicle M based on at least one of the camera lane markings CL and the map lane markings ML. For example, in the case of ALC control, the first driving control generates a driving trajectory for the host vehicle M to change lanes from the driving lane (e.g., lane L1) to a lane (e.g., lane L2) of the destination lane change (course change destination), and drives the host vehicle M so that a representative point of the host vehicle M moves on a trajectory that follows the generated driving trajectory. In addition, in the case of LKAS control, the first driving control drives the host vehicle M so that a representative point of the host vehicle M passes through the center of the lane marked by the lane markings. In the first driving control, for example, if the camera recognition accuracy is below a threshold, driving control may be performed by prioritizing the map dividing line ML, and if the map information is old (for example, the map update date is earlier than a specified date and time), driving control may be performed by prioritizing the camera dividing line CL.

[0062] The second driving control is, for example, driving control that executes at least steering control of the steering or speed of the host vehicle M based on an object (e.g., a physical boundary, another vehicle) recognized by the first recognition unit 132. The second driving control, for example, identifies the position of a lane based on the physical boundary or the position of another vehicle, and drives the host vehicle M so that a representative point of the host vehicle M runs in the center of the identified lane. In addition, the second driving control drives the host vehicle M so that a representative point of the host vehicle M runs on a trajectory that follows the running trajectory of the other vehicle.

[0063] Furthermore, the driving control may include a plurality of driving control functions with different automation levels (an example of the degree of automation). The automation levels may include, for example, a first level, a second level with a lower degree of automation of driving control than the first level, and a third level with a lower degree of automation of driving control than the second level. The automation levels may also include a fourth level with a lower degree of automation of driving control than the third level. Here, the automation level may be a level determined by standardized information, laws and regulations, or an index value set independently. Therefore, the types, contents, and number of automation levels are not limited to the following examples. A low degree of automation of driving control means, for example, a low automation rate in driving control and a large (heavy) task assigned to the driver. A low degree of automation of driving control means a low degree of control of the steering or acceleration / deceleration of the host vehicle M by the automatic driving control device 100 (a high degree of need for the driver to intervene in steering or acceleration / deceleration operations). Tasks assigned to the driver include, for example, monitoring the surroundings of the host vehicle M and operating driving controls. The operation of the driving controls includes, for example, the driver gripping the steering wheel (hereinafter referred to as a hands-on state). The tasks assigned to the driver are, for example, tasks for the occupant (driver tasks) necessary to maintain the autonomous driving of the host vehicle M. Therefore, if the occupant is unable to perform the assigned tasks, the automation level will be reduced. For example, the first level of driving control may include driving controls such as ALC, LKAS, ACC, TJP, and CMBS. Furthermore, the second or third level of driving control may include driving controls such as ALC, LKAS, ACC, and CMBS. The fourth level of driving control may include manual driving. Furthermore, the fourth level of driving control may include driving controls such as ACC and CMBS. Of the first to fourth levels, the first level has the highest degree of automation of driving control, and the fourth level has the lowest degree of automation of driving control.

[0064] Furthermore, at the first level, no tasks are assigned to the occupant (the tasks assigned to the driver are the lightest), and therefore, for example, driving control is permitted when the driver of the host vehicle M is not gripping the steering wheel (hereinafter referred to as a hands-off state). At the second level, a task assigned to the driver is, for example, monitoring the surroundings (particularly the front) of the host vehicle M. At the third level, a task assigned to the driver is, for example, monitoring the surroundings of the host vehicle M and being in a hands-on state. At the fourth level, a task assigned to the driver is, for example, monitoring the surroundings of the host vehicle M and being in a hands-on state, as well as operating the driving operator 80 to control the steering and speed of the host vehicle M. In other words, at the fourth level, the occupant can immediately take over driving, and the tasks assigned to the driver are the most severe. The content of driving control and the tasks assigned to the occupant at each automation level are not limited to the examples described above. The automatic driving control device 100 executes driving control at one of first to fourth levels based on the surrounding conditions of the vehicle M and the task being performed by the occupant.

[0065] For example, the driving control unit 144 executes the first driving control when the determination unit 142 determines that the camera lane markings and the map lane markings are not separated, and executes the second driving control when the determination unit 142 determines that the camera lane markings and the map lane markings are separated. Furthermore, the driving control unit 144 may, for example, execute control to switch from the first driving control to the second driving control when a predetermined condition is satisfied, or may execute control to terminate driving control of the vehicle M and switch to manual driving by the occupant. Furthermore, the driving control unit 144 may switch the automation level depending on the surrounding circumstances and the type of driving control.

[0066] In a first scene shown in FIG. 3 , the automatic driving control device 100 executes a lane change from lane L1 to lane L2 by ALC control according to the first driving control at time T1. In this case, the driver is assumed to have his / her hands off the vehicle. For example, at time T1, the determination unit 142 determines whether or not the camera marking CL3 and the map marking ML3, which are markings that demarcate the lane L2 to which the host vehicle M is changing lanes, are separated from each other. The camera marking CL3 and the map marking ML3 are markings that demarcate the lane L2 to which the host vehicle M is changing lanes (in other words, markings that are farther away from the host vehicle M before the lane change). The determination unit 142 may also determine whether or not the camera marking CL and the map marking ML are separated from each other, by determining whether or not the camera marking CL2 and the map marking ML2 are separated from each other, or whether or not the camera marking CL1 and the map marking ML1 are separated from each other, to determine whether or not the camera marking CL and the map marking ML are separated from each other as a whole.

[0067] Here, it is assumed that at time T1, the vicinity of the host vehicle M (within a predetermined distance from the host vehicle M) does not include point P1 shown in FIG. 3 where the camera lane marking CL and the map lane marking ML diverge. Therefore, the determination unit 142 determines that at least the camera lane marking CL3 and the map lane marking ML3 do not diverge. Based on the determination result, the driving control unit 144 executes ALC using the first driving control, generates a target trajectory K1 for changing lanes from lane L1 to lane L2 based on at least one of the camera lane marking CL3 and the map lane marking ML3, and controls the steering and speed of the host vehicle M so that the host vehicle M travels along the generated target trajectory K1.

[0068] The determination unit 142 continues to determine the deviation between the camera lane markings CL3 and the map lane markings ML3 even during a lane change. In this case, the lateral movement of the vehicle M (movement in the road width direction (the Y-axis direction in the figure)) due to the lane change also causes a deviation in the lateral positions of the camera lane markings CL3 and the map lane markings ML3. Even in this case, if the deviation distance is not equal to or greater than a predetermined distance based on the determination result that the camera lane markings CL3 and the map lane markings ML3 matched at time T1, a matching process is performed to move the position of one of the camera lane markings CL3 and the map lane markings ML3 (for example, the camera lane markings CL3) so that it overlaps with the position of the other (for example, the map lane markings ML3). The lateral movement of the vehicle M is controlled while the positions of the lane marks as seen from the vehicle M are adjusted as needed. When matching is performed, since driving control is in progress, the greater the distance between the camera division line CL and the map division line ML, the faster the moving speed of the moving division line (the greater the amount of lateral movement in a specified time) is set so that matching can be completed quickly (within a specified time).

[0069] Then, in the first scene, at time T2 while the host vehicle is changing lanes, the determination unit 142 determines that at least the camera lane marking CL3 and the map lane marking ML3 will diverge (more specifically, determines that there is a divergence based on the divergence angle). In this case, the driving control unit 144 performs driving control of the host vehicle M in accordance with an object (an object other than a lane marking) present in the vicinity. "In accordance with an object" may mean in accordance with the type or position of the object, or in accordance with the presence or absence of the object.

[0070] For example, at time T2, if a first condition is satisfied, the cruise control unit 144 performs cruise control of the host vehicle M using the second driving control in accordance with the physical boundary OB2 located on the right side as viewed from the host vehicle M. The first condition is, for example, that the degree of deviation between the camera lane marking CL3 and the map lane marking ML3 is equal to or greater than a threshold, and that the camera lane marking CL3 deviates from the map lane marking ML3 in the opposite direction (leftward in FIG. 3 ) to the lane change direction (course change direction, rightward in FIG. 3 ) of the host vehicle M. In the example of FIG. 3 , to satisfy the first condition, the cruise control unit 144 controls the host vehicle M to travel along a target trajectory generated based on the position of the physical boundary OB2 so that the host vehicle M travels along the physical boundary OB2 at a position a predetermined distance away from the physical boundary OB2. This allows the cruise control unit 144 to continue driving control in a hands-off state using the physical boundary OB2, even if the camera lane marking CL3 and the map lane marking ML3 deviate from each other. Note that the cruise control unit 144 may use the physical boundary OB1 in addition to (or instead of) the physical boundary OB2. Furthermore, the traveling control unit 144 ends the ALC control when the host vehicle M is positioned at a predetermined position in the lane L2 (for example, the center of the lane).

[0071] As described above, in the first scenario, for example, in a situation where temporary construction is being carried out at the lane change destination and the number of lanes remains the same but the lanes are temporarily detoured to avoid the construction site, the detour markings actually drawn on the road surface are recognized as camera markings CL and therefore deviate from the map markings ML. Therefore, if a lane change is being performed near the start point of such deviation, continuing lane change control using the map markings may result in the vehicle approaching the construction site. Therefore, in the embodiment, when there is such deviation from the markings, the host vehicle M is allowed to travel in a lane that is estimated based on a physical boundary (assuming a structure that separates the construction site from the road). This makes it possible to continue driving control while suppressing unstable vehicle behavior during driving control.

[0072] In addition to the first condition, the driving control unit 144 may include as a condition that the road on which the vehicle M is traveling (a road within a predetermined distance in the direction of travel as seen from the vehicle M) is not a branching road or a merging road, is not a section where lanes increase or decrease, the curvature of the road is less than a threshold, and the speed of the vehicle M is equal to or greater than a predetermined speed. In this way, by limiting the conditions to a situation where there is no increase or decrease in lanes, no false detection due to the influence of a curved road, and the speed is appropriate for changing lanes, and by allowing the continuation of ALC control at a point where a demarcation line that temporarily detours due to construction or the like begins even when the number of lanes remains the same, it is possible to prevent excessive continuation of driving control.

[0073] If the surrounding conditions of the host vehicle M do not satisfy the first condition at time T2 in the first scene, the HMI control unit 180 may output an alarm or the like to the HMI 30 to notify the occupant, and the driving control unit 144 may stop the ALC control currently being performed and switch to control such as manual driving. The driving control unit 144 may also determine whether to immediately terminate or continue the lane change control depending on the state of the host vehicle M's lane change at time T2. For example, if the position of the host vehicle M crosses the camera division line CL2 (or the map division line ML2) (at least a part of the vehicle body passes over the division line), the ALC control may be continued until the lane change is completed, and if the position does not cross the division line, the driving control unit 144 may perform control such as switching to manual driving. If driving control is to be continued until the lane change is completed, the driving control unit 144 may also perform control such as lowering the automation level.

[0074] [Second Scene] FIG. 4 is a diagram illustrating driving control of the host vehicle M in the second scenario. In the second scenario, the position (or timing) at which the camera lane marking CL and the map lane marking ML diverge during a lane change is different from that in the first scenario. Specifically, the second scenario illustrates a situation in which the camera lane marking CL and the map lane marking ML diverge at point P1, which is ahead of point Pa at which the entire body of the host vehicle M has passed over the camera lane marking CL2 (or the map lane marking ML2) (in other words, the entire body of the host vehicle M is on lane L2). In the second scenario, instead of the entire body of the host vehicle M having passed over the lane, another criterion may be used, such as the passage of a representative point of the host vehicle M. The following description will mainly focus on the control steps that differ from those in the first scenario, and will omit a description of the control steps that are similar to those in the first scenario (e.g., control at time T1).

[0075] At time T2 in the second scenario, the determination unit 142 determines that at least the camera lane marking CL3 and the map lane marking ML3 will deviate from each other at point P1, a distance Da ahead of point Pa where the host vehicle M passed over the camera lane marking CL2 (or map lane marking ML2). In this case, the driving control unit 144 determines whether the distance Da from point Pa to point P1 is within a first predetermined distance. If the distance Da is within the first predetermined distance, the driving control unit 144 continues the ALC control of the host vehicle M using the first driving control in accordance with an object present in the vicinity, as in the first scenario. Note that the object in the second scenario is a road boundary OB2 present on the right side of lane L2 as seen from the host vehicle M, as in the first scenario. However, a physical boundary OB1 may be used in addition to (or instead of) the physical boundary OB2.

[0076] In addition to the first condition, the traveling control unit 144 may also include as a condition that the road on which the host vehicle M is traveling is not a branching road or a merging road, is not a section where the number of lanes included in the road increases or decreases, the curvature of the road is less than a threshold, and the speed of the host vehicle M is equal to or greater than a predetermined speed. The second scenario can achieve the same effect as the first scenario. Therefore, the second scenario can perform more appropriate movement control depending on the surrounding situation after the lane change.

[0077] Thus, in the first and second scenarios, if the camera lane markings CL and the map lane markings ML diverge while the vehicle M is moving laterally in the road width direction (travel path width direction) due to ALC control, the vehicle has not been traveling continuously in the lane L2 after the lane change, so there is no travel history of the lane L2 up until that point, and the reliability is lower than that of the lane L1 that the vehicle had been traveling in until then. Therefore, in the embodiment, in such a situation, the lane to which the vehicle will change lanes is estimated using the physical boundaries of the road, making it possible to continue cruise control while preventing the behavior of the vehicle M from becoming unstable during the lane change.

[0078] [Scene 3] 5 is a diagram for explaining driving control of the host vehicle M in the third scenario. The third scenario illustrates a situation in which ALC control is performed in which, for example, a lane marking temporarily drawn on a road during road repair work is forgotten or not erased after the road repair, or the temporarily drawn lane marking is painted over with a different color, but is recognized as a lane marking in a camera image due to a difference in color from the surrounding road surface, etc.

[0079] For example, the third scene differs from the second scene in that, in addition to the camera lane markings CL1-CL3, camera lane markings CL4 and CL5 that were supposed to disappear after repair (or were still visible in the camera image despite having been removed) are present in the direction of travel of the host vehicle M. Also, unlike the second scene, in lane L1, another vehicle m1 is traveling ahead of the host vehicle M at a speed of Vm1, and in lane L2, another vehicle m2 is traveling ahead of the host vehicle M at a speed of Vm2. The other vehicle m1 is an example of a "first preceding moving object," and the other vehicle m2 is an example of a "second preceding moving object." Furthermore, in the example of FIG. 5, in addition to times T1 and T2, there is also a time T3, and time progresses in the order of times T1, T2, and T3. The other vehicles m1 and m2 move over time, just like the host vehicle M, but for convenience of explanation, this is omitted here. In the following, the third scenario will be described mainly focusing on the differences from the second scenario, and a description of the parts that perform similar control will be omitted.

[0080] At time T1 in the third scene, as in the first and second scenes, the determination unit 142 determines whether the camera lane markings CL around the host vehicle M deviate from the map lane markings ML, and determines that they do not deviate. In this case, the driving control unit 144 allows ALC control in a hands-off state, generates a target trajectory K1 for changing lanes from lane L1 to lane L2, and controls at least one of the steering and speed of the host vehicle M so that the host vehicle M travels along the generated target trajectory K1.

[0081] Time T2 is the time when the host vehicle M passes point P2 where the camera lane marking CL and the map lane marking ML diverge while changing lanes, and is the time when part of the host vehicle M passes over the camera lane marking CL2 (or map lane marking ML2). At time T2, the camera lane marking CL5 and the map lane marking ML3 become misaligned. In this situation, the host vehicle M is moving laterally in conjunction with the lane change, so the determination unit 142 performs a matching process to move the position of one of the camera lane marking CL and the map lane marking ML (for example, the camera lane marking CL) so that it overlaps with the position of the other (for example, the map lane marking ML), and performs lateral movement control while adjusting the position of the lane marking as seen from the host vehicle M as needed.

[0082] If the driving control unit 144 continues to change lanes based on the camera lane marking CL5 after time T2 shown in FIG. 5, the lane change may be completed at time T3 not in the center of lane L2 but to the left of the center (or the entire body of the host vehicle M may not be on lane L2), and the host vehicle M may be traveling at a position offset from the map lane marking ML3. Therefore, for example, when the surrounding situation of the host vehicle M satisfies a second condition different from the first condition described above, the driving control unit 144 executes driving control using a second driving control according to an object around the host vehicle M. In the third scenario, the object may be, for example, at least one of another vehicle m1 located ahead of the host vehicle M in the lane L1 before the host vehicle M changes lanes and another vehicle m2 located ahead of the host vehicle M in the lane L2 to which the host vehicle M will change lanes. The second condition is, for example, that the lane width direction (travel path width direction, Y-axis direction in the figure, horizontal direction) movement speed VL of the lane marking on at least one side (e.g., the right side) of the lane L2 to which the lane is to be changed and the map lane marking ML3 are adjusted so that their positions change from different to overlapping, and that the deviation distance (travel path width direction deviation) W1 between the camera lane marking CL5 and the map lane marking ML3 in the lane width direction (travel path width direction) is greater than or equal to a threshold value.

[0083] When the second condition is satisfied, the driving control unit 144 estimates that the other vehicles m1 and m2 are traveling in the center of the lane based on position information (lateral position relative to the lane) of at least one of the other vehicles m1 and m2, and controls the lateral position of the host vehicle M when changing lanes. For example, when traveling based on the other vehicle m2, the driving control unit 144 controls at least the steering of the host vehicle M so that the representative point of the host vehicle M travels along the traveling trajectory of the representative point of the other vehicle m2. Furthermore, when traveling based on the other vehicle m1, the driving control unit 144 estimates the center position of the lane L2 relative to the position of the other vehicle m1 based on the position of the other vehicle m1 and the road widths (lateral widths) of the lanes L1 and L2 obtained from map information or camera images, and controls at least the steering of the host vehicle M so that the representative point of the host vehicle M is positioned at the estimated center position.

[0084] In the third scenario, as shown at time T3 in Figure 5, if a marking line is not erased, the old marking line may be recognized, causing the vehicle to travel at a position offset from the map marking line ML, which may result in an unstable position at the lane change destination. However, as described above, when the surrounding conditions of the vehicle M satisfy the second condition, the ALC driving control can be continued according to the surrounding conditions by generating a driving position (target trajectory K1) based on other vehicles m1 and m2 ahead and changing lanes. This allows, for example, when the system determines that a lane change is to be made to overtake a vehicle ahead, driving control can be performed to avoid interference with the overtaken vehicle or the preceding vehicle ahead of the lane change destination, allowing for safer continuation of driving control.

[0085] In the third scenario, in addition to the second condition, the driving control unit 144 may include, as in the first condition, the conditions that the road on which the vehicle M is traveling (a road within the processing distance in the direction of travel as seen from the vehicle M) is not a branching or merging road, is not a section in which lanes increase or decrease, the curvature of the road is less than a threshold, and the speed of the vehicle M is equal to or greater than a predetermined speed. In this way, by limiting the conditions to situations in which there is no increase or decrease in lanes, no false detection due to the influence of a curved road, and the speed is appropriate for changing lanes, and by allowing the continuation of ALC control at points where the number of lanes remains the same but the old dividing line remains, it is possible to prevent excessive continuation of driving control.

[0086] Furthermore, in the third scenario, if the surrounding conditions of the host vehicle M do not satisfy the second condition, the HMI control unit 180 may cause the HMI 30 to output an alarm or the like to notify the occupant, and the driving control unit 144 may stop the ALC control currently being performed and switch to control such as manual driving. Furthermore, the driving control unit 144 may determine whether to immediately terminate or continue the lane change control depending on the state of the host vehicle M's lane change at time T2. For example, if the position of the host vehicle M crosses the camera division line CL2 (or the map division line ML2) (at least a part of the vehicle body passes over the division line), the ALC control may be continued until the lane change is completed, and if the position does not cross the division line, the ALC control may be switched to manual driving. Furthermore, if the driving control unit 144 continues driving control until the lane change is completed, the driving control unit 144 may perform control such as lowering the automation level.

[0087] [Scene 4] FIG. 6 is a diagram illustrating driving control of the host vehicle M in a fourth scenario. In the fourth scenario, the location (or timing) at which the camera lane markings CL and the map lane markings ML diverge during a lane change is different from that in the third scenario. Specifically, the fourth scenario illustrates a situation in which the camera lane markings CL4 and CL5 are recognized and diverge from the map lane markings ML at point P2, which is ahead of point Pb at which the entire body of the host vehicle M has passed over the camera lane markings CL2 (or map lane markings ML2) (in other words, the entire body of the host vehicle M is on lane L2). In the fourth scenario, instead of the entire body of the host vehicle M having passed over the camera lane markings CL2, another criterion, such as the representative point of the host vehicle M having passed over the camera lane markings, may be used. The following description will mainly focus on the control steps that differ from those in the third scenario, and a description of the control steps that are similar to those in the third scenario will be omitted.

[0088] In the fourth scene, the first recognition unit 132 recognizes the camera lane lines CL4 and CL5 at point P2, which is ahead of the current point Pb, at time T2. The determination unit 142 determines that at point P2, at least the camera lane line CL5 and the map lane line ML3 deviate from each other. In this case, the driving control unit 144 determines whether the distance Db from point Pb to point P2 is within a second predetermined distance. If the distance Db is within the second predetermined distance, the driving control unit 144 performs driving control of the host vehicle M in accordance with an object present in the vicinity, as in the third scene. Note that the object in the fourth scene is, for example, at least one of other vehicles m1 and m2, as in the third scene.

[0089] In addition to the second condition, the traveling control unit 144 may include as a condition that the road on which the host vehicle M is traveling is not a branching road or a merging road, is not a section where the number of lanes included in the road increases or decreases, the curvature of the road is less than a threshold, and the speed of the host vehicle M is equal to or greater than a predetermined speed. In this way, the fourth scenario can achieve the same effect as the third scenario. Therefore, according to the fourth scenario, more appropriate movement control can be executed depending on the surrounding situation after the lane change.

[0090] In the lane line matching process in the third and fourth scenes, instead of moving the camera lane line CL to the map lane line ML, the map lane line ML may be moved to the camera lane line CL, in which case control is performed based on the movement speed of the map lane line ML. The second predetermined distance in the third and fourth scenes may be longer than the first predetermined distance. The camera lane lines CL4 and CL5 in the third and fourth scenes are lane lines remaining after repairs, and therefore pose a lower risk than the first and second scenes, where there is a possibility of entering a construction area. Furthermore, since the camera lane lines CL4 and CL5 are redrawn sections, they are expected to exist over a long distance, and therefore, even if they are determined over a long distance, there is no significant impact on driving control.

[0091] Furthermore, in the third and fourth scenes of the embodiment, the lateral position of the host vehicle M when changing lanes may be controlled based on position information (lateral position relative to the lane) of at least one of another vehicle m1 (first leading moving body) present in front of the lane L1 before the host vehicle M changes lanes and another vehicle m2 (second leading moving body) present in front of the lane L2 to which the host vehicle M will change lanes, instead of (or in addition to) at least one of another vehicle m1 (first leading moving body) present in front of the lane L1 before the host vehicle M changes lanes and another vehicle m2 (second leading moving body) present in front of the lane L2 to which the host vehicle M will change lanes. Furthermore, the first leading moving body and the second leading moving body in the embodiment were originally leading moving bodies, but may also be used as leading moving bodies even when they become trailing moving bodies after being overtaken by the host vehicle M.

[0092] [Processing flow] The following describes the processing executed by the automatic driving control device 100 of the embodiment. Of the processing executed by the automatic driving control device 100, the following mainly describes the driving control processing based on the recognition status of lane markings, etc. It should be noted that when the flow starts, the host vehicle M is assumed to be executing a predetermined driving control (for example, LKAS control in a first driving state (for example, the driver's hands are off)). The processing described below may be executed repeatedly at a predetermined timing or at a predetermined cycle (for example, while driving control by the automatic driving control device 100 is being executed).

[0093] 7 is a flowchart showing an example of the flow of a driving control process in an embodiment. In the example of FIG. 7, the first recognition unit 132 recognizes the surrounding situation including lane lines (camera lane lines) present around the host vehicle M based on the output of the detection device DD that detects the surrounding situation of the host vehicle M (step S100). In the processing of step S100, for example, objects present around the host vehicle M (e.g., physical boundaries, other vehicles, etc.) may be recognized. Next, the second recognition unit 134 refers to map information based on the position information of the host vehicle M and recognizes lane lines (map lane lines) present around the host vehicle M from the map information (step S110).

[0094] Next, the driving control unit 144 determines whether or not to change lanes to a lane on one side (e.g., the right side) of the lane in which the camera lane markings CL and the map lane markings ML are present (recognized) as viewed from the lane in which the host vehicle M is traveling (step S120). If it is determined that a lane change will be performed, the driving control unit 144 executes lane change control (ALC control) based on the first driving state (step S130). Next, the determination unit 142 determines whether or not there is a deviation between the camera lane markings CL and the map lane markings ML on at least one side (e.g., the right side) of the lane to which the host vehicle M is to change lanes (step S140). If it is determined that there is a deviation, the driving control unit 144 determines whether or not the host vehicle M is currently changing lanes (step S150). If it is determined that the host vehicle M is currently changing lanes, the driving control unit 144 executes a driving control process according to the surrounding conditions of the host vehicle M (step S160). Specific processing of the driving control process in step S160 will be described later.

[0095] Furthermore, if it is determined in step S120 that the vehicle will not change lanes to one side where the camera division lines CL and the map division lines ML are present, if it is determined in step S140 that the camera division lines and the map division lines on at least one side of the lane to which the vehicle will change lanes do not diverge, or if it is determined in step S150 that the vehicle is not changing lanes, the driving control unit 144 executes control appropriate to each situation (step S170). In step S170, for example, if it is determined in step S120 that the vehicle will not change lanes, the driving control unit 144 continues the current control (e.g., LKAS control). Furthermore, if it is determined in step S140 that the camera division lines and the map division lines do not diverge, the driving control unit 144 executes control such as continuing the first driving state until the lane change is completed. Furthermore, if the process of step S150 indicates that the vehicle is not changing lanes (for example, if the vehicle M is not passing through a dividing line that separates the driving lane from the adjacent lane), the cruise control unit 144 performs control such as suspending ALC control and switching to LKAS control for the lane before the lane change. In other situations, control appropriate for each situation is also performed.

[0096] [Driving control process: first embodiment] Next, a specific example of the process of step S160 will be described. Fig. 8 is a flowchart showing a first embodiment of the driving control process. The example of Fig. 8 shows an example of the driving control process in the first and second situations described above. In the example of Fig. 8, the driving control unit 144 determines whether the deviation angle between the camera lane line CL and the map lane line ML determined by the determination unit 142 is equal to or greater than a threshold value (step S161A).

[0097] If it is determined that the deviation angle is equal to or greater than the threshold, the determination unit 142 determines whether the camera lane markings CL deviate from the map lane markings ML in the opposite direction (e.g., leftward relative to the lane markings ML) from the lane change direction (e.g., rightward relative to the lane markings ML) in which the host vehicle M is changing lanes (step S162A). If it is determined that the camera lane markings CL deviate from the map lane markings ML in the opposite direction (e.g., rightward relative to the lane markings ML), the determination unit 142 continues driving control (ALC control) using the second driving control based on physical boundaries present around the host vehicle M (step S163A). Furthermore, if it is determined in step S161A that the deviation between the camera lane markings CL and the map lane markings ML is not equal to or greater than the threshold, or if it is determined in step S162A that the camera lane markings CL do not deviate from the map lane markings ML in the opposite direction to the lane change direction, the determination unit 142 executes driving control based on the map lane markings ML (step S164A), for example. This ends the processing of this flowchart.

[0098] If the physical boundary is not recognized in the process of step S163A, the driving control unit 144 may perform control such as lowering the automation level or terminating driving control and switching to manual driving. Also, in the process of step S164A, control may be performed to continue driving control based on the camera lane markings CL.

[0099] [Driving control process: second embodiment] FIG. 9 is a flowchart showing a second embodiment of the driving control process. The example of FIG. 9 shows an example of the driving control process in the third and fourth situations described above. In the example of FIG. 9, the driving control unit 144 determines whether the movement speed of one of the camera lane markings CL and the map lane markings ML during lateral position adjustment is equal to or greater than a predetermined speed (step S161B). If it is determined that the movement speed of the lane markings is equal to or greater than the predetermined speed, the driving control unit 144 determines whether the deviation distance between the camera lane markings CL and the map lane markings ML in the road width direction (lateral direction, lane width direction) determined by the determination unit 142 is equal to or greater than a threshold (step S162B). If it is determined that the difference is equal to or greater than the threshold, the driving control unit 144 continues driving control (ALC control) using the second driving control based on at least one of a first preceding vehicle (an example of a first preceding moving body) traveling ahead of the host vehicle M in the lane before the lane change and a second preceding vehicle (an example of a second preceding moving body) traveling ahead of the host vehicle M in the lane to which the lane change is to be made (step S163B). Also, if it is determined in the processing of step S161B that the moving speed during position adjustment of the camera division line CL and the map division line ML is not equal to or greater than a predetermined speed, or if it is determined in the processing of step S162B that the deviation distance in the lane width direction between the camera division line CL and the map division line ML is not equal to or greater than a threshold, the driving control unit 144 continues driving control based on the map division line ML, for example (step S164B).

[0100] In the process of step S163B, if neither the first preceding moving body nor the second preceding moving body is recognized, the traveling control unit 144 may lower the automation level or terminate driving control and switch to manual driving. In addition, in the process of step S164B, traveling control may be continued based on the camera lane markings CL.

[0101] [Variations] In the above-described embodiment, ALC control in a hands-off state has been mainly described, but similar control may also be performed for ALC control in a hands-on state. Furthermore, in the embodiment, in addition to ALC control based on a system-determined decision, similar control may also be performed when ALC control is executed based on a lane change instruction from an occupant of the host vehicle M (e.g., an instruction from the HMI 30). Furthermore, the control during lane change in the embodiment may be applied to a case where the host vehicle changes lanes and then returns to the original lane due to overtaking control, for example. Furthermore, in the embodiment, instead of determining whether the camera lane markings CL and the map lane markings ML diverge, it may be determined whether the camera lane markings CL and the map lane markings ML coincide with each other. Furthermore, in addition to the above-described driving control, at least one of the steering and speed of the host vehicle M may be controlled to avoid contact with an object recognized by the recognition unit 130.

[0102] According to the above-described embodiment, the automatic driving control device 100 (an example of a mobile body control device) includes a first recognition unit 132 that recognizes a surrounding situation including objects around the host vehicle M (an example of a mobile body) and camera demarcation lines (first demarcation lines) that demarcate the lane (an example of a travel path) on which the host vehicle M is traveling, based on the output of a detection device DD that detects the surrounding situation of the host vehicle M (an example of a mobile body); a second recognition unit 134 that recognizes map demarcation lines (second demarcation lines) that demarcate the lane on which the host vehicle M is traveling from map information, based on position information of the host vehicle M; and a determination unit 142 that determines a deviation between the camera demarcation lines and the map demarcation lines. and a travel control unit 144 (an example of a travel control unit) that controls the movement of the host vehicle M based on at least one of the camera division lines and the map division lines, and when the camera division lines and the map division lines are present on one side, either left or right, as viewed from the host vehicle M, and the host vehicle M changes course (changes lanes) to one side, and when the camera division lines on one side and the map division lines diverge while the host vehicle M is changing course, the travel control unit 144 controls the movement of the host vehicle M in accordance with the object, thereby enabling more appropriate travel control to be performed in accordance with the surrounding conditions of the moving body when changing course. This can ultimately contribute to the development of a sustainable transportation system.

[0103] For example, in the past, if the camera lane markings CL and the map lane markings ML diverged during a lane change, the host vehicle was in the process of moving laterally due to the lane change or had just recently moved laterally, so cruise control was canceled and a warning was issued. However, according to the embodiment, if the travel distance during a lane change or after moving to an adjacent lane is within a predetermined range and the camera lane markings of the lane after the lane change diverge from the map lane markings, the travel lane can be estimated from surrounding objects (physical boundaries and surrounding vehicles), and cruise control can be continued while preventing the behavior of the host vehicle M from becoming unstable.

[0104] The above-described embodiment can be expressed as follows. a storage medium for storing computer-readable instructions; a processor connected to the storage medium; The processor executes the computer-readable instructions to: Based on an output from a detection device that detects the surrounding situation of the moving body, the surrounding situation including objects around the moving body and a first dividing line that divides a path along which the moving body moves is recognized; Recognizing a second dividing line that divides a travel path around the moving object from map information based on the position information of the moving object; Determining a deviation between the first demarcation line and the second demarcation line; controlling the movement of the moving object based on at least one of the first demarcation line and the second demarcation line; When the first and second dividing lines are present on one side of the left and right sides as seen from the moving body, and the moving body changes course to the one side, and when the first and second dividing lines on the one side diverge during the course change, movement control of the moving body is performed in accordance with the object. Mobile control device.

[0105] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0106] 1...vehicle system, 10...camera, 12...radar device, 14...LIDAR, 16...object recognition device, 20...communication device, 30...HMI, 40...vehicle sensor, 50...navigation device, 60...MPU, 80...driving operator, 100...automatic driving control device, 120...first control unit, 130...recognition unit, 132...first recognition unit, 134...second recognition unit, 140...action plan generation unit, 142...judgment unit, 144...travel control unit, 160...second control unit, 162...target trajectory acquisition unit, 164...speed control unit, 166...steering control unit, 180...HMI control unit, 190...memory unit, 200...traveling drive force output device, 210...brake device, 220...steering device, M...host vehicle, m1, m2...other vehicles

Claims

1. a first recognition unit that recognizes a surrounding situation including objects around the moving object and a first division line that divides a path along which the moving object moves, based on an output from a detection device that detects the surrounding situation of the moving object; a second recognition unit that recognizes a second division line that divides a travel path around the moving object from map information based on position information of the moving object; a determination unit that determines a deviation between the first demarcation line and the second demarcation line; a movement control unit that controls movement of the moving object based on at least one of the first demarcation line and the second demarcation line, the movement control unit controls the movement of the moving body in accordance with the object when the first and second demarcation lines are present on one side of the left or right side as seen from the moving body, the moving body changes course to the one side, and the first and second demarcation lines on the one side diverge during the course change; Mobile control device.

2. the movement control unit performs movement control of the moving object in accordance with the physical boundary present on the one side when a surrounding situation of the moving object satisfies a first condition; the first condition is that a deviation angle between the first lane marking line and the second lane marking line on the one side is equal to or greater than a threshold value, and the first lane marking line on the one side deviates from the second lane marking line in a direction opposite to a lane change direction of the moving object; The mobile object control device according to claim 1 .

3. the movement control unit performs movement control of the moving object in accordance with the physical boundary present on the one side when, in addition to the first condition, the movement path is not a branching path or a merging path, the movement path is not a section where an increase or decrease occurs, the curvature of the movement path is less than a threshold, and the speed of the moving object is equal to or greater than a predetermined speed; The mobile object control device according to claim 2 .

4. when a surrounding situation of the moving body satisfies a second condition, the movement control unit performs movement control in accordance with at least one of a first preceding moving body that was present ahead of the moving path before the moving body changed course and a second preceding moving body that is present ahead of the moving path after the moving body changed course, The second condition is that the moving speed when the positions of the first and second demarcation lines on one side are adjusted from a state in which the positions of the first and second demarcation lines in the width direction of the travel path are different to one another so that the position of the first and second demarcation lines overlaps the position of the other is equal to or greater than a predetermined speed, and the distance of deviation between the positions of the first and second demarcation lines on one side in the width direction of the travel path is equal to or greater than a threshold value. The mobile object control device according to claim 1 .

5. the movement control unit performs movement control according to at least one of the first preceding moving body and the second preceding moving body when, in addition to the second condition, the movement path is not a branching path or a merging path, the movement path is not a section where the movement path increases or decreases, the curvature of the movement path is less than a threshold, and the speed of the moving body is equal to or greater than a predetermined speed; The mobile object control device according to claim 4.

6. The computer based on an output of a detection device that detects the surrounding situation of the moving object, recognizing the surrounding situation including objects around the moving object and a first dividing line that divides a path along which the moving object moves; Recognizing a second dividing line that divides a travel path around the moving object from map information based on the position information of the moving object; determining a deviation between the first demarcation line and the second demarcation line; controlling the movement of the moving object based on at least one of the first demarcation line and the second demarcation line; When the first and second demarcation lines are present on one side of the left or right side as seen from the moving body, and the moving body changes course to the one side, and when the first and second demarcation lines on the one side diverge during the course change, movement control of the moving body is performed in accordance with the object. A mobile object control method.

7. On the computer, Based on an output from a detection device that detects the surrounding situation of the moving body, a surrounding situation including objects around the moving body and a first dividing line that divides a path along which the moving body moves is recognized; a second division line that divides a travel path around the moving object from map information based on the position information of the moving object; determining a deviation between the first demarcation line and the second demarcation line; controlling the movement of the moving object based on at least one of the first demarcation line and the second demarcation line; When the first and second demarcation lines are present on one side of the left and right sides as seen from the moving body, and the moving body changes course to the one side, and when the first and second demarcation lines on the one side diverge during the course change, movement control of the moving body is performed in accordance with the object. program.

Images