Vehicle control device, vehicle control method, and program

The vehicle control system addresses route inaccuracies by combining detection device data with map information and adaptive driving modes, ensuring precise route estimation and safer automated driving.

JP7584488B2Active Publication Date: 2024-11-15HONDA MOTOR CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2022203047
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2022-12-20
Publication Date
2024-11-15
Estimated Expiration
2042-12-20

AI Technical Summary

Technical Problem

Existing vehicle route determination systems face inaccuracies due to discrepancies between detected road conditions and map information, particularly in areas with road construction or maintenance issues, leading to unreliable route estimation.

Method used

A vehicle control system that utilizes both detection device data and map information to estimate a vehicle's route, incorporating a learning unit to generate a route estimation model and switch driving modes based on deviation thresholds, ensuring accurate route determination and driver notification.

Benefits of technology

Improves the accuracy of vehicle route determination by integrating multiple data sources and adaptive driving modes, enhancing safety and reliability in automated driving systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007584488000001
    Figure 0007584488000001
  • Figure 0007584488000002
    Figure 0007584488000002
  • Figure 0007584488000003
    Figure 0007584488000003
Patent Text Reader

Abstract

To improve accuracy in determining a travel path of a vehicle.SOLUTION: A vehicle control device comprises: a travel control part for controlling traveling of a vehicle in a travel path determined according to first information based on an output of a detection device which detects a peripheral situation of the vehicle and second information based on map information; a travel path estimation part for estimating the travel path of the vehicle on the basis of the first information and the second information; and a travel path determination part for determining the travel path of the vehicle on the basis of the first information, the second information and travel path estimation information relating to the travel path of the vehicle estimated by the travel path estimation part.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a program. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transportation systems that take into consideration vulnerable traffic participants have been gaining momentum. To achieve this, we are focusing on research and development to further improve traffic safety and convenience through research and development of driving assistance technology. In this regard, a self-location estimation technology is known that improves the quality of operations of autonomous driving of moving objects by calculating reliability taking into account multiple selected self-locations (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-125563 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in driving assistance technology, when estimating the vehicle's route using information recognized by a detection device that recognizes the surrounding conditions and information obtained from map information based on the vehicle's position detected by a position sensor, there is a problem in that the vehicle's route may not be obtained accurately because the actual road conditions may differ from the map information due to road construction or poor maintenance, or the route recognized by the detection device may be recognized differently from the actual route due to changes in road conditions such as tunnels and forks.

[0005] In order to solve the above-mentioned problems, an object of the present invention is to provide a vehicle control device, a vehicle control method, and a program that can improve the accuracy of determining a vehicle's route, thereby contributing to the development of a sustainable transportation system. [Means for solving the problem]

[0006] A vehicle control device, a vehicle control method, and a program according to the present invention employ the following configuration. (1): A vehicle control device according to one embodiment of the present invention is a vehicle control device including: a driving control unit that controls driving of the vehicle on a driving path determined by first information based on the output of a detection device that detects the surrounding conditions of the vehicle and second information based on map information; a driving path estimation unit that estimates a driving path of the vehicle based on the first information and the second information; and a driving path determination unit that determines the driving path of the vehicle based on the first information, the second information, and driving path estimation information regarding the driving path of the vehicle estimated by the driving path estimation unit.

[0007] (2) In the aspect (1) above, the path estimating unit estimates the path of the vehicle when the degree of deviation between the first information and the second information is equal to or greater than a threshold value.

[0008] (3): In the above aspect (1), a learning unit is further provided that generates a path estimation model based on the first information, the second information, and true value data, the path estimation model having the first information and the second information as input and the path estimation information as output, and the path estimation unit uses the path estimation model to estimate a path of the vehicle based on the first information and the second information.

[0009] (4): In the above aspect (1), the first information includes information on a first dividing line that divides the vehicle's lane and is recognized based on the output of the detection device, and the second information includes information on a second dividing line that divides the vehicle's lane and is obtained from the map information based on the vehicle's position information.

[0010] (5): In the above aspect (1), the path estimation unit estimates the path of the vehicle based on the first information, the second information, and the vehicle's driving conditions and / or vehicle type information.

[0011] (6): In the above aspect (2), the driving control unit executes driving control to control one or both of the steering and the speed of the vehicle, and the driving control includes a first driving mode and a second driving mode in which the task imposed on the driver of the vehicle is heavier than in the first driving mode or the degree of assistance provided to the driver is less than in the first driving mode, and the driving control unit switches from the first driving mode to the second driving mode when a state in which the deviation degree is equal to or greater than a threshold continues for a predetermined period of time or more while executing the first driving mode.

[0012] (7): In the aspect of (6) above, a notification control unit is further provided which notifies a driver of the vehicle of the control content in the driving control unit, and the notification control unit changes the content to be notified to the driver in accordance with the switching of the driving mode when the state in which the deviation degree is equal to or greater than a threshold continues for a predetermined period of time or more.

[0013] (8): In the above aspect (3), the learning unit re-learns the path estimation model using the path of the vehicle estimated by the path estimation unit and the path on which the vehicle actually traveled.

[0014] (9): A vehicle control method according to one embodiment of the present invention is a vehicle control method in which a computer controls driving of the vehicle on a route determined by first information based on the output of a detection device that detects the surrounding conditions of the vehicle and second information based on map information, estimates the route of the vehicle based on the first information and the second information, and determines the route of the vehicle based on the first information, the second information, and route estimation information related to the estimated route of the vehicle.

[0015] (10): A program according to one embodiment of the present invention is a program that causes a computer to control driving of a vehicle on a path determined by first information based on the output of a detection device that detects the surrounding conditions of the vehicle and second information based on map information, estimate the path of the vehicle based on the first information and the second information, and determine the path of the vehicle based on the first information, the second information, and path estimation information regarding the estimated path of the vehicle. Effect of the Invention

[0016] According to the above aspects (1) to (10), the accuracy of determining the vehicle's travel route can be improved. [Brief description of the drawings]

[0017] [Figure 1] 1 is a configuration diagram of a vehicle system 1 including a vehicle control device according to an embodiment. [Diagram 2] 2 is a functional configuration diagram of a first control unit 120 and a second control unit 160. FIG. [Diagram 3] A figure showing an example of the correspondence between driving modes, control states of the vehicle M, and tasks. [Figure 4] 11 is a diagram for explaining the processing of a learning unit 180. FIG. [Diagram 5] 11 is a diagram for explaining the processing of a recognition unit 130 and a deviation determination unit 152. FIG. [Figure 6] 13 is a diagram for explaining the determination of a travel path based on the determination result of the deviation determination unit 152. FIG. [Figure 7] 4 is a flowchart showing an example of a flow of processing executed by the automatic driving control device 100. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, with reference to the drawings, an embodiment of a vehicle control device, a vehicle control method, and a program of the present invention will be described. Hereinafter, an embodiment in which the vehicle control device is applied to an autonomous vehicle will be described as an example. Autonomous driving means, for example, automatically (without relying on the driver's operation) controlling one or both of the steering and speed of the vehicle to execute driving control. The driving control may include various driving controls such as LKAS (Lane Keeping Assistance System), ALC (Auto Lane Changing), ACC (Adaptive Cruise Control System), CMBS (Collision Mitigation Brake System), etc. In addition, the driving control may include driving assistance control for the driver such as ADAS (Advanced Driver Assistance System). The driving of an autonomous vehicle may be controlled by the driver's manual driving.

[0019] [Overall configuration] 1 is a configuration diagram of a vehicle system 1 including a vehicle control device according to an embodiment. The vehicle (hereinafter referred to as vehicle M) on which the vehicle system 1 is mounted is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination of these. The electric motor operates using power generated by a generator connected to the internal combustion engine, or discharged power from a battery (storage battery) such as a secondary battery or a fuel cell.

[0020] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driver monitor camera 70, a driving operation device 80, an automatic driving control device 100, a driving force output device 200, a brake device 210, and a steering device 220. These devices and devices are connected to each other by a multiple communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. Note that the configuration shown in FIG. 1 is merely an example, and a part of the configuration may be omitted, or another configuration may be added. The automatic driving control device 100 is an example of a "vehicle control device". A combination of the camera 10, the radar device 12, the LIDAR 14, and the object recognition device 16 is an example of a "detection device DD". The HMI 30 is an example of an “output unit.” The detection device DD detects the surrounding conditions of the vehicle M.

[0021] The camera 10 is, for example, a digital camera using a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to any location of the vehicle M in which the vehicle system 1 is mounted. When capturing an image of the front, the camera 10 is attached to the top of the front windshield, the back of the room mirror, the front of the vehicle body, or the like. When capturing an image of the rear, the camera 10 is attached to the top of the rear windshield, the back door, or the like. When capturing an image of the side, the camera 10 is attached to a door mirror or the like. The camera 10 periodically and repeatedly captures images of the periphery of the vehicle M, for example. The camera 10 may be a stereo camera.

[0022] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by surrounding objects (reflected waves) to detect at least the position (distance and direction) of the objects. The radar device 12 is attached to any location of the vehicle M. The radar device 12 may detect the position and speed of the objects by an FM-CW (Frequency Modulated Continuous Wave) method.

[0023] The LIDAR 14 irradiates light around the vehicle M and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time between light emission and light reception. The irradiated light is, for example, a pulsed laser light. The LIDAR 14 is attached to an arbitrary position of the vehicle M.

[0024] The object recognition device 16 performs sensor fusion processing on the detection results of some or all of the camera 10, the radar device 12, and the LIDAR 14 to recognize the position, type, speed, and the like of objects around the vehicle M. The objects include, for example, other vehicles (for example, surrounding vehicles existing within a predetermined distance from the vehicle M), pedestrians, bicycles, road structures, and the like. The road structures include, for example, road signs, traffic signals, railroad crossings, curbs, medians, guard rails, fences, and the like. The road structures may also include, for example, road markings such as road dividing lines (hereinafter simply referred to as "dividing lines") drawn or affixed to the road surface, crosswalks, bicycle crossings, and stop lines. The object recognition device 16 outputs the recognition results to the automatic driving control device 100. The object recognition device 16 may output the detection results of the camera 10, the radar device 12, and the LIDAR 14 to the automatic driving control device 100 as they are. In that case, the object recognition device 16 may be omitted from the configuration of the vehicle system 1 (specifically, the detection device DD). In addition, the object recognition device 16 may be included in the automatic driving control device 100.

[0025] The communication device 20 communicates with, for example, other vehicles in the vicinity of the vehicle M, a terminal device of a user using the vehicle M, or various server devices, using a network such as a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), a LAN (Local Area Network), a WAN (WiDe Area Network), or the Internet.

[0026] The HMI 30 outputs various information to occupants (including the driver) of the vehicle M and accepts input operations by the occupants. The HMI 30 includes, for example, various display devices, a touch panel, switches, keys, a speaker, a buzzer, a microphone, and the like. The HMI 30 may also include light-emitting units such as indicators and lamps that can light up or flash in one or more colors. The HMI 30 is provided in a position visible to the occupants, such as the instrument panel or the steering wheel 82.

[0027] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the vehicle M, an acceleration sensor that detects the acceleration, a yaw rate sensor that detects the yaw rate (for example, the rotational angular velocity around a vertical axis passing through the center of gravity of the vehicle M), and a direction sensor that detects the orientation of the vehicle M. The vehicle sensor 40 may also be provided with a position sensor that detects the position of the vehicle M. The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. The position sensor may also be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver 51 of the navigation device 50. The results detected by the vehicle sensor 40 are output to the automatic driving control device 100.

[0028] The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 holds first map information 54 in a storage device such as a hard disk drive (HDD) or a flash memory. The GNSS receiver 51 identifies the position of the vehicle M based on a signal received from a GNSS satellite. The position of the vehicle M may be identified or complemented by an inertial navigation system (INS) that uses the output of the vehicle sensor 40. The GNSS receiver 51 may be provided in the vehicle sensor 40. The above-mentioned position sensor and the GNSS receiver 51 are an example of a "position detection unit." The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, etc. The navigation HMI 52 may be partially or entirely shared with the above-mentioned HMI 30. The route determination unit 53 determines, for example, a route (hereinafter, a route on a map) from the position of the vehicle M specified by the GNSS receiver 51 (or an arbitrary position input) to a destination input by the occupant using the navigation HMI 52, with reference to the first map information 54. The first map information 54 is, for example, information in which a road shape is expressed by links indicating roads in a predetermined section and nodes connected by the links. The first map information 54 may include POI (Point Of Interest) information and the like. The route on the map is output to the MPU 60. The navigation device 50 may perform route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may transmit the current position and the destination to a navigation server via the communication device 20, and obtain a route equivalent to the route on the map from the navigation server. The navigation device 50 outputs the determined route on the map to the MPU 60.

[0029] The MPU 60 includes, for example, a recommended lane determination unit 61, and holds second map information 62 in a storage device such as an HDD or a flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, every 100 m in the vehicle travel direction), and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 determines, for example, which lane from the left the vehicle should travel in. Lanes are divided by dividing lines. When a branch point is present on the route on the map, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel along a reasonable route to proceed to the branch point.

[0030] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on road shapes and road structures. The road shapes include, for example, branching and merging, tunnels (entrances and exits), curved roads (entrances and exits), curvature of roads or demarcation lines, curvature radius, number of lanes, width, gradient, and the like, as road shapes that are more detailed than those in the first map information 54. The above information may be stored in the first map information 54. The information on road structures may include information on the type, position, orientation with respect to the extension direction of the road, size, shape, color, and the like of the road structures. For example, the demarcation line may be one type, and lane marks, curbs, median strips, and the like belonging to the demarcation line may be different types. In addition, the types of demarcation lines may include, for example, demarcation lines indicating that lane changes are possible and demarcation lines indicating that lane changes are not possible. The type of lane marking may be set for each road or lane section based on the link, for example, or multiple types may be set within one link.

[0031] The second map information 62 may include location information (latitude and longitude) of roads and buildings, address information (address and postal code), facility information, telephone number information, information on prohibited sections where mode A or mode B described later is prohibited, and the like. The second map information 62 may be updated as needed by the communication device 20 communicating with an external device. The first map information 54 and the second map information 62 may be provided as an integrated piece of map information. The map information (the first map information 54 and the second map information 62) may be stored in the storage unit 190.

[0032] The driver monitor camera 70 is, for example, a digital camera using a solid-state imaging element such as a CCD or a CMOS. The driver monitor camera 70 is attached to any location in the vehicle M in a position and orientation that allows it to capture an image of the head (in an orientation that captures the face) of the driver seated in the driver's seat of the vehicle M and other passengers seated in the passenger seat or the rear seat from the front. For example, the driver monitor camera 70 is attached to the upper part of a display device provided in the center of the instrument panel of the vehicle M, the upper part of the front windshield, the rearview mirror, etc. The driver monitor camera 70 periodically and repeatedly captures images including the interior of the vehicle cabin, for example.

[0033] The driving operator 80 includes, for example, a steering wheel 82, an accelerator pedal, a brake pedal, a shift lever, and other operators. The driving operator 80 is equipped with a sensor that detects the amount of operation or the presence or absence of operation, and the detection result is output to the automatic driving control device 100, or to some or all of the driving force output device 200, the brake device 210, and the steering device 220. The steering wheel 82 is an example of an "operator that accepts steering operation by the driver." The operator does not necessarily have to be annular, and may be in the form of an irregular steering wheel, a joystick, a button, or the like. The steering wheel 82 is equipped with a steering grip sensor 84. The steering grip sensor 84 is realized by a capacitance sensor or the like, and outputs a signal to the automatic driving control device 100 that can detect whether the driver is gripping the steering wheel 82 (meaning that the driver is in contact with the steering wheel in a state in which force can be applied). In addition, the driving operator 80 may be equipped with, for example, a reaction force device that adjusts the amount of steering or speed operation by the driver's manual driving.

[0034] The automatic driving control device 100 executes driving control to control one or both of the steering and the speed of the vehicle M based on the surrounding conditions of the vehicle M. For example, the automatic driving control device 100 includes a first control unit 120, a second control unit 160, an HMI control unit 170, a learning unit 180, and a storage unit 190. The first control unit 120, the second control unit 160, the HMI control unit 170, and the learning unit 180 are each realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). In addition, some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by cooperation between software and hardware. The above-mentioned programs may be stored in advance in a storage device (a storage device having a non-transient storage medium) such as an HDD or flash memory of the autonomous driving control device 100, or may be stored in a removable storage medium such as a DVD, CD-ROM, or memory card, and installed in the storage device of the autonomous driving control device 100 by mounting the storage medium (non-transient storage medium) in a drive device, card slot, or the like. The HMI control unit 170 is an example of a "notification control unit".

[0035] The storage unit 190 may be realized by the above-mentioned various storage devices, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a ROM (Read Only Memory), a RAM (Random Access Memory), or the like. For example, a road estimation model 192, a program, and various other information are stored in the storage unit 190. The road estimation model 192 is, for example, a trained model trained by the learning unit 180 to estimate the road of the vehicle M. In addition, the storage unit 190 may store map information (first map information 54, second map information 62).

[0036] FIG. 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130, a behavior plan generation unit 140, and a mode determination unit 150. The first control unit 120 realizes, for example, a function by AI (Artificial Intelligence) and a function by a pre-given model in parallel. For example, the function of "recognizing an intersection" may be realized by executing in parallel the recognition of an intersection by deep learning or the like and the recognition based on a pre-given condition (there are signals, road markings, etc. that can be pattern matched), and scoring and comprehensively evaluating both. This ensures the reliability of the automatic driving. In addition, the first control unit 120 executes control related to the automatic driving of the vehicle M based on, for example, instructions from the MPU 60, the HMI control unit 170, etc. A combination of the recognition unit 130, the behavior plan generation unit 140, and the second control unit 160 is an example of a "travel control unit". The driving control unit controls the driving of the vehicle M on a route (drivable area) determined by, for example, first information based on the output of a detection device that detects the surrounding conditions of the vehicle M and second information based on map information.

[0037] The recognition unit 130 recognizes the surrounding situation of the vehicle M based on the recognition result of the detection device DD (information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16). For example, the recognition unit 130 recognizes the type, position, speed, acceleration, and other conditions of the vehicle M and objects present around the vehicle M. The type of object may be, for example, whether the object is a vehicle or a pedestrian, or may be a type for identifying each vehicle. The position of the object is recognized as the position of an absolute coordinate system (hereinafter, vehicle coordinate system) with a representative point of the vehicle M (center of gravity, center of drive shaft, etc.) as the origin, and is used for control. The position of the object may be represented by a representative point such as the center of gravity, corner, or tip of the object in the traveling direction, or may be represented by a represented area. The speed includes, for example, the speed of the vehicle M and other vehicles in the traveling direction (longitudinal direction) of the lane on which the vehicle is traveling (hereinafter, referred to as longitudinal speed) and the speed of the vehicle M and other vehicles in the lateral direction of the lane (hereinafter, lateral speed). The "state" of an object may include, for example, if the object is a moving body such as another vehicle, the acceleration or jerk of the object, or its "behavioral state" (e.g., whether it is changing lanes or about to do so).

[0038] The recognition unit 130 includes, for example, a first recognition unit 132 and a second recognition unit 134. The first recognition unit 132 recognizes first information about the road of the vehicle M based on the output of the detection device DD that detects the surrounding situation of the vehicle M. The second recognition unit 134 recognizes second information about the road of the vehicle M based on the position information of the vehicle M and map information. Each of the first information and the second information includes, for example, information about the lane of the vehicle M (for example, shape, type, width, etc.) and demarcation line information that demarcates the lane of the vehicle M. The first information and the second information may also include lanes around the vehicle M (including lanes that exist in the traveling direction (for example, ahead)) and their demarcation line information. The first information and the second information may also include information about a drivable area within the lane, such as the right or left side of the lane.

[0039] The behavior plan generating unit 140 generates a behavior plan for driving the vehicle M by driving control such as automatic driving based on the result of recognition by the recognition unit 130, the driving mode determined by the mode determining unit 150, and the like. For example, the behavior plan generating unit 140 generates a target trajectory for the vehicle M to travel in the recommended lane determined by the recommended lane determining unit 61 in principle, and further generates a target trajectory for the vehicle M to travel automatically (without the driver's operation) in the future so as to be able to respond to the surrounding situation of the vehicle M based on the recognition result by the recognition unit 130 or the recognition result of the surrounding road shape and the lane marking based on the current position of the vehicle M acquired from the map information. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequence of points (trajectory points) to be reached by the vehicle M. The trajectory points are points to be reached by the vehicle M at every predetermined driving distance (for example, about several [m]) along the road, and separately, a target speed (and a target acceleration) at every predetermined sampling time (for example, about a few tenths [sec]) is generated as part of the target trajectory. Also, the trajectory points may be positions to be reached by the vehicle M at each predetermined sampling time. In this case, the information on the target speed (and target acceleration) is expressed by the intervals between the trajectory points.

[0040] The behavior plan generating unit 140 may set an event of the automatic driving when generating the target trajectory. The event includes, for example, a constant speed driving event in which the vehicle M drives in the same lane at a constant speed, a following driving event in which the vehicle M follows another vehicle (hereinafter referred to as a forward vehicle) that exists within a predetermined distance (for example, within 100 [m]) ahead of the vehicle M and is closest to the vehicle M, a lane change event in which the vehicle M changes lanes from the own lane to an adjacent lane, a branching event in which the vehicle M branches into a lane on the destination side at a branching point on the road, a merging event in which the vehicle M merges into a main line at a merging point, a takeover event in which the automatic driving is terminated and the vehicle M is switched to manual driving, and the like. The behavior plan generating unit 140 generates a target trajectory according to the activated event.

[0041] The mode determination unit 150 determines the driving mode of the vehicle M to be one of a plurality of driving modes in which tasks are differently assigned to a driver (an example of a passenger). The mode determination unit 150 includes, for example, a driver state determination unit 151, a deviation determination unit 152, a travel path estimation unit 153, a travel path determination unit 154, and a mode change processing unit 155.

[0042] FIG. 3 is a diagram showing an example of the correspondence between the driving modes, the control state of the vehicle M, and the tasks. The driving modes of the vehicle M include, for example, five modes, from mode A to mode E. In the above five modes, the control state, i.e., the degree of automation of the driving control of the vehicle M, is highest in mode A, followed by mode B, mode C, mode D, and the lowest in mode E. Conversely, the tasks imposed on the driver are the lightest in mode A, followed by mode B, mode C, mode D, and the most severe in mode E. In addition, since the control state is not automatic driving in modes D and E, the automatic driving control device 100 has a responsibility to end the control related to automatic driving and to transition to driving assistance or manual driving. Below, the contents of each driving mode are illustrated.

[0043] In mode A, the vehicle is in an automatic driving state, and the driver is not required to monitor the surroundings of the vehicle M or to grip the steering wheel 82 (grip the steering wheel in the figure). The surrounding monitoring includes at least monitoring the area ahead of the vehicle M. However, even in mode A, the driver is required to be in a position to be able to quickly switch to manual driving in response to a request from a system centered on the automatic driving control device 100. Note that automatic driving here means that both steering and acceleration / deceleration are controlled without the driver's operation. The area ahead means the space in the traveling direction of the vehicle M that is visible through the front windshield. Mode A is a driving mode that can be executed when conditions are met, such as the vehicle M traveling at a predetermined speed (for example, about 50 [km / h]) or less on a motor vehicle-only road such as an expressway, and there is a vehicle ahead that is to be followed, and is sometimes called TJP (Traffic Jam Pilot). When this condition is no longer met, the mode determination unit 150 changes the driving mode of the vehicle M to mode B.

[0044] In mode B, the vehicle is in a driving assistance state, and the driver is tasked with monitoring the front of the vehicle M (hereinafter, forward monitoring), but is not tasked with gripping the steering wheel 82. In mode C, the vehicle is in a driving assistance state, and the driver is tasked with the task of monitoring the front and the task of gripping the steering wheel 82. Mode D is a driving mode in which the driver is required to perform some driving operation for at least one of steering and acceleration / deceleration of the vehicle M. For example, in mode D, driving assistance such as ACC and LKAS is performed. In mode E, the vehicle is in a manual driving state in which the driver is required to perform both steering and acceleration / deceleration. In both modes D and E, the driver is naturally tasked with the task of monitoring the front of the vehicle M. In the embodiment, for example, when mode A is the "first driving mode", modes B to E are examples of the "second driving mode". Also, when mode B is the "first driving mode", modes C to E are examples of the "second driving mode". In other words, the task imposed on the driver in the second driving mode is more severe than the task in the first driving mode.

[0045] The driving modes are not limited to those illustrated in FIG. 3, and may be defined by other definitions. For example, among driving modes that require both forward monitoring and gripping of the steering wheel, there may be driving modes with looser thresholds for determining that the steering wheel is being gripped and driving modes with stricter thresholds. More specifically, driving modes may be defined such that in one driving mode, it is sufficient for the driver to have either the left or right hand touching the steering wheel 82, while in another driving mode in which the task imposed on the driver is heavier, the driver must grip the steering wheel 82 with both hands with a strength equal to or greater than the threshold. Driving modes with different levels of task imposed on the driver may be defined in any other way.

[0046] In addition, the degree of driving assistance for the driver of the vehicle M (in other words, the degree of control of automatic driving) may differ depending on the driving mode. For example, in the second driving mode, the degree of driving assistance for the driver (the degree of control by automatic driving) is made smaller than in the first driving mode. In addition, when the mode is different even in the second driving mode, the degree of driving assistance may be made smaller as the task imposed on the driver becomes more severe. Reducing the degree of driving assistance includes, for example, lowering the maximum amount of steering assistance (for example, maximum torque). Reducing the degree of driving assistance may also include, instead of (or in addition to) the above, reducing the speed range in which assistance is possible in speed assistance (for example, upper speed limit).

[0047] The driver state determination unit 151 determines whether the occupant (driver) is in a state suitable for driving. For example, the driver state determination unit 151 monitors the state of the occupant for the above-mentioned mode change, and determines whether the state of the occupant is in a state corresponding to the task. For example, the driver state determination unit 151 analyzes an image captured by the driver monitor camera 70, performs a posture estimation process, and determines whether the occupant is in a position that does not allow the occupant to shift to manual driving in response to a request from the system. In addition, the driver state determination unit 151 analyzes an image captured by the driver monitor camera 70, performs a gaze estimation process, and determines whether the occupant is monitoring the periphery (more specifically, the front) of the vehicle M. If it is determined that the occupant is not in a state suitable for driving the task for a predetermined time or more, the driver state determination unit 151 determines that the occupant is in a state not suitable for driving the task, or that the task related to the driving mode is not being executed by the driver. If it is determined that the occupant is in a state suitable for driving the task, or that the task related to the driving mode is being executed by the driver, the driver state determination unit 151 determines that the occupant is in a state suitable for driving the task, or that the task related to the driving mode is being executed by the driver. In addition, if it is determined that the occupant is in a state suitable for driving the task, the driver state determination unit 151 determines that the occupant is in a state suitable for driving the task, or that the task related to the driving mode is being executed by the driver. The driver state determination unit 151 may also determine whether or not the occupant is in a state where he or she can take over as driver.

[0048] The deviation determination unit 152 determines whether there is a deviation (a difference) between the first information recognized by the first recognition unit 132 and the second information recognized by the second recognition unit. The deviation determination unit 152 may also obtain the degree of deviation between the first information and the second information.

[0049] When the deviation determination unit 152 determines that the first information and the second information deviate from each other, the path estimation unit 153 estimates the path of the vehicle M based on the first information and the second information, and outputs the estimation result (path estimation information) to the path determination unit 154. The estimation result includes information indicating which lane the vehicle M is estimated to be traveling in, between the lane included in the first information and the lane included in the second information. The estimation result may also include information regarding a drivable area within the lane, such as the right or left side of the lane. The estimation result may also include priority information indicating which information, the first information or the second information, is to be prioritized. The priority information may be, for example, information regarding the order of priority, or information indicating the priority ranking of each piece of information (for example, a priority ranking of 0.7 for the first information, a priority ranking of 0.3 for the second information, etc.).

[0050] The path determination unit 154 determines a path (drivable area) on which the vehicle M will travel based on the first information and the second information. For example, when the deviation determination unit 152 determines that the first information and the second information do not deviate from each other, the path determination unit 154 determines a path using one or both of the first information and the second information that are determined in advance, and when the deviation determination unit 152 determines that the first information and the second information deviate from each other, the path determination unit 154 determines a path based on the first information, the second information, and the estimation result by the path estimation unit 153. Details of the processing in the deviation determination unit 152, the path estimation unit 153, and the path determination unit 154 will be described later.

[0051] The mode change processing unit 155 determines or changes the driving mode to be executed by the vehicle M based on the determination result by the driver state determination unit 151, the determination result by the deviation determination unit 152, and the like. For example, when the driver does not execute a task related to the determined driving mode, the mode change processing unit 155 changes the driving mode of the vehicle M to a driving mode with a heavier task. For example, when the driver is in a position in which he cannot shift to manual driving in response to a request from the system in mode A (for example, when the driver continues to look away from the vehicle outside the permitted area or when a sign of driving difficulty is detected), the mode change processing unit 155 uses the HMI 30 to prompt the driver to shift to manual driving, and if the driver does not comply, the vehicle M is gradually stopped by pulling over to the shoulder of the road, and the automatic driving is stopped. After the automatic driving is stopped, the vehicle M enters a state of mode D or E, and the vehicle M can be started by the driver's manual operation. The same applies to "stopping automatic driving" below. In mode B, when the driver is not monitoring the road ahead, the mode change processing unit 155 uses the HMI 30 to prompt the driver to monitor the road ahead, and if the driver does not comply, the vehicle M is moved to the shoulder of the road and gradually stopped, and the automatic driving is stopped. In mode C, when the driver is not monitoring the road ahead or is not gripping the steering wheel 82, the mode change processing unit 155 uses the HMI 30 to prompt the driver to monitor the road ahead and / or grip the steering wheel 82, and if the driver does not comply, the vehicle M is moved to the shoulder of the road and gradually stopped, and the automatic driving is stopped. Changing of the driving mode based on the determination result by the deviation determination unit 152 will be described later.

[0052] The second control unit 160 controls the driving force output device 200, the brake device 210, and the steering device 220 so that the vehicle M passes the target trajectory generated by the behavior plan generation unit 140 at the scheduled time. The second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires information on the target trajectory (trajectory points) generated by the behavior plan generation unit 140, and stores it in a memory (not shown). The speed control unit 164 controls the driving force output device 200 or the brake device 210 based on a speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the degree of curvature of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the radius of curvature (or curvature) of the road ahead of the vehicle M and feedback control based on the deviation from the target trajectory.

[0053] The HMI control unit 170 notifies the occupant of predetermined information through the HMI 30. The predetermined information includes, for example, information related to the traveling of the vehicle M, such as information related to the state of the vehicle M and information related to driving control. The information related to the state of the vehicle M includes, for example, the speed of the vehicle M, the engine speed, the shift position, and the like. The information related to the driving control includes the control contents in the driving control unit, such as whether or not driving control by automatic driving is performed, information inquiring whether or not automatic driving is to be started, the status of driving control by automatic driving (for example, the driving mode being performed and the contents of an event), information related to the degree of driving assistance, information related to switching of the driving mode, and the like. The predetermined information may also include, for example, information related to the current position and destination of the vehicle M, and information related to the remaining amount of fuel. The predetermined information may also include information not related to the driving control of the vehicle M, such as television programs, content (for example, movies) stored in a storage medium such as a DVD, and the like.

[0054] For example, the HMI control unit 170 may generate an image including the above-mentioned predetermined information and display the generated image on the display device of the HMI 30, or may generate a sound indicating the predetermined information and output the generated sound from a speaker of the HMI 30. The HMI control unit 170 may also light or blink a light-emitting unit such as an indicator or lamp included in the HMI 30 in a predetermined color. The HMI control unit 170 may also output information received by the HMI 30 to the communication device 20, the navigation device 50, the first control unit 120, or the like. The HMI control unit 170 may also transmit various pieces of information to be output by the HMI 30 to a terminal device used by an occupant of the vehicle M via the communication device 20. The terminal device is, for example, a smartphone or a tablet terminal.

[0055] The learning unit 180 receives the first information recognized by the first recognition unit 132 and the second information recognized by the second recognition unit 134 as inputs, and generates a lane estimation model 192 that outputs an estimated result of the lane of travel of the vehicle M. FIG. 4 is a diagram for explaining the processing of the learning unit 180. For example, the learning unit 180 receives the first information and the second information as inputs, and generates a lane estimation model 192 that outputs an estimated result of the lane of the vehicle M, by a function of AI (Artificial Intelligence) such as machine learning (neural network) or deep learning using the first information and the second information prepared in advance and true value (correct answer) data. The true value data is, for example, information about the lane on which the vehicle actually traveled at the time when the first information and the second information are recognized.

[0056] Furthermore, the learning unit 180 may generate a route estimation model 192 that uses the first information, the second information, and information on the driving conditions as inputs and outputs an estimated result of the route by including information on the driving conditions when the first information and the second information are recognized in addition to the first information and the second information. The driving conditions are, for example, information on the time period during which the vehicle traveled, weather, and the like. The time period may include classification such as daytime or nighttime, information on the date or period, and information on the season. The driving conditions may also include traffic congestion, and the like. In particular, the first information recognized by the output of the detection device DD may have a different recognition result depending on the driving conditions at the time of detection, or the route may change due to the influence of road construction (for example, nighttime construction or end-of-year construction) or traffic congestion, and therefore, by learning including information on the driving conditions, a more accurate route estimation model 192 can be generated. Furthermore, the learning unit 180 may generate a lane estimation model 192 that takes the first information, the second information, and information about the vehicle type (and the driving conditions) as inputs and outputs the estimated result of the lane by including vehicle type information in addition to (or instead of) the information about the driving conditions. Since the installation positions and number of the detection devices DD, the recognition performance, etc. differ for each vehicle type, and the recognition results accordingly differ, a more accurate lane estimation model 192 can be generated by learning including information about the vehicle type.

[0057] The learning unit 180 may update the path estimation model 192 based on the path estimated by the path estimation unit 153 (or the path determined by the path determination unit 154) and the path actually traveled by the driver by manual driving (mode E) or the like. For example, when the path estimated by the path estimation unit 153 using the path estimation model 192 is a path based on the first information, and the path actually traveled by the driver by manual driving is a path based on the second information, the paths are different, so the learning unit 180 re-learns based on the first information, the second information, and the information on the actually traveled path, and updates the path estimation model. Note that the re-learning may be performed when the degree of deviation between the first information and the second information is equal to or greater than a threshold value, or when the path estimated by the path estimation unit 153 is different from the path actually traveled by the vehicle M by the driver's manual driving operation. By performing the re-learning, a more accurate path estimation model 192 can be generated. Furthermore, the lane estimation model 192 may be generated or updated by the learning unit 180, or may be acquired from an external device via a network.

[0058] The driving force output device 200 outputs a driving force (torque) for driving the vehicle M to the driving wheels. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, etc., and an ECU (Electronic Control Unit) that controls these. The ECU controls the above components according to information input from the second control unit 160 or information input from the accelerator pedal of the driving operation device 80.

[0059] The braking device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or information input from the brake pedal of the driving operation element 80, so that a brake torque corresponding to a braking operation is output to each wheel. The braking device 210 may include a backup mechanism that transmits hydraulic pressure generated by the operation of the brake pedal to the cylinder via a master cylinder. Note that the braking device 210 is not limited to the configuration described above, and may be an electronically controlled hydraulic brake device that controls an actuator according to information input from the second control unit 160 to transmit hydraulic pressure in the master cylinder to the cylinder.

[0060] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor changes the direction of the steered wheels by applying a force to, for example, a rack and pinion mechanism. The steering ECU drives the electric motor according to information input from the second control unit 160 or information input from the steering wheel of the driving operation device 80, to change the direction of the steered wheels.

[0061] [Recognition section, mode determination section] Hereinafter, the details of each function (excluding the function of the driver state determination unit 151) included in the recognition unit 130 and the mode determination unit 150 will be described. Note that in the following example, the first information recognized by the first recognition unit 132 and the second information recognized by the second recognition unit 134 will be described as lane marking information.

[0062] FIG. 5 is a diagram for explaining the processing of the recognition unit 130 and the deviation determination unit 152. In the example of FIG. 5, it is assumed that the vehicle M is traveling at a speed VM in the extension direction of the lane L1 (X-axis direction in the figure). Also, FIG. 5 shows the first demarcation lines LL1 and RL1 recognized by the first recognition unit 132 on the plane (XY plane) of the vehicle coordinate system, and the second demarcation lines LL2 and RL2 of the same coordinate system recognized by the second recognition unit 134. Also, the position (X1, Y1) shown in FIG. 5 indicates a representative point of the vehicle M when the first demarcation line is recognized by the first recognition unit 132, and the position (X2, Y2) indicates a representative point of the vehicle M detected by the position detection unit when the second demarcation line is recognized by the second recognition unit 134.

[0063] The first recognition unit 132 recognizes the left and right demarcation lines LL1 and RL1 that demarcate the driving lane of the vehicle M, for example, based on the output of the detection device DD. The demarcation lines LL1 and RL1 are an example of a "first demarcation line". For example, the first recognition unit 132 analyzes an image captured by the camera 10, extracts edge points in the image that have a large brightness difference with adjacent pixels, and recognizes the demarcation lines LL1 and RL1 in the image plane by connecting the edge points. In addition, the first recognition unit 132 converts the positions of the demarcation lines LL1 and RL1 into a vehicle coordinate system based on the position (X1, Y1) of a representative point (for example, the center of gravity or center) of the vehicle M. In addition, the first recognition unit 132 may recognize, for example, the radius of curvature or the curvature of the first demarcation lines LL1 and RL1.

[0064] The second recognition unit 134 recognizes, for example, the demarcation lines LL2 and RL2 that demarcate the lane of the vehicle M from the map information based on the position of the vehicle M detected by the position detection unit. The demarcation lines LL2 and RL2 are an example of a "second demarcation line." For example, the second recognition unit 134 acquires the position information of the vehicle M detected by the position detection unit, refers to the second map information 62 based on the acquired position information (position (X2, Y2)), and recognizes, from the second map information 62, the demarcation lines LL2 and RL2 that demarcate the lane present at the position of the vehicle M. In addition, the second recognition unit 134 recognizes the respective radii of curvature or curvatures of the demarcation lines LL2 and RL2 from the second map information 62.

[0065] The deviation determination unit 152 superimposes the position (X1, Y1) and the position (X2, Y2) on the plane (XY plane) of the vehicle coordinate system, and superimposes them so that the orientation of the vehicle M is the same, and acquires the degree of deviation between the left and right sides (the demarcation lines LL1 and LL2, and the demarcation lines RL1 and RL2). The deviation may be, for example, the deviation in the horizontal position (the Y-axis direction in the figure) (for example, the lateral deviation amount W1 between the demarcation lines LL1 and LL2 in the figure), the difference in the vertical position (the length of the distance in the X-axis direction in the figure), or a combination thereof. Moreover, instead of (or in addition to) the above content, the deviation may be the angle (peeling angle) between the demarcation lines LL1 and LL2, or the angle between the demarcation lines RL1 and RL2. The deviation determination unit 152 increases the degree of deviation as the lateral deviation amount W1 or the peeling angle increases.

[0066] The deviation determination unit 152 determines that the first and second demarcation lines are not deviated (match) when the degree of deviation between the demarcation lines LL1 and LL2 and between the demarcation lines RL1 and RL2 is less than a threshold value, and determines that the first and second demarcation lines are deviated (mismatch) when the degree of deviation is equal to or greater than the threshold value. The deviation determination unit 152 may also determine that the first and second demarcation lines are not matched when the state in which the degree of deviation is equal to or greater than the threshold value continues for a predetermined time or more. This makes it possible to prevent the determination result of the degree of deviation from being frequently switched, thereby making it possible to further stabilize driving control based on the determination result.

[0067] 6 is a diagram for explaining the determination of the path based on the determination result of the deviation determination unit 152. When it is determined that the first and second demarcation lines are deviated, the deviation determination unit 152 outputs information on the first and second demarcation lines to the path estimation unit 153 and the path determination unit 154. The path estimation unit 153 uses the path estimation model 192 to obtain an estimation result of the path of the vehicle M in response to the input of the first and second demarcation lines, and outputs the estimation result to the path determination unit 154.

[0068] The road estimation unit 153 may acquire information about the driving conditions (such as the current time and weather information around the vehicle M) from the vehicle M or an external device connected via the communication device 20, and may also input the acquired information to the road estimation model 192 to acquire an estimation result. The road estimation unit 153 may also input vehicle type information of the vehicle M to the road estimation model 192 in addition to (or instead of) the information about the driving conditions to acquire an estimation result. This makes it possible to acquire an estimation result according to differences in recognition performance based on the driving conditions and vehicle type.

[0069] The path determination unit 154 determines the path of the vehicle M from information on the first and second demarcation lines. For example, if the degree of deviation between the first and second demarcation lines is less than a threshold (if they match), the path determination unit 154 uses one or both of the first and second demarcation lines to determine the path (drivable area) on which the vehicle M will travel. Using both means, for example, using both demarcation lines in an overlapping manner, or using one demarcation line while also using at least a portion of the other demarcation line in an interpolating manner.

[0070] Also, when the degree of deviation between the first and second demarcation lines is equal to or greater than a threshold (does not match), the path determination unit 154 determines the path of the vehicle M based on the first and second demarcation lines and the estimation result (e.g., priority information) of the path estimation unit 153. For example, when the first demarcation line has a higher priority than the second demarcation line in the estimation result, the path determination unit 154 determines the lane defined by the first demarcation line as the path. Also, when the first information indicates a travelable area on the right side of the lane and the second information indicates a travelable area on the left side of the same lane in the estimation result, and the first information has a higher priority than the second information, the path determination unit 154 determines the right side of the lane as the path. Also, the path determination unit 154 may adjust the position of the path according to the respective priority levels of the first information and the second information to determine the final path. In this way, when the first lane marking and the second lane marking deviate from each other, the lane estimation is performed using the lane estimation model 192, thereby making it possible to further improve the accuracy of lane determination.

[0071] For example, when autonomous driving or driving assistance is performed based on the driving mode to be executed, the action plan generation unit 140 generates a target trajectory for the vehicle M to travel along the center of the road determined by the road determination unit 154, and outputs the generated target trajectory to the second control unit 160 to execute driving control.

[0072] The mode change processing unit 155 changes the driving mode based on, for example, the determination result by the deviation determination unit 152. For example, when both the left and right demarcation lines of the first and second demarcation lines match (the deviation degree is less than a threshold value) and other conditions for enabling the first driving mode are satisfied, the mode change processing unit 155 changes the driving mode from the second driving mode to the first driving mode, changes the second driving mode to a mode in which the task imposed on the driver is light, or continues the first driving mode being executed. In addition, when the deviation degree of at least one of the left and right demarcation lines between the first and second demarcation lines is equal to or greater than a threshold value while the first driving mode is being executed, the mode change processing unit 155 executes a process for switching from the first driving mode to a second driving mode (for example, mode C to mode E). Note that the mode change processing unit 155 may suppress switching from the first driving mode to the second driving mode until the state in which the deviation degree is equal to or greater than the threshold value continues for a predetermined time or more. This makes it possible to suppress the change of the driving mode when the deviation degree is equal to or greater than the threshold only temporarily, and to realize more stable driving control. In addition, the mode change processing unit 155 may gradually switch the multiple modes included in the second driving mode to a mode in which the task imposed on the occupant becomes heavier depending on the duration of the state in which the deviation degree is equal to or greater than the threshold. In this case, the longer the duration, the more the mode is switched to a mode in which the task becomes heavier.

[0073] When the driving mode is switched from the first driving mode to the second driving mode due to the deviation between the first and second dividing lines, the HMI control unit 170 may output information indicating the reason for the switching to the HMI 30 in addition to the switching of the driving mode. The HMI control unit 170 may also output information on the task (for example, forward monitoring or gripping the steering wheel 82) assigned to the occupant for switching the driving mode to the HMI 30. When the driving mode is switched from the first driving mode to the second driving mode due to the deviation between the first and second dividing lines, the HMI control unit 170 may also output information on the support state according to the driving support degree to the HMI 30. The HMI control unit 170 may also change the light emission state (display state) of the light emitting unit such as an indicator or lamp included in the HMI 30 according to the change of the driving mode or the change of the support state. The light emission mode is, for example, lighting, blinking, blinking cycle, light emission color, etc. This allows the driver to clearly understand the change in the driving assistance state.

[0074] [Variations] In the above-described embodiment, the learning unit 180 may train the lane estimation model 192 in a server (external device) that can communicate with the communication device 20. In this case, the learning unit 180 transmits information such as the recognition results and driving history of the first recognition unit 132 and the second recognition unit 134, the driving conditions and the vehicle type of the vehicle M, etc., to the server via the communication device 20. The server receives the information transmitted from the vehicle M and also receives similar information from other vehicles, generates the lane estimation model 192 using the received information, and distributes the generated lane estimation model 192 to each vehicle. As a result, the lane estimation model can be generated using more information, so that more accurate lane estimation can be realized. In addition, the processing load of the vehicle M can be reduced by having the server execute the learning process.

[0075] In addition, in the embodiment, the deviation judgment unit 152 may output the first information and the second information to the path estimation unit 153 and execute the path estimation process even if the degree of deviation between the first information and the second information is less than the threshold (if they match). In this case, the path determination unit 154 determines the path based on the first information, the second information, and the estimation result obtained from the path estimation unit 153, regardless of the degree of deviation between the first information and the second information. In addition, the path determination unit 154 may switch whether to refer to the estimation result when the degree of deviation between the first information and the second information is less than the threshold (if they match) according to the instruction content from the driver input from the HMI 30. This allows the path determination process to be performed according to the driver's intention.

[0076] [Processing flow] Next, a flow of processing executed by the automatic driving control device 100 according to the embodiment will be described. FIG. 7 is a flowchart showing an example of a flow of processing executed by the automatic driving control device 100. In the following, among the processing executed by the automatic driving control device 100, a processing of determining a route and switching a driving mode of the vehicle M based on the degree of deviation of the lane markings recognized by the first recognition unit 132 and the second recognition unit 134 will be mainly described. In addition, it is assumed that the route estimation model 192 has already been generated at the start of the processing shown in FIG. 7. It is also assumed that the vehicle M is being controlled by the first driving mode (for example, mode A). In addition, in the following processing, it is assumed that the driver's state is a state suitable for the mode being executed or the mode after switching in the determination result by the driver state determination unit 151 (that is, a situation in which mode switching does not occur based on the determination result of the driver state determination unit 151). The processing shown in FIG. 7 may be repeatedly executed at a predetermined timing.

[0077] 7, the first recognition unit 132 recognizes a first dividing line that divides the lane of the vehicle M based on the output of the detection device DD (step S100). Next, the second recognition unit 134 refers to map information based on the position information of the vehicle M obtained from the position detection unit, and recognizes a second dividing line that divides the lane of the vehicle M (step S110). Note that the processing of steps S100 and S110 may be performed in the reverse order or in parallel.

[0078] Next, the deviation determination unit 152 compares the first and second demarcation lines and determines whether the deviation of the demarcation lines is equal to or greater than a threshold (step S120). If it is determined that the deviation is not equal to or greater than the threshold, the path determination unit 154 determines the path of the vehicle M using one or both of the recognized first and second demarcation lines (step S130). Next, the driving control unit continues the first driving mode and executes driving control to drive the vehicle M along the determined path (step S140).

[0079] In addition, in the process of step S120, if it is determined that the deviation degree between the first and second demarcation lines is equal to or greater than the threshold, the deviation determination unit 152 determines whether the state in which the deviation degree is equal to or greater than the threshold continues for a predetermined time or more (step S150). If it is determined that the state in which the deviation degree is equal to or greater than the threshold does not continue for a predetermined time or more, the path estimation unit 153 estimates the path of the vehicle M based on the first and second demarcation lines using the path estimation model (step S160). Next, the path determination unit 154 determines the path of the vehicle M based on the recognized demarcation lines and the estimation result (step S170). Next, the driving control unit continues the first driving mode and executes driving control to drive the vehicle M along the determined path (step S180).

[0080] Furthermore, if it is determined in the process of step S150 that the state in which the deviation degree is equal to or greater than the threshold continues for a predetermined time or more, the driving control unit executes driving control to switch from the first driving mode to the second driving mode (step S190). Next, the HMI control unit 170 outputs information on the state of vehicle control (driving assistance) to the HMI 30 to notify the driver (step S200). This ends the process of this flowchart.

[0081] According to the embodiment described above, the vehicle control device is equipped with a driving control unit that controls the driving of the vehicle M on a route determined by first information based on the output of the detection device DD that detects the surrounding conditions of the vehicle and second information based on map information, a route estimation unit 153 that estimates the route of the vehicle M based on the first information and the second information, and a route determination unit 154 that determines the route of the vehicle M based on the first information, the second information, and route estimation information regarding the route of the vehicle M estimated by the route estimation unit 153, thereby improving the accuracy of determining the route of the vehicle.

[0082] According to the embodiment, when determining the route of the vehicle M based on the information of the lane markings based on the output of the detection device DD and the information of the lane markings acquired from the map information, the route is estimated using a model learned in advance, and the final route of the vehicle M is determined based on the estimated route, so that a more reliable and safer route can be determined. Furthermore, by determining the route using the route estimation result using the previously learned route estimation model, route determination can be realized in response to changes in the driving conditions, etc., rather than determining the route based on a simple rule base determined in advance. According to the embodiment, the route including not only the range of the vehicle position but also the traveling direction (for example, forward) of the vehicle M can be determined with high accuracy. Particularly in the field of autonomous driving, a target trajectory corresponding to the route on which the vehicle M will travel in the future is generated, and driving control is performed along the target trajectory, so that more appropriate driving control can be realized by improving the accuracy of route determination (route selection). Therefore, according to the embodiment, it is possible to contribute to the development of a sustainable transportation system.

[0083] The above-described embodiment can be expressed as follows. a storage medium for storing computer-readable instructions; a processor coupled to the storage medium; The processor executes the computer-readable instructions to: Controlling the travel of the vehicle on a route determined by first information based on an output of a detection device that detects a surrounding situation of the vehicle and second information based on map information; Estimating a path of the vehicle based on the first information and the second information; determining a route of the vehicle based on the first information, the second information, and route estimation information relating to the estimated route of the vehicle; Vehicle control device.

[0084] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0085] 1...vehicle system, 10...camera, 12...radar device, 14...LIDAR, 16...object recognition device, 20...communication device, 30...HMI, 40...vehicle sensor, 50...navigation device, 60...MPU, 70...driver monitor camera, 80...driving operator, 100...automatic driving control device, 120...first control unit, 130...recognition unit, 132...first recognition unit, 134...second recognition unit, 140...action plan generation generation unit, 150...mode determination unit, 151...driver state determination unit, 152...deviation determination unit, 153...roadway estimation unit, 154...roadway determination unit, 155...mode change processing unit, 160...second control unit, 162...acquisition unit, 164...speed control unit, 166...steering control unit, 170...HMI control unit, 180...learning unit, 190...storage unit, 200...traveling drive force output device, 210...brake device, 220...steering device

Claims

1. a travel control unit that controls travel of the vehicle on a route determined by first information based on an output of a detection device that detects a surrounding situation of the vehicle and second information based on map information; A path estimation unit that estimates a path of the vehicle based on the first information and the second information; a path determination unit that determines a path of the vehicle based on the first information, the second information, and path estimation information regarding the path of the vehicle estimated by the path estimation unit; A learning unit including a trained model that outputs the roadway estimation information based on the first information and the second information, The path estimation unit estimates a path of the vehicle by using the trained model. Vehicle control device.

2. a travel control unit that controls travel of the vehicle on a route determined by first information based on an output of a detection device that detects a surrounding situation of the vehicle and second information based on map information; A path estimation unit that estimates a path of the vehicle based on the first information and the second information; a path determination unit that determines a path of the vehicle based on the first information, the second information, and path estimation information regarding the path of the vehicle estimated by the path estimation unit; a learning unit that generates a roadway estimation model based on the first information, the second information, and true value data, the model having the first information and the second information as input and the roadway estimation information as output, The path estimation unit estimates a path of the vehicle based on the first information and the second information by using the path estimation model. Vehicle control device.

3. The trained model receives the first information and the second information as input. The vehicle control device according to claim 1.

4. The path estimation unit estimates a path of the vehicle when a degree of deviation between the first information and the second information is equal to or greater than a threshold value. The vehicle control device according to claim 1.

5. The first information includes information of a first dividing line that divides a travel lane of the vehicle and is recognized based on an output of the detection device; The second information includes information of a second dividing line that divides a lane in which the vehicle is traveling, the second information being obtained from the map information based on the position information of the vehicle. The vehicle control device according to claim 1.

6. The path estimation unit estimates a path of the vehicle based on the first information, the second information, and a driving condition and / or vehicle type information of the vehicle. The vehicle control device according to claim 1.

7. The traveling control unit executes driving control to control one or both of the steering and the speed of the vehicle, The driving control includes a first driving mode and a second driving mode in which a task imposed on a driver of the vehicle is heavier than in the first driving mode or a degree of assistance to the driver is smaller than in the first driving mode; The traveling control unit switches from the first driving mode to the second driving mode when a state in which the deviation degree is equal to or greater than a threshold continues for a predetermined time or more while the first driving mode is being executed. The vehicle control device according to claim 4.

8. A notification control unit notifies a driver of the vehicle of the control content in the driving control unit, The notification control unit changes the content of the notification to be given to the driver in accordance with the switching of the driving mode when a state in which the deviation degree is equal to or greater than a threshold continues for a predetermined time or more. The vehicle control device according to claim 7.

9. The learning unit re-learns the path estimation model by using the path of the vehicle estimated by the path estimation unit and a path on which the vehicle has actually traveled. The vehicle control device according to claim 2.

10. The computer Controlling the travel of the vehicle on a route determined by first information based on an output of a detection device that detects a surrounding situation of the vehicle and second information based on map information; Estimating a route of the vehicle based on the first information and the second information; determining a route of the vehicle based on the first information, the second information, and route estimation information relating to the estimated route of the vehicle; estimating a path of the vehicle using a trained model that outputs the path estimation information based on the first information and the second information; A vehicle control method.

11. On the computer, controlling the traveling of the vehicle on a route determined by first information based on an output of a detection device that detects a surrounding situation of the vehicle and second information based on map information; estimating a path of the vehicle based on the first information and the second information; determining a route of the vehicle based on the first information, the second information, and route estimation information relating to an estimated route of the vehicle; estimating a route of the vehicle using a trained model that outputs the route estimation information based on the first information and the second information; program.

Citation Information

Patent Citations

  • Virtual lane generation device and program

    JP2015005132A

  • Ambient environment information generation method and ambient environment information generation device

    JP2019152667A

  • Driving control device, vehicle, driving control method and program

    JP2021149321A

  • Self position estimation device, movable body, self position estimation method, and self position estimation program

    JP2022125563A

  • Vehicle control device, vehicle control method, and program

    JP2022129695A