Vehicle control device, vehicle control method, and program

Abstract

To provide a vehicle control device, a vehicle control method, and a program capable of executing more appropriate driving control according to a peripheral situation of a vehicle.SOLUTION: A vehicle control device according to an embodiment includes a recognition unit configured to recognize a surrounding situation of a vehicle, a driving control unit configured to execute driving control for controlling at least one of steering and a speed of the vehicle based on the surrounding situation, and an operation unit configured to receive an operation from an occupant of the vehicle, in which the driving control unit executes the driving control based on any one of a plurality of control modes including a first control and a second control having a content of driving assistance different from that of the first control, in a case where the control mode can be switched on the basis of the surrounding situation, switching from the first control to the second control or switching from the second control to the first control is executed in a case where an instruction from the operation unit is received, and the switching is not executed in a case where the instruction from the operation unit is not received.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a program. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transportation systems that take into consideration vulnerable traffic participants have been gaining momentum. To achieve this, efforts are being focused on research and development into driving assistance technologies to further improve traffic safety and convenience. In this context, a technology is known that includes a reception unit that receives from a passenger a selection operation of one or more driving modes from a plurality of driving modes that have different control characteristics regarding the acceleration / deceleration or turning of the vehicle, and that performs automatic driving of the vehicle based on the driving mode selected by the reception unit (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-206152 Summary of the Invention [Problem to be solved by the invention]

[0004] In driving assistance technology, it is desirable to encourage switching of control modes according to the vehicle's surrounding conditions and to prevent the operation from becoming complicated due to the occupant having to perform different operations for each control mode, but there is still room for further study on this point.

[0005] The present application has been made in consideration of the above circumstances, and one of its objects is to provide a vehicle control device, a vehicle control method, and a program that can execute more appropriate driving control in accordance with the surrounding conditions of the vehicle, thereby contributing to the development of a sustainable transportation system. [Means for solving the problem]

[0006] A vehicle control device, a vehicle control method, and a program according to the present invention employ the following configuration. (1): A vehicle control device according to one embodiment of the present invention includes a recognition unit that recognizes the surrounding conditions of the vehicle, a driving control unit that executes driving control to control at least one of the steering and speed of the vehicle based on the surrounding conditions, and an operation unit that receives operations from an occupant of the vehicle, wherein the driving control unit executes the driving control based on any one of a plurality of control modes including a first control and a second control that has a different driving assistance content from the first control, and when it is possible to switch the control mode based on the surrounding conditions and an instruction is received from the operation unit, the vehicle control device switches from the first control to the second control or from the second control to the first control, and does not execute the switching when an instruction is not received from the operation unit.

[0007] (2): In the above aspect (1), when the second control is being executed, the operation control unit determines whether or not to execute a predetermined operation based on the content received by the operation unit, and when the first control is being executed, the operation control unit executes the predetermined operation regardless of whether or not an operation is performed by the operation unit.

[0008] (3) In the above aspect (2), the predetermined action includes changing lanes of the vehicle.

[0009] (4): In the above aspect (1), when the state of the vehicle satisfies a predetermined condition while the first control or the second control is being executed, the driving control unit switches to a third control that is different from the first control and the second control, and the instruction to switch to the third control is performed by an operation that is different from the operation for switching from the first control to the second control and the operation for switching from the second control to the first control.

[0010] (5): In the above aspect (4), the operation control unit switches to a control different from the first control or the second control when the execution conditions of the third control are no longer satisfied during execution of the third control.

[0011] (6): In the above aspect (1), the driving control unit includes a first control device that executes the first control and a second control device that executes the second control, and in the first control, the first control device generates a target trajectory of the vehicle, and in the second control, the second control device generates a target trajectory of the vehicle.

[0012] (7): In the above aspect (1), the operation unit includes a plurality of operators including a first operator, a second operator, and a third operator, the first operator accepts instructions to execute or switch between the first control or the second control, and accepts instructions to start or end execution of a predetermined operation when the second control is being executed, the second operator accepts a set speed of the vehicle for the predetermined operation, and accepts lane change instructions, and the third operator operates the information displayed on the display unit when the second control is being executed, and accepts instructions regarding lane changes when the first control is being executed.

[0013] (8) In the above aspect (1), a notification unit is further provided that notifies the driver of the current and future vehicle states when the execution conditions for the control aspect are satisfied.

[0014] (9): In the above aspect (8), the driving control unit does not switch to the first control if, during execution of the second control, the operation unit is not operated for a predetermined period of time after making an inquiry to the notification unit regarding switching to the first control.

[0015] (10): In the aspect (1) above, the driving control unit distinguishes between an area where the first control can be performed and an area where the second control can be performed based on the road conditions of the vehicle.

[0016] (11): A vehicle control method according to one embodiment of the present invention is a vehicle control method in which a computer recognizes a surrounding situation of a vehicle, and executes driving control to control at least one of the steering and speed of the vehicle based on the surrounding situation, and executes the driving control based on any one of a plurality of control modes including a first control and a second control that has a different driving assistance content from the first control, and when it is possible to switch the control mode based on the surrounding situation and an instruction is received from an operation unit that receives operations from an occupant of the vehicle, the computer switches from the first control to the second control or switches from the second control to the first control, and when an instruction is not received from the operation unit, the computer does not execute the switching.

[0017] (12): A program according to one aspect of the present invention causes a computer to recognize the surrounding conditions of a vehicle, and executes driving control that controls at least one of the steering and speed of the vehicle based on the surrounding conditions, and executes the driving control based on any one of a plurality of control modes including a first control and a second control that provides driving assistance different from the first control, and when it is possible to switch the control mode based on the surrounding conditions and an instruction is received from an operation unit that receives operations from an occupant of the vehicle, the program switches from the first control to the second control or from the second control to the first control, and does not execute the switching when an instruction is not received from the operation unit. [Effects of the Invention]

[0018] According to the above aspects (1) to (12), more appropriate driving control can be executed in accordance with the surrounding conditions of the vehicle. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a configuration diagram of a vehicle system 1 that uses a vehicle control device according to an embodiment. [Figure 2] FIG. 2 is a functional configuration diagram of a first control unit 140. [Figure 3] FIG. 2 is a diagram for explaining a driving scene in which the first control and the second control can be executed. [Figure 4] FIG. 4 is a diagram for explaining a specific example of the application range of the first control and the second control. [Figure 5] FIG. 10 is a diagram for explaining a specific example of a further application range of the first control. [Figure 6] FIG. 2 is a diagram showing an example of an operation unit 46. [Figure 7] 4 is a flowchart showing an example of a flow of processing executed by the vehicle system 1 of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, embodiments of a vehicle control device, a vehicle control method, and a program according to the present invention will be described with reference to the drawings.

[0021] [Overall configuration] FIG. 1 is a configuration diagram of a vehicle system 1 that uses a vehicle control device according to an embodiment. The vehicle on which the vehicle system 1 is mounted may be, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source may be an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination of these. The electric motor operates using power generated by a generator connected to the internal combustion engine, or discharge power from a secondary battery or a fuel cell. In the following description, the vehicle will be described as a hybrid vehicle that uses a four-wheeled internal combustion engine and an electric motor as drive sources.

[0022] The vehicle system 1 includes, for example, a camera 10, a LIDAR (Light Detection and Ranging) unit 20, a communication device 30, an HMI (Human Machine Interface) 40, a vehicle sensor 50, a driver monitor camera 60, a driving operator 70, a steering grip sensor 74, a power supply unit 78, a navigation device 80, an MPU (Map Positioning Unit) 90, and a first control device 100.

[0023] The vehicle system 1 further includes, for example, a second control device 200, a camera 310, a radar device 320, a driving force output device 400, a braking device 410, and a steering device 420. The vehicle system 1 further includes, for example, a switching control unit 300. The HMI 40 is an example of a "notification unit." The first control device 100, the second control device 200, the switching control unit 300, the HMI 40, the driving operator 70, and the navigation device 80 are examples of a "driving control device." The first recognition unit 120 and the second recognition unit 210 are examples of a "recognition unit." The first control unit 140, the first vehicle control unit 160, the second control unit 220, the second vehicle control unit 230, and the switching control unit 300 are examples of a "driving control unit." The driving control unit in the embodiment executes driving control based on one of a plurality of control modes, including a first control (described later) and a second control (described later) that provides driving assistance different from the first control.

[0024] These devices and equipment are connected to each other via multiplexed communication lines such as a CAN (Controller Area Network) communication line, serial communication lines, wireless communication networks, etc. The configurations shown in FIG. 1 and FIG. 2 (described later) are merely examples, and some of the configurations may be omitted, or other configurations may be added. Furthermore, each functional configuration may be integrated or provided in a distributed manner.

[0025] The camera 10 is, for example, a digital camera using a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to any location of a vehicle (hereinafter referred to as vehicle M) in which the vehicle system 1 is installed. When capturing an image of the front, the camera 10 is attached to the top of the front windshield, the back of the rearview mirror, or the like. The camera 10, for example, periodically captures images of the periphery of the vehicle M. The camera 10 may be a stereo camera.

[0026] The LIDAR unit 20 irradiates the surroundings of the vehicle M with light (or electromagnetic waves with a wavelength similar to that of light) and measures the scattered light. The LIDAR unit 20 detects the distance to the target based on the time between emitting and receiving the light. The irradiated light is, for example, pulsed laser light. The LIDAR unit 20 may be attached to any location. However, the sensor portion of the LIDAR unit 20 is attached to a position, such as the roof, from which information about the surroundings including the front of the vehicle M can be acquired. The LIDAR unit 20 may be provided with an ECU (Electronic Control Unit).

[0027] The communication device 30 communicates with other vehicles in the vicinity of the vehicle M, for example, using a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), etc., or communicates with various server devices via a wireless base station.

[0028] The HMI 40 presents various information to an occupant of the vehicle M and accepts input operations by the occupant. The HMI 40 includes, for example, a display unit 42, a speaker 44, and an operation unit 46. The display unit 42 is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display device. The display unit 42 is provided, for example, on an instrument panel or a meter display unit. The display unit 42 displays various images (including videos) in the embodiment. The display unit 42 may be integrated with the input unit as a touch panel. The speaker 44 outputs a predetermined sound (e.g., an alarm) into the vehicle cabin. The operation unit 46 accepts operations by the occupant to, for example, start or end a control mode of the vehicle M described below or to switch to another control mode. The operation unit 46 may also accept operations such as setting parameters (e.g., speed adjustment) for an ongoing control or switching modes. The operation unit 46 may be provided, for example, on the steering wheel 72 or the like, or on a touch panel of the display unit 42. The HMI 40 may also include a microphone, a buzzer, a vibration generator (vibrator), keys, and the like in addition to the display unit 42, speaker 44, and operation unit 46. The HMI 40 may also include an output unit that prompts the driver to grip the steering wheel, and a HUD (Head Up Display).

[0029] The vehicle sensor 50 includes various sensors used for controlling the vehicle, such as a vehicle speed sensor that detects the speed of the vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity around a vertical axis, and a direction sensor that detects the orientation of the vehicle M. The vehicle sensor 50 may also be provided with a position sensor that detects the position of the vehicle M. The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. The position sensor may also be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver 81 of the navigation device 80.

[0030] The driver monitor camera 60 is, for example, a digital camera that uses a solid-state imaging element such as a CCD or CMOS. The driver monitor camera 60 is attached to any location in the vehicle M in a position and orientation that allows it to capture an image of the head of the driver seated in the driver's seat of the vehicle M from the front (in an orientation that captures an image of the face). For example, the driver monitor camera 60 is attached to the top of a display device provided in the center of the instrument panel of the vehicle M.

[0031] The driving operators 70 include, for example, a steering wheel 72, an accelerator pedal, a brake pedal, a shift lever, and other operators. The driving operators 70 are equipped with sensors that detect the amount of operation or the presence or absence of operation, and the detection results are output to the first control device 100, the second control device 200, the switching control unit 300, or some or all of the driving force output device 400, the brake device 410, and the steering device 420. The steering wheel 72 may be formed, for example, in an annular shape, or in the form of an irregularly shaped steering wheel, a joystick, buttons, or the like. A steering grip sensor 74 is attached to the steering wheel 72. The steering grip sensor 74 is realized by a capacitance sensor or the like, and outputs a signal capable of detecting whether the driver is gripping the steering wheel 72 (meaning whether the driver is in contact with the steering wheel in a state where force can be applied) to the first control device 100, the second control device 200, or the switching control unit 300. The steering wheel 72 may also be provided with an operation unit 46 for switching the control mode for the vehicle M, setting parameters, and the like.

[0032] The power supply unit 78 is a battery that supplies power to the vehicle system 1. The power supply unit 78 may include multiple batteries and may be configured as redundant so that if one battery fails, power is supplied from the other battery.

[0033] The navigation device 80 includes, for example, a GNSS receiver 81, a navigation HMI 82, and a route determination unit 83. The navigation device 80 stores first map information 84 in a storage device such as a hard disk drive (HDD) or flash memory. The GNSS receiver 81 identifies the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be identified or supplemented by an inertial navigation system (INS) that uses the output of the vehicle sensors 50. The navigation HMI 82 includes a display device, a speaker, a touch panel, keys, etc. The navigation HMI 82 may share some or all of the components with the HMI 40 described above. The route determination unit 83, for example, determines a route (hereinafter, a route on a map) from the position of the vehicle M identified by the GNSS receiver 81 (or any input position) to a destination input by the occupant using the navigation HMI 82, by referring to the first map information 84. The first map information 84 is, for example, information that represents road shapes using links indicating roads and nodes connected by the links. The first map information 84 may also include POI (Point Of Interest) information and the like. The route on the map is output to the MPU 90. The navigation device 80 may provide route guidance using the navigation HMI 82 based on the route on the map. The navigation device 80 may be realized, for example, by the functions of a terminal device such as a smartphone or tablet device owned by the occupant. The navigation device 80 may transmit the current position and destination to a navigation server via the communication device 30 and obtain a route equivalent to the route on the map from the navigation server.

[0034] The MPU 90 includes, for example, a recommended lane determination unit 91, and stores second map information 92 in a storage device such as an HDD or flash memory. The recommended lane determination unit 91 divides the route on the map provided by the navigation device 80 into a plurality of blocks (for example, by dividing each block into 100 m in the vehicle travel direction), and determines a recommended lane for each block by referring to the second map information 92. The recommended lane determination unit 91 determines, for example, which lane from the left the vehicle should travel in. When a branch point is present on the route on the map, the recommended lane determination unit 91 determines the recommended lane so that the vehicle M can travel on a reasonable route to the branch point. The MPU 90 also recognizes the position of the vehicle M based on the detection results of a gyro sensor (not shown), the position of the vehicle M identified by the GNSS receiver 81, and the like.

[0035] The second map information 92 is map information with higher accuracy than the first map information 84. The second map information 92 includes, for example, information on the centers of lanes or information on lane boundaries. The second map information 92 may also include road information, traffic regulation information, address information (address and postal code), facility information, telephone number information, and the like. The road information may include, for example, road type information such as expressways and general roads, road shape information such as merging, branching, T-junctions, and curvature (or curvature radius), and other road information such as the number of lanes, road gradient, junctions (JCTs), service areas, toll booths, and zebra strips (guide zones). The second map information 92 may be updated as needed by the communication device 30 communicating with other devices.

[0036] [First control device] The first control device 100 includes, for example, a first recognition unit 120, a first control unit 140, and a first vehicle control unit 160. The first recognition unit 120, the first control unit 140, and the first vehicle control unit 160 are each realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or an SOC (System On Chip), or may be realized by a combination of software and hardware. The program may be stored in advance in a device (a storage device with a non-transitory storage medium) such as the HDD or flash memory of the first control device 100, or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the first control device 100 by inserting the storage medium (non-transitory storage medium) into a drive device.

[0037] The first recognition unit 120 performs sensor fusion processing on the detection results from some or all of the camera 10 and the LIDAR unit 20 to recognize the surrounding situation of the vehicle M. For example, the second recognition unit 210 recognizes the position, type, speed, etc. of objects present around the vehicle M (within a predetermined distance) from the results of the sensor fusion processing. This function may be included in the LIDAR unit 20, or may be provided as a configuration separate from the LIDAR unit 20 and the first control device 100. The first recognition unit 120 may also perform sensor fusion processing using the detection results from the camera 310 or the radar device 320 (described later). The position of an object is recognized as a position on an absolute coordinate system with a representative point of the vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or may be represented by a represented area. The "state" of an object may include the acceleration or jerk of the object, or its "behavioral state" (for example, whether the object is changing lanes or is about to change lanes).

[0038] The first recognition unit 120 recognizes, for example, the lane in which the vehicle M is traveling (driving lane). For example, the first recognition unit 120 recognizes the driving lane by comparing the pattern of road dividing lines (e.g., an arrangement of solid lines and dashed lines) obtained from the second map information 92 with the pattern of road dividing lines around the vehicle M recognized from an image captured by the camera 10. The first recognition unit 120 may recognize the driving lane by recognizing road boundaries (road boundaries) including not only road dividing lines but also road dividing lines, shoulders, curbs, medians, guardrails, etc. In this recognition, the position of the vehicle M obtained from the navigation device 80 and the processing results by the INS may be taken into consideration. The first recognition unit 120 recognizes stop lines, obstacles, red lights, toll booths, and other road phenomena.

[0039] When recognizing the driving lane, the first recognition unit 120 recognizes the position and attitude of the vehicle M with respect to the driving lane. For example, the first recognition unit 120 may recognize the deviation of the reference point of the vehicle M from the center of the lane and the angle it forms with a line connecting the centers of the lanes in the traveling direction of the vehicle M as the relative position and attitude of the vehicle M with respect to the driving lane. Alternatively, the first recognition unit 120 may recognize the position of the reference point of the vehicle M with respect to either side edge of the driving lane (a road dividing line or a road boundary) as the relative position of the vehicle M with respect to the driving lane.

[0040] The first recognition unit 120, for example, performs functions based on AI (Artificial Intelligence) and functions based on a predefined model in parallel. For example, the function of "recognizing intersections" may be realized by executing intersection recognition using deep learning or the like and recognition based on predefined conditions (such as traffic lights and road signs that can be pattern matched) in parallel, and then scoring and comprehensively evaluating both. This ensures the reliability of autonomous driving (driving control). The first recognition unit 120 may be omitted, and the processing results of the second recognition unit 210, which will be described later, may be used.

[0041] The first control unit 140 executes control to assist the occupant (driver) in driving based on the recognition result by the first recognition unit 120. Fig. 2 is a functional configuration diagram of the first control unit 140. The first control unit 140 includes, for example, a behavior plan generation unit 142 and a mode determination unit 144.

[0042] The behavior plan generation unit 142 generates a target trajectory along which the vehicle M will automatically travel in the future (without relying on the driver's operation) so that, in principle, the vehicle M will travel along the recommended lane determined by the recommended lane determination unit 91 and will be able to respond to the surrounding conditions of the vehicle M. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequence of points (trajectory points) that the vehicle M should reach. The trajectory points are points that the vehicle M should reach at every predetermined travel distance (for example, about several meters) along the road. Separately, target speeds and target accelerations are generated as part of the target trajectory for every predetermined sampling time (for example, about a few tenths of a second). Furthermore, the trajectory points may be positions that the vehicle M should reach at each sampling time for each predetermined sampling time. In this case, information on the target speed and target acceleration is expressed as the interval between trajectory points.

[0043] The behavior plan generation unit 142 may set an autonomous driving event when generating the target trajectory. The autonomous driving events include a constant speed driving event, a low speed following driving event, a lane change event, a branching event, a merging event, a takeover event, etc. The behavior plan generation unit 142 generates a target trajectory according to the activated event.

[0044] The mode determination unit 144 determines the driving mode of the vehicle M to be one of a plurality of driving modes that impose different tasks on the driver. The mode determination unit 144 includes, for example, a driver state determination unit 146 and a mode change processing unit 148.

[0045] Here, the vehicle system 1 is capable of executing a plurality of driving modes of the vehicle M. The plurality of driving modes are, for example, modes with different control states, i.e., different degrees of automation of the driving control of the vehicle M. A high degree of automation means that the vehicle system 1 controls the vehicle M to a high degree, in other words, the degree to which the driver intervenes in the control (driving operation) of the vehicle M is low. The tasks assigned to the driver vary depending on the degree of automation. For example, the higher the degree of automation, the lighter the tasks. Examples of tasks include the driver's monitoring of the road ahead, gripping the steering wheel 72, and operating acceleration and deceleration. For example, in a driving mode with a high degree of automation, automatic driving is performed without the driver being required to monitor the road ahead, grip the steering wheel 72, or operate acceleration and deceleration. Automatic driving means that both steering and acceleration and deceleration are controlled without the driver's operation. The term "forward" refers to the space in the traveling direction of the vehicle M as viewed through the front windshield. For example, when vehicle M is traveling at a predetermined speed (for example, about 60 km / h) or less on a motorway such as an expressway and a preceding vehicle to be followed is present, a driving mode is executed in which the driver is not required to perform the above tasks. This driving mode is sometimes referred to as TJP (Traffic Jam Pilot). When this condition is no longer satisfied, mode determination unit 144 changes the driving mode to another driving mode.

[0046] When the driver does not perform a task related to the determined driving mode (hereinafter, the current driving mode), the mode determination unit 144 changes the driving mode of the vehicle M to a driving mode with a more difficult task. For example, when the driver is in a highly automated driving mode and is in a position where he or she cannot switch to manual driving in response to a request from the system (for example, when the driver continues to look away from the vehicle outside the permitted area or when a sign of driving difficulty is detected), the mode determination unit 144 prompts the driver to switch to manual driving using the HMI 40 or a predetermined output unit that prompts the occupant to grip the steering wheel. If the driver does not comply, the mode determination unit 144 controls the vehicle M to move to the shoulder of the road and gradually stop, thereby stopping the automated driving. After the automated driving is stopped, the vehicle M switches to a driving mode with a lower automated level, and the driver can start the vehicle M manually. The same applies to "stopping automated driving" below.

[0047] The driver state determination unit 146 monitors the driver's state for the above mode change and determines whether the driver's state is appropriate for the task. For example, the driver state determination unit 146 analyzes the image captured by the driver monitor camera 60 and performs posture estimation processing to determine whether the driver is in a position that prevents them from switching to manual driving in response to a request from the system. The driver state determination unit 146 analyzes the image captured by the driver monitor camera 60 and performs line-of-sight estimation processing to determine whether the driver is monitoring the road ahead.

[0048] The mode change processing unit 148 performs various processes for changing the mode. For example, the mode change processing unit 148 instructs the behavior plan generation unit 142 to generate a target trajectory for stopping on the shoulder of the road, instructs the second control device 200 to operate, and controls the HMI 40 to prompt the driver to take action.

[0049] The first vehicle control unit 160 controls the driving force output device 400, the braking device 410, and the steering device 420 so that the vehicle M passes through the target trajectory generated by the action plan generation unit 142 at the scheduled time. The second vehicle control unit 230 may provide information about the target trajectory to the second control unit 200 and control the driving force output device 400, the braking device 410, and the steering device 420 via the second control unit 200. As an example, the first vehicle control unit 160 executes junction passage control and merging control for the vehicle M. The junction passage control is control for causing the vehicle M to maintain its lane within a junction and for selecting a lane in which the vehicle M will travel within a junction. The merging control is control for causing the vehicle M to change lanes into the merging lane when merging from a merging lane onto a main lane.

[0050] The driving force output device 400 outputs a driving force (torque) to the driving wheels for driving the vehicle M. The driving force output device 400 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, etc., and an ECU that controls these. The ECU controls the above components according to information input from the first control device 100 or the second control device 200, or information input from the accelerator pedal of the driving operator 70.

[0051] The braking device 410 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the first control device 100 or the second control device 200 or information input from the brake pedal of the driving operation device 70, so that a braking torque corresponding to the braking operation is output to each wheel. The braking device 410 may include a backup mechanism that transmits hydraulic pressure generated by operation of the brake pedal to the cylinder via a master cylinder. Note that the braking device 410 is not limited to the configuration described above, and may also be an electronically controlled hydraulic braking device that controls an actuator according to information input from the first control device 100 or the second control device 200 to transmit hydraulic pressure from the master cylinder to the cylinder.

[0052] The steering device 420 includes, for example, a steering ECU and an electric motor. The electric motor changes the direction of the steered wheels by applying a force to, for example, a rack and pinion mechanism. The steering ECU drives the electric motor to change the direction of the steered wheels in accordance with information input from the first control device 100 or the second control device 200, or information input from the steering wheel of the driving operator 70.

[0053] The camera 310 is, for example, a digital camera using a solid-state imaging element such as a CCD or CMOS. The camera 310 is attached to any location on the vehicle M. For example, the camera 310 periodically and repeatedly captures images of the surroundings of the vehicle M. The camera 310 may be a stereo camera. Furthermore, the camera 310 and the camera 10 may be the same camera.

[0054] The radar device 320 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by an object (reflected waves) to detect at least the position (distance and direction) of the object. The radar device 320 is attached to any location on the vehicle M. The radar device 320 may detect the position and speed of an object by an FM-CW (Frequency Modulated Continuous Wave) method.

[0055] [Second control device] The second control device 200 includes, for example, a second recognition unit 210, a second control unit 220, and a second vehicle control unit 230. The second recognition unit 210, the second control unit 220, and the second vehicle control unit 230 are realized, for example, by a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI, ASIC, FPGA, GPU, or SOC, or may be realized by a combination of software and hardware. The program may be stored in advance in a device (a storage device having a non-transitory storage medium) such as an HDD or flash memory of the second control device 200, or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the second control device 200 by inserting the storage medium (non-transitory storage medium) into a drive device.

[0056] The second recognition unit 210 performs sensor fusion processing on the detection results from some or all of the camera 310 and the radar device 320 to recognize the surrounding situation of the vehicle M. For example, the second recognition unit 210 recognizes the position, type, speed, etc. of objects present around the vehicle M (within a predetermined distance) from the result of the sensor fusion processing. The second recognition unit 210 may have the same function as the first recognition unit 120, for example. The second recognition unit 210 may use the detection results of the camera 10 or the LIDAR unit 20 in the sensor fusion processing. The second recognition unit 210 may be omitted, and the processing results of the first recognition unit 120 described above may be used.

[0057] The second control unit 220 executes control to assist the occupant (driver) in driving based on the recognition result by the second recognition unit 210. The second control unit 220 generates a target trajectory along which the vehicle M will travel in the future based on the running state of the vehicle M (position and speed of the vehicle M) and surrounding conditions (road conditions, positions of surrounding objects, etc.). The second control unit 220 may execute driving control of the vehicle M, similar to the first control unit 140.

[0058] The second vehicle control unit 230, for example, acquires information on the target trajectory (trajectory points) generated by the second control unit 220 and stores it in a memory (not shown). The second vehicle control unit 230 controls the driving force output device 400 and the brake device 410 based on the speed element associated with the target trajectory stored in the memory. The second vehicle control unit 230 controls the steering device 420 according to the curvature of the target trajectory stored in the memory. The processing of the second vehicle control unit 230 is realized, for example, by a combination of feedforward control and feedback control. As an example, the second vehicle control unit 230 executes a combination of feedforward control according to the curvature of the road ahead of the vehicle M and feedback control based on the deviation from the target trajectory. The second vehicle control unit 230 may also perform driving control of the vehicle M based on the target trajectory generated by the first control unit 140.

[0059] [Switching control unit 300] The switching control unit 300 switches between the control (first control) executed by the first control unit 100 and the control (second control) executed by the second control unit 200, and controls the start and end of each control. In the embodiment, the first control and the second control differ in the control mode for the vehicle M. The control mode is, for example, the method or behavior of the driving control, and more specifically, the conditions, processing performance, type, etc. for executing the control. For example, in the first control, a predetermined operation in the driving control based on the surrounding situation is executed by the operation unit 46 without receiving an instruction or operation indicating permission from the occupant (driver) (regardless of whether or not an operation is performed), and in the second control, a decision is made as to whether or not to execute a predetermined operation in the driving control based on the content of the operation from the occupant, and control is executed according to the decision.

[0060] Furthermore, in terms of the above-mentioned processing performance, the processing performance of the first control device 100 is higher than that of the second control device 200. In this case, for example, the first control device 100 can perform driving control in other road conditions in addition to the road conditions (driving scenes) in which the second control device 200 can perform driving control.

[0061] Furthermore, in the above-described types, when the second control has, for example, an ADAS (Advanced Driver Assistance System) function, the first control may have, in addition to the ADAS function, a junction passing function, a merging function, etc. ADAS includes, for example, predetermined operations in driving control such as ACC (Adaptive Cruise Control), LKAS (Lane Keeping Assistance System), FCW (Forward Collision Warning), CMBS (Collision Mitigation Braking System), and ALCA (Auto Lane Changing Assist). Furthermore, in the lane change (ALCA) in the second control, a lane change is executed after receiving an operation instruction from the driver (e.g., an instruction on the destination of the lane change), whereas in the lane change (ALC; Auto Lane Changing) in the first control, a lane change is executed based on the surrounding conditions without receiving an operation instruction from the driver (regardless of whether or not an operation has been performed).

[0062] [About the driving scene] Here, road conditions (driving scenes) in which the above-described first control and second control are executed will be specifically described. FIG. 3 is a diagram for explaining driving scenes in which the first control and the second control can be executed. In the example of FIG. 3, a first driving scene SN1 and a second driving scene SN2 are shown. Also, in the example of FIG. 3, an example of driving control of a vehicle M1 traveling in the first driving scene SN1 and an example of driving control of a vehicle M2 traveling in the second driving scene SN2 are shown, including the driving trajectories of the vehicles M1 and M2. Also, in the example of FIG. 3, the road in the first driving scene SN1 is a general road (e.g., a main road of a general road) in an urban area, and the road in the second driving scene SN2 is a highway, but the road conditions and the like are not limited thereto. For example, the first driving scene SN1 may include a highway in the connecting portion between the first driving scene SN1 and the second driving scene SN2. Also, the roads other than the first driving scene SN1 and the second driving scene SN2 are, for example, narrow streets, but this is not particularly limited thereto. The switching control unit 300 distinguishes between an area where the first control is executable and an area where the second control is executable based on the road conditions on which the vehicle M is traveling, as shown in FIG.

[0063] The first driving scene SN1 is an area (scene) in which the first control can be executed. By executing the first control, the vehicle M1 can turn right or left at an intersection, stop or start at a traffic light, merge, drive at a junction (JCT), select a route, branch off, enter a T-junction, and drive in accordance with traffic rules based on various traffic participants and road signs on public roads. The first control can provide advanced technology and convenience through driving control (driving assistance) that responds to various situations the vehicle M1 encounters while driving to a destination. The first control can also perform driving control when the destination is not clearly determined. When the destination is not clearly determined, optimal and rational driving control is executed for the situation encountered based on the surrounding conditions, for example. The first control can also continuously evolve functions by successively updating software for executing the first control installed in the vehicle M via communication means such as OTA (Over The Air), thereby providing added value that allows the occupants to experience the growth of the vehicle M.

[0064] Furthermore, the second driving scene SN2 is an area (scene) in which the second control is executable. By executing the second control, the vehicle M2 is controlled to perform hands-off driving (driving with the driver not touching (holding) the steering wheel 72) when driving on a highway, automatic lane change (ALCA) to an adjacent lane, lane change to a branching lane, etc. In the second control, the junction passing function and merging function executable in the first control cannot be executed, and the sensors (e.g., LIDAR) and algorithms used by the first control device 100 cannot be used.

[0065] FIG. 4 is a diagram illustrating a specific example of the application range of the first control and the second control. The example of FIG. 4 shows the application range of each of the first control and the second control for the "expressway driving load reduction function." In the example of FIG. 4, the first control can execute a lane keeping function (LKAS), a lane change function (ALC), a diverging function (changing lanes to a branching lane), a junction (JCT) passing function, a merging function, and the like when driving on an expressway. All of these functions reduce the driver's driving load and thereby reduce stress. In this control, the first control device 100 generates a target trajectory TT along which the vehicle M will travel in the future, and executes driving control so that the vehicle M travels along the generated target trajectory TT. By executing each of the above-described functions in this way, for example, the lane keeping function allows the occupant to take their hands off the steering wheel 72 (handle), and the other functions allow the occupant to change lanes, diverge, pass through a JCT, and merge without anxiety. 4, the second control can execute the lane keeping function, lane change function, and diverging function, but does not execute the junction passing function or merging function. Furthermore, for each function in the second control, the occupant is notified of, for example, inquiry information about whether or not to execute the function, and the occupant then decides whether or not to execute the function by operating the operation unit 46, etc.

[0066] FIG. 5 is a diagram illustrating a specific example of the application range of the first control. In addition to the above-described "expressway driving load reduction function," the first control also includes a "general road driving load reduction function." As shown in FIG. 5, the first control performs basic road-following behaviors, such as speed setting according to speed limits indicated by road signs, speed adjustment and lane keeping according to curves (curvature), following the road at intersections, and lane keeping control in lanes with blurred lane markings. The first control also performs controls related to hands-off driving and lane change assistance (e.g., lane changes using driver triggers) during traffic jams on general roads, and behavioral responses specific to general roads (e.g., responses to intersections, pedestrians and bicycles, and vehicles parked on the roadside). These controls are realized, for example, by performing more detailed surrounding recognition results, including the recognition results (recognition results of the LIDAR unit 20) by the first recognition unit 120 of the first control device 100. The first control device 100 generates a target trajectory TT as needed and causes the vehicle M to travel along the generated target trajectory TT.

[0067] In addition, the first control can also perform control in the second driving scene SN2 as shown in FIG. 3. In this case, the first control executes controls such as hands-off driving, automatic lane change (ALC in the first control), and lane change to a branching exit lane. Furthermore, in the first control, if a destination is not entered into the navigation device 80, road-following driving is performed on expressways in all driving except when exiting and passing through junctions (JCTs). This allows the execution of driving assistance functions such as ACC and LKAS that assist road-following driving without linking with the navigation system. In this way, the first control can execute driving control with reduced operational intervention frequency even in driving situations that eliminate the need to enter a destination, which is expected in everyday driving. Furthermore, in the second control, only predetermined driving controls (e.g., ACC and LKAS) may be executed in the first driving scene SN1, such as on an ordinary road.

[0068] Here, based on the position of vehicle M, the surrounding driving scene, etc., when vehicle M is driving in the first driving scene SN1 or the second driving scene SN2, or is predicted to drive in the near future, switching control unit 300 notifies the occupant (driver, etc.) of inquiry information via HMI 40, and controls the start (or end) of first control or second control, or switching between first control and second control, by command input using a predetermined operator provided on operation unit 46. In addition to switching the control mode described above, switching control unit 300 controls so that the settings of various parameters selected by the driver in each control mode, such as speed adjustment, vehicle speed setting, and adjustment of the distance from the vehicle ahead, can be received from the same operator.

[0069] FIG. 6 is a diagram showing an example of the operation unit 46. In the example of FIG. 6, the operation unit 46 is provided on the right side of the steering wheel 72. In addition, in the example of FIG. 6, three switches (a first switch SW1, a second switch SW2, and a third switch SW3) are arranged vertically in the operation unit 46. The operation unit 46 in the embodiment only needs to have a plurality of operators, and the number, arrangement, and types of the switches SW are not limited to the example of FIG. 6. The switch is an example of an "operator," and a button, a lever, a touchpad, a track button, etc. may be used instead of the switch.

[0070] In the example of FIG. 6, the first switch (first operator) SW1 is a button switch, and the second switch (second operator) SW2 is a wheel switch with a wheel portion in the center that can rotate up and down. The wheel portion can be tilted left and right. The third switch (third operator) SW3 has a sensor that detects finger movement in the up and down direction, and further has a sensor on the right side that detects a confirmation instruction by finger contact. The switching control unit 300 accepts different operation contents for each control mode (e.g., first control, second control) to be executed, even if the operation is the same for each control.

[0071] Here, in the driving scenes (road conditions) shown in FIG. 3, it is assumed that the first driving scene SN1 satisfies the execution conditions for the first control, and the second driving scene SN2 satisfies the execution conditions for the second control. Based on the position of vehicle M and map information (first map information 84, second map information 92) obtained from the vehicle sensor 50, etc., the switching control unit 300 determines whether vehicle M is present in the first driving scene SN1 or the second driving scene SN2, or whether vehicle M will be present in the first driving scene SN1 or the second driving scene SN2 in the near future (within a predetermined time) based on the position, speed, traveling direction, etc. of vehicle M. Furthermore, the switching control unit 300 may determine whether vehicle M is present in the first driving scene SN1 or the second driving scene SN2 or whether vehicle M will be present in the near future based on the surrounding road conditions, road signs, etc., based on the recognition result of the first recognition unit 120 or the second recognition unit 210. Furthermore, the switching control unit 300 may perform the above-mentioned determination by combining the map information and the recognition result.

[0072] For example, when it is determined that the vehicle M is in the first driving scene SN1 (or will be in the first driving scene SN1 in the near future), the switching control unit 300 generates information (e.g., images and sounds) indicating the current and future vehicle states and notifies the occupant (driver) by having the HMI 40 output the generated information. The information indicating the vehicle state includes, for example, information indicating that the vehicle M is capable of executing the first control (or will be capable of executing the first control in the near future) and inquiry information as to whether or not to execute the first control. Furthermore, the information indicating the vehicle state may include information regarding the driving controls (operations) that can be executed in the first control, the benefits of executing the first control, and an explanation of the content that can be received when the first switch SW1 to the third switch SW3 are operated during execution of the second control.

[0073] Similarly, when it is determined that the vehicle M is in the second driving scene SN2 (or will be in the second driving scene SN2 in the near future), the switching control unit 300 generates information indicating the current and future vehicle states and notifies the occupant (driver) by having the HMI 40 output the generated information. In this case, the information indicating the vehicle state includes, for example, information indicating that the vehicle M is capable of executing the second control (or will be capable of executing the second control in the near future) and inquiry information regarding whether or not to execute the second control. Furthermore, the information indicating the vehicle state may include information regarding the driving control (operation) that can be executed in the second control, the benefits of executing the second control, and an explanation of the content that can be received when the first switch SW1 to the third switch SW3 are operated during execution of the second control.

[0074] If the first switch SW1 is operated (turned On) after notification of inquiry information about whether or not to execute the first control, the switching control unit 300 executes the first control at a timing when the first control becomes executable (for example, when the vehicle M enters the area of ​​the first traveling scene SN1). If the second control is being executed, the switching control unit 300 switches from the second control to the first control. If the first switch SW1 is operated after notification of inquiry information about whether or not to execute the second control, the switching control unit 300 executes the second control at a timing when the second control becomes executable (for example, when the vehicle M enters the area of ​​the second traveling scene SN2). If the first control is being executed, the switching control unit 300 switches from the first control to the second control.

[0075] Furthermore, if the first switch SW1 is operated while the first control or the second control is being executed, the switching control unit 300 ends the currently executed control. Furthermore, if a preset execution condition for ACC or LKAS, etc. is satisfied while the second control is being executed, the switching control unit 300 generates information indicating that ACC or LKAS is executable and inquiry information as to whether to execute them, and outputs this to the HMI 40. If the first switch SW1 is then operated, the ACC or LKAS is executed. Furthermore, if the first switch SW1 is operated while the ACC or LKAS operation control is being executed, the switching control unit 300 ends the currently executed operation control.

[0076] Furthermore, when the wheel portion of the second switch SW2 is scrolled up or down while the first control is being executed, the switching control unit 300 accepts an adjustment of the set speed of the vehicle M when a predetermined operation of the first control (for example, ACC) is being executed, or an adjustment of the distance between the vehicle and the vehicle in front. Furthermore, the switching control unit 300 accepts an operation (lane change instruction) as to whether or not to execute a lane change (for example, ALC) when a predetermined condition (for example, a predetermined automation level) is met, by operating the wheel portion while the first control is being executed.

[0077] Furthermore, during execution of the first control, the switching control unit 300 receives, via the third switch SW3, instructions regarding ALC (for example, an instruction to agree to inquiry information regarding ALC), adjustment of parameters for other functions, and customization of other setting information by the driver. Furthermore, during execution of the second control, the third switch SW3 receives operations on the information (interaction information) displayed on the display unit 42. For example, pressing the up / down switch switches the display content displayed on the display unit 42, such as scrolling or increasing or decreasing a numerical value, and pressing the enter switch receives instructions regarding the information selected on the screen.

[0078] In this manner, in the embodiment, the same operator can be used to receive different instructions depending on the driving assistance mode being executed, which reduces costs associated with an increase in the number of operators and also reduces the workload of the user, as the user only needs to know the positions and functions of a small number of operators.

[0079] For example, when switching of the control mode is possible based on the surrounding circumstances (when the execution conditions are satisfied) and an instruction is received from the operation unit 46, the switching control unit 300 executes switching from the first control to the second control or from the second control to the first control. The switching control unit 300 notifies the driver by outputting inquiry information regarding whether to execute the first control or the second control from the display unit 42 or the speaker 44, and then determines whether to execute switching based on the information instructed by the first switch SW1. For example, when an instruction to switch is received, the switching control unit 300 executes switching from the first control to the second control or from the second control to the first control, but does not execute switching to the first control or the second control if an instruction is not received from the first switch SW1. In this way, when the control of the vehicle M is to be switched, the occupant is asked whether or not to execute the control and is prompted to perform the instruction operation, thereby enabling the user to correctly understand the change in control. In addition, the switching control unit 300 also notifies the inquiry information when executing the first control or the second control from a state in which no driving control is being executed (fully manual driving state), and determines whether or not to execute the control based on the subsequent operation content.

[0080] Furthermore, the switching control unit 300 may be configured not to switch to the first control if, while the second control is being executed, query information as to whether to switch to the first control is notified and the operation unit 46 has not been operated for a predetermined time or more since the notification. In this case, even if the vehicle M has entered the first traveling scene SN1, control is executed to continue the second control as long as possible. Conversely, while the first control is being executed, query information as to whether to switch to the second control is notified and the operation unit 46 has not been operated for a predetermined time or more since the notification. In this case, control is executed to continue the first control as long as possible, even if the vehicle M has entered the second traveling scene SN2.

[0081] Furthermore, when the second control is being executed, the switching control unit 300 may determine whether or not to execute a predetermined operation based on the content received by the operation unit 46, and when the first control is being executed, the switching control unit 300 may execute the predetermined operation regardless of whether or not an operation is performed by the operation unit 46 (regardless of the content received by the operation unit 46). Note that the predetermined operation is, for example, a lane change of the vehicle M, but is not limited to this and may be other driving control (for example, ACC or LKAS), etc.

[0082] Furthermore, when the operating state of the vehicle M satisfies a predetermined condition while the first control or the second control is being executed, the switching control unit 300 may execute a third control that is different from the first control and the second control. The predetermined condition here is, for example, when there is a traffic jam around the vehicle M (particularly ahead) and the vehicle is traveling (or stopped) at a low speed that is equal to or lower than a predetermined speed. In this case, the third control becomes, for example, a TJP control that maintains a safe distance from the vehicle ahead while adapting to changes in the vehicle speed when traveling at a low speed such as in a traffic jam. Note that the predetermined condition and the third control are not limited to this.

[0083] Furthermore, an instruction from the driver as to whether to switch to the third control (switching instruction) may be received by operating an operator (for example, the second switch SW2, the third switch SW3, or another operator) different from an operation (for example, an operation of the first switch SW1) different from an operation for switching from the first control to the second control and from the second control to the first control. This allows the driver to more accurately understand the distinction between the first control and the second control for control under specific conditions (third control) by changing the content of the instruction operation.

[0084] Furthermore, when the conditions for executing the third control are no longer satisfied (for example, when traffic congestion is resolved) during the execution of the third control, the switching control unit 300 switches to a control mode different from the first control or the second control. The different control mode may be, for example, a mode in which the driving control is terminated and the driving mode is switched to manual driving, or a control in which the driving mode (automation level) is lowered. This makes it possible to issue a new instruction to execute the first control or the second control when the conditions for the third control are no longer satisfied, allowing the occupant to more clearly understand the transition of control.

[0085] In the embodiment, in the first control, the first control device 100 generates a target trajectory of the vehicle M, and in the second control, the second control device 200 generates a target trajectory of the vehicle M. In addition, in the embodiment, based on the target trajectory generated by the first control device 100 or the second control device 200, one control device (for example, the second control device 200) controls the traveling driving force output device 400, the braking device 410, and the steering device 420 to perform driving control that controls at least one of the speed or velocity of the vehicle M. This makes it possible to unify the processing related to the driving control, thereby making it possible to further stabilize the behavior of the vehicle M.

[0086] [flowchart] Next, a description will be given of the flow of processing executed by the vehicle system 1. The following processing will mainly focus on the processing executed by the vehicle system 1, which switches the control mode depending on the operation status of the operation unit 46. It is also assumed that the first control mode by the first control device 100 or the second control mode by the second control device 200 is already being executed when the processing starts.

[0087] FIG. 7 is a flowchart showing an example of the flow of processing executed by the vehicle system 1 of the embodiment. In the example of FIG. 7, the switching control unit 300 recognizes the surrounding conditions of the vehicle M based on the recognition result by the recognition unit (the first recognition unit 120 or the second recognition unit 210) included in the control unit (the first control unit 100 or the second control unit 200) corresponding to the control currently being executed (step S100). Note that in the processing of step S100, when the first control is being executed, the surrounding conditions may be recognized based on the recognition results of both the first recognition unit 120 and the second recognition unit 210. Next, the switching control unit 300 determines whether to switch the currently executed control mode based on the surrounding conditions of the vehicle M (step S110). If it is determined that the driving assistance mode should be switched, the switching control unit 300 generates inquiry information and notifies the occupant (step S120).

[0088] Next, the switching control unit 300 determines whether an instruction from the occupant using a predetermined operator included in the operation unit 46 has been received in response to the inquiry information (step S130). If it is determined that an instruction has been received, the switching control unit 300 switches the control mode (step S140). Next, the control unit (first control unit 100 or second control unit 200) that has become the execution target after the switching receives an instruction from a predetermined operator from the operation unit corresponding to the control being executed (step S150), and executes driving control corresponding to the received instruction (step S160). This ends the processing of this flowchart. Also, if it is determined in the processing of step S110 that the control mode will not be switched, or if it is determined in the processing of step S130 that an instruction from the occupant using the operator has not been received, the processing ends without switching the control.

[0089] [Variations] In the above-described embodiment, a portion of the first driving scene SN1 shown in FIG. 3 may include an area (section) in which the second control is executable, and a portion of the second driving scene SN2 may include an area (section) in which the first control is executable. In this case, switching control may be performed in response to an inquiry corresponding to the switching section or an operation of the operation unit. In addition, in the embodiment, information regarding the section in which the first control is executable and the section in which the second control is executable may be included in map information, and the above-described control mode switching control may be performed in response to the section acquired from the map information. In addition, the function of the switching control unit 300 in the embodiment may be provided in either the first control device 100 or the second control device 200.

[0090] According to the embodiment described above, the vehicle control device includes a recognition unit (first recognition unit 120, second recognition unit 210) that recognizes the surrounding conditions of vehicle M, a driving control unit (first control unit 140, first vehicle control unit 160, second control unit 220, second vehicle control unit 230, switching control unit 300) that performs driving control to control at least one of the steering and speed of vehicle M based on the surrounding conditions, and an operation unit 46 that accepts operations from an occupant of vehicle M. The driving control unit performs driving control based on any of a plurality of control modes including a first control and a second control that has a different driving assistance content from the first control, and when it is possible to switch the control mode based on the surrounding conditions and an instruction is received from the operation unit 46, it switches from the first control to the second control or from the second control to the first control, and when it is not receiving an instruction from the operation unit 46, it does not perform the switching, thereby enabling more appropriate driving control to be performed according to the surrounding conditions of the vehicle. Specifically, according to the embodiment, when the control mode is changed, the occupant is allowed to perform an operation, so that the occupant can correctly understand the change in the control mode.

[0091] The above-described embodiment can be expressed as follows. a storage medium for storing computer-readable instructions; a processor connected to the storage medium; The processor executes the computer-readable instructions to: Recognizes the vehicle's surroundings, Execute driving control to control at least one of steering and speed of the vehicle based on the surrounding conditions; The driving control is executed based on any one of a plurality of control modes including a first control and a second control having a different driving assistance content from the first control; When the control mode can be switched based on the surrounding conditions, When an instruction is received from an operation unit that receives an operation from an occupant of the vehicle, switching from the first control to the second control or switching from the second control to the first control is performed; When an instruction from the operation unit is not received, the switching is not performed. Vehicle control device.

[0092] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0093] 1...vehicle system, 10, 310...camera, 20...LIDAR unit, 30...communication device, 40...HMI, 50...vehicle sensor, 60...driver monitor camera, 70...driving operator, 78...power supply unit, 80...navigation device, 90...MPU, 100...first control device, 120...first recognition unit, 140...first control unit, 142...action plan generation unit, 144...mode determination unit, 146...driver state determination unit, 148...mode change processing unit, 160...first vehicle control unit, 200...second control device, 210...second recognition unit, 220...second control unit, 230...second vehicle control unit, 300...switching control unit, 400...traveling drive force output device, 410...brake device, 420...steering device

Claims

1. a recognition unit that recognizes the surrounding situation of the vehicle; a driving control unit that executes driving control to control at least one of steering and speed of the vehicle based on the surrounding situation; an operation unit that accepts an operation from an occupant of the vehicle, The operation control unit The driving control is executed based on any one of a plurality of control modes including a first control and a second control in which the driving assistance content is different from that of the first control; When the control mode can be switched based on the surrounding conditions, When an instruction from the operation unit is received, switching from the first control to the second control or switching from the second control to the first control is executed; When an instruction from the operation unit is not received, the switching is not performed. Vehicle control device.

2. The operation control unit When the second control is being executed, determining whether or not to execute a predetermined operation based on the content received by the operation unit; When the first control is being executed, the predetermined operation is executed regardless of whether or not the operation unit is operated. The vehicle control device according to claim 1 .

3. The predetermined operation includes changing lanes of the vehicle. The vehicle control device according to claim 2.

4. The operation control unit When a state of the vehicle satisfies a predetermined condition during execution of the first control or the second control, switching to a third control different from the first control and the second control; The instruction to switch to the third control is performed by an operation different from the operation to switch from the first control to the second control and the operation to switch from the second control to the first control. The vehicle control device according to claim 1 .

5. the operation control unit switches to a control different from the first control or the second control when an execution condition of the third control is no longer satisfied during execution of the third control. The vehicle control device according to claim 4.

6. the operation control unit includes a first control device that executes the first control and a second control device that executes the second control, In the first control, a target trajectory of the vehicle is generated by the first control device; In the second control, a target trajectory of the vehicle is generated by the second control device. The vehicle control device according to claim 1 .

7. the operation unit includes a plurality of operators including a first operator, a second operator, and a third operator; the first operator accepts an instruction to execute or switch between the first control and the second control, and also accepts an instruction to start or end execution of a predetermined action when the second control is being executed; the second operator accepts a set speed of the vehicle in the predetermined operation and also accepts a lane change instruction; the third operator operates the information displayed on the display unit when the second control is being executed, and receives an instruction regarding a lane change when the first control is being executed. The vehicle control device according to claim 1 .

8. a notification unit that notifies the driver of the current and future vehicle states when the execution condition for the control mode is satisfied; The vehicle control device according to claim 7.

9. the driving control unit does not switch to the first control if a state in which the operation unit is not operated continues for a predetermined time or more after making an inquiry to the notification unit regarding switching to the first control during execution of the second control; The vehicle control device according to claim 8.

10. The driving control unit distinguishes between an area in which the first control can be performed and an area in which the second control can be performed based on road conditions of the vehicle. The vehicle control device according to claim 1 .

11. The computer Recognizes the vehicle's surroundings, Execute driving control to control at least one of steering and speed of the vehicle based on the surrounding conditions; The driving control is executed based on any one of a plurality of control modes including a first control and a second control in which the driving assistance content is different from that of the first control; When the control mode can be switched based on the surrounding conditions, When an instruction is received from an operation unit that receives an operation from an occupant of the vehicle, switching from the first control to the second control or switching from the second control to the first control is performed; When an instruction from the operation unit is not received, the switching is not performed. Vehicle control method.

12. On the computer, Recognize the vehicle's surroundings, Execute driving control to control at least one of steering and speed of the vehicle based on the surrounding conditions; Executing the driving control based on any one of a plurality of control modes including a first control and a second control having a different driving assistance content from the first control; When the control mode can be switched based on the surrounding conditions, When an instruction is received from an operation unit that receives an operation from an occupant of the vehicle, switching from the first control to the second control or switching from the second control to the first control is executed; When an instruction from the operation unit is not received, the switching is not executed. program.

Images