Vehicle control device and vehicle control method

Abstract

A vehicle control device 1 of the present invention addresses the problem of obtaining a vehicle control device that enables an occupant to easily grasp a surrounding situation and vehicle control in autonomous driving when a confidence level of prediction is low in the autonomous driving. This vehicle control device 1 comprises a processor 201 and a memory 202, and is characterized in that the processor predicts an action of a moving body on the basis of external environment information acquired by an external environment sensor that recognizes an external environment of an host vehicle, generates an action plan of the host vehicle for an action prediction result of the moving body, determines to output a notification including at least one of the action prediction result and the action plan when a prediction confidence level of the action prediction result falls below a threshold, and controls a notification device that outputs the notification.

Description

Vehicle Control Device and Vehicle Control Method

[0001] The present invention relates to a vehicle control device and a vehicle control method.

[0002] In automatic driving of a vehicle, for example, the driving behavior of a surrounding moving object is predicted based on external information, a behavior plan of the host vehicle for the prediction result is generated, and vehicle control of the host vehicle is performed according to the behavior plan.

[0003] Patent Document 1 describes a technology that "even when the reliability of driving behavior prediction is low, if a certain risk occurs due to the estimated driving behavior and the risk level is high, such as when sudden braking of the host vehicle is required, the evaluation value will be equal to or higher than the threshold, and it is determined that the predicted driving behavior of surrounding vehicles should be notified to the occupants of the host vehicle".

[0004] Japanese Patent Laid-Open No. 2017-204071

[0005] For example, when the reliability of the prediction of the driving behavior of a moving object is low, the behavior intention of the host vehicle in automatic driving may differ from the intention of the occupant. In such a case, if the movement actually performed by the host vehicle based on the behavior intention of the host vehicle differs from the movement intended by the occupant, this will give the occupant a sense of incongruity, which may impair the convenience of automatic driving.

[0006] The present invention has been made in view of the above points, and an object of the present invention is to provide a vehicle control device that allows an occupant to easily grasp the surrounding situation and vehicle control in automatic driving when the prediction confidence is low in automatic driving.

[0007] The vehicle control device of the present invention that solves the above problem is a vehicle control device comprising a processor and a memory, wherein the processor predicts the behavior of a moving object based on external information acquired by an external sensor that recognizes the external environment of the host vehicle, generates a behavior plan of the host vehicle for the behavior prediction result of the moving object, and when the prediction confidence of the behavior prediction result is below a threshold, determines to output a notification including at least one of the behavior prediction result or the behavior plan, and controls a notification device that outputs the notification.

[0008] According to the present invention, appropriate notifications can be made to the occupants regarding the surrounding conditions and the content of vehicle control during autonomous driving. Further features related to the present invention will become apparent from the description herein and the accompanying drawings. In addition, problems, configurations, and effects other than those described above will become apparent from the following description of embodiments.

[0009] A diagram showing the configuration of the driving drive system and sensors of an autonomous vehicle equipped with a vehicle control device according to the first embodiment of the present invention. Hardware configuration diagram of the vehicle control device according to the first embodiment of the present invention. Functional block diagram of the vehicle control device according to the first embodiment of the present invention. Functional block diagram of the autonomous driving planning unit of the vehicle control device according to the first embodiment of the present invention. Functional block diagram of the occupant notification management unit of the vehicle control device according to the first embodiment of the present invention. Control flow diagram of the notification output determination unit of the vehicle control device according to the first embodiment of the present invention. Control flow diagram of the notification content generation unit of the vehicle control device according to the first embodiment of the present invention. Functional block diagram of the track planning unit of the vehicle control device according to the first embodiment of the present invention. A diagram showing an example of a notification device of the vehicle control device according to the first embodiment of the present invention. A diagram showing an example of a notification from the notification device of the vehicle control device according to the first embodiment of the present invention. A diagram illustrating an example of the behavior prediction result of another vehicle by the vehicle control device according to the first embodiment of the present invention. A diagram illustrating an example of the behavior prediction result of another vehicle by the vehicle control device according to the first embodiment of the present invention. Functional block diagram of the occupant notification management unit of the vehicle control device according to the second embodiment of the present invention. Control flow diagram of the notification content generation unit of the vehicle control device according to the second embodiment of the present invention. Functional block diagram of the track planning unit of the vehicle control device according to the second embodiment of the present invention. A diagram showing an example of a notification from the notification device of the vehicle control device according to the second embodiment of the present invention. A functional block diagram of the occupant notification management unit of the vehicle control device according to the third embodiment of the present invention. A control flow diagram of the notification content generation unit of the vehicle control device according to the third embodiment of the present invention. A diagram showing an example of a notification from the notification device of the vehicle control device according to the third embodiment of the present invention. A functional block diagram of the occupant notification management unit of the vehicle control device according to the fourth embodiment of the present invention. A diagram showing an example of the behavior prediction results of other vehicles and the trajectory plan of the own vehicle by the vehicle control device according to the fourth embodiment of the present invention. A graph showing the confirmation method selected according to the degree of behavioral impact and the degree of confidence.

[0010] Embodiments of the present invention will be described in detail below with reference to the drawings.

[0011] [First Embodiment] Figure 1 is an explanatory diagram showing the overall configuration of an autonomous driving vehicle (hereinafter sometimes simply referred to as the vehicle or the self-vehicle) 1001 equipped with a vehicle control device 1 according to the first embodiment of the present invention. In the figure, FL wheel refers to the left front wheel, FR wheel refers to the right front wheel, RL wheel refers to the left rear wheel, and RR wheel refers to the right rear wheel.

[0012] Vehicle 1001 is equipped with a vehicle control device 1 that calculates command values ​​for a steering control mechanism 10, a brake control mechanism 13, and a throttle control mechanism 20, which act as actuators that control the direction of travel of vehicle 1001, based on information (external information) from external sensors (hereinafter sometimes simply referred to as sensors) 2, 3, 4, and 5 that recognize the outside world. Furthermore, vehicle 1001 is equipped with a steering control device 8 that controls the steering control mechanism 10 based on command values ​​from the vehicle control device 1, a braking control device 15 that controls the brake control mechanism 13 based on the command values ​​to adjust the brake force distribution to each wheel (FL wheel, FR wheel, RL wheel, RR wheel), an acceleration control device 19 that controls the throttle control mechanism 20 based on the command values ​​to adjust the torque output of the engine (not shown), and a notification device (notification device) 24 that notifies the vehicle 1001's travel plan and predictions of the behavior of moving objects in the surrounding area. In this example, an engine is used as the drive source, but of course, the present invention can also be applied to electric vehicles that use a motor as the drive source.

[0013] Vehicle 1001 is equipped with external sensors 2, 3, 4, and 5, including a camera 2 at the front, laser radars 3 and 4 on the left and right sides, and a millimeter-wave radar 5 at the rear, which can detect the relative distance and relative speed between vehicle 1001 and surrounding vehicles. Vehicle 1001 is also equipped with a communication device 23 for communication between the vehicle and the road or between vehicles. In this embodiment, the above combination of external sensors is shown as an example of a sensor configuration, but it is not limited to this, and combinations with ultrasonic sensors, stereo cameras, infrared cameras, etc., may also be used. The above sensor signals (sensor output signals) are input to the vehicle control device 1.

[0014] Figure 2 is a hardware configuration diagram of a vehicle control device according to the first embodiment of the present invention. As shown in Figure 2, the vehicle control device 1 includes, for example, a CPU 201 which is a processor, a memory 202 such as ROM or RAM, an input / output unit 203, and an I / F device 204. The ROM of the memory 202 stores the flow of vehicle driving control, which will be described below. As will be described in detail later, the vehicle control device 1 calculates command values ​​for each actuator 10, 13, and 20 to control vehicle driving according to the generated driving plan. The control devices 8, 15, and 19 for each actuator 10, 13, and 20 receive the command values ​​from the vehicle control device 1 via communication and control each actuator 10, 13, and 20 based on the command values.

[0015] Next, returning to the explanation of Figure 1, we will describe the operation of the brakes. When the driver is operating the vehicle 1001, the force applied to the brake pedal 12 by the driver is amplified by a brake booster (not shown), and a master cylinder (not shown) generates hydraulic pressure corresponding to that force. The generated hydraulic pressure is supplied to the wheel cylinders 16FL to 16RR via the brake control mechanism 13. The wheel cylinders 16FL to 16RR consist of cylinders, pistons, pads, etc. (not shown), and the piston is propelled by the working fluid supplied from the master cylinder, and the pad connected to the piston is pressed against the disc rotor. The disc rotor rotates together with the wheel. Therefore, the brake torque acting on the disc rotor becomes the braking force acting between the wheel and the road surface. As a result, braking force can be generated on each wheel in response to the driver's brake pedal operation.

[0016] Although not shown in detail in Figure 1, the braking control device 15, like the vehicle control device 1, includes, for example, a CPU, ROM, RAM, and an input / output unit. The braking control device 15 receives input from a combine sensor 14 capable of detecting longitudinal acceleration, lateral acceleration, and yaw rate, wheel speed sensors 11FL to 11RR installed on each wheel, the brake force command value from the vehicle control device 1 mentioned above, and sensor signals from the steering angle detection device 21 via the steering control device 8 described later. The output of the braking control device 15 is connected to a brake control mechanism 13 having a pump and control valve (not shown), and can generate arbitrary braking force on each wheel independently of the driver's brake pedal operation. Based on the above information, the braking control device 15 estimates vehicle 1001 spin, drift out, and wheel lock, and controls the brake control mechanism 13 etc. to generate braking force on the corresponding wheel to suppress these, thereby playing a role in improving the driver's steering stability. Furthermore, the vehicle control device 1 communicates a brake force command value to the brake control device 15, thereby generating an arbitrary brake force on the vehicle 1001. In autonomous driving, where no driver intervention is required, the vehicle control device 1 plays a role in automatically applying the brakes. However, the present invention is not limited to the brake control device 15 described above, and other actuators such as brake-by-wire may be used.

[0017] Next, the operation of the steering will be explained. When the driver is operating the vehicle 1001, the steering torque and steering angle input by the driver via the steering wheel 6 are detected by the steering torque detection device 7 and the steering angle detection device 21, respectively. Based on this information, the steering control device 8 controls the motor 9 to generate assist torque. Although not shown in detail in Figure 1, the steering control device 8 also has components similar to the vehicle control device 1, such as a CPU, ROM, RAM, and input / output unit. The combined force of the driver's steering torque and the assist torque from the motor 9 causes the steering control mechanism 10 to move, turning the front wheels. Meanwhile, in accordance with the turning angle of the front wheels, the reaction force from the road surface is transmitted to the steering control mechanism 10 and transmitted to the driver as a road reaction force.

[0018] The steering control device 8 can generate torque using the motor 9 and control the steering control mechanism 10 independently of the driver's steering input. Therefore, the vehicle control device 1 can control the front wheels to any desired steering angle by communicating steering force command values ​​to the steering control device 8, and plays a role in automatically steering in autonomous driving where no driver input is required. However, the present invention is not limited to the steering control device 8 described above, and other actuators such as steer-by-wire may be used.

[0019] Next, let's explain the accelerator. The amount the driver depresses the accelerator pedal 17 is detected by the stroke sensor 18 and input to the acceleration control device 19. Although not shown in detail in Figure 1, the acceleration control device 19, like the vehicle control device 1, includes, for example, a CPU, ROM, RAM, and an input / output unit. The acceleration control device 19 adjusts the throttle opening according to the amount the accelerator pedal is depressed and controls the engine (torque output). As a result, the vehicle 1001 can be accelerated in response to the driver's accelerator pedal operation. Furthermore, the acceleration control device 19 can control the throttle opening independently of the driver's accelerator pedal operation. Therefore, the vehicle control device 1 can generate any acceleration in the vehicle 1001 by communicating an acceleration command value to the acceleration control device 19, and plays the role of automatically accelerating in autonomous driving where no driver operation is required.

[0020] Next, the configuration of the vehicle control device 1 of this embodiment will be explained using the block diagram shown in Figure 3. Figure 3 is a functional block diagram of the vehicle control device according to the first embodiment of the present invention.

[0021] The vehicle control device 1 basically includes an automatic driving planning unit 301 that plans the movements of the vehicle 1001 to automatically drive the vehicle 1001 to a destination (as described later), a trajectory planning unit 302 that plans the trajectory of the vehicle 1001 to automatically park the vehicle 1001 in a parking space in a parking lot, etc., and the trajectory of the vehicle 1001 to travel on public roads and highways, a vehicle motion control unit 303 that generates command values ​​to control the vehicle motion of the vehicle 1001, an actuator control unit 304 that controls the actuators 10, 13, and 20 such as the steering, brake, and engine (via control devices 8, 15, and 19), and a notification device control unit 305 that controls a notification device 24 having an HMI (Human Machine Interface) such as a visual device 902 and an audio device 901 mounted on the vehicle, and these are implemented on different controllers (CPUs). Therefore, a vehicle network 306 such as a CAN is required for communication between each controller. However, the vehicle network 306 may also be connected wirelessly in addition to via a wired connection. Furthermore, the actuator control unit 304 may be implemented on different hardware such as an engine control controller or a brake control controller.

[0022] Next, the configuration and operation of the automatic driving planning unit 301 included in the vehicle control device 1 of this embodiment will be explained using the block diagram shown in Figure 4. Figure 4 is a functional block diagram of the automatic driving planning unit of the vehicle control device according to the first embodiment of the present invention.

[0023] The automated driving planning unit 301 acquires road lane information 411 necessary for automated driving, map information 412 of the area around the vehicle, environmental information 413 of the area around the vehicle, route information 414 from the current location to the target location, and user input information 415 such as an action plan entered by the user, and generates occupant notification management information 421 and action candidates 422. The occupant notification management information 421 is information for notifying occupants of the actions of the vehicle 1001 under automated driving, and the action candidates 422 have information on at least one action candidate in which the vehicle 1001 will act under automated driving.

[0024] The automated driving planning unit 301 has, as internal functions, an action prediction unit 401, an action planning unit 402, and an occupant notification management unit 403. The action prediction unit 401 predicts the actions of other vehicles, pedestrians, and other moving objects based on external information acquired by external sensors that recognize the outside world of the vehicle 1001. In this embodiment, moving objects include parked vehicles and the like that may move in the future, even if their speed is zero at the present time.

[0025] The behavior prediction unit 401 calculates the future position and velocity information (behavior prediction information) of the moving object based on the input information 411 to 415. Along with predicting the behavior of the moving object, the behavior prediction unit 401 also calculates the prediction confidence level (%) of that behavior prediction. If multiple behaviors can be predicted for the behavior of the moving object, the prediction confidence level is calculated for each of them.

[0026] The action planning unit 402 generates an action plan for the vehicle 1001 based on the action prediction results of the moving object by the action prediction unit 401. Based on the moving object's action prediction results and map information, the action planning unit 402 calculates an action plan that is suitable for the vehicle 1001 to avoid collisions with other vehicles and objects in its vicinity, and is appropriate for the route information 414 and the current vehicle state (speed, position, direction, etc.), and outputs it as an action candidate 422 for the vehicle 1001. When the action planning unit 402 receives manual operation via the HMI of the notification device 24 or natural language voice input, it generates an action plan for the vehicle 1001 based on that input.

[0027] The occupant notification management unit 403 generates occupant notification management information 421 to be notified to the occupants of its own vehicle 1001 and outputs it to the notification device control unit 305. Based on the predicted behavior result of the moving object and the confidence level of the predicted behavior result, the occupant notification management unit 403 sets whether or not to include at least one of the predicted behavior result or the action plan in the notification and the content of the notification. The occupant notification management unit 403 outputs the occupant notification management information 421 when the prediction confidence level of the predicted behavior result of the moving object falls below a preset threshold (first predetermined value, second predetermined value). The occupant notification management information 421 includes at least one of the predicted behavior result of another vehicle or the action candidate 422 of its own vehicle 1001.

[0028] Figure 5 is a functional block diagram of the occupant notification management unit of a vehicle control device according to the first embodiment of the present invention. The occupant notification management unit 403 sets whether or not to issue a notification that includes at least one of the behavior prediction result or the behavior plan, based on the behavior prediction result and the prediction confidence level of the moving object. The occupant notification management unit 403 generates occupant notification management information 421 based on the behavior prediction result 401a, the prediction confidence level 401b, and the behavior candidate 422. The occupant notification management unit 403 has a notification output determination unit 501 and a notification content generation unit 502 as internal functions.

[0029] The notification output determination unit 501 determines whether or not it is necessary to notify the occupants of the predicted behavior of the moving object, based on the prediction confidence level. If it determines that notification to the occupants is necessary, it selects either a screen notification or an audio notification, depending on the prediction confidence level.

[0030] The notification output determination unit 501 determines the notification device 24 to be used for notification according to the method of notifying the occupants. For example, if the notification is to be made with an image, the visual device 902 is selected as the notification device 24, and if the notification is to be made with sound, the audio device 901 is selected as the notification device 24. Alternatively, the visual device 902 may be used together with the audio device 901.

[0031] Figure 6 is a control flow diagram of the notification output determination unit of the vehicle control device according to the first embodiment of the present invention. The notification output determination unit 501 acquires various information, including the prediction confidence level, from the action prediction unit 401 and the action planning unit 402 (S601), and determines whether the prediction confidence level is less than a first predetermined value (S602). If the prediction confidence level is less than a first predetermined value (Yes in S602), it determines to notify the occupants (S603), and compares the prediction confidence level with a second predetermined value to determine the notification method (S604).

[0032] If the prediction confidence is less than the second predetermined value (Yes in S604), the system selects to provide notification using the voice device 901 (S605) to ensure that the notification is reliably conveyed to the crew. On the other hand, if the prediction confidence is equal to or greater than the second predetermined value, the system selects to provide notification using a visual device (S606) to ensure that the notification is conveyed only to crew members who are looking at the visual device. The first predetermined value is set to a value greater than the second predetermined value.

[0033] The notification content generation unit 502 generates specific content to be notified to the occupant from the notification device 24 selected by the notification output determination unit 501. For example, when notifying of the predicted movement of a moving object via an audio device, it generates specific content for the audio output to be output from the audio device 901. On the other hand, when notifying of the predicted movement of a moving object via a visual device, it generates specific content for the image output to be output from the visual device 902. The notification content generation unit 502 may also change the notification content based on the occupant's gaze, face orientation, and degree of concentration captured by the in-vehicle camera.

[0034] Figure 7 is a control flow diagram of the notification content generation unit of the vehicle control device according to the first embodiment of the present invention.

[0035] The notification content generation unit 502 acquires various information, including the prediction confidence level, from the action prediction unit 401 and the action planning unit 402 (S701), and determines whether the notification output destination, which is the result of the notification output determination unit 501, is an audio device or not (S702). If the notification output destination is an audio device, it generates audio output content (S703), and if the notification output destination is not an audio device, it generates output screen display content to be displayed on a visual device (S704). Then, it generates crew notification management information 421 to be conveyed to the crew using the audio output content or output screen display content (S705).

[0036] Figure 8 is a functional block diagram of the track planning unit of a vehicle control device according to the first embodiment of the present invention. The track planning unit 302 acquires lane information 411, map information 412, environmental information 413, route information 414, and action candidate information 422, and calculates the target track 821 of the vehicle 1001.

[0037] The track planning unit 302 includes a target track candidate generation unit 801 and a target track selection unit 802. The target track candidate generation unit 801 generates at least one target track candidate based on the vehicle's action candidates 422. The target track selection unit 802 selects one target track 821 from the target track candidates based on at least one of the crew's instructions and confidence level. The vehicle 1001 performs automatic driving based on the selected target track 821.

[0038] Figure 9 shows an example of a notification device for a vehicle control device according to the first embodiment of the present invention. Figure 9 is a schematic diagram showing the front of the passenger compartment. A steering wheel 912 for manual driving is mounted on a console panel 911 in front of the driver's seat 921, and the notification device 24 is located in the center of the console panel 911 next to the steering wheel 912. The notification device 24 has a display monitor 902, which is a visual device that displays images, and a speaker 901, which is an audio device that outputs sound. The display monitor 902 is configured as, for example, a touch panel, and the screen can be operated by the occupant.

[0039] Figure 10 shows an example of a notification from the notification device of a vehicle control device according to the first embodiment of the present invention. The image shown in Figure 10 is an example of an image displayed on the visual device 902 when the notification output determination unit 501 determines that the prediction confidence level is less than a first predetermined value and a notification should be sent by the notification device 24, and further determines that the prediction confidence level is equal to or greater than a second predetermined value and a notification should be sent by the visual device 902.

[0040] The visual device 902 of the vehicle 1001 displays an image of the current situation, showing the vehicle 1001 intending to enter the parking lot 1000, and another vehicle 1002 in the process of exiting a parking space 1011. The behavior prediction result 1021 of the other vehicle 1002, calculated by the behavior prediction unit 401, is superimposed on the image. The occupant of the vehicle 1001 can recognize the behavior prediction result 1021 of the other vehicle 1002, calculated by the vehicle control device 1, by looking at the visual device 902. Therefore, the occupant can infer the intention of the vehicle 1001's actions under autonomous driving, and the occupant's discomfort with the vehicle 1001's movements can be suppressed.

[0041] Figures 11 and 12 illustrate an example of the behavior prediction results of another vehicle by a vehicle control device according to the first embodiment of the present invention. The example shown in Figures 11 and 12 shows a situation in which the own vehicle 1001 is positioned at the entrance of the parking lot 1000 in order to park in the parking lot 1000, and another vehicle 1102 is moving within the parking lot 1000 in a direction that is approaching from the opposite direction.

[0042] In the example shown in Figure 11, the behavior prediction unit 401 of the automated driving planning unit 301 predicts two actions for the other vehicle 1102: a predicted action 1121 in which the vehicle parks in the parking space 1111, and a predicted action 1122 in which the vehicle continues straight to the exit area 1112. The unit then determines that the prediction confidence level for the parking action 1121 is 90%, and the prediction confidence level for the continuing straight action 1122 is 10%.

[0043] For example, when the first predetermined value is set to 95% and the second predetermined value is set to 80% in the notification output determination unit 501 of the crew notification management unit 403, the prediction confidence level of the highest prediction of behavior 1121, which is 90%, is lower than the first predetermined value. Therefore, the determination process S602 by the notification output determination unit 501 is Yes, and since the prediction confidence level is equal to or greater than the second predetermined value, the determination process S604 is No. Consequently, it is determined that a notification should be sent by the notification device 24, and the crew is notified of the predictions of behavior 1121 and 1122 of the other vehicle 1102 and their prediction confidence levels via the visual device 902.

[0044] In the example shown in FIG. 12, the behavior prediction unit 401 of the automatic driving planning unit 301 predicts three types of behaviors: a first behavior prediction 1221 that another vehicle 1202 parks in the parking space 1211, a second behavior prediction 1222 that the another vehicle 1202 travels straight to the vicinity of the exit 1212 as it is, and a third behavior prediction 1223 that the another vehicle 1202 parks in another parking space 1213. Further, the behavior prediction unit 401 determines that the prediction confidence of the behavior prediction 1221 of parking in the parking space 1211 is 35%, determines that the prediction confidence of the behavior prediction 1222 of traveling straight is 40%, and determines that the prediction confidence of the behavior prediction 1223 of parking in another parking space 1213 is 25%.

[0045] Therefore, among the prediction confidences of the plurality of behavior predictions, the prediction confidence of 40% of the behavior prediction 1222 having the highest prediction confidence is lower than both a first predetermined value and a second predetermined value, so that the determination result is Yes in the determination processing S602 performed by the notification output determination unit 501, and further the determination result is Yes in the determination processing S604. Accordingly, the audio device 901 notifies the occupant of the behavior predictions 1221, 1222, 1223 and the respective prediction confidences thereof. In this case, in addition to the notification by the audio device 901, notification by the visual device 902 may also be performed, which allows the occupant to confirm the information visually in addition to acoustically, thereby further improving the convenience of the device.

[0046] The vehicle control device according to the present embodiment predicts a behavior of a moving object based on external world information acquired by an external world sensor, generates a behavior plan of the own vehicle 1001 according to the behavior prediction result of the moving object, determines to output a notification including at least one of the behavior prediction result and the behavior plan when the prediction confidence of the behavior prediction result is lower than a threshold value, and controls a notification device 24 that outputs the notification. Therefore, when the confidence of the prediction of the driving behavior of the moving object is low, the occupant can confirm the behavior and intention of the own vehicle under automatic driving, the occupant can easily grasp the behavior intention of the own vehicle 1001 under automatic driving, and can communicate with the vehicle control device 1. Accordingly, inconsistency between the behavior intention of the own vehicle 1001 and the intention of the occupant can be eliminated, the occupant's discomfort with the behavior of the own vehicle 1001 can be suppressed, and the convenience of automatic driving can be ensured. Further, when the confidence of the prediction of the driving behavior of the moving object is high, notification to the occupant can be suppressed. This makes it possible to provide only information necessary for the driver.

[0047] [Modification 1] As Modification 1 of the present embodiment, the vehicle control apparatus 1 may turn on / off the context-based output trigger according to the driving conditions of the host vehicle 1001. Specifically, the vehicle control apparatus 1 may be configured to perform event-driven output that outputs the action intention of the host vehicle in natural language only when the host vehicle 1001 encounters a specific event or situation. For example, the action intention such as "The vehicle ahead has suddenly braked. Please slow down." may be output only in situations important for the driver, such as sudden braking, lane change, approaching an intersection, and the like.

[0048] Furthermore, the vehicle control apparatus 1 may be configured to perform request-based output that outputs an action intention in natural language only when an occupant requests information via voice or button operation. For example, when the occupant asks "What will the vehicle ahead do?", the vehicle control apparatus 1 may be configured to reply "The vehicle ahead is trying to turn right." via the notification device 24.

[0049] [Modification 2] As Modification 2 of the present embodiment, the vehicle control apparatus 1 may be provided with filtering conditions such as switching based on priority. Specifically, a priority is set for information to be output according to a driving scene in which the host vehicle or another vehicle travels, and only information with high priority is output in natural language. This provides only information necessary for the driver. For example, action intentions of other vehicles at intersections and merging points are classified as high priority, and small behaviors of other vehicles on straight roads are classified as low priority.

[0050] Furthermore, a notification method based on priority may be adopted, in which low-importance information is only displayed visually, and voice output is limited to high-priority information. The priority here can be set according to, for example, the positional relationship between the host vehicle and other vehicles, driving scenes, and the like. For example, a high-priority notification "The vehicle ahead is trying to turn right" may be output by voice, and a low-priority notification "The adjacent vehicle has changed lanes" may be displayed on a display.

[0051] [Modification 3] As a third modification of this embodiment, the vehicle control device switches the granularity of notifications and performs adaptive output according to the situation, adjusting the frequency and level of detail of output of action intentions based on factors such as vehicle speed, road conditions, and traffic volume. For example, on a highway, it notifies briefly and concisely, "The vehicle ahead is slowing down," while in urban areas or parking lots, it notifies longer and more specifically, "The vehicle on the right is backing into a parking space. Please be careful."

[0052] Furthermore, the system may allow users to customize settings to their liking, such as adjusting the frequency and level of detail of the output. For example, occupants could be able to select between a "detailed information mode" and a "simplified information mode." This would enable the provision of information tailored to each individual driver.

[0053] [Second Embodiment] Next, a second embodiment of the present invention will be described below. A characteristic feature of this embodiment is that it is configured to receive correction instructions from the occupants and reflect them in the vehicle's action plan.

[0054] Figure 13 is a functional block diagram of the occupant notification management unit of the vehicle control device according to the second embodiment of the present invention. The occupant notification management unit 403 has a correction information receiving unit 1303 in addition to the configuration of the first embodiment. The correction information receiving unit 1303 receives correction instructions from the occupants of the vehicle 1001 for the action plan of the vehicle 1001 output from the notification device 24. The correction information receiving unit 1303 receives correction instructions by manual operation via the HMI of the notification device 24 or by natural language voice input. The notification content generation unit 502 updates the action plan of the vehicle 1001 notified by the notification device 24 based on the correction information from the correction information receiving unit 1303.

[0055] Figure 14 is a control flow diagram of the notification content generation unit of a vehicle control device according to the second embodiment of the present invention. The notification content generation unit 502 acquires various information (S701) and performs a correction information reception determination to receive correction information (occupant correction information) from the occupant based on the action candidate (S1401). Then, if an audio device is selected as the output destination device, it generates audio output content that takes the correction information into consideration (S1403), and if an audio device is not selected, it generates output screen display content that takes the correction information into consideration (S1404). Then, it generates occupant notification management information including the occupant correction information (S705).

[0056] Figure 15 is a functional block diagram of the track planning unit of a vehicle control device according to a second embodiment of the present invention. The track planning unit 302 plans the target track of the vehicle 1001 using various input information, including crew correction information 1511. In addition to the configuration of the first embodiment, the track planning unit 302 has a crew correction information reflection unit 1503 that reflects the crew correction information to the track plan.

[0057] Figure 16 shows an example of a notification from the notification device of a vehicle control device according to a second embodiment of the present invention. The image shown in Figure 16 is an example of an image displayed when the notification output determination unit 501 determines that the prediction confidence level is less than a first predetermined value and a notification should be sent by the notification device 24, and further determines that the prediction confidence level is equal to or greater than a second predetermined value and a notification should be sent by the visual device 902.

[0058] The visual device 902 of the vehicle 1001 displays an image of the current situation: the vehicle 1001 intending to enter the parking lot 1000, and another vehicle 1002 in the process of exiting a parking space 1011. The behavior prediction result 1021 of the other vehicle 1002, calculated by the behavior prediction unit 401, is superimposed on the image. The action plan for the vehicle 1001 is also displayed.

[0059] In the example shown in Figure 16, two action candidates are presented as action plans: action A, which moves to standby position 1611, and action B, which moves to standby position 1612. A button for selecting an action for the vehicle 1001 is displayed at the top of the visual device 902, and the selection can be made, for example, by operating the touch panel or buttons, or by the occupant inputting in natural language verbally. For example, if action B is selected, the trajectory planning unit 302 generates a target trajectory 1602 for moving to action B, and the vehicle 1001 automatically drives based on the selected target trajectory 1602.

[0060] Alternatively, initially, only action A is displayed on the visual device 902 as the track plan for the vehicle 1001. When the crew verbally provides crew correction information in natural language, such as "I want to go further in," the action plan is updated, and action B, which reflects the crew correction information, is added to the visual device 902, resulting in a configuration where two action candidates are displayed as shown in Figure 16.

[0061] Thus, when the confidence level of the predicted behavior of the moving object falls below a first predetermined value, the vehicle control device 1 outputs a notification from the visual device 902 that includes both the predicted behavior and the action plan, and controls the notification device to receive correction instructions from the occupants of the vehicle 1001. Therefore, the occupants of the vehicle 1001 can recognize the predicted behavior of the moving object 1002 and instruct the vehicle control device 1 to update the action plan of the vehicle 1001, and the vehicle control device 1 can receive sensitive commands from the occupants. In other words, the vehicle control device 1 is configured to receive correction instructions from the occupants of the vehicle 1001 when the confidence level of the predicted behavior of the moving object falls below a first predetermined value, that is, when there is a possibility that there is an error in the judgment made by the vehicle control device 1 regarding the prediction of the behavior of the moving object 1002 or the action plan of the vehicle 1001. As a result, even if there is an error in the judgment made by the vehicle control device 1, the vehicle 1001 can execute control based on an appropriate action plan.

[0062] [Third Embodiment] Next, a third embodiment of the present invention will be described below. A distinctive feature of this embodiment is that it is configured to learn correction instructions from the occupant and reflect them in the action plan of the vehicle 1001 in similar scenes. For example, if the vehicle has received instructions from the occupant in the past in a driving scene similar to the current driving scene, the content of the instructions is memorized and reflected in notification control (such as changing the notification frequency and timing) and action plan (such as changing the stopping position) in the current driving scene.

[0063] Figure 17 is a functional block diagram of the occupant notification management unit of a vehicle control device according to the third embodiment of the present invention. The occupant notification management unit 403 receives the behavior prediction result 401a and prediction confidence level 401b, as well as current environmental information 431 and past environmental information 432. The current environmental information 431 is information about the current surrounding conditions detected by external sensors.

[0064] The past environmental information 432 is information about past surrounding conditions that has been stored in the memory of the vehicle control device 1 in the past, and includes driving environment information regarding the driving environment in which the vehicle 1001 has driven in the past, and past input information regarding past inputs received from the occupants in the driving environment.

[0065] The memory 202 of the vehicle control device 1 stores past environmental information 432, which includes scenes that the vehicle 1001 has actually encountered in the past and correction information received from the occupants regarding those scenes. The correction information included in the past environmental information 432 is stored for each occupant. This occupant-specific information can be obtained through manual input by the occupant, key information, or facial recognition. The occupant notification management unit 403 generates notification content for the notification device 24 based on a comparison between the predicted behavior of the moving object predicted by the behavior prediction unit 401 and the correction information received from the occupants in the past. The occupant notification management unit 403 controls the notification device 24 based on external information and past environmental information.

[0066] The occupant notification management unit 403, in addition to the configuration of the second embodiment, includes a similarity search unit 1701 that searches for similarity to past scenes and a correction information generation unit 1702 based on past information. The similarity search unit 1701 compares the current surrounding situation with past surrounding situations and examines the occupant correction information at that time. The similarity search unit 1701 searches for past scenes, which are past surrounding situations that have a high degree of similarity to the current surrounding situation that the vehicle 1001 is actually encountering, on an image basis, and extracts correction information from the occupant in those past scenes. The correction information generation unit 1702 uses the correction information, which is the past result extracted by the similarity search unit 1701, to generate correction information that matches the current surrounding situation. The notification content generation unit 502 generates occupant notification management information 421, which is the content of the notification to the occupant, based on the information output from the notification output determination unit 501, the correction information reception unit 1303, and the correction information generation unit 1702.

[0067] Figure 18 is a control flow diagram of the notification content generation unit of a vehicle control device according to the third embodiment of the present invention. The notification content generation unit 502 acquires various information including current environment information 431 and past environment information 432 (S701), and generates correction information based on the past information (S1801). Here, it searches for occupant correction information in similar past situations and generates correction information based on the learning results. Then, it performs a correction information reception determination to accept correction information from the occupant (occupant correction information) based on the action candidate (S1401). Then, if an audio device is selected as the output destination device (Yes in S702), it generates audio output content that takes the correction information into consideration (S1403), and if an audio device is not selected (No in S702), it generates output screen display content that takes the correction information into consideration (S1403). Then, it generates occupant notification management information including occupant correction information (S705).

[0068] Figure 19 shows an example of a notification from the notification device of a vehicle control device according to a third embodiment of the present invention. The image shown in Figure 19 is an example of an image that is displayed when the notification output determination unit 501 determines that the prediction confidence level is less than a first predetermined value and a notification should be sent by the notification device 24, and further determines that the prediction confidence level is greater than or equal to a second predetermined value and a notification should be sent by the visual device 902.

[0069] The visual device 902 of the vehicle 1001 displays an image of the current situation: the vehicle 1001 intending to enter the parking lot 1000, and another vehicle 1002 in the process of exiting a parking space 1011. The behavior prediction result 1021 of the other vehicle 1002, calculated by the behavior prediction unit 401, is superimposed on the image. The action plan for the vehicle 1001 is also displayed.

[0070] In the example shown in Figure 19, two action candidates are presented as an action plan: action A, which moves to a waiting position 1911, and action B, which moves to a waiting position 1912. A button for selecting an action candidate is displayed at the top of the visual device 902, allowing the occupant to select an action candidate, for example, via a touch panel, buttons, or verbally. In this embodiment, in addition to "Action A" 1621, "Action B" 1622, and "Other" 1623, there are additional adjustable selection buttons: "Move forward" 1921 and "Move backward" 1922. The "Move forward" 1921 and "Move backward" 1922 selection buttons are used to fine-tune the position of action A or action B.

[0071] For example, if a crew member wishes to wait at a position slightly ahead of action A, they select "action A" 1621 on the visual device 902, and then select "further ahead" 1921. As a result, the trajectory planning unit 302 outputs a target trajectory 1901 that leads to the selected position. Therefore, when a crew member of their own vehicle 1001 recognizes the predicted action result of the moving body 1002 and instructs the vehicle control device 1 to update the action plan of their own vehicle 1001, they can make minor adjustments to the action plan to their liking. This correction information, which is used for these minor adjustments, is learned in conjunction with information that identifies the individual crew member and stored in the memory 202. Therefore, by repeating the learning process, it becomes possible to prevent unnecessary notifications. Specifically, the timing at which the same crew member has requested corrections in the past is recorded, and the crew member's behavioral tendencies are accumulated as past environmental information 432.

[0072] The occupant notification management unit 403 can adjust the timing or content of notifications based on past environmental information 432 stored in the memory 202. For example, the occupant notification management unit 403 omits notifications to the occupant in scenes where corrections have not been repeated for a certain period of time. In another example, the occupant notification management unit 403 notifies the occupant the first time (e.g., in a scene being driven for the first time) to confirm the action plan, but omits notifications in the same or similar scenes if the same action is repeated in those scenes. This reduces the burden on the occupant and creates a comfortable driving environment. Furthermore, the vehicle control device 1 may generate an action plan or control its own vehicle 1001 based on past environmental information 432 stored in the memory 202. For example, if the occupant makes multiple corrections such as "I want the vehicle to decelerate earlier when passing an oncoming vehicle," correction information that changes the deceleration timing in scenes where the vehicle passes an oncoming vehicle will be stored in the occupant action profile.

[0073] The vehicle control device 1 refers to the past environmental information 432 stored in the memory 202 and automatically performs early deceleration in similar situations in the future. Alternatively, correction information can be accumulated for each scene and used when similar scenes are reproduced. For example, if there are many corrections such as "go ahead of oncoming traffic" at a particular intersection or narrow road, correction information related to behavioral instructions at that particular intersection or narrow road will be accumulated in the past environmental information 432. The vehicle control device 1 refers to the occupant behavior profile stored in the memory 202 and reflects it in subsequent operations. Furthermore, scene changes may also be recorded as past environmental information 432 so that the system can respond appropriately even when the environment or conditions change. In this way, the burden on the occupants is reduced and a comfortable driving environment is achieved.

[0074] [Fourth Embodiment] Next, a fourth embodiment of the present invention will be described below. A characteristic feature of this embodiment is that it is configured to enable notification that takes into account the degree of influence of the vehicle 1001 on the target.

[0075] Figure 20 is a functional block diagram of the occupant notification management unit of the vehicle control device according to the fourth embodiment of the present invention. The occupant notification management unit 403 has an action influence calculation unit 2101 in addition to the configuration of the second embodiment. The occupant notification management unit 403 generates notification content in the notification content generation unit 502, taking into account the occupant's target state (for example, the occupant's request to park quickly or park in a position close to the store entrance) from the calculation results of the action influence calculation unit 2101. The vehicle control device 1 receives input of the occupant's target state by the occupant's operation of the HMI of the notification device 24.

[0076] The action impact calculation unit 2101 determines the degree of influence that the vehicle 1001's action plan has on the target state. The action planning unit 402 determines the target state, including the target position of the vehicle 1001. The notification device control unit 305 controls the notification device 24 based on the degree of influence determined by the action impact calculation unit 2101.

[0077] Figure 21 is a diagram showing an example of the prediction results of the behavior of other vehicles and the trajectory plan of the own vehicle by the vehicle control device according to the fourth embodiment of the present invention. The example shown in Figure 21 shows a situation in which the own vehicle 1001 is positioned at the entrance of the parking lot 1000 in order to park in the parking lot 1000, and another vehicle 2003 is moving within the parking lot 1000 in the direction of approaching from the opposite direction.

[0078] In the example shown in Figure 21, the behavior prediction unit 401 of the autonomous driving planning unit 301 predicts two actions for the other vehicle 2003: a behavior prediction 2021 in which the other vehicle 2003 parks in the parking space 1211, and a behavior prediction 2022 in which the other vehicle moves to the waiting position and waits until its own vehicle 1001 parks. The unit determines that there is a 70% chance of the parking behavior prediction 2021 and a 30% chance of the waiting behavior prediction 2022.

[0079] For example, with the configuration of the second embodiment described above, if actions are planned based solely on the degree of prediction confidence, the vehicle 1001 may enter a waiting state without notifying the occupants, potentially resulting in movements that differ from the occupants' intentions. Therefore, in this embodiment, an action impact calculation unit 2101 is provided to implement notification control that takes into account the degree of impact on the vehicle's target. Here, the details of the action impact calculation unit 2101 and the action impact will be explained. The vehicle control device 1 considers the concept of the degree of action impact on the actions of the autonomous vehicle and makes appropriate driving decisions for each driving scene. Here, the target state refers to the state that the vehicle should ultimately achieve. For example, when driving on a highway, the target state is to safely exit at an interchange near the destination. On the other hand, in a parking lot, the target state is to stop safely and reliably in an available parking space.

[0080] The degree of behavioral impact is determined based on the extent to which coordinated actions with surrounding vehicles and objects affect the achievement of the final goal during the vehicle's journey to reach its target state. On ordinary roads and highways, actions such as lane changes and acceleration / deceleration during driving are considered to have a relatively small impact on the final target state. These actions are temporary adjustments and do not significantly hinder reaching the destination, so occupants are less interested in them. On the other hand, in a parking lot entry scenario, the target state is to stop in an available parking space, so how the vehicle responds to the movements and actions of surrounding vehicles greatly affects the achievement of the target state. For example, yielding a parking space to a surrounding vehicle necessitates searching for another available space, potentially delaying the completion of the final parking maneuver. Thus, in a parking lot, the results of coordinated actions are more directly linked to the target state, and occupants tend to be more interested in these behavioral changes. Therefore, the degree of behavioral impact differs depending on the driving scenario; it is judged to have a low impact when driving on ordinary roads and highways, and a high impact when entering a parking lot. By considering the impact on these behaviors, and by providing detailed notifications and explanations in situations of high passenger interest, and omitting unnecessary notifications in situations of low interest, a comfortable and safe autonomous driving experience can be achieved.

[0081] To specifically evaluate the impact of an action, it is effective to make a quantitative judgment based on multiple evaluation indicators. Examples of evaluation indicators include the degree of contribution to the target state (how much the action contributes to achieving the final goal), the irreversibility of the action (whether the action can be reversed once it has been performed), the degree of constraint on action selection (the range of selectable actions), prediction of changes in the surrounding environment (how the surrounding situation will change as a result of the action), and the degree of agreement with the crew's intentions (the consistency between the action and past correction history). The action impact calculation unit 2101 evaluates the impact of an action by setting scores for these indicators and calculating an overall score by summing the scores. Note that the method of calculating the impact of an action by the action impact calculation unit 2101 is not limited to this example. For example, the action impact calculation unit 2101 may determine the impact of an action using some of the evaluation indicators or other evaluation indicators from the above-mentioned multiple evaluation indicators, or it may determine the impact of an action after assigning weights to the evaluation indicators.

[0082] For example, if the notification device 24 notifies the occupant that "a vehicle 2003 is expected to park in parking space 1211, so we will wait at waiting position 2011 by taking action A," the occupant can input to the HMI, "No, I want to park before the vehicle 2003, so please move a little further." The notification content generation unit 502 generates notification content indicating a candidate action to move to action B, and notifies the occupant via the notification device 24. When the occupant selects action B via the notification device 24, the action plan of the vehicle 1001 is updated, and the action to move the vehicle to action B is executed.

[0083] Figure 22 is a graph showing the confirmation method selected according to the degree of behavioral impact and the level of confidence. For example, if the degree of behavioral impact is high and the level of confidence is low, the notification device control unit 305 (1) performs control that allows confirmation by voice. On the other hand, if both the degree of behavioral impact and the level of confidence are at an intermediate level, the notification device control unit 305 (2) performs control that allows confirmation on the screen. And if the degree of behavioral impact is low and the level of confidence is high, (3) the process of not performing confirmation by the notification device is performed. This ensures that appropriate notifications are made and that actions are taken in accordance with the intentions of the crew.

[0084] Although embodiments of the present invention have been described in detail above, the present invention is not limited to the embodiments described above, and various design modifications can be made without departing from the spirit of the invention as described in the claims. For example, the embodiments described above are described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add a configuration of another embodiment to the configuration of one embodiment. Moreover, it is possible to add, delete, or replace a part of the configuration of each embodiment with other configurations.

[0085] 1...Vehicle control device, 2-5...External sensors, 24...Notification device (notification equipment), 201...CPU (processor), 202...Memory, 301...Automatic driving planning unit, 302...Track planning unit, 303...Vehicle motion control unit, 304...Actuator control unit, 305...Notification device control unit, 401...Action prediction unit, 402...Action planning unit, 403...Crew notification management unit, 501...Notification output determination unit, 502...Notification content Generation unit, 801...Target trajectory candidate generation unit, 802...Target trajectory selection unit, 901...Audio device, 902...Visual device, 1001...Own vehicle, 1000...Parking lot, 1002, 1102, 1202, 2003...Other vehicles (moving objects), 1021...Action prediction result, 1303...Correction information reception unit, 1503...Crew correction information reflection unit, 1701...Similarity search unit, 1702...Correction information generation unit, 2101...Action impact calculation unit

Claims

1. A vehicle control device comprising a processor and memory, wherein the processor predicts the behavior of a moving object based on external information acquired by an external sensor that recognizes the external environment of the vehicle, generates an action plan for the vehicle in relation to the predicted behavior of the moving object, and determines to output a notification including at least one of the predicted behavior or the action plan when the prediction confidence level of the behavior prediction falls below a threshold, and controls a notification device that outputs the notification.

2. A vehicle control device according to claim 1, wherein the processor sets whether or not to provide a notification including at least one of the behavior prediction result or the action plan, based on the behavior prediction result of the moving body and the prediction confidence level of the behavior prediction result.

3. A vehicle control device according to claim 2, wherein the processor determines to give the notification when the prediction confidence level is less than a first predetermined value.

4. A vehicle control device according to claim 3, wherein the processor sets the implementation of notification using the voice device of the notification device when the prediction confidence level is less than a second predetermined value, and sets the implementation of notification using the visual device of the notification device when the prediction confidence level is equal to or greater than the second predetermined value.

5. A vehicle control device according to claim 1, wherein the processor receives natural language input from the occupants of the vehicle regarding the action plan output from the notification device, and updates the action plan based on the input.

6. A vehicle control device according to claim 2, wherein the memory stores as past environment information a record of the driving environment in which the vehicle has driven in the past, and past input information a record of past inputs received from the occupants of the vehicle in the driving environment, and the processor controls the notification device based on the external information and the past environment information.

7. A vehicle control device according to claim 1, wherein the processor determines a target state including the target position of the vehicle, determines the degree of action influence that the action plan has on the target state, and controls the notification device based on the degree of action influence.

8. A vehicle control method performed by a vehicle control device comprising a processor and memory, wherein the processor performs the steps of: predicting the behavior of a moving object based on external information acquired by an external sensor that recognizes the external environment of the vehicle; generating an action plan for the vehicle in response to the predicted behavior of the moving object; and determining that a notification including at least one of the predicted behavior or the action plan should be output when the prediction confidence level of the behavior prediction falls below a threshold, and controlling a notification device that outputs the notification.

Images