Control device, control method, and control program

Abstract

To provide a technique enabling effective monitoring and operation of automatic operation control using a machine learning model, with respect to a control device of a vehicle.SOLUTION: A control device includes one or more processors for performing automatic operation control of a vehicle by using a machine learning model. The vehicle is configured to be operated by an operator. The one or more processors are configured to execute: processing for calculating reliability related to an inference result by the machine learning model during executing the automatic operation control; processing for issuing a notification to urge the operator to confirm an operation environment of the vehicle when a predetermined first condition indicating deterioration of the reliability is satisfied; and processing for issuing a notification to request a hands-on operation to the operator when a predetermined second condition indicating the further deterioration of the reliability in addition to the first condition is satisfied.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to a technology for performing autonomous driving control of a vehicle using a machine learning model. [Background technology]

[0002] In recent years, various techniques have been proposed to utilize machine learning models as artificial intelligence (AI) in the field of vehicle control.

[0003] For example, Patent Document 1 discloses a vehicle control assistance system that aims to assist vehicle control using a machine learning model (trained model) that reflects the vehicle state, the driver's intentions, etc. Other documents that show the technical level in this technical field include the following Patent Documents 2 and 3. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2022-067454 A [Patent Document 2] JP 2020-004402 A [Patent Document 3] JP 2021-195933 A Summary of the Invention [Problem to be solved by the invention]

[0005] Among vehicle controls, autonomous driving control in particular is expected to improve control performance by utilizing machine learning models in each of the functions of recognition, judgment, and operation. However, autonomous driving control requires appropriate judgment as to whether control can be continued normally. On the other hand, the inference results of machine learning models contain a probabilistic element. For this reason, when using machine learning models to control autonomous driving of a vehicle, it is expected that it will be difficult to judge whether control can be continued normally.

[0006] One objective of the present disclosure has been made in consideration of the above-mentioned problems, and is to provide technology that enables effective monitoring and operation of autonomous driving control that uses machine learning models. [Means for solving the problem]

[0007] A first aspect of the present disclosure relates to a control device for a vehicle.

[0008] A control device according to a first aspect includes one or more processors that perform autonomous driving control of a vehicle using a machine learning model. The vehicle is configured to be driven by an operator, and the one or more processors are further configured to execute a process of calculating a reliability of an inference result by the machine learning model while executing autonomous driving control, a process of issuing a notification to the operator to prompt the operator to check the driving environment of the vehicle when a predetermined first condition indicating that the reliability is decreasing is satisfied, and a process of issuing a notification to the operator requesting a hands-on operation when a predetermined second condition indicating that the reliability is decreasing in addition to the first condition is satisfied.

[0009] A second aspect of the present disclosure relates to a control method for a vehicle configured to be driven by an operator.

[0010] A control method relating to a second aspect includes the steps of: performing autonomous driving control of a vehicle using a machine learning model; calculating the reliability of an inference result by the machine learning model while executing the autonomous driving control; notifying an operator to check the vehicle's driving environment when a predetermined first condition indicating a decrease in reliability is met; and notifying the operator to take a hands-on approach when a predetermined second condition indicating a further decrease in reliability in addition to the first condition is met.

[0011] A third aspect of the present disclosure relates to a control program that causes a computer to execute control of a vehicle configured to be driven by an operator.

[0012] The control program relating to the third aspect is configured to cause a computer to execute the following processes: a process of performing autonomous driving control of a vehicle using a machine learning model; a process of calculating the reliability of the inference results by the machine learning model while executing the autonomous driving control; a process of notifying an operator to check the driving environment of the vehicle when a predetermined first condition indicating a decrease in reliability is met; and a process of notifying the operator to take a hands-on approach when a predetermined second condition indicating a further decrease in reliability in addition to the first condition is met. Effect of the Invention

[0013] According to the present disclosure, the reliability of the inference result by the machine learning model is calculated while the autonomous driving control is being executed. Then, the decrease in the reliability is evaluated in two stages, and when the reliability satisfies a first condition, a notification is sent to the operator of the vehicle 1 to prompt the operator to check the driving environment, and when the reliability satisfies a second condition in addition to the first condition, a notification is sent to the operator requesting a hands-on operation. This makes it possible to effectively monitor and operate the autonomous driving control using the machine learning model. [Brief description of the drawings]

[0014] [Figure 1] FIG. 2 is a diagram illustrating an example of a configuration related to an automatic driving function of a vehicle according to the present embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a configuration of a control device according to the present embodiment. [Diagram 3] 11 is a diagram for explaining an example of a reliability calculated in a notification process executed by a processor; FIG. [Figure 4] 11 is a flowchart illustrating an example of a process executed by a processor in a notification process. [Diagram 5] FIG. 2 is a diagram showing an example of the control device according to the present embodiment. [Figure 6] 13 is a flowchart illustrating an example of a process executed by a processor in a notification process according to another embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, an embodiment will be described with reference to the drawings.

[0016] 1.Autonomous driving of vehicles This embodiment relates to a control device for a vehicle. The control device according to this embodiment performs at least automatic driving control of the vehicle. FIG. 1 is a block diagram showing a configuration example related to automatic driving control of a vehicle 1 by the control device according to this embodiment. Automatic driving control is a concept including not only fully automatic driving control but also risk avoidance control, lane keep assist control, and the like.

[0017] The vehicle 1 includes a sensor group 10, a recognition unit 20, a planner 30, a control amount calculation unit 40, and a traveling device 50.

[0018] The sensor group 10 includes a recognition sensor 11 used to recognize the situation around the vehicle 1. Examples of the recognition sensor 11 include a camera, a LIDAR (Laser Imaging Detection and Ranging), a radar, and the like. The sensor group 10 may further include a status sensor 12 that detects the status of the vehicle 1, a position sensor 13 that detects the position of the vehicle 1, and the like. Examples of the status sensor 12 include a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, and the like. Examples of the position sensor 13 include a GNSS (Global Navigation Satellite System) sensor.

[0019] The sensor detection information SEN is information obtained by the sensor group 10. For example, the sensor detection information SEN includes images captured by a camera. As another example, the sensor detection information SEN may include point cloud information obtained by a LIDAR. The sensor detection information SEN may include vehicle state information indicating the state of the vehicle 1. The sensor detection information SEN may include position information indicating the position of the vehicle 1.

[0020] The recognition unit 20 receives the sensor detection information SEN. The recognition unit 20 recognizes the situation around the vehicle 1 based on the information obtained by the recognition sensor 11. For example, the recognition unit 20 recognizes objects around the vehicle 1. Examples of objects include pedestrians, other vehicles (preceding vehicles, parked vehicles, etc.), white lines, road structures (e.g., guardrails, curbs), fallen objects, traffic lights, intersections, signs, etc. The recognition result information RES indicates the recognition result by the recognition unit 20. For example, the recognition result information RES includes object information indicating the relative position and relative speed of the object with respect to the vehicle 1.

[0021] The planner 30 receives the recognition result information RES from the recognition unit 20. The planner 30 may also receive vehicle state information, position information, and pre-generated map information. The map information may be high-precision three-dimensional map information. The planner 30 generates a driving plan for the vehicle 1 based on the received information. The driving plan may be for reaching a pre-set destination, or for avoiding risks. Examples of the driving plan include maintaining the current driving lane, changing lanes, overtaking, turning right or left, steering, accelerating, decelerating, and stopping. Furthermore, the planner 30 generates a target trajectory TRJ required for the vehicle 1 to travel according to the driving plan. The target trajectory TRJ includes a target position and a target speed.

[0022] The control amount calculation unit 40 receives the target trajectory TRJ from the planning unit 30. The control amount calculation unit 40 calculates a control amount CON required for the vehicle 1 to follow the target trajectory TRJ. It can also be said that the control amount CON is a control amount required to reduce the deviation between the vehicle 1 and the target trajectory TRJ. The control amount CON includes at least one of a steering control amount, a drive control amount, and a braking control amount. Examples of the steering control amount include a target steering angle, a target torque, a target motor angle, a target motor drive current, etc. Examples of the drive control amount include a target speed, a target acceleration, etc. Examples of the braking control amount include a target speed, a target deceleration, etc.

[0023] The traveling device 50 includes a steering device 51, a driving device 52, and a braking device 53. The steering device 51 steers the wheels. For example, the steering device 51 includes an electric power steering (EPS) device. The driving device 52 is a power source that generates a driving force. Examples of the driving device 52 include an engine, an electric motor, and an in-wheel motor. The braking device 53 generates a braking force. The traveling device 50 receives a control amount CON from the control amount calculation unit 40. The traveling device 50 operates the steering device 51, the driving device 52, and the braking device 53 according to the steering control amount, the driving control amount, and the braking control amount, respectively. As a result, the vehicle 1 travels so as to follow the target trajectory TRJ.

[0024] The recognition unit 20 includes at least one of a rule-based model and a machine learning model. The rule-based model performs recognition processing based on a group of predetermined rules. Examples of the machine learning model include a neural network (NN), a support vector machine (SVM), a regression model, a decision tree model, and the like. The NN may be a convolutional neural network (CNN), a recurrent neural network (RNN), or a combination thereof. The type of each layer, the number of layers, and the number of nodes in the NN are arbitrary. The machine learning model is generated in advance through machine learning. The recognition unit 20 performs recognition processing by inputting the sensor detection information SEN to the model. The recognition result information RES is output from the model, or is generated based on the output from the model. Hereinafter, the model included in the recognition unit 20 is also referred to as a "recognition model."

[0025] Similarly, the planning unit 30 includes at least one of a rule-based model and a machine learning model. The planning unit 30 performs planning processing by inputting recognition result information RES to the model. The target trajectory TRJ is output from the model or is generated based on the output from the model. Hereinafter, the model included in the planning unit 30 is also referred to as a "planning model."

[0026] Similarly, the control amount calculation unit 40 includes at least one of a rule-based model and a machine learning model. The control amount calculation unit 40 performs a control amount calculation process by inputting the target trajectory TRJ to the model. The control amount CON is output from the model, or is generated based on the output from the model. Hereinafter, the model included in the control amount calculation unit 40 is referred to as a "control amount calculation model."

[0027] Two or more of the recognition unit 20, the planner 30, and the control amount calculation unit 40 may be integrally configured. All of the recognition unit 20, the planner 30, and the control amount calculation unit 40 may be integrally configured (end-to-end configuration). For example, the recognition unit 20 and the planner 30 may be integrally configured by a NN that outputs a target trajectory TRJ from the sensor detection information SEN. Even in the case of an integral configuration, intermediate products such as recognition result information RES and target trajectory TRJ may be output. For example, when the recognition unit 20 and the planner 30 are integrally configured by a NN, the recognition result information RES may be the output of an intermediate layer of the NN.

[0028] The recognition unit 20, the planning unit 30, and the control amount calculation unit 40 constitute an "automatic driving control unit" that controls automatic driving of the vehicle 1. In this embodiment, a machine learning model is used for at least a part of the automatic driving control unit. That is, the automatic driving control unit uses a machine learning model as at least one of the recognition model, the planning model, and the control amount calculation model.

[0029] 2 is a block diagram showing an example of a hardware configuration of the control device 101 according to this embodiment. The control device 101 is mounted on the vehicle 1, and performs at least automatic driving control of the vehicle 1. In other words, the control device 101 has at least the functions of the automatic driving control unit described above.

[0030] The control device 101 is configured to be able to communicate with the sensors 10, the traveling device 50, the display unit 60, the speaker 70, and the actuators.

[0031] The control device 101 is a computer including one or more processors 110 (hereinafter simply referred to as processors 110) and one or more storage devices 120 (hereinafter simply referred to as storage devices 120).

[0032] The processor 110 executes various processes. Examples of the processor 110 include a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), and a field-programmable gate array (FPGA). The recognition unit 20, the planning unit 30, and the control amount calculation unit 40 may be realized by a single processor 110, or may be realized by separate processors 110. The storage device 120 stores various information. Examples of the storage device 120 include a hard disk drive (HDD), a solid state drive (SSD), a volatile memory, and a non-volatile memory.

[0033] The storage device 120 stores a computer program 130 and model data 140 .

[0034] The computer program 130 is executed by the processor 110. Various processes by the control device 101 may be realized by cooperation between the processor 110, which executes the computer program 130, and the storage device 120. The computer program 130 may be recorded on a computer-readable recording medium.

[0035] The model data 140 is data of a recognition model, a planning model, and a control amount calculation model. In executing the autonomous driving control, the processor 110 configures the recognition unit 20, the planning unit 30, and the control amount calculation unit 40 by using the models read from the model data 140. As described above, in this embodiment, a machine learning model is used as at least one of the recognition model, the planning model, and the control amount calculation model.

[0036] The display unit 60 performs various displays. The display unit 60 is configured to be controllable by the control device 101. Examples of the display unit 60 include a meter panel display, a multi-information display, a head-up display, an indicator, and the like.

[0037] The speaker 70 emits various sounds. The speaker 70 is configured to be controllable by the control device 101.

[0038] In this embodiment, the vehicle 1 is configured to be capable of being driven by an operator. For example, the vehicle 1 is equipped with a driving operation device (steering wheel, accelerator pedal, brake pedal, etc.) that accepts driving operations by the operator. In this case, the operator is typically a driver who boards the vehicle 1. However, the operator may also be a remote operator who remotely operates the vehicle 1. The vehicle 1 may be configured so that, when automatic driving control is being performed, the operator can override and drive the vehicle 1 by operating the driving operation device.

[0039] 2. Notification Processing As described above, the control device 101 according to the present embodiment performs autonomous driving control of the vehicle 1 by using a machine learning model. In autonomous driving control, it is required to appropriately determine whether control can be continued normally. Meanwhile, the inference result by the machine learning model is calculated based on probabilistic inference based on training data. For this reason, there is a risk of an error in the inference result, and it may be inappropriate to determine whether control can be continued normally from the inference result alone.

[0040] Therefore, in the control device 101 according to this embodiment, the processor 110 calculates the reliability of the inference result by the machine learning model while executing the automatic driving control. Then, the processor 110 is configured to execute a process of notifying the operator of the vehicle 1 according to the calculated reliability (hereinafter, referred to as "notification process"). The notification process will be described in detail below.

[0041] 2-1. Reliability The reliability calculated in the notification process may be the confidence level of a machine learning model for the inference result. An example of the reliability calculated in the notification process will be described with reference to FIG.

[0042] Fig. 3(A) shows a case where a machine learning model is used as the control amount calculation model 144. In Fig. 3(A), the control amount calculation model 144 is configured to receive the target trajectory TRJ as an input and output three control amounts CON1, CON2, and CON3 as inference results. For example, the three control amounts CON1, CON2, and CON3 are a steering control amount, a driving control amount, and a braking control amount, respectively. The control amount calculation unit 40 outputs the control amount CON including the three control amounts CON1, CON2, and CON3.

[0043] In FIG. 3(A), the controlled variable calculation model 144 is further configured to output a confidence level for each of the three controlled variables CON1, CON2, and CON3. At this time, the processor 110 can calculate a reliability level for the inference result by the controlled variable calculation model 144 based on each confidence level. For example, the processor 110 calculates the lowest confidence level as the reliability level. In this case, in FIG. 3(A), the processor 110 calculates the reliability level for the inference result by the controlled variable calculation model 144 to be 30%. Also, for example, the processor 110 calculates the average value of each confidence level as the reliability level. In this case, in FIG. 3(A), the processor 110 calculates the reliability level for the inference result by the controlled variable calculation model 144 to be 70%.

[0044] When the autonomous driving control unit uses multiple machine learning models, the processor 110 can calculate the reliability for all of the multiple machine learning models in the same manner as described above. Alternatively, the processor 110 may calculate the reliability for each machine learning model as follows.

[0045] FIG. 3(B) shows a case where machine learning models are used as the recognition model 142 and the planning model 143. In FIG. 3(B), the recognition model 142 is configured to receive sensor detection information SEN as an input and output two pieces of recognition result information RES1 and RES2 as inference results. For example, the two pieces of recognition result information RES1 and RES2 are the recognition results of other vehicles and the recognition results of white lines, respectively. The recognition unit 20 outputs recognition result information RES including the two pieces of recognition result information RES1 and RES2. The planning model 143 is configured to receive recognition result information RES as an input and output a target trajectory TRJ as an inference result. The planning unit 30 outputs the target trajectory TRJ output from the planning model 143.

[0046] 3B, the recognition model 142 is further configured to output a confidence level for each of the two pieces of recognition result information RES1 and RES2. The planning model 143 is further configured to output a confidence level for the target trajectory TRJ. At this time, the processor 110 may calculate the reliability of the inference result by the recognition model 142 and the reliability of the inference result by the planning model 143 based on each confidence level. For example, the processor 110 calculates the reliability of the inference result by the recognition model 142 to be 70% and the reliability of the inference result by the planning model 143 to be 90%.

[0047] As described above, the processor 110 can calculate the reliability. As another example of the reliability, when a machine learning model outputs a plurality of inference results, the reliability can be calculated using the degree of variation between the plurality of inference results as an index. For example, when a machine learning model is used as a planning model, the reliability can be calculated using the time-series variation between the plurality of output target trajectories TRJ as an index.

[0048] 2-2. Processing Fig. 4 is a flowchart showing an example of the notification process executed by the processor 110. The process according to the flowchart shown in Fig. 4 is repeatedly executed at a predetermined processing cycle while the control device 101 is running.

[0049] First, in step S100, the processor 110 calculates the reliability of an inference result by a machine learning model used in the autonomous driving control unit. For example, as described above, the processor 110 obtains a confidence level for each inference result, and calculates the reliability based on the obtained confidence levels.

[0050] Next, in step S110, the processor 110 determines whether the calculated confidence has decreased to below a first predetermined threshold. If a confidence is calculated for each machine learning model, the processor 110 may be configured to determine whether any of the confidences has decreased to below a first threshold.

[0051] If the reliability is equal to or greater than the first threshold (step S110; No), the processor 110 ends the current process without notifying the operator. If the reliability has decreased to less than the first threshold (step S110; Yes), the process proceeds to step S120.

[0052] In step S120, the processor 110 determines whether the calculated confidence has decreased to less than a second predetermined threshold value that is less than the first threshold value. When confidence is calculated for each machine learning model, the processor 110 may be configured to determine whether any of the confidence values ​​determined to be less than the first threshold value has decreased to less than the second threshold value.

[0053] When the reliability is equal to or greater than the second threshold (step S120; No), that is, when the reliability has decreased to less than the first threshold but not to less than the second threshold, the processor 110 notifies the operator to prompt the operator to check the driving environment of the vehicle 1 (step S130). "Checking the driving environment" refers to checking surrounding objects, checking surrounding traffic conditions, checking the running state of the vehicle, checking the status of the automatic driving control (the driving plan and the target trajectory TRJ), and the like. For example, the processor 110 causes the speaker 70 to output a voice such as "Check the surroundings of the vehicle and be careful when driving by automatic driving." Also, for example, the processor 110 turns on an indicator indicating that the situation requires checking the driving environment. Also, for example, the processor 110 causes the meter panel display or the multi-information display to display the driving plan and the target trajectory TRJ of the automatic driving control.

[0054] After step S130, processor 110 ends the current process.

[0055] By issuing a notification to prompt the operator to check the driving environment, the operator of the vehicle 1 is expected to check the driving environment and monitor whether the autonomous driving control is being performed normally. The operator of the vehicle 1 can prepare for the case where the autonomous driving control is not being performed normally.

[0056] When the reliability falls below the second threshold (step S120; Yes), that is, when the reliability falls below the first threshold and then further falls below the second threshold, the processor 110 notifies the operator to request a hands-on state (step S140). The "hands-on" state here refers to the state of the operator when driving the vehicle 1, and various states can be assumed depending on the environment to which the present embodiment is applied. Typically, it is a state in which the operator sits in the driver's seat and grips the steering wheel. For example, the processor 110 causes the speaker 70 to output a voice such as "Be prepared to immediately perform driving operations and be careful when driving in automatic driving mode." Also, for example, the processor 110 turns on an indicator indicating that a hands-on state is being requested. Also, for example, the processor 110 causes the meter panel display or the multi-information display to display a window warning that a hands-on state is being requested.

[0057] After step S140, processor 110 ends the current process.

[0058] When the reliability has decreased to a considerable extent, it is considered that there is a high possibility that the autonomous driving control will suddenly not be performed normally. By issuing a notification requesting a hands-on, the operator of the vehicle 1 is expected to immediately start driving the vehicle 1 by terminating the autonomous driving control or by overriding the autonomous driving control when it is determined that the autonomous driving control is not being performed normally.

[0059] As described above, the notification process is executed in the control device 101 according to the present embodiment. The control method according to the present embodiment is implemented by the processor 110 executing the process in this manner. The control program according to the present embodiment is realized by the computer program 130 that causes the processor 110 to execute the process in this manner.

[0060] Incidentally, it is considered that the reliability during execution of the automatic driving control changes more significantly as the traffic situation around the vehicle 1 becomes more complex. For example, in a situation where there are many objects around the vehicle 1, it is predicted that the reliability is likely to change due to frequent object recognition. In such a situation, since the reliability is likely to change suddenly, it is desirable to notify the operator early.

[0061] Therefore, the processor 110 may be configured to change the first threshold value or the second threshold value depending on the complexity of the traffic situation around the vehicle 1 in the notification process.

[0062] The complexity of the traffic situation can be indexed by the type of road on which the vehicle 1 is traveling, the traffic volume, the geographical conditions, etc. For example, the complexity of the traffic situation is low when the vehicle 1 is traveling on a highway, and high when the vehicle 1 is traveling on a city street. Also, for example, the greater the traffic volume around the vehicle 1, the higher the complexity of the traffic situation. Also, for example, the complexity of the traffic situation is low when the weather around the vehicle 1 is sunny, and high when the weather around the vehicle 1 is rainy.

[0063] The processor 110 can obtain the surrounding environment information of the vehicle 1 as an index, and calculate the complexity of the traffic situation around the vehicle 1 based on the obtained surrounding environment information. The processor 110 obtains the surrounding environment information from, for example, the sensor detection information SEN or the recognition result information RES. The processor 110 may evaluate the complexity of the traffic situation in stages or with a continuous numerical value.

[0064] When the complexity of the traffic situation is high, the processor 110 changes the first threshold value or the second threshold value to be larger. Fig. 5 is a diagram showing an embodiment in which the first threshold value or the second threshold value is changed according to the complexity of the traffic situation. Fig. 5 shows an example of a time change in reliability, and the first threshold value and the second threshold value according to the complexity of the traffic situation. In Fig. 5, the processor 110 changes the first threshold value and the second threshold value to be larger when the complexity of the traffic situation is high.

[0065] 5, by changing the first threshold value or the second threshold value according to the complexity of the traffic situation, it is possible to advance the timing of issuing a notification (confirmation notification) urging the operator to check the driving environment of the vehicle 1 or a notification (hands-on notification) requesting a hands-on operation. In this way, in a situation where the reliability is likely to change, it is possible to notify the operator earlier.

[0066] Additionally, the processor 110 may be configured to change the first threshold value or the second threshold value depending on the state of the vehicle 1. For example, the processor 110 may change the first threshold value or the second threshold value to be larger when the speed of the vehicle 1 is high.

[0067] 2-3. Other embodiments As described above, in the notification process, the decrease in reliability is evaluated in two stages, and when the condition indicating the first stage (first condition) is met, a notification is issued urging the driver to check the driving environment of the vehicle 1, and when the condition indicating the second stage (second condition) is met, a notification is issued requesting a hands-on operation. In the above description, the first condition is that the reliability is less than a first threshold, and the second condition is that the reliability is less than a second threshold that is further smaller than the first threshold.

[0068] Meanwhile, the autonomous driving control unit uses a machine learning model as at least one of the recognition model, the planning model, and the control amount calculation model. In particular, the autonomous driving control unit is greatly influenced by using a machine learning model as the upstream recognition model and planning model. In addition, the processor 110 can calculate the reliability of each of the recognition model and the planning model.

[0069] Therefore, when machine learning models are used as the recognition model and the planning model, it is possible to evaluate the decrease in reliability in two stages using the reliability of the inference result by the recognition model (hereinafter referred to as "first reliability") and the reliability of the inference result by the planning model (hereinafter referred to as "second reliability"). From this point of view, the notification process can also adopt the following embodiment.

[0070] Fig. 6 is a flowchart showing an example of a notification process according to another embodiment, which is executed by the processor 110. The process according to the flowchart shown in Fig. 6 is repeatedly executed at a predetermined processing cycle while the control device 101 is running.

[0071] First, in step S200, the processor 110 calculates a first reliability and a second reliability.

[0072] Next, in step S210, the processor 110 determines whether either the first reliability or the second reliability has decreased to below a predetermined threshold.

[0073] If both the first reliability and the second reliability are equal to or greater than the threshold (step S210; No), the processor 110 ends the current process without notifying the operator. If either the first reliability or the second reliability has decreased to below a predetermined threshold (step S210; Yes), the process proceeds to step S220.

[0074] In step S220, the processor 110 determines whether both the first reliability and the second reliability have decreased to below a threshold value.

[0075] If either the first reliability or the second reliability is equal to or greater than the threshold value (step S220; No), that is, if only one of the first reliability or the second reliability has decreased to below the threshold value, the processor 110 notifies the operator to check the driving environment of the vehicle 1 (step S230).

[0076] After step S230, processor 110 ends the process.

[0077] When both the first reliability and the second reliability have decreased to be less than the threshold value (step S220; Yes), the processor 110 notifies the operator to request a hands-on session (step S240).

[0078] After step S240, processor 110 ends the process.

[0079] As described above, the processor 110 may execute a notification process according to another embodiment. In the notification process according to the other embodiment, a first condition is that either the first reliability or the second reliability is less than a predetermined threshold, and a second condition is that both the first reliability and the second reliability are less than a threshold.

[0080] 3.Effects As described above, according to this embodiment, the reliability of the inference result by the machine learning model is calculated while the automatic driving control is being executed. Then, the decrease in reliability is evaluated in two stages, and when the reliability satisfies the first condition, a notification is given to the operator of the vehicle 1 to prompt him / her to check the driving environment, and when the reliability satisfies the second condition in addition to the first condition, a notification is given to the operator requesting a hands-on operation. By giving a notification to prompt him / her to check the driving environment, the operator can check the driving environment and monitor whether the automatic driving control is being performed normally. Also, by giving a notification requesting a hands-on operation, the operator can be put into a state where he / she can immediately start driving the vehicle 1 in preparation for the case where it is determined that the automatic driving control is not being performed normally. In this way, according to this embodiment, the automatic driving control using the machine learning model can be effectively monitored and operated. [Explanation of symbols]

[0081] 1 vehicle, 10 sensor group, 20 recognition unit, 30 planning unit, 40 control amount calculation unit, 50 running device, 60 display unit, 70 speaker, 101 control device, 110 processor, 120 storage device, 130 computer program, 142 Recognition model, 143 Planning model, 144 Control quantity calculation model

Claims

1. A control device for a vehicle, One or more processors that utilize a machine learning model to perform autonomous driving control of the vehicle; The vehicle is configured to be driven by an operator; The one or more processors further include: A process of calculating a reliability of an inference result by the machine learning model while executing the autonomous driving control; When a predetermined first condition indicating that the reliability is decreased is satisfied, a process of notifying the operator to check the driving environment of the vehicle; When a predetermined second condition indicating that the reliability is decreased is satisfied in addition to the first condition, a process of notifying the operator of a request for a hands-on operation is performed; It is configured to run Control device.

2. The control device according to claim 1 , The one or more processors calculate the confidence based on a confidence level of the machine learning model for the inference result. Characterized by Control device.

3. The control device according to claim 1 or 2, the first condition is that the reliability is less than a predetermined first threshold value; The second condition is that the reliability is less than a second predetermined threshold value that is smaller than the first threshold value. Characterized by Control device.

4. The control device according to claim 3, The one or more processors further include: A process of acquiring surrounding environment information of the vehicle; A process of calculating a complexity of a traffic situation around the vehicle based on the surrounding environment information; A process of changing the first threshold value or the second threshold value according to the complexity; It is configured to run Control device.

5. The control device according to claim 1 or 2, The machine learning model includes a recognition model that recognizes a situation around the vehicle and a planning model that generates a target trajectory for the autonomous driving control; the reliability includes a first reliability regarding the inference result by the recognition model and a second reliability regarding the inference result by the planning model; the first condition is that either the first reliability or the second reliability is less than a predetermined threshold; The second condition is that both the first reliability and the second reliability are less than the threshold value. Characterized by Control device.

6. A method for controlling a vehicle configured to be driven by an operator, comprising: Performing automatic driving control of the vehicle using a machine learning model; Calculating a reliability of an inference result by the machine learning model while executing the autonomous driving control; When a predetermined first condition indicating that the reliability is decreased is satisfied, a notification is given to the operator to prompt the operator to check a driving environment of the vehicle. When a predetermined second condition indicating that the reliability is decreased is satisfied in addition to the first condition, a notification requesting a hands-on is given to the operator; is executed by a computer Control methods.

7. A control program for causing a computer to execute control of a vehicle configured to be driven by an operator, A process of performing automatic driving control of the vehicle using a machine learning model; A process of calculating a reliability of an inference result by the machine learning model while executing the autonomous driving control; When a predetermined first condition indicating that the reliability is decreased is satisfied, a process of notifying the operator to check the driving environment of the vehicle; When a predetermined second condition indicating that the reliability is decreased is satisfied in addition to the first condition, a process of notifying the operator of a request for a hands-on operation is performed; The computer is configured to execute Control program.

Images