Recognition state notification device

Abstract

To provide a recognition state notification device for enabling a monitoring person to appropriately obtain a degree of self-confidence in recognition of an automatic operation system.SOLUTION: A recognition state notification device 5 includes: a self-confidence calculation section 13 for calculating a degree of self-confidence in recognition on a recognized target recognized by a recognition section 11; and a notification section 15 for notifying a monitoring person of an own vehicle V of the recognized target where a degree of self-confidence is equal to or less than a reference value.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a recognition state notification device. [Background technology]

[0002] For example, Patent Document 1 describes an apparatus for designing a mechanical structure. This apparatus designs the mechanical structure based on machine learning. This apparatus also calculates the confidence level of a design proposal based on the results of the machine learning. This apparatus then displays the calculated confidence level. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-004069 A Summary of the Invention [Problem to be solved by the invention]

[0004] AI [Artificial Intelligence] is also used in the field of autonomous driving, which allows vehicles to drive automatically. For example, there is a possibility that the person monitoring the autonomous vehicle may be overconfident in the output of the autonomous driving system using AI. For this reason, it is necessary to notify the person monitoring the autonomous vehicle so that they can properly understand the confidence level of the autonomous driving system's recognition.

[0005] For this reason, the present disclosure describes a recognition state notification device that allows an observer to appropriately grasp the degree of confidence in the recognition of an autonomous driving system. [Means for solving the problem]

[0006] One aspect of the present disclosure is a recognition state notification device in an autonomous driving system that performs autonomous driving of a vehicle using a recognition result of a recognition unit that recognizes surrounding targets based on a detection result of an external sensor. The recognition state notification device includes a confidence level calculation unit that calculates a recognition confidence level for a recognized target that is a target recognized by the recognition unit, and a notification unit that notifies a recognized target whose confidence level is equal to or lower than a reference value to a supervisor of the vehicle.

[0007] In the above-mentioned recognition state notification device, the notification unit may notify a recognized object having a confidence level equal to or lower than a reference value in a manner according to the value of the confidence level. In the above-mentioned recognition state notification device, when notifying a recognized object having a confidence level equal to or lower than a reference value, the notification unit may display, in different manners, a portion of the recognized object that has been recognized by the recognition unit and a portion that has not been recognized by the recognition unit.

[0008] The above-mentioned device may further include a starting point setting unit that sets a target that is a starting point of control for the autonomous driving based on a route plan for the autonomous driving of the vehicle. The notification unit may notify a recognized target that is set as a starting point of control by the starting point setting unit and has a confidence level equal to or lower than a reference value. Effect of the Invention

[0009] According to one aspect of the present disclosure, a monitor can appropriately grasp the degree of confidence in the autonomous driving system's recognition. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing an example of an autonomous driving system equipped with a recognition state notification device. [Diagram 2] FIG. 2 is a flowchart showing a flow of a process for notifying the degree of confidence of a recognized target, which is performed by the recognition state notifying device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, exemplary embodiments will be described with reference to the drawings. In each drawing, the same or corresponding elements are denoted by the same reference numerals, and duplicated explanations will be omitted.

[0012] The autonomous driving system 1 shown in FIG. 1 is mounted on a host vehicle (vehicle) V that is capable of traveling autonomously. In other words, the host vehicle V is an autonomously driven vehicle. The autonomous driving system 1 executes autonomous driving to drive the host vehicle V autonomously. Note that some of the functions of the autonomous driving system 1 may be installed outside the host vehicle V. The autonomous driving system 1 includes an external sensor 2, an HMI [Human Machine Interface] 3, and an ECU [Electronic Control Unit] 4.

[0013] The external sensor 2 is an on-board sensor that detects the external environment of the host vehicle V. The external sensor 2 may include a camera. The camera is an imaging device that captures images of the external environment of the host vehicle V. The camera is provided, for example, on the back side of the windshield of the host vehicle V, and captures images of the area in front of the vehicle. The camera transmits imaging information related to the external environment of the host vehicle V to the ECU 4. The camera may be a monocular camera or a stereo camera. Multiple cameras may be provided. The camera may capture images of the area in front of the host vehicle V, as well as the left and right sides, etc.

[0014] The external sensor 2 may include a radar sensor. The radar sensor is a detection device that detects objects around the vehicle V by using radio waves (e.g., millimeter waves) or light. The radar sensor includes, for example, a millimeter wave radar or a lidar (LIDAR: Light Detection and Ranging). The radar sensor detects objects by transmitting radio waves or light to the vicinity of the vehicle V and receiving the radio waves or light reflected by the objects. The radar sensor transmits information about the detected objects to the ECU 4.

[0015] The HMI 3 is an interface for inputting and outputting information between the host vehicle V and a passenger. The HMI 3 has a display, a speaker, and the like. The passenger is a monitor who monitors the traveling of the host vehicle V. The monitor may switch the driving state of the host vehicle V from automatic driving to manual driving or assist in the driving of the host vehicle V based on the information output from the HMI 3.

[0016] The ECU 4 is an electronic control unit having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The ECU 4 realizes various functions by, for example, loading a program recorded in the ROM into the RAM and executing the program loaded into the RAM with the CPU. The ECU 4 may be composed of a plurality of electronic units. Functionally, the ECU 4 includes a recognition unit 11, a driving execution unit 12, a confidence level calculation unit 13, a starting point setting unit 14, and a notification unit 15.

[0017] The recognition unit 11 recognizes surrounding targets based on the detection results of the external sensor 2. For example, the recognition unit 11 can recognize surrounding targets using AI technology such as machine learning based on the detection results of the external sensor 2. Targets recognized by the recognition unit 11 include, for example, surrounding vehicles, white lines (division lines) on the road, pedestrians, etc. The recognition unit 11 can recognize the type of target in addition to the position and shape of the target.

[0018] The driving execution unit 12 executes automatic driving of the host vehicle V based on a well-known method. The driving execution unit 12 can control the driving of the host vehicle V by issuing instructions to actuators (drive actuators, brake actuators, steering actuators, etc.) used to control the driving of the host vehicle V. For example, the driving execution unit 12 generates a route plan for automatic driving based on the recognition results of the recognition unit 11, etc., and controls the driving of the host vehicle V according to the route plan.

[0019] The confidence level calculation unit 13 calculates the confidence level of the recognition of the recognized target, which is a target recognized by the recognition unit 11. The confidence level calculation unit 13 can calculate the confidence level of the recognition of the recognized target based on a well-known method. For example, the confidence level calculation unit 13 may calculate the confidence level based on the temporal stability and continuity of the recognition result. For example, when the external sensor 2 includes a plurality of sensors, the confidence level calculation unit 13 may calculate the confidence level based on the consistency of the recognition results between the sensors. In addition, there are cases where the recognition results of the surrounding targets can be acquired from other surrounding vehicles, etc. In this case, for example, the confidence level calculation unit 13 may calculate the confidence level by comparing the recognition results acquired from other surrounding vehicles, etc. with the recognition results of the recognition unit 11. For example, the confidence level calculation unit 13 may calculate the confidence level based on the effectiveness (characteristics) of the external sensor 2 used for recognition.

[0020] The starting point setting unit 14 sets a target object that is a starting point of the control of the automatic driving based on a route plan of the automatic driving of the vehicle V. The target object that is a starting point of the control of the automatic driving is a target object that should be noted in order to execute the automatic driving. For example, the starting point setting unit 14 may set the left and right dividing lines of the vehicle V's own lane as targets that are starting points of the control in order to drive along the center of the lane in the vehicle V's own lane. For example, the starting point setting unit 14 may set the preceding vehicle as a target object that is a starting point of the control in order to maintain an appropriate inter-vehicle distance from the preceding vehicle while driving in the vehicle V's own lane. For example, there may be a case where another vehicle enters the vehicle's own lane from an adjacent lane. In this case, the starting point setting unit 14 may set the other vehicle that enters the vehicle's own lane as a target object that is a starting point of the control in order to maintain an appropriate inter-vehicle distance from the other vehicle even after the other vehicle enters the vehicle's own lane.

[0021] For example, there is a case where the host vehicle V changes lanes from the host vehicle lane to an adjacent lane. In this case, the start point setting unit 14 may set the left and right dividing lines of the adjacent lane as landmarks that are the start points of control. Furthermore, in this case, the start point setting unit 14 may set another vehicle (a vehicle in the adjacent lane) traveling within a predetermined distance in front or behind a position directly to the side of the host vehicle V in the adjacent lane as a landmark that is the start point of control.

[0022] For example, the lane on which the vehicle V is traveling may be a road on which a pedestrian or a bicycle may jump out into the lane. In this case, the starting point setting unit 14 may set a parked vehicle or a wall that is large enough to hide a pedestrian or a bicycle as a target that serves as a starting point for control. This parked vehicle or wall may be a parked vehicle or a wall that is within a predetermined lateral distance from the travel route of the route plan created up to the previous time. Note that a route for a travel distance within a predetermined time may be used as this travel route.

[0023] The notification unit 15 notifies a monitor of the vehicle V of a recognized target having a confidence level calculated by the confidence level calculation unit 13 equal to or lower than a reference value among the recognized targets recognized by the recognition unit 11. The notification unit 15 notifies the recognized target using the HMI 3. Notifying the recognized target may include notifying the position of the recognized target so that the monitor can grasp it. Notifying the recognized target may include notifying the shape and / or type of the recognized target so that the monitor can grasp it.

[0024] The notification unit 15 notifies a recognized target whose confidence level is equal to or lower than a reference value in a manner according to the value of the confidence level. For example, the notification unit 15 may change the color or brightness according to the value of the confidence level. This allows the observer to know the level of confidence level based on the manner of notification of the recognized target whose confidence level has decreased.

[0025] In addition, when reporting a recognized target whose confidence level is equal to or lower than a reference value, the notification unit 15 may display the recognized target in a different manner from the recognized target in a portion that has been recognized by the recognition unit 11 and the portion that has not been recognized. Specifically, for example, the notification unit 15 may display the recognized target whose confidence level is equal to or lower than a reference value together with detection information such as a camera image. This detection information may be point cloud information detected by a lidar. That is, the notification unit 15 displays an image of the recognized target whose confidence level is equal to or lower than a reference value on the detection information such as a camera image. In this case, the notification unit 15 displays the portion of the recognized target whose confidence level is equal to or lower than a reference value in a different manner from the other portions. For example, the notification unit 15 displays the portion of the recognized target whose confidence level is equal to or lower than a reference value, the portion of the recognized target whose confidence level exceeds the reference value, and the other portions in a different manner. This allows the observer to understand how the target whose confidence level has decreased is recognized by the recognition unit 11 by looking at the display of the HMI 3. For example, when a dividing line is displayed, the observer can distinguish and recognize parts of the dividing line with low confidence, parts with high confidence, and parts that have been detected by a camera or the like but not recognized by the recognition unit 11.

[0026] The notification unit 15 can also perform notification based on a target object that is set by the start point setting unit 14 as a starting point for control of the autonomous driving. Here, the notification unit 15 may notify a recognized target object that is set by the start point setting unit 14 as a starting point for control and has a confidence level equal to or lower than a reference value. This allows the monitor to grasp the recognition state of the recognized target object that is the starting point for control in the autonomous driving among the recognized targets.

[0027] In this way, the confidence level calculation unit 13, the starting point setting unit 14, and the notification unit 15 function as a recognition state notification device 5 that notifies the recognition state of the recognized target by the recognition unit 11. In this embodiment, the recognition state notification device 5 is provided integrally with the automatic driving system 1, as an example. However, the recognition state notification device 5 may be provided separately from the automatic driving system 1 that performs automatic driving. In addition, for example, the recognition state notification device 5 can also notify a monitor other than the occupants of the vehicle V. In this case, for example, the recognition state notification device 5 may notify a monitor (for example, a remote operator of the vehicle V) in an information center that manages the traveling of multiple vehicles.

[0028] The recognition unit 11 of the autonomous driving system 1 may calculate the confidence level of recognition of the recognized object. In this case, the confidence level calculation unit 13 may be configured by the recognition unit 11 that calculates the confidence level. Even if the recognition unit 11 calculates the confidence level of recognition of the recognized object, the confidence level calculation unit 13 may be provided separately from the recognition unit 11. In other words, the confidence level calculation unit 13 may calculate the confidence level of the recognized object exclusively for making a notification.

[0029] Next, a flow of a process of notifying the degree of confidence of a recognized target, which is performed by the recognition state notifying device 5, will be described. The process shown in Fig. 2 is performed, for example, while the vehicle V is in autonomous driving mode. However, this process may be performed at a timing other than when the vehicle V is in autonomous driving mode. In addition, when the process shown in Fig. 2 reaches an end, the process is restarted from the start after a predetermined time.

[0030] 2, in S101, the confidence level calculation unit 13 calculates the confidence level of recognition for the recognized target recognized by the recognition unit 11. In S102, the notification unit 15 notifies the monitor of the vehicle V of the recognized target whose confidence level is equal to or lower than a reference value among the recognized targets whose confidence levels are calculated by the confidence level calculation unit 13. Here, the notification unit 15 can notify the recognized target in the various modes described above. Also, here, the notification unit 15 can notify the recognized target whose confidence level is equal to or lower than a reference value for the target that is the starting point of the control of the autonomous driving set by the starting point setting unit 14 as described above.

[0031] As described above, the recognition state notification device 5 in the automatic driving system 1 allows the monitor of the vehicle V to appropriately grasp the degree of confidence in the automatic driving system's recognition.

[0032] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments. At least a portion of the above-described embodiments and various modified examples may be combined in any desired manner. [Explanation of symbols]

[0033] 1...automatic driving system, 5...recognition state notification device, 11...recognition unit, 13...confidence level calculation unit, 14...start point setting unit, 15...notification unit, V...own vehicle (vehicle).

Claims

1. A recognition state notification device in an autonomous driving system that performs autonomous driving of a vehicle using a recognition result of a recognition unit that recognizes surrounding targets based on a detection result of an external sensor, a confidence level calculation unit that calculates a confidence level of recognition of a recognition target that is the target recognized by the recognition unit; a notification unit that notifies a person monitoring the vehicle of the recognized object whose confidence level is equal to or lower than a reference value.

2. The recognition state informing device according to claim 1 , wherein the informing unit informs the recognized target of the degree of confidence being equal to or lower than a reference value in a manner corresponding to the value of the degree of confidence.

3. 2. The recognition state notification device according to claim 1, wherein when notifying the recognized target of which the confidence level is below a reference value, the notification unit displays, of the recognized target, portions that have been recognized by the recognition unit and portions that have not been recognized by the recognition unit in different manners.

4. A starting point setting unit sets a target that is a starting point of control of the autonomous driving based on a route plan of the autonomous driving in the vehicle, The recognition state notification device according to any one of claims 1 to 3, wherein the notification unit notifies the recognized target that is set by the starting point setting unit as the target that is the starting point of control and whose confidence level is equal to or lower than a reference value.

Images