Systems, control circuitry, methods, devices, and storage media for operating a vehicle

Abstract

The present application relates to systems, control circuitry, methods, apparatuses, and storage media for operating a vehicle. A system for operating a vehicle includes an object detector and an operator monitor, and a controller. The object detector is to detect one or more targets proximate to a host vehicle, which can be operated in an autonomous mode and a manual mode. The operator monitor is to detect a gaze direction of an operator of the host vehicle. The controller circuitry is in communication with the object detector and the operator monitor. The controller circuitry is configured to determine a classification of each target detected by the object detector. The classification includes a primary target and an ignore target. The controller circuitry is further configured to determine that a switch of operation of the host vehicle from the autonomous mode to the manual mode is recommended, perform a verification that the operator has gazed at each primary target more recently than a primary time, and perform the switch in response to the verification.

Description

[0001] This application is a divisional application of the invention patent application filed on June 26, 2019, with application number 201910559193.7 and invention title "Verification of vehicle operator awareness before switching from autonomous mode to manual mode". Technical Field

[0002] This disclosure generally relates to systems for operating vehicles, and more specifically to systems that verify that the operator has been looking at each key objective more recently than the main time before performing the switch of vehicle control from autonomous mode to manual mode. Background Technology

[0003] In the field of autonomous vehicles, vehicles can operate in fully automatic mode, in which case the human operator only needs to specify the destination to operate the vehicle. However, in some cases, full automation is undesirable; instead, it is necessary, for example, to automate the vehicle's speed control while steering control is performed by a human operator. Summary of the Invention

[0004] In one aspect, this disclosure provides a system for operating a vehicle, the system comprising: an object detector for detecting one or more targets near a master vehicle, the master vehicle being operable in an autonomous mode and a manual mode; an operator monitor for detecting the gaze direction of the operator of the master vehicle; and controller circuitry communicating with the object detector and the operator monitor, the controller circuitry being configured to determine a classification of each target detected by the object detector, the classification including primary targets and ignored targets, determine that a switch of operation of the master vehicle from the autonomous mode to the manual mode is recommended, perform verification that the operator has gazed at each primary target more recently than the primary time, and perform the switch in response to the verification.

[0005] In another aspect, this disclosure provides a controller circuit for operating a vehicle, the controller circuit comprising: a first input configured to communicate with an object detector for detecting one or more targets near a master vehicle, the master vehicle being operable in autonomous and manual modes; a second input configured to communicate with an operator monitor for detecting the gaze direction of an operator of the master vehicle; and a processor communicating with the object detector and the operator monitor, the processor being configured to determine a classification of each target detected by the object detector, the classification including primary targets and ignored targets, determine that a switch of operation of the master vehicle from the autonomous mode to the manual mode is recommended, perform verification that the operator has gazed at each primary target more recently than the primary time, and perform the switch in response to the verification.

[0006] In another aspect, this disclosure provides a method for operating a vehicle, the method comprising: detecting one or more targets near a master vehicle using an object detector, the master vehicle being operable in an autonomous mode and a manual mode; detecting the gaze direction of the operator of the master vehicle using an operator monitor; determining a classification for each target detected by the object detector, the classification including primary targets and ignored targets; determining that a switch of operation of the master vehicle from the autonomous mode to the manual mode is recommended; performing verification that the operator has gazed at each primary target more recently than the primary time; and performing the switch in response to the verification.

[0007] In another aspect, this disclosure provides a first device comprising: one or more processors; a memory; and one or more programs stored in the memory, said one or more programs including instructions for performing the methods described above.

[0008] In another aspect, this disclosure provides a non-transitory computer-readable storage medium comprising one or more programs for execution by one or more processors of a first device, the one or more programs comprising instructions that, when executed by the one or more processors, cause the first device to perform the method described above. Attached Figure Description

[0009] The invention will now be described by way of example with reference to the accompanying drawings, in which:

[0010] Figure 1 This is a diagram of a system for operating a vehicle according to one embodiment;

[0011] Figure 2 According to one embodiment Figure 1 The scenarios encountered by the system; and

[0012] Figure 3 This is an operation according to one embodiment. Figure 1 The systematic approach. Detailed Implementation

[0013] Reference will now be made in detail to embodiments, examples of which are shown in the accompanying drawings. In the following detailed description,

[0014] Many specific details are elaborated to provide a thorough understanding of the various described embodiments. However,

[0015] It will be apparent to those skilled in the art that the various described embodiments can be practiced without these specific details. In other instances, well-known methods, processes, components, circuits, and networks have not been described in detail to avoid unnecessarily obscuring aspects of the embodiments.

[0016] "One or more" includes functions performed by one element, functions performed by more than one element (e.g., in a distributed manner), functions performed by one element, functions performed by several elements, or any combination of the above.

[0017] It should also be understood that, although in some instances, the terms "first" and "second" are used herein.

[0018] Terms such as "+" are used to describe various elements, but these elements should not be limited by these terms. These terms are only used to distinguish elements from one another. For example, without departing from the scope of the various described embodiments, a first contact may be referred to as a second contact, and similarly, a second contact may be referred to as a first contact. Both a first contact and a second contact are contacts, but they are not the same contact.

[0019] 5. The terminology used in the description of the various embodiments described herein is for the purpose of describing the embodiments only.

[0020] This is not intended to be restrictive. As used in the description of the various described embodiments and in the appended claims, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to also include the plural forms. It should also be understood that, as herein…

[0021] The term “and / or” as used refers to and includes all possible combinations of one or more of the associated listed items. It should also be understood that, when used in this specification, the terms “includes,” “including,” “comprises,” and / or “comprising” specify the presence of the stated features, wholes, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or groups thereof.

[0022] 5. As used herein, the term "if" may optionally be interpreted, depending on the context, as indicating

[0023] "When..." or "at..." or "in response to determining..." or "in response to detecting..."

[0024] Similarly, the phrases "if it is determined that..." or "if [the condition or event stated] is detected"

[0025] Depending on the context, it can be optionally interpreted as meaning "when determining..." or "in response to determining..."

[0026] Either "when [the stated condition or event] is detected" or "in response to the detection of [the stated condition or event]".

[0027] Figure 1 A non-limiting example of a system 10 for operating a vehicle (e.g., master vehicle 12) is shown. Master vehicle 12 can be characterized as an autonomous vehicle. As used herein, the term autonomous vehicle is used to characterize a vehicle that can be operated in autonomous mode 14 (i.e., fully automatic mode), where a human operator of master vehicle 12 (hereinafter referred to as operator 18) can simply specify a destination to operate master vehicle 12. However, full automation is not required all the time. For example, autonomous mode 14 may include a situation where only the speed control of master vehicle 12 is automated, but the steering is controlled by operator 18. As will be explained in more detail below, some of the unique advantages of system 10 described herein are found when master vehicle 12 switches from autonomous mode 14 to manual mode 16, where the degree or level of automation can simply be to provide audible or visual warnings to the human operator who normally controls the steering, accelerator, and brakes of master vehicle 12.

[0028] System 10 includes an object detector 20, which may contain or include one or more instances of a camera, radar, lidar, ultrasonic transducer, or any combination thereof. One function of the object detector 20 is to detect one or more targets 22 (e.g., other vehicles, pedestrians, traffic signals) approaching the host vehicle 12 (e.g., within 100 meters). Methods and algorithms for processing information from the devices forming the object detector 20 to detect targets 22 are well known to those skilled in the art of autonomous vehicles. While various devices forming the object detector 20 are shown as part of the host vehicle 12 (e.g., mounted on the host vehicle 12), this is not a requirement. It should be contemplated that external devices such as traffic observation cameras mounted on poles or traffic signals may provide information or images to the host vehicle 12. As previously mentioned, the host vehicle 12 can operate in both autonomous and manual modes.

[0029] System 10 includes an operator monitor 24 for detecting the gaze direction 26 of operator 18 of host vehicle 12, among other possible uses and functions. Operator monitor 24 may consist of or be formed from one or more instances of an infrared (IR) and / or visible light internal camera, radar, and / or ultrasonic transducer. As used herein, gaze direction 26 is characterized as the location inside or outside host vehicle 12 that operator 18 is looking at. For example, looking forward at another vehicle in front of host vehicle 12, looking in the rearview or side mirrors of host vehicle 12 to observe another vehicle behind or beside host vehicle 12, or making necessary head or eye movements to check for so-called blind spots beside and partially behind host vehicle 12. Methods and algorithms for processing information from the devices forming operator monitor 24 to detect gaze direction 26 toward something inside or outside host vehicle 12 are well known to those skilled in the art of driver monitoring.

[0030] System 10 includes controller circuitry 30, which communicates with object detector 20 via a first input 36, with operator monitor 24 via a second input 38, and with vehicle control 28, which allows for automatic and manual operation of steering, braking, and accelerator (but not limited to) of the master vehicle 12. As will be appreciated by those skilled in the art, communication may be via wires, fiber optics, or wireless communication. As will be apparent to those skilled in the art, controller circuitry 30 (hereinafter sometimes referred to as controller 30) may include one or more instances of processor 32, such as a microprocessor or other control circuitry (e.g., analog and / or digital control circuitry including application-specific integrated circuits (ASICs) for processing data). Although system 10 described herein is generally described with respect to a single instance of controller 30, it should be appreciated that the functionality of controller 30 may be shared or distributed among several instances of the controller, each instance of the controller being configured for certain specific tasks. Hereafter, any reference to controller 30 configured for something is also interpreted as implying that processor 32 may also be configured for the same thing. It should also be recognized that multiple instances of the processor may exist within any instance of controller 30. Controller 30 may include memory 34 (i.e., a non-transitory computer-readable storage medium), including non-volatile memory, such as electrically erasable programmable read-only memory (EEPROM) for storing one or more routines, thresholds, and captured data. Memory 34 may be part of processor 32, part of controller 30, or separate from controller 30 (e.g., remote memory stored in the cloud). As described herein, one or more routines may be executed by controller 30 or processor 32 to perform steps for determining the relative positions of various instances of target 22 and the gaze direction 26 of operator 18 based on signals received by controller 30.

[0031] Figure 2 A non-limiting example of scenario 38 that the master vehicle 12 may encounter is shown. This situation has been observed when an autonomous vehicle such as the master vehicle 12 is operating in autonomous mode 14, and when the operation of the master vehicle 12 should switch from autonomous mode 14 to manual mode. Examples of this situation include, but are not limited to: malfunction or damage to one or more of the devices forming the object detector 20, or approaching a construction area or accident scene when the available driving path of the master vehicle 12 does not correspond to the configuration of the road indicated in the digital map. When the system 10 detects a recommended switch from autonomous mode 14 to manual mode 16 (hereinafter sometimes referred to as switch 40), Figure 1In the event of a situation described herein, system 10 verifies that operator 18 is aware of the existence of target 22 before completing switchover 40. As used herein, switchover 40 is characterized as being initiated by system 10 and should not be confused with takeover control performed by operator 18 (where operator 18 initiates a transition from autonomous mode 14 to manual mode 16).

[0032] To this end, controller 30 is configured to determine a classification 42 for each target 22 detected by object detector 20. Options for classification 42 include, but are not limited to, primary target 44 and ignored target 46. By way of example, and not limitation, an instance of target 22 classified as primary target 44 could be a traffic signal 48. Figure 2 ), a stop sign (not shown), or a pedestrian 50, or another vehicle 52 moving on or toward or across the travel path 54 of the main vehicle 12 and / or within seventy-five meters (75m) of the main vehicle 12. That is, the instances of the classification 42 of the primary target 44 assigned to target 22 are these instances of target 22: something that the system 10 (i.e., the controller 30 or processor 32) has already considered to be something that the operator 18 should be aware of (i.e., see) before completing the switch 40 from autonomous mode 14 to manual mode 16.

[0033] In contrast, instances of target 22 that are classified as ignored target 46 are those instances of target 22 that system 10 (i.e., controller 30 or processor 32) has determined to be something that operator 18 does not need to see before completing the switch 40 from autonomous mode 14 to manual mode 16. By way of example, rather than limitation, instances of target 22 classified as ignored target 46 could be oncoming vehicles 56 traveling in the opposite direction to the main vehicle 12 and already behind or past the main vehicle 12, or stationary objects outside the travel path of the main vehicle 12 (e.g., a trash container 58 on the shoulder). One possible general definition of which of target 22 should be classified as ignored target 46 is those instances of target 22 that are not moving and are not in the travel path or road of the main vehicle.

[0034] When the master vehicle 12 is operating in autonomous mode 14, it can continuously or periodically track each instance of target 22 and determine the classification 42 of each instance of target 22. When system 10 determines that it is recommended to switch the operation of the master vehicle from autonomous mode 40 to manual mode, system 10 (i.e., controller 30 or processor 32) is configured to perform a verification 60 that the operator 18 has looked at each instance of target 22 of the classification 42 assigned to the master target 44 more recently than the primary time 62 (e.g., three seconds (3s)). That is, verification 60 is to check that the operator 18's gaze direction 26 has been directed at each of the master targets, and to check that the time 64 since looking at each of the master targets is less than the primary time 62. In response to the affirmative result of verification 60 that each of the master targets has been looked at or gazed at more recently than the primary time 62, controller 30 performs the switch 40.

[0035] It is expected that there will be instances where verification 60 yields a negative result, i.e., operator 18 does not look at one or more of the main targets for a longer period than the main time 62. Figure 2 A passing vehicle 66 is shown that is currently in the operator 18's so-called blind spot and is traveling faster than the primary vehicle 12. The fact that the passing vehicle 66 is traveling faster than the primary vehicle 12 and / or that the passing vehicle 66 has recently accelerated to a speed faster than the primary vehicle 12 can serve as an indication that the passing vehicle 66 is attempting to change lanes into the primary vehicle 12's travel path 54. Even if the passing vehicle 66 is traveling at the same speed as the primary vehicle 12, the operator 18 needs to be aware of the presence of the passing vehicle 66 before taking manual control of the primary vehicle 12 to avoid a sudden lane change towards the passing vehicle 66. Therefore, the controller 30 can be configured to classify the passing vehicle as a primary target 44, so the presence of the passing vehicle 66 is likely something that the operator 18 should be aware of before completing the switch 40. However, in this non-limiting example, the operator 18 does not show a gaze direction 26 towards the passing vehicle 66 for a longer period than the primary time 62.

[0036] In response to this expectation, system 10 may include an alarm device 68 perceptible to operator 18, which communicates with controller circuitry 30 via output 70. Alarm device 68 may include, or include, a reconfigurable display or head-up display on a dashboard, a speaker outputting a voice notification such as “Perform a visual scan,” an arrangement of lights located in a position to attract operator 18’s attention, or any combination thereof. The alarm device is configured to be operable by controller 30 to indicate to operator 18 the direction of an instance that operator 18 needs to see, for example, toward a primary target 44. The information conveyed to operator 18 by alarm device 68 may include, but is not limited to, the specific direction and distance to the target, direction only, which part of the 360-degree radius, for example, the quadrant located at one or more primary targets that have not been viewed more recently than the primary target (e.g., left front, right rear, etc.).

[0037] It should be considered that there might be instances where objective 22 is less important than the primary objective but more important than ignoring the objective. For example, Figure 2 A following vehicle 72 is shown traveling at essentially the same speed as the primary vehicle 12. While operator 18 should not ignore the following vehicle 72, operator 18 may not need to observe the following vehicle as frequently as instances of the primary vehicle 12 that are in front of or adjacent to it. Therefore, classification 42 may include secondary targets 74. By further exemplary differentiation, instances of primary targets 44 may be another vehicle moving close to the primary vehicle 12 or a pedestrian 50 moving in or toward the primary vehicle 12's path of travel 54, and instances of secondary targets 74 may be another vehicle that is not moving close to the primary vehicle (i.e., moving at the same speed and direction, or moving away from the primary vehicle 12). Since secondary targets 74 may not need to be viewed or observed as frequently as instances of primary targets 44, verification 60 may include or be configured to verify that operator 18 has observed each instance of a secondary target more recently than a secondary time 76 (e.g., 5 seconds), where the secondary time 76 is longer than the primary time 62.

[0038] It should be envisioned that the classification 42 of some instances of target 22 may require more information than is available from object detector 20, and / or the line of sight from object detector 20 to an object may be obstructed or otherwise impeded. Therefore, system 10 may include a digital map 78 indicating the location of instances of primary target 44 and / or secondary target 74 near the main vehicle 12. By way of example, and not limitation, instances of target 22 depicted in digital map 78 may include traffic signals 48 or stop signs (not shown), and / or boundaries defining inaccessible areas such as cliffs.

[0039] A situation may arise when it is difficult to determine the operator 18's gaze direction 26 (for example, because the operator is wearing sunglasses), and / or when the main targets 44 are too close to each other to distinguish instances of changes in gaze direction 26 required to view each instance of the main targets. Therefore, as an addition to or alternative to monitoring gaze direction 26, verification 60 may include that the operator 18 has performed a visual scan 80 around the main vehicle 12 more recently than scan time 82. If gaze direction 26 cannot be detected or distinguished, the movement of the operator 18's head may be tracked to determine that the operator 18 has performed visual scan 80. Requirements for completing visual scan 80 may include looking in the rearview and side mirrors, a general scan of the area in front of the main vehicle, and sufficient head movement to indicate that blind spots have been checked.

[0040] Figure 3 A non-limiting example of a method 100 for operating a vehicle (e.g., main vehicle 12) is shown.

[0041] Step 105 (operating the master vehicle in autonomous mode) may include operator 18 specifying or entering a destination, and controller 30 controlling the master vehicle 12 by operating vehicle control 28 to “drive” the master vehicle 12 to the destination in autonomous mode 14. The master vehicle 12 can also be operated in manual mode 16, where operator 18 manually controls one or more of the steering, braking, and accelerator of vehicle control 28. As will become apparent in the following description, one aspect of method 100 is verifying that operator 18 is aware of a relevant object approaching the master vehicle 12 before completing the transition from autonomous mode 14 to manual mode 16.

[0042] Step 110 (detecting (multiple) targets) may include using object detector 20 to detect one or more targets 22 approaching the host vehicle 12. Controller 30 may be configured or programmed to process one or more signals from object detector 20 to detect and / or track one or more targets 22.

[0043] Step 115 (accessing the digital map) may include accessing a digital map 78 that indicates one or more instances of the primary target 44 and / or secondary targets 74 near the primary vehicle 12. The digital map 78 may be stored in a memory 34 on the primary vehicle 12, or it may be stored in the cloud and remotely accessed from the primary vehicle 12.

[0044] Step 120 (detecting (multiple) gaze directions) may include detecting the gaze direction 26 of the operator 18 of the host vehicle 12 using the operator monitor 24. The gaze direction 26 may be characterized by the azimuth angle relative to the longitudinal axis of the host vehicle and the elevation angle relative to the longitudinal axis or the horizontal angle relative to gravity.

[0045] Step 125 (classifying the targets) may include determining a classification 42 for each target 22 detected by object detector 20. The value or type of classification 42 may include, but is not limited to, primary target 44, secondary target 74, and ignored target 46. Classification 42 may be based on, but is not limited to, the relative position of the target (e.g., in front of, beside, or behind the main vehicle 12, within, near, or outside the main vehicle's travel path 54) and / or relative movement (e.g., toward or away from the main vehicle 12). Classification 42 may also take into account the identification of target 22 (e.g., traffic signal 48, pedestrian 50, other vehicles 52, or stationary objects such as trash can 58).

[0046] Step 130 (Recommended Switch?) may include determining that the operation of the master vehicle 12 is recommended to be switched from autonomous mode 14 to manual mode 16 because one or more of the devices forming the object detector 20 have malfunctioned or been damaged.

[0047] Step 135 (Gazing at (multiple) primary targets < primary time?) may include determining that operator 18 has gazed at each instance of primary target 44 more recently than primary time 62. That is, less than primary time 62 has elapsed since operator 18 looked at or gazed at each instance of target 22 that is classified as primary target 44.

[0048] Step 140 (focusing on (multiple) secondary targets < secondary time?) is an optional step that may include determining that operator 18 has focused on each secondary target 74 more recently than secondary time 76, where secondary time 76 is longer than primary time 62.

[0049] Step 145 (Visual scan < scan time?) is an optional step that may include determining that operator 18 has performed a visual scan 80 around the main vehicle 12 more recently than scan time 82.

[0050] Step 150 (Operator Awareness Verification?) may include verification 60, which confirms that the tests performed by steps 135, 140, and 145 have been passed or at least that these steps have been performed. If verification 60 is confirmed, the method proceeds to step 160 to perform switching 40. Otherwise, if one or more of steps 135, 140, and 145 fail, method 100 proceeds to step 155 to activate alarm device 68. In addition to one or more of steps 135, 140, and 145, verification 60 may include confirming that operator 18 has grasped or touched the steering wheel controlling the steering of the master vehicle and that the operator's feet are properly positioned to operate the brakes and accelerator of the master vehicle 12.

[0051] Step 155 (operating the alarm device) may include operating the alarm device to instruct operator 18 to: move toward one or more primary targets 44 that were not viewed more recently than primary time 62, and / or move toward one or more secondary targets 74 that were not viewed more recently than secondary time 76, and / or perform a visual scan 80 that was not performed more recently than scan time 82.

[0052] Step 160 (Perform a switch?) may include performing a switch 40 in response to verification by stopping the automatic control of vehicle control 28 and / or providing some indication that manual mode 16 is being used. This indication may be a voice message emitted from a speaker (e.g., “Use manual driving” or “Disengage automatic driving”) and / or an indicator illuminated on the dashboard of the primary vehicle 12.

[0053] Step 165 (Safely stop the master vehicle) may include maintaining operation in autonomous mode 14 until the controller 30 can steer the master vehicle 12 to the shoulder and stop, or exit the exit ramp and find a suitable parking position. This step may be taken if the operator 18 is unable, for some reason, to satisfactorily visually scan the area around the master vehicle 12 in preparation for taking manual control of the master vehicle 12.

[0054] Step 170 (operating the master vehicle in manual mode) may include operator 18 continuing to operate vehicle control 28 until it is possible to resume operating the master vehicle in autonomous mode 14.

[0055] This document describes a first device 30, which includes one or more processors 32; a memory 34; and one or more programs 105-170 stored in the memory 34. The one or more programs 105-170 include instructions for performing all or part of method 100. Additionally, this document describes a non-transitory computer-readable storage medium 34, which includes one or more programs 105-170 for execution by one or more processors 32 of the first device 30, the one or more programs 105-170 including instructions that, when executed by one or more processors 32, cause the first device 30 to perform all or part of method 100.

[0056] Therefore, a system 10, a controller 30 for the system 10, and a method 100 for the operating system 10 are provided. The system 10 and the method 100 are used to determine when the operator is ready to take manual control of the master vehicle 12 when it is recommended to switch the operation from autonomous mode 14 to manual mode 16 for various reasons.

[0057] While the invention has been described with respect to preferred embodiments thereof, it is not intended to limit the invention thereto, but rather to limit it only to the extent set forth in the appended claims.

Claims

1. A system for operating a vehicle, the system comprising: An object detector for detecting one or more targets approaching the vehicle, which is capable of operating in autonomous and manual modes; An operator monitor is used to detect the direction of the operator's gaze in the vehicle. as well as A controller circuit, which communicates with the object detector and the operator monitor, is configured to: Determine the classification of each target detected by the object detector, the classification options including primary targets and non-primary targets; The recommended operation for the vehicle is to switch from the autonomous mode to the manual mode. Verification is performed that the operator has observed at least one primary target within a first threshold time period, and The switching is performed in response to the verification.

2. The system of claim 1, wherein the system includes an alarm device operable to indicate to the operator a direction toward one or more primary targets that the operator has not yet looked at within the first threshold time period.

3. The system of claim 1, wherein the classification option further includes secondary targets, and the verification includes the operator having observed each secondary target for a second threshold time duration, the second threshold time duration being longer than the first threshold time duration.

4. The system according to any one of claims 1-3, wherein the system includes a digital map indicating the location of the at least one primary target near the vehicle.

5. The system according to any one of claims 1-3, wherein the verification includes the operator having performed a visual scan around the vehicle more recently than the scan time.

6. A controller circuit for operating a vehicle, the controller circuit comprising: A first input terminal is configured to communicate with an object detector for detecting one or more targets approaching the vehicle, which is capable of operating in autonomous and manual modes. The second input is configured to communicate with an operator monitor used to detect the gaze direction of the operator of the vehicle; as well as A processor, which communicates with the object detector and the operator monitor, is configured to: Determine the classification of each target detected by the object detector, the classification options including primary targets and non-primary targets; The recommended operation for the vehicle is to switch from the autonomous mode to the manual mode. Verification is performed to confirm that the operator has been looking at at least one primary target within a first threshold time period, and The switching is performed in response to the verification.

7. The controller circuit according to claim 6, wherein the controller circuit comprises: The output is configured to communicate with an alarm device that is operable to indicate to the operator a direction toward one or more primary targets that the operator has not yet looked at within the first threshold time period.

8. The controller circuit of claim 6, wherein the classification option further includes secondary targets, and the verification includes the operator having observed each secondary target for a second threshold time duration, the second threshold time duration being longer than the first threshold time duration.

9. The controller circuit according to any one of claims 6-8, wherein the controller circuit includes a digital map indicating the location of the at least one primary target near the vehicle.

10. The controller circuit according to any one of claims 6-8, wherein the verification includes the operator having performed a visual scan around the vehicle more recently than the scan time.

11. A method for operating a vehicle, the method comprising: The vehicle can detect one or more targets approaching it using an object detector, and the vehicle can operate in both autonomous and manual modes. The operator's gaze direction is detected using an operator monitor; Determine the classification of each target detected by the object detector, the classification options including primary targets and non-primary targets; The recommended operation for the vehicle is to switch from the autonomous mode to the manual mode. Verification is performed to confirm that the operator has been looking at at least one primary target within a first threshold time period, and The switching is performed in response to the verification.

12. The method of claim 11, wherein the method comprises: The alarm device is activated to instruct the operator to move toward one or more primary targets that have not yet been observed within the first threshold time period.

13. The method of claim 11, wherein the classification option further includes secondary targets, and the verification includes the operator having observed each secondary target for a second threshold time duration, the second threshold time duration being longer than the first threshold time duration.

14. The method according to any one of claims 11-13, wherein the method comprises: Access a digital map indicating the location of the at least one primary target near the vehicle.

15. The method according to any one of claims 11-13, wherein the method comprises: The verification confirmed that the operator had performed a visual scan around the vehicle more recently than the scan time.

16. A first device, comprising: One or more processors; Memory; as well as One or more programs stored in memory, the one or more programs including instructions for performing the method according to any one of claims 11 to 15.

17. A non-transitory computer-readable storage medium comprising one or more programs for execution by one or more processors of a first device, the one or more programs comprising instructions that, when executed by the one or more processors, cause the first device to perform the method according to any one of claims 11 to 15.

18. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 11 to 15.

Images