Adaptive Lighting Systems for Autonomous Vehicles

Abstract

The system (100) includes a headlamp (110) mounted on the autonomous vehicle (502). The headlamp (110) is configured to illuminate at least a portion of a road (102) on which the autonomous vehicle (502) is located. The system (100) further includes a control device (550) associated with the autonomous vehicle (502). The processor (122, 570) obtains information (310) about an environment surrounding the autonomous vehicle (502). The processor (122, 570) determines that at least a portion of the road (102) should be illuminated if the information (310) indicates that an illumination level (340) of the portion of the road (102) is lower than a threshold illumination level (342). In response to determining that at least a portion of the road (102) should be illuminated, the processor (122, 570) adjusts the headlamp (110) to illuminate at least a portion of the road (102).

Description

[Technical Field]

[0001] (Related Applications and Priority Claims) This application claims priority to U.S. Provisional Application No. 63 / 266,547, filed January 7, 2022, and entitled "AN ADAPTIVE ILLUMINATION SYSTEM FOR AN AUTONOMOUS VEHICLE," which is incorporated herein by reference.

[0002] (Technical field) The present disclosure relates generally to autonomous vehicles, and more particularly to adaptive lighting systems for autonomous vehicles. [Background technology]

[0003] One aim of autonomous vehicle technology is to provide a vehicle that can safely navigate toward a destination. Sometimes, an autonomous vehicle may need to travel at night or in a dark area. Navigating an autonomous vehicle while traveling at night or in a dark area can be difficult. For example, detecting objects, road markings, and road signs in a night or dark area can be difficult due to certain sensors in the autonomous vehicle. Summary of the Invention [Means for solving the problem]

[0004] The present disclosure recognizes various problems and heretofore unmet needs associated with implementing safe navigation for autonomous vehicles in the following situations: the autonomous vehicle is traveling at night or the environment around the autonomous vehicle is otherwise dark; the autonomous vehicle is traveling toward a road corner (e.g., a left or right turn) where at least one blind spot is on the road and the sensor data captured by the autonomous vehicle's sensors does not provide sufficient information about the blind spot; or the autonomous vehicle is traveling through a bridge or tunnel. Certain embodiments of the present disclosure implement an adaptive lighting system for illuminating the traveling path of an autonomous vehicle by providing unique technical solutions to the technical problems of current autonomous vehicle technology, including those problems described above.

[0005] The present disclosure contemplates systems and methods for implementing an adaptive lighting system for an autonomous vehicle. For example, the lighting system may include one or more headlights associated with the autonomous vehicle. Each headlight may be adapted to turn on and off a low-beam light source, a high-beam light source, a left-side light source, and a right-side light source. Each light source of the headlight may be an optical light source (e.g., a visible light source) or an infrared light source (e.g., an invisible light source). The headlight may be further adapted to adjust the intensity of the light source. For example, the disclosed system may adjust the intensity of the light source from 0% to 100% depending on the particular situation.

[0006] The disclosed system may adjust the headlights depending on the situation the autonomous vehicle encounters. For example, if oncoming traffic is detected on the opposite side of the road from the autonomous vehicle, the disclosed system may turn on a low-beam light source and / or an infrared light source. This may avoid blinding the driver in oncoming traffic. In another example, if oncoming traffic is not detected, the disclosed system may turn on a high-beam light source and / or an infrared light source. In another example, if it is determined that the autonomous vehicle is approaching a tunnel, the disclosed system may turn on a low-beam or high-beam light source and / or an infrared light source in accordance with traffic rule policies (e.g., depending on oncoming traffic inside the tunnel). In another example, if it is determined that the autonomous vehicle is approaching a left turn location, the disclosed system may turn on a left-side light source and / or a left-side infrared light source to provide better lighting conditions for the curved portion of the left turn location. In another example, the disclosed system may perform a similar operation for a right turn location. If it is determined that the autonomous vehicle is approaching a right turn location, the disclosed system may turn on a right-side light source and / or a right-side infrared light source to provide better lighting conditions for the curved portion of the right turn location.

[0007] In another example, if the light condition level of an image captured by a camera sensor of an autonomous vehicle is low (e.g., below a threshold light condition level) or if the image is dark, the disclosed system may increase the intensity of the light source (e.g., optical and / or infrared light source) until the light condition level of a second image (e.g., captured after increasing the intensity of the light source) is at least equal to the threshold light condition level.

[0008] In another example, if the light condition level of an infrared image captured by an infrared camera sensor of an autonomous vehicle is low (e.g., below a threshold light condition level) or if the infrared image is dark, the disclosed system may increase the intensity of the light source (e.g., optical and / or infrared light source) until the light condition level of a second infrared image (e.g., captured after increasing the intensity of the light source) is at least equal to the threshold light condition level.

[0009] In an embodiment, in any of the example situations described above, the disclosed system may adjust (e.g., increase) the intensity of a light source (e.g., optical and / or infrared light source) until the illumination level of the path of travel of the autonomous vehicle is at least equal to a threshold illumination level. The threshold illumination level may be configured to indicate a preferred illumination level (e.g., a minimum amount of illumination required) of the path of travel that is safe for the autonomous vehicle to travel.

[0010] Thus, the disclosed system can be integrated into practical applications that improve the perception and visibility of autonomous vehicle sensors. For example, by implementing an adaptive lighting system for an autonomous vehicle, the autonomous vehicle's path of travel is illuminated, which allows the autonomous vehicle's sensors to more accurately detect objects in the autonomous vehicle's path of travel. Thus, the disclosed system can determine a safer path of travel for the autonomous vehicle. Thus, the autonomous vehicle can travel more safely, and vehicles surrounding the autonomous vehicle can also travel more safely.

[0011] The disclosed system may further be integrated into additional practical applications that improve the autonomous navigation of autonomous vehicles. In other words, the disclosed system improves autonomous vehicle navigation technology. For example, the disclosed system may enable an autonomous vehicle to navigate at night with little or no human intervention. In another example, the disclosed system may enable an autonomous vehicle to navigate within a route that includes a tunnel or bridge with little or no human intervention. In another example, the disclosed system may enable an autonomous vehicle to navigate within a route that includes a non-traditional left or right turn location that would be difficult to navigate without the disclosed system's proposed solution for adjusting lighting on the left or right corner. In another example, the disclosed system may enable an autonomous vehicle to identify and maneuver around (or through) blind spots (which may arise due to insufficient sensor perception or light) by eliminating such blind spots by adjusting lighting directed toward those blind spots. In this way, the disclosed system provides a technical solution to overcome current problems that specifically arise in autonomous vehicle navigation technology.

[0012] In one embodiment, the system includes at least one headlight mounted on the autonomous vehicle. In another embodiment, the system includes at least two headlights mounted on the autonomous vehicle. The headlights may be configured to illuminate at least a portion of a road along which the autonomous vehicle is traveling. The system further includes a control device associated with the autonomous vehicle. The processor may obtain information about an environment surrounding the autonomous vehicle. The processor may determine that at least a portion of the road should be illuminated if the information indicates that an illumination level of the portion of the road is below a threshold illumination level. In response to determining that at least a portion of the road should be illuminated, the processor may adjust the at least one headlight to illuminate at least a portion of the road.

[0013] Certain embodiments of the present disclosure may include some, all, or none of these advantages. These advantages and other features will be more clearly understood from the following detailed description considered in conjunction with the accompanying drawings and claims. [Brief explanation of the drawings]

[0014] For a more complete understanding of the present disclosure, reference is now made to the following brief description taken in conjunction with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.

[0015] [Figure 1] FIG. 1 illustrates an embodiment of a system for implementing an adaptive lighting system for an autonomous vehicle.

[0016] [Figure 2A] 2A and 2B illustrate an exemplary embodiment of a headlamp for an autonomous vehicle. [Figure 2B] 2A and 2B illustrate an exemplary embodiment of a headlamp for an autonomous vehicle.

[0017] [Figure 3] FIG. 3 illustrates an example operational flow of the system of FIG. 1 for determining lighting levels for a path of travel of an autonomous vehicle.

[0018] [Figure 4] FIG. 4 illustrates an embodiment of a method for implementing an adaptive lighting system for an autonomous vehicle.

[0019] [Figure 5] FIG. 5 illustrates a block diagram of an example autonomous vehicle configured to implement autonomous driving operations.

[0020] [Figure 6] FIG. 6 illustrates an example system for providing autonomous driving operations used by the autonomous vehicle of FIG.

[0021] [Figure 7] FIG. 7 illustrates a block diagram of an onboard control computer included in the autonomous vehicle of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0022] As described above, previous techniques fail to provide an efficient, reliable, and safe navigation solution for an autonomous vehicle in situations where the autonomous vehicle is traveling at night or when the environment around the autonomous vehicle is otherwise dark; the autonomous vehicle is traveling toward a road corner (e.g., a left or right turn) where at least one blind spot exists on the road where sensor data captured by the autonomous vehicle's sensors does not provide sufficient information; or the autonomous vehicle is traveling through a bridge or tunnel. The present disclosure provides various systems, methods, and devices that implement an adaptive lighting system for illuminating the traveling path of an autonomous vehicle.

[0023] Exemplary Systems for Implementing Adaptive Lighting Systems for Autonomous Vehicles FIG. 1 illustrates an embodiment of a system 100 configured to implement adaptive lighting for an autonomous vehicle 502. FIG. 1 further illustrates a simplified schematic diagram of a road 102 traveled by the autonomous vehicle 502. In one embodiment, the system 100 may comprise the autonomous vehicle 502. The autonomous vehicle 502 may comprise a control device 550. The control device 550 may comprise a processor 122 in signal communication with a memory 126. The memory 126 may store software instructions 128 that, when executed by the processor 122, cause the control device 550 to perform one or more operations described herein. For example, when the software instructions 128 are executed, the control device 550 may determine whether at least a portion of the road 102 on which the autonomous vehicle 502 is located should be illuminated, and if it is determined that at least a portion of the road 102 in front of the autonomous vehicle 502 should be illuminated, the control device 550 may adjust the headlights 110 to illuminate at least a portion of the road. 1 illustrates autonomous vehicle 502 with a single headlamp 110 extending horizontally across the front of the vehicle, it should be understood that autonomous vehicle 502 may include any suitable number and combination of headlamp 110 shaped, positioned, and located as needed for the particular design of autonomous vehicle 502. In some embodiments, system 100 may further comprise network 112. Network 112 enables autonomous vehicle 502 to communicate with other autonomous vehicles 502, systems, servers, devices, etc. In other embodiments, system 100 may not have all of the listed components and / or may have other elements instead of or in addition to those listed above. System 100 may be configured as shown or in any other configuration.

[0024] One optional approach for determining whether a portion of a road 102 should be illuminated is to use a light sensor. The light sensor may be configured to detect the brightness of its surroundings. However, the light sensor's detection of the brightness of the environment around it may depend on the location of the light sensor (e.g., in front of the autonomous vehicle 502, behind the autonomous vehicle 502, or any other location relative to the autonomous vehicle 502) and the detection range of the light sensor (e.g., 1 foot, 2 feet, etc.). Thus, relying solely on a light sensor may not provide a comprehensive understanding of the lighting conditions surrounding the autonomous vehicle 502 and whether the autonomous vehicle's path of travel 502 is or should be illuminated to provide safer driving conditions for the autonomous vehicle 502, other vehicles surrounding the autonomous vehicle 502, and pedestrians.

[0025] Accordingly, system 100 may be configured to use the additional data to provide a more comprehensive understanding of the lighting conditions surrounding autonomous vehicle 502 and to determine lighting levels for a path of travel for autonomous vehicle 502. Accordingly, system 100 may be configured to dynamically illuminate a path of travel for autonomous vehicle 502 by adjusting headlights 110. One example scenario in which system 100 may adjust headlights 110 to dynamically illuminate a path of travel for autonomous vehicle 502 is described below.

[0026] In one example scenario, while traveling along road 102, autonomous vehicle 502 may approach a covered portion of road 102, such as a bridge or tunnel 104. In such a case, system 100 may determine that the travel path of autonomous vehicle 502 should be illuminated because, among other reasons, lighting conditions inside tunnel 104 are often reduced compared to daylight conditions. Accordingly, system 100 may adjust headlights 110 to illuminate the travel path of autonomous vehicle 502 before autonomous vehicle 502 enters tunnel 104 (or travels under a bridge). For example, system 100 may turn on the high-beam light source of headlight 110 while autonomous vehicle 502 is traveling inside tunnel 104 (or travels under a bridge). In one embodiment, system 100 may turn on one or more of the high-beam light source and the high-beam infrared light source of headlight 110. In one embodiment, if oncoming traffic is detected inside the tunnel 104, the system 100 may turn on the low beam light source and / or the low beam infrared light source of the headlights 110.

[0027] In another example scenario, while traveling along road 102, autonomous vehicle 502 may encounter oncoming traffic, such as vehicle 108. In such a case, system 100 may determine that headlights 110 may need to be adjusted to avoid blinding occupants within vehicle 108. For example, system 100 may turn on the low-beam light sources of headlights 110 until autonomous vehicle 502 has passed vehicle 108. In some embodiments, system 100 may turn on one or more of the low-beam light sources and low-beam infrared light sources of headlights 110.

[0028] In another example scenario, while traveling along road 102, autonomous vehicle 502 may approach left turn location 106. In such a case, system 100 may determine that a side light source (e.g., left light source 202b (see FIGS. 2A and 2B)) of headlight 110 may need to be turned on to provide better illumination to the curved side of at least a portion of the traveling path of autonomous vehicle 502. In some embodiments, system 100 may turn on one or more of left light source 202b (see FIGS. 2A and 2B) and left infrared light source 202b (see FIGS. 2A and 2B) of headlight 110.

[0029] Similarly, in another example scenario, while traveling along road 102, autonomous vehicle 502 may approach a right turn. In such a case, system 100 may determine that a side light source (e.g., right side light source 202a (see FIGS. 2A and 2B)) of headlight 110 may need to be turned on to provide better illumination to the curved side of at least a portion of the autonomous vehicle's 502 path of travel. In some embodiments, system 100 may turn on one or more of right side light source 202a (see FIGS. 2A and 2B) and right side infrared light source 202a (see FIGS. 2A and 2B) of headlight 110.

[0030] In any of the example scenarios described above and any other example scenarios described below, system 100 may dynamically adjust the intensity of the light source of headlamp 110 and / or the intensity of the infrared light source, e.g., between 0% and 100%, based on the determined illumination level 340 of the path of travel of autonomous vehicle 502. For example, if the illumination level 340 of at least a portion of the path of travel of autonomous vehicle 502 is determined to be low (e.g., 10%), system 100 may increase the intensity of the light source of headlamp 110 and / or the intensity of the infrared light source to, e.g., 80%, 85%, or any other suitable intensity level, until the illumination level 340 of at least a portion of the path of travel of autonomous vehicle 502 is determined to be above a threshold illumination level 342. In one embodiment, threshold illumination level 342 may be configured to indicate a minimum illumination level 340 of at least a portion of the path of travel of autonomous vehicle 502 at which it is safe for autonomous vehicle 502 to proceed. The threshold lighting level 342 may be adjusted depending on different road, traffic, safety, and weather conditions experienced by the autonomous vehicle 502, lighting conditions experienced by the autonomous vehicle 502 at different times, and the traveling speed of the autonomous vehicle 502.

[0031] Details of the operation of system 100 for determining lighting levels 340 for a path of travel of autonomous vehicle 502, adjusting headlights 110, and dynamically illuminating a path of travel of autonomous vehicle 502 are described in more detail below in conjunction with operational flow 300 of system 100 illustrated in FIG. 3 .

[0032] (System Components) (Exemplary Autonomous Vehicle) In one embodiment, the autonomous vehicle 502 may include a semi-truck tractor unit (see FIG. 5) attached to a trailer for transporting freight or cargo from one location to another. The autonomous vehicle 502 is generally configured to travel along roads 102 in an autonomous mode. The autonomous vehicle 502 navigates using several components, which are described in detail in FIGS. 5-7. The operation of the autonomous vehicle 502 is described in more detail in FIGS. 5-7. The corresponding description below includes a brief description of certain components of the autonomous vehicle 502.

[0033] Control device 550 may generally be configured to control the operation of autonomous vehicle 502 and its components and facilitate autonomous driving of autonomous vehicle 502. Control device 550 may be further configured to determine a path ahead for autonomous vehicle 502 that is safe to proceed along and free of objects or obstacles, and to navigate and direct autonomous vehicle 502 within that path. This process is described in more detail in FIGS. 5-7. Control device 550 may generally include one or more computing devices in signal communication with other components of autonomous vehicle 502 (see FIG. 5). In this disclosure, control device 550 may be synonymously referred to as on-board control computer 550.

[0034] The control device 550 may be configured to detect objects on and around the road 102 by analyzing the sensor data 130 and / or the map data 142. For example, the control device 550 may detect objects on and around the road 102 by implementing the object detection machine learning module 140. The object detection machine learning module 140 may be implemented using neural networks and / or machine learning algorithms to detect objects from images, videos, infrared images, point clouds, radar data, etc. The object detection machine learning module 140 is described in further more detail below. The control device 550 may receive sensor data 130 from sensors 546 positioned on the autonomous vehicle 502 and determine a safe path to proceed. The sensor data 130 may include data captured by the sensors 546.

[0035] The sensors 546 may be configured to capture any objects within their detection area or field of view, such as landmarks, lane markers, lane boundaries, road boundaries, vehicles, pedestrians, road / traffic signs, among others. In some embodiments, the sensors 546 may be configured to detect rain, fog, snow, and / or any other weather conditions.

[0036] Sensors 546 may include rain sensors, cameras, infrared cameras, (light detection and ranging) LiDAR sensors, motion sensors, infrared sensors, etc. In some embodiments, sensors 546 may be positioned around autonomous vehicle 502 to capture the environment surrounding autonomous vehicle 502. For further description of sensors 546, see the corresponding description of FIG. 5.

[0037] (control device) The control device 550 is described in further detail in FIG. 5. Briefly, the control device 550 may include a processor 122 in signal communication with a memory 126 and a network interface 124. The processor 122 may include one or more processing units that perform various functions as described herein. The memory 126 may store any data and / or instructions used by the processor 122 to perform its functions. For example, the memory 126 may store software instructions 128 that, when executed by the processor 122, cause the control device 550 to perform one or more functions described herein.

[0038] The processor 122 may be one of the data processors 570 illustrated in FIG. 5. The processor 122 comprises one or more processors operatively coupled to the memory 126. The processor 122 is any electronic circuitry including a state machine, one or more central processing unit (CPU) chips, logic units, cores (e.g., multi-core processors), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or digital signal processors (DSPs). The processor 122 may be a programmable logic device, a microcontroller, a microprocessor, or any suitable combination of the foregoing. The processor 122 is communicatively coupled to and in signal communication with the network interface 124 and the memory 126. The one or more processors may be configured to process data and may be implemented in hardware or software. For example, the processor 122 may be an 8-bit, 16-bit, 32-bit, 64-bit, or any other suitable architecture. The processor 122 may include an arithmetic logic unit (ALU) for performing arithmetic and logical operations, processor registers that supply operands to the ALU and store results of ALU operations, and a control unit that fetches instructions from memory and executes them by directing the coordinated operation of the ALU, registers, and other components. One or more processors may be configured to implement various instructions. For example, one or more processors may be configured to execute software instructions 128 to implement the functionality disclosed herein, such as some or all of those described with respect to Figures 1-7. In some embodiments, the functionality described herein is implemented using a logic unit, an FPGA, an ASIC, a DSP, or any other suitable hardware or electronic circuitry.

[0039] The network interface 124 may be a component of the network communications subsystem 592 illustrated in FIG. 5 . The network interface 124 may be configured to enable wired and / or wireless communications. The network interface 124 may be configured to communicate data between the autonomous vehicle 502 and other devices, systems, or domains. For example, the network interface 124 may comprise an NFC interface, a Bluetooth® interface, a Zigbee® interface, a Z-WAVE interface, a radio frequency identification (RFID) interface, a WIFI interface, a local area network (LAN) interface, a wide area network (WAN) interface, a metropolitan area network (MAN) interface, a personal area network (PAN) interface, a wireless PAN (WPAN) interface, a modem, a switch, and / or a router. The processor 122 may be configured to send and receive data using the network interface 124. The network interface 124 may be configured to use any suitable type of communications protocol, as would be understood by one skilled in the art.

[0040] 5. Memory 126 may store any of the information described in FIGS. 1-7 along with any other data, instructions, logic, rules, or code that, when executed by processor 122, is operable to implement the functions described herein. For example, memory 126 may store software instructions 128, sensor data 130, image quality 132, threshold image quality 134, infrared image quality 136, threshold infrared image quality 138, object detection machine learning module 140, map data 142, route planning 144, driving instructions 148, environmental information 310 (around autonomous vehicle 502), lighting level 340, threshold lighting level 342, light control fusion module 330, and / or any other data / instructions. Software instructions 128 include code that, when executed by processor 122, causes control device 550 to perform the functions described herein, such as some or all of those described in FIGS. 1-7. The memory 126 may comprise one or more disks, tape drives, or solid-state drives, may be used as overflow data storage devices, may store programs when such programs are selected for execution, and may store instructions and data read during program execution. The memory 126 may be volatile or non-volatile and may comprise read-only memory (ROM), random access memory (RAM), ternary content addressable memory (TCAM), dynamic random access memory (DRAM), and static random access memory (SRAM). The memory 126 may include one or more of a local database, a cloud database, a network-attached storage device (NAS), etc.

[0041] The object detection machine learning module 140 may be implemented by the processor 122 executing software instructions 128 and may generally be configured to detect objects and obstacles from the sensor data 130. The object detection machine learning module 140 may be implemented using neural networks and / or machine learning algorithms to detect objects from any type of data, such as images, videos, infrared images, point clouds, radar data, etc.

[0042] In some embodiments, the object detection machine learning module 140 may be implemented using machine learning algorithms such as support vector machines (SVMs), naive Bayes, logistic regression, k-nearest neighbors, decision trees, etc. In some embodiments, the object detection machine learning module 140 may utilize multiple neural network layers, convolutional neural network layers, and / or the like, where the weights and biases of these layers are optimized during the training process of the object detection machine learning module 140. The object detection machine learning module 140 may be trained by a training dataset that may include samples of one or more object-classified data types within each sample. For example, the training dataset may include sample images of object-classified objects (e.g., vehicles, lane markings, pedestrians, road signs, obstacles, etc.) within each sample image. Similarly, the training dataset may include samples of other data types, such as object-classified video, infrared imagery, point clouds, radar data, etc. within each sample data. The object detection machine learning module 140 may be trained, tested, and refined by the training dataset and the sensor data 130. The object detection machine learning module 140 uses the sensor data 130 (not labeled with objects) to increase its predictive accuracy in detecting objects. For example, supervised and / or unsupervised machine learning algorithms may be used to validate the predictions of the object detection machine learning module 140 in detecting objects in the sensor data 130.

[0043] Map data 142 may include a virtual map of a city or area, including roads 102. In some examples, map data 142 may include map 658 and map database 636 (see FIG. 6 for a description of map 658 and map database 636). Map data 142 may include drivable areas, such as roads 102, routes, and highways, and non-drivable areas, such as terrain (as determined by occupancy grid module 660 (see FIG. 6 for a description of occupancy grid module 660)). Map data 142 may define location coordinates for road signs, lanes, lane markings, lane boundaries, road boundaries, traffic lights, obstacles, and the like.

[0044] The route plan 144 may be a plan for traveling from a start location (e.g., a first autonomous vehicle departure / arrival point) to a destination (e.g., a second autonomous vehicle departure / arrival point). For example, the route plan 144 may specify one or more combinations of streets, roads, and highways in a specific order from the start location to the destination. The route plan 144 may specify stages, including a first stage (e.g., traveling out from the start location / origin), multiple intermediate stages (e.g., traveling along specific lanes of one or more specific streets / roads / highways), and a final stage (e.g., entering the destination / arrival point). The route plan 144 may include other information about the route from the start location to the destination, such as road / traffic signs within the route plan 144.

[0045] The driving instructions 148 may be implemented by the planning module 662 (see the description of the planning module 662 in FIG. 6 ). The driving instructions 148 may include instructions and rules for adapting the autonomous driving of the autonomous vehicle 502 according to the driving rules of each stage of the route plan 144. For example, the driving instructions 148 may include instructions for staying within a speed range for the road 102 being traveled by the autonomous vehicle 502, instructions for adapting the speed of the autonomous vehicle 502 to changes observed by the sensor 546 (such as the speed of surrounding vehicles or objects within the detection area of ​​the sensor 546), etc. In one example, if the path of travel of the autonomous vehicle 502 is not illuminated or if the illumination level 340 of at least a portion of the path of travel of the autonomous vehicle 502 is below the threshold illumination level 342, the driving instructions 148 may send a front light assist request to the control device 550. The front light assist request may indicate that the lighting level 340 of at least a portion of the autonomous vehicle's 502 path of travel is below the threshold lighting level 342 and / or that it is unsafe for the autonomous vehicle to proceed along the path. In another example, assuming the autonomous vehicle 502 is approaching a road corner (e.g., a left or right turn), the driving instructions 148 may send a side light assist request to the control device 550. The side light assist request may indicate that the lighting level 340 of at least a portion of the autonomous vehicle's 502 path of travel is below the threshold lighting level 342 and / or that it is unsafe for the autonomous vehicle to proceed along the path.

[0046] Headlights 110 may comprise light-emitting devices and may generally be configured to emit optical and / or infrared light. In some embodiments, system 100 may include any number of headlights 110 mounted on autonomous vehicle 502. Headlights 110 may be communicatively coupled to control device 550 via a wireless communication protocol, such as Wifi, Bluetooth, or by wire. In some embodiments, headlights 110 may include light-emitting diodes (LEDs), bulbs, infrared light LEDs (IR LEDs), infrared light emitters, among other light sources. Exemplary embodiments of headlights 110 are further described below in conjunction with FIGS. 2A and 2B .

[0047] In some embodiments, headlamp 110 may include multiple light sources. For example, headlamp 110 may include one or more optical light sources (e.g., visible light sources), one or more infrared light sources (e.g., invisible light sources), one or more near-infrared light sources, light sources with wavelengths within the light spectrum, or any combination thereof. Each light source of headlamp 110 may be controlled to emit a high beam or a low beam of light. In some embodiments, each light source of headlamp 110 may be individually controlled. For example, each light source of headlamp 110 may be individually controllable in terms of intensity, e.g., power level, and / or direction.

[0048] In some embodiments, each portion of the light sources of headlamp 110 may be individually controlled. For example, each portion of the light sources of headlamp 110 may be individually controllable in terms of intensity, e.g., power level, and / or direction.

[0049] The headlights 110 may be controlled by a control device 550. In some examples, the control device 550 may adjust the headlights 110 by operating one or more light sources of the headlights 110, such as turning on / off a front low beam light source, turning on / off a front high beam light source, turning on / off a left light source, turning on / off a right light source, adjusting the intensity of the front light source, adjusting the intensity of the left light source, and adjusting the intensity of the right light source.

[0050] In other examples, the control device 550 may adjust the headlight 110 by activating one or more infrared light sources of the headlight 110, such as turning on / off the front low-beam infrared light source, turning on / off the front high-beam infrared light source, turning on / off the left infrared light source, turning on / off the right infrared light source, adjusting the intensity of the front infrared light source, adjusting the intensity of the left infrared light source, and adjusting the intensity of the right infrared light source.

[0051] Various example scenarios in which the control device 550 may adjust the headlamps 110 are illustrated in FIG. 3 in conjunction with an operational flow 300 of the system 100.

[0052] Network 112 may include any interconnected system capable of transmitting audio, video, signals, data, messages, or any combination of the foregoing. Network 112 may include all or a portion of a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a personal area network (PAN), a wireless PAN (WPAN), an overlay network, a software-defined network (SDN), a virtual private network (VPN), a packet data network (e.g., the Internet), a mobile phone network (e.g., a cellular network such as 4G or 5G), a plain old telephone (POT) network, a wireless data network (e.g., Wifi, WiGig, WiMax, etc.), a long-term evolution (LTE) network, a universal mobile telecommunications system (UMTS) network, a peer-to-peer (P2P) network, a Bluetooth® network, a near field communication (NFC) network, a Zigbee® network, a Z-WAVE network, a Wifi network, and / or any other suitable network.

[0053] (Example headlight) 2A and 2B illustrate exemplary embodiments of a headlamp 110. FIG. 2A illustrates one embodiment of a headlamp 110 in which the right light source 202a and the left light source 202b of the headlamp 110 are separated. For example, the right light source 202a and the left light source 202b of the headlamp 110 may be implemented in different devices. FIG. 2B illustrates another embodiment of a headlamp 110 in which the headlamp 110 comprises a strip of light sources 202 from one side of the autonomous vehicle 502 to another. For example, the strip of light sources 202 of the headlamp 110 may comprise the right light source 202a and the left light source 202b of the headlamp 110. In the example of FIG. 2B, the headlamp 110 consists of six sections. Each section of the headlamp 110 may comprise one or more light sources 202. 2B, each of the right and left light sources 202a and 202b is illustrated as having three sections, however, this disclosure contemplates that each of the right and left light sources 202a and 202b can have any number of sections. Any number and combination of light sources 202 can be LEDs, bulbs, IR LEDs, infrared light emitters, among other light sources. Each light source 202 can be controlled to emit a high beam and a low beam of light.

[0054] (Operational flow for determining lighting levels for an autonomous vehicle's path of travel) 3 illustrates an example operational flow 300 of the system 100 of FIG. 1 for determining an illumination level 340 for a path of travel of an autonomous vehicle 502. The operational flow 300 may begin when a control device 550 obtains information 310. The information 310 may comprise, among other things, one or more of road curvature indication data 310a, time data 310b, location data 310c, light sensor data 310d, light condition levels 316a, images 310e, light condition levels 316b, infrared images 310f, light condition levels 316c, rain sensor data 310g, and precipitation levels 320.

[0055] (Get road curvature indication data) The road curvature indication data 310a may be detected by a vehicle control subsystem 548. The vehicle control subsystem 548 is described in detail in FIG. 5. Briefly, the vehicle control subsystem 548 may include hardware and / or software modules configured to control the autonomous operation of the autonomous vehicle 502. For example, the vehicle control subsystem 548 may control the throttle, gears, brakes, steering, and turn signals.

[0056] With respect to turn indicators, when the autonomous vehicle 502 approaches a corner or a curved road, the vehicle control subsystem 548 may activate (e.g., turn on) the corresponding turn indicator. For example, when the autonomous vehicle 502 approaches a left turn location (e.g., the road ahead of the autonomous vehicle 502 curves or turns to the left), the vehicle control subsystem 548 may turn on the left turn indicator. In another example, when the autonomous vehicle 502 approaches a right turn location (e.g., the road ahead of the autonomous vehicle 502 curves to the right), the vehicle control subsystem 548 may turn on the right turn indicator.

[0057] The activated turn signals may be included in the road curvature indication data 310a. Thus, the road curvature indication data 310a may include data indicating the status of the turn signals, such as turned on turn signals (e.g., left or right) and turned off turn signals. In some examples, the road curvature indication data 310a may include data indicating the high beam and / or low beam of the turned on turn signals.

[0058] The control device 550 may determine the status of the turn indicators from the road curvature indication data 310a in the light status detection stage 312a. For example, the control device 550 may analyze the road curvature indication data 310a through signal processing and determine the status of the turn indicators from the road curvature indication data 310a. The control device 550 may communicate the road curvature indication data 310a to the light control fusion module 330. The light control fusion module 330 may use this information to determine the illumination level 340 of the travel path of the autonomous vehicle 502. This process is further described below in conjunction with the description of the light control fusion module 330.

[0059] (Light condition level is determined from time data) Time data 310b may be detected from network 112. Network 112 may generally represent the Internet. Control device 550 may obtain the current time from network 112. For example, control device 550 may determine the location (e.g., Global Positioning System (GPS) location) of autonomous vehicle 502 and the corresponding time zone to determine the current time. Thus, time data 310b may include the current time. For example, time data 310b may indicate that the current time is 9:00 AM, 1:00 PM, etc.

[0060] In the time-based light status detection stage 312b, the control device 550 may determine the ambient lighting status at the current time and the current location of the autonomous vehicle 502. In the time-based light status detection stage 312b, the ambient lighting status may be determined based on what is expected at a particular time and location. For example, the control device 550 may determine that the ambient lighting status around the autonomous vehicle 502 indicates that the ambient lighting level is at a particular level (e.g., 15%, 20%, or any other lighting level). For example, the control device 550 may determine that the ambient lighting status around the autonomous vehicle 502 indicates that it is nighttime (e.g., the ambient lighting level is 15%). The control device 550 may communicate the time data 310b to the light control fusion module 330. The light control fusion module 330 may use this information to determine the lighting level 340 of the autonomous vehicle 502's travel path (e.g., the actual lighting level 340 of the autonomous vehicle 502's travel path). This process is further described below in conjunction with the description of the light control fusion module 330.

[0061] (Light condition levels are determined from location data) The location data 310c may be detected by a GPS sensor 546g. The GPS sensor 546g is illustrated in FIG. 5. Briefly, the GPS sensor 546g may be configured to determine a GPS location of the autonomous vehicle 502. Thus, the location data 310c may include a current GPS location of the autonomous vehicle 502. For example, the location data 310c may include current GPS location coordinates of the autonomous vehicle 502. The location data 310c may be updated as the autonomous vehicle 502 travels.

[0062] In a location-based light status detection stage 312c, the control device 550 may determine the expected ambient lighting status around the autonomous vehicle 502 based on the autonomous vehicle's 502's current GPS location from the location data 310c. The control device 550 may communicate the location data 310c to the light control fusion module 330. The light control fusion module 330 may use this information to determine the lighting level 340 for the autonomous vehicle's 502's path of travel. This process is described further below in conjunction with the description of the light control fusion module 330.

[0063] In one example scenario, assume that autonomous vehicle 502 is approaching tunnel 104 (see FIG. 1). Control device 550 may determine the location of autonomous vehicle 502 from location data 310c. Control device 550 may compare the location of autonomous vehicle 502 with map data 142 (see FIG. 1). Map data 142 may include the location and type of road traveled by autonomous vehicle 502, the road on which autonomous vehicle 502 is located, and roads near autonomous vehicle 502 (e.g., within a threshold radius from autonomous vehicle 502). Control device 550 may determine that autonomous vehicle 502 is approaching tunnel 104 (see FIG. 1) based on the comparison between the location of autonomous vehicle 502 and map data 142 (see FIG. 1). In response, control device 550 may turn on headlights 110 before autonomous vehicle 502 enters tunnel 104 (see FIG. 1). For example, control device 550 may turn on the high beam light source and / or the high beam infrared light source of headlamp 110 when oncoming traffic is not detected inside tunnel 104 (see FIG. 1). In another example, control device 550 may turn on the low beam light source and / or the low beam infrared light source of headlamp 110 when oncoming traffic is detected inside tunnel 104 (see FIG. 1).

[0064] The control device 550 may adjust the intensity of the light source (e.g., infrared and / or optical light source) of the headlamp 110 based on feedback indicating that the illumination level 340 of at least a portion of the path of travel of the autonomous vehicle 502 needs to be increased until it is determined that the illumination level 340 of the path of travel of the autonomous vehicle 502 exceeds a threshold illumination level 342. In one example, the feedback may be determined from the quality (e.g., image quality 132) of the image 310e captured after an initial increase in the intensity of the light source of the headlamp 110, the quality (e.g., infrared image quality 136) of the infrared image 310f, the light condition level 316b of the image 310e, and the light condition level 316c of the infrared image 310f. In the same or another example, the feedback may be determined from the light condition level 316a detected from the light sensor data 310d captured after an initial increase in the intensity of the light source of the headlamp 110. This process is further described below in conjunction with the description of the light control fusion module 330. Image quality 132 may indicate whether objects shown in image 310e can be properly recognized by object detection machine learning module 140. For example, if fewer than a threshold number of objects shown in image 310e are not recognized by object detection machine learning module 140, control device 550 may determine that image 310e is of low quality (e.g., less than 60%) and below a threshold image quality 134 (e.g., 70%). Infrared image quality 136 may indicate whether objects shown in infrared image 310f can be properly recognized by object detection machine learning module 140. For example, if fewer than a threshold number of objects shown in infrared image 310f are not recognized by object detection machine learning module 140, control device 550 may determine that infrared image 310f is of low quality (e.g., less than 60%) and below a threshold infrared image quality 138 (e.g., 70%).

[0065] Similarly, when autonomous vehicle 502 is exiting tunnel 104 (see FIG. 1 ), control device 550 may determine that autonomous vehicle 502 is exiting tunnel 104 (see FIG. 1 ) based on a comparison between the current location of autonomous vehicle 502 and map data 142 (see FIG. 1 ). In response, control device 550 may turn off the light source and / or the infrared light source of headlamp 110 after autonomous vehicle 502 exits tunnel 104 (see FIG. 1 ).

[0066] (Light condition levels are determined from light sensor data) The light sensor data 310d may be detected by a light sensor 546k. The light sensor 546k is illustrated in FIG. 5. Briefly, the light sensor 546k may include a photoelectric device configured to detect light condition levels within its detection range. The light sensor 546k may be configured to convert light energy (e.g., photons) into an electrical signal. The amplitude of the electrical signal may be proportional to the detected light energy. When the light sensor 546k detects a more concentrated light energy flux, the amplitude of the electrical signal may be higher. The light sensor 546k may determine its ambient light condition level 316a based on the generated electrical signal. Examples of the light sensor 546k may include a photoresistor, a photosensitive resistor, a photodiode, a phototransistor, and / or any other device configured to convert light into an electrical signal.

[0067] In some embodiments, multiple light sensors 546k may be associated with the autonomous vehicle 502. For example, a first set of light sensors 546k may be located on the front of the autonomous vehicle 502, a second set of light sensors 546k may be located on the rear of the autonomous vehicle 502, a third set of light sensors 546k may be located on the left side of the autonomous vehicle 502, and a fourth set of light sensors 546k may be located on the right side of the autonomous vehicle 502. Each set of light sensors 546k may be located adjacent to a corresponding camera 546a and / or infrared camera 546j. For example, a light sensor 546k located on the front of the autonomous vehicle 502 may be located adjacent to a front-facing camera 546a and / or a front-facing infrared camera 546j, etc.

[0068] The light sensor data 310d captured by the front light sensor 546k may be used to determine a light condition level 316a for the roadway the autonomous vehicle 502 is on. The light condition level 316a detected by the light sensor 546k may represent the amount of light energy detected by the light sensor 546k. The light condition level 316a may be an output of the light sensor 546k. The light sensor data 310d may include the light condition level 316a detected by the light sensor 546k. The light condition level 316a may be expressed in a percentage format, such as 0%, 10%, 15%, etc.

[0069] During the illumination-based light status detection stage 312d, the control device 550 may determine a light condition level 316a indicated in the light sensor data 310d. In one example scenario, assume that the autonomous vehicle 502 is traveling along a road. The control device 550 may determine that at least a portion of the road (on which the autonomous vehicle 502 is located) should be illuminated if the light condition level 316a (determined from the light sensor data 310d) of the portion of the road is determined to be below a threshold light condition level (e.g., 50%, 70%, or any other suitable value). In other words, the control device 550 may determine that a portion of the road is dark.

[0070] The control device 550 may communicate the light sensor data 310d to the light control fusion module 330. The light control fusion module 330 may use this information to determine the lighting level 340 for the path of travel of the autonomous vehicle 502. This process is further described below in conjunction with the description of the light control fusion module 330.

[0071] (Light condition level is determined from the image) The image 310e may be captured by a camera 546a. The camera 546a is illustrated in FIG. 5. Briefly, each camera 546a may generally include any device configured to capture images and / or video 310e of its surrounding environment. Examples of cameras 546a may include digital cameras, video cameras, webcams, and / or any other type of camera. Each camera 546a may communicate the captured images and / or video 310e to the control device 550 for processing. In this disclosure, the images and videos 310e may be collectively referred to as images 310e. The images 310e may include images of one or more portions of a roadway on which the autonomous vehicle 502 is located.

[0072] In the image-based light status detection stage 312e, the control device 550 may determine the light condition level 316b of each image 310e. In one embodiment, the control device 550 may determine the light condition level 316b of the image 310e based on the color of the pixels of the image 310e. In this operation, the control device 550 may determine the color of the pixels of the image 310e by, for example, an image processing algorithm implemented by the control device 550 executing software instructions 128 (see FIG. 1).

[0073] The control device 550 may determine the light condition level 316b of the image 310e based on the percentage of pixels of the image 310e that are black, whereby the light condition level 316b of the image 310e may be inversely proportional to the percentage of black pixels of the image 310e. For example, as the percentage of black pixels of the image 310e increases, the control device 550 may determine that the light condition level 316b of the image 310e decreases.

[0074] In one example scenario, if it is determined that a greater than a threshold percentage of pixels in image 310e of the path of travel of autonomous vehicle 502 are black (e.g., greater than 50%, 55%, etc.), control device 550 may determine that the path of travel of autonomous vehicle 502 should be illuminated.

[0075] In one embodiment, the control device 550 may determine the light condition level 316b of the image 310e by implementing a light detection algorithm 314. The light detection algorithm 314 may be configured to detect the light condition level 316b of the image 310e. For example, the light detection algorithm 314 may include a support vector machine, a neural network, a random forest, k-means clustering, an image processing algorithm, etc. In other examples, the light detection algorithm 314 may include, without limitation, a multi-layer perceptron, a recurrent neural network (RNN), an RNN long short-term memory (LSTM), a convolutional neural network (CNN), a transformer, or any other suitable type of neural network model.

[0076] The light detection algorithm 314 may be trained by a training dataset comprising a set of images classified by various light condition levels 316b. The light detection algorithm 314 may be trained to learn the association between each image in the training dataset (e.g., training image) and its corresponding light condition level 316b. In this process, the light detection algorithm 314 may feed the training images into a neural network and extract features from the training images, such as brightness, contrast, shading, and / or other aspects of the training images. The light detection algorithm 314 may learn to associate the extracted features with the light condition levels 316b classified for the training images. In a testing phase, the light detection algorithm 314 may be given another image from the training dataset (e.g., a test image) without its light condition level 316b and asked to predict the light condition level 316b of the test image. The light detection algorithm 314 may extract features of the test image, similar to those described above. The light detection algorithm 314 may compare the features of the test image with the features of the training image. The light detection algorithm 314 may predict that the test image will have the same (or similar) light condition levels 316b as the training images, which have matching (or corresponding) features compared to the test images. In this manner, the control device 550 (via the light detection algorithm 314) may determine the light condition levels 316b of the image 310e.

[0077] The control device 550 may communicate the image 310e and the light condition level 316b to the light control fusion module 330. The light control fusion module 330 may use this information to determine the lighting level 340 for the path of travel of the autonomous vehicle 502. This process is further described below in conjunction with the description of the light control fusion module 330.

[0078] (Light condition levels are determined from the thermal image) The infrared images 310f may be captured by infrared cameras 546j. Infrared cameras 546j are illustrated in FIG. 5. Briefly, each infrared camera 546j may generally include any device configured to capture infrared images and / or infrared video 310f of its surrounding environment. Examples of infrared cameras 546j may include digital cameras, thermal cameras, etc. The infrared images 310f (or infrared video) may indicate the temperature of an object in different colors. For example, the color of an object shown in the infrared image 310f may represent the temperature of the object. Each infrared camera 546j may communicate the captured infrared images and / or video 310f to the control device 550 for processing. In this disclosure, the infrared images and / or video 310f may be collectively referred to as infrared image 310f. The infrared images 310f may include an infrared image of one or more portions of a roadway on which the autonomous vehicle 502 is located.

[0079] In the infrared image-based light status detection stage 312f, the control device 550 may determine the light condition level 316c of each infrared image 310f, similar to that described above with respect to determining the light condition level 316b of image 310e.

[0080] In one embodiment, the control device 550 may determine the light condition level 316c of the infrared image 310f based color of a pixel of the infrared image 310f. In this operation, the control device 550 may determine the color of the pixel of the infrared image 310f by, for example, an infrared image processing algorithm implemented by the control device 550 executing software instructions 128 (see FIG. 1).

[0081] The control device 550 may determine the light condition level 316c of the thermal image 310f based on the percentage of pixels in the thermal image 310f that are black, such that the light condition level 316c of the thermal image 310f is inversely proportional to the percentage of black pixels in the thermal image 310f. For example, as the percentage of black pixels in the thermal image 310f increases, the control device 550 may determine that the light condition level 316c of the thermal image 310f decreases. In one example scenario, if a greater than a threshold percentage of pixels in the thermal image 310f of the traveling path of the autonomous vehicle 502 is determined to be black (e.g., greater than 50%, 55%, etc.), the control device 550 may determine that the traveling path of the autonomous vehicle 502 should be illuminated.

[0082] In one embodiment, the control device 550 may determine the light condition level 316c of the infrared image 310f by implementing a light detection algorithm 318. The light detection algorithm 318 may be configured to detect the light condition level 316c of the infrared image 310f. For example, the light detection algorithm 318 may include, but is not limited to, a support vector machine, a neural network, a random forest, k-means clustering, image processing, etc. In other examples, the light detection algorithm 318 may include, but is not limited to, a multi-layer perceptron, an RNN, an RNN LSTM, a CNN, a transformer, or any other suitable type of neural network model.

[0083] The light detection algorithm 318 may be trained with a training dataset comprising a set of infrared images classified by various light condition levels 316c. The light detection algorithm 318 may be trained to learn the association between each infrared image in the training dataset (e.g., training infrared image) and its corresponding light condition level 316c. In this process, the light detection algorithm 318 may feed the training infrared images into a neural network and extract features from the training infrared images, such as brightness, contrast, shading, and / or other aspects of the training infrared images. The light detection algorithm 318 may learn to associate the extracted features with the light condition levels 316c classified for the training infrared images. In a testing phase, the light detection algorithm 318 may be given another infrared image from the training dataset (e.g., a test infrared image) without its light condition classification indication and asked to predict the light condition level 316c of the test infrared image. The light detection algorithm 318 may extract features of the test infrared image, similar to those described above. The light detection algorithm 318 may compare the features of the test infrared image with the features of the training infrared images. The light detection algorithm 318 may predict that the test infrared image will have the same (or similar) light condition levels 316c as the training infrared image that has matching (or corresponding) features compared to the test infrared image. In this manner, the control device 550 (via the light detection algorithm 318) may determine the light condition levels 316c of the infrared image 310f.

[0084] The control device 550 may communicate the thermal image 310f and the light condition level 316c to the light control fusion module 330. The light control fusion module 330 may use this information to determine the lighting level 340 for the path of travel of the autonomous vehicle 502. This process is further described below in conjunction with the description of the light control fusion module 330.

[0085] (Light condition levels are determined from rain sensor data) The rain sensor data 310g may be captured by rain sensors 546i. Rain sensors 546i are illustrated in FIG. 5. Briefly, each rain sensor 546i may include a liquid sensing module and generally configured to sense a liquid level (e.g., humidity level, raindrops) on or within the housing of the rain sensor 546i. At least a portion of the housing of the rain sensor 546i may be the sensing area of ​​the rain sensor 546i, thereby enabling the rain sensor 546i to detect a liquid level on the sensing area. Examples of the rain sensor 546i may include a capacitance-based sensor, an optical sensor, an infrared-based sensor, and / or any other type of sensor capable of detecting a liquid level on or within its sensing area.

[0086] The capacitance-based rain sensor 546i may be configured to detect the liquid level on a sensing area by determining the difference in capacitance detected from the sensing area before and after humidity (such as raindrops) is added onto the sensing area.

[0087] The optical rain sensor 546i may be configured to detect liquid levels by determining the difference in the transmitted and reflected optical signals (e.g., laser beams) before and during rain. The infrared-based rain sensor 546i may be configured to detect liquid levels by determining the difference in the transmitted and reflected infrared signals before and during rain. For example, the difference in the transmitted and reflected infrared signals before and during rain may include frequency changes, phase signal changes, and energy (e.g., power level, intensity) changes between the transmitted and reflected infrared signals.

[0088] The rain sensor data 310g may indicate, for example, a liquid level detected due to rain. The control device 550 may determine the rainfall level 320 based on the rain sensor data 310g. For example, the rainfall level may be expressed in a percentage format, such as 10%, 20%, etc.

[0089] In one example scenario, assume that the autonomous vehicle 502 is traveling along a road while it is raining. The control device 550 may determine the rainfall level 320 based on the rain sensor data 310g. The control device 550 may determine that the path of travel (e.g., at least a portion of the road on which the autonomous vehicle 502 is located) should be illuminated if it is determined that the rain sensor data 310g indicates that the rainfall level 320 is above a threshold rainfall level, e.g., above 40%, 45%, etc.

[0090] The control device 550 may communicate the rain sensor data 310g and the rainfall level 320 to the light control fusion module 330. The light control fusion module 330 may use this information to determine the lighting level 340 for the path of travel of the autonomous vehicle 502. This process is described further below in conjunction with the description of the light control fusion module 330.

[0091] (Determines whether the autonomous vehicle's path should be illuminated) The control device 550 may implement the light control fusion module 330 by the processor 122 executing the software instructions 128. The light control fusion module 330 may include hardware and / or software modules and is generally configured to determine, based on the information 310, a lighting level 340 for at least a portion of a path of travel of the autonomous vehicle 502, and to determine, based on the information 310, whether the path of travel of the autonomous vehicle 502 should be illuminated. The lighting level 340 may indicate the amount of light in an environment in the path of travel of the autonomous vehicle 502.

[0092] In one embodiment, the light control fusion module 330 may determine that the path of travel of the autonomous vehicle 502 should be illuminated if the determined illumination level 340 of the path of travel is less than a threshold illumination level 342, e.g., 40%, 45%, etc.

[0093] In one embodiment, the light control fusion module 330 may determine the lighting level 340 based on the information 310. For example, the light control fusion module 330 may determine the lighting level 340 of any suitable number and combination of factors associated with at least a portion of the travel path of the autonomous vehicle 502 based on, for example, 1) the current time at the location of the autonomous vehicle 502 indicated in the time data 310b (e.g., 9:00 AM), 2) the location of the autonomous vehicle 502 indicated in the location data 310c (e.g., the autonomous vehicle 502 is approaching a tunnel), 3) the light condition level 316a (e.g., 65%) indicated in the light sensor data 310d, 4) the light condition level 316b (e.g., 70%) detected from the travel path image 310e, and 5) the light condition level 316c (e.g., 73%) detected from the travel path infrared image 310f. In one embodiment, the light control fusion module 330 may determine the lighting level 340 by determining an average of the light condition levels 316a-316c. The current time and location of the autonomous vehicle 502 may be used to determine the accuracy (e.g., confidence level) of the lighting level 340.

[0094] In one embodiment, if it is determined that the path of travel of autonomous vehicle 502 should be illuminated, control device 550 may adjust headlights 110 to illuminate the path of travel. For example, if information 310 indicates that 1) the current time is nighttime, and / or 2) autonomous vehicle 502 is approaching a tunnel, and / or 3) light condition level 316b in image 310e is below a threshold light condition level, and / or 4) light condition level 316c in infrared image 310f is below a threshold light condition level, and / or 5) light condition level 316a indicated in light sensor data 310d is below a threshold light condition level, control device 550 may adjust headlights 110 to illuminate the path of travel.

[0095] In one embodiment, if the determined illumination level 340 is lower than the threshold illumination level 342, the control device 550 may adjust the headlights 110 to illuminate the path of travel. Example scenarios in which the control device 550 adjusts the headlights 110 are described below.

[0096] In one example scenario, with respect to road curvature indication data 310a, if control device 550 determines that road curvature indication data 310a indicates that autonomous vehicle 502 is approaching a left turn location (e.g., the road ahead of autonomous vehicle 502 is curving to the left), light control fusion module 330 may determine that the left light source and / or the left infrared light source of headlamp 110 may need to be turned on. In response, control device 550 may adjust headlamp 110 by turning on the left light source and / or the left infrared light source of headlamp 110 (e.g., light source 202b (see FIGS. 2A and 2B)).

[0097] In another example scenario, with respect to time data 310b, if time data 310b indicates that the current time is nighttime (e.g., 9:00 P.M.), light control fusion module 330 may determine that the front light source and / or the front infrared light source of headlamp 110 may need to be turned on. In response, control device 550 may adjust headlamp 110 by turning on the front light source and / or the front infrared light source of headlamp 110.

[0098] In another example scenario, with respect to location data 310c, if location data 310c indicates that autonomous vehicle 502 is approaching a tunnel (e.g., tunnel 104 in FIG. 1 ), light control fusion module 330 may determine that the front high beam light source and / or the front high beam infrared light source of headlight 110 may need to be activated. In response, control device 550 may turn on the front high beam light source and / or the front high beam infrared light source of headlight 110 before autonomous vehicle 502 enters the tunnel.

[0099] In another example scenario, with respect to light sensor data 310d, if the light sensor data 310d indicates that the ambient light surrounding the autonomous vehicle 502 is low (e.g., 20%, 25%), the light control fusion module 330 may determine that one or more light sources (e.g., low beam, high beam, optical light source, infrared light source) of the headlight 110 may need to be activated. In response, the control device 550 may activate one or more light sources of the headlight 110.

[0100] In another example scenario, for image 310e, if the light condition level 316b of image 310e is determined to be below a threshold light condition level (e.g., below 70%, 75%, etc.), the light control fusion module 330 may determine that one or more light sources (e.g., low beam, high beam, optical light source, infrared light source) of headlamp 110 may need to be activated. In response, control device 550 may activate one or more light sources of headlamp 110.

[0101] In another example scenario, with respect to the thermal image 310f, if the light condition level 316c of the thermal image 310f is determined to be below a threshold light condition level (e.g., below 70%, 75%, etc.), the light control fusion module 330 may determine that one or more light sources (e.g., low beam, high beam, optical light source, infrared light source) of the headlamp 110 may need to be activated. In response, the control device 550 may activate one or more light sources of the headlamp 110.

[0102] In another example scenario, if the light sensor data 310d is determined to indicate that the light condition level of the environment surrounding the autonomous vehicle 502 is below a threshold light condition level (e.g., below 70%, 75%, etc.), and / or if the time data 310b is determined to indicate that the current time is nighttime, and / or if the driving command 148 is determined to request front light assist from the control device 550, the light control fusion module 330 may determine that the front low beam light source or the front high light source of the headlamp 110 may need to be turned on (depending on the oncoming traffic, such that if oncoming traffic is detected by analyzing the sensor data 130, the low beam light source may be turned on, and if oncoming traffic is not detected by analyzing the sensor data 130, the high beam light source may be turned on).

[0103] The light control fusion module 330 may further determine that the frontal infrared light source of the headlamp 110 may need to be turned on. The control device 550 (via the light control fusion module 330) may adjust the intensity of the frontal infrared source based on the quality of the infrared image 310f and the light condition level 316c determined from the infrared image 310f. For example, if the light condition level 316c of the infrared image 310f is below a threshold light condition level (e.g., below 70%, 75%, etc.), the control device 550 may increase the intensity of the frontal infrared source until the light condition level 316c of the infrared image 310f exceeds the threshold light condition level. Example pseudocode for this example scenario may be as follows: If(light condition level < threshold light condition level; and / or current time == nighttime; and / or front light assist request received == yes) { If(oncoming traffic detected == yes) { Turn on low beam front light source;} Else if(oncoming traffic detected == no) { Turn on high beam front light source;} Turn on the front infrared light source; adjusting the intensity of the front infrared light source based on the infrared image quality and the light condition level determined from the infrared image;

[0104] In another example scenario, if 1) light sensor data 310d is determined to indicate that light condition level 316a of the autonomous vehicle's 502 path of travel is below a threshold light condition level (e.g., below 70%, 75%, etc.) and / or time data 310b is determined to indicate that the current time is nighttime, and 2) location data 310c is determined to indicate that the autonomous vehicle 502 is approaching a particular turn (e.g., a left or right turn) and / or driving command 148 requests side light assist from control device 550, then light control fusion module 330 may determine that the side light source of headlamp 110 may need to be turned on. The light control fusion module 330 may further determine that the side infrared light source of headlamp 110 may need to be turned on.

[0105] The control device 550 (via the light control fusion module 330) may adjust the intensity of the side infrared light source based on the quality of the infrared image 310f and the light condition level 316c determined from the infrared image 310f. For example, if the light condition level 316b of the infrared image 310f is below a threshold light condition level (e.g., below 70%, 75%, etc.), the control device 550 may increase the intensity of the side infrared source infrared until the light condition level 316b of the image 310f exceeds the threshold light condition level. Example pseudocode for this example scenario may be as follows: If(light condition level < threshold light condition level; and / or current time == night) { If(autonomous vehicle is approaching a turn location == yes; and / or side light assistance request received == yes) { Turn on the side light source; Turn on the side infrared light source; adjust the intensity of the side infrared light source based on the thermal image quality and the light condition level determined from the thermal image;

[0106] In any of the example scenarios described above, if control device 550 detects oncoming traffic, for example, by analyzing sensor data 130 (see FIG. 1 ), control device 550 may determine that the high beam light source of headlamp 110 may need to be turned off and the infrared light source of headlamp 110 may need to be turned on. Control device 550 may also determine that the low beam light source and / or the low beam infrared light source of headlamp 110 may need to be turned on, depending on the scenario encountered.

[0107] In any of the example scenarios described above, control device 550 may adjust the intensity of activated light sources from 0% to 100% according to a comparison between illumination level 340 and threshold illumination level 342, light condition levels 316a-316c, and the quality of image 310e and / or infrared image 310f. The quality of image 310e and / or infrared image 310f may indicate whether an object shown in image 310e and / or infrared image 310f can be properly recognized by object detection machine learning module 140.

[0108] In some embodiments, object detection machine learning module 140 may determine the current lighting status of at least a portion of a road along which autonomous vehicle 502 is traveling. For example, whether headlights 110 are on or off, object detection machine learning module 140 may determine the current lighting status of at least a portion of a road based on image quality 132 of image 310 e and / or infrared image quality 136 of infrared image 310 f. This information, along with other information obtained from sensors 546, network 112, and vehicle control subsystem 548, as described above, may be used as additional information in adjusting headlights 110.

[0109] Exemplary Methods of Implementing an Adaptive Lighting System for Autonomous Vehicles FIG. 4 illustrates an example flowchart of a method 400 for implementing an adaptive lighting system for an autonomous vehicle 502. Modifications, additions, or omissions may be made to method 400. Method 400 may include more, fewer, or other operations. For example, operations may be performed in parallel or in any suitable order. At times, the autonomous vehicle 502, the control device 550, or any component thereof is discussed as performing operations, but any suitable system or component of the system may perform one or more operations of method 400. For example, one or more operations of method 400 may be implemented, at least in part, in the form of software instructions 128 and processing instructions 580 from FIGS. 1 and 5, respectively, stored on a non-transitory, tangible, machine-readable medium (e.g., memory 126 and data storage 590 from FIGS. 1 and 5, respectively) that, when invoked by one or more processors (e.g., processors 122 and 570 from FIGS. 1 and 5, respectively), may cause the one or more processors to perform operations 402-408.

[0110] Method 400 may begin 402, in which control device 550 may obtain information 310 about the environment surrounding autonomous vehicle 502. Information 310 may comprise one or more of road curvature indication data 310a, time data 310b, location data 310c, light sensor data 310d, light condition levels 316a, images 310e, light condition levels 316b, infrared images 310f, light condition levels 316c, rain sensor data 310g, and rainfall levels 320. Control device 550 may obtain information 310 from one or more of vehicle control subsystem 548, network 112, GPS sensor 546g, light sensor 546k, camera 546a, infrared camera 546j, and rain sensor 546i, similar to those described in FIG.

[0111] At 404, the control device 550 may determine, based at least on the information 310, whether at least a portion of the road on which the autonomous vehicle 502 is located should be illuminated. In this process, the control device 550 may determine an illumination level 340 of at least a portion of the road on which the autonomous vehicle 502 is located (e.g., the path of travel of the autonomous vehicle 502), similar to that described in the operational flow 300 of FIG. 3. For example, the control device 550 may determine an increase or decrease by which at least a portion of the road (e.g., the path of travel of the autonomous vehicle 502) should be illuminated if the information 310 indicates that the illumination level 340 of at least a portion of the road is lower than a threshold illumination level 342. Other example scenarios in which the control device 550 may determine whether at least a portion of the road should be illuminated based on the information 310 are described in FIGS. 1 and 3. If it is determined that at least a portion of the road should be illuminated, the method 400 may proceed to 408. Otherwise, the method 400 may proceed to 406.

[0112] At 406 , the control device 550 may not adjust the headlights 110 associated with the autonomous vehicle 502 .

[0113] At 408, the control device 550 may adjust the headlights 110 to illuminate at least a portion of the road on which the autonomous vehicle 502 is located.

[0114] For example, with respect to image 310e of the path of travel of autonomous vehicle 502, control device 550 may determine the light conditions level 316a of image 310e. If control device 550 determines that light conditions level 316a is less than a threshold light conditions level 150 (e.g., less than 70%, 65%, etc.), control device 550 may adjust headlights 110 to increase the illumination level 340 of at least a portion of the path of travel of autonomous vehicle 502 until light conditions level 316a is at least equal to the threshold light conditions level 150. For example, control device 550 may increase the intensity of the light source of headlights 110 until light conditions level 316a (of future images 310e captured after the initial adjustment of headlights 110) is at least equal to the threshold light conditions level. Additionally or alternatively, the control device 550 may adjust the headlights 110 to increase the illumination level 340 of at least a portion of the path of travel of the autonomous vehicle 502 until the illumination level 340 is at least equal to the threshold illumination level 342.

[0115] In another example, with respect to an infrared image 310f of a path of travel of the autonomous vehicle 502, the control device 550 may determine a light conditions level 316c of the infrared image 310f. If the control device 550 determines that the light conditions level 316c is less than a threshold light conditions level 150 (e.g., less than 70%, 65%, etc.), the control device 550 may adjust the headlights 110 to increase the illumination level 340 of at least a portion of the path of travel of the autonomous vehicle 502 until the light conditions level 316c is at least equal to the threshold light conditions level 150.

[0116] For example, control device 550 may increase the intensity of the infrared light source of headlamp 110 until light conditions level 316c (of future infrared images 310f captured after the initial adjustment of headlamp 110) is at least equal to the threshold light conditions level. Additionally or alternatively, control device 550 may adjust headlamp 110 to increase the illumination level 340 of at least a portion of the path of travel of autonomous vehicle 502 until the illumination level 340 is at least equal to threshold illumination level 342. Control device 550 may cause one or more of the actions described above to be performed when control device 550 receives sensor data 130 and detects the presence of oncoming traffic in sensor data 130. In response to oncoming traffic being detected, control device 550 may turn on one or more of the low-beam infrared light source of headlamp 110 and the low-beam light source of headlamp 110. The control device 550 may determine the infrared image quality 136 of at least a portion of the infrared image 310f of the path of travel of the autonomous vehicle 502. If the control device 550 determines that the infrared image quality 136 is lower than the threshold infrared image quality 138, the control device 550 may increase the intensity of the infrared light emitted from the low-beam infrared light source of the headlamp 110 until the infrared image quality 136 is at least equal to the threshold infrared image quality 138. The control device 550 may perform similar operations with respect to the image 310e of at least a portion of the path of travel of the autonomous vehicle 502. For example, the control device 550 may determine the image quality 132 of the image 310e of the path of travel of the autonomous vehicle 502. If the control device 550 determines that the image quality 132 is lower than the threshold image quality 134, the control device 550 may increase the intensity of the light emitted from the low-beam light source of the headlamp 110 until the image quality 132 is at least equal to the threshold image quality 134.

[0117] In another example, assume that control device 550 receives sensor data 130 and detects from the sensor data 130 that there is no oncoming traffic. In such a case, control device 550 may turn on one or more of the high-beam infrared light source of headlight 110 and the high-beam light source of headlight 110. Control device 550 may determine the infrared image quality 136 of at least a portion of infrared image 310f of the traveling path of autonomous vehicle 502. If control device 550 determines that the infrared image quality 136 is lower than a threshold infrared image quality 138, control device 550 may increase the intensity of infrared light emitted from the high-beam infrared light source of headlight 110 until the infrared image quality 136 is at least equal to the threshold infrared image quality 138. Control device 550 may perform similar operations with respect to image 310e of at least a portion of the traveling path of autonomous vehicle 502. For example, control device 550 may determine the image quality 132 of image 310e of at least a portion of the traveling path of autonomous vehicle 502. If the control device 550 determines that the image quality 132 is lower than the threshold image quality 134, the control device 550 may increase the intensity of the light emitted from the high beam light source of the headlamp 110 until the image quality 132 is at least equal to the threshold image quality 134.

[0118] In another example, assume that information 310 indicates that the illumination level 340 of at least a portion of the path of travel of autonomous vehicle 502 is below a threshold illumination level 342. Also assume that control device 550 is detecting oncoming traffic by analyzing sensor data 130. In this example, control device 550 may turn on one or more of the low-beam infrared light source and the low-beam light source of headlight 110. Control device 550 may determine the quality (e.g., infrared image quality 136) of infrared image 310f of at least a portion of the path of travel of autonomous vehicle 502. The quality of infrared image 310f may indicate whether objects shown in infrared image 310f can be properly recognized by object detection machine learning module 140. For example, if less than a threshold number of objects shown in infrared image 310f are not recognized by object detection machine learning module 140, control device 550 may determine that the quality of infrared image 310f is low (e.g., below 60%) and below threshold infrared image quality 138 (e.g., 70%). If the control device 550 determines that the quality of the infrared image 310f is lower than the threshold infrared image quality 138, the control device 550 may increase the intensity of the infrared light emitted from the low-beam infrared light source until the quality of the infrared image 310f is at least equal to the threshold infrared image quality 138.

[0119] The control device 550 may perform a similar operation with respect to image 310e, increasing the intensity of the light source of the headlamp 110 until the quality (e.g., image quality 132) of image 310e is at least equal to the threshold image quality 134. The control device 550 may determine the quality (e.g., image quality 132) of image 310e of at least a portion of the path of travel of the autonomous vehicle 502. The quality of image 310e may indicate whether objects shown in image 310e can be properly recognized by the object detection machine learning module 140. For example, if less than a threshold number of objects shown in image 310e are not recognized by the object detection machine learning module 140, the control device 550 may determine that the quality of image 310e is low (e.g., less than 60%) and less than the threshold image quality 134 (e.g., 70%).

[0120] In another example, assume that information 310 indicates that the illumination level 340 of at least a portion of the path of travel of autonomous vehicle 502 is below threshold illumination level 342. Assume that the control device also determines that there is no oncoming traffic detected from sensor data 130. In this example, control device 550 may turn on one or more of the high-beam infrared light source and the high-beam light source of headlight 110.

[0121] Control device 550 may determine the quality of infrared image 310f of at least a portion of the path of travel of autonomous vehicle 502, similar to that described above. If control device 550 determines that the quality of infrared image 310f is lower than threshold infrared image quality 138, control device 550 may increase the intensity of infrared light emitted from the high-beam infrared light source until the quality of infrared image 310f is at least equal to threshold infrared image quality 138. Control device 550 may perform similar operations with respect to image 310e, increasing the intensity of the light source of headlamp 110 until the quality of image 310e is at least equal to threshold image quality 134.

[0122] In another example, assume that road curvature indication data 310a and / or location data 310c indicate that autonomous vehicle 502 is approaching a left turn location (e.g., the road ahead of autonomous vehicle 502 curves to the left). In this example, control device 550 may turn on one or more of the left infrared light source of headlamp 110 and the left light source (e.g., light source 202b (see FIGS. 2A and 2B)). Control device 550 may determine the quality of infrared image 310f of at least a portion of the travel path of autonomous vehicle 502, similar to that described above. If control device 550 determines that the quality of infrared image 310f is lower than threshold infrared image quality 138, control device 550 may increase the intensity of infrared light emitted from the left infrared light source until the quality of infrared image 310f is at least equal to threshold infrared image quality 138. The control device 550 may perform similar operations with respect to the image 310 e , increasing the intensity of the light source of the headlamp 110 until the quality of the image 310 e is at least equal to the threshold image quality 134 .

[0123] Control device 550 may perform similar operations for the case where autonomous vehicle 502 approaches a right turn location (e.g., the road ahead of autonomous vehicle 502 curves to the left). For example, assume that information 310 indicates that the illumination level 340 of at least a portion of autonomous vehicle 502's path of travel is below threshold illumination level 342. Assume that road curvature indication data 310a and / or location data 310c also indicate that autonomous vehicle 502 is approaching a right turn location. In this example, control device 550 may turn on one or more of right infrared light source and right light source (e.g., light source 202a (see FIGS. 2A and 2B)) of headlamp 110. Control device 550 may determine the quality of infrared image 310f of at least a portion of autonomous vehicle 502's path of travel, similar to that described above.

[0124] If the control device 550 determines that the quality of the infrared image 310f is lower than the threshold infrared image quality 138, the control device 550 may increase the intensity of the infrared light emitted from the right infrared light source until the quality of the infrared image 310f is at least equal to the threshold infrared image quality 138. The control device 550 may perform similar operations with respect to image 310e, increasing the intensity of the light source of the headlamp 110 until the quality of image 310e is at least equal to the threshold image quality 134.

[0125] Exemplary Autonomous Vehicles and Their Operation FIG. 5 shows a block diagram of an example vehicle ecosystem 500 in which autonomous driving behavior may be determined. As shown in FIG. 5, the autonomous vehicle 502 may be a semi-trailer truck. The vehicle ecosystem 500 may include several systems and components that may generate and / or deliver one or more sources of information / data and related services to an onboard control computer 550, which may be located within the autonomous vehicle 502. The onboard control computer 550 may be in data communication with multiple vehicle subsystems 540, all of which may reside within the autonomous vehicle 502. A vehicle subsystem interface 560 may be provided to facilitate data communication between the onboard control computer 550 and the multiple vehicle subsystems 540. In some embodiments, the vehicle subsystem interface 560 may include a controller area network (CAN) controller and communicate with devices within the vehicle subsystem 540.

[0126] Autonomous vehicle 502 may include various vehicle subsystems that support operation of autonomous vehicle 502. Vehicle subsystems 540 may include a vehicle driving subsystem 542, a vehicle sensor subsystem 544, a vehicle control subsystem 548, and / or a network communication subsystem 592. The components or devices of vehicle driving subsystem 542, vehicle sensor subsystem 544, and vehicle control subsystem 548 shown in FIG. 5 are examples. Autonomous vehicle 502 may be configured as shown or in any other configuration.

[0127] Vehicle driving subsystem 542 may include components operable to provide powered motion for autonomous vehicle 502. In an exemplary embodiment, vehicle driving subsystem 542 may include engine / motor 542a, wheels / tires 542b, transmission 542c, electrical subsystem 542d, and power source 542e.

[0128] The vehicle sensor subsystem 544 may include several sensors 546 configured to sense information about the environment or conditions of the autonomous vehicle 502. The vehicle sensor subsystem 544 may include one or more cameras 546a or image capture devices, a radar unit 546b, one or more temperature sensors 546c, a wireless communication unit 546d (e.g., a cellular communication transceiver), an inertial measurement unit (IMU) 546e, a laser rangefinder / LiDAR unit 546f, a global positioning system (GPS) transceiver 546g, a wiper control system 546h, one or more rain sensors 546i, and / or an infrared camera 546j. The vehicle sensor subsystem 544 may also include sensors configured to monitor internal systems of the autonomous vehicle 502 (e.g., an O monitor, a fuel gauge, engine oil temperature, etc.).

[0129] The IMU 546e may include any combination of sensors (e.g., accelerometers and gyroscopes) configured to sense position and orientation changes of the autonomous vehicle 502 based on inertial acceleration. The GPS transceiver 546g may be any sensor configured to estimate the geographic location of the autonomous vehicle 502. To this end, the GPS transceiver 546g may include a receiver / transmitter operable to provide information regarding the position of the autonomous vehicle 502 relative to the Earth. The radar unit 546b may represent a system that utilizes radio signals to sense objects within the local environment of the autonomous vehicle 502. In some embodiments, in addition to sensing objects, the radar unit 546b may additionally be configured to sense the speed and heading of objects proximate to the autonomous vehicle 502. The laser rangefinder or LiDAR unit 546f may be any sensor configured to sense objects within the environment in which the autonomous vehicle 502 is located using lasers. The camera 546a may include one or more devices configured to capture multiple images of the environment of the autonomous vehicle 502. Camera 546a may be a still image camera or a motion video camera. Infrared camera 546j may include one or more devices configured to capture multiple infrared images of the environment of autonomous vehicle 502. Camera 546j may be a still image infrared image camera or a motion infrared camera. Rain sensor 546i may include one or more devices configured to detect a liquid level (e.g., raindrops, humidity) on a sensing area of ​​rain sensor 546i.

[0130] Vehicle control subsystem 548 may be configured to control the operation of autonomous vehicle 502 and its components. Thus, vehicle control subsystem 548 may include various elements such as throttle and gear selector 548a, brake unit 548b, navigation unit 548c, steering system 548d, and / or autonomous control unit 548e. Throttle and gear selector 548a may be configured, for example, to control the operating speed of an engine, which in turn controls the speed of autonomous vehicle 502. Throttle and gear selector 548a may be configured to control gear selection in a transmission. Brake unit 548b may include any combination of mechanisms configured to slow autonomous vehicle 502. Brake unit 548b may slow autonomous vehicle 502 in a standard manner, including using friction or engine braking to slow the wheels. Brake unit 548b may include an anti-lock braking system (ABS) that may prevent the brakes from locking when the brakes are applied. Navigation unit 548c may be any system configured to determine a driving path or route for autonomous vehicle 502. Navigation unit 548c may additionally be configured to dynamically update the driving path while autonomous vehicle 502 is operating. In some embodiments, navigation unit 548c may be configured to incorporate data from GPS transceiver 546g and one or more pre-defined maps to determine a driving path for autonomous vehicle 502. Steering system 548d may represent any combination of mechanisms that may be operable to adjust the heading of autonomous vehicle 502 in autonomous mode or driver-controlled mode.

[0131] Autonomous control unit 548e may represent a control system configured to identify, evaluate, and avoid or otherwise negotiate potential obstacles or obstructions in the environment of autonomous vehicle 502. In general, autonomous control unit 548e may be configured to control autonomous vehicle 502 for driverless operation or to provide driver assistance in controlling autonomous vehicle 502. In some embodiments, autonomous control unit 548e may be configured to incorporate data from GPS transceiver 546g, radar unit 546b, LiDAR unit 546f, camera 546a, and / or other vehicle subsystems to determine a driving path or trajectory for autonomous vehicle 502.

[0132] Network communications subsystem 592 may comprise network interfaces such as routers, switches, modems, etc. Network communications subsystem 592 may be configured to establish communications between autonomous vehicle 502 and other systems, servers, etc. Network communications subsystem 592 may be further configured to send and receive data to and from other systems.

[0133] Many or all of the functions of autonomous vehicle 502 can be controlled by onboard control computer 550. Onboard control computer 550 may include at least one data processor 570 (which may include at least one microprocessor) that executes processing instructions 580 stored in a data storage device 590 or a non-transitory computer-readable medium such as memory. Onboard control computer 550 may also represent multiple computing devices that may be responsible for controlling individual components or subsystems of autonomous vehicle 502 in a distributed manner. In some embodiments, data storage device 590 may include processing instructions 580 (e.g., program logic) executable by data processor 570 to implement various methods and / or functions of autonomous vehicle 502, including those described with respect to FIGS. 1-7 .

[0134] Data storage device 590 may similarly include additional instructions, including instructions for transmitting data to, receiving data from, interacting with, or controlling one or more of vehicle driving subsystem 542, vehicle sensor subsystem 544, and vehicle control subsystem 548. Onboard control computer 550 may be configured to include data processor 570 and data storage device 590. Onboard control computer 550 may control functions of autonomous vehicle 502 based on inputs received from various vehicle subsystems (e.g., vehicle driving subsystem 542, vehicle sensor subsystem 544, and vehicle control subsystem 548).

[0135] FIG. 6 shows an example system 600 for providing precise autonomous driving operation. System 600 may include several modules that may operate within onboard control computer 550, as described in FIG. 5. Onboard control computer 550 may include sensor fusion module 602, shown in the upper left corner of FIG. 6, which may perform at least four image or signal processing operations. Sensor fusion module 602 may acquire images from cameras located on the autonomous vehicle, perform image segmentation 604, and detect the presence of moving objects (e.g., other vehicles, pedestrians, etc.) and / or static obstacles (e.g., stop signs, speed bumps, terrain, etc.) located around the autonomous vehicle. Sensor fusion module 602 may acquire LiDAR point cloud data items from LiDAR sensors located on the autonomous vehicle, perform LiDAR segmentation 606, and detect the presence of objects and / or obstacles located around the autonomous vehicle. The sensor fusion module 602 may obtain the illumination level 340 (see FIG. 3) from the light control fusion module 330 (see FIG. 3). The sensor fusion module 602 may use the illumination level 340 to perform one or more functions described herein, such as determining the quality of the image and infrared image and detecting objects from the image and infrared image.

[0136] The sensor fusion module 602 may perform instance segmentation 608 on the images and / or point cloud data items to identify contours (e.g., boxes) around objects and / or obstacles located around the autonomous vehicle. The sensor fusion module 602 may perform temporal fusion 610 in which objects and / or obstacles from one image and / or frame of a point cloud data item are correlated or associated with objects and / or obstacles from one or more images or frames received subsequently in time.

[0137] The sensor fusion module 602 may fuse objects and / or obstacles from images acquired from cameras and / or point cloud data items acquired from a LiDAR sensor. For example, the sensor fusion module 602 may determine, based on the locations of two cameras, that an image from one of the cameras comprising one half of a vehicle located in front of the autonomous vehicle is the same as the vehicle captured by another camera. The sensor fusion module 602 may send the fused object information to the tracking and prediction module 646 and the fused obstacle information to the occupancy grid module 660. The onboard control computer may include the occupancy grid module 660, which may retrieve landmarks from a map database 658 stored within the onboard control computer. The occupancy grid module 660 may determine a drivable area and / or obstacles from the fused obstacles acquired from the sensor fusion module 602 and the landmarks stored in the map database 658. For example, the occupancy grid module 660 may determine that the drivable area may include a speed bump obstacle.

[0138] Under the sensor fusion module 602, the onboard control computer 550 may include a LiDAR-based object detection module 612 that may perform object detection 616 based on point cloud data items acquired from a LiDAR sensor 614 located on the autonomous vehicle. The object detection 616 technique may provide a location (e.g., in 3D world coordinates) of an object from the point cloud data items. Under the LiDAR-based object detection module 612, the onboard control computer may include an image-based object detection module 618 that may perform object detection 624 based on images acquired from a camera 620 located on the autonomous vehicle. The object detection 618 technique may use deep machine learning techniques 624 to provide a location (e.g., in 3D world coordinates) of an object from images provided by the camera 620.

[0139] A radar 656 on the autonomous vehicle may scan the area in front of the autonomous vehicle or the area toward which the autonomous vehicle is being driven. The radar data may be transmitted to a sensor fusion module 602, which may use the radar data to correlate objects and / or obstacles detected by the radar 656 with objects and / or obstacles detected from both the LiDAR point cloud data items and the camera images. The radar data may also be transmitted to a tracking and prediction module 646, which may perform data processing on the radar data to track objects via an object tracking module 648, as described further below.

[0140] The onboard control computer may include a tracking and prediction module 646 that receives the locations of objects from the point cloud and images and the fused objects from the sensor fusion module 602. The tracking and prediction module 646 may also receive radar data with which the tracking and prediction module 646 can track objects, via an object tracking module 648, from one point cloud data item and one image acquired at one time instance to another (or next) point cloud data item and another image acquired at another subsequent time instance.

[0141] The tracking and prediction module 646 may perform object attribute estimation 650 to estimate one or more attributes of an object detected in an image or point cloud data item. The one or more attributes of an object may include the type of object (e.g., pedestrian, car, truck, etc.). The tracking and prediction module 646 may perform behavior prediction 652 to estimate or predict the movement pattern of an object detected in the image and / or point cloud. The behavior prediction 652 may be performed to detect the location of an object within a set of images received at different times (e.g., sequential images) or a set of point cloud data items received at different times (e.g., sequential point cloud data items). In some embodiments, the behavior prediction 652 may be performed for each image received from a camera and / or each point cloud data item received from a LiDAR sensor. In some embodiments, the tracking and prediction module 646 can be implemented (e.g., launched or executed) to reduce computational load by performing behavior prediction 652 on every other image received from a camera or every other point cloud data item received from a LiDAR sensor, or every other image or every other point cloud data item received from a predetermined number of cameras or LiDAR sensors (e.g., every two images or every three point cloud data items).

[0142] The behavior prediction 652 feature may determine the speed and direction of objects surrounding the autonomous vehicle from the radar data, and the speed and direction information can be used to predict or determine the object's motion pattern. The motion pattern may comprise predicted trajectory information of the object over a predetermined amount of time into the future after images are received from the camera. Based on the predicted motion pattern, the tracking and prediction module 646 may assign a motion pattern context tag to the object (e.g., "located at coordinate (x,y)," "parked," "driving at 50 mph," "accelerating," or "decelerating"). The context tag may describe the object's motion pattern. The tracking and prediction module 646 may send one or more object attributes (e.g., object type) and the motion pattern context tag to the planning module 662. The tracking and prediction module 646 may perform environment analysis 654 using any information obtained by the system 600 and any number or combination of its components.

[0143] The on-board control computer may include a planning module 662 that receives object attribute and motion pattern situation tags, drivable areas and / or obstacles from the tracking and prediction module 646, and vehicle location and attitude information from the fusion localization module 626 (described further below).

[0144] The planning module 662 may perform navigation planning 664 to determine a set of trajectories on which the autonomous vehicle may be driven. The set of trajectories may be determined based on drivable area information, one or more object attributes of the object, the object's motion pattern situation tag, the location of obstacles, and the drivable area information. In some embodiments, the navigation planning 664 may include determining an area adjacent to a road where the autonomous vehicle may be safely parked in the event of an emergency. The planning module 662 may include behavior decision-making 666 to determine driving actions (e.g., steering, braking, throttle) in response to determining changing conditions on the road (e.g., a traffic light has turned yellow, or the autonomous vehicle is experiencing unsafe driving conditions because another vehicle has cut in front of the autonomous vehicle and entered an area within a predetermined safe distance of the autonomous vehicle's location). The planning module 662 may perform trajectory generation 668 to select a trajectory from the set of trajectories determined by the navigation planning operation 664. The selected trajectory information may be transmitted by the planning module 662 to the control module 670.

[0145] The onboard control computer may include a control module 670 that receives the proposed trajectory from the planning module 662 and the autonomous vehicle location and attitude from the fusion localization module 626. The control module 670 may include a system identifier 672. The control module 670 may perform model-based trajectory refinement 674 to refine the proposed trajectory. For example, the control module 670 may apply filtering (e.g., a Kalman filter) to smooth the proposed trajectory data and / or minimize noise. The control module 670 may perform robust control 676 by determining the amount of brake pressure to apply, the steering angle, the throttle amount to control the vehicle's speed, and / or the transmission gear based on the refined proposed trajectory information and the autonomous vehicle's current location and / or attitude. The control module 670 may transmit the determined brake pressure, steering angle, throttle amount, and / or transmission gear to one or more devices within the autonomous vehicle to control and facilitate precise driving behavior of the autonomous vehicle.

[0146] Deep image-based object detection 624 performed by image-based object detection module 618 can also be used to detect landmarks on the road (e.g., stop signs, speed bumps, etc.). The onboard control computer may include a fusion localization module 626, which obtains landmarks detected from images, landmarks obtained from a map database 636 stored on the onboard control computer, landmarks detected from point cloud data items by LiDAR-based object detection module 612, speed and displacement from an odometer sensor 644, and an estimated location of the autonomous vehicle from GPS / IMU sensors 638 (e.g., GPS sensor 640 and IMU sensor 642) located on or in the autonomous vehicle. Based on this information, fusion localization module 626 can perform localization operations 628 and determine the location of the autonomous vehicle, which can be transmitted to planning module 662 and control module 670.

[0147] The fusion localization module 626 can estimate the pose 630 of the autonomous vehicle based on GPS and / or IMU sensors 638. The pose of the autonomous vehicle can be transmitted to the planning module 662 and the control module 670. The fusion localization module 626 can also estimate the status (e.g., location, possible movement angles) of the trailer unit (e.g., trailer status estimation 634), for example, based on information (e.g., angular velocity and / or linear velocity) provided by the IMU sensors 642. The fusion localization module 626 may also check map content 632.

[0148] FIG. 7 shows an example block diagram of an onboard control computer 550 included in an autonomous vehicle 502. The onboard control computer 550 may include at least one processor 704 and a memory 702 having stored instructions (e.g., software instructions 128 and process instructions 580 of FIGS. 1 and 5, respectively). The instructions, when executed by the processor 704, configure the onboard control computer 550 and / or various modules of the onboard control computer 550 to perform the operations described in FIGS. 1-6. The transmitter 706 may transmit or send information or data to one or more devices within the autonomous vehicle. For example, the transmitter 706 may send instructions to one or more motors of a steering wheel to steer the autonomous vehicle. The receiver 708 may receive information or data transmitted or sent by one or more devices. For example, the receiver 708 may receive the status of the current speed from an odometer sensor or the current transmission gear from a transmission. The transmitter 706 and receiver 708 may also be configured to communicate with multiple vehicle subsystems 540 and on-board control computers 550 described above in FIGS.

[0149] While several embodiments are provided in this disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the disclosure. The examples are to be considered illustrative, not restrictive, and the intention is not to be limited to the details provided herein. For example, various elements or components may be combined or integrated into another system, or certain features may be omitted, or not implemented.

[0150] Additionally, the techniques, systems, subsystems, and methods described and illustrated in various embodiments, either separately or distinctly, may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present disclosure. Other items shown or discussed as coupled or directly coupled or in communication with each other may also be indirectly coupled or communicate through some interface, device, or intermediate component, whether electrically, mechanically, or otherwise. Other examples of changes, substitutions, and alterations are ascertainable by those skilled in the art and could be made without departing from the spirit and scope disclosed herein.

[0151] In order to assist the Patent Office and any reader of any patent issued hereto in interpreting the claims appended hereto, Applicant emphasizes that Applicant does not intend any of the appended claims to invoke 35 U.S.C. § 112(f) as they exist as of the filing date hereof, unless the words "means for" or "step for" are expressly used in a particular claim.

[0152] Implementations of the present disclosure can be described in terms of the following notes, the features of which can be combined in any reasonable manner.

[0153] Appendix 1. A system, comprising: a headlamp mounted on the autonomous vehicle and configured to illuminate at least a portion of a roadway along which the autonomous vehicle is located; a control device associated with the autonomous vehicle and including at least one processor; Equipped with The control device includes at least: Obtaining information about an environment surrounding the autonomous vehicle; determining that at least a portion of the road should be illuminated if the information indicates that the illumination level of the portion of the road is below a threshold illumination level; adjusting the headlights to illuminate at least a portion of the road in response to determining that at least a portion of the road should be illuminated; A system that is configured to:

[0154] Appendix 2. Information is Road curve indication, current time, autonomous vehicle location, light sensor data captured by at least one light sensor; an image of at least a portion of a road captured by a camera associated with the autonomous vehicle; or an infrared image of at least a portion of a roadway captured by an infrared camera associated with the autonomous vehicle; 2. The system of claim 1, comprising one or more of:

[0155] Appendix 3. The system of Appendix 1, wherein the information comprises a road curvature indication indicating that the road ahead of the autonomous vehicle is curving to the left, and wherein to adjust the headlights, the at least one processor is further configured to turn on at least a left light source of the headlights.

[0156] Clause 4. The information comprises a current time, and to determine that at least a portion of the road should be illuminated, the at least one processor includes at least: determining that the current time is nighttime; adjusting headlights by turning on infrared light sources associated with the autonomous vehicle; 2. The system of claim 1, further configured to:

[0157] Clause 5. The information comprises the location of the autonomous vehicle; At least one processor includes at least: comparing the location of the autonomous vehicle with map data comprising the location and type of road ahead of the autonomous vehicle; determining that the autonomous vehicle is approaching a tunnel based at least in part on a comparison between a location of the autonomous vehicle and the map data; Before an autonomous vehicle enters a tunnel, it must turn on its headlights. 2. The system of claim 1, further configured to:

[0158] Appendix 6. At least one processor includes at least: determining that the autonomous vehicle is exiting a tunnel based on at least a second location of the autonomous vehicle and the map data; After an autonomous vehicle exits a tunnel, it turns off its headlights. 6. The system of claim 5, further configured to:

[0159] Appendix 7. The system of Appendix 1, wherein the information comprises light sensor data captured by at least one light sensor, the at least one light sensor configured to detect an illumination level of an environment surrounding the at least one light sensor, and wherein, to determine that at least a portion of the road should be illuminated, the at least one processor is further configured to determine that an illumination level of at least the road is below a threshold illumination level.

[0160] Appendix 8. A method, comprising: Obtaining information about an environment surrounding the autonomous vehicle; determining that at least a portion of a road on which the autonomous vehicle is located should be illuminated if the information indicates that an illumination level of the portion of the road is below a threshold illumination level; adjusting the headlights to illuminate at least a portion of the road in response to determining that at least a portion of the road should be illuminated; Including, A method, wherein the headlights are mounted on the autonomous vehicle and configured to illuminate at least a portion of a road.

[0161] Clause 9. The information comprises an image of at least a portion of a road, and the method further comprises: determining a first lighting condition level for an image; determining that a first light condition level of the image is below a threshold light condition level; In response to determining that the first light condition level of the image is less than the threshold light condition level, adjusting the headlights to increase the illumination level of the roadway until the first light condition level is at least equal to the threshold light condition level. 9. The method of claim 8, further comprising:

[0162] Appendix 10. The method of Appendix 9, wherein determining a first light condition level of the image includes determining a color of pixels of the image, and determining that the first light condition level of the image is below a threshold light condition level includes determining that a greater than a threshold percentage of pixels of the image are black.

[0163] Clause 11. The information comprises an image of at least a portion of an infrared roadway, and the method further comprises: determining a second light condition level for the thermal image; determining that a second light condition level of the thermal image is less than a threshold light condition level; In response to determining that the second light condition level of the thermal image is less than the threshold light condition level, adjusting the headlights to increase the illumination level of the roadway until the second light condition level is at least equal to the threshold light condition level. 9. The method of claim 8, further comprising:

[0164] Appendix 12. The method of Appendix 11, wherein determining a second light condition level of the infrared image includes determining a color of pixels of the infrared image, and determining that the second light condition level of the infrared image is below a threshold light condition level includes determining that a greater than threshold percentage of pixels of the infrared image are black.

[0165] Appendix 13. The method of Appendix 8, wherein adjusting the headlights to illuminate at least a portion of the roadway includes one or more of turning on a low beam light source, turning on a high beam light source, turning on a left light source, turning on a right light source, and increasing the intensity of infrared light emitted from at least one infrared light source.

[0166] Clause 14. Receiving sensor data captured by at least one sensor associated with an autonomous vehicle; Detecting a lack of oncoming traffic based on the received sensor data; turning on one or more of a headlight high beam infrared light source and a headlight high beam light source; determining a quality of an infrared image captured by an infrared camera associated with the autonomous vehicle; determining that the quality of the thermal image is below a threshold thermal image quality; increasing the intensity of the infrared light emitted from the high beam infrared light source until the quality of the infrared image is at least equal to a threshold infrared image quality; 9. The method of claim 8, further comprising:

[0167] Clause 15. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, Obtaining information about an environment surrounding the autonomous vehicle; determining that at least a portion of a road on which the autonomous vehicle is located should be illuminated if the information indicates that an illumination level of the portion of the road is below a threshold illumination level; adjusting the headlights to illuminate at least a portion of the road in response to determining that at least a portion of the road should be illuminated; by at least one processor; A non-transitory computer-readable medium, wherein the headlamp is mounted on an autonomous vehicle and configured to illuminate at least a portion of a roadway.

[0168] Clause 16. The instructions, when executed by at least one processor, at least: receiving sensor data captured by at least one sensor associated with the autonomous vehicle; Detecting the presence of oncoming traffic based on the received sensor data; in response to oncoming traffic being detected, turning on one or more of a headlamp low beam infrared light source and a headlamp low beam light source; determining a quality of an infrared image captured by an infrared camera associated with the autonomous vehicle; determining that the quality of the thermal image is below a threshold thermal image quality; increasing the intensity of the infrared light emitted from the low-beam infrared light source until the quality of the infrared image is at least equal to a threshold infrared image quality; 16. The non-transitory computer-readable medium of claim 15, further causing at least one processor to:

[0169] Clause 17. The non-transitory computer-readable medium of Clause 16, wherein the at least one sensor comprises at least one of a camera, a light detection and ranging sensor, and an infrared sensor.

[0170] Clause 18. The instructions, when executed by at least one processor, at least: determining that the autonomous vehicle is approaching a left turn location; turning on one or more of a left infrared light source and a left light source; determining a quality of an infrared image captured by an infrared camera associated with the autonomous vehicle; determining that the quality of the thermal image is below a threshold thermal image quality; increasing the intensity of the infrared light emitted from the left infrared light source until the quality of the infrared image is at least equal to the threshold infrared image quality; 16. The non-transitory computer-readable medium of claim 15, further causing at least one processor to:

[0171] Clause 19. The instructions, when executed by at least one processor, at least: determining that the autonomous vehicle is approaching a right turn location; turning on one or more of a right infrared light source and a right light source; determining a quality of an infrared image captured by an infrared camera associated with the autonomous vehicle; determining that the quality of the thermal image is below a threshold thermal image quality; increasing the intensity of the infrared light emitted from the right infrared light source until the quality of the infrared image is at least equal to the threshold infrared image quality; 16. The non-transitory computer-readable medium of claim 15, further causing at least one processor to:

[0172] Clause 20. The non-transitory computer-readable medium of Clause 15, wherein the obtained information further comprises rain sensor data captured by at least one rain sensor, and wherein, to determine that at least a portion of the roadway should be illuminated, the at least one processor is further configured to determine that at least the rain sensor data indicates a rainfall level that is higher than a threshold rainfall level.

[0173] Appendix 21. The system of any of Appendixes 1-7, wherein the processor is further configured to perform one or more operations of any of Appendixes 8-14.

[0174] Addendum 22. The system of any of Addendums 1-7, wherein the processor is further configured to cause one or more of the operations of any of Addendums 15-20 to be performed.

[0175] Appendix 23. The method of any of Appendixes 8-14, further comprising one or more operations of any of Appendixes 1-7.

[0176] Clause 24. The method of any of Clauses 8-14, further comprising one or more operations of any of Clauses 15-20.

[0177] Clause 25. An apparatus comprising means for performing one or more of the operations set forth in any of Clauses 8-14.

[0178] Appendix 26. The non-transitory computer-readable medium of any of Appendixes 15-20, the instructions, when executed by at least one processor, further causing the at least one processor to perform one or more operations of any of Appendixes 1-7.

[0179] Appendix 27. The non-transitory computer-readable medium of any of Appendixes 15-20, the instructions, when executed by at least one processor, further causing the at least one processor to perform one or more operations of any of Appendixes 8-14.

Claims

1. A system (100) comprising: a headlamp (110) mounted on an autonomous vehicle (502) and configured to illuminate at least a portion of a road (102) on which the autonomous vehicle (502) is located; a control device (550) associated with the autonomous vehicle (502) and comprising at least one processor (122, 570); Equipped with The control device includes at least Obtaining information (310) about an environment surrounding the autonomous vehicle (502); determining that at least a portion of the roadway (102) should be illuminated if the information (310) indicates that a lighting level (340) of the portion of the roadway (102) is below a threshold lighting level (342); in response to determining that at least the portion of the roadway (102) should be illuminated, adjusting the headlights (112) to illuminate at least the portion of the roadway (102); A system configured to:

2. The information (310) is Road curvature indication, Current time, a location of the autonomous vehicle (502); optical sensor data captured by at least one optical sensor (310d); an image (310e) of at least the portion of the road (102) captured by a camera (546a) associated with the autonomous vehicle (502); or an infrared image (310f) of at least the portion of the road (102) captured by an infrared camera (546j) associated with the autonomous vehicle (502); The system of claim 1 , comprising one or more of:

3. 2. The system of claim 1, wherein the information (310) comprises a road curvature indication indicating that the road (102) ahead of the autonomous vehicle (502) is curving to the left, and to adjust the headlights, the at least one processor (122, 570) is further configured to turn on at least a left side light source (202b) of the headlight (110).

4. The information (310) comprises a current time, and to determine that at least the portion of the road (102) should be illuminated, the at least one processor (122, 570) comprises at least: determining that the current time is nighttime; adjusting the headlights (110) by turning on an infrared light source associated with the autonomous vehicle; The system of claim 1 , further configured to:

5. the information (310) comprising a location of the autonomous vehicle (502); The at least one processor (122, 570) includes at least comparing the location of the autonomous vehicle (502) to map data (142) comprising the location and type of roads ahead of the autonomous vehicle (502); determining that the autonomous vehicle (502) is approaching a tunnel (104) based at least in part on a comparison between the location of the autonomous vehicle (502) and the map data (142); before the autonomous vehicle (502) enters the tunnel (104), turning on the headlights (110); The system of claim 1 , further configured to:

6. The at least one processor (122, 570) includes at least determining that the autonomous vehicle (502) is exiting the tunnel (104) based on at least a second location of the autonomous vehicle (502) and the map data (142); turning off the headlights (110) after the autonomous vehicle (502) exits the tunnel (104); The system of claim 5 , further configured to:

7. 2. The system of claim 1, wherein the information (310) comprises light sensor data (310d) captured by at least one light sensor (546k), the at least one light sensor (564k) configured to detect a lighting level of the environment surrounding the at least one light sensor (546k), and wherein to determine that at least the portion of the road (102) should be illuminated, the at least one processor (122, 570) is further configured to determine that the lighting level (340) of at least the road (102) is lower than the threshold lighting level (342).

8. 1. A method (400), comprising: Obtaining (402) information (310) about an environment surrounding an autonomous vehicle (502); determining (404) that at least the portion of the road (102) on which the autonomous vehicle (502) is located should be illuminated if the information (310) indicates that the illumination level (340) of the portion of the road is below a threshold illumination level (342); In response to determining that at least the portion of the roadway (102) should be illuminated, adjusting headlights (110) to illuminate (408) at least the portion of the roadway (102); Including, The method of claim 1, wherein the headlights (110) are mounted on the autonomous vehicle (502) and configured to illuminate at least a portion of the roadway (102).

9. The information (310) comprises an image (310e) of at least the portion of the road (102), and the method further comprises: determining a first lighting condition level for the image; determining that the first light condition level (316b) of the image (310e) is below a threshold light condition level (150); in response to determining that the first light condition level (316b) of the image (310e) is lower than the threshold light condition level (150), adjusting the headlights (110) to increase the illumination level (340) of the roadway (102) until the first light condition level (316b) is at least equal to the threshold light condition level (150); The method of claim 8 , further comprising:

10. 10. The method of claim 9, wherein determining the first light condition level (316b) of the image (310e) comprises determining a color of pixels of the image (310e), and determining that the first light condition level (316b) of the image (310e) is below the threshold light condition level (150) comprises determining that a greater than a threshold percentage of pixels of the image are black.

11. The information (310) comprises an infrared image (310f) of at least the portion of the road (102), and the method further comprises: determining a second light condition level (316c) of the thermal image (310f); determining that the second light condition level (316c) of the thermal image (310f) is below a threshold light condition level (150); in response to determining that the second light condition level (316c) of the thermal image (310f) is lower than the threshold light condition level (150), adjusting the headlights (110) to increase the illumination level (340) of the roadway (102) until the second light condition level (316c) is at least equal to the threshold light condition level (150); The method of claim 8 , further comprising:

12. 12. The method of claim 11, wherein determining the second light condition level (316c) of the infrared image (310f) includes determining a color of pixels of the infrared image (310f), and determining that the second light condition level (316c) of the infrared image (310f) is below the threshold light condition level (316c) includes determining that a greater than a threshold percentage of pixels of the infrared image (310f) are black.

13. 9. The method of claim 8, wherein adjusting the headlights (110) to illuminate at least the portion of the road (102) comprises one or more of turning on a low beam light source, turning on a high beam light source, turning on a left light source (202b), turning on a right light source (202a), and increasing an intensity of infrared light emitted from at least one infrared light source.

14. receiving sensor data (130) captured by at least one sensor (546) associated with the autonomous vehicle (502); detecting a lack of oncoming traffic based on the received sensor data (130); turning on one or more of a high beam infrared light source of the headlight (110) and a high beam light source of the headlight (110); determining a quality (136) of an infrared image (310f) captured by an infrared camera (546j) associated with the autonomous vehicle (502); determining that the quality (136) of the thermal image (310f) is below a threshold thermal image quality (138); increasing an intensity of infrared light emitted from said high beam infrared light source until said quality (136) of said infrared image (310f) is at least equal to said threshold infrared image quality (138); The method of claim 8 , further comprising:

15. A non-transitory computer readable medium (126, 590) storing instructions (128), which when executed by at least one processor (122, 570) perform: Obtaining information (310) about an environment surrounding an autonomous vehicle (502); determining that at least a portion of a road (102) on which the autonomous vehicle (502) is located should be illuminated if the information indicates that an illumination level (340) of the portion of the road (102) is below a threshold illumination level (342); in response to determining that at least the portion of the roadway (102) should be illuminated, adjusting headlights (110) to illuminate at least the portion of the roadway (102); by said at least one processor (122, 570); The headlamp (110) is mounted on the autonomous vehicle (502) and configured to illuminate at least a portion of the roadway (102).

16. The instructions (128), when executed by the at least one processor (122, 570), perform at least: receiving sensor data (130) captured by at least one sensor (546) associated with the autonomous vehicle (502); detecting the presence of oncoming traffic based on the received sensor data (130); in response to the oncoming traffic being detected, turning on one or more of a low beam infrared light source of the headlight (110) and a low beam light source of the headlight (110); determining a quality (136) of an infrared image (310f) captured by an infrared camera (546j) associated with the autonomous vehicle (502); determining that the quality (136) of the thermal image (310f) is below a threshold thermal image quality (138); increasing an intensity of infrared light emitted from said low-beam infrared light source until said quality of said infrared image (310f) is at least equal to said threshold infrared image quality (138); 20. The non-transitory computer readable medium of claim 15, further causing the at least one processor (122, 570) to:

17. 20. The non-transitory computer readable medium of claim 16, wherein the at least one sensor (546) comprises at least one of a camera, a light detection and ranging sensor, and an infrared sensor.

18. The instructions (128), when executed by the at least one processor (122, 570), perform at least: determining that the autonomous vehicle (502) is approaching a left turn location; turning on one or more of the left infrared light source (202b) and the left light source (202b); determining a quality (136) of an infrared image (310f) captured by an infrared camera (546j) associated with the autonomous vehicle (502); determining that the quality (136) of the thermal image (310f) is below a threshold thermal image quality (138); increasing an intensity of infrared light emitted from the left infrared light source (202b) until the quality (136) of the infrared image (310f) is at least equal to the threshold infrared image quality (138); 20. The non-transitory computer readable medium of claim 15, further causing the at least one processor (122, 570) to:

19. The instructions (128), when executed by the at least one processor (122, 570), perform at least: determining that the autonomous vehicle (502) is approaching a right turn location; turning on one or more of the right infrared light source (202a) and the right light source (202a); determining a quality (136) of an infrared image (130f) captured by an infrared camera (546j) associated with the autonomous vehicle (502); determining that the quality (136) of the thermal image (130f) is below a threshold thermal image quality (138); increasing an intensity of infrared light emitted from the right infrared light source (202a) until the quality (136) of the infrared image (310f) is at least equal to the threshold infrared image quality (138); 20. The non-transitory computer readable medium of claim 15, further causing the at least one processor (122, 570) to:

20. 16. The non-transitory computer readable medium of claim 15, wherein the acquired information (310) further comprises rain sensor data (310g) captured by at least one rain sensor (546i), and wherein to determine that at least the portion of the road (102) should be illuminated, the at least one processor (122, 570) is further configured to determine that at least the rain sensor data (310g) indicates a rainfall level (320) that is higher than a threshold rainfall level.

21. The system of any of claims 1-7, wherein the processor (122, 570) is further configured to perform one or more operations of any of claims 8-14.

22. The system of any of claims 1-7, wherein the processor (122, 570) is further configured to perform one or more operations of any of claims 15-20.

23. The method of any of claims 8-14, further comprising one or more of the operations of any of claims 1-7.

24. The method of any of claims 8-14, further comprising one or more operations as recited in any of claims 15-20.

25. Apparatus comprising means for performing one or more of the operations recited in any one of claims 8-14.

26. A non-transitory computer-readable medium (126, 590) according to any of claims 15-20, wherein the instructions, when executed by the at least one processor (122, 570), further cause the at least one processor (122, 570) to perform one or more operations according to any of claims 1-7.

27. A non-transitory computer-readable medium (126, 590) according to any one of claims 15-20, wherein the instructions, when executed by the at least one processor (122, 570), further cause the at least one processor (122, 570) to perform one or more operations according to any one of claims 8-14.

Images