System and method for vehicle monitoring of infrastructure lighting

Abstract

The present disclosure provides "VEHICLE MONITORING OF INFRASTRUCTURE LIGHTING". Methods and apparatus for vehicle monitoring of infrastructure lighting are disclosed. An example vehicle includes a camera, a vehicle-to-vehicle communication module, and a controller. The controller is used to identify a fixed infrastructure object within an image captured by the camera, and in response to determining that the fixed infrastructure object includes a light, identify whether the light is inoperable. The controller is also used to send an alert to an infrastructure communication node indicating that the light is inoperable via vehicle-to-infrastructure communication using the vehicle-to-vehicle communication module.

Description

Technical Field

[0001] The present disclosure relates generally to infrastructure and, more particularly, to vehicular monitoring of infrastructure lighting. Background Art

[0002] Typically, infrastructure systems utilize lighting to facilitate vehicle travel through the infrastructure system. For example, street lights are often located on the side of a road to facilitate vehicle travel along the road when it is dark. Traffic lights are generally located at intersections to manage traffic through the intersection. In addition, signs (e.g., traffic signs, commercial signs) are located on the side of the road to facilitate the driver of the vehicle to navigate through the infrastructure system. In addition, vehicles typically include exterior lights to facilitate the vehicle's travel along the road when it is dark and / or to warn other vehicles that it is braking and / or turning. Summary of the invention

[0003] The appended claims define the application. This disclosure summarizes aspects of the embodiments and should not be used to limit the claims. Other implementations are contemplated based on the techniques described herein, as will be apparent to one of ordinary skill in the art upon studying the following drawings and detailed description, and these implementations are intended to fall within the scope of the present application.

[0004] Example embodiments for vehicle monitoring of infrastructure lighting are shown. A disclosed example vehicle includes a camera, a vehicle-to-vehicle communication module, and a controller. The controller is configured to identify a fixed infrastructure object within an image captured by the camera, and in response to determining that the fixed infrastructure object includes a light, identify whether the light is inoperable. The controller is further configured to send an alert to an infrastructure communication node via vehicle-to-infrastructure communication using the vehicle-to-vehicle communication module indicating that the light is inoperable.

[0005] Some examples also include a proximity sensor for further detecting the presence of the fixed infrastructure object.In some examples, to identify the fixed infrastructure object and the operability of the light, the controller utilizes machine learning to perform image recognition.

[0006] In some examples, the controller monitors light emission of the lamp to determine if the lamp is inoperable. Some such examples also include a light sensor configured to collect light emission measurements to enable the controller to monitor the light emission of the lamp. In some such examples, to monitor the light emission of the lamp, the controller processes the light distribution of the lamp using optical frequency filtering. In some such examples, the controller determines that the lamp is inoperable in response to determining that one or more LEDs of the lamp are inoperable based on the light emission of the lamp.

[0007] Some examples further include: a second communication module configured to send the alert to a server via a cellular network; and a GPS receiver configured to identify a vehicle location. In such examples, the controller includes in the alert the vehicle location at which the camera captured the image including the fixed infrastructure object, so as to identify the location of the inoperable light.

[0008] In some examples, the controller identifies that the fixed infrastructure object is a commercial sign. The light of the lamp post is a first street light configured to illuminate a portion of a road. Some such examples also include a light sensor for monitoring ambient light. In such examples, the controller is configured to identify the operability of the first street light in response to the light sensor detecting that the ambient light is below an ambient light threshold. In some such examples, when the controller identifies that the lamp post includes the first street light and a second street light, the controller determines the operability of the first street light by comparing the light emissions of the first street light and the second street light. In some such examples, the controller is configured to: identify a pattern of lamp posts along the road based on a plurality of images captured by the camera; determine an illumination pattern corresponding to the pattern of lamp posts; and identify that the first street light of the lamp post is inoperable in response to determining that the illumination measurement of the lamp post does not correspond to the illumination pattern.

[0009] In some examples, the controller identifies that the fixed infrastructure object is a first traffic light including a traffic light. In such examples, each of the traffic lights includes a first light for indicating that traffic is proceeding, a second light for indicating that the traffic is yielding, and a third light for indicating that the traffic is stopped. In some such examples, when the controller identifies that the intersection includes the first traffic light and the second traffic light, the controller determines the operability of one or more of the traffic lights of the first traffic light by comparing the light emissions of the first traffic light and the second traffic light. In some such examples, the controller is configured to: retrieve an illumination pattern of the traffic light; and identify one of the traffic lights as inoperable in response to determining that a measured illumination sequence of the traffic light does not correspond to the illumination pattern.

[0010] In some examples, the controller identifies the fixed infrastructure object as a commercial sign.

[0011] The disclosed example method includes identifying, via a processor, an infrastructure object within an image captured by a camera of a vehicle. The disclosed example method also includes determining whether the infrastructure object includes a light; and identifying whether the light is inoperable in response to determining that the infrastructure object includes the light. The disclosed example method includes sending an alert to an infrastructure communication node via vehicle-to-infrastructure communication utilizing a vehicle-to-vehicle communication module of the vehicle indicating that the light is inoperable.

[0012] Some examples also include identifying a pattern of lamp posts along the road based on a plurality of images captured by the camera. In such examples, one of the lamp posts is the infrastructure object. Such examples also include determining an illumination pattern corresponding to the pattern of lamp posts; and identifying that the lamp of the infrastructure object is inoperable in response to determining that the illumination measurement of the infrastructure object does not correspond to the illumination pattern.

[0013] Some examples also include identifying an intersection including a first traffic signal and a second traffic signal based on the image captured by the camera. In such examples, the first traffic signal is the infrastructure object. These examples also include determining the operability of the light of the first traffic signal by comparing the light emissions of the first traffic signal and the second traffic signal.

[0014] Some examples also include: identifying that the infrastructure object is a traffic light; retrieving a lighting pattern of the traffic light of the traffic light; and identifying that one of the traffic lights is inoperable in response to determining that a measured lighting sequence of the traffic light does not correspond to the lighting pattern. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] For a better understanding of the present invention, reference may be made to the embodiments illustrated in the following drawings. The components in the drawings are not necessarily drawn to scale, and related elements may be omitted or, in some cases, may be exaggerated in proportion in order to emphasize and clearly illustrate the novel features described herein. In addition, as is known in the art, the system components may be arranged differently. Furthermore, in the drawings, like reference numerals represent corresponding parts in the several views.

[0016] Figure 1 An example vehicle according to the teachings herein is shown.

[0017] Figure 2 Shown by Figure 1 Example image of a lamp post captured by a vehicle's camera.

[0018] Figure 3 Shows Figure 1 of vehicles travel along a road adjacent to multiple lamp posts.

[0019] Figure 4 Shown by Figure 1 Example images of two traffic lights captured by a vehicle's camera.

[0020] FIG. 5A to FIG. 5C Shown by Figure 1 An example sequence of images of traffic lights captured by a vehicle's camera.

[0021] Figure 6 Shows Figure 1 Vehicles are approaching the intersection.

[0022] Figure 7 Shown by Figure 1 Example image of a commercial sign captured by a vehicle's camera.

[0023] Figure 8 Shown by Figure 1 An example image of another vehicle captured by the camera of a vehicle.

[0024] Fig. 9 yes Figure 1 Block diagram of the electronic components of a vehicle.

[0025] Fig.10 is a flow chart for monitoring infrastructure lighting via a vehicle in accordance with the teachings herein. DETAILED DESCRIPTION

[0026] While the invention may be embodied in various forms, some exemplary and non-limiting embodiments are shown in the drawings and will be described below, it being understood that this disclosure is considered as exemplification of the invention and is not intended to limit the invention to the particular embodiments shown.

[0027] Typically, infrastructure systems utilize lighting to facilitate vehicle travel through the infrastructure system. For example, street lights are often located on the side of a road to facilitate vehicle travel along the road when dark. Traffic lights are generally located at intersections to manage traffic through the intersection. In addition, signs (e.g., traffic signs, commercial signs) are located on the side of the road to facilitate the driver of the vehicle to navigate through the infrastructure system. In addition, vehicles typically include exterior lights to facilitate the vehicle travel along the road when dark and / or to warn other vehicles that it is braking and / or turning. When such lighting is inoperable, the vehicle driver's ability to navigate through the infrastructure system may be impeded.

[0028] The examples disclosed herein enable a vehicle to identify when a nearby infrastructure object (e.g., a lamp post, a traffic light, a commercial sign, etc.) and / or a vehicle's lights are inoperable and send an alert to an infrastructure node, a vehicle communication node, and / or a remote server to facilitate the subsequent repair of the inoperable lights. The example methods and devices disclosed herein include a vehicle system that detects when nearby lights are off. The system includes a camera that captures images of nearby objects. The vehicle may also include other sensors (e.g., a radar sensor, an ultrasonic sensor) to further detect the presence of nearby objects. The system uses image recognition to analyze the captured images, for example, to detect when the lights of nearby objects are inoperable. For example, the system is configured to detect when each of the following nearby lights is off: a light (e.g., a taillight) of another vehicle (e.g., when it is recognized that it is dimmed), a brake light of another vehicle (e.g., when it is recognized that the other vehicle is braking), a street light (e.g., when a street light pattern is recognized), a traffic light, a light of a commercial sign, etc. For example, if another vehicle's lights are off, the system sends an alert to the other vehicle via vehicle-to-vehicle (V2V) and / or vehicle-to-infrastructure (V2I) communications using the vehicle-to-vehicle communication module (e.g., dedicated short-range communication (DSRC) module). If a street light or traffic signal is off, the system sends an alert to an infrastructure node via V2I communications using the vehicle-to-vehicle communication module. If a commercial sign's lights are off, the system sends an alert to a remote server. The system can further use crowdsourcing to map intersections and / or other features of the road.

[0029] Go to the attached picture, Figure 1 An example vehicle 100 is shown according to the teachings herein. The vehicle 100 may be a standard gasoline-powered vehicle, a hybrid vehicle, an electric vehicle, a fuel cell vehicle, and / or any other type of vehicle for mobility. The vehicle 100 includes parts related to mobility, such as a powertrain having an engine, a transmission, a suspension, a drive shaft, and / or wheels, etc. The vehicle 100 may be non-autonomous, semi-autonomous (e.g., some routine power functions are controlled by the vehicle 100), or autonomous (e.g., power functions are controlled by the vehicle 100 without direct driver input).

[0030] In the example shown, the vehicle 100 includes one or more cameras 102, one or more light sensors 104, and one or more proximity sensors 106. The camera 102 is configured to capture images and / or videos of the surrounding area of ​​the vehicle 100 for image recognition. In some examples, the camera 102 is configured to measure the luminosity, brightness, color, and / or other characteristics of the ambient light. For example, the camera 102 includes a sensor that detects the characteristics of the ambient light around the camera 102 and / or the vehicle 100 (e.g., luminosity, brightness, color, and / or other characteristics). In addition, the light sensor 104 is configured to detect the characteristics of the ambient light around the light sensor 104 and / or the vehicle 100 (e.g., luminosity, brightness, color, and / or other characteristics). For example, the light sensor 104 is configured to collect light emission measurements in order to monitor the light emitted near the vehicle 100. The proximity sensor 106 is configured to detect the presence, relative position, and / or distance from an object (e.g., a fixed infrastructure object, another vehicle, etc.) positioned in the surrounding area of ​​the vehicle 100. For example, proximity sensor 106 includes a radar sensor that detects and locates objects via radio waves, a lidar sensor that detects and locates objects via lasers, an ultrasonic sensor that detects and locates objects via ultrasonic waves, and / or any other type of sensor configured to detect and locate nearby objects.

[0031] In addition, the illustrated example vehicle 100 includes a navigation system 108 and a global positioning system (GPS) receiver 110. For example, the navigation system 108 is an electronic system that facilitates an operator (e.g., a driver) of the vehicle 100 to navigate the vehicle 100. For example, the navigation system 108 presents a map, provides directions, and / or identifies traffic conditions and / or obstacles to facilitate the driver to navigate from a current location to a target destination. In addition, the GPS receiver 110 receives signals from the global positioning system to identify the current location of the vehicle 100.

[0032] like Figure 1As shown, the vehicle 100 includes a communication module 112, which includes a wired or wireless network interface to enable communication with an external network. The communication module 112 also includes hardware (e.g., processor, memory, storage, antenna, etc.) and software for controlling the wired or wireless network interface. In the example shown, the communication module 112 includes one or more communication controllers for cellular networks (e.g., Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), Long Term Evolution (LTE), Code Division Multiple Access (CDMA)), Near Field Communication (NFC) and / or other standard-based networks (e.g., WiMAX (IEEE 802.16m); Near Field Communication (NFC), Wireless Local Area Network (including IEEE 802.11a / b / g / n / ac or others), Wireless Gigabit (IEEE 802.11ad), etc.). The external network can be: a public network, such as the Internet; a private network, such as an intranet; or a combination thereof, and can utilize various networking protocols that are now available or developed later, including but not limited to TCP / IP-based network protocols.

[0033] The illustrated example vehicle 100 also includes a communication module 114. The communication module 114 is a vehicle-to-vehicle communication module that includes an antenna, a radio, and software for broadcasting messages, and establishes connections between the vehicle 100 and other vehicles via vehicle-to-vehicle (V2V) communications, between the vehicle 100 and infrastructure-based modules via vehicle-to-infrastructure (V2I) communications, and between the vehicle 100 and mobile device-based modules via V2X communications. For more information on vehicle-to-vehicle communication networks and how the network communicates with vehicle hardware and software, see the U.S. Department of Transportation June 2011 System Requirements Specification (SyRS) report (available at http: / / www.its.dot.gov / meetings / pdf / CoreSystem_SE_SyRS_RevA%20(2011-06-13).pdf), the entire contents of which and all documents cited on pages 11 to 14 of the SyRS report are incorporated herein by reference. The vehicle-to-vehicle communication system can be installed on the vehicle and on the infrastructure along the roadside. Vehicle-to-vehicle communication systems that are incorporated into infrastructure (e.g., traffic lights, street lamps, municipal cameras, etc.) are referred to as "roadside" systems or units. Vehicle-to-vehicle communication can be combined with other technologies (such as global positioning systems (GPS), visible light communications (VLC), cellular communications, and short-range radar) to facilitate vehicles to communicate their position, speed, heading, relative position to other objects and exchange information with other vehicles or external computer systems. Vehicle-to-vehicle communication systems can be integrated with other systems such as mobile phones.

[0034] In some examples, the communication module 114 implements a dedicated short range communication (DSRC) protocol. Currently, DSRC networks are identified by the abbreviation or name DSRC. However, other names are sometimes used, which are usually related to connected vehicle programs, etc. Most of these systems are pure DSRC or variants of the IEEE 802.11 wireless standard. However, in addition to pure DSRC systems, dedicated wireless communication systems between cars and roadside infrastructure systems are also meant to be included, which are integrated with GPS and are based on IEEE 802.11 protocols for wireless local area networks (such as 802.11p, etc.). Additionally or optionally, the communication module 114 implements one or more other communication protocols, such as long-term evolution (LTE), 5G communication, etc.

[0035] In addition, the vehicle 100 includes a lighting controller 116 that monitors the operability of lights in the vicinity of the vehicle 100. For example, the lighting controller 116 identifies objects (e.g., vehicles, fixed infrastructure objects) within an image captured via one or more of the cameras 102. For example, the lighting controller 116 utilizes image recognition software to identify objects located within an image captured via the cameras 102. In some examples, the image recognition software identifies boundaries of the objects within the image. In addition, the image recognition software identifies the boundaries corresponding to the objects by comparing the identified boundaries corresponding to the objects to a database (e.g., Fig. 9 The lighting controller 116 identifies objects within the image by comparing them to a database 916 of the vehicle 100 that includes entries that relate object boundaries to known objects. That is, the lighting controller 116 identifies objects within the image via image recognition software by identifying boundaries within the image and comparing these boundaries to boundaries of known objects. In addition, in some examples, the lighting controller 116 utilizes machine learning to perform image recognition on objects located within images captured by one or more of the cameras 102 of the vehicle 100. Additionally or optionally, the lighting controller 116 also identifies and / or detects the presence of an object based on data collected from one or more of the proximity sensors 106 (e.g., a lidar sensor). For example, the lighting controller 116 utilizes data collected from the proximity sensors 106 to identify the shape and / or relative position of an object.

[0036] The lighting controller 116 also determines, via, for example, image recognition software, whether the identified object includes a light (e.g., an exterior light of a vehicle, a light of a fixed infrastructure object). In response to determining that the object includes a light, the lighting controller 116 of the vehicle 100 identifies the operability of the light. For example, to determine whether the light is operable or inoperable, the lighting controller 116 monitors the light emission of the light based on the collected light emission measurements. For example, the light emission measurements and / or lighting measurements include color temperature measurements, relative intensity measurements, time series (e.g., flicker) measurements, etc. In some examples, the lighting controller 116 collects the light emission measurements of the light from images captured by one or more of the cameras 102. Additionally or alternatively, one or more of the light sensors 104 collect the light emission measurements of the light. Furthermore, in some examples, the lighting controller 116 monitors the light emission of the light by processing the light distribution of the light using optical frequency filtering. For example, the lighting controller 116 applies optical frequency filtering techniques to the captured images and / or the collected data to determine the light distribution of the light.

[0037] In addition, in some examples, the lighting controller 116 determines whether the lamp of the identified object is operable, partially inoperable and / or completely inoperable. For example, some lamps are formed by multiple light emitting diodes (LEDs). In such an example, when more of the LEDs of the lamp are operable, the lamp emits more light. That is, the light emission characteristics of the lamp correspond to the number of operable LEDs. The lighting controller 116 is configured to identify the inoperability, partial operability and / or complete operability of the lamp by comparing the light emission measurement of the lamp with the known threshold corresponding to the lamp. In some examples, when identifying an object, the lighting controller 116 controller identifies one or more thresholds of operability of the LED of the lamp corresponding to the object. For example, the lighting controller 116 identifies the upper threshold and the lower threshold of the lamp of the identified object. In response to determining that the light emission measurement is greater than the upper threshold, the lighting controller 116 determines that the lamp is completely operable. That is, the lighting controller 116 determines that the LEDs of the lamp are all operable. In response to determining that the light emission measurement is between the lower threshold and the upper threshold, the lighting controller 116 determines that the lamp is partially operable. That is, the lighting controller 116 determines that some of the LEDs of the lamp are inoperable. In addition, in response to determining that the light emission measurement value is less than the lower threshold, the lighting controller 116 determines that the lamp is inoperable. That is, the lighting controller 116 determines that none of the LEDs of the lamp are inoperable.

[0038] In addition, the lighting controller 116 is configured to send an alert via the communication module 114 to indicate that the light of the identified object is (partially or completely) inoperable. For example, the lighting controller 116 sends the alert to a nearby infrastructure communication node via V2I communication and / or sends the alert to a nearby vehicle via V2V communication. In some examples, the alert issued by the lighting controller 116 identifies the location of the inoperable light and / or the location of the vehicle 100 at which one or more of the cameras 102 capture an image including the object. In some examples, the infrastructure communication node and / or the vehicle communication node relays the alert to a remote server (e.g., a cloud server) via a network (e.g., a cellular network). For example, the lighting controller 116 identifies the location of the inoperable light of the object by determining the location of the vehicle 100 via the GPS receiver 110 and determining the location of the object relative to the vehicle 100 via the collected images and / or data. The lighting controller 116 includes the location of the vehicle 100 and / or the identified object in the alert to facilitate repair and / or replacement of the inoperable light. Additionally, in some examples, lighting controller 116 communicates with the lighting controller 116 via communication module 112 via a cellular and / or other network (e.g., Fig. 9 The network 920) sends the alert to the server (e.g., Fig. 9 For example, a remote server (e.g., a cloud server) stores information of inoperable lights identified by the lighting controller 116 of the vehicle 100 to facilitate repair and / or replacement of the inoperable lights (e.g., by a third party).

[0039] Figure 2 An example image 200 of a lamp post 202 captured by one or more of the cameras 102 of the vehicle 100 is shown. The lighting controller 116 of the vehicle 100 recognizes that the fixed infrastructure object is within the image 200 and determines, via, for example, an image recognition system, that the fixed infrastructure object is a lamp post.

[0040] like Figure 2 As shown, lamp post 202 includes a post 204, a street lamp 206 (e.g., a first street lamp), and another street lamp 208 (e.g., a second lamp). For example, street lamp 206 and / or street lamp 208 are configured to illuminate a portion of a road along which vehicle 100 is traveling (e.g., Figure 3 In other examples, lamp post 202 includes more or fewer street lights.

[0041] The lighting controller 116 is configured to identify the operability of the street lights 206, 208 of the lamp post 202 based on the light emission measurements corresponding to the lamp post 202. For example, when the vehicle 100 approaches the lamp post 202, one or more of the cameras 102 and / or one or more of the light sensors 104 collect light emission measurements near the lamp post 202. In some examples, the lighting controller 116 is configured to monitor the light emission measurements of the lamp post 202 and / or other lamp posts in response to one or more of the cameras 102 and / or one or more of the light sensors 104 detecting that the ambient light around the vehicle 100 is below an ambient light threshold associated with a dark driving condition in which the street lights are activated. In addition, the lighting controller 116 compares the light emission measurements with the light emission threshold corresponding to the lamp post 202. For example, if the light emission measurement is greater than a first threshold, the lighting controller 116 determines that the street lights 206, 208 are fully operational; if the light emission measurement is less than the first threshold, one of the street lights 206, 208 is partially inoperable; if the light emission measurement is less than a second threshold, both street lights 206, 208 are partially inoperable; if the light emission measurement is less than a third threshold, one of the street lights 206, 208 is completely inoperable; and / or if the light emission measurement is less than a fourth threshold, both street lights 206, 208 are completely inoperable.

[0042] Additionally or alternatively, the lighting controller 116 is configured to compare light emission measurements of the plurality of streetlights when the lamp post includes more than one streetlight. For example, when the lighting controller 116 determines that the lamp post 202 includes the streetlight 206 and the streetlight 208, the lighting controller 116 uses image recognition to compare the light emission of the streetlight 206 (e.g., the first light emission) with the light emission of the streetlight 208 (e.g., the second light emission) to identify the operability of the streetlight 206 and the streetlight 208. In the example shown, the lighting controller 116 determines that the streetlight 206 is operable and the streetlight 208 is inoperable in response to the lighting controller 116 determining via image recognition that the light emitted by the streetlight 206 is greater than the light emitted by the streetlight 208.

[0043] Upon detecting that the streetlight 208 is inoperable, the lighting controller 116 issues an alert to facilitate repair and / or replacement of the streetlight 208. For example, the lighting controller 116 sends an alert to the infrastructure node 210 of the lamp post 202 via the communication module 114. Additionally or alternatively, the infrastructure node 210 and / or the communication module 112 of the vehicle 100 sends an alert to a remote server (e.g., via a cellular network) to alert a third party that the streetlight 208 is inoperable and is in a state of repair and / or replacement.

[0044] Figure 3The vehicle 100 is shown traveling along a road 302 adjacent to a plurality of lamp posts 304 including street lamps. Figure 3 As shown, the lamp posts 304 are spaced apart from each other by a distance 306. In the example shown, the lighting controller 116 identifies a pattern that the lamp posts 304 follow along the road. For example, the lighting controller 116 identifies the pattern based on a plurality of images captured by one or more of the cameras 102, data collected by one or more of the light sensors 104, and / or data collected by one or more of the proximity sensors 106. In addition, the lighting controller 116 determines (e.g., using machine learning) a lighting pattern corresponding to the lamp posts 304. For example, the lighting controller 116 determines the lighting pattern based on lighting measurements collected via one or more of the cameras 102 and / or one or more of the light sensors 104. The lighting controller 116 identifies that the street light of one of the lamp posts 304 is inoperable in response to determining that the lighting measurement corresponding to one of the lamp posts 304 does not correspond to the lighting pattern. That is, when the lighting measurement corresponding to one of the lamp posts 304 is less than the lighting measurement of other lamp posts 304 in the vicinity, the lighting controller 116 determines that the street light of one of the lamp posts 304 is inoperable. In the example shown, the lighting controller 116 identifies that the streetlight of the lamp post 304 a is inoperable upon detecting that the lighting measurements corresponding to the lamp post 304 a do not correspond to the lighting pattern formed by the lamp post 304 along the road 302 .

[0045] Upon detecting that the streetlight of the lamp post 304a is inoperable, the lighting controller 116 issues an alert to facilitate repair and / or replacement of the streetlight 208. For example, the lighting controller 116 sends an alert to the infrastructure node 308 of the lamp post 304a and / or the infrastructure node 310 of another lamp post 304 in the vicinity. In some examples, when the lamp post 304a including the inoperable lamp does not include an infrastructure node, the lighting controller 116 sends an alert to the infrastructure node 310. Additionally or alternatively, the infrastructure node 308, the infrastructure node 310, and / or the communication module 112 of the vehicle 100 sends an alert to a remote server (e.g., via a cellular network) to warn third parties that the streetlight is inoperable and in a state of repair and / or replacement.

[0046] Figure 4 An example image 400 of a traffic light 402 (e.g., a first traffic light) and another traffic light 404 (e.g., a second traffic light) located at an intersection is shown. For example, when the vehicle 100 approaches the intersection, the image 400 is captured by one or more of the cameras 102 of the vehicle 100. The lighting controller 116 recognizes that the fixed infrastructure object is within the image 400 and determines, via, for example, an image recognition system, that the fixed infrastructure object is a traffic light.

[0047] like Figure 4 As shown, the traffic light 402 includes a light 406 (e.g., a first light, a green light) that is lit to indicate that traffic is proceeding, a light 408 (e.g., a second light, a yellow light) that is lit to indicate that traffic is avoiding, and a light 410 (e.g., a third light, a red light) that is lit to indicate that traffic is stopping. In addition, the traffic light 404 includes a light 412 (e.g., a first light, a green light) that is lit to indicate that traffic is proceeding, a light 414 (e.g., a second light, a yellow light) that is lit to indicate that traffic is avoiding, and a light 416 (e.g., a third light, a red light) that is lit to indicate that traffic is stopping. In addition, the traffic lights 402 and 404 at the intersection are configured to operate in unison. That is, the lights 406 and 412 are configured to be lit and disabled in unison, the lights 408 and 414 are configured to be lit and disabled in unison, and the lights 410 and 416 are configured to be lit and disabled in unison.

[0048] In operation, the lighting controller 116 of the vehicle 100 is configured to identify, via image recognition, that the intersection includes the traffic lights 402 and the traffic lights 404 as the vehicle 100 approaches the intersection. Upon identifying the traffic lights 402, 404, the lighting controller 116 determines the operability of the lights 406, 408, 410 of the traffic lights 402 and the operability of the lights 412, 414, 416 of the traffic lights 404 by comparing the light emissions of the traffic lights 402 and the traffic lights 404. In the example shown, the lighting controller 116 determines that the lights 414 of the traffic lights 404 are inoperable in response to detecting that the lights 408 of the traffic lights 402 are illuminated and the lights 414 of the traffic lights 404 are deactivated.

[0049] Upon detecting that one or more of the lights 406, 408, 410, 412, 414, 416 of the traffic lights 402, 404 are inoperable, the lighting controller 116 issues an alert to facilitate repair and / or replacement of one or more of the lights 406, 408, 410, 412, 414, 416. For example, upon detecting that one or more of the lights 406, 408, 410, 412, 414, 416 are inoperable, the lighting controller 116 sends an alert to the infrastructure node 418 of the traffic light 402 and / or the infrastructure node 420 of the traffic light 404. Additionally or alternatively, the infrastructure node 418, the infrastructure node 420, and / or the communication module 112 of the vehicle 100 sends an alert to a remote server (e.g., via a cellular network) to warn third parties that the street light is inoperable and is in a state of repair and / or replacement.

[0050] FIG. 5A to FIG. 5CAn example sequence of images 502, 504, 506 of a traffic light 508 captured by one or more of the cameras 102 of the vehicle 100 is shown. More specifically, Figure 5A A traffic light 508 is shown in a first state, Figure 5B The traffic light 508 is shown in a second state, and Figure 5C Traffic light 508 is shown in a third state. FIG. 5A to FIG. 5C As shown, traffic lights 508 include lights 510 (e.g., a first light, a green light) that are illuminated to indicate that traffic is proceeding, lights 512 (e.g., a second light, a yellow light) that are illuminated to indicate that traffic is yielding, and lights 514 (e.g., a third light, a red light) that are illuminated to indicate that traffic is stopping.

[0051] In the example shown, images 502, 504, 506 are captured sequentially within a short period of time such that image 502 is captured before image 504 and image 504 is captured before image 506. Figure 5A In the first state of the traffic light 508 captured in the image 502, the light 510 is illuminated and the lights 512, 514 are deactivated. Figure 5B In the second state of the traffic light 508 captured in the image 504, the lights 510, 512, 514 are deactivated. Figure 5C In a third state of the traffic light 508 captured in the image 506 of FIG. 502 , the light 514 is illuminated and the lights 510 and 512 are deactivated. Based on the images 502 , 504 , and 506 , the lighting controller 116 determines a measured lighting sequence (e.g., a time-varying sequence) of the traffic light 508. For example, the lighting controller 116 determines that the measured lighting sequence includes a first state in which the light 510 is illuminated, a subsequent second state in which the lights 510 , 512 , and 514 are not illuminated, and a subsequent third state in which the light 514 is illuminated.

[0052] In addition, the lighting controller 116 retrieves known lighting patterns (eg, time-varying patterns) of the lights 510, 512, 514 of the traffic light 508. For example, the lighting controller 116 retrieves known lighting patterns (eg, time-varying patterns) of the lights 510, 512, 514 of the traffic light 508 from a database (eg, Fig. 9The lighting controller 116 retrieves the known lighting pattern from a database 916 of the image 504 and / or a remote server (e.g., a cloud server). The lighting controller 116 identifies that one of the lights 510, 512, 514 is inoperable in response to determining that the measured lighting sequence of the traffic light does not correspond to the known lighting pattern of the traffic light 508. For example, the lighting controller 116 identifies that one of the lights 510, 512, 514 is inoperable when comparing the traffic light 508 as captured in the image 504 to the known lighting pattern of the traffic light 508. In addition, the lighting controller 116 is configured to identify which of the lights 510, 512, 514 is inoperable by comparing two or more of the states of the measured lighting sequence to the known lighting pattern. For example, when comparing the images 502, 504 and / or the images 504, 506 to the known lighting pattern of the traffic light 508, the lighting controller 116 identifies that the light 512 is inoperable.

[0053] Upon detecting that one or more of the lamps 510, 512, 514 of the traffic light 508 are inoperable, the lighting controller 116 issues an alert to facilitate repair and / or replacement of one or more of the lamps 510, 512, 514. For example, upon detecting that one or more of the lamps 510, 512, 514 are inoperable, the lighting controller 116 sends an alert to the infrastructure node 516 of the traffic light 508. Additionally or alternatively, the infrastructure node 516 and / or the communication module 112 of the vehicle 100 sends an alert to a remote server (e.g., via a cellular network) to alert a third party that the street light is inoperable and is in a state of repair and / or replacement.

[0054] In addition, in some examples, for example, the lighting controller 116 is configured to identify whether a road marking (e.g., a lane marking, a reflective infrastructure element, etc.) requires maintenance. For example, the lighting controller 116 uses image recognition to detect the presence of a road marking and identifies the road marking. In addition, the lighting controller 116 uses the camera 102 (e.g., in conjunction with image recognition software) and / or the proximity sensor 106 (e.g., a lidar sensor) to measure the reflectivity of the road marking. If the reflectivity measurement is less than a corresponding threshold, the lighting controller 116 issues an alert to facilitate the maintenance of the road marking.

[0055] Figure 6The vehicle 100 is shown approaching an intersection 600. In the example shown, the intersection 600 is formed by a road 602 along which the vehicle 100 is traveling and a road 604 along which another vehicle 606 is traveling. In addition, the vehicle 100 is approaching a flashing light 608 (e.g., a flashing red light) indicating that traffic is stopped, and the vehicle 606 is approaching another flashing light 610 (e.g., a flashing yellow light) indicating that traffic is yielding. In some cases, the driver of the vehicle 100 may not be aware that the flashing light 610 indicates that the vehicle 606 is yielding and / or may assume that the flashing light 610 indicates that the vehicle 606 is stopped.

[0056] To facilitate the driver to safely drive the vehicle 100 through the intersection 600, the lighting controller 116 obtains additional information about the intersection 600 (e.g., flashing lights 610 instructing the vehicle 606 to avoid) and presents additional information to the driver (e.g., via Figure 4 Infotainment head unit 904). For example, lighting controller 116 retrieves additional information from a remote server (e.g., a cloud server) via communication module 112. In some such examples, the remote server receives additional information from vehicles (e.g., vehicle 100, vehicle 606) traveling through intersection 600. Additionally or alternatively, lighting controller 116 retrieves additional information from vehicle 606 via V2V communication, retrieves additional information from flashing light 608 via V2I communication, and / or retrieves additional information from flashing light 610 via V2I communication.

[0057] Figure 7 An example image 700 of a business sign 702 captured by one or more of the cameras 102 of the vehicle 100 is shown. The business sign 702 includes a name 704, a slogan 706, a logo 708, and an address 710. In the example shown, the business sign 702, the name 704, the slogan 706, the logo 708, and / or the address 710 are illuminated by lights of the business sign 702.

[0058] The lighting controller 116 of the vehicle 100 is configured to identify the operability of the lights of the commercial sign 702 based on the light emission measurements corresponding to the commercial sign 702. For example, when the vehicle 100 approaches the commercial sign 702, one or more of the cameras 102 and / or one or more of the light sensors 104 collect light emission measurements. In addition, the lighting controller 116 compares the light emission measurements with light emission thresholds corresponding to the commercial sign 702. For example, if the light emission measurements are greater than a first threshold, then the lighting controller 116 determines that the lights of the commercial sign 702 are fully operable, if the light emission measurements are between the first threshold and a second threshold, then partially inoperable, and if the light emission measurements are less than the second threshold, then fully inoperable. Additionally or alternatively, the lighting controller 116 utilizes image recognition to identify when one or more of the lights of the commercial sign 702 are inoperable. In addition, the lighting controller 116 is configured to utilize image recognition and / or machine learning to identify graffiti 712 on the commercial sign 702.

[0059] Upon detecting that commercial sign 702 includes inoperable lights and / or graffiti, lighting controller 116 issues an alert to facilitate repair and / or replacement of commercial sign 702. For example, lighting controller 116 is configured to send an alert to an infrastructure node of commercial sign 702 via communication module 114. Additionally or alternatively, lighting controller 116 is configured to send an alert to a remote server (e.g., via a cellular network) via communication module 112 to alert a third party that commercial sign 702 requires repair.

[0060] The lighting controller 116 may also utilize image recognition software to recognize information presented by the business sign 702 based on the image captured via the camera 102. For example, the lighting controller 116 utilizes image recognition to recognize the name 704, slogan 706, logo 708, and / or address 710 corresponding to the business sign 702. Furthermore, in some examples, the lighting controller 116 detects when new businesses open and / or old businesses close, and provides such information to a remote server to be used for online business and / or map searches via the communication module 112 and / or the communication module 114. Additionally or alternatively, the lighting controller 116 detects when a business opens and / or closes for the day based on detected light changes in the business's storefront, and provides such information to a remote server via the communication module 112 and / or the communication module 114.

[0061] Figure 8 800 is shown of an example image of another vehicle 802 captured by one or more cameras 102 of the vehicle 100. More specifically, the image 800 shows the rear side 804 of the vehicle 802 located in front of the vehicle 100. Figure 8As shown, the rear side 804 of the vehicle 802 includes a brake light 806 (eg, a first brake light, a left brake light), a brake light 808 (eg, a second brake light, a right brake light), a brake light 810 (eg, a third brake light, an upper brake light), and a license plate 812 .

[0062] The lighting controller 116 of the vehicle 100 is configured to identify the operability of the brake lights 806, 808, 810 of the vehicle 802 in front of the vehicle 100. Additionally or alternatively, the lighting controller 116 is configured to identify the operability of the brake lights of the vehicles behind and / or to the side of the vehicle 100. For example, the lighting controller 116 is configured to monitor the operability of the brake lights 806, 808, 810 of the vehicle 802 when it is identified that the vehicle 802 is braking. In some examples, the lighting controller 116 identifies that the vehicle 802 is braking based on V2V communications received from the vehicle 802 via the communication module 114 of the vehicle 100. Furthermore, in some examples, the lighting controller 116 identifies that the vehicle 802 is braking, reversing, changing lanes, etc. based on data collected from sensors of the vehicle 100 (such as one or more of the proximity sensor 106, the vehicle speed sensor, the accelerometer, etc.).

[0063] Upon detecting that the vehicle 802 is braking, the lighting controller 116 applies image recognition to the image 800 to monitor the operability of the brake lights 806, 808, 810 of the vehicle 802. For example, the lighting controller 116 utilizes image recognition to identify the location of the brake lights 806, 808, 810 and determine the light emission measurement of each of the brake lights 806, 808, 810. Subsequently, the lighting controller 116 compares the light emission measurement to determine whether one or more of the brake lights 806, 808, 810 are inoperable. In the example shown, the lighting controller 116 determines that the light emission measurement corresponding to the brake light 806 is less than the light emission measurement of the brake light 808 and the light emission measurement of the brake light 810. In response, the lighting controller 116 determines that the brake lights 808, 810 are illuminated and the brake light 806 is deactivated. Since the lighting controller 116 detects that the brake light 806 is deactivated when the vehicle 802 is braking, the lighting controller 116 determines that the brake light 806 is inoperable.

[0064] Upon detecting that one or more of the brake lights 806, 808, 810 are inoperable, the lighting controller 116 issues an alert. For example, the lighting controller 116 sends the alert to the vehicle 802 via V2V communication using the communication module 114. Additionally or alternatively, the lighting controller 116 sends the alert to a remote server (e.g., via a cellular network) using the communication module 112. For example, the alert includes identification of the vehicle 802 detected as having inoperable brake lights and / or other exterior lights based on the license plate 812. In some such examples, the lighting controller 116 utilizes image recognition to identify the license plate 812 within the image 800. Furthermore, while the illustrated example includes monitoring the brake lights 806, 808, 810 of the vehicle 802, the lighting controller 116 of the vehicle 100 may be used to monitor any other exterior lighting (e.g., daytime running lights) of the vehicle 802.

[0065] Fig. 9 is a block diagram of an electronic component 900 of a vehicle 100. Fig. 9 As shown, electronic components 900 include an onboard computing platform 902 , a GPS receiver 110 , a communication module 112 , a communication module 114 , an infotainment head unit 904 , a camera 102 , a sensor 906 , an electronic control unit (ECU) 908 , and a vehicle data bus 910 .

[0066] The onboard computing platform 902 includes: a microcontroller unit, controller or processor 912; a memory 914; and a database 916. In some examples, the processor 912 of the onboard computing platform 902 is configured to include the lighting controller 116. Alternatively, in some examples, the lighting controller 116 is incorporated into another electronic control unit (ECU) along with its own processor 912, memory 914 and / or database 916. The database 916 stores entries that, for example, correlate characteristics of objects within an image captured by the camera 102 of the vehicle 100 with characteristics of known objects. For example, the lighting controller 116 retrieves the boundaries of objects within the image, lamp post patterns, lighting patterns of traffic lights, geographic information, and / or other information, and compares the retrieved information with data collected by the camera 102 and / or sensor 906 of the vehicle 100 to identify and monitor the lighting of detected objects.

[0067] The processor 912 may be any suitable processing device or a group of processing devices, such as, but not limited to, a microprocessor, a microcontroller-based platform, an integrated circuit, one or more field programmable gate arrays (FPGAs), and / or one or more application-specific integrated circuits (ASICs). The memory 914 may be a volatile memory (e.g., a RAM including non-volatile RAM, magnetic RAM, ferroelectric RAM, etc.), a non-volatile memory (e.g., a disk memory, a flash memory, an electrically programmable read-only memory, an electrically erasable programmable read-only memory, a memristor-based solid-state memory, etc.), an unchangeable memory (e.g., an electrically programmable read-only memory), a read-only memory, and / or a high-capacity storage device (e.g., a hard disk drive, a solid-state drive, etc.). In some examples, the memory 914 includes a variety of memories, particularly volatile memories and non-volatile memories.

[0068] The database 916 stores entries that relate, for example, known objects (e.g., lamp posts, street lights, traffic signals, traffic lights, commercial signs, commercial sign lights, vehicles, vehicle lights, operable lights, inoperable lights, graffiti, etc.) to objects included in images and / or videos captured by one or more cameras 102 of the vehicle 100. For example, the lighting controller 116 accesses one or more entries of the database 916 to identify an object within a captured image and / or video as a lamp post, an operable street light, an inoperable street light, a traffic signal, an operable traffic light, an inoperable traffic light, a commercial sign, an operable light of a commercial sign, an inoperable light of a commercial sign, graffiti, a vehicle, an operable light of a vehicle, an inoperable light of a vehicle, etc.

[0069] The memory 914 is a computer-readable medium on which one or more sets of instructions, such as software for operating the methods of the present disclosure, may be embedded. The instructions may embody one or more methods or logic as described herein. For example, the instructions may reside completely or at least partially in any one or more of the memory 914, the computer-readable medium, and / or the processor 912 during the execution of the instructions.

[0070] The terms "non-transitory computer-readable medium" and "computer-readable medium" include a single medium or multiple media, such as centralized or distributed databases, and / or associated caches and servers that store one or more sets of instructions. In addition, the terms "non-transitory computer-readable medium" and "computer-readable medium" include any tangible medium that can store, encode, or carry a set of instructions for execution by a processor or cause a system to perform any one or more of the methods or operations disclosed herein. As used herein, the term "computer-readable medium" is expressly defined to include any type of computer-readable storage device and / or storage disk and excludes propagating signals.

[0071] In addition, if Fig. 9As shown, the electronic component 900 includes a GPS receiver 110, a communication module 112, and a communication module 114. For example, the communication module 112 wirelessly communicates with a server 918 of a network 920. For example, the communication module 112 transmits captured images and / or information identifying detected inoperable infrastructure lights (e.g., the locations of the detected inoperable infrastructure lights, which detected infrastructure lights are identified as inoperable), detected graffiti (e.g., the locations of the detected graffiti), and / or detected inoperable vehicle lights (e.g., the license plates of the vehicles having the inoperable lights, which detected infrastructure lights are identified as inoperable) to the server 918 via a cellular network to enable others to identify and resolve the inoperable lights, graffiti, etc.

[0072] The illustrated example of the infotainment head unit 904 provides an interface between the vehicle 100 and the user. The infotainment head unit 904 includes digital and / or analog interfaces (e.g., input devices and output devices) to receive input from the user and display information for the user. The input device includes, for example, a control knob, a dashboard, a digital camera for image capture and / or visual command recognition, a touch screen, an audio input device (e.g., a cabin microphone), a button or a touch pad. The output device may include an instrument cluster output (e.g., a dial, a lighting device), an actuator, a display 922 (e.g., a head-up display, a center console display, such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a flat panel display, a solid-state display, etc.) and / or a speaker 924. In the illustrated example, the infotainment head unit 904 includes hardware (e.g., a processor or controller, memory, storage device, etc.) and software (e.g., an operating system, etc.) for an infotainment system (such as SYNC® and MyFord Touch®). In addition, the infotainment head unit 904 displays the infotainment system on, for example, a display 922. Additionally or alternatively, the display 922 , speaker 924 and / or other output devices of the infotainment head unit 904 present information collected via the communication module 112 and / or from an infrastructure node from a server 918 of the network 920 and / or from other vehicles via the communication module 114 to facilitate the driver of the vehicle 100 to control the vehicle 100 .

[0073] Sensors 906 are arranged in and around the vehicle 100 to monitor properties of the vehicle 100 and / or the environment in which the vehicle 100 is located. One or more sensors 906 can be installed to measure properties around the exterior of the vehicle 100. Additionally or alternatively, one or more of the sensors 906 can be installed in the cabin of the vehicle 100 or in the body of the vehicle 100 (e.g., engine compartment, wheel well, etc.) to measure properties inside the vehicle 100. For example, the sensors 906 include accelerometers, odometers, tachometers, pitch and yaw sensors, wheel speed sensors, microphones, tire pressure sensors, biometric sensors, and / or any other suitable type of sensor. In the example shown, the sensors 906 include light sensors 104 and proximity sensors 106.

[0074] ECU 908 monitors and controls the subsystems of vehicle 100. For example, ECU 908 is a group of discrete electronic devices that include their own circuits (e.g., integrated circuits, microprocessors, memory, storage devices, etc.) and firmware, sensors, actuators, and / or mounting hardware. ECU 908 communicates and exchanges information via a vehicle data bus (e.g., vehicle data bus 910). In addition, ECU 908 can communicate properties (e.g., the state of ECU 908, sensor readings, control states, errors and diagnostic codes, etc.) to each other and / or receive requests from each other. For example, vehicle 100 can have 70 or more ECU 908, which are positioned at various locations around vehicle 100 and are communicatively coupled via vehicle data bus 910.

[0075] In the example shown, the ECU 908 includes a body control module 926 and a telematics control unit 928. For example, the body control module 926 controls one or more subsystems throughout the vehicle 100, such as power windows, power locks, anti-theft systems, power mirrors, etc. For example, the body control module 926 includes circuits to drive one or more of relays (e.g., for controlling wiper fluid, etc.), brushed direct current (DC) motors (e.g., for controlling power seats, power locks, power windows, wipers, etc.), stepper motors, LEDs, etc. The telematics control unit 928 controls tracking of the vehicle 100, for example, using data received by a GPS receiver 110 of the vehicle 100.

[0076] The vehicle data bus 910 communicatively couples the camera 102, the GPS receiver 110, the communication module 112, the communication module 114, the vehicle computing platform 902, the infotainment head unit 904, the sensor 906, and the ECU 908. In some examples, the vehicle data bus 910 includes one or more data buses. The vehicle data bus 910 can be implemented according to the controller area network (CAN) bus protocol defined by the International Standards Organization (ISO) 11898-1, the media oriented system transport (MOST) bus protocol, the CAN flexible data (CAN-FD) bus protocol (ISO 11898-7) and / or the K-line bus protocol (ISO 9141 and ISO 14230-1) and / or the Ethernet™ bus protocol IEEE 802.3 (since 2002), etc.

[0077] Fig.10 is a flow chart of an example method 1000 for monitoring infrastructure lighting. Fig.10 The flowchart represents a memory (such as Fig. 9 914) in a memory 914) and includes one or more programs that when executed by a processor (such as Fig. 9 When executed by the processor 912 of the vehicle 100, one or more programs enable the vehicle 100 to achieve Figure 1 and Fig. 9 Example lighting controller 116. Although reference Fig. 9 The flowchart shown in describes an example procedure, and many other methods of implementing the example lighting controller 116 may alternatively be used. For example, the order of execution of the blocks may be rearranged, changed, eliminated, and / or combined to perform the method 1000. In addition, due to the combination of Figures 1 to 9 The method 1000 is disclosed by means of the components, and therefore some functions of these components will not be described in detail below.

[0078] Initially, at block 1002, one or more of the cameras 102 capture images of the surrounding area of ​​the vehicle 100. At block 1004, the lighting controller 116 determines whether an object is detected within the captured image. For example, the lighting controller 116 utilizes image recognition to detect an object (e.g., a fixed infrastructure object, a vehicle, etc.) within one or more of the captured images. In response to the lighting controller 116 not detecting an object within the captured image, the method 1000 proceeds to block 1016. Otherwise, in response to the lighting controller 116 detecting an object within the captured image, the method 1000 proceeds to block 1006.

[0079] At block 1006, the lighting controller 116 identifies an object within the captured image. For example, the lighting controller 116 identifies the object by comparing characteristics of the captured image with characteristics of a known object. In some examples, the lighting controller 116 performs the comparison after retrieving the characteristics of the known object from a database (e.g., the database 916 of the vehicle 100, a database of the server 918). In some examples, the lighting controller 116 identifies that the object includes a fixed infrastructure object, such as a lamp post, a traffic light, a commercial sign, etc. In other examples, the lighting controller 116 identifies that the object includes another vehicle.

[0080] At block 1008, the lighting controller 116 determines whether the object detected within the captured image includes a light. For example, the lighting controller 116 determines that the detected object includes a light by utilizing image recognition and / or by retrieving such information from a database (e.g., database 916 of the vehicle 100, database of the server 918). In response to the lighting controller 116 determining that the detected object does not include a light, the method 1000 proceeds to block 1016. Otherwise, in response to the lighting controller 116 determining that the detected object includes a light, the method 1000 proceeds to block 1010.

[0081] At block 1010, the lighting controller 116 determines whether the lights of the object are inoperable. For example, the lighting controller 116 determines the operability of the lights by applying image recognition to the captured image and / or based on the light emission measurements. For example, the lighting controller 116 compares the collected light emission measurements to the light emission threshold to determine whether the lights of the detected object are operable. In response to the lighting controller 116 determining that none of the lights are inoperable, the method 1000 proceeds to block 1016. Otherwise, in response to the lighting controller 116 determining that one or more of the lights are inoperable, the method 1000 proceeds to block 1012. At block 1012, the lighting controller 116 sends an alert identifying the inoperable lights to an infrastructure node via V2I communication and / or to a communication node of another vehicle via V2V communication using the communication module 114 (i.e., a vehicle-to-vehicle communication module, such as a DSRC module). At block 1014, the lighting controller 116 sends the alert to a remote server of the network (e.g., server 918 of network 920) using the communication module 112.

[0082] At block 1016, the lighting controller 116 determines whether the communication module 114 and / or the communication module 112 has received information that facilitates the driver of the vehicle 100 to operate the vehicle 100. In some examples, the communication module 114 and / or the communication module 112 receives navigation information to facilitate the driver of the vehicle 100 as the vehicle 100 approaches an intersection. In response to the lighting controller 116 determining that such information has not been received, the method 1000 returns to block 1002. Otherwise, in response to the lighting controller 116 determining that such information has been received, the method 1000 proceeds to block 1018, where the lighting controller 116 presents such information to the driver of the vehicle 100 (e.g., via the infotainment head unit 904).

[0083] In the present application, the use of disjunctive words is intended to include conjunctions. The use of definite or indefinite articles is not intended to indicate cardinality. In particular, reference to "the" object or "a" and "an" object is also intended to indicate one of possible multiple such objects. In addition, the conjunction "or" can be used to convey simultaneous features rather than mutually exclusive alternatives. In other words, the conjunction "or" should be understood to include "and / or". The terms "include", "include" and "include" are inclusive and have the same scope as "include", "include" and "include", respectively. In addition, as used herein, the terms "module", "unit" and "node" refer to hardware having a circuit that provides communication, control and / or monitoring capabilities, usually in combination with a sensor. "Module", "unit" and "node" can also include firmware executed on the circuit.

[0084] The above embodiments, particularly any "preferred" embodiments, are possible examples of implementations and are intended only for a clear understanding of the principles of the present invention. Many changes and modifications may be made to the above embodiments without departing substantially from the spirit and principles of the technology described herein. All modifications are intended to be included herein within the scope of the present disclosure and protected by the following claims.

[0085] According to the present invention, a vehicle is provided, the vehicle having: a camera; a vehicle-to-vehicle communication module; and a controller, the controller being used to: identify a fixed infrastructure object within an image captured by the camera; identify whether the light is inoperable in response to determining that the fixed infrastructure object includes a light; and send an alert indicating that the light is inoperable to an infrastructure communication node via vehicle-to-infrastructure communication utilizing the vehicle-to-vehicle communication module.

[0086] According to one embodiment, the above invention is further characterized by a proximity sensor for further detecting the presence of the fixed infrastructure object.

[0087] According to one embodiment, in order to identify the fixed infrastructure object and the operability of the light, the controller utilizes machine learning to perform image recognition.

[0088] According to one embodiment, the controller monitors light emissions from the lamp to determine if the lamp is inoperable.

[0089] According to one embodiment, the above invention is further characterized by a light sensor configured to collect light emission measurements to enable the controller to monitor the light emission of the lamp.

[0090] According to one embodiment, in order to monitor the light emission of the lamp, the controller processes the light distribution of the lamp using optical frequency filtering.

[0091] According to one embodiment, the controller determines that the lamp is inoperable in response to determining that one or more LEDs of the lamp are inoperable based on the light emissions of the lamp.

[0092] According to one embodiment, the above invention is also characterized by: a second communication module, which is configured to send the alarm to a server via a cellular network; and a GPS receiver, which is configured to identify the vehicle location, wherein the controller includes in the alarm the vehicle location at which the camera captured the image including the fixed infrastructure object, so as to facilitate the identification of the location of the inoperable light.

[0093] According to one embodiment, the controller identifies that the fixed infrastructure object is a lamp post, the lamp of the lamp post being a first street lamp configured to illuminate a portion of a road.

[0094] According to one embodiment, the above invention is further characterized by a light sensor for monitoring ambient light, wherein the controller is configured to identify operability of the first street light in response to the light sensor detecting that the ambient light is below an ambient light threshold.

[0095] According to one embodiment, when the controller identifies that the lamp post includes the first street light and the second street light, the controller determines the operability of the first street light by comparing the light emissions of the first street light and the second street light.

[0096] According to one embodiment, the controller is configured to: identify a pattern of lamp posts along the road based on multiple images captured by the camera; determine a lighting pattern corresponding to the pattern of lamp posts; and identify that the first street light of the lamp post is inoperable in response to determining that the lighting measurement value of the lamp post does not correspond to the lighting pattern.

[0097] According to one embodiment, the controller identifies that the fixed infrastructure object is a first traffic light including traffic lights, each of the traffic lights including a first light for instructing traffic to proceed, a second light for instructing the traffic to avoid, and a third light for instructing the traffic to stop.

[0098] According to one embodiment, when the controller identifies that the intersection includes the first traffic signal and the second traffic signal, the controller determines the operability of one or more of the traffic lights of the first traffic signal by comparing the light emissions of the first traffic signal and the second traffic signal.

[0099] According to an embodiment, the controller is configured to: retrieve an illumination pattern of the traffic lights; and identify one of the traffic lights as inoperable in response to determining that a measured illumination sequence of the traffic lights does not correspond to the illumination pattern.

[0100] According to one embodiment, the controller identifies the fixed infrastructure object as a commercial sign.

[0101] According to the present invention, a method includes: identifying, via a processor, an infrastructure object within an image captured by a camera of a vehicle; determining whether the infrastructure object includes a light; identifying whether the light is inoperable in response to determining that the infrastructure object includes the light; and sending an alert indicating that the light is inoperable to an infrastructure communication node via vehicle-to-infrastructure communication utilizing a vehicle-to-vehicle communication module of the vehicle.

[0102] According to one embodiment, the above invention is also characterized by: identifying a pattern of lamp posts along a road based on multiple images captured by the camera, one of the lamp posts being the infrastructure object; determining a lighting pattern corresponding to the pattern of lamp posts; and identifying that the lamp of the infrastructure object is inoperable in response to determining that the lighting measurement value of the infrastructure object does not correspond to the lighting pattern.

[0103] According to one embodiment, the above invention is also characterized by: identifying an intersection including a first traffic light and a second traffic light based on the image captured by the camera, the first traffic light being the infrastructure object; and determining the operability of the light of the first traffic light by comparing the light emissions of the first traffic light and the second traffic light.

[0104] According to one embodiment, the above invention is further characterized by: identifying that the infrastructure object is a traffic light; retrieving a lighting pattern of the traffic light of the traffic light; and identifying that one of the traffic lights is inoperable in response to determining that the measured lighting sequence of the traffic light does not correspond to the lighting pattern.

Claims

1. A vehicle, comprising: camera; Inter-vehicle communication module; as well as A controller, the controller being used to: identifying fixed infrastructure objects within images captured by the camera; responsive to determining that the fixed infrastructure object includes a light, identifying whether the light is inoperable; and sending an alert to an infrastructure communications node via vehicle-to-infrastructure communications utilizing the inter-vehicle communications module indicating that the light is inoperable; as well as The controller is also configured to retrieve additional information from another vehicle approaching the intersection and to retrieve additional information from a first flashing light and a second flashing light at the intersection.

2. The vehicle of claim 1, further comprising a proximity sensor for further detecting the presence of the fixed infrastructure object.

3. The vehicle of claim 1 , wherein to identify the fixed infrastructure object and the operability of the light, the controller utilizes machine learning to perform image recognition.

4. The vehicle of claim 1 , further comprising a light sensor configured to collect light emission measurements to enable the controller to monitor the light emission of the lamp, wherein the controller monitors the light emission of the lamp to determine whether the lamp is inoperable, and wherein in order to monitor the light emission of the lamp, the controller utilizes optical frequency filtering to process the light distribution of the lamp. 5 . The vehicle of claim 4 , wherein the controller determines that the lamp is inoperable in response to determining that one or more LEDs of the lamp are inoperable based on the light emissions of the lamp.

6. The vehicle of claim 1, further comprising: a second communication module configured to send the alert to a server via a cellular network; as well as a GPS receiver configured to identify a vehicle location, Wherein the controller includes in the alert the vehicle location at which the camera captured the image including the fixed infrastructure object to facilitate identifying the location of the inoperable light.

7. The vehicle of claim 1, wherein the controller identifies the fixed infrastructure object as a lamp post, the lamp of the lamp post being a first street light configured to illuminate a portion of a roadway.

8. The vehicle of claim 7, further comprising a light sensor for monitoring ambient light, wherein the controller is configured to identify operability of the first streetlight in response to the light sensor detecting that the ambient light is below an ambient light threshold.

9. The vehicle of claim 7, wherein when the controller identifies that the lamp post includes the first and second streetlights, the controller determines operability of the first streetlight by comparing light emissions of the first and second streetlights.

10. The vehicle of claim 7, wherein the controller is configured to: identifying a pattern of lamp posts along the roadway based on a plurality of images captured by the camera; determining a lighting pattern corresponding to the lamp post pattern; as well as The first streetlight of the lamp post is identified as inoperable in response to determining that the illumination measurement of the lamp post does not correspond to the illumination pattern.

11. The vehicle of claim 1 , wherein the controller identifies that the fixed infrastructure object is a first traffic light including traffic lights, each of the traffic lights including a first light for instructing traffic to proceed, a second light for instructing the traffic to avoid, and a third light for instructing the traffic to stop.

12. The vehicle of claim 11, wherein when the controller identifies that the intersection includes the first traffic signal and the second traffic signal, the controller determines the operability of one or more of the traffic lights of the first traffic signal by comparing the light emissions of the first traffic signal and the second traffic signal.

13. The vehicle of claim 11, wherein the controller is configured to: retrieving a lighting pattern of the traffic light; and One of the traffic lights is identified as inoperable in response to determining that the measured illumination sequence of the traffic light does not correspond to the illumination pattern.

14. The vehicle of claim 1, wherein the controller recognizes that the fixed infrastructure object is a commercial sign.

15. A method for vehicle monitoring of infrastructure lighting, the method comprising: identifying, via a processor, an infrastructure object within an image captured by a camera of the vehicle; determining whether the infrastructure object includes a light; identifying whether the light is inoperable in response to determining that the infrastructure object includes the light; as well as sending an alert to an infrastructure communications node via vehicle-to-infrastructure communications utilizing a vehicle-to-vehicle communications module of the vehicle indicating that the light is inoperable; as well as Additional information is retrieved from another vehicle approaching the intersection and from a first flashing light and a second flashing light at the intersection.

Images