Outdoor lighting system integrated pothole detection

Abstract

A method of street pothole detection includes detecting a vehicle moving on a street and determining, by a processor, a type of the vehicle. The method further includes detecting, by the processor, a presence of a pothole in the street based on sounds received by an outdoor luminaire and based on vibrations of the outdoor luminaire sensed by the outdoor luminaire, where detecting the presence of the pothole is performed if the type of the vehicle is a three-plus axle vehicle. The sounds and the vibrations are caused by the vehicle moving through the pothole.

Description

[0001] OUTDOOR LIGHTING SYSTEM INTEGRATED POTHOLE DETECTION

[0002] FIELD OF THE INVENTION

[0003] The present disclosure relates generally to street pothole detection, and more particularly to artificial intelligence (Al) based pothole detection and pothole severity estimation integrated in outdoor light system.

[0004] BACKGROUND OF THE INVENTION

[0005] Potholes pose a threat to road safety. Several factors such as freeze-thaw cycles, traffic loads, low-quality material, bad design, and extreme weather (e.g., heavy rain, sleet, etc.) can contribute to pothole formation. Once a pothole is formed, the pothole can increase in size, for example, due to water seeping into the pavement, vehicles damaging the pavement around the pothole, etc. Potholes can cause significant damage to vehicles. For example, vehicle components and systems such as tires, rims, suspension systems, wheel alignment, etc. can be damaged by potholes. In some cases, the extent of the damage to a vehicle caused by a pothole may depend on the size of the pothole. For example, a relatively larger (e.g., a wider and / or deeper) pothole may cause more damage than a smaller pothole. Thus, a solution that enables detecting street potholes and estimating pothole sizes may be desirable.

[0006] SUMMARY OF THE INVENTION

[0007] The present disclosure relates generally to street pothole detection, and more particularly to Al based pothole detection and pothole severity estimation integrated in outdoor light system. In an example embodiment, a method of street pothole detection includes detecting a vehicle moving on a street and determining, by a processor, a type of the vehicle. The method further includes detecting, by the processor, a presence of a pothole in the street based on sounds received by an outdoor luminaire and based on vibrations of the outdoor luminaire sensed by the outdoor luminaire, where detecting the presence of the pothole is performed if the type of the vehicle is a three-plus axle vehicle. The sounds and the vibrations are caused by the vehicle moving through the pothole. In some another example embodiment, an outdoor luminaire system for street pothole detection includes an outdoor luminaire that includes a radar device, a microphone device, and a vibration sensor. The outdoor luminaire system is configured to detect a vehicle moving on a street and to determine a type of the vehicle. The outdoor luminaire system is also configured to detect a presence of a pothole in the street based on sounds received by the microphone device and based on vibrations of the outdoor luminaire sensed by the vibration sensor, where detecting the presence of the pothole is performed if the type of the vehicle is a three-plus axle vehicle. The sounds and the vibrations are caused by the vehicle moving through the pothole.

[0008] These and other aspects, objects, features, and embodiments will be apparent from the following description and the appended claims.

[0009] BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and wherein:

[0011] Fig. 1 illustrates an outdoor luminaire system for detecting potholes and estimating pothole severity according to an example embodiment;

[0012] Fig. 2 illustrates a block diagram of an outdoor luminaire corresponding to each luminaire of the outdoor luminaires of FIG. 1 according to an example embodiment;

[0013] Fig. 3 illustrates a method of operation of the outdoor luminaire system of FIG. 1 to detect potholes and estimate pothole severity according to an example embodiment; and

[0014] Fig. 4 illustrates a method of detecting potholes and estimating pothole severity by the outdoor luminaire system of FIG. 1 according to an example embodiment.

[0015] The drawings illustrate only example embodiments and are therefore not to be considered limiting in scope. The elements and features shown in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the example embodiments. Additionally, certain dimensions or placements may be exaggerated to help visually convey such principles. In the drawings, the same reference numerals used in different figures may designate like or corresponding but not necessarily identical elements. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] In the following paragraphs, example embodiments will be described in further detail with reference to the figures. In the description, well known components, methods, and / or processing techniques are omitted or briefly described. Furthermore, reference to various feature(s) of the embodiments is not to suggest that all embodiments must include the referenced feature(s).

[0017] FIG. 1 illustrates an outdoor luminaire system 100 for detecting potholes and estimating pothole severity according to an example embodiment. In some example embodiments, the system 100 includes outdoor luminaires 102, 104, 106, 108, 110, 112, 114, and 116. The outdoor luminaires 102-116 may be streetlight luminaires. For example, the outdoor luminaires 102-108 may be on one side of a street 118, and the luminaires 110-116 may be on opposite side of the street 118 from the outdoor luminaires 102-108. The street 118 may include lanes 120, 122 for automobile traffic flow in two opposite directions as shown by arrows 142 and 144. For example, the street 118 may include a lane separator 124. Alternatively, the lanes 120, 122 of the street 118 may be for same-direction automobile traffic flow. In some alternative embodiments, the street 118 may have more or fewer lanes than shown without departing from the scope of this disclosure. The locations of the outdoor luminaires 102-116 are known, for example, from installation data and / or from global positioning system (GPS) data.

[0018] In some example embodiments, the outdoor luminaires 102-116 may each be capable of detecting a pothole and estimating the severity of the pothole. For example, the outdoor luminaires 102-116 may each include sensors and other devices that may be used in the detection of potholes in the street 118 and to estimate the severity of detected potholes. To illustrate, each luminaire of the outdoor luminaires 102-116 may detect the presence of a pothole based on information sensed or otherwise obtained by sensors of the particular luminaire. Each luminaire of the outdoor luminaires 102-116 may also estimate the severity of a detected pothole based on information sensed or otherwise obtained by sensors of the particular luminaire.

[0019] In some example embodiments, each luminaire of the luminaires 102-116 may include a light module, a controller, a radar device, a microphone device, a vibration sensor, and a tilt sensor. FIG. 2 illustrates a block diagram of an outdoor luminaire 200 corresponding to each luminaire of the outdoor luminaires 102-116 of FIG. 1 according to an example embodiment. Referring to FIGS. 1 and 2, in some example embodiments, the outdoor luminaire 200 includes a controller 202 that may include a processor 216 (e.g., a microprocessor) and a memory device 218 (e.g., a nonvolatile memory device such as a flash memory device). The memory device 218 may contain software code and data 220 including Al models 222. The software code including the Al models 222 may be executed by the processor 216 to perform and / or control operations described herein with respect to the outdoor luminaire 200, for example, to detect a pothole, to estimate severity of a detected pothole, and other operations.

[0020] In some example embodiments, the outdoor luminaire 200 may include a light module 204 that provides a light provided by the luminaire 200. For example, the luminaire 200 may be a streetlight that is attached to a pole. The controller 202 may control the operation of the light module 204 that may, for example, include one or more light sources such as light emitting diode light sources.

[0021] In some example embodiments, the outdoor luminaire 200 may include a radar device 206 that can transmit and receive radar signals that can be used to detect vehicles that are within the radar detection range of the radar device 206. For example, the radar device 206 may detect vehicles and / or perform other operations based on radar signals transmitted and received by the radar device 206. Alternatively or in addition, the controller 202 may process information from the radar device 206 to detect vehicles and / or perform other operations. With respect to vehicles that are within the radar detection range, the radar device 206 that can transmit and receive radar signals that can be used to determine speeds of vehicles, relative locations of detected vehicles (e.g., relative to the location of the outdoor luminaire 100), and vehicle movement directions, and to identify types of vehicles. For example, reflected radar signal amplitude and / or duration may be used to identify a type of a vehicle as belonging to a category from among two or more categories of vehicles such as a small car (e.g., a sedan or a small or mid-size sport utility vehicle (SUV)), a small truck (e.g., pick-up truck), a mid-size truck, a heavy truck (e.g., an 18-wheeler truck), a bus, or a trailer as can be readily understood by those of ordinary skill in the art with the benefit of this disclosure.

[0022] In some example embodiments, the luminaire 200 may include a microphone device 208 that may include one or more microphones (e.g., an array of microphones) that can capture sounds. For example, the microphone device 208 may receive / capture sounds caused by or resulting from a vehicle driving through a pothole (i.e., tires of a vehicle moving through a pothole, entering and leaving a pothole, or otherwise making contact with the pothole) within the sound sensing range of the microphone device 208. A vehicle driving through a pothole may be expressed as the vehicle hitting the pothole. In general, sounds generated from a heavy truck (e.g., an 18-wheeler), a bus, a trailer, and generally vehicles with at least three axles (i.e., three-plus axle vehicles) hitting a pothole (i.e., driving through a pothole) are generally louder (i.e., a higher amplitude) than sounds generated from a smaller vehicle or generally vehicles with maximum of two axles (i.e., two-axle vehicles) hitting the same pothole. The microphone device 208 may also receive sounds produced by vehicles (e.g., by the engine of vehicles) driving within the sound sensing range of the microphone device 208. The microphone device 208 may also capture other sounds such as wind sounds. In general, the controller 202 may process information from the microphone device 208, for example, to perform operations such as determining or otherwise estimating a type of a vehicle, detecting a pothole, estimating a severity of a detected pothole, etc.

[0023] In some example embodiments, the luminaire 200 may include a vibration sensor 210 to measure vibrations of the outdoor luminaire 200. To illustrate, the vibration sensor 210 may measure vibrations of the outdoor luminaire 200 caused by a moving vehicle passing within a vibration-causing range from the outdoor luminaire 200. For example, vibration sensor 210 may measure vibrations caused by a vehicle driving through a pothole (i.e., one or more tires moving over a pothole or entering and leaving a pothole) that is within a vibration-causing range from the location of the outdoor luminaire 200. Vibrations of the outdoor luminaire 200 caused by a vehicle driving over a smooth pavement area near the outdoor luminaire 200 are typically weaker than vibrations caused by the same vehicle hitting a pothole at a comparably same location as the smooth pavement area. Vibrations caused by a heavy truck (e.g., an 18-wheeler) are typically stronger (i.e., a higher amplitude) than vibrations caused by a smaller vehicle. In general, the controller 202 may process information from the vibration sensor 210, for example, to perform operations such as detecting a pothole, estimating a severity of a detected pothole, etc.

[0024] In some example embodiments, the luminaire 200 may include a tilt sensor 212 to measure the amount of tilting of the outdoor luminaire 200. To illustrate, a vehicle passing by the outdoor luminaire 200 may cause the outdoor luminaire 200 (e.g., the pole of the outdoor luminaire 200) to tilt. For example, the amount of tilting may be, for example, with respect to a vertical axis or with respect to the default tilting angle of the pole of the outdoor luminaire 200. In general, the amount of tilting of the outdoor luminaire 200 may depend on the type of vehicle that caused the tilting. To illustrate, a tilting caused by a heavy truck (e.g., an 18-wheeler) is typically larger than a tilting caused by a smaller vehicle at a comparably same location. The controller 202 may process information from the tilt sensor 212 indicating the amount of tilting of the pole of the outdoor luminaire 200, for example, to identify the type of a vehicle that caused the tilting. The controller 202 may also process information from the tilt sensor 212, for example, to perform operations such as detecting a pothole, estimating a severity of a detected pothole, etc.

[0025] In some example embodiments, the outdoor luminaire 200 may also include a communication interface unit 214. The communication interface unit 214 may transmit and receive signals complaint with one or more wireless communication standards and / or wired communication protocol. For example, the communication interface unit 214 may transmit and receive Bluetooth Low Energy (BLE) signals, Wi-Fi signals, other wireless signals, and / or Ethernet signals. The controller 202 may use the communication interface unit 214 to transmit information, for example, to other outdoor luminaires, to a server 146 (local or in the cloud), to a public works authority, and / or other destinations. The controller 202 may also receive information through the communication interface unit 214, for example, from other outdoor luminaires.

[0026] In some example embodiments, the luminaire 200 may include one or more other devices 224. For example, the one or more other devices 224 may include an anemometer unit to measure wind speed. For example, because wind may interfere with the detection of potholes based on sounds and vibrations, the controller 202 may perform detections of potholes and estimation of severity of potholes if the speed of wind as measured by the anemometer unit is at or below a threshold wind speed (e.g., 1 mile per hour, 3 miles per hour, 7 miles per hour, or 12 miles per hour). As described above, the outdoor luminaire 200 of FIG. 2 may correspond to each luminaire of the luminaires 102-116 and other outdoor luminaires, if any, of the system 100 of FIG. 1.

[0027] In some example embodiments, the outdoor luminaire 200 may include a driver that may receive AC power, for example, from a municipality power line, and provide DC power to the components of the outdoor luminaire 200 as can be readily understood by those of ordinary skill in the art with the benefit of this disclosure. In some alternative embodiments, the outdoor luminaire 200 may include more or fewer components than shown in FIG. 2 without departing from the scope of this disclosure. In some alternative embodiments, some of the components of the outdoor luminaire 200 may be integrated into a single component without departing from the scope of this disclosure. In some alternative embodiments, the outdoor luminaire 200 may include multiple processors, including some processors that may be inside some of the components of the outdoor luminaire 200, that perform different functions and may generally be referred to herein, together or individually, as a processor without departing from the scope of this disclosure. Referring to FIGS. 1-3, in some example embodiments, the light module of each luminaire of the outdoor luminaires 102-116 may provide a respective light to a respective portion of the street 118. The controller of each luminaire of the outdoor luminaires 102-116 may process information, such as information from sensors of the particular outdoor luminaire and control the operations of the respective outdoor luminaire. As described with respect to the outdoor luminaire 200, the controller of each luminaire of the outdoor luminaires 102-116 may execute software code including Al models to perform operations of the respective luminaire such as detecting potholes and estimating severity of detected potholes. Each luminaire of the luminaires 102-116 may also include a communication interface to send and receive information, for example, to and from other luminaires of the system 100 and / or the server 146 wirelessly or via one or more wired connections. To be clear, the server 146, which includes a processor, may be a local server or a cloud server as can be readily understood by those of ordinary skill in the art with the benefit of this disclosure.

[0028] In some example embodiments, each luminaire of the outdoor luminaires 102- 116 may detect the presence of a pothole based on sounds and vibrations generated or otherwise caused by a vehicle driving through the pothole. Because heavy trucks and other three-plus axle vehicles driving through potholes are more likely to produce louder sounds and stronger vibrations than smaller vehicles (i.e., two-axle vehicles) driving through potholes, the luminaires 102-116 may use sounds and vibrations caused by three-plus axle vehicles to detect potholes and may exclude the use of sounds and vibrations caused by smaller vehicles for the purpose of detecting potholes. Indeed, the luminaires 102-116 may use sounds and vibrations caused by heavy trucks (e.g., heavier than 14,000 pounds with no load or trucks with at least three axles) and other three-plus axle vehicles and may exclude sounds and vibrations caused by other vehicles including other trucks that are smaller trucks (e.g., pick-up trucks, small commercial trucks, etc.). For example, the luminaires 102-116 may limit detection of potholes to sounds and vibrations caused by trucks that have at least three axles, such as 18-wheeler trucks.

[0029] To illustrate, the luminaires 102-116 may each distinguish heavy trucks such as, for example, an 18-wheeler truck from other vehicles based on one or more of radar signals, luminaire pole tilting caused vehicles, engine sounds, etc. For example, the amplitude and duration of reflected radar signals along with the speed of a vehicle may be used to identify the type of a vehicle, for example, as a sedan, a small truck, a mid-size truck, or a heavy truck (e.g., an 18-wheeler truck) as can be readily understood by those of ordinary skill in the art with the benefit of this disclosure. Alternatively or in addition, sounds produced by engines of different types of vehicles may be processed by the luminaires 102- 116 to identify the type of a vehicle. Alternatively or in addition, because different types / classes of vehicles cause different degrees of tilting of the poles of outdoor luminaires in general, the luminaires 102-116 may each use the amount of tilting of the particular outdoor luminaire to identify the type of a vehicle that caused the tilting.

[0030] FIG. 3 illustrates a method 300 of operations of the outdoor luminaire system 100 of FIG. 1 to detect potholes and estimate pothole severity according to an example embodiment. Referring to FIGS. 1-3, to illustrate with respect to the outdoor luminaire 108, at step 302 of the method 300, the outdoor luminaire 108 may perform radar sensing using a radar device of the outdoor luminaire 108 corresponding to the radar device 208 of the outdoor luminaire 200. For example, as the vehicle 136 moves through the radar detection range of the outdoor luminaire 108, the outdoor luminaire 108 may use radar signals transmitted and received by the radar device of the outdoor luminaire 108 to detect the vehicle 136. Based on radar sensing by the radar device, the outdoor luminaire 108 (e.g., the controller of the outdoor luminaire 108 corresponding to the controller 202 of the outdoor luminaire 200) may also determine whether other vehicles are present with the radar detection range of the radar device of the outdoor luminaire. For example, the radar detection range of the radar device of the outdoor luminaire 108 may serve as a threshold distance for detecting vehicles by the outdoor luminaire 108 relative to the location of the outdoor luminaire 108. At step 304 of the method 300, the outdoor luminaire 108 may perform audio sensing using the microphone device of the outdoor luminaire 108 corresponding to the microphone device 206 of the outdoor luminaire 200. At step 306 of the method 300, the outdoor luminaire 108 may perform vibration sensing of the outdoor luminaire 108 using the vibration sensor of the outdoor luminaire 108 corresponding to the vibration sensor 210 of the outdoor luminaire 200. At step 308 of the method 300, the outdoor luminaire 108 may perform tilt sensing of the pole of the outdoor luminaire 108 using the tilt sensor of the outdoor luminaire 108 corresponding to the tilt sensor 212 of the outdoor luminaire 200.

[0031] In some example embodiments, at step 310 of the method 300, the outdoor luminaire 108 may determine the type of the vehicle 136. For example, the outdoor luminaire 108 may use information from the radar sensing operation at step 302 to identify the type of the vehicle 136 as a heavy truck (e.g., an 18-wheeler truck) or generally as a three-plus axle vehicle (i.e., a vehicle with at least three axles). Alternatively or in addition, the outdoor luminaire 108 may perform engine sound classification of the sound of the engine of the vehicle 136 captured by the microphone device of the outdoor luminaire 108 to identify the type of the vehicle 136 as a heavy truck or generally as a three-plus axle vehicle. Alternatively or in addition, the outdoor luminaire 108 may process (e.g., classify) vibration information from the vibration sensor of the outdoor luminaire 108 to identify the type of the vehicle 136 as a heavy truck or generally as a three-plus axle vehicle. Alternatively or in addition, the outdoor luminaire 108 may use information from the tilt sensor at step 308 to identify the type of the vehicle 136 as a heavy truck or generally as a three-plus axle vehicle.

[0032] In some example embodiments, at step 312 of the method 300, the outdoor luminaire 108 may detect a pothole 126 in the street 118 based on sounds received by the microphone device of the outdoor luminaire 108 and based on vibrations of the outdoor luminaire 108 sensed by the vibration sensor of the outdoor luminaire 108. The sounds and the vibrations used by the outdoor luminaire 108 to detect the pothole 126 may be generated or otherwise caused by the vehicle 136 moving through (e.g., driving over or entering and leaving) the pothole 126.

[0033] In some example embodiments, the outdoor luminaire 108 may perform operations to detect the pothole 126 based on the sounds captured at step 304 and vibrations sensed at step 306 and caused by the vehicle 136 hitting (i.e., driving over or entering and leaving) the pothole 126 if the outdoor luminaire 108 detects at step 310 of the method 300 that the vehicle 136 is a truck (e.g., a heavy truck such as an 18 -wheel er truck). Alternatively or in addition, the outdoor luminaire 108 may perform operations to detect the pothole 126 based on the sounds and vibrations caused by the vehicle 136 hitting the pothole 126 if the outdoor luminaire 108 determines that no other vehicles are within a radar detection range of the radar device of the outdoor luminaire 108. Alternatively or in addition, the outdoor luminaire 108 may perform operations to detect the pothole 126 based on the sounds and vibrations caused by the vehicle 136 hitting the pothole 126 if the outdoor luminaire 108 determines that the wind speed at the location of the outdoor luminaire 108 is at or below a threshold wind speed (e.g., 1 mile per hour, 3 miles per hour, 7 miles per hour, or 12 miles per hour).

[0034] In some example embodiments, the outdoor luminaire 108 may execute an Al model such as one or more Al models of the Al models 222 shown in FIG. 2 to detect the pothole 126 based on the sounds and vibrations caused by the vehicle 136 hitting the pothole 126. For example, the controller of the outdoor luminaire 108 corresponding to the controller 202 of the outdoor luminaire 200 may execute one or more Al models to detect the pothole 126. To illustrate, training data used to train one or more Al models of the Al models 222 may be obtained by having different vehicles, including trucks, drive over smooth pavement and through potholes of different sizes at different locations relative to different outdoor luminaires used in generating the training data. Sounds received by microphone devices of outdoor luminaires and vibrations of the outdoor luminaires sensed by respective vibration sensors of the outdoor luminaires may be labelled and used as training data for the one or more Al models of the Al models 222. In some cases, the labelled data can also be used to generate additional training data, for example, by executing an autoregressive predictive coding (APC) algorithm as readily understood by those of ordinary skill in the art, and the APC generated training data can also be used to train one or more Al models of the Al models 222, for example, to detect potholes.

[0035] In some example embodiments, a data unit or an input vector used during training or inference may represent sounds and vibrations generated by all tires of a vehicle that go over a pothole. For example, a single data unit / input vector of a training data or inference data may include or otherwise represent sounds and vibrations generated by all same-side tires of a truck that hit a pothole. To illustrate with respect to a three-axle truck, a time window that is large enough to accommodate sounds and vibrations generated by all three same-side longitudinally separated tires of the truck that hit a pothole (equivalently, pass over the same location) may be used in obtaining labelled training data units. Because of length variations among different trucks, a time window that is large enough to accommodate sounds and vibrations generated by all same-side tires of a vehicle that has the largest length between the same-side tires may be used as a fixed time window with respect to all trucks used in generating training data and during inference. A minimum speed (e.g., 20 miles per hour (mph), 30 mph, or 40 mph) of the longest truck may be considered for selecting the size of the fixed time window. In general, the time window used during training data generation may be the same size as the time window used during inferencing for input vector generation from sounds and vibrations caused by trucks. Individual input vectors used during training and inferencing may be a concatenation of a sound vector and a vibration vector as can be readily understood by those of ordinary skill in the art with the benefit of this disclosure.

[0036] To be clear, both during training and inferencing, a fixed time window may serve as a sliding window, where one instance of the sliding window includes occurrences of sounds and vibrations generated by all relevant tires of a truck hitting a pothole, where some instances of the sliding window include occurrences of sounds and vibrations generated by some but not all relevant tires of the truck hitting the pothole, and where some instances of the sliding window do not include occurrences of sounds and vibrations generated by the tires of the truck hitting the pothole. For example, during training data generation, data derived from or otherwise associated with the instance of the sliding window that encompasses the occurrences of sounds and vibrations generated by all relevant tires of the truck hitting a pothole may be labelled as indicating a pothole while the other instances are labelled as not indicating a pothole. Such labelled data may be used to train one or more Al models of the Al models 222. Alternatively or in addition, an APC algorithm may be executed on the data associated with the different instances of the sliding window, and the execution of the APC algorithm may produce data that can be used to train one or more Al models of the Al models 222.

[0037] In general, one or more Al models of the Al models 222 may be trained using supervised learning or self-learning as can be readily understood by those of ordinary skill in the art with the benefit of this disclosure. One or more Al models of the Al models 222 used to detect a pothole may be, for example, a convolutional neural network that can be trained and executed to classify time windows of sounds and vibrations as indicating a presence of a pothole or not.

[0038] As described above, at step 312 of the method 300, the outdoor luminaire 108 may execute one or more Al models of the Al models 222 to detect the pothole 126 in the street 118 based on sounds received by the microphone device of the outdoor luminaire 108 and based on vibrations of the outdoor luminaire 108 sensed by the vibration sensor of the outdoor luminaire 108. As described above, the sounds and vibrations processed by the outdoor luminaire 108 include the sounds and vibrations resulting from the vehicle 136 or another vehicle that is identified, for example, as a heavy truck driving through the pothole 126. Because the location of the outdoor luminaire 108 is known, the general location of the pothole 126 may be estimated or otherwise determined to be in the vicinity of the known location of the outdoor luminaire 108, for example, at step 314 of the method 300. The outdoor luminaire 108 may determine that the presence of the pothole 126 in the street 118 after detecting the presence of the pothole 126 multiple times based on sounds and vibrations caused by multiple vehicles, such as multiple heavy trucks.

[0039] In some example embodiments, the outdoor luminaires 102-106 and 110-116 of the system 100 may detect potholes in the street 118 in the manner described with respect to the outdoor luminaire 108 and the pothole 126. For example, the outdoor luminaire 106 may execute one or more Al models of the Al models 222 to detect the pothole 126 in the street 118 based on sounds received by the microphone device of the outdoor luminaire 106 and based on vibrations of the outdoor luminaire 106 sensed by the vibration sensor of the outdoor luminaire 106. For example, the sounds and vibrations used by the outdoor luminaire 106 to detect the pothole 126 may be generated, for example, when the vehicle 136 drives through the pothole 126. Indeed, the sounds used by the outdoor luminaire 106 to detect the pothole 126 may be the same sounds used by the outdoor luminaire 108 to detect the pothole 126 except for variations resulting from, for example, different distances and directions of the outdoor luminaires 106 and 108 relative to the pothole 126. The vibrations of the outdoor luminaire 106 and the vibrations of the outdoor luminaire 108 may be caused by the same occurrence of a vehicle, such as the vehicle 136, driving through the pothole 126.

[0040] In some example embodiments, after the detection of the pothole 126 by the outdoor luminaires 106 and 108 individually, the general location of the pothole 126 may be estimated or otherwise determined to be in the vicinity of the known locations of the outdoor luminaires 106 and 108, for example, at step 314 of the method 300. To illustrate, time stamp information associated with sounds received by the outdoor luminaires 106 and 108 and with vibrations sensed by the outdoor luminaires 106 and 108 may be used to determine that the pothole 126 detected by the outdoor luminaires 106 and 108 is the same pothole.

[0041] In some cases, after detecting the pothole 126, the outdoor luminaire 108 may send a message to nearby outdoor luminaires, such as the outdoor luminaires 106, 114, 116, for example, indicating the detection of a pothole (i.e., in this case the pothole 126) by the outdoor luminaire 108. Alternatively or in addition, the message may include instructions to check for a pothole. For example, because the pothole 126 may be too far to be detected by the outdoor luminaires 114, 116 based on sounds and vibrations resulting from a vehicle moving through the pothole 126, the outdoor luminaires 114, 116 may not detect a relevant pothole (i.e., the pothole 126 in this case). To illustrate, the outdoor luminaires 114, 116 may determine that no pothole is detected by the outdoor luminaires 114, 116 that lines up with the timing information related to the detection of the pothole 126 by the outdoor luminaire 108, for example, as indicated the message from the outdoor luminaire 108.

[0042] In contrast, the outdoor luminaire 106 may determine, based on relevant information provided by the outdoor luminaire 108, that a pothole detected by the outdoor luminaire 106 (i.e., already detected or detected in response to the received message) is the same pothole (i.e., the pothole 126 in this case) detected by the outdoor luminaire 108. Alternatively or in addition, the outdoor luminaire 106 may respond to the message from the outdoor luminaire 108 with relevant information (e.g., pothole detection related timing information) to enable the outdoor luminaire 108 to determine, at step 314 of the method 300, that the pothole detected by the outdoor luminaire 106 is the same pothole (i.e., the pothole 126 in this case) detected by the outdoor luminaire 108.

[0043] In some example embodiments, at step 314, the outdoor luminaire 108 and / or the outdoor luminaire 106 may determine that the pothole 126 is located, for example, between the outdoor luminaires 106 and 108, based on the detection of the pothole 126 by both of the outdoor luminaires 106 and 108. At step 314, the outdoor luminaire 108 may also validate / confirm the detection of the presence of the pothole 126 performed by the outdoor luminaire 108 at step 312. For example, at step 314, the outdoor luminaire 108 may validate / confirm the detection of the presence of the pothole 126 based on the detection of the presence of the pothole 126 by the outdoor luminaire 106 that is adjacent to the outdoor luminaire 108. At step 314, the outdoor luminaire 108 may validate / conform the detection of the presence of the pothole 126, for example, based on speed reduction of the vehicle 136 or another vehicle as the vehicle exits the pothole 126. To illustrate, drivers typically slow down immediately after exiting a pothole, and the typical speed reduction may be used to validate / confirm the detection of the presence of a pothole performed at step 312.

[0044] In some example embodiments, the outdoor luminaire 104 may execute one or more Al models of the Al models 222 to detect a pothole 128 in the same manner as described above with respect to the outdoor luminaire 108. To illustrate, the outdoor luminaire 104 may detect the pothole 128 in the street 118 based on sounds received by the microphone device of the outdoor luminaire 104 and based on vibrations of the outdoor luminaire 104 sensed by the vibration sensor of the outdoor luminaire 104. The sounds and vibrations used by the outdoor luminaire 104 to detect the pothole 128 may be generated by a vehicle, such as the vehicle 136, identified at step 310 as a truck or a heavy truck hitting the pothole 128. In contrast, a vehicle 138 may be identified, at step 310 of the method 300, as a small vehicle (e.g., a sedan), a pick-up truck, a small or mid-size SUV, or categorically as a vehicle with maximum two axles (i.e., a two-axle vehicle) or a non-heavy truck, and the outdoor luminaire 104 may not use sounds and vibrations that may be caused by the vehicle 138 to detect the pothole 128.

[0045] Instead, in some example embodiments, in addition to detecting the pothole 128 based on sounds and vibrations caused by a relatively large vehicle such as a heavy truck (e.g., the vehicle 136) hitting the pothole 128, the outdoor luminaire 104 may detect the pothole 128, at step 318 of the method 300, based on reductions in speed and changes of direction of relatively small vehicles (e.g., sedans) as such vehicles approach the pothole 128. In general, drivers of relatively small vehicles tend to slow down as the vehicles approach potholes and also tend to maneuver around the potholes. To illustrate, at step 310 of the method 300, the outdoor luminaire 104 may identify the vehicle 138 approaching the pothole 128 as a sedan. At step 316 of the method 300, the outdoor luminaire 104 may detect a reduction in speed and changes of direction by the vehicle 138. For example, the outdoor luminaire 104 may use the radar device of the outdoor luminaire 104 corresponding to the radar device 208 of the outdoor luminaire 200 to perform radar sensing at step 302 of the method 300. Based on information from the radar sensing performed at step 302, at step 316, the outdoor luminaire 104 may detect a reduction of speed and change(s) of direction of the vehicle 138 as the vehicle 138 approaches the pothole 128.

[0046] In some example embodiments, at step 318, the outdoor luminaire 104 may detect the pothole 128 based on the reduction(s) in speed and the change(s) of direction of the vehicle 138. The outdoor luminaire 104 may detect the pothole 128 multiple times based on reductions in speed and changes of direction of, for example, multiple sedans. At step 314, the outdoor luminaire 104 may validate / confirm the detection of the presence of the pothole 128 performed by the outdoor luminaire 104 at step 312. For example, at step 314, the outdoor luminaire 104 may validate / confirm the detection of the presence of the pothole 128 based on the detection by the outdoor luminaire 104 at step 318. Alternatively or in addition, at step 314, the outdoor luminaire 104 may validate / confirm the detection of the presence of the pothole 128 performed at step 312 based on the detection of the pothole 128 by the outdoor luminaire 102, that is on the same side of the street 118 as the outdoor luminaire 104. The outdoor luminaire 102 may detect the pothole 128 in the manner described above, for example, with respect to the outdoor luminaire 104 and the pothole 128 and with respect to the outdoor luminaire 108 and the pothole 126.

[0047] Alternatively or in addition, at step 314, the outdoor luminaire 104 may validate / confirm the detection of the presence of the pothole 128 performed at step 312 based on the detection of the pothole 128 by the outdoor luminaire 110 that is on the opposite side of the street 118 from the outdoor luminaire 104. The outdoor luminaire 110 may detect the pothole 128 in the manner described above, for example, with respect to the outdoor luminaire 104 and the pothole 128 and with respect to the outdoor luminaire 108 and the pothole 126. Based on the detections of the pothole 128 by the outdoor luminaires 102, 104, and 110 that have known locations, a relatively more accurate location of the pothole 128 may be determined at step 314 of the method 300, for example, by one or more outdoor luminaires of the outdoor luminaires 102, 104, 110. For example, the location of the pothole 128 may be determined to be relatively closer to the lane separator 124 (e.g., a median) because of the detection of the pothole 128 by the outdoor luminaire 110.

[0048] In some example embodiments, other potholes in the street 118, such as potholes 130, 132, may be detected in the same manner as described with respect to the potholes 126, 128. For example, the potholes 130, 132 may be detected based on sounds and vibrations caused by other vehicles, such as a vehicle 140 (e.g., an 18-wheeler truck), hitting the potholes 130, 132. To illustrate, the outdoor luminaire 112 may detect the pothole 130, which may be a relatively smaller (e.g., narrower and / or shallower) pothole than the potholes 126, 128 in the manner described above with respect to the potholes 126, 128. The outdoor luminaires 114, 116 may each detect the pothole 132 in the manner described above with respect to the potholes 126, 128. The general locations of the potholes 130, 132 may be determined based on the known locations of the outdoor luminaires 112, 114, 116 in the manner described above with respect to the potholes 126, 128.

[0049] In some cases, a relatively larger pothole is followed by one or more smaller potholes. For example, the pothole 132 may be followed by a pothole 134 that is smaller (e.g., narrower and / or shallower) than the pothole 132. In some example embodiments, the outdoor luminaire 114, 116 may each determine that the pothole 132 is followed by the pothole 134, based on, for example, sound and / or vibration patterns (e.g., a pattern of primary and secondary sound amplitude peaks) caused by, for example, the vehicle 140 as the tires of the vehicle 140 move through the pothole 132 and the pothole 134 in the direction shown by the arrow 144. For example, a front tire of the vehicle 140 may move through both of the potholes 132, 134 before another tire of the vehicle 140 reaches the pothole 132. Because the amplitude peaks of the sounds produced by the tires of vehicle 140 hitting the pothole 132 are different from the amplitude peaks of the sounds produced by the tires of vehicle 140 hitting the pothole 134, the outdoor luminaire 114, 116 may each determine that the pothole 132 is followed by the pothole 134 based on the sound patterns resulting from the tires of the vehicle 140 hitting the potholes 132, 134. Alternatively or in addition, the patterns of amplitude peaks of vibrations of the outdoor luminaire 114, 116 caused the vehicle 140 hitting the potholes 132, 134 can be used to determine that the pothole 132 is followed by the pothole 134.

[0050] In some example embodiments, the outdoor luminaires 102-116 may each estimate a severity of a pothole detected by the particular outdoor luminaire. For example, the outdoor luminaires 102-116 may each execute one or more Al models of the Al models 222 to estimate the severity of a pothole detected by the particular outdoor luminaire. To illustrate, the outdoor luminaire 108 may estimate the severity of the pothole 126. As another example, the outdoor luminaire 106 may estimate the severity of the pothole 126. As yet another example, the outdoor luminaire 112 may estimate the severity of the pothole 130. In general, the severity of a particular pothole as estimated by a particular outdoor luminaire of the system 100 is intended to indicate the level of deterioration of the pavement at the pothole and may correspond to, for example, the dimensions (e.g., length, width, and / or depth) of the pothole.

[0051] In some example embodiments, the outdoor luminaires 102-116 may indicate the severity of a pothole in categories such as, for example, low, medium, high, and extreme or other suitable categories that indicate different severity levels. For example, the outdoor luminaires 102-116 may each execute one or more Al models of the Al models 222 to classify a detected pothole into one of the severity categories.

[0052] In some example embodiments, each luminaire of the outdoor luminaires 102- 116 may estimate a severity of a pothole based on sounds and vibrations generated or otherwise caused by a vehicle driving through the pothole, the location of the vehicle relative to the particular outdoor luminaire, and the speed of the vehicle. The location of a vehicle relative to a particular outdoor luminaire and the speed of the vehicle may be determined using radar signals as can be readily understood by those of ordinary skill in the art with the benefit of this disclosure. The location of the vehicle and the speed of the vehicle that are relevant to the estimation of pothole severity may be the relative location of the vehicle and the speed of the vehicle at the time the vehicle moves through the pothole. Because amplitudes of sounds and vibrations received or sensed by a particular outdoor luminaire may depend on the location of a pothole relative to the outdoor luminaire, the location of a vehicle hitting the pothole relative to the location of the outdoor luminaire may be relevant to the reliable estimation of the severity of the pothole. Because amplitudes of sounds and vibrations caused by a vehicle hitting (i.e., moving through) a pothole may depend on the speed of the vehicle, the speed of the vehicle at the time that the vehicle hits the pothole may be relevant to the reliable estimation of the severity of the pothole. Equation 1 shows an equation for estimating the severity of a pothole by an outdoor luminaire with respect to vehicle i that hits the pothole:

[0053] Severity = argmax Prob Pi = A |S)([t, t + T]), Li [t, t + T ), Gi [t, t + T]) In Equation 1, X ={0, 1, 2, 3} and corresponds to different severity levels, such as low, medium, high, and extreme; Si is the speed of vehicle I; Li is the location of vehicle i (relative to the outdoor luminaire); and Gi is a concatenation of an audio signal vector and a vibration signal vector representing sounds and vibrations caused by vehicle i that hit the pothole. T is the time window that is large enough to encompass sounds and vibrations caused by the tires of the vehicle that hit (i.e., move through) the pothole.

[0054] In some example embodiments, one or more of the Al models 222 used in estimating the severity of potholes may be trained and used in the manner described above with respect to Al models used to detect potholes. In addition to sounds and vibrations caused by vehicles hitting potholes, vehicle speed and vehicle location may be collected and labelled for training Al models used in pothole severity estimation.

[0055] To illustrate with respect to the outdoor luminaire 108, at step 320 of the method 300, the outdoor luminaire 108 may determine the location of the vehicle 136 relative to the location of the outdoor luminaire 108, for example, as a distance from the outdoor luminaire 108. The outdoor luminaire 108 may also determine the speed of the vehicle 136. For example, the outdoor luminaire 108 may determine or otherwise obtain the location and speed of the vehicle 136 from step 302 of the method 300 executed by the outdoor luminaire 108. The location and speed of the vehicle 136 are determined aligned with the timing of the sounds and vibrations caused by the vehicle hitting the pothole 126.

[0056] In some example embodiments, at step 322, the outdoor luminaire 108 may execute one or more Al models of the Al models 222 to estimate the severity of the pothole 126 based on sounds and vibrations generated or otherwise caused by the vehicle 136 driving through the pothole 126, the location of the vehicle 136 relative to the outdoor luminaire 108, and the speed of the vehicle 136. In general, the outdoor luminaire 108 may estimate the severity of the pothole 126 at step 322 of the method 300 after the outdoor luminaire 108 detected the pothole 126 at step 312 of the method 300 as described above. For example, the outdoor luminaire 108 may estimate the severity of the pothole 126 as high, for example, from among severity categories of low, medium, high, and extreme. To illustrate, at step 322, the outdoor luminaire 108 may execute one or more Al models of the Al models that perform classification to estimate the severity of the pothole 126 as high.

[0057] In some example embodiments, the outdoor luminaires 102-106 and 110-116 of the system 100 may perform estimation of severity of relevant potholes in the manner described with respect to the outdoor luminaire 108 and the pothole 126. For example, the outdoor luminaire 106 may execute one or more Al models of the Al models to estimate the severity of the pothole 126 in the manner described with respect to the outdoor luminaire 108. In some cases, after determining the severity of the pothole 126, the outdoor luminaire 106 may provide the severity information, for example, to the outdoor luminaire 108. For example, the outdoor luminaire 106 may automatically send the severity information, for example, to nearby outdoor luminaires including the outdoor luminaire 108 or may send the severity information, for example, in response to a message sent by the outdoor luminaire 108 about the detection of the pothole 126 as described above with respect to steps 312 and 314. The outdoor luminaire 106 may estimate that the severity of the pothole 126 to be the same as the severity of the pothole 126 estimated by the outdoor luminaire 108 (e.g., high). Alternatively, the outdoor luminaire 106 may estimate that the severity of the pothole 126 to be different from the severity of the pothole 126 estimated by the outdoor luminaire 108.

[0058] As another example, the outdoor luminaire 112 may execute one or more Al models of the Al models to estimate the severity of the pothole 126 in the manner described with respect to the outdoor luminaire 108. For example, the outdoor luminaire 112 may estimate the severity of the pothole 130 as low, and the outdoor luminaire 108 may estimate the severity of the pothole 126 as high. In general, the amplitudes of the sounds and vibrations caused by a vehicle hitting a pothole may be directly correlated to the severity of the pothole, where the severity of the pothole may be related to the dimensions of the pothole such as depth, width, and / or length.

[0059] In some example embodiments, the outdoor luminaires 102-116 may each process, at step 324 of the method 300, severity information indicating the severity of the pothole. To illustrate, a particular outdoor luminaire may process the severity information indicating the severity of the pothole as estimated at step 322 by the particular outdoor luminaire and possibly other outdoor luminaires. For example, the outdoor luminaire 108 may process, at step 322, the estimated severity of the pothole 126 as estimated by the outdoor luminaire 108 as well as by the outdoor luminaire 106. The outdoor luminaire 108 may calculate, for example, the arithmetic mean of the estimates of the severity of the pothole 126 made by the outdoor luminaires 106, 108 and use that value as the estimate of the severity of the pothole 126. Alternatively, the outdoor luminaire 108 may select the highest estimate of the estimates of the severity of the pothole 126 made by the outdoor luminaires 106, 108 as the estimate of the severity of the pothole 126.

[0060] In some example embodiments, the outdoor luminaires 102-116 can each send, at step 326 of the method 300, a respective notification indicating a detection of a respective pothole and / or an estimate of the severity of the pothole. For example, the outdoor luminaires 102-116 may send notifications to a relevant governmental authority (e.g., a public works authority) and / or other entities. To illustrate, the outdoor luminaire 108 may send, at step 326, a notification indicating the detection of the presence of the pothole 126 as determined at step 312 or as confirmed at step 314. The notification sent by the outdoor luminaire 108 may, alternatively or in addition, include information indicating the severity of the pothole 126 as estimated by the outdoor luminaire 108 at step 322 or as determined at step 324. The notification sent by the outdoor luminaire 108 may also include other information such as an identifier of the outdoor luminaire 108, the GPS location of the outdoor luminaire 108, and / or other relevant information.

[0061] In general, some operations of the method 300 of FIG. 3, such as pothole detection and severity estimation described herein with respect to the outdoor luminaires 102- 116, may be performed by the server 146 in the manner described with respect to the outdoor luminaires 102-116. The server 146 may receive relevant information from the outdoor luminaires 102-116 to perform some steps of the method 300 as can be readily understood by those of ordinary skill in the art with the benefit of this disclosure.

[0062] By using sounds generated by the tires of a vehicle driving through (i.e., hitting or entering and leaving) a pothole and vibrations of an outdoor luminaire caused by the tires of a vehicle driving through the pothole, the outdoor luminaires 102-116 may each detect the presence of a street pothole and may estimate the severity of a detected pothole as described above with respect to FIGS. 1-3. Because outdoor luminaires are widely available in many areas, an outdoor luminaire system, such as the outdoor luminaire system 100, may provide a large coverage area of pothole detection and severity estimation.

[0063] In some alternative embodiments, the system 100 may include more or fewer outdoor luminaires than shown without departing from the scope of this disclosure. In some example embodiments, the system 100 may include outdoor luminaires that are installed at a median (e.g., the lane separator 124) of the street 118 and that operate in the manner described above with respect to the outdoor luminaires 102-116 to detect pothole and estimate pothole severity without departing from the scope of this disclosure. In some alternative embodiments, some operations of detecting potholes and estimating severity of potholes may be performed by one or more luminaires of the outdoor luminaires 102-116, and some other operations of detecting potholes and estimating severity of potholes may be performed by the server 146 without departing from the scope of this disclosure.

[0064] In some alternative embodiments, the outdoor luminaire 200 may include more or fewer components than shown in FIG. 2 without departing from the scope of this disclosure. In some alternative embodiments, some of the components of the outdoor luminaire 200 may be integrated into a single component without departing from the scope of this disclosure.

[0065] In some alternative embodiments, the method 300 may include other operations than shown in FIG. 3 without departing from the scope of this disclosure. In some example embodiments, the outdoor luminaires 102-116 may each use the amount of tilting of the pole of the particular outdoor luminaire to as measured by the tilt sensor 212, for example, to perform operations such as detecting potholes, estimating severity of detected potholes, etc. In some alternative embodiments, one or more operations of the method 300 may be omitted without departing from the scope of this disclosure. For example, pothole severity analysis at step 322 may be omitted.

[0066] FIG. 4 illustrates a method 400 of detecting potholes and estimating pothole severity by the outdoor luminaire system 100 of FIG. 1 according to an example embodiment. As described above, the outdoor luminaire 200 of FIG. 2 corresponds to each luminaire of the luminaires 102-116 of the outdoor luminaire system 100, and the components of the luminaire 200 correspond to respective components of the luminaires 102- 116. Referring to FIGS. 1-4, in some example embodiments, at step 402, the method 400 includes detecting a vehicle moving on a street. For example, the outdoor luminaires 102-116 may each detect vehicles that are, for example, in the radar detection range of the radar device of the respective outdoor luminaire. For example, the outdoor luminaire 108 may detect the vehicle 136 using radar signals. The outdoor luminaire 108 may also determine the speed and relative location of the vehicle 136 using radar signals.

[0067] In some example embodiments, at step 404, the method 400 includes determining that no other vehicle is within a threshold distance from the outdoor luminaire. For example, after or while detecting the vehicle 136 at step 402, the outdoor luminaire 108 may determine that no other vehicle (i.e., no vehicle other than the vehicle 136) is within a threshold distance (e.g., a radar detection range) of the outdoor luminaire 108. As another example, after or while detecting the vehicle 140 at step 402, the outdoor luminaire 110 may determine that no other vehicle (i.e., no vehicle other than the vehicle 140) is within a threshold distance (e.g., a radar detection range) of the outdoor luminaire 110. To illustrate, the outdoor luminaire 108 may determine that no other vehicle is present to avoid interference, for example, during subsequent steps of the method 400. The method 400 may also include determining that wind speed around the vicinity of a particular outdoor luminaire (e.g., the outdoor luminaire 108) is below a threshold speed before attempting to detect a pothole.

[0068] In some example embodiments, at step 406, the method 400 includes determining a type of the vehicle. To illustrate, the outdoor luminaires 102-116 may each detect the type of a vehicle based on radar signals, engine sounds, vibrations of the particular outdoor luminaire caused by the vehicle passing by, and / or based on the amount of tilting of the particular outdoor luminaire caused by the vehicle passing by. For example, the outdoor luminaire 108 may determine whether the vehicle 136 is a truck (e.g., a heavy truck such as an 18-wheeler truck) or generally as a three-plus axle vehicle or another type of vehicle such as a sedan or generally as a two-axle vehicle.

[0069] In general, in contrast to sounds and vibrations caused by relatively smaller vehicles, sounds and vibrations caused by trucks, such as heavy trucks (e.g., at least three- axle trucks), hitting potholes have relatively higher amplitudes and more amplitude peaks (i.e., from three or more tires hitting a pothole). As such, sounds and vibrations caused by trucks hitting potholes can enable more reliable detections of potholes. In contrast, drivers of smaller vehicles, such as sedans, typically slow down when approaching potholes and maneuver around potholes. Identifying the type of a vehicle may enable a reliable pothole detection of potholes.

[0070] In some example embodiments, at step 408, the method 400 includes detecting a pothole in the street based on sounds received by the outdoor luminaire and based on vibrations of the outdoor luminaire sensed by the outdoor luminaire. For example, step 408 may be executed if the type of the vehicle detected at step 406 is a truck or a heavy truck (i.e., a truck with at least three axles) or generally as a three-plus axle vehicle. In general, the outdoor luminaires 102-116 may each execute one or more Al models of the Al models 222 to detect potholes. To illustrate with respect to the outdoor luminaire 108, the outdoor luminaire 108 may detect the pothole 126 based on sounds received by the outdoor luminaire 108 and based on vibrations of the outdoor luminaire 108 sensed by the vibration sensor of the outdoor luminaire 108. As described above, the sounds received by the outdoor luminaire 108 and used to detect the pothole 126 are sounds generated or otherwise caused by the vehicle 136 or another truck (e.g., an 18-wheeler truck) hitting the pothole 126. Also as described above, the vibrations of the outdoor luminaire 108 sensed by the vibration sensor of the outdoor luminaire 108 and used to detect the pothole 126 are vibrations caused by the vehicle 136 or another truck (e.g., an 18-wheeler truck) hitting the pothole 126. In some alternative embodiments, the amount of tilting of a particular outdoor luminaire caused by a vehicle hitting a pothole may be used to detect the presence of the pothole in a similar manner as described above with respect to the use of sounds and vibrations.

[0071] In some example embodiments, at step 410, the method 400 includes estimating a severity of the pothole detected at step 408 at least based on the sounds, the vibrations, a relative location of the vehicle with respect to the outdoor luminaire, and a speed of the vehicle. For example, the outdoor luminaires 102-116 may each execute one or more Al models of the Al models 222 to estimate severities of potholes in a manner described above with respect to step 322 of the method 300. For example, the outdoor luminaire 108 may estimate the severity of the pothole 126 detected by the outdoor luminaire 108 at step 408 based on the sounds caused by the vehicle 136 hitting the pothole 126 and received by the outdoor luminaire 108, the vibrations of the outdoor luminaire 108 caused by the vehicle 136 hitting the pothole 126, the location (e.g., distance) of the vehicle 136 relative to the outdoor luminaire 108, and the speed of the vehicle 136. The relevant location (e.g., distance) of the vehicle 136 and the relevant speed of the vehicle 136 are those that are time- wise aligned with the sounds and the vibrations caused by the vehicle 136 hitting the pothole 126.

[0072] In some example embodiments, at step 412, the method 400 includes validating / confirming the presence of the pothole detected at step 408. To illustrate, a validation of the presence of the pothole 126 may be performed by the outdoor luminaire 108 repeatedly based on sounds and vibrations caused by multiple trucks (e.g., heavy trucks). For example, after the outdoor luminaire 108 detects the pothole 126 based on sounds and vibrations caused by the vehicle 136 hitting the pothole 126, the outdoor luminaire 108 may detect the pothole 126 based on sounds and vibrations caused by another truck hitting the pothole 126. The outdoor luminaire 108 may validate the detection of the pothole by repeatedly detecting the pothole 126 based on sounds and vibrations caused by other trucks hitting the pothole 126.

[0073] In some example embodiments, the outdoor luminaire 108 may detect the pothole 126 based on smaller cars slowing down and maneuvering around the pothole 126 as described with respect to steps 316 and 318 of the method 300 and the pothole 128. For example, after the outdoor luminaire 108 detects the pothole 126 as described at step 408, the outdoor luminaire 108 may validate / confirm, at step 412, the detection of the pothole 126 at step 408 based on, for example, one or more smaller cars (e.g., the vehicle 138) slowing down and maneuvering around the pothole 126. The validation of the detection of the presence of a pothole may be performed at step 412 based on the detection of the presence of the same pothole by another luminaire. For example, the detection of the presence of the pothole 126 by the outdoor luminaire 108 may be confirmed by the detection of the presence of the pothole 126 by the outdoor luminaire 106 as described above with respect to FIGS. 1- 3. The validation of the detection of the presence of a pothole at step 412 can also be done based on the speed reduction of vehicles that typically occurs immediately after vehicles drive though a pothole as described above with respect to step 314 of FIG. 3.

[0074] In some example embodiments, at step 414, the method 400 includes determining a final severity of the pothole detected at step 408. For example, the outdoor luminaire 108 may calculate, for example, the average of the multiple values of the severity of the pothole 126 and use that average value as the final value of the severity of the pothole 126. The multiple severity values may be from, for example, the outdoor luminaire 108 and the outdoor luminaire 106 may be at a location to be able to detect the pothole 126. In addition, the outdoor luminaires 106 and 108 may each estimate the severity of the pothole 126 multiple times based on different vehicles (e.g., heavy trucks) driving through the pothole at different times. In some alternative embodiments, the outdoor luminaire 108 may select, for example, the highest value or the lowest value from among multiple values of the severity of the pothole 126 and use the selected value as the final value of the severity of the pothole 126.

[0075] In some example embodiments, at step 416, the method 400 includes estimating a location of the pothole detected at step 408. For example, if the pothole 130 is detected by the outdoor luminaire 112 and by no other outdoor luminaire of the outdoor luminaire system 100, the location of the pothole 130 may be estimated to be close to the GPS location of the outdoor luminaire 112. As another example, if the pothole 126 is detected by the outdoor luminaires 106 and 108 and by no other outdoor luminaires, the location of the pothole 126 may be estimated to be between the GPS locations of the outdoor luminaires 106 and 108. As another example, if the pothole 128 is detected by the outdoor luminaires 102, 104, and 110 and by no other outdoor luminaires, the location of the pothole 128 may be estimated to be between the GPS locations of the outdoor luminaires 102 and 104 and relatively closer to the center of the street 118.

[0076] In some example embodiments, at step 418, the method 400 includes sending a notification related to the presence of the pothole detected at step 408. For example, the notification may include information indicating the presence of the pothole 126, the severity of the pothole as determined at step 410 and / or at step 414, the GPS location of, for example, the outdoor luminaire 108 and possibly the location of the outdoor luminaire 106, and / or identifier information of the outdoor luminaire 108 and possibly that of the outdoor luminaire 106. The notification may also include other information such as date and time of notification, detection statistics, etc. The notification may be sent, for example, multiple times a day, after each detection at step 408, once a day, once every few days, once a week, or at another rate. The notification may be sent, at step 418, by each outdoor luminaire of the outdoor luminaire system 100 that detects a pothole, by one of multiple outdoor luminaires that detect the same pothole, or by another entity such as the server 146. In some example embodiments, the notification may be sent based on pothole detection at step 408 and severity estimation at step 410 and without considering the validation / confirmation of the presence of the pothole at step 412 and / or the final severity of the pothole determined at step 414.

[0077] In general, some operations of the method 400 of FIG. 4, such as pothole detection and severity estimation described herein with respect to the outdoor luminaires 102- 116, may be performed by the server 146 in the manner described with respect to the outdoor luminaires 102-116. The server 146 may receive relevant information from the outdoor luminaires 102-116 to perform some steps of the method 400 as can be readily understood by those of ordinary skill in the art with the benefit of this disclosure.

[0078] In some alternative embodiments, the method 400 may include other steps than shown and / or described without departing from the scope of this disclosure. In some alternative embodiments, the method 400 may include performing operations such as detecting potholes, estimating severity of detected potholes, etc. by the outdoor luminaires 102-116 based on the amount of tilting of the pole of the particular outdoor luminaire. In some alternative embodiments, some of the steps of the method 400 may be performed in a different order than shown without departing from the scope of this disclosure. In some alternative embodiments, some of the steps of the method 400 may be omitted without departing from the scope of this disclosure.

[0079] Although particular embodiments have been described herein in detail, the descriptions are by way of example. The features of the example embodiments described herein are representative and, in alternative embodiments, certain features, elements, and / or steps may be added or omitted. Additionally, modifications to aspects of the example embodiments described herein may be made by those skilled in the art without departing from the scope of the following claims, the scope of which are to be accorded the broadest interpretation so as to encompass modifications and equivalent structures.

Claims

CLAIMS:

1. A method of street pothole detection, the method comprising: detecting (302, 402) a vehicle (136, 138, 140) moving on a street (118); determining (310, 406), by a processor (202, 146), a type of the vehicle; and detecting (312, 408), by the processor (202, 146), a presence of a pothole (126, 128, 130, 132) in the street based on sounds received by an outdoor luminaire (102-116) and based on vibrations of the outdoor luminaire sensed by the outdoor luminaire, wherein the sounds and the vibrations are caused by the vehicle moving through the pothole, and wherein the detecting the presence of the pothole is performed if the type of the vehicle is a three-plus axle vehicle by determining if the sounds and the vibrations have relatively higher amplitudes and more amplitude peaks than if the type of the vehicle is not the three-plus axle vehicle.

2. The method of Claim 1, further comprising estimating (322, 410) a severity of the pothole (126) at least based on the sounds, the vibrations, a relative location of the vehicle (136) with respect to the outdoor luminaire (108), and a speed of the vehicle (136).

3. The method of Claim 2, further comprising sending a notification (326, 418) indicating the presence of the pothole (126) at least as determined based on the sounds received by the outdoor luminaire (108) and the vibrations of the outdoor luminaire (108).

4. The method of Claim 1, further comprising the validating (314, 412) the presence of the pothole (126) detected based on the sounds and the vibrations.

5. The method of Claim 4, wherein the validating the presence of the pothole is performed at least based a second detection of the presence of the pothole (126) and wherein the second detection of the presence of the pothole is performed based on the sounds as received by a second outdoor luminaire (106) and based on vibrations of the second outdoor luminaire (106) sensed by the second outdoor luminaire (106).

6. The method of Claim 5, further comprising estimating (314, 416) a location of the pothole (126) with respect to a location of the outdoor luminaire (108) and a location of the second outdoor luminaire (106).

7. The method of Claim 6, wherein the second outdoor luminaire (106, 116) is located on a same side of the street (118) as the outdoor luminaire (108), in a median of the street (118) across from the outdoor luminaire (108), or on an opposite side of the street from the outdoor luminaire (108).

8. The method of Claim 4, wherein the validating the presence of the pothole is performed at least based on speed changes of vehicles (138), direction changes of the vehicles (138), or both.

9. The method of Claim 1, wherein the detecting the presence of the pothole is performed if the type of the vehicle is the truck with at least three axles.

10. The method of Claim 1, further comprising, before detecting the presence of the pothole (126), determining (404) that no other vehicle is present within a threshold distance from the outdoor luminaire (108), wherein the threshold distance is a radar detection range of the outdoor luminaire.

11. The method of Claim 1, further comprising determining that a wind speed at the outdoor luminaire is below a threshold wind speed such that detecting the presence of the pothole is unaffected by sound of wind and vibrations of the outdoor luminaire caused by the wind.

12. The method of Claim 1, wherein the processor (202, 146) is configured to execute an artificial intelligence (Al) model (222) to detect the presence of the pothole (126, 128, 130, 132) in the street.

13. An outdoor luminaire system (100) for street pothole detection, the outdoor luminaire system comprising:an outdoor luminaire (102-116, 200) comprising a radar device (208), a microphone device (206), and a vibration sensor (210), wherein the outdoor luminaire system (100) is configured to: detect (302, 402) a vehicle (136, 138, 140) moving on a street (118); determine (310, 406) a type of the vehicle; and detect (312, 408) a presence of a pothole (126, 128, 130, 132) in the street based on sounds received by the microphone device and based on vibrations of the outdoor luminaire sensed by the vibration sensor, wherein the sounds and the vibrations are caused by the vehicle moving through the pothole, and wherein the detecting the presence of the pothole is performed if the type of the vehicle is a three-plus axle vehicle by determining if the sounds and the vibrations have relatively higher amplitudes and more amplitude peaks than if the type of the vehicle is not the three-plus axle vehicle.

14. The outdoor luminaire system of Claim 13, wherein the outdoor luminaire system is further configured to estimate (322, 410) a severity of the pothole (126) at least based on the sounds, the vibrations, a relative location of the vehicle (136) with respect to the outdoor luminaire (108), and a speed of the vehicle (136).

15. The outdoor luminaire system of Claim 13, wherein the outdoor luminaire system is further configured to validate (314, 412) the presence of the pothole (126) detected based on the sounds and the vibrations, wherein validating the presence of the pothole (126) is performed at least based a second detection of the presence of the pothole, and wherein the second detection of the presence of the pothole is performed based on the sounds as received by a second outdoor luminaire (106) of the outdoor luminaire system and based on vibrations of the second outdoor luminaire (106) sensed by a vibration sensor of the second outdoor luminaire (106).

Images