Lane violation alerts

Abstract

Lane violation events may be detected based on processing of visual data captured by a device in a vehicle, and a driver may be alerted depending on the class of vehicle being driven, and provided that no exceptions of an enumerated list apply. In-cab and remote alerts may be generated as part of a system of driver safety and compliance.

Description

LANE VIOLATION ALERTSCROSS-REFERENCE TO RELATED APPLICATIONS[0001.1] This application claims the benefit of priority to US Provisional Patent Application No. 63 / 614,935, filed December 27, 2023, the entirety of which is incorporated by reference herein.TECHNICAL FIELD

[0001] The present disclosure relates to systems and methods for monitoring driving behavior, particularly, detecting a lane violation by a driver of a vehicle and alerting the driver and / or others to the lane violation.BACKGROUND

[0002] Traffic rules / laws are in place to reduce congestion on roads and allow smooth flow of traffic. Traffic rules can be different for different countries and can also vary from state to state, or province to province, in a country. One such traffic rule that is applicable in several state of the United States indicates that vehicles belonging to certain vehicle classes should be driven on a defined lane on certain roads having multiple lanes, and / or that driving in certain other lanes on such roads is prohibited, subject to exceptions. In some cases, drivers may not be aware of such rules. Furthermore, given the variability of rules and driving scenarios, it may be difficult for a driver to ascertain whether such rules apply to a given road and / or context, particularly while driving. Therefore, there is a need to reliably and accurately detect lane violations and alert the driver and / or fleet managers upon detecting lane violations so that drivers may follow such rules, avoid any applicable fines, and improve traffic predictability for all drivers and vehicles on the road, which in turn improves road safety generally.SUMMARY

[0003] Certain aspects of the present disclosure are directed to solving technical traffic scene comprehension challenges, which may include proper handling of noisy outputs of a scene comprehension neural network, properly detecting exception conditions, properly identifying which detectable lanes are applicable for the purposes ofcomplying with lane violation rules, the like, so that drivers can be beneficially alerted to instances when their vehicle is travelling in an inappropriate lane.

[0004] Certain aspects of the present disclosure provide methods for detection of a lane violation by a device, such as an edge device coupled to an external environment-facing camera installed in a vehicle. The method comprises receiving, by at least one processor of the computing device, visual data captured by at least one camera, the computing device being installed in the cabin of the vehicle; determining, by the at least one processor, a number of applicable lanes on a road on which the vehicle is traveling; identifying, by the at least one processor and based on the visual data, a lane in which the vehicle is travelling of the determined number of lanes; and generating, by the at least one processor, an alert based at least on the vehicle’s lane of travel on the road.

[0005] Certain aspects of the present disclosure provide a device for detection of a lane violation by a vehicle in which the device is installed, wherein the device is configured to perform any of the methods disclosed herein.

[0006] Certain aspects of the present disclosure provide a computer program product for detection of a lane violation by a vehicle. The computer program product includes a non-transitory computer-readable medium having stored instructions, the instructions being executable by one or more processors configured to detect a lane violation by a vehicle.BRIEF DESCRIPTION OF FIGURES

[0007] FIG. 1 illustrates a block diagram of the edge device for detection of a lane violation, according to an embodiment of the disclosure.

[0008] FIG. 2 illustrates a flow diagram for detection of a lane violation and generation of an alert, according to an embodiment of the disclosure.

[0009] FIG. 3 illustrates a flow diagram for determination of a number of lanes in a road, according to an embodiment of the disclosure.

[0010] FIG. 4 depicts an example scenario to illustrate lane sorting for the detected lanes.

[0011] FIG. 5 depicts an example scenario to illustrate removing supposed exit or entrance lanes from the detected lanes.

[0012] FIG. 6 depicts an example scenario to illustrate the compensation for missing lane boundary detections.

[0013] FIG. 7 depicts four example scenarios in which an in-cab alert may be generated or suppressed when the vehicle is in the left-most lane, according to an embodiment of the disclosure.

[0014] FIG. 8 depicts two example scenarios in which a lane violation alert may be suppressed based on various detected objects in a road scene, according to an embodiment of the disclosure.

[0015] FIG. 9 depicts a table of vehicle classes for which lane violation rules apply, by US State.DETAILED DESCRIPTION

[0016] The detailed description set forth below, in connection with the appended drawings, is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein may be practiced. The detailed description includes specific details for the purpose of providing a thorough understanding of the various concepts. However, it will be apparent to those skilled in the art that these concepts may be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form to avoid obscuring such concepts.

[0017] Based on the teachings, one skilled in the art should appreciate that the scope of the disclosure is intended to cover any aspect of the disclosure, whether implemented independently of or combined with any other aspect of the disclosure. For example, an apparatus may be implemented, or a method may be practiced using any number of the aspects set forth. In addition, the scope of the disclosure is intended to cover such an apparatus or method practiced using other structure, functionality, or structure and functionality in addition to or other than the various aspects of the disclosure set forth. Any aspect of the disclosure disclosed may be embodied by one or more elements of a claim.

[0018] The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

[0019] Although particular aspects are described herein, many variations and permutations of these aspects fall within the scope of the disclosure. Although some benefits and advantages of the preferred aspects are mentioned, the scope of the disclosure is not intended to be limited to particular benefits, uses or objectives. Rather,aspects of the disclosure are intended to be broadly applicable to different technologies, and system configurations, some of which are illustrated by way of example in the figures and in the following description of the preferred aspects. The detailed description and drawings are merely illustrative of the disclosure rather than limiting, the scope of the disclosure being defined by the appended claims and equivalents thereof.

[0020] Traffic rules may impose restrictions on which lanes certain vehicles may use or may be prohibited from using. Signs, laws indicating such restrictions and the applicability of these laws to vehicle classes may not standardized across different jurisdictions. In fact, traffic rules may vary from state to state in a country (e.g., U.S.A) and these traffic rules may evolve separately. One such rule is that trucks on a multiple lane roadway cannot use the left-hand lane (which may be referred to as a “passing” lane) unless an exception applies, such as they are taking an exit from the left or in an emergency. Drivers of vehicles may not be aware of these rules and may violate these rules. It may also be unclear when and how the rules apply, for example, where there is an entrance or exit lane, or when traffic is temporarily routed by traffic officers or construction crews.

[0021] In the United States and some other countries, a generally applicable component of these rules provides that for any roadway with 3 or more lanes in the same direction of travel, heavy commercial vehicles (except buses in some locales) are restricted only to the right hand lane, except when overtaking (passing), when they are allowed to do so only from the adjacent lane (i.e. middle lane of a three-lane road). Similarly, on any highway with more than one passing lane in the same direction, heavy commercial vehicles (again, except buses in some locales), may be restricted in ordinary operation to the right-hand travel lane (not counting exit or entrance lanes), and in overtaking and passing may be restricted to the next adjacent passing or travel lane, and shall not use any other lanes except in an emergency.

[0022] To detect a lane violation as defined by such rules and in appropriate circumstances, and to alert the driver of such violations, an edge device is installed in the cabin of the vehicle. The edge device may monitor driving behavior generally, which may include functionality to detect and determine road characteristics, such as the presence, location, and characteristics of visually detectable lane boundaries. In accordance with certain aspects of the present disclosure, an installed edge device may be configured (or updated) to include lane violation detection and / or alert generationfunctionality. FIG. 1 depicts a block diagram of the edge device 101, in accordance with certain aspects of the present disclosure. The edge device 101 is a computing device that includes a processor, one or more sensors, a communication module, memory, input / output module, etc. The processor includes a lane violation module for detection of a lane violation. The lane violation module may also determine if and when to alert the driver of the lane violation, such as based on whether a threshold period of time has passed since the driver entered a certain lane, such as the left-hand lane. The one or more sensors may include a positional sensor (GPS, GNSS), and / or one or more cameras to capture the visual data related to a path travelled by the vehicle. The visual data can be processed by one or more machine learning models on the edge device to detect objects depicted in visual data. In one embodiment, the edge device is configured to detect and categorize at least one lane on the road from the captured visual data. The edge device may be configured to perform all or some of the steps shown in FIG. 2.

[0023] To determine the number of lanes for traffic in the vehicle’s direction of travel for the vehicle in which the edge device is installed, the lane violation module includes a road information module. The road information module receives visual data related to detections of lane boundaries and road boundaries in image frames, each of which may be organized into lane tracks and road boundary tracks, respectively, across multiple frames of processed visual data. The lane boundary and road boundary tracks may then be further processed to identify the applicable lanes (which may be lanes in the vehicle’s direction of travel that are not entrance / exit lanes, carpool lanes, emergency shoulder lanes, and the like) for the vehicle in which the edge device is installed. The road information module may optionally output the number of lanes on the road that the ego vehicle is driving on (applicable, in the direction of travel, and / or all). The road information module localizes the lane that the ego vehicle is currently in. The road information module can include one or more machine learning models. In some embodiments, the road information module may be configured to perform all or some of the steps illustrated in FIG. 3 to localize the lane on which the vehicle is travelling.Step 1 : The module organizes detected road boundary and lane boundary tracks in a data structure, such that the road boundary and lane boundary tracks have unique track IDs. For the lane tracker, additional lane attributes of the detected lanes may be available based on the output of a ML model that detects lanes in individual frames, which may be referred to as a LaneDNN model. Lane attributes may include, forexample, a determination as to whether each detected lane boundary is solid or dashed and whether the color is white or yellow.Step 2: In some embodiments, the module sorts all lanes and road boundaries from left to right. Sorting may be done using the slopes of the detected lanes, based on the heuristic that parallel lines point to a vanishing point from the perspective of a vehiclemounted camera (example shown in FIG. 4). In FIG. 4, the lines indicated with a 5, 6, or 7 are each detected road boundaries. The line indicated with a 2 is a solid yellow lane boundary. The line indicated with a 3 is a dashed white lane boundary. The line indicated with a 4 is a solid white lane boundary. In this image(FIG. 4), all of the detected road boundaries and lane boundaries point in the direction of a common vanishing point. Alternatively, or in addition, sorting may be based at least in part on a location that each detected lane line (and / or road boundary) would intersect the bottom of the camera frame, if extended away from the vanishing point, which may be referred to as the “x-intercept”. Sorting based on the x-intercept may also incorporate points at which a detected lane may intersect a side boundary of the camera frame. Sorting by the x-intercept may provide more reliable sorting in situations where not all detected lane boundaries are parallel, as may be expected when a different detected lane boundaries mark the edge of a merging entrance lane or an exit lane. In such cases, lane or road boundaries that are not parallel to others on the road will point to a different vanishing point. Similarly, a left-lane exit lane may point away from a vanishing point associated with other detectable lanes in the same traffic scene. Next, the width of every detected lane (defined by detected lane boundaries) is calculated by subtracting the x- intercept of the far lane boundary (relative to the ego vehicle) from the near lane boundary. Where a standard ego lane width as viewed from the camera is known (i.e. calibrated), the calculated width of each lane in the image may be normalized by the width of the lane in which the vehicle is travelling. This may either be the lane width as currently detected (the right ego-lane boundary minus the left ego-lane boundary), or the lane width as calibrated over time, which may be the median value of such calculations over several minutes (e.g. 10 minutes). The lane width calculations may be used to filter out spurious lane boundary detections that correspond to overlapping, duplicate, or noisy lane boundaries detected by the ML model, or to identify road locations where a lane boundary detection may have been missed.Step 3 : The module selects the lanes within the nearest left and right road boundaries where the vehicle is located. The module selects all the lanes to the left of the vehicle’sleft lane boundary up to the first road boundary and all the lanes to the right of the vehicle’s right lane boundary up to the first road boundary.Step 4: The module may optionally account for potential entrance or exit lanes, such as on the freeway, based on lane boundary characteristics. If there are any dashed lanes in the road, the module may be configured (depending on local norms) to treat lanes having a dashed lane as a left lane boundary as an entrance or exit lane. Detected entrance and exit lanes are then removed (or otherwise ignored) for the purposes of detecting on which lane the vehicle is travelling of the applicable through-traffic lanes of the road. In general, applicable lanes may be considered to be through-travel lanes that are intended for traffic in a same direction of travel as the vehicle. Applicable lanes for the purpose of detecting lane violations would not include entrance lanes, exit lanes, road shoulders. Applicable lanes would likewise exclude specialized lanes such as carpool lanes. Detection of a large vehicle in a carpool lane may be considered a lane violation as soon as the truck enters the carpool lane. And, further, on the same road, the truck may be considered to have triggered a lane violation if the truck is in the leftmost lane before the carpool lane. Applicable lanes would also exclude specialized lanes, such as bike lanes, that are not intended for regular vehicles. An example of this type of processing is shown in FIG. 5. In FIG. 5 the lines indicated with 8 or 9 are each detected road boundaries. There are no detected solid yellow lane boundaries. The line indicated with a 3 is a dashed white lane boundary. The lines indicated with a 2, 4, or 7 are solid white lane boundaries. In FIG. 5, the right ego lane boundary, indicated with a 3, is characterized by the edge device processing as having a “dashed” attribute. A lane boundary further to the right, indicated with a 4, demarcating the far side of the vehicle’s right adjacent lane, is “solid.” A further lane boundary to the right, indicated with a 7, demarcating a second lane to the right of the vehicle, is also solid. In this situation, the vehicle’s second right adjacent lane (between the solid white lane boundaries marked with a 4 and a 7) is treated as an entrance or exit lane. In some embodiments, this lane is removed from the data structure that is used to determine the lane in which the vehicle is travelling, for the purpose of determining if there is a lane violation. In this situation, even though there are two detected lanes of travel to the right of the ego-vehicle, the ego-vehicle is still properly characterized as travelling on a two- lane road (in the left-most lane) for this purpose of determining if there is a lane violation.Step 5: The module may optionally compensate for cases where the model potentially missed detecting lanes on the left and / or right sides of road. Such a compensation may be based on an observation that highway roads in the relevant jurisdiction (e.g. the United States) are painted with lane lines such that the left-most lane line (lane boundary) is solid yellow, and the right-most lane is solid white. If either the left-most lane or the right-most lane (between the vehicle and a detected road boundary) is detected as “dashed” or “dotted” rather than “solid” then the road information module may, depending on local norms, add a lane to the left or right, respectively, for a determination of the number of lanes to the left or right of the vehicle. Such a compensation may be computed prior to Step 4, since such added lanes may be pruned as described in that optional step. Alternatively, or in addition, compensation for cases where the model potentially missed detecting lanes may be based on the calculated width of each lane.

[0024] In addition, or alternatively, the lane violation module may include a locationbased rules module that determines the location of the vehicle (e.g. using GPS) and compares the location of the vehicle with a map stored on (or accessible to) the edge device. In one embodiment, the map includes the area of country divided into polygons (with a polygon defined for each State of the United States for example, and may have additional polygons defined for cities, counties, provinces, etc. as may be relevant). The map is stored in the edge device (or accessible to the edge device, such as via a cellular network connection). For example, the map may be downloaded at the time of installation of the edge device in the vehicle, or prior, and / or may be updated or overwritten at a later time such as by an over-the-air (OTA) update. For example, the map can be updated over time upon receiving a newer version of the map from a cloud server. Further, the cloud server sends the state level traffic rules (or more generally, traffic rules corresponding to one or more of the defined polygons in the map) to the edge device upon initial set up of the edge device, upon establishing a cellular connection with cloud server for the first time, and / or as part of an OTA update. The edge device may then use the stored map and the received location information from a sensor to determine a state (e.g., California) or other jurisdiction in which the vehicle is travelling, based on, for example, a comparison of the location of the vehicle with the map.

[0025] In some embodiments, exception suppression may be accomplished with a query to a map database, such as a database of linked road segments with attributes.For example, when there is a detection that the vehicle is travelling in a left most-lane, a map may be queried for road links within a distance of the vehicle’s location, such as a two-mile radius. The returned road links may be connected to make a path for where the vehicle can go from the current location. Using lane attributes, road segment links ahead of the driver may be queried for an attribute that corresponds to a left-exit lane. For example, the target attribute may include “deceleration lane.” If such an attribute is found, then determination of a lane violation event may be suppressed and / or delayed to account for the possibility that the driver may be in the process of making an allowed exit via a left-exit lane.

[0026] Upon determination of the state (or city, county, province, or other relevant area), the location-based rules module retrieves the traffic rules associated with the state from the memory (or from a remote server, such as via a cellular network connection). The application of traffic rules to the vehicles may be also based on the vehicle class (the class of vehicle to which the vehicle belongs), in addition to the location of the vehicle. In one example scenario, the edge device may determine that there are at least three lanes on the road in which the vehicle is travelling. The edge device may determine that the vehicle is travelling on the left-most lane of the at least three lanes, which may be a lane violation for a vehicle of the vehicle’s class in the state in which the vehicle is travelling.

[0027] Whether there is a lane violation may further depend on whether an exception applies, and so the alert may only be triggered after the vehicle remains in the same lane for at least a threshold period of time, and in that time no valid exception is detected. Upon detection of the vehicle traveling in the left-most lane (or more generally, a prohibited or discouraged lane), in a jurisdiction where such lane positioning may result in a lane violation, the edge device may initialize a timer (or counter) and monitor the vehicle path. For example, the processor may detect any lane changes made after the start of the vehicle entering the prohibited lane (which may be the left-most applicable lane). The edge device updates the timer if there is no lane change detected but may terminate the timer if there is a lane change detected such that the vehicle moves one lane to the right. The edge device periodically compares the timer value with a threshold value. The timer value indicates the time duration for which the vehicle continued to violate lane restrictions imposed by the traffic rules. If the time lapse derived from the timer is greater than the threshold value, then the edge devicemay generate an alert to notify the driver of the lane violation. For example, the edge device uses its speaker to present an audio alert to the driver.

[0028] Different types of alerts may be generated based on threshold values. There can be more than one threshold value, so that a different alert or alert type may be generated for every threshold met. In some embodiments, the threshold values can be configured by a fleet manager and can be changed based on their preference. The generated alert may be a moderate alert if the timer value is greater than the first threshold but less than the second threshold. The generated alert may be a severe alert if the timer value is greater than the second threshold. The generated alert is output to the driver through one or more components present in the output module of the edge device. In addition, the edge device may be configured to report the generated alerts to the cloud server. For example, the device may be configured so that only severe alerts are reported to the cloud server. The report can include the alert type, time at which the alert was generated, and the duration for which the alert remained relevant.

[0029] In another embodiment, to determine if the condition for a lane violation is satisfied, the edge device determines two factors. The initial (first) condition is satisfied when the road information module outputs that “the road has 3 or more lanes and the number of lanes to the left of the vehicle is 0.” This condition may be computed when a left lane change occurs, and the event processing logic is initiated in response to each detected lane change. That is, upon detecting a left lane change, the device may be configured to determine if the first condition is satisfied.

[0030] The second condition is the temporal condition, which is used to determine if the first condition continues for a period of time that would be considered longer than a temporary lane change such as when passing / overtaking another vehicle, which may be permitted. Before the temporal condition is met, the event computations (timer, counter, or scanning for exception conditions) may be terminated. If so, both conditions will not have been met, and no lane violation event will have been determined to occur. To calculate ongoing event computation before the temporal condition is met, the edge device may consider analyzing outputs over multiple individual frames, which may help handle noisy detection outputs. For example, the processor may calculate a histogram of the last 5 seconds of (#lanes, #lanes to the left of the vehicle) and pick the top 2 most common pairs. The temporal condition may be considered valid if one of these two most likely pairs satisfies the condition of “the road has 3 or more lanes andthe number of lanes to the left of the ego vehicle is 0” and this remains true for a threshold period of time.

[0031] While monitoring the path of the vehicle after detection of the first lane violation condition, the edge device is configured to suppress the alert generation and / or reset the timer upon detection of any of the following example scenarios: a. detection of a traffic light crossing, b. detection of a left turn made by the vehicle, c. detection of the presence of certain road marks (e.g. a U-turn permitted road mark), d. detection of certain objects on the road, such as a construction cone on the road, in the visual data, captured subsequent to or around the time of the detection of the first condition of a lane violation.

[0032] FIG. 4 shows an example to illustrate lane boundary sorting, lane width cleanup, and selecting lanes within the road boundaries. On the left side of FIG. 4, the lines indicated with a 5, 6, or 7 represent the road boundaries. As noted above, the line indicated with a 2 is a solid yellow lane boundary, the line indicated with a 3 is a dashed white lane boundary, and the line indicated with a 4 is a solid white lane boundary. The yellow, white, and dashed / solid lane boundary attributes were inferred for each lane boundary by processing the image data with a neural network model that was trained to detect lines, localize them within an image, and to infer attributes including color (which may be binary - yellow / white), and line type (e.g. solid, dashed). The road boundaries, indicated with a 5, 6, or 7, were inferred by a separate neural network head, which may be preferable road boundaries are often just next to lane boundaries. The rectangle surrounding the word EGO indicates where the vehicle is located.

[0033] After sorting the lane boundaries and road boundaries from left to right, the resulting order is [5, 1, 6, 2, 3, 4, 7], The road information module removes the “lane” bounded by lane boundary 1 and road boundary 5 since the width is small. Alternatively, or in addition, this potential lane may be removed as part of a step to remove lanes that are outside of the nearest road boundaries. In this case, all lane boundaries to the left of road boundary 6 could be removed.

[0034] To get the lanes between the left side of the vehicle’s lane (indicated with a 3) and the first boundary to the left (indicated with a 6), and the right side of the vehicle’s lane (indicated with a 4) and the first boundary on the right (indicated with a 7), the road information module outputs the final applicable lanes shown on right side of FIG. 4. The pattern of lane boundaries illustrated in FIG. 4 indicates that the ego-vehicle istravelling on the right-most lane of a divided road having a shoulder on the right and two lanes for through-traffic in the vehicle’s direction of travel.

[0035] FIG. 5 shows an example to illustrate removing potential exit or entrance lanes. In FIG. 5, both the lane boundaries indicated with a 4 and a 7 are solid. This is an indication of a separated entrance lane, so the potential lane between boundaries (4) and (7) may be removed. Additionally, there is no road boundary detected on the right side. In such a case, the method (or configured module) will select all lanes associated with detected lane boundaries to the right side initially, followed by pruning, such as just described.

[0036] FIG. 6 shows an example to illustrate the compensation for missing lanes. In FIG. 6, the lane boundary indicated with a 4 is dashed. This indicates that the ML model potentially failed to detect a lane on the right side, because the lane boundaries that are nearest to the edge of a road tend to be solid lane lines. The road information module may compensate for the potentially missing lane so that the number of lanes will be 3 instead of 2.

[0037] FIG. 7 depicts four example scenarios in which an in-cab alert may be generated or suppressed when the vehicle is in the left-most lane. In each example, a timer starts when the vehicle makes a left lane change, and the processor determines that all other necessary conditions are met (such as jurisdiction and class of vehicle). In these examples, there are three or more lanes detected in the same direction of travel as the vehicle, the vehicle has made a lane change into the left-most lane, the vehicle is a class 8 truck, and the vehicle is driving in a jurisdiction (e.g. state) having a rule that prohibits class 8 vehicles for driving in the left lane unless an exception applies.

[0038] In the top left scenario of FIG. 7, an in-cab alert is generated 50 seconds after the vehicle first enters the left-most lane. In this example, there may be two thresholds. After 50 seconds, the driver is presented with an in-cab alert (which may be an audio sound, a visual display, and the like). A second threshold may be configured, such as an additional 30 seconds, so that a remote alert (visible to a fleet safety manager, for example) may be triggered if the driver remains in the same lane for 30 or more additional seconds, i.e. a total of 80 seconds.

[0039] In the top right scenario, the in-cab alert is generated 50 seconds after the vehicle first enters the left-most lane, but the timer (or counter) is terminated soon afterwards, after the driver of the vehicle makes a right lane change. In this example, the driver may have avoided a remote report of the lane violation. In-cab and remotealerts may both be components of an incentive program to encourage drivers to comply with all relevant rules and regulations, when safe and appropriate to do so. A 50 second threshold that may be configured for a lane violation module to generate an alert that is presented directly to the driver in the vehicle cabin (in-cab). Other time durations may be used. In this example scenario, a second time duration, such as 70 seconds (total), may have been configured as the threshold above which to generate a remote report of the lane violation, which could be considered a way of encouraging a driver to remedy the lane violation quickly.

[0040] In the bottom left scenario, an in-cab alert is generated 50 seconds after the vehicle first enters the left-most lane, but the timer (or counter) is terminated soon afterwards because the number of lanes on the road has decreased to two. In the depicted scenario, the vehicle is in a state in which a lane violation is defined as driving in the left-most lane when there are three or more lanes of traffic in the same direction, but there is no lane violation when there are only two lanes.

[0041] In the bottom right scenario, the vehicle turned left soon after entering the leftmost lane. Because the vehicle was not in the left-most lane for the configured time interval, no lane violation alert was generated, either to the driver in-cab, or as a remote alert.

[0042] The schematic in the bottom right depicts additional factors that may cause a suppression of a lane violation alert / termination of a corresponding counter, in appropriate scenarios. These scenarios include a detection of a road marking, such as the lettering “BUS” when the device in installed in a vehicle that is a bus (vehicle class = “bus”). Another scenario is a detection of a traffic light. Because the lane violation rules only apply on certain roads, such as class 1 highways, and not residential roads, the presence of a traffic light may be treated as evidence that the vehicle is not travelling on a highway. A third scenario is a detection of a construction cone or other object(s) indicative of a construction zone, for which lane violation rules may be altered or processing of lane violation steps may be paused and / or ignored.

[0043] FIG. 8 depicts two example scenarios in which a lane violation alert may be suppressed based on various detected objects in a road scene. In the scene depicted on the top, the vehicle is travelling in a lane that is bounded on the left by a solid lane. While being bounded on the left by a solid lane may indicate that the vehicle is in the left-most lane, the left lane boundary may be typically colored white rather than yellow in the local area. In this example, the left adjacent lane is a U-turn lane. There is also aroad marking, indicating that a U-turn is permissible. Even though the road marking does not apply to the lane in which the vehicle is travelling, the presence of the road marking indicates that the vehicle is not travelling on a class 1 highway. In addition, several traffic lights are detected, including traffic lights that appear to govern traffic in the lane in which the vehicle is travelling. The presence of such traffic lights is additional evidence that the vehicle is not travelling on a class 1 highway, and so the lane violation rules or regulations should not be applied. In certain embodiments, the lane violation processing may be suppressed or ignored in such cases.

[0044] In the scene depicted on the bottom of FIG. 8, the vehicle is travelling in a lane that is bounded on the left by a solid lane, and the solid lane boundary is yellow. In this scene, the vehicle is travelling in the left-most lane. However, the presence of construction cones indicates that the vehicle is subject to special traffic control rules in place for the construction work. In this case, the vehicle is only allowed to travel in the left-most lane. According to alert suppression logic, no alerts will be generated (or if an alert is generated, it will be suppressed), when the vehicle is in a construction zone such as this one. In this case, the detection of a construction cone could reset a timer that monitors how long the driver has been in the left-most lane.

[0045] The edge device may include one or more machine learning models that are trained to implement one or more methods described above.

[0046] FIG. 9 depicts a table of vehicle classes for which Lane Violation rules and regulations are applicable. For most states within the United States, Lane violation rules apply to vehicles of class 6 and higher.

[0047] In countries or jurisdictions for which vehicles travel on the left side of undivided highways and roads, rather than the right side of the road (or for which travelling on the right side of the road is considered the “wrong” side of the road), a lane violation alert may be generated when certain vehicles travel in the right-most lane and no exception applies. The examples disclosed herein apply mutatis mutandis for all lane violation logic. For example, rather than initiating certain method steps in response to a left lane change, such method steps may be initiated in response to a right lane change, and the like.

[0048] As used herein, the term “determining” encompasses a wide variety of actions. For example, “determining” may include calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, a database or another data structure), ascertaining and the like. Additionally, “determining” may include receiving(e.g., receiving information), accessing (e.g., accessing data in a memory) and the like. Furthermore, “determining” may include resolving, selecting, choosing, establishing and the like.

[0049] As used herein, a phrase referring to “at least one of’ a list of items refers to any combination of those items, including single members. As an example, “at least one of: a, b, or c” is intended to cover: a, b, c, a-b, a-c, b-c, and a-b-c.

[0050] The methods disclosed herein comprise one or more steps or actions for achieving the described method. The method steps and / or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims.

[0051] The processing system may be configured as a general-purpose processing system with one or more microprocessors providing the processor functionality and external memory providing at least a portion of the machine-readable media, all linked together with other supporting circuitry through an external bus architecture. Alternatively, the processing system may comprise one or more specialized processors for implementing the neural networks, for example, as well as for other processing systems described herein.

[0052] Thus, certain aspects may comprise a computer program product for performing the operations presented herein. For example, such a computer program product may comprise a computer-readable medium having instructions stored (and / or encoded) thereon, the instructions being executable by one or more processors to perform the operations described herein. For certain aspects, the computer program product may include packaging material.

[0053] Further, it should be appreciated that modules and / or other appropriate means for performing the methods and techniques described herein can be downloaded and / or otherwise obtained by a user terminal and / or base station as applicable. For example, such a device can be coupled to a server to facilitate the transfer of means for performing the methods described herein. Alternatively, various methods described herein can be provided via storage means (e.g., RAM, ROM, a physical storage medium such as a compact disc (CD) or floppy disk, etc.), such that a user terminal and / or base station can obtain the various methods upon coupling or providing the storage means to the device. Moreover, any other suitable technique for providing the methods and techniques described herein to a device can be utilized.

[0054] It is to be understood that the claims are not limited to the precise configuration and components illustrated above. Various modifications, changes and variations may be made in the arrangement, operation and details of the methods and apparatus described above without departing from the scope of the claims.

Claims

Claims1. A computer-implemented method, comprising: receiving, by at least one processor of a computing device, visual data of a road on which a vehicle is travelling, captured by at least one camera of the computing device, the computing device being installed in the vehicle; determining, by the at least one processor and based on the visual data, a number of applicable lanes on a road on which the vehicle is traveling; identifying, by the at least one processor, a lane in which the vehicle is travelling from the determined number of applicable lanes; and generating, by the at least one processor, an alert based at least on the vehicle’s identified lane of travel on the road.

2. The method of claim 1, wherein generating the alert comprises: initiating a timer in response to determining that the vehicle is traveling on a left-hand lane of the determined number of applicable lanes; and generating the alert if a value of the timer is greater than a first threshold value.

3. The method of claim 2, wherein the alert is a moderate alert if the timer value is greater than a first threshold value, wherein the alert is a severe alert if the timer value is greater than the second threshold value; and wherein the second threshold value is greater than the first threshold value.

4. The method of claim 2, further comprising: terminating the timer upon determination of a left turn by the vehicle.

5. The method of claim 2, further comprising: querying a database of road segments and road segment attributes in a vicinity of the vehicle; and terminating the timer upon determination of a road segment attribute corresponding to an availability of a left exit in the vehicle’s direction of travel.

6. The method of claim 1, wherein generating the alert is further based on a class of the vehicle to which the vehicle belongs.

7. The method of claim 1, wherein generating the alert is further based on the location of the vehicle.

8. The method of claim 7, further comprising: determining, by the at least one processor, a location of the vehicle based on location data received from at least one sensor of the computing device; and receiving a lane violation rule for the location.

9. The method of claim 1, further comprising: determining a type of road for the road on which the vehicle is travelling, wherein generating an alert is further based on the determined type of road.

10. The method of claim 1, wherein determining the number of applicable lanes on the road on which the vehicle is travelling comprises: processing the visual data with a neural network, wherein a first head of the neural network is configured to output locations of lane boundaries, and wherein a second head of the neural network is configured to output locations of road boundaries.

11. The method of claim 1 , wherein determining the number of applicable lanes on the road on which the vehicle is travelling comprises: identifying a first left road boundary and a first right road boundary; and selecting detected lane boundaries between the first left road boundary and the first right road boundary.

12. The method of claim 1, further comprising: processing the visual data with an object detection neural network, wherein the object detection neural network is configured to detect one or more of: traffic lights, construction cones, or road markings; and wherein the alert is suppressed if one or more of: a traffic light, a construction cone, or a road marking indicative of a residential road is detected by the object detection neural network.

13. The method of claim 1, further comprising: detecting a lane change by the vehicle.

14. The method of claim 13, wherein the lane change is a left lane change, and wherein the number of applicable lanes on the road is determined in response to detecting the left lane change.

15. The method of claim 1, wherein determining the number of applicable lanes comprises: identifying a first road boundary in a first lateral direction from the vehicle; and excluding one or more detected lane boundaries, wherein the excluded one or more detected lane boundaries are farther from the vehicle than the first road boundary in the first lateral direction from the vehicle.

16. The method of claim 1, wherein determining the number of applicable lanes comprises: identifying a first road boundary in a first lateral direction from the vehicle; and wherein the method further comprises: identifying a shoulder of a road based on a calculated distance between the identified first road boundary and the closest identified lane boundary in the first lateral direction that is closer to the vehicle.

17. The method of claim 1, wherein determining the number of applicable lanes comprises: identifying a nearest detected lane boundary in a first lateral direction from the vehicle, for which an attribute of the detected lane boundary is solid; excluding one or more detected lane boundaries, wherein the excluded one or more detected lane boundaries are farther from the vehicle in the first lateral direction from the vehicle than the identified nearest detected lane boundary.18 The method of claim 17, further comprising: identifying an entrance lane of a road based on a calculated distance between the identified nearest detected lane boundary for which an attribute of the detected lane boundary is solid, and the closest lane boundary of the excluded one or more detected lane boundaries.

19. The method of claim 1, wherein determining the number of applicable lanes comprises: identifying a farthest detected lane boundary in a first lateral direction from the vehicle;in response to determining that an attribute of the identified farthest detected lane boundary is not solid, adding one lane to the number of applicable lanes, wherein the added one lane is farther in the first lateral direction from the vehicle, and wherein the added one lane is bounded on the side nearer to the vehicle by the identified farthest detected lane boundary.

20. The method of claim 1, wherein the number of applicable lanes is the number of lanes of through-travel lanes on the road, wherein each of the through-travel lanes are intended for traffic in a same direction of travel as the vehicle, and are not an entrance lane, an exit lane, or a road shoulder.

21. A computer program product comprising a non-transitory computer-readable medium having instructions stored thereon, the instructions being executable by one or more processors configured to carry out the method of any preceding claim.

22. A device comprising a processor configured to perform the steps of the method of any of claims 1 to 20.

Images