Technologies for handling incidents on road segments based on radar data and camera data

Abstract

This disclosure enables a system including a plurality of radars, a Pan-Tilt-Zoom (PTZ) camera, and a computing unit in communication with the plurality of radars and the PTZ camera. The PTZ camera and the computing unit may be embodied as separate and distinct physical devices, or the PTZ camera and the computing unit may be embodied as a single physical device, with functionality correspondingly adapted accordingly. The plurality of radars have a many-to-one correspondence to the PTZ camera. The computing unit commands the PTZ camera to move to a position capable of imaging an incident on a road segment based on the computing unit detecting the incident as observed by one radar of the plurality of radars.

Description

Docket: 15257367-000100 Patent SpecificationTITLE TECHNOLOGIES FOR HANDLING INCIDENTS ON ROAD SEGMENTS BASED ON RADAR DATA AND CAMERA DATACROSS-REFERENCE TO RELATED PATENT APPLICATION

[0001] This patent application claims a benefit of US Provisional Patent Application 63 / 778,324 filed 26 March 2025, which is incorporated by reference herein for all purposes.TECHNICAL FIELD

[0002] This disclosure relates to technologies for handling incidents on road segments based on radar data and camera data.BACKGROUND

[0003] Conventionally, an Intelligent Transportation System (ITS) may have a control center having a computing terminal (e.g., a desktop computer) operated by a dispatcher (e.g., a person). As such, the ITS may receive a plurality of video feeds from a plurality of cameras positioned to image a plurality of road segments (e.g., a plurality of control zones) and present the plurality of video feeds on the computing terminal for view by the dispatcher, as needed. Sometimes, the ITS may process the plurality of video feeds to identify (e.g., by a computer vision algorithm) an incident (e.g., a car accident) on one of the road segments and present a notice on the computing terminal to prompt the dispatcher for attention. If appropriate, then the dispatcher may request (e.g., by a telephone) an emergency responder (e.g., a police officer) to handle the incident.

[0004] To enhance monitoring of the plurality of road segments, the ITS may receive a plurality of tracking feeds from a plurality of radars positioned to scan the plurality of road segments. As such, the ITS may process the plurality of tracking feeds to supplement the plurality of video feeds to augment or supplement identifying the incident.

[0005] The plurality of cameras and the plurality of radars may be embodied via a plurality of monitoring units (e.g., a plurality of housings) where each monitoring unit has a single camera and a single radar. This approach is technologically disadvantageousDocket: 15257367-000100 Patent Specificationdue to an excessive computational load, a complicated data conflict resolution algorithm, and an excessive financial cost. Although distributing the plurality of cameras and the plurality of radars from each other may be possible, this approach is also technologically disadvantageous due to an even greater excessive computational load, an even more complicated data conflict resolution algorithm, and an even more excessive financial cost.SUMMARY

[0006] Broadly, this disclosure solves these technological disadvantages by having a system including a plurality of radars (e.g., a plurality of Doppler radars), a Pan-Tilt-Zoom (PTZ) camera, and a computing unit (e.g., an edge computer, an embedded computer, an industrial computer). The PTZ camera and the computing unit may be embodied as separate and distinct physical devices, or the PTZ camera and the computing unit may be embodied as a single physical device (e.g., one housing), with functionality correspondingly adapted accordingly. The plurality of radars may be attached (e.g., bracketed, fastened, adhered) to a plurality of outdoor structures (e.g., a plurality of utility poles, a plurality of transmission towers, a plurality of buildings) sequentially positioned (e.g., one after another) lateral to a lane of a road segment such that (a) each radar of the plurality of radars has a radar Field of View (FOV), (b) the radar FOVs collectively monitor the road segment in its entirety without overlapping each other, and (c) the plurality of radars have a one-to-one correspondence to the plurality of outdoor structures relative to the road segment. The PTZ camera may be attached (e.g., bracketed, fastened, adhered) to one outdoor structure of the plurality of outdoor structures such that (a) the PTZ camera has a camera FOV, (b) the camera FOV is capable of individually monitoring any section of the road segment, (c) the PTZ camera is positioned higher in elevation than the plurality of radars, and (d) the plurality of radars have a many-to-one correspondence (e.g., 4 to 1) to the PTZ camera relative to the road segment. The computing unit may be attached (e.g., bracketed, fastened, adhered) to one outdoor structure of the plurality of outdoor structures. The computing unit may be programmed to (a) command the PTZ camera to move (e.g., on an X, Y, or Z axis) the camera FOV from a first position or orientation not capable of imaging an incident on the road segment observed by one radar of the plurality of radars to a second position or orientation capableDocket: 15257367-000100 Patent Specificationof imaging the incident responsive to the computing unit detecting the incident based on the one radar of the plurality of radars observing the incident, (b) command the PTZ camera to generate an imagery (e.g., a photo or a video) depicting the incident as the camera FOV is positioned in the second position or orientation, (c) receive the imagery from the PTZ camera positioned in the second position or orientation, (d) perform a validation (e.g., run a set of computer vision heuristics on the imagery) of the incident (e.g., to minimize a false positive) while the PTZ camera positioned in the second position or orientation, (e) notify a control center of an ITS of the incident (or perform another action) responsive to the validation being successful while the PTZ camera is positioned in the second position or orientation, and (f) ignore the incident (or perform another action) responsive to the validation being unsuccessful.DESCRIPTION OF DRAWINGS

[0007] FIG. 1 shows a schematic diagram of an embodiment of a physical arrangement according to this disclosure.

[0008] FIG. 2 shows a schematic diagram of an embodiment of a communication arrangement according to this disclosure.

[0009] FIG. 3 shows a schematic diagram of an embodiment of a logical arrangement according to this disclosure.

[0010] FIG. 4 shows a schematic diagram of an embodiment of a subsystem arrangement according to this disclosure.

[0011] FIG. 5 shows a schematic diagram of an embodiment of a plurality of incidents that can be identified according to this disclosure.

[0012] FIG. 6 shows a schematic diagram of an embodiment of a computing terminal operated by a dispatcher where the computing terminal runs an application program presenting a Graphical User Interface (GUI) that received a communication from a computing unit controlling a plurality of radars and a PTZ camera according to this disclosure.

[0013] FIG. 7 shows an embodiment of a plurality of screenshots showing a GUI depicting a plurality of viewing panes presenting an incident involving a pedestrian on a road segment according to this disclosure.Docket: 15257367-000100 Patent Specification

[0014] FIG. 8 shows an embodiment of a plurality of screenshots showing a GUI depicting a plurality of viewing panes presenting incident involving a car accident on a road segment according to this disclosure.

[0015] FIG. 9 shows an embodiment of a plurality of screenshots showing a GUI depicting a plurality of viewing panes presenting an incident involving a vehicle moving in a wrong direction on a road segment according to this disclosure.

[0016] FIG. 10 shows an embodiment of a plurality of screenshots showing a GUI depicting a plurality of viewing panes presenting an incident involving a vehicle stopped on a road segment according to this disclosure.

[0017] FIG. 11 shows an embodiment of a plurality of screenshots showing a GUI depicting a plurality of viewing panes presenting an incident involving a slowed movement of a vehicle on a road segment according to this disclosure.

[0018] FIG. 12 shows an embodiment of method comprising a plurality of steps of detecting an incident according to this disclosure.

[0019] FIG. 13 shows an embodiment of a plurality of screens of a GUI notified of an incident after validation of the incident according to this disclosure.

[0020] FIGS. 14-15 shows an embodiment of a plurality of technical specifications for a radar according to this disclosure.

[0021] FIGS. 16-17 shows an embodiment of a plurality of technical specifications for a camera according to this disclosure.

[0022] FIG. 18 shows an embodiment of an incident management platform according to this disclosure.

[0023] FIG. 19 shows an embodiment of a plurality of screens a GUI for management of incidents as a function of time according to this disclosure.

[0024] FIG. 20 shows an embodiment of a screen of a GUI for viewing performance statistics according to this disclosure.

[0025] FIGS. 21-29 shows an embodiment of a plurality of screens of a GUI for configuring operation of a plurality of radars and a PTZ camera according to this disclosure.

[0026] FIGS. 31-32 shows an embodiment of a mathematical calculation enabling operation of a plurality of radars and a PTZ camera according to this disclosure.Docket: 15257367-000100 Patent Specification

[0027] FIG. 33 shows an embodiment of a video feed where one vehicle is kept and other vehicle are hidden according to this disclosure.

[0028] FIG. 34 shows an embodiment of a video fee where a vehicle is detected to be moving in an anomalous manner according to this disclosure.DETAILED DESCRIPTION

[0029] This disclosure is now described more fully with reference to drawings, in which some embodiments of this disclosure are shown. This disclosure may, however, be embodied in many different forms and should not be construed as necessarily being limited to the embodiments disclosed herein. Rather, these embodiments are provided so that this disclosure is thorough and complete, and fully conveys various concepts of this disclosure to skilled artisans.

[0030] Various terminology used herein can imply direct or indirect, full or partial, temporary or permanent, action or inaction. For example, when an element is referred to as being "on," "connected" or "coupled" to another element, then the element can be directly on, connected or coupled to the other element and / or intervening elements can be present, including indirect and / or direct variants. In contrast, when an element is referred to as being "directly connected" or "directly coupled" to another element, there are no intervening elements present.

[0031] As used herein, a term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless specified otherwise, or clear from context, "X employs A or B" is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then "X employs A or B" is satisfied under any of the foregoing instances.

[0032] Although the terms first, second, etc. can be used herein to describe various elements, components, regions, layers and / or sections, these elements, components, regions, layers and / or sections should not necessarily be limited by such terms. These terms are used to distinguish one element, component, region, layer or section from another element, component, region, layer or section. Thus, a first element, component, region, layer, or section discussed below could be termed a second element, component, region, layer, or section without departing from the teachings of the present disclosure.Docket: 15257367-000100 Patent Specification

[0033] Furthermore, relative terms such as "below," "lower," "above," and "upper" can be used herein to describe one element's relationship to another element as illustrated in the accompanying drawings. Such relative terms are intended to encompass different orientations of illustrated technologies in addition to the orientation depicted in the accompanying drawings. For example, if a device in the accompanying drawings were turned over, then the elements described as being on the "lower" side of other elements would then be oriented on "upper" sides of the other elements. Similarly, if the device in one of the figures were turned over, elements described as "below" or "beneath" other elements would then be oriented "above" the other elements. Therefore, the example terms "below" and "lower" can encompass both an orientation of above and below.

[0034] The terminology used herein is for describing particular example embodiments and is not intended to be necessarily limiting of the present disclosure. As used herein, the singular forms "a," "an" and "the" are intended to include the plural forms (e.g., two, three, four, five, six, seven, eight, nine, ten, tens) as well, unless the context clearly indicates otherwise. Also, as used herein, the term "a" and / or "an" shall mean "one or more," even though the phrase "one or more" is also used herein. The terms "comprises," "includes" and / or "comprising," "including" when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence and / or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Furthermore, when the present disclosure states herein that something is "based on" something else, then such statement refers to a basis which may be based on one or more other things as well. In other words, unless expressly indicated otherwise, as used herein "based on" inclusively means "based at least in part on" or "based at least partially on."

[0035] FIG. 1 shows a schematic diagram of an embodiment of a physical arrangement according to this disclosure. In particular, there is a physical arrangement 100 containing a road segment 102, a divider 104, a plurality of lanes 106, a plurality of lateral areas 108, a vehicle 110, a plurality of outdoor structures 112, a plurality of radars 114, a plurality of radar FOVs 116, a PTZ camera 118, a PTZ camera FOV 120, and a communication block 122. As further described below, the PTZ camera 118 and the communication block 122 may be embodied as separate and distinct physical devices, orDocket: 15257367-000100 Patent Specificationthe PTZ camera 118 and the communication block 122 may be embodied as a single physical device (e.g., one housing), with functionality correspondingly adapted accordingly.

[0036] The road segment 102 extends between the plurality of lateral areas 108. The divider 104 divides the road segment 102 into the plurality of lanes 106. One (distal) of the lateral areas 108 has the plurality of outdoor structures 112 disposed thereon, on one side (distal) of the road segment 102, and the other (proximal) of the lateral areas 108 has the vehicle 110 positioned thereon, whether stationary (e.g. , disabled) or moving, on the other side (proximal) of the road segment 102.

[0037] The road segment 102 may be paved or unpaved (e.g., dirt, concrete, cobblestone), flat or non-flat (e.g., sloped), longitudinally rectilinear or longitudinally non-rectilinear (e.g., curved). The road segment 102 may have a length of about 1 mile or less (e.g., 0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2, or 0.1 mile), although other lengths are possible, depending on how the plurality of radars 114 and the PTZ camera 118 are constituted, configured, and positioned, as disclosed herein. The road segment 102 may be a free road, a toll road, a public road, a private road, or other suitable type of road, whether on land, including within a tunnel, or a bridge over a body of water (e.g., a river, a canal, a lake). The road segment 102 may be in a rural area or an urban area. The road segment 102 may be a highway, a parkway, a street, an avenue, a boulevard, a drive, a dirt path, a gravel path, a cul-de-sac, or any other suitable path on which the vehicle 110 can move, whether forwards, backwards, or laterally. For example, the road segment 102 may allow the vehicle 110 to move within one lane 106 of the plurality of lanes 106 among a stream of vehicles (similar to the vehicle 110) moving on the road segment 102 within the plurality of lanes 106, which may be at city speeds of about 30 miles per hour or less or highway speeds of about 65 miles per hour or less, although lower or higher speeds may be possible (e.g., about 3 miles per hour, about 25 miles per hour, 80 miles per hour).

[0038] The divider 104 is embodied as a dashed line indicating that each lane in the plurality of lanes 106 enables a traffic flow of vehicles 110 to move in single or opposite directions. However, this configuration is not required and the divider 104 may be a single solid line or a pair of solid lines indicating that each lane of the plurality of lanes 106 enables the traffic flow of vehicles 110 in single or opposite directions. Although FIG. 1Docket: 15257367-000100 Patent Specificationshows the plurality of lanes 106 embodied as two lanes, this configuration is not required and there may be a single lane or multiple lanes (e.g., three, four, five or more) on the road segment 102, whether per the road segment 102 or per direction on the road segment 102, whether for the traffic flow in single or opposite directions. Alternatively, the divider 104 may be a physical barrier (e.g., a wall) dividing the road segment 102 into the plurality of lanes 106, whether for the traffic flow in single or opposite directions. Note that the divider 104 may be absent, whether as a line or as a physical barrier.

[0039] The road segment 102 is interposed between the plurality of lateral areas 108. The plurality of lateral areas 108 may be embodied as shoulders, raised area, sidewalks, grass areas, or other suitable lateral areas. The plurality of lateral areas 108 may each be paved or unpaved (e.g., dirt, concrete, cobblestone) along the road segment 102. The plurality of lateral areas 108 may be any suitable lateral areas on which the vehicle 110 can stand, park, or move, whether forwards, backwards, or laterally, without endangering other passing vehicles or nearby pedestrians.

[0040] The vehicle 110 may be any land vehicle, whether manned or unmanned, whether powered by a fossil fuel (e.g., gasoline, diesel), an electric battery, a renewable energy source (e.g., hydrogen, solar panel), or a human action (e.g., a pair of pedals), whether governmental or private. For example, the vehicle 110 may be a bicycle, a motorcycle, a skateboard, a car, a minivan, a van, a jeep, a Sports Utility Vehicle (SUV), a truck, a bus, a tank, an armored vehicle, or any other suitable land vehicle able to travel, move, or drive on the road segment 102 in any lane in the plurality of lanes 106 or in any lateral area of the plurality of lateral areas. As shown, the vehicle 110 is positioned, whether stationary (e.g., disabled or damaged) or moving (e.g., accelerating or decelerating), on one lateral area 108 (proximal) of the plurality of lateral areas 108, although the vehicle 110 can be positioned in any lane in the plurality of lanes 106, whether moving (e.g., in a normal manner) or stationary (e.g., disabled or damaged), or on the other lateral area 108 (distal) of the plurality of lateral areas 108, whether moving (e.g., in a normal manner) or stationary (e.g., disabled or damaged).

[0041] The plurality of outdoor structures 112 may be embodied as utility poles, transmission towers, external building walls, flag poles, cell towers, lampposts, water towers, columns, pillars, clocktowers, street signs, outdoor signage, bridge towers, trees,Docket: 15257367-000100 Patent Specificationbus stops, train stops, or other suitable outdoor structures that are stationary, whether of one type or multiple types. For example, the plurality of outdoor structures 112 may be of one type (e.g., all are utility poles) or of multiple types (e.g., some are utility poles and some are transmission towers).

[0042] The plurality of outdoor structures 112 is sequentially positioned (e.g. , one after another) in a rectilinear pattern (e.g., from an overhead view) along one side (distal) of the road segment 102 along one lateral area 108 (distal) of the plurality of lateral areas 108, whether internal or external to that respective lateral area 108. For example, one outdoor structure 112 of the plurality of outdoor structures 112 may be non-offset relative to two other outdoor structures 112 of the plurality of outdoor structures 112, when viewed from above, such that the rectilinear pattern is formed. However, this configuration is not required. For example, the plurality of outdoor structures 112 may be sequentially positioned (e.g., one after another) in a non-rectilinear pattern (e.g., arcuate, sinusoidal, zigzag, sawtooth from an overhead view) along one side (distal) of the road segment 102 along one lateral area 108 of the plurality of lateral areas 108, whether internal or external to that respective lateral area 108. For example, one outdoor structure 112 of the plurality of outdoor structures 112 may be offset relative to two other outdoor structures 112 of the plurality of outdoor structures 112, when viewed from above. Likewise, for example, the plurality of outdoor structures 112 may be sequentially or alternately positioned along both sides (proximal and distal) of the road segment 102 along both lateral areas 108 (proximal and distal) of the plurality of lateral areas 108, whether internal or external to those respective lateral areas 108, whether in a rectilinear pattern or a non-rectilinear pattern (e.g., arcuate, sinusoidal, zigzag, sawtooth), whether along one side of the road segment 102, both sides of the road segment 102, or collectively on both sides of the road segment (e.g., three outdoor structures 112 positioned on one side of the road segment 102 and two outdoor structures 112 positioned on the other side of the road segment 102 may collectively define a sinusoidal or W-pattern). For example, the plurality of outdoor structures 112 may be sequentially positioned lateral to the lane 106 of the road segment 102 on one side of the road segment 102 or the plurality of outdoor structures 112 may be sequentially positioned lateral to the lane 106 of the road segment 102 on both sides of the road segment 102.Docket: 15257367-000100 Patent Specification

[0043] The plurality of outdoor structures 112 may be substantially equidistantly spaced apart (e.g., about 0.25 miles or less) from each other (e.g., from a profile view), although this configuration is not required. For example, the plurality of outdoor structures 112 may be substantially non-equidistantly spaced apart (e.g., some may be spaced apart about 0.3 miles and some be spaced apart about 0.2 miles) from each other (e.g., from a profile view). This may be technologically advantageous due to potential uniform coverage (substantially equidistant) or customized coverage (not substantially equidistant).

[0044] The plurality of outdoor structures 112 avoids extending over the road segment 102. However, this configuration is not required. For example, at least one outdoor structure 112 of the outdoor structures 112 may extend over the road segment 102. Although FIG. 1 shows the vehicle 110 and the plurality of outdoor structures 112 positioned on opposite sides of the road segment 102, whether internal or external to the respective lateral areas 108, this configuration is not required. For example, the plurality of outdoor structures 112 and the vehicle 110 may be positioned on one side of the road segment 102, whether internal or external to the respective lateral area 108. For example, the vehicle 110 may be stationary or moving on one lateral area 108 of the plurality of lateral areas 108 between one outdoor structure 112 of the plurality of outdoor structures 112 and the road segment 102, along one side of the road segment 102.

[0045] The plurality of radars 114 may be embodied as a plurality of Time-Of-Flight (TOF) radars (e.g., radiofrequency), a plurality of Doppler radars (e.g., radiofrequency), or a mixture of TOF radars (e.g., radiofrequency) and Doppler radars (e.g., radiofrequency). The plurality of TOF radars may be pulse radars may be pulse TOF radars, Frequency Modulated Continuous Wave (FMCW) TOF radars, or other suitable TOF radars. The plurality of Doppler radars may be Continuous Wave (CW) Doppler radars, FMCW Doppler radars, pulse Doppler radars, coherent pulse Doppler radars, or other suitable Doppler radars. The Doppler radars may operate in X-band, K-band, Ka-Band, Ku-band, E-band, or other suitable frequency bands. For example, each radar 114 of the plurality of radars 114 may be a Doppler radar. For example, each radar 114 of the plurality of radars 114 may operate in a Ku-band inclusively between about 12 GHz and about 18 GHz, a K-band inclusively between about 18 GHz and about 27 GHz, or a Ka-Docket: 15257367-000100 Patent Specificationband inclusively between about 26.5 GHz and about 40 GHz. Note that the plurality of radars 11 may run stabilization algorithms to compensate for potential inaccuracies (e.g. , jitter) resulting from or introduced by vibrations caused at least from the stream of vehicles or the communication block 122 may run stabilization algorithms on radar data to compensate for potential inaccuracies (e.g., jitter) resulting from or introduced by vibrations caused at least from the stream of vehicles.

[0046] The TOF radars may be technologically advantageous for their high-detail and high-precision relative to the Doppler radars, but may involve data filtering operations in view of such abundance of data. The Doppler radars may be technologically advantageous for their speed or velocity measurements, detection of moving objects against stationary backgrounds or objects, clutter rejection, distinguishing between different types of moving targets based on motion characteristics, long-range detection, all-weather performance, wide area of coverage, vehicle classification and traffic analysis relative to the TOF radars, especially since the plurality of Doppler radars may maximize detection of moving objects (e.g., the vehicle 110, pedestrians, animals) and minimize detection of stationary objects (e.g., rocks, outdoor structures 112), which would minimize data filtering operations and improve computational efficiency relative to the TOF radars. Note that the plurality of radars 114 is powered by a mains electricity source (e.g., a power line).

[0047] As shown in FIG. 1, the plurality of radars 114 may be embodied in a plurality of housings (e.g., weatherproof) attached (e.g., bracketed, fastened, adhered) to the plurality of outdoor structures 112 sequentially positioned (e.g., one after another) lateral to the lane 106 of the plurality of lanes 106 of the road segment 102. For example, the housings may be attached to the plurality of outdoor structures 112 via a plurality of arms (e.g., weatherproof), which may be cantilevered from the plurality of outdoor structures 112, as shown in FIG. 1, although this configuration is not required and the plurality of arms may be omitted or the plurality of arms may not be cantilevered. For example, the plurality of outdoor structures 112 may have a plurality of external sidewalls to or through which the plurality of housings are attached, whether via the plurality of arms or not, to enable the radar FOVs 116 collectively monitor the road segment 102 in its entirety without overlapping each other, as shown in FIG. 1. However, note that this configurationDocket: 15257367-000100 Patent Specificationis not required. For example, the plurality of outdoor structures 112 may have a plurality of apex sides (e.g., top sides, roofs) to or through which the plurality of housings are attached, whether via the plurality of arms or not, to enable the radar FOVs 116 collectively monitor the road segment 102 in its entirety without overlapping each other, as shown in FIG. 1. Note that each housing of the plurality of housings can be elevated at about 100 feet or lower, as measured from a ground level (e.g., a flat area, an inclined area, a grass area, a sidewalk area, a pavement area) from which that respective outdoor structure 112 of the plurality of outdoor structures 112 extends.

[0048] Each radar 114 of the plurality of radars 114 has the radar FOV 116, which may be pyramidal (see FIG. 1) or conical in shape, for example. Correspondingly, the radar FOVs 116 are sequentially positioned and thereby collectively monitor the road segment 102 in its entirety, without overlapping each other. For example, the radar FOVs 116 are sequentially formed adjacent to each other, without overlapping each other, and thereby sequentially cover a plurality of sections of the road segment 102, as shown in FIG. 1, where the plurality of sections collectively define the road segment, as shown in FIG. 1. Additionally, the plurality of radars 114 have a one-to-one correspondence to the plurality of outdoor structures 112 relative to the road segment 102. Although the plurality of FOVs 116 are formed to be as close to each as practical, there may still be a plurality of coverage gaps (e.g., about 50 feet or less) between the plurality of FOVs 116.

[0049] The plurality of radars 114 is sequentially positioned (e.g. , one after another) in a rectilinear pattern (e.g., from an overhead view). For example, one radar 114 of the plurality of radars 114 may be non-offset relative to two other radars 114 of the radars 114, when viewed from above, such that the rectilinear pattern is formed. However, this configuration is not required. For example, the plurality of radars 114 may be sequentially positioned (e.g., one after another) in a non-rectilinear pattern (e.g., arcuate, sinusoidal, zigzag, sawtooth from an overhead view). For example, one radar 114 of the plurality of radars 114 may be offset relative to two other radars 114 of the plurality of radars 114, when viewed from above. Likewise, for example, the plurality of radars 114 may be sequentially or alternately positioned along both sides (proximal and distal) of the road segment 102 along both lateral areas 108 (proximal and distal) of the plurality of lateral areas 108, whether in a rectilinear pattern or a non-rectilinear pattern (e.g., arcuate,Docket: 15257367-000100 Patent Specificationsinusoidal, zigzag, sawtooth), whether along one side of the road segment 102, both sides of the road segment 102, or collectively on both sides of the road segment (e.g., three radars 114 positioned on one side of the road segment 102 and two radars 114 positioned on the other side of the road segment 102 may collectively define a sinusoidal or W-pattern). The plurality of radars 114 may be substantially equidistantly spaced apart (e.g., about 0.25 miles or less) from each other (e.g., from a profile view), although this configuration is not required and the plurality of radars 114 may be substantially non-equidistantly spaced apart (e.g., some may be spaced apart about 0.3 miles and some be spaced apart about 0.2 miles) from each other (e.g., from a profile view). Note that the plurality of radars 114 avoids extending over the road segment 102. However, this configuration is not require and at least one radar 114 of the radars 114 may extend over the road segment 102.

[0050] The PTZ camera 118 may be a Power over Ethernet (PoE) PTZ camera, an analog PTZ camera, an Internet Protocol (IP) PTZ camera, or other suitable PTZ camera, whether imaging in a visible spectrum or an Infrared (IR) spectrum. The PTZ camera 118 is powered by a mains electricity source (e.g., a power line). The PTZ camera 118 may be movable (e.g., pan or tilt) relative to its outdoor structure 114 on an X, Y, or Z axis such that the camera FOV 120 may be movable along the X, Y, or Z axis relative to its outdoor structure 114.

[0051] As shown in FIG. 1, the PTZ camera 118 may be embodied in a housing (e.g., weatherproof) attached (e.g., bracketed, fastened, adhered) to one outdoor structure 114 of the plurality of outdoor structures 112 sequentially positioned (e.g., one after another) lateral to the lane 106 of the plurality of lanes 106 of the road segment 102. For example, the housing may be attached to the one outdoor structure 114 of the plurality of outdoor structures 112 via an arm (e.g., weatherproof), which may be cantilevered from the one outdoor structure 114 of the plurality of outdoor structures 112, as shown in FIG. 1, although this configuration is not required and the arm may be omitted or not be cantilevered. For example, the one outdoor structure 114 of the plurality of outdoor structures 112 may have an external sidewall to or through which the housing may be attached, whether via the arm or not, to enable the PTZ camera 118 to operate for the camera FOV 120 to be able to individually monitor the road segment 102 in its entirety,Docket: 15257367-000100 Patent Specification(any section of the road segment 102), as shown in FIG. 1. For example, the PTZ camera 118 can move (e.g., pan or tilt) for the camera FOV 120 to be able to image any section of the road segment 102. However, note that this configuration is not required. For example, the one outdoor structure 114 of the plurality of outdoor structures 112 may have an apex side (e.g., a top side, a roof) to or through which the housing may be attached, whether via the arm or not, to enable the camera FOV 120 to be able to individually monitor the road segment 102 in its entirety (any section of the road segment), as shown in FIG. 1. For example, the PTZ camera 118 can move (e.g., pan or tilt) for the camera FOV 120 to be able to image any section of the road segment 102. For example, the PTZ camera 118 may move (e.g., pan or tilt) to enable the camera FOV 120 to be able to individually monitor the road segment 102 in its entirety (any section of the road segment 102), as shown in FIG. 1, without any other cameras (e.g., government or company) monitoring the road segment 102 in its entirety.

[0052] As shown in FIG. 1, the PTZ camera 118 avoids extending over the road segment 102. However, this configuration is not required. For example, the PTZ camera 118 may extend over the road segment 102.

[0053] As shown in FIG. 1 , the PTZ camera 118 and one radar 114 of the plurality of radars 114 are attached to one common outdoor structure 112 of the plurality of outdoor structure 112. However, this configuration is not required. For example, the PTZ camera 118 may not share any outdoor structure 112 of the plurality of outdoor structure 112 with any radar 114 of the plurality of radars 114.

[0054] As shown in FIG. 1, the PTZ camera 118 has the camera FOV 120, which may be pyramidal (see FIG. 1) or conical in shape. The PTZ camera 118 operates (e.g., moves, pans, tilts) to enable the camera FOV 120 to be able to individually monitor any section of the road segment 102, such that the camera FOV 120 can individually monitor the road segment 102 in its entirety, as needed. For example, the PTZ camera 118 can move (e.g., pan or tilt) for the camera FOV 120 to be able to image any section of the road segment 102, as needed. Likewise, for example, the road segment 102 has the plurality of sections monitored by the plurality of radars 114 via the plurality of radar FOVs 116 in a one-to-one correspondence (one road section per radar FOV), whereas the PTZ camera 118 is capable of monitoring all sections of the road segment 102 when the camera FOV 120 isDocket: 15257367-000100 Patent Specificationmoved (e.g., panned or tilted) from section to section, whether sequentially (e.g., from section three to section two) or skipping (e.g., from section two to section four), among all sections of the road segment 102. Therefore, as shown in FIG. 1 , the PTZ camera 118 is positioned higher in elevation, as measured from a ground level (e.g., a flat area, an inclined area, a grass area, a sidewalk area, a pavement area) from which that respective outdoor structure 112 of the plurality of outdoor structures 112 extends, than the plurality of radars 114. For example, the PTZ camera 118 can be elevated at about 150 feet or lower. As shown in FIG. 1 , the plurality of radars 11 have a many-to-one correspondence (e.g., 4 to 1, 6 to 1, 9 to 1, 13 to 1) to the PTZ camera 118 relative to the road segment 102, which enables for the plurality of radars 114 to operate as an initial screen / filter for getting the PTZ camera 118 involved (e.g., to start imaging), as disclosed herein, which minimizes computational load, simplifies data conflict resolution algorithms, and minimizes financial costs. Likewise, as shown in FIG. 1, the plurality of radars 114 have a many-to-one correspondence (e.g., 4 to 1, 6 to 1, 9 to 1, 13 to 1) to the communication block 122 relative to the road segment 102. Similarly, as shown in FIG. 1 , the PTZ camera 118 has a one-to-one correspondence to the communication block 122 relative to the road segment 102, although the PTZ camera 118 and the communication block 122 may be embodied as one physical device. Note that the PTZ camera 118 may run stabilization algorithms to compensate for potential inaccuracies (e.g., jitter) resulting from or introduced by vibrations caused at least from the stream of vehicles or the communication block 122 may run stabilization algorithms on imagery data to compensate for potential inaccuracies (e.g., jitter) resulting from or introduced by vibrations caused at least from the stream of vehicles.

[0055] The communication block 122 may be a computing unit embodied as an edge computer, an embedded computer, an industrial computer, a physical server, or another suitable computing form factor. As shown in FIG. 1, the communication block 122 may be embodied in a housing (e.g., weatherproof) attached (e.g., bracketed, fastened, adhered) to one outdoor structure 114 of the plurality of outdoor structures 112 sequentially positioned (e.g., one after another) lateral to the lane 106 of the plurality of lanes 106 of the road segment 102. For example, the housing may be attached to the one outdoor structure 114 of the plurality of outdoor structures 112 via an arm (e.g., weatherproof),Docket: 15257367-000100 Patent Specificationwhich may be cantilevered from the one outdoor structure 114 of the plurality of outdoor structures 112, as shown in FIG. 1 , although this configuration is not required and the arm may be omitted or not be cantilevered. For example, the one outdoor structure 114 of the plurality of outdoor structures 112 may have an external sidewall to or through which the housing may be attached, whether via the arm or not, to enable the communication block 122 to operate, as disclosed herein. However, note that this configuration is not required. For example, the one outdoor structure 114 of the plurality of outdoor structures 112 may have an apex side (e.g., a top side, a roof) to or through which the housing is attached, whether via the arm or not, to enable the communication block 122 to operate, as disclosed herein. Note that the communication block 122 may be omitted as a separate physical device and, instead, the communication block 122 may be integrated into the PTZ camera 118, i.e., to form one physical device (e.g., one housing), which may further optimize system architecture by enabling the PTZ camera 118 to perform all functionality of the communication block 122, as disclosed herein. As such, the PTZ camera 118 may be or host the computing unit embodied as the edge computer, the embedded computer, the industrial computer, the physical server, or another suitable computing form factor described above, and corresponding functionality can be adapted accordingly. For example, the PTZ camera 118 may be programmed to perform computer vision (e.g., image processing, object detection, Optical Character Recognition (OCR)) and radar control, as disclosed herein. For example, the PTZ camera 118 may host a firmware (e.g., embedded or built-in) or another suitable software logic (e.g., a software module, a software engine) running locally that may be programmed to perform computer vision (e.g., image processing, object detection, OCR) and radar control, as disclosed herein. For example, the PTZ camera 118 may have a processing unit that is powerful enough to do Artificial Neural Network (ANN)-based image recognition or run third-party applications, to further augment, expand, or supplement all functionality, as disclosed herein.

[0056] The PTZ camera 118, the communication block 122, and one radar 114 of the plurality of radars 114 are attached to one common outdoor structure 112 of the plurality of outdoor structure 112. For example, the PTZ camera 118 and the communication block 122 may be attached to a common outdoor structure 112 of the plurality of outdoorDocket: 15257367-000100 Patent Specificationstructures 112, whether embodied as separate and distinct physical devices or as one physical device (e.g., one housing). For example, one radar 114 of the plurality of radars 114, the PTZ camera 118, and the communication block 122 may be attached to a common outdoor structure 112 of the plurality of outdoor structures 112. However, this configuration is not required. For example, the PTZ camera 118 or the communication block 122 may not share any outdoor structure 112 of the plurality of outdoor structure 112 with any radar 114 of the plurality of radars 114, the communication block 122, or the PTZ camera 118. For example, the PTZ camera 118 and the communication block 122 may be attached to two different outdoor structures 112 of the plurality of outdoor structures 112.

[0057] The communication block 122 includes a processor (e.g., multicore), a memory (e.g., random, persistent, flash) coupled to the processor, a communication transmitter (e.g., wired, wireless, waveguide) coupled to the processor, and a communication receiver (e.g., wired, wireless, waveguide) coupled to the processor. The communication block 122 may include a power supply powering the processor, the memory, the communication transmitter, and the communication receiver. The power supply may be powered by a mains electricity source (e.g., a power line). The communication block 122 may include a modem, a network card, or another suitable network interface hosting the communication transmitter and the communication receiver. The communication transmitter and the communication receiver may be embodied to form a transceiver. When the communication block 122 is integrated into the PTZ camera 118, i.e., one physical device (e.g., one housing), the PTZ camera 118 may host the processor, the memory, the communication transmitter, and the communication receiver, with corresponding functionality being adapted accordingly.

[0058] The communication block 122, the plurality of radars 114, and the PTZ camera 118 may collectively form a communication network (e.g., a Local Area Network (LAN)). For example, the communication block 122, the plurality of radars 114, and the PTZ camera 118 may be logically connected to each other in a wired manner (e.g., cables, switches, routers), a wireless manner (e.g., via respective wireless communication transmitters and wireless communication receivers), or a waveguide manner (e.g., optical fibers, switches, routers). For example, the communication block 122 may beDocket: 15257367-000100 Patent Specificationprogrammed to control (e.g., on, off, start capture, pause capture, stop capture) the plurality of radars 114 and the PTZ camera 118, which may be in parallel. Likewise, for example, the communication block 122 may be programmed to communicate (e.g., wired, wireless, waveguide) with a control center (e.g., a network equipment item, a network router, a network switch, a physical server, a computing terminal) of an ITS.

[0059] Each radar 114 of the plurality of radars 114 measure speeds of objects and their geodesic coordinates, and generate corresponding statistical data on traffic flow on the road segment 102. How many radars 114 of the plurality of radars 114 may vary (e.g., depending on how radar configurations and positioning and how long the road segment 102 extends), but the PTZ camera 118 is positioned to allow inspections of all available control areas of the plurality of radar detectors 114 to enable the communication block 122 to process of data from the plurality of radars 114 and the PTZ camera 118. Therefore, the memory stores a set of instructions (e.g., a program) instructing the processor to perform various methods as disclosed herein. For example, the set of instructions may instruct the processor to generate event files that are transmitted to the control center 124 and contain a sufficient amount of information to enable the dispatcher or software at the control center to make an informed decision to respond to the incident. For example, the set of instructions may instruct the processor to control (e.g., on, off, start capture, pause capture, stop capture) the plurality of radars 114 and the PTZ camera 118. Likewise, for example, the set of instructions may instruct the processor to start, pause, or stop movement (e.g., pan or tilt) or zoom of the PTZ camera 118 for the camera FOV 120 to be capable of individually monitoring any section of the road segment 102, such that the camera FOV 120 can individually monitor the road segment 102 in its entirety, as needed. For example, the PTZ camera 118 can move (e.g., pan or tilt) or zoom the camera FOV 120 to be able to image any section of the road segment 102, as needed. Likewise, for example, the road segment 102 has the plurality of sections monitored by the plurality of radars 114 via the plurality of radar FOVs 116 in a one-to-one correspondence (one road section per radar FOV), whereas the PTZ camera 118 is capable of monitoring all sections of the road segment 102 when the camera FOV 120 is moved (e.g., panned or tilted) from section to section, whether sequentially (e.g., from section three to section two) orDocket: 15257367-000100 Patent Specificationskipping (e.g., from section two to section four), among all sections of the road segment 102.

[0060] The set of instructions enables the communication block 122 to be programmed to communicate (e.g., wired, wireless, waveguide) via the communication receiver and the communication transmitter with a control center (e.g., a network equipment item, a network router, a network switch, a physical server, a computing terminal) of an ITS over a communication network (e.g., a cellular network, a satellite network, a fiberoptic network, a LAN, a Wide Area Network (WAN), Internet). For example, the communication block 122 may communicate with the control center of the ITS in a wireless manner, a wired manner, or a waveguide manner. The control center may be remote from the communication block 122, the plurality of radars 114, and the PTZ camera 118.

[0061] The communication block 122 may command the PTZ camera 118 to move (e.g., pan or tilt) the camera FOV 120 from a first position or orientation (e.g., imaging a first section of the road segment 102) to a second position or orientation (e.g., imaging a second section of the road segment 102). In the first position or orientation, the camera FOV 120 is not capable of imaging (e.g., not within the camera FOV 120) an incident on the road segment 102 (e.g., within one lane 106 of the plurality of lanes 106) detected by the communication block 122 based on one radar 114 of the plurality of radars 114 observing the incident. In the second position or orientation, the camera FOV 120 is capable of imaging (e.g., within the camera FOV 120) the incident based on the communication block 122 detecting the incident based on the one radar 114 of the plurality of radars 114 observing the incident. For example, the road segment 102 has the plurality of sections monitored by the plurality of radars 114 via the plurality of radar FOVs 116 in a one-to-one correspondence (one road section per radar FOV), whereas the PTZ camera 118 is capable of monitoring all sections of the road segment 102 when the camera FOV 120 is moved (e.g., panned or tilted) from section to section, whether sequentially (e.g., from section three to section two) or skipping (e.g., from section two to section four), among all sections of the road segment 102. This approach is technologically advantageous, because the plurality of radars 114 operate as the initial screen / filter for the incident and the PTZ camera 118 is moved, on-demand, when the communication block 122 detects the incident based on the one radar 114 of the pluralityDocket: 15257367-000100 Patent Specificationof radars 114 observing the incident. Note that each radar 114 of the plurality of radars 114 may be configured to self-diagnose itself and inform the communication block 122 of whatever issues are detected such that the communication block 122 can act accordingly, as programmed. For example, some of issues may include sensor blindness, sensor interference, sensor misalignment in roll or pitch angle, rain, snow, smoke, or fog. For example, the communication block 122 can accordingly pause operation until the issue is resolved, continue operating, or notify the computing terminal of such issue. Note that to enable operations, as described above, the plurality of radars 114 and the PTZ camera 118 may need to be initially calibrated or recalibrated. For example, this initial calibration or recalibration may involve determining a position of the PTZ camera 118 axes in space by measuring a plurality of tilts of a plurality of landmarks in proximity of the road segment 102 or the lateral area 108, which may have been introduced due to installation errors. Then, taking these errors into account, for each target (e.g., the vehicle 110), a set of coordinates is calculated in the PTZ camera 118 space and a plurality of rotation angles of the PTZ camera 118 from a global geodetic coordinate system, in which a set of radar target coordinates are located.

[0062] The incident may involve the vehicle 110 moving (e.g., accelerating, decelerating) or being stationary (e.g., damaged, disabled) on the road segment 102 (e.g., within one or both lanes 106 or crossing the divider 104) or one lateral area 108 of the plurality of lateral areas 108. The incident may be on the road segment 102 (e.g., within one or both lanes 106 or crossing the divider 104) or one lateral area 108 of the plurality of lateral areas 108. As explained above, the one radar 114 of the plurality of radars 114 may have a respective radar FOV 116 enabling the incident to be observed on the road segment 102 (e.g., within one or both lanes 106) or one lateral area 108 of the plurality of lateral areas 108, for the communication block 122 to detect the incident. Note that the incident may involve individual vehicles in a traffic flow that move slowly (e.g., less than 20%-30% of a permitted speed on this section in an absence of traffic jams), a vehicle moving in a prohibited direction or in a territory prohibited for movement, a stopped vehicle, a burning or smoking vehicle, a vehicle that braked sharply up to a complete stop, people, animals, fallen trees or objects (e.g., boards, boxes, rocks), possible consequences of an accident (e.g., more than two stopped vehicles, one vehicle and people around, oneDocket: 15257367-000100 Patent Specificationstopped vehicle and animals, flames or smoke), traffic difficulties and traffic jams, or other suitable objects.

[0063] Once the PTZ camera 118 is moved (e.g., panned or tilted) from the first position or orientation to the second position or orientation, the communication block 122 may command the PTZ camera 118 to start, un-pause, or continue to generate an imagery (e.g., a photo, a video) based on the camera FOV 120 depicting the incident, as the camera FOV 120 is positioned in the second position or orientation. The PTZ camera 118 may generate the imagery in a visible spectrum during daytime or in an infrared spectrum during nighttime. The communication block 122 receives the imagery from the PTZ camera 118 positioned in the second position or orientation and attempts to perform a validation (e.g., to minimize a false positive) of the incident, while the PTZ camera 118 is positioned in the second position or orientation. For example, the validation may include running a set of computer vision heuristics (e.g., object detection, smoke detection, fire detection, pedestrian detection) on the imagery. Note that if the PTZ camera 118 has a wide-angle lens (e.g., a fisheye lens) warping the imagery, then the imagery may be dewarped, such as by using an algorithm disclosed in United States Patent 10565680, which is incorporated by reference herein for all purposes. This dewarping can happen at the PTZ camera 118 before the validation (although after is possible) or at the communication block 122 before the validation (although after is possible).

[0064] If the validation is successful (e.g., passes), then the communication block 122 uses the communication transmitter to notify the control center of the ITS of the incident (or perform another action), while the PTZ camera 118 is positioned in the second position or orientation should additional imagery from the PTZ camera 118 depicting the incident be needed if requested from the control center. For example, this notification may include the communication transmitter sending the imagery to the control center for display therein (e.g., at a computing terminal operated by a dispatcher) and the control center (e.g., at a computing terminal operated by a dispatcher) may request additional imagery from the PTZ camera 118 depicting the incident if the imagery as originally sent was informationally insufficient to the control center (e.g., unclear, blurry). However, if the validation is not successful (e.g., fails), then the communication block 122 ignores (or performs another action) the incident and continues with its operation. For example, ifDocket: 15257367-000100 Patent Specificationanother incident is subsequently detected by the communication block 122 based on tracking data from another radar 114 of the plurality of radars 114 on another section of the road segment 102 or one lateral area 108 of the plurality of lateral areas 108 not currently in view of the camera FOV 120, then the communication block 122 may command the PTZ camera 118 to move (e.g., pan or tilt) the camera FOV 120 from the second position or orientation to another position orientation (e.g., a third position or orientation) where the communication block 122 detected that other incident, to enable the camera FOV 120 to now image that section of the road segment 102 or one lateral area 108 of the plurality of lateral areas 108, to repeat processing described above. For example, the road segment 102 has the plurality of sections monitored by the plurality of radars 114 via the plurality of radar FOVs 116 in a one-to-one correspondence (one road section per radar FOV), whereas the PTZ camera 118 is capable of monitoring all sections of the road segment 102 when the camera FOV 120 is moved (e.g., panned or tilted) from section to section, whether sequentially (e.g., from section three to section two) or skipping (e.g., from section two to section four), among all sections of the road segment 102. For example, the radar FOVs 116 may collectively monitor the road segment 102 in its entirety (e.g., all its sections) or a shoulder thereof (e.g., lateral area 108) without overlapping each other. The camera FOV 120 may be capable of individually monitoring any section of the road segment 102 or the shoulder thereof. As such, if the incident is a first incident, the imagery is a first imagery, the validation is a first validation, then the communication block 122 may be programmed to command the PTZ camera 118 to move (e.g., pan or tilt) the camera FOV 120 from the first position or orientation or the second position or orientation each not capable of imaging a second incident (e.g., a car accident) on the road segment 102 or the shoulder thereof observed by another radar 114 of the plurality of radars 114 to a third position or orientation (e.g., imaging a third section of the road segment 102 or lateral area 108) capable of imaging the second incident responsive to the communication block 122 detecting the second incident based on the another radar 114 of the plurality of radars 114 observing the second incident. This movement of the PTZ camera 118 may be sequential (e.g., from section three to section two) or skipping (e.g., from section two to section four), among all sections of the road segment 102. As such, the communication block 122 may be programmed to (a) command the PTZ cameraDocket: 15257367-000100 Patent Specification118 to generate a second imagery (similar to the first imagery) depicting the second incident as the camera FOV 120 is positioned in the third position or orientation, (b) receive the second imagery from the PTZ camera 118 positioned in the third position or orientation, (c) perform a second validation (similar to the first validation) of the second incident while the PTZ camera 118 is positioned in the third position or orientation, (d) notify the control center of the ITS of the second incident responsive to the second validation being successful while the PTZ camera positioned in the third position or orientation, and (e) ignore the second incident responsive to the second validation being unsuccessful. For example, the second incident may be on the shoulder thereof.

[0065] The incident may involve an object (e.g., the vehicle 110, a person, an animal) having a distance (e.g., about 0.2 miles) from the one radar 114 of the plurality of radars 114 when the incident is observed by the one radar 114 of the plurality of radars 114. The object may have a speed (e.g., about 50 miles per hour, about 3 miles per hour) on the road segment 102 or one lateral area 108 of the plurality of lateral areas 108 when the incident is observed by the one radar 114 of the plurality of radars 114. The one radar 114 of the plurality of radars 114 may measure the distance and the speed relative thereto such that the communication block 122 may determine a geographical or positional coordinate (e.g., an X-plane coordinate, a Y-plane coordinate, a Z-plane coordinate, a Cartesian plane coordinate, a longitude coordinate, a latitude coordinate, a geolocation coordinate, a Global Positioning System (GPS) coordinate) associated with the object based on the distance and the speed. As such, the communication block 122 may command the PTZ camera 118 to move (e.g., pan or tilt) the camera FOV 120 from the first position or orientation to the second position or orientation based on the geographical or positional coordinate such that the camera FOV 120 captures the object in the second position or orientation.

[0066] Since the PTZ camera 118 is elevated at a height (e.g., within about 150 feet or less) from a ground level (e.g., a flat area, an inclined area, a grass area, a sidewalk area, a pavement area) from which that respective outdoor structure 112 of the plurality of outdoor structures 112 extends, the PTZ camera 118 may generate the imagery depicting the incident as the camera FOV 120 is in the second position or orientation by capturing at least two images (e.g., photos) with a time period therebetween (e.g., withinDocket: 15257367-000100 Patent Specificationabout 5, 4, 3, 2, or 1 second or less), to enable depiction of displacement of the object during the time period, for ease of analysis (e.g., human review) at the control center of the ITS when those two images are send by the communication transmitter from the communication block 122 to the control center of the ITS responsive to the incident being detected by the communication block 122 and validated. The communication block 122 may determine the time period based on the height, the distance, and the speed, to enable depiction of displacement of the object during the time period. As such, on a per incident basis, the time period may be dynamically set, depending on the height, the distance, and the speed. Therefore, the time period may vary based on the height, the distance, and the speed.

[0067] The control center of the ITS may include a computing terminal (e.g., a desktop computer, a laptop computer, a vehicular computer) having a display. The computing terminal may be operated by a dispatcher. The control center of the ITS may be notified of the incident by the communication block 122 based on the communication block 122 sending the two images along with a set of corresponding metadata (e.g., a date identifier, a time identifier, a camera identifier, a pan degree identifier, a tilt degree identifier, a zoom degree identifier, an object identifier, an object attribute, an object distance, an object speed, an incident identifier, an incident descriptor) such that the display displays the two images simultaneously, which may be side-by-side. Also, the display may display a digital map (e.g., an overhead map, a terrain map, a schematic map, a satellite map) having a spatiotemporal annotation by color to locationally show the incident based on the geographical or positional coordinate. Note that the spatiotemporal annotation by color (e.g., a higher altitude shows one color and a lower altitude shows another color) may be enabled to present the imagery over or adjacent to the digital map when the spatiotemporal annotation by color is user activated from the computing terminal.

[0068] The communication block 122 may command the PTZ camera 118 to move the camera FOV 120 from the second position or orientation to a default position or orientation based on the computing terminal not receiving a user input (e.g., from a physical or virtual keyboard, a cursor control unit, a mouse unit, a trackball unit, a touchpad unit) from the dispatcher relative to the incident within a time period (e.g., within about 1 minute, 30 seconds). The time period may start to be counted from when the display of the computingDocket: 15257367-000100 Patent Specificationterminal displayed a notice indicating the incident as being detected and validated by the communication block 122. If the computing terminal termina does receive the user input within the time period, then the PTZ camera 118 remains in the second position or orientation, until the dispatcher operates the computing terminal to command the PTZ camera 118 to move to another position or orientation or the default position or orientation. This approach enables the PTZ camera 118 to resume its visual patrol, on-demand, if the dispatcher does not view the incident as taking place (e.g., false positive) or not worthy of a response from the dispatcher.

[0069] The computing terminal may host an Operating System (OS) and an application running on the OS, where the application is programmed to operate in a plurality of modes. For example, the plurality of modes may correspond to a plurality of user profiles associated with a plurality of dispatchers, including the dispatcher. For example, some dispatchers may set the time period to 1 minute and some dispatchers may set the time period to 30 seconds, although this timing can be adjusted. Likewise, for example, the plurality of modes may correspond to a day mode and a night mode, where the day mode may have the time period be shorter than the night mode, due to potentially poor visibility, although this timing can be adjusted. Similarly, for example, the plurality of modes may corresponds to a normal weather mode (e.g., a clear visual conditions) and an abnormal weather mode (e.g., a foggy visual condition, a rainy visual condition, a snowy visual condition, a smokey visual condition), where the normal weather mode may have the time period be shorter than the abnormal weather mode, although this timing can be adjusted. Therefore, the time period may have a length that varies depending on which mode of the plurality of modes the application is operating in when the computing terminal is notified of the incident (e.g., when the notification indicating the incident is displayed).

[0070] FIG. 2 shows a schematic diagram of an embodiment of a communication arrangement according to this disclosure. In particular, there is a communication arrangement 200 formed using the physical arrangement 100, where the communication arrangement 200 includes the plurality of radars 114, the PTZ camera 118, the communication block 122 having a computational unit 122.1 (e.g., the processor and the memory described above) and a router 122.2 (e.g., the communication transmitter and the communication receiver described above), and a control center 124 (the controlDocket: 15257367-000100 Patent Specificationcenter described above) of an ITS 126 (the ITS described above). The communication arrangement 200 enables the physical arrangement 100 to operate as described above.

[0071] FIG. 3 shows a schematic diagram of an embodiment of a logical arrangement according to this disclosure. In particular, there is a logical arrangement 300 including a plurality of blocks corresponding to a plurality of logic units (e.g., software modules) for a plurality of components of the communication arrangement 200 and the physical arrangement 100 to operate as described above. For example, the plurality of logic units includes interfaces for creating a map of an area (the road segment 102 or lateral area 108), marking control zones on the area with setting parameters for event detection, filters for suppressing interference and repeated events, interfaces for equipment calibration, interfaces for controlling operating modes, Artificial Neural Network (ANN) and logic detectors for object detection and analysis of types of events, a database for storing statistical data on traffic from the plurality of radars 114, servers for transferring data to the control center 124 of the ITS 126 upon request from there and on an initiative basis as requested.

[0072] FIG. 4 shows a schematic diagram of an embodiment of a subsystem arrangement according to this disclosure. In particular, there is a subsystem 400 disclosing how the physical arrangement 100, the communication arrangement 200, and the logical arrangement 300 operate as described above.

[0073] FIG. 5 shows a schematic diagram of an embodiment of a plurality of incidents that can be identified according to this disclosure. In particular, there is a plurality of incidents 500 detectable as enabled by the physical arrangement 100, the communication arrangement 200, and the logical arrangement 300 operating as described above in view of the subsystem 400. For example, the incident described in context of FIG. 1 or the plurality of incidents 500 may be detected on the road segment 102 or lateral area 108 and may involve pedestrians, a road accident involving the vehicle 110, a wrong direction of movement of the vehicle 110, the vehicle 110 being stopped, a slowdown of the vehicle 110, or other suitable incidents disclosed herein.

[0074] FIG. 6 shows a schematic diagram of an embodiment of a computing terminal operated by a dispatcher where the computing terminal runs an application programing presenting a Graphical User Interface (GUI) that received a communication from aDocket: 15257367-000100 Patent Specificationcomputing unit controlling a plurality of radars and a PTZ camera according to this disclosure. In particular, as shown in FIG. 1, there is a computing terminal 600 (e.g., a laptop computer, a vehicular computer, a desktop computer) operated by a dispatcher. The computing terminal 600 hosts an OS (e.g., Windows) and runs an application program running on the OS. The application program presents a Graphical User Interface (GUI) that received a communication (e.g., a notice) from the communication block 122 controlling the plurality of radars 114 and the PTZ camera 118, as described above in context of the physical arrangement 100, the communication arrangement 200, and the logical arrangement 300. As exemplified, the plurality of radars 114 has a 4-to-1 correspondence to the PTZ camera 118 for every road segment 102 extending for about 0.62 miles (having 1 lane to 12 lanes with 6 lanes in each direction), where each radar 114 of the plurality of radars 114 monitors a section of the road segment 102 and this section is about 0.155 miles in length. Note that this configuration is adjustable, depending on configuration of the plurality of radars 114, the PTZ camera 118, and how long the road segment 102 extends. For example, there may be more or less radars 114 corresponding to the PTZ camera 118 or the road segment 102 may be longer or shorter or there may be more or less lanes 106.

[0075] As described above, the GUI presents a screen with two images (in two viewing panes one above another) taken by the PTZ camera 118 at two different points in time to enable depiction of displacement of the vehicle 110 during the time period, for ease of analysis (e.g., human review) at the control center of the ITS when those two images are send by the communication transmitter from the communication block 122 to the control center of the ITS responsive to the incident being detected by the communication block 122 and validated. The communication block 122 may determine the time period based on the height, the distance, and the speed, to enable depiction of displacement of the object during the time period. As such, on a per incident basis, the time period may be dynamically set, depending on the height, the distance, and the speed. Therefore, the time period may vary based on the height, the distance, and the speed. Note that the GUI also presents related metadata (see right viewing pane) and list of alerts (see left viewing pane).Docket: 15257367-000100 Patent Specification

[0076] FIG. 7 shows an embodiment of a plurality of screenshots showing a GUI depicting a plurality of viewing panes presenting an incident involving a pedestrian on a road segment according to this disclosure. In particular, as shown in FIG. 1, the GUI presents a screen 700 with four viewing panes presenting an incident involving a pedestrian on the road segment 102. The screen 700 enables the dispatcher to select or activate (e.g., click) a user input element (e.g., a hyperlink, a button) to view the imagery (e.g., a video) depicting the incident, along with presenting a bounding box (e.g., orange) enclosing what the communication block 122 detected as the incident, i.e., the pedestrian on the road segment 102. Note that two images (in two viewing panes one above another) taken by the PTZ camera 118 at two different points in time to enable depiction of displacement of the vehicle 110 during the time period, for ease of analysis (e.g., human review) at the control center of the ITS when those two images are send by the communication transmitter from the communication block 122 to the control center of the ITS responsive to the incident being detected by the communication block 122 and validated. The communication block 122 may determine the time period based on the height, the distance, and the speed, to enable depiction of displacement of the object during the time period. As such, on a per incident basis, the time period may be dynamically set, depending on the height, the distance, and the speed. Therefore, the time period may vary based on the height, the distance, and the speed. Note that the GUI also presents related metadata (see right viewing pane) and list of alerts (see left viewing pane). Likewise, note that the digital map is shown as a satellite map presenting a plurality of bounding boxes (e.g., green) indicating a plurality of identified objects (e.g., a stream of vehicles), a bounding box where first of the two images was taken (e.g., yellow), and a bounding box the second of the two images was taken (e.g., red). Note that this color coding scheme can be adjusted to different colors.

[0077] FIG. 8 shows an embodiment of a plurality of screenshots showing a GUI depicting a plurality of viewing panes presenting incident involving a car accident on a road segment according to this disclosure. In particular, as shown in FIG. 1, the GUI presents a screen 800 similar to the screen 700, where the incident involves a car accident on the road segment 102.Docket: 15257367-000100 Patent Specification

[0078] FIG. 9 shows an embodiment of a plurality of screenshots showing a GUI depicting a plurality of viewing panes presenting an incident involving a vehicle moving in a wrong direction on a road segment according to this disclosure. In particular, as shown in FIG. 1, the GUI presents a screen 900 similar to the screen 700 or the screen 800, where the incident involves a vehicle (the vehicle 110) moving (e.g., accelerating or decelerating) in a wrong direction on the road segment 102.

[0079] FIG. 10 shows an embodiment of a plurality of screenshots showing a GUI depicting a plurality of viewing panes presenting an incident involving a vehicle stopped on a road segment according to this disclosure. In particular, as shown in FIG. 1 , the GUI presents a screen 1000 similar to the screen 700, the screen 800, or the screen 900, where the incident involves a vehicle (the vehicle 110) stopped (e.g., disabled or damaged) on the road segment 102.

[0080] FIG. 11 shows an embodiment of a plurality of screenshots showing a GUI depicting a plurality of viewing panes presenting an incident involving a slowed movement of a vehicle on a road segment according to this disclosure. In particular, as shown in FIG.1 , the GUI presents a screen 1000 similar to the screen 700, the screen 800, the screen 900, or the screen 1000, where the incident involves a slowed movement of a vehicle (the vehicle 110) on the road segment 102.

[0081] FIG. 12 shows an embodiment of method comprising a plurality of steps of detecting an incident according to this disclosure. In particular, in view of FIGS. 1-11, there is a method 1200 illustrated.

[0082] The method 1200 includes a first step involving the communication block 122 detecting the incident on a section of the road segment 102 or the lateral area 108, as observed by one radar 114 of the plurality of radars 114, and commanding the PTZ camera 118 to move from the first position or orientation not capable of imaging the incident to the second position or orientation capable of imaging the incident, based on a set of coordinates (e.g., a set of GPS coordinates) associated with the section of the road segment 102 or the lateral area 108, to enable the incident to be within the camera FOV 120.

[0083] The method 1200 includes a second step involving the PTZ camera 118 moving from the first position or orientation to the second position or orientation, asDocket: 15257367-000100 Patent Specificationcommanded by the communication block 122, for the incident to be within the camera FOV 120, and captures the two images, as described above, with the time period therebetween (e.g., about 1 second difference), to enable depiction of displacement of the object during the time period, for ease of analysis (e.g., human review) at the control center of the ITS when those two images are send by the communication transmitter from the communication block 122 to the control center of the ITS responsive to the incident being detected by the communication block 122 and validated. The communication block 122 may determine the time period based on the height, the distance, and the speed, to enable depiction of displacement of the object during the time period. As such, on a per incident basis, the time period may be dynamically set, depending on the height, the distance, and the speed. Therefore, the time period may vary based on the height, the distance, and the speed.

[0084] The method 122 includes a third step involving the communication block 122 sending a notification (e.g., a message, an email message, a text message, a chat message, an archive file, a zip file) of the incident to the control center 124 of the incident, along with the two images and corresponding metadata. As such, as described above, the notification may prompt the dispatcher for attention such that the dispatcher operating the computing terminal may review the notification, along with the two images and the corresponding metadata. If appropriate, then the dispatcher may request (e.g., by a telephone) an emergency responder (e.g., a police officer) to handle the incident.

[0085] FIG. 13 shows an embodiment of a plurality of screens of a GUI notified of an incident after validation of the incident according to this disclosure. In particular, in view of FIGS. 1-12, the GUI may present a plurality of screens 1300 notifying the dispatcher of the incident, after validation thereof by the communication block 122, as described above. On screen 1 , as presented on the display of the computing terminal, the dispatcher may operate the computing terminal (e.g., from a physical or virtual keyboard, a cursor control unit, a mouse unit, a trackball unit, a touchpad unit) to select an incident identifier from a list of incident identifiers that have been previously detected, as described above. On screen 2, as presented on the display of the computing terminal, the dispatcher may operate the computing terminal (e.g., from a physical or virtual keyboard, a cursor control unit, a mouse unit, a trackball unit, a touchpad unit) to explore a set of data correspondingDocket: 15257367-000100 Patent Specificationto the incident identifier, as described above. For example, the screen 2 may depict (i) two viewing panes presenting the two images captured, as described above, (ii) two viewing panes presenting the metadata for the two images captured, as described above, (iii) a viewing pane presenting a set of location data associated with the incident, and (iv) an action pane presenting a plurality of user input elements (e.g., hyperlinks, buttons) for corresponding action associated with the incident. For example, the plurality of user input elements may include a plurality of buttons, where (i) one button initiates an alarm (e.g., request assistance of an emergency responder), (ii) another button initiates a live video feed from the PTZ camera 118 in the second position or orientation depicting the incident based on the communication block 118 commanding the PTZ camera 118 to initiate the live video feed, (iii) another button initiates playback video (e.g., captured in the second position or orientation after the two images), or (iv) another button may be accepting the incident to enter further information related to the incident. On screen 3, as presented on the display of the computing terminal, the dispatcher may operate the computing terminal (e.g., from a physical or virtual keyboard, a cursor control unit, a mouse unit, a trackball unit, a touchpad unit) to handle the incident as per protocol based on selecting the accepting the incident button. On screen 4, as presented on the display of the computing terminal, the dispatcher may operate the computing terminal (e.g., from a physical or virtual keyboard, a cursor control unit, a mouse unit, a trackball unit, a touchpad unit) to submit the report for documentation and enable the PTZ camera 118 to resume visual patrol.

[0086] FIGS. 14-15 shows an embodiment of a plurality of technical specifications for a radar according to this disclosure. In particular, in view of FIGS. 1-13, each radar 114 of the plurality of radars 114 may have a set of characteristics 1400 and a set of characteristics 1500. Note that all of these characteristics are examples and any of these characteristics can change (e.g., higher or lower) independent of others or depending on at least one other characteristic.

[0087] FIGS. 16-17 shows an embodiment of a plurality of technical specifications for a camera according to this disclosure. In particular, in view of FIGS. 1 -15, the PTZ camera 118 may have a set of characteristics 1600. Note that all of these characteristics areDocket: 15257367-000100 Patent Specificationexamples and any of these characteristics can change (e g., higher or lower) independent of others or depending on at least one other characteristic.

[0088] FIG. 18 shows an embodiment of an incident management platform according to this disclosure. In particular, in view of FIGS. 1-17, the control center 124 may have a video wall, a plurality of computer monitors, or the computing terminal described above to provide a visual interface 1800 to handle the incidents, as described above. The viewing panes of the GUI described above may be presented on a single monitor or be distributed among the video wall or the plurality of computer monitors, which may be user customizable or movable on the video wall or the plurality of computer monitors as needed.

[0089] FIG. 19 shows an embodiment of a plurality of screens a GUI for management of incidents as a function of time according to this disclosure. In particular, in view of FIGS.1-18, there is a plurality of screens 1900 presented on the video wall, the plurality of computer monitors, or the computing terminal described above. The plurality of screens 1900 includes a first viewing pane presenting the digital map described above and a second viewing pane presenting a general list of events and a filtered list of events, where the first viewing pane is presented side-by-side the second viewing pane, although one-above-another presentation is possible.

[0090] The first viewing pane depicts the digital map presenting a plurality of icons corresponding to a plurality of locations where a plurality of PTZ cameras 118 are installed, each with a corresponding plurality of radars 114, as described above. The digital map can be panned and zoomed in or out, as needed. The digital map can be proprietary or accessed from an Application Programming Interface (API) availed by a map creator (e.g., Google, Apple, TomTom), which can be user-selectable (see dropdown menu). When each icon of the plurality of icons is user-activated by clicking, then a respective location window is presented over or adjacent to the digital map, which may be external to the first viewing pane. The respective location window presents a set of user input elements (e.g., checkboxes, dropdown menus, text fields) and a set of corresponding labels, which collectively display of relevant locational metadata and allow adjustment of relevant camera information, as shown in FIG. 19.

[0091] The second viewing pane presents the general list of events and the filtered list of events, one above another, although side-by-side presentation is possible. The generalDocket: 15257367-000100 Patent Specificationlist of events are events (e.g., incidents) that observed by the plurality of PTZ cameras 118, each with the corresponding plurality of radars 114, associated with the icons presented on the digital map. The general list of events is filtered to generate the filtered list of events, where such filtering is classified by event type, such as emergency events, interference events, traffic issue events, technical issue events, or any other suitable event type. The filtered list of events is color-coded by type, such that each row in the filtered set of events is color-coded to correspond to event type. When a row in the filtered set of events is activated (e.g., clicked), then a respective event window may be presented, whether internal to the first viewing pane or the second viewing pane, or external to the first viewing pane and the second viewing pane, which may be side-by-side or one-above-another, as described above. The respective event window presents a form populated with event information corresponding to a specific event at a specific location on the digital map corresponding to a specific icon described above. As shown in FIG. 19, the form may be prepopulated with date, time, location identifier, event identifier, parameter identifier, and allow for comment entry (e.g., text field) or opening (e.g., by clicking a button on the form) of another window to view of statistics for that location.

[0092] FIG. 20 shows an embodiment of a screen of a GUI for viewing performance statistics according to this disclosure. In particular, in view of FIGS. 1 -19, there is a screen 200 shown that presents a dashboard to present performance statistics, based on an individual location having the PTZ camera 118 with the plurality of radars 114 or on a plurality basis for a plurality of locations, as shown on the digital map, where the plurality of PTZ cameras 118 with corresponding pluralities of radars 114 are installed. For example, the screen 200 (or any other screen or GUI disclosed herein) may be accessible via a web browser (e.g., Firefox, Edge, Chrome, Safari).

[0093] FIGS. 21-29 shows an embodiment of a plurality of screens of a GUI for configuring operation of a plurality of radars and a PTZ camera according to this disclosure. In particular, in view of FIGS. 1-20, there is a preparatory phase involving a cartographic and indicatory interface 2100 a satellite image of an area centered on a location of a proposed installation of the PTZ camera 118 near the road segment 102 or the lateral area 108. The satellite image is formed in such a size that slightly exceeds a size of a circle with a radius of a possible maximum distance of the PTZ camera 118. InDocket: 15257367-000100 Patent Specificationa screen 2200, on a terrain image, information about a plurality of control zones on the road segment 102 is plotted by means of a graphical interface. For each control zone of the plurality of control zones, restrictions are set for a minimum permissible speed, permitted directions of movement, traffic ban, ban on data analysis, statistical data collection boundaries, positioning points for video image analysis, or other suitable criteria. In a screen 2300 and a screen 2400, there is a control interface enabling a selection of a plurality of parameters of an interference rejection filter for radar data and a system operating mode. In a screen 2500, a user may calibrate each radar 114 of the plurality of radars 114 using geolocation data such that a set of coordinates of a position of a target relative to that respective radar 114 that this respective radar 114 measures are correctly converted into geodetic and GPS (or other geolocation system) coordinates. The PTZ camera 118 is calibrated using geodata so that the PTZ camera 118 can pan or tilt to different positions so that its optical axis can be pointed at the road segment 102 or the lateral area 108 at a specified set of GPS coordinates of the target. In a screen 2600, on the digital map of the area, a sequence of patrol points is set for the PTZ camera 118 to view at a time when all existing active targets from the plurality of radars 114 have already been worked out, and new active targets have not yet appeared, so that when the PTZ camera 118 bypasses this sequence of points, the PTZ camera 118 can analyze images of scenes and events found by the plurality of radars 114 based on ANNs (e.g., people, animals, objects standing vehicles, flames, smoke), so that when the PTZ camera 118 is operating in a patrol mode, these events can be detected.

[0094] There may be an algorithm used to form a digital map of an area with a demarcation of a plurality of control zones. This algorithm may involve obtaining an image of the area with reference to a set of GPS coordinates, marking the plurality of control zones in the image, and setting allowed target parameter limits for each control zone such that the digital map is ready.

[0095] There may be an algorithm used to calibrate a radar 114 of the plurality of radars 114. The algorithm may involve inserting a marker for the radar 114 on the digital map, displaying a plurality of target markers on the digital map using the a set of a target relative to the radar 114, and entering an azimuth angle and shifting a location of theDocket: 15257367-000100 Patent Specificationmarker for the radar 114 on the digital map so that the target corresponds to a real position of the vehicle 110 on the road segment 102.

[0096] There may be an algorithm used to calibrate the PTZ camera 118. The algorithm may involve obtaining a set of GPS coordinates of the PTZ camera 118, plotting a location of the PTZ camera 118 on the digital map of the area, selecting landmarks and obtaining their coordinates on the digital map, calculating a position of the PTZ camera 118 axes in space from terrain landmarks, and calculating of a set of coefficients of a matrix for converting the set GPS coordinates of targets into rotation angles of the PTZ camera 118.

[0097] There may be an algorithm for forming an event file. The algorithm may involve generating a set of radar data indicating measured speed, coordinates and statistics of a traffic flow by a radar 114 of the plurality of radars 114 generate, the set of radar data is filtered for interferences, determining which control zone contains a target based on a set of coordinates for the target, analyze acceleration and statistical data to determine whether a set of parameters for the target satisfy or do not satisfy limits set of that control zone. If not, then indicating the target on the image of the digital map of the area by its coordinates, as not active. If yes, then using the set of GPS coordinates to calculate the rotation angles of the PTZ camera 118, turning the PTZ camera 118 as appropriate, causing the PTZ camera 118 to capture at least two images of a scene taken at two different points in time, as described above. Further, the algorithm involves indicating the target as being active and the at least two images are processed by ANNs seeing vehicles, flames, smoke, people, animals, or other suitable aspects indicative of the incident to have been occurred or occurring. Further, the algorithm involves analyzing whatever ANN has found and an event type is specified (e.g., a car accident, a car on fire). If the event type indicates that the incident occurred, then the event file is formed where an event scene image with feature markups is displayed on top of an image of a section of the digital map where the event scene is located and the event file is sent to the control center 124 as described above. Note that if the PTZ camera 118 is not positioned or oriented to capture the at least two images described above and not currently imaging a detected and validated incident described above, the algorithm may cause the PTZ camera 118 to move (e g., pan or tilt) to one of several predefined points capable of imaging the roadDocket: 15257367-000100 Patent Specificationsegment 102 or the lateral area 108 from a sequence of predefined points. At that time, a set of image data is accumulated to detect stationary objects (e.g., cars). As such, if those objects are found and an incident is detected, then the event file is formed to indicate a stationary object with a sign of an accident and the event file is sent to the control center 124 as described above to enable display of an event scene image with feature markups is displayed on top of an area map image. However, if those objects are not found, then the event file is formed to indicate a stationary object and the event file is sent to the control center 124 as described above to enable display of an event scene image with feature markups is displayed on top of an area map image.

[0098] For example, the radar 114 may analyzes the speed of the vehicle 110, the acceleration of the vehicle 110 is calculated based on changes in the speed value, the direction of movement of the vehicle 110, and the current coordinates of the vehicle 110 are revealed. The map of the area, on which the road zones are marked, containing prohibited zones, zones with a certain permitted direction of movement, zones with a given permissible level of speed and acceleration of braking, or others. The coordinates of targets received from the radar 114 are analyzed and determined in which control zone of the terrain the digital map the moving object is located, checks the conditions for the permissible movement of the object in this zone are checked.

[0099] If the coordinates of the target or the magnitude of the speed or acceleration of braking or the direction of movement are in the area of prohibited values, then the rotation angles for the PTZ camera 118 are calculated from the geodetic coordinates of such targets and the PTZ camera 118 is pointed at the section of the road segment 102 where there is an event that goes beyond the specified limits and two or more pictures of the scene are taken with a certain time interval. The time interval is a value which is calculated by an algorithm to ensure the visible movement (displacement) of the target on the image of the scene by a given value in pixels e.g., 200 pixels), with the help of ANNs that identify identical objects in the scene image and by the distance between such objects with a given value (e.g., 200 pixels). The operator may be assigned exactly the target to which the radar measurements related against the background of many other targets, road users, an emergency braking, for example, event file is generated, and a background detecting software is used and the scene is checked for a possible accident using anDocket: 15257367-000100 Patent Specificationalgorithm for signs of difference between the scene of the accident (e.g., signs of an accident: two or more vehicles stopped on the stage in the same place, or there are people and one or more stopped vehicles, or there are one or more stopped vehicles and animals or objects on the roadway, or flames and smoke). From the scene of a temporary stop of the vehicle 110, if an accident is detected, then filtering is carried out against repeated triggers and false alarms. If all filters are passed successfully, then files of the appropriate types are generated with images and metadata included in them.

[0100] The at least two or more images and the time interval between them may be needed so that video analytics algorithms can identify and indicate on the image of the scene against the background of the flow of vehicles (all road users) exactly the target of interest to the operator with anomalous movement parameters. As such, the dispatcher in the control center 124 can unambiguously visually understand and check how objects move around the scene and, comparing with the service information about the type of event, and make an informed decision on the reliability of the identified events and take the necessary actions. If people, animals, other objects that should not be on the road, smoke, flames or signs of an accident are found, then new data is entered into the service information of the event file, where the image marks the areas to which the dispatcher should pay special attention and everything in general can significantly affect the priority and essence of the operator's reaction to events.

[0101] While the PTZ camera 118 is not busy shooting scenes based on information from the radar 114, the communication block 122 receives the values of pre-calculated rotation angles for certain geodetic or GPS coordinates of points of road sections for which the software of the communication block 122 uses video analytics methods and ANN algorithms to detect fire, smoke, stopped vehicles, people, animals, fallen trees or objects (e.g., boards, boxes, rocks), checks the admissibility of the location of such objects for zones on the road map and can generate the appropriate types of event files in case of non-compliance of the position of the detected objects with the permissible zones for them and the presence of an accident. In terms of the number of frames and the structure of service data, the files are similar to those formed on the basis of radar data, with the difference that they relate to stationary or very slow-moving objects that are not recorded by the radar 114 reliably enough.Docket: 15257367-000100 Patent Specification

[0102] Based on the analysis of statistical data from the plurality of radars 114, the state of difficulties and traffic jams is calculated, the PTZ camera 118 is moved (e.g., panned or tilted) to the point at the coordinates of the place where the traffic impediment is detected and the two or more images of this place are taken, files of the corresponding event are formed, using a background detecting software for which the time of the data set is calculated by a special algorithm so as to enable its work to suppress images of moving targets at speeds higher of the specified threshold at any angles of road observation (e.g., the time of the data set depends on the angles of observation on the target and the value of the speed threshold, which will be considered equivalent to no traffic) and the accident search algorithm checks the scene for a possible accident and generates files of the appropriate types.

[0103] Based on what is described above, this technology employs a method that combines data from two different types of devices: the radar 114 and a video analytics detector based on the PTZ camera 118. This approach enables the joint processing of data to provide extensive information about road traffic and events with high accuracy and improved responsiveness, surpassing the capabilities of each method individually. To ensure reliable operation, the method is used for calculating the data acquisition time and identifying anomalous targets based on radar-measured vehicle speeds relative to traffic flow and camera positioning parameters. For the video analytics detector's background extraction algorithm, the method is used for calculating data acquisition time based on threshold vehicle speed (treated as stationary) and camera positioning parameters. Additionally, in the accident detection module, the method is used to prevent repeated qualification of multiple braking events as accidents at locations where an accident has already been recorded or where braking occurs near parked vehicles. Sets of filters, filter interfaces, detector control interfaces, and interfaces for working with road elevation maps have been developed. For calibrating the vertical position of the PTZ axis in space, the method is used that avoids using PTZ sensor angle measurements, thus eliminating errors introduced by these sensors.

[0104] As shown in FIGS. 1-29, this is technologically advantageous for several reasons. Initially, there is a challenge in identifying anomalous vehicles in that the vehicles moving outside established limits (e.g., speed, acceleration, or position) must be identifiedDocket: 15257367-000100 Patent Specificationagainst general traffic flow and marked on scene images for operators. Radar data on speed and position must correspond precisely to vehicles marked on scene images. However, aligning radar data with scene images using PTZ cameras is highly challenging due to inaccuracies in PTZ angle settings, zoom levels, and external calibration difficulties caused by constant camera rotations to unknown angles and the lack of landmarks with known coordinates. At longer distances, these inaccuracies significantly increase, making it difficult to correctly identify anomalous vehicles in scene images. Further, there are radar limitations in that the radars cannot reliably detect completely stationary vehicles. Vibrations and other interference often result in false targets with speeds of a few km / h, requiring all such radar measurements — including real targets with low speeds — to be ignored. This enable an alternative channel for detecting stationary or slow-moving vehicles (below a few km / h). Although this challenged is addressed above through a video background detection algorithm that sums and normalizes pixel data over several dozen or hundreds of images from the camera during a specific data acquisition period. However, determining this period depends on factors such as the angle between the camera's optical axis and vehicle movement direction, distance to vehicles, camera height above the road, and threshold speed for stationary classification. Additionally, there are PTZ camera calibration issues in that for stationary cameras, data acquisition time can be set as a constant based on detector performance results. However, using such constants for PTZ cameras is suboptimal or slows detection when the camera moves across different road areas at varying angles to vehicle movement direction. Accurate targeting of road surface points based on GPS or geodetic coordinates requires elevation data. Unfortunately, such information is not always available with acceptable accuracy, limiting radar-camera integration capabilities since mid-range radars are insensitive to road elevation. Also, for detecting possible traffic accidents, data from radars regarding vehicle braking acceleration values are used, which are calculated based on changes in vehicle speeds measured by radar. The algorithm that distinguishes between braking with no reason, even if such braking ended with the vehicle stopping, and an actual accident, applies the condition that the image of the potential accident scene, which is analyzed after pointing a video camera at the presumed accident site, must contain two or more stationary vehicles (possible accident participants) or one vehicle and people or animalsDocket: 15257367-000100 Patent Specificationor objects. However, this condition can be met not only during an accident but possibly during vehicle braking against the background of a parking lot or during braking against the background of a previously occurred accident that has already been processed earlier, which leads to numerous false designations of simple vehicle braking with no reason as accidents. Moreover, the ITS may be connected to several hundred to several thousand PTZ cameras of various models and costs. Each camera during installation may have errors that deflect the axis of rotation of the PTZ mechanism in azimuth from the vertical position. The error in the form of deviation from vertical leads to a significant targeting error in range along the road. Cost-effective and easy-to-use methods for calibrating the position of the PTZ axis in space are needed. Currently, some technologies apply a constant target acquisition time value, which has the maximum possible value and is selected for the worst possible camera position, when the target moves directly toward the camera or at a minimum possible angle (a few degrees) to the camera's optical axis. This value ensures functionality, but with excessive time costs for situations when the target moves relative to the camera's optical axis not at acute angles, but at angles close to 90 degrees. The selection of this value is very approximate and experimental. Such a solution is not always possible for identifying abnormally moving vehicles in a scene image, because if the time between two frames is constant, the vehicle may leave the field of view or its displacement will be very small, and in any case, the displacement will not have a constant value and there will not be the necessary criterion for distinguishing a specific vehicle against other vehicles. In situations where a large number of false positives may occur, which would lead to the dispatcher overload in the control center, the accident detector algorithm (e g., software module) is turned off and accidents in such locations are not recorded. Operation may be limited to recording simple vehicle braking events for unknown reasons. For calibrating the position of PTZ cameras, additional inclinometers are used or a large number of reference points and numerous operations by personnel are required. When using coordinate conversions of the target from a geodetic basis to the PTZ mechanism basis using an inverse matrix, some technologies use angle measurement sensors of the PTZ mechanism, which leads to a noticeable increase in the calculation error of the PTZ mechanism position angles. In situations where a height map for the road is unavailable, the targeting accuracy deteriorates.Docket: 15257367-000100 Patent SpecificationTherefore, technologies described above in context of FIGS. 1-29 solve these technological problems and provide substantial technological advantages.

[0105] Algorithm for calculating data accumulation time for correct operation of background detector and anomalously moving vehicle detector:a) Initial data for calculations obtained from radar equipment, PTZ camera, constants depending on the camera model used, constants set during commissioning::• B3: Number of frames for pixel-by-pixel summation to ensure required quality of suppression of moving object artifacts• B4: Camera mounting height, meters:• B5: Angle between the bearing to the target or patrol point and the road line, degrees• B6: Threshold target speed, below which the target will be identified by the background detector as stationary, and for the anomalously moving vehicle detector, this is the target speed measured by radar to highlight the image of this specific target for slow-moving vehicles against other vehicles in traffic:• B7: Distance to target, m:• B8: Camera matrix pixel size, pm• B9: PPM, as an image quality parameter for object recognition by neural networks, pixels / m:• B10: Estimated target size, m• B11 : Estimated projection movement of the target on the camera matrix (preferably approximately half the height of the camera matrix), pixels• B12: Camera FPS, frames / sec• B13: Number of pixels in the camera matrix horizontally, pixelsb) Calculation:• Target speed km / hB15 = B6 * 3.6:• Camera field of view at the distance to the target, mB16 = B13 / B9• Required focal length of the camera lens, mmB17 = (B7 * 1920 * 0.001 * B8) / B16:• Vco.n target speed orthogonal to bearing, m / sB18 = B6 * COS(RADIANS(ABS(B5)))• Vcp.n target speed along the bearing, m / sB19 = B6 * SIN(RADIANS(ABS(B5)))• B angle of target observation from mounting height, degreesB20 = DEGREES(ATAN(B4 I B7))• Vymatr speed of target projection movement on the camera matrix plane, m / s B21 = B18 * SIN(RADIANS(B20)):• Vco.n. m speed of target projection movement on the camera matrix plane in the transverse direction, m / sB22 = B21 * (B17 / 1000) / B7Docket: 15257367-000100 Patent Specification• Vcp.n.m speed of target projection movement on the camera matrix plane in the vertical direction, m / sB23 = B19 * (B17 / 1000) / B7• Vco.n.m speed of target projection movement on the camera matrix plane in the transverse direction, pixelsB24 = 1000000 * B22 / B8• Vcp.n.m speed of target projection movement on the camera matrix plane in the vertical direction, pixelsB25 = 1000000 * B23 / B8• T accumulation time, sT = B11 / (SQRT(B24A2 + B25A2))• R = B9 * B10, where R is the target size in pixels in the scene image

[0106] For the background detector in the stopped vehicle search algorithm, the value T calculated according to the given relations guarantees the optimal time for summing scene frames. For the abnormally moving vehicle detector, it is necessary to use neural network methods to find two identical vehicles of size R pixels in two scene images separated by a time interval T, and the value B11 in pixels, as the distance at which the found vehicle images should be from each other. The task is to identify two images of precisely those vehicles to which the velocity value B6 relates, which for the abnormal vehicle movement detector is the velocity value measured by radar.

[0107] Algorithm for filtering false accident status assignments during repeated detection of accident conditions at the same scene for an accident detector:: 1. Create Table #1 with the following columns:a. Patrol point number 0-Nb. Camera pitch angle for point Nc. Camera azimuth angle for point Nd. M - vehicle number at the scene (from zero to M) for each point Ne. X-coordinate of the bounding box center for the Mth vehicle (from 0 to M) in pixels at scene Nf. Y-coordinate of the bounding box center for the Mth vehicle (from 0 to M) in pixels at scene Ng. Time of entry for the row with parameters a-f in Table #1h. X-dimension of the object's bounding box in pixels for the Mth vehicle (from 0 to M)i. Y-dimension of the object's bounding box in pixels for the Mth vehicle (from 0 to M)The commissioning of the parking filter and old accident filter begins with operation in PATROL mode:1. Set N patrol points so that the entire road is examined within the required limits with 50% scene overlap.Docket: 15257367-000100 Patent Specification2. If a stationary vehicle is detected when pointing at a location, a row with data is created in Table #1 for each such vehicle.3. The patrol finds all vehicles parked at the time of its operation at point N.4. Other modes are not activated until the patrol has passed all points at least once (to complete commissioning faster without interference).5. In "Patrol" mode, the FD continues its work with low priority and, as possible, updates old records for scenes by completely removing all old vehicle objects for point N and replacing them with new ones if it finds anything.When acceleration magnitude exceeds the specified threshold, the following sequence of actions is performed:1. Calculate the coordinate of the point for aiming at the stopping point.2. Find the coordinate of patrol point N closest to the calculated stopping point and position the camera at this position according to 1b, 1c.3. Search for vehicles after appropriate accumulation according to patrol settings for this point N.4. Check the found vehicles for matches with existing rows for number N.5. If two or more of the vehicles found at this step previously had no match in Table #1 for the scene at point N from the center coordinates with an error of up to 20% of the bounding box size along the corresponding X or Y coordinate, then we consider this an accident with multiple vehicles and enter rows corresponding to the new vehicles in Table #1.:6. If only one vehicle was not previously in Table #1 , then we look for people, animals, objects at scene N. If people, animals, objects are found, then this is an accident with one vehicle and people, etc. around. Add an entry to Table #1 for the single vehicle for point N.7. If there is only one new vehicle and no people, then increase the size of the new vehicle's bounding box by 10% and check if the new vehicle's bounding box is in contact with the bounding box of any existing vehicle in Table #1. If YES, this is an accident with a parked vehicle. Add an entry to Table #1 for the single vehicle for point N.8. If none of the above is present, but one new stationary vehicle is detected, then this is simply a vehicle stop without qualifying as an accident. Add an entry to Table #1 for the single vehicle for point N, as for a patrol event.9. If after the braking detector is triggered there are no stationary vehicles, then this is simply a braking event.Function for updating records in Table #1. May be required if there are too many events and the points in the table have an entry time greater than the value T1. It can be run once an hour.1. Calculate the value T 1 = Sum of all operating times of all patrol points, if they were not hindered by higher priority tasks +20%.2. Check if there are any points in Table #1 whose entry time exceeds T1 from the current time.:3. If such points are found, delete them from Table #1.Filter control elements in the interface:1. In the logic menu:a. Acceleration threshold value (modulus), which will always be taken with aDocket: 15257367-000100 Patent Specificationminus sign in calculations.b. Checkbox "use parking filter" (PF description above in the text) yes / no. If no, then everything written here is collected and formed but does not affect the formation of accident / non-accident (just in case until we are sure that everything is good and to quickly switch and feel the difference).c. Checkbox "Use Patrol filter for accidents" (meaning the filter by sizes and probabilities to screen out false scene objects) yes / nod. PF recovery time (value in the range from 2 to 24 hours, and if nothing, then according to the calculated value in item 1 of the update function).2. In the menu of each zone:a. Checkbox "Apply acceleration detector" yes / no

[0108] FIG. 27 shows a screen 2700 enabled for obtaining elevation maps from various sources (selectable) and correcting the road surface elevation map if necessary. FIG. 28 shows a screen 2800 enabled for verification of the quality of radar target data matches and pan-tilt camera guidance, where the screen 2800 visualizes the maximum amount of service information from radars and the system as a whole. For example, the screen 2800 enables visualization of radar tracks.

[0109] FIG. 29 shows a screen 2900 illustrating a situation requiring a correction of an elevation map. The dispatcher observes the movement of a radar marker on a terrain map and compares the marker's position on the map with the scene image displayed in the scene frame output window. If the marker's position on the map does not correspond to the scene within an acceptable margin of error, then the dispatcher can either change the geodetic zero position of the radar in the appropriate interface or edit the terrain elevation map, depending on what cause of misalignment they suspect in the given case. For example, if the road relief has pronounced height irregularities, as shown in FIG. 29, then adjusting only the radar's position on the map will not help reduce the alignment error between the radar data on target coordinates and the position of the scene from the PTZ camera 118.

[0110] FIG. 30 shows a screenshot 3000 depicting a visible deviation of a knowingly vertical reference point from a vertical plane (e.g., about 3 degrees). FIGS. 31-32 shows an embodiment of a mathematical calculation 3100 and 3200 enabling operation of a plurality of radars and a PTZ camera according to this disclosure. In particular, there is an algorithm for calculating rotation angles for the PTZ camera 118. The communication block may perform this algorithm. Initially, GPS coordinates of the camera placementDocket: 15257367-000100 Patent Specificationlocation are obtained, two reference points in the camera's control zone, and the height of the camera suspension from the road surface. Then, the PTZ camera 118 is set to the horizon position by pitch (e.g., 90 degrees according to PTZ pitch sensors). The PTZ camera 118 is sequentially aimed at each of the two reference points that contain vertical elements. The center of the camera's field of view (matrix), which is maximally free from optical distortions, is aimed at building corners or vertical elements of engineering structures. For engineering structures, an additional check of their vertical ity is carried out during camera installation. The visible angle of inclination of the vertical element is measured for each of the two landmarks. The GPS coordinates of the two landmarks are converted to geodetic coordinates with the center at the camera placement point. Two planes are defined by the center point of intersection of the PTZ camera 118 rotation axes and the normal vectors N1 and N2 to these planes, which are calculated during the transformation of vectors to landmarks from spherical coordinates to Cartesian coordinates. By multiplying the normal vectors, the direction vector r is found, N1xN2=r, which is the axis of rotation of the PTZ camera 118 in azimuth in space starting at the point of intersection of the azimuth and pitch rotation axes of the PTZ camera 118, which is at height H of the camera suspension from the road surface. Thus, when targeting an object on the road segment 102, the height of the camera suspension should be taken into account by subtracting this value from the II axis coordinate of the target on the road in geodetic coordinates. The correction angle <|) of the azimuth angle of the PTZ camera 118 is calculated, which differs from the geodetic angle to the target by one of the available landmarks. Note that coordinate systems and transformation from spherical coordinates into Cartesian coordinates and vice versa is illustrated in FIG. 31. As such, correction of the target aiming vector is applied by recalculating the coordinates of the target point from the Earth's coordinate system ENP to the PTZ-related system E'N'U by rotating the vector r by angle t|) around the previously determined axis of rotation, using the rotation matrix Rr, c|>, where x=E'; y=N'; z=U. In the E'N'U coordinate system, the target coordinates are converted from Cartesian coordinates to spherical coordinates to obtain the required rotation angles for the PTZ camera 118. The process of equipment configuration, operation algorithms and filters are proposed, taking into account limitations to achieve the most optimal equipment operation, reducing the scanning cycleDocket: 15257367-000100 Patent Specificationtime for all control zones and providing the maximum possible reliability and speed in terms of detecting accidents and other types of events. Note that the algorithm for calculating PTZ rotation angles during calibration does not use an inverse matrix and tilt angle measurement data of the PTZ camera 118.

[0111] FIG. 33 shows an embodiment of a video feed where one vehicle is kept and other vehicle are hidden according to this disclosure. FIG. 34 shows an embodiment of a video fee where a vehicle is detected to be moving in an anomalous manner according to this disclosure. In particular, as shown in FIG. 33, there is a method 3300 for optimizing the background detector operation time. As shown in FIG. 34, there is a method for identifying abnormally moving vehicles in the scene image against the background of other vehicles and a method for the detector to detect false situations when analyzing a scene for an accident event against the background of parking or previously occurred accidents. The proposed algorithm for camera calibration and calculation of rotation angles of the PTZ camera 118 does not increase the error when calculating PTZ rotation angles by more than the error in measuring the pan angle. A visually well-controlled interface for actions with equipment is described above, allowing to avoid potentially dangerous actions of personnel on the road and the use of additional expensive measuring instruments.

[0112] Various embodiments of the present disclosure may be implemented in a data processing system suitable for storing and / or executing program code that includes at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements include, for instance, local memory employed during actual execution of the program code, bulk storage, and cache memory which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution.

[0113] I / O devices (including, but not limited to, keyboards, displays, pointing devices, DASD, tape, CDs, DVDs, thumb drives and other memory media, etc.) can be coupled to the system either directly or through intervening I / O controllers. Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices throughDocket: 15257367-000100 Patent Specificationintervening private or public networks. Modems, cable modems, and Ethernet cards are just a few of the available types of network adapters.

[0114] This disclosure may be embodied in a system, a method, and / or a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure. The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing.

[0115] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0116] Computer readable program instructions for carrying out operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmwareDocket: 15257367-000100 Patent Specificationinstructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. A code segment or machine-executable instructions may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, among others. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0117] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions. The various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computerDocket: 15257367-000100 Patent Specificationsoftware, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0118] The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustration, and combinations of blocks in the block diagrams and / or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions. Words such as “then,” “next,” etc. are not intended to limit the order of the steps; these words are simply used to guide the reader through the description of the methods. Although process flow diagrams may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or the main function.

[0119] Features or functionality described with respect to certain example embodiments may be combined and sub-combined in and / or with various other exampleDocket: 15257367-000100 Patent Specificationembodiments. Also, different aspects and / or elements of example embodiments, as disclosed herein, may be combined and sub-combined in a similar manner as well. Further, some example embodiments, whether individually and / or collectively, may be components of a larger system, wherein other procedures may take precedence over and / or otherwise modify their application. Additionally, a number of steps may be required before, after, and / or concurrently with example embodiments, as disclosed herein. Note that any and / or all methods and / or processes, at least as disclosed herein, can be at least partially performed via at least one entity or actor in any manner.

[0120] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and should not be interpreted in an idealized and / or overly formal sense unless expressly so defined herein. As used herein, the term "about" and / or "substantially" refers to a + / - 10% variation from the nominal value / term. Such variation is always included in any given.

[0121] If any disclosures are incorporated herein by reference and such disclosures conflict in part and / or in whole with the present disclosure, then to the extent of conflict, and / or broader disclosure, and / or broader definition of terms, the present disclosure controls. If such disclosures conflict in part and / or in whole with one another, then to the extent of conflict, the later-dated disclosure controls.

[0122] Although preferred embodiments have been depicted and described in detail herein, it will be apparent to those skilled in the relevant art that various modifications, additions, substitutions and the like can be made without departing from the spirit of the disclosure, and these are, therefore, considered to be within the scope of the disclosure, as defined in the following claims.

Claims

Docket: 15257367-000100 Patent SpecificationCLAIMSWhat is claimed is:

1. A method, comprising:causing a plurality of radars to be attached to a plurality of outdoor structures sequentially positioned lateral to a lane of a road segment such that (a) each radar of the plurality of radars has a radar Field of View (FOV), (b) the radar FOVs collectively monitor the road segment in its entirety without overlapping each other, and (c) the plurality of radars have a one-to-one correspondence to the plurality of outdoor structures relative to the road segment;causing a Pan-Tilt-Zoom (PTZ) camera to be attached to one outdoor structure of the plurality of outdoor structures such that (a) the PTZ camera has a camera FOV, (b) the camera FOV is capable of individually monitoring any section of the road segment, (c) the PTZ camera is positioned higher in elevation than the plurality of radars, and (d) the plurality of radars have a many-to-one correspondence to the PTZ camera relative to the road segment; andcausing a computing unit to be attached to one outdoor structure of the plurality of outdoor structures such that the computing unit is able to (a) command the PTZ camera to move the camera FOV from a first position or orientation not capable of imaging an incident on the road segment observed by one radar of the plurality of radars to a second position or orientation capable of imaging the incident responsive to the computing unit detecting the incident based on the one radar of the plurality of radars observing the incident, (b) command the PTZ camera to generate an imagery depicting the incident as the camera FOV is positioned in the second position or orientation, (c) receive the imagery from the PTZ camera positioned in the second position or orientation, (d) perform a validation of the incident while the PTZ camera positioned in the second position or orientation, (e) notify a control center of an Intelligent Transportation System (ITS) of the incident responsive to the validation being successful while the PTZ camera positioned in the second position or orientation, and (f) ignore the incident responsive to the validation being unsuccessful.Docket: 15257367-000100 Patent Specification2. The method of claim 1 , wherein each radar of the plurality of radars is a Doppler radar.

3. The method of claim 1, wherein the PTZ camera and the computing unit are attached to a common outdoor structure of the plurality of outdoor structures.

4. The method of claim 1, wherein the PTZ camera and the computing unit are attached to two different outdoor structures of the plurality of outdoor structures.

5. The method of claim 1 , wherein the plurality of radars are substantially equidistantly spaced apart from each other.

6. The method of claim 1 , wherein the plurality of radars are not substantially equidistantly spaced apart from each other.

7. The method of claim 1, wherein the incident involves an object having a distance from the one radar of the plurality of radars when the incident is observed by the one radar of the plurality of radars, wherein the object has a speed on the road segment when the incident is observed by the one radar of the plurality of radars, wherein the one radar of the plurality of radars measures the distance and the speed, wherein the computing unit determines a geographical or positional coordinate associated with the object based on the distance and the speed, wherein the computing unit commands the PTZ camera to move the camera FOV from the first position or orientation to the second position or orientation based on the geographical or positional coordinate such that the camera FOV captures the object in the second position or orientation.

8. The method of claim 7, wherein the PTZ camera is elevated at a height from a ground level, wherein the PTZ camera generates the imagery depicting the incident as the camera FOV is in the second position or orientation by capturing two images with a time period therebetween, wherein the computing unit determines the time period based on the height, the distance, and the speed.Docket: 15257367-000100 Patent Specification9. The method of claim 8, wherein the control center of the ITS includes a computing terminal having a display, wherein the control center of the ITS is notified of the incident based on the computing unit sending the two images along with a set of corresponding metadata such that the display displays the two images simultaneously.

10. The method of claim 8, wherein the time period varies based on the height, the distance, and the speed.

11. The method of claim 7, wherein the control center of the ITS includes a computing terminal having a display, wherein the display displays a digital map having a spatiotemporal annotation by color to locationally show the incident based on the geographical or positional coordinate.

12. The method of claim 11, wherein the spatiotemporal annotation by color is enabled to present the imagery over or adjacent to the digital map when the spatiotemporal annotation by color is user activated from the computing terminal.

13. The method of claim 1, wherein the control center of the ITS includes a computing terminal having a display, wherein the computing terminal is operated by a dispatcher, wherein the computing unit commands the PTZ camera to move the camera FOV from the second position or orientation to a default position or orientation based on the computing terminal not receiving a user input from the dispatcher relative to the incident within a time period.

14. The method of claim 13, wherein the computing terminal hosts an Operating System (OS) and an application running on the OS, wherein the application is programmed to operate in a plurality of modes, wherein the time period has a length that varies depending on which mode of the plurality of modes the application is operating in when the computing terminal is notified of the incident.Docket: 15257367-000100 Patent Specification15. The method of claim 1, wherein the computing unit communicates with the control center of the ITS in a wireless manner.

16. The method of claim 1, wherein the computing unit communicates with the control center of the ITS in a wired or waveguide manner.

17. The method of claim 1 , wherein the imagery is generated in a visible spectrum during daytime.

18. The method of claim 1, wherein the imagery is generated in an infrared spectrum during nighttime.

19. The method of claim 1, wherein the plurality of outdoor structures is sequentially positioned lateral to the lane of the road segment on one side of the road segment.

20. The method of claim 1, wherein the plurality of outdoor structures is sequentially positioned lateral to the lane of the road segment on both sides of the road segment.

21. The method of claim 1 , wherein the radar FOVs collectively monitor the road segment in its entirety or a shoulder thereof without overlapping each other, wherein the camera FOV is capable of individually monitoring any section of the road segment or the shoulder thereof, wherein the incident is a first incident, wherein the imagery is a first imagery, wherein the validation is a first validation, wherein the computing unit is programmed to command the PTZ camera to move the camera FOV from the first position or orientation or the second position or orientation each not capable of imaging a second incident on the road segment or the shoulder thereof observed by one radar of the plurality of radars to a third position or orientation capable of imaging the second incident responsive to the computing unit detecting the second incident based on the one radar of the plurality of radars observing the second incident, wherein the computing unit is programmed to (a) command the PTZ camera to generate a second imagery depicting the second incident as the camera FOV is positioned in the third position or orientation, (b) receive the secondDocket: 15257367-000100 Patent Specificationimagery from the PTZ camera positioned in the third position or orientation, (c) perform a second validation of the second incident while the PTZ camera positioned in the third position or orientation, (d) notify the control center of the ITS of the second incident responsive to the second validation being successful while the PTZ camera positioned in the third position or orientation, and (e) ignore the second incident responsive to the second validation being unsuccessful.

22. The method of claim 21 , wherein the second incident is on the shoulder thereof.

23. The method of claim 1 , wherein each radar of the plurality of radars is a Doppler radar, wherein the PTZ camera and the computing unit are attached to a common outdoor structure of the plurality of outdoor structures, wherein the plurality of radars are not substantially equidistantly spaced apart from each other, wherein the incident involves an object having a distance from the one radar of the plurality of radars when the incident is observed by the one radar of the plurality of radars, wherein the object has a speed on the road segment when the incident is observed by the one radar of the plurality of radars, wherein the one radar of the plurality of radars measures the distance and the speed, wherein the computing unit determines a geographical or positional coordinate associated with the object based on the distance and the speed, wherein the computing unit commands the PTZ camera to move the camera FOV from the first position or orientation to the second position or orientation based on the geographical or positional coordinate such that the camera FOV captures the object in the second position or orientation, wherein the PTZ camera is elevated at a height from a ground level, wherein the PTZ camera generates the imagery depicting the incident as the camera FOV is in the second position or orientation by capturing two images with a time period therebetween, wherein the computing unit determines the time period based on the height, the distance, and the speed, wherein the control center of the ITS includes a computing terminal having a display, wherein the control center of the ITS is notified of the incident based on the computing unit sending the two images along with a set of corresponding metadata such that the display displays the two images simultaneously, wherein the time period varies based on the height, the distance, and the speed, wherein the display displays a digitalDocket: 15257367-000100 Patent Specificationmap having a spatiotemporal annotation by color to locationally show the incident based on the geographical or positional coordinate, wherein the spatiotemporal annotation by color is enabled to present the imagery over or adjacent to the digital map when the spatiotemporal annotation by color is user activated from the computing terminal, wherein the computing terminal is operated by a dispatcher, wherein the computing unit commands the PTZ camera to move the camera FOV from the second position or orientation to a default position or orientation based on the computing terminal not receiving a user input from the dispatcher relative to the incident within a time period.

24. The method of claim 1 , wherein one radar of the plurality of radars, the PTZ camera, and the computing unit are attached to a common outdoor structure of the plurality of outdoor structures.

25. The method of claim 1, wherein the PTZ camera is calibrated using an algorithm to calculate a movement angle thereof without using an inverse matrix and an tilt angle measurement data associated with the PTZ camera 118.

26. The method of claim 1 , wherein the PTZ camera and the computing unit are embodied as separate and distinct physical devices.

27. The method of claim 1 , wherein the PTZ camera and the computing unit are embodied as one physical device.

28. A system, comprising:a plurality of radars attached to a plurality of outdoor structures sequentially positioned lateral to a lane of a road segment such that (a) each radar of the plurality of radars has a radar Field of View (FOV), (b) the radar FOVs collectively monitor the road segment in its entirety without overlapping each other, and (c) the plurality of radars have a one-to-one correspondence to the plurality of outdoor structures relative to the road segment;Docket: 15257367-000100 Patent Specificationa Pan-Tilt-Zoom (PTZ) camera attached to one outdoor structure of the plurality of outdoor structures such that (a) the PTZ camera has a camera FOV, (b) the camera FOV is capable of individually monitoring any section of the road segment, (c) the PTZ camera is positioned higher in elevation than the plurality of radars, and (d) the plurality of radars have a many-to-one correspondence to the PTZ camera relative to the road segment; anda computing unit attached to one outdoor structure of the plurality of outdoor structures, wherein the computing unit is programmed to (a) command the PTZ camera to move the camera FOV from a first position or orientation not capable of imaging an incident on the road segment observed by one radar of the plurality of radars to a second position or orientation capable of imaging the incident responsive to the computing unit detecting the incident based on the one radar of the plurality of radars observing the incident, (b) command the PTZ camera to generate an imagery depicting the incident as the camera FOV is positioned in the second position or orientation, (c) receive the imagery from the PTZ camera positioned in the second position or orientation, (d) perform a validation of the incident while the PTZ camera positioned in the second position or orientation, (e) notify a control center of an Intelligent Transportation System (ITS) of the incident responsive to the validation being successful while the PTZ camera positioned in the second position or orientation, and (f) ignore the incident responsive to the validation being unsuccessful.

29. A method, comprising:causing a plurality of radars to be attached to a plurality of outdoor structures sequentially positioned lateral to a lane of a road segment such that (a) each radar of the plurality of radars has a radar Field of View (FOV), (b) the radar FOVs collectively monitor the road segment in its entirety without overlapping each other, and (c) the plurality of radars have a one-to-one correspondence to the plurality of outdoor structures relative to the road segment;causing a Pan-Tilt-Zoom (PTZ) camera to be attached to one outdoor structure of the plurality of outdoor structures such that (a) the PTZ camera has a camera FOV, (b) the camera FOV is capable of individually monitoring any section of the road segment,Docket: 15257367-000100 Patent Specification(c) the PTZ camera is positioned higher in elevation than the plurality of radars, and (d) the plurality of radars have a many-to-one correspondence to the PTZ camera relative to the road segment; andcausing the PTZ camera to (a) move the camera FOV from a first position or orientation not capable of imaging an incident on the road segment observed by one radar of the plurality of radars to a second position or orientation capable of imaging the incident responsive to the PTZ camera detecting the incident based on the one radar of the plurality of radars observing the incident, (b) generate an imagery depicting the incident as the camera FOV is positioned in the second position or orientation, (c) perform a validation of the incident while the PTZ camera positioned in the second position or orientation, (d) notify a control center of an Intelligent Transportation System (ITS) of the incident responsive to the validation being successful while the PTZ camera positioned in the second position or orientation, and (e) ignore the incident responsive to the validation being unsuccessful.

30. A system, comprising:a plurality of radars attached to a plurality of outdoor structures sequentially positioned lateral to a lane of a road segment such that (a) each radar of the plurality of radars has a radar Field of View (FOV), (b) the radar FOVs collectively monitor the road segment in its entirety without overlapping each other, and (c) the plurality of radars have a one-to-one correspondence to the plurality of outdoor structures relative to the road segment; anda Pan-Tilt-Zoom (PTZ) camera attached to one outdoor structure of the plurality of outdoor structures such that (a) the PTZ camera has a camera FOV, (b) the camera FOV is capable of individually monitoring any section of the road segment, (c) the PTZ camera is positioned higher in elevation than the plurality of radars, and (d) the plurality of radars have a many-to-one correspondence to the PTZ camera relative to the road segment, wherein the PTZ camera is programmed to (a) move the camera FOV from a first position or orientation not capable of imaging an incident on the road segment observed by one radar of the plurality of radars to a second position or orientation capable of imaging the incident responsive to the PTZ camera detecting the incident based on the one radar ofDocket: 15257367-000100 Patent Specificationthe plurality of radars observing the incident, (b) command the PTZ camera to generate an imagery depicting the incident as the camera FOV is positioned in the second position or orientation, (c) perform a validation of the incident while the PTZ camera positioned in the second position or orientation, (d) notify a control center of an Intelligent Transportation System (ITS) of the incident responsive to the validation being successful while the PTZ camera positioned in the second position or orientation, and (e) ignore the incident responsive to the validation being unsuccessful.

Images