Automated Virtual Tripwire Placement

Abstract

One or more computing devices implement a method for placing a virtual trip wire on grid points of a geographic coordinate grid. A processor of the device controls a camera to capture a series of images within the camera's field of view. The processor identifies an object of interest in the images and determines the object's movement through the images. The processor identifies the object's direction of movement and determines a variance in the direction of movement. The processor also determines a median direction of movement vector for a vector point on the grid corresponding to the object of interest. The processor then determines multiple candidate locations for placing a trip wire across the grid points in a direction approximately perpendicular to the direction of movement of at least one object of interest, and automatically selects one candidate location based on the median direction of movement vector and the variance in the direction of movement.

Description

[Technical field]

[0001] The present invention relates generally to airborne mounted nodes positioned above roadways for video collection and analysis, and more particularly, but not exclusively, to airborne mounted nodes that analyze video and automatically determine suitable locations for virtual tripwires. [Background technology]

[0002] Towns, cities, and various other municipalities often desire to analyze traffic flow along various roads. For example, traffic flow can be analyzed to determine which roads carry relatively more automobiles in a given time period. Such traffic flow information may be used to determine which roads to widen or narrow to accommodate bicycle lanes, for example. Such information may be used to determine where to add, remove, or modify warning signals (e.g., additional traffic lights, audible warnings, etc.), crosswalks, signs, and other useful urban appurtenances.

[0003] In some traffic monitoring systems, video may be continuously recorded, photographed, or monitored along a portion of a roadway. The captured video or images may be processed to determine the number of vehicles traveling along the roadway, or other information about the flow of traffic, including times of relatively heavy traffic, times of relatively light traffic, or other characteristics.

[0004] In one conventional example, for example, a particular portion of a road may be manually selected by a traffic engineer to monitor traffic passing through that portion of the road. In another particular example, a traffic engineer may manually select a particular tripwire or line that extends in a direction perpendicular to the direction of traffic movement along the road, in which case traffic crossing that line may be monitored. Summary of the Invention [Problem to be solved by the invention]

[0005] Not all subject matter discussed in the "Background Art" is necessarily prior art, and it should not be assumed that it is prior art merely because of a discussion in the "Background Art." Also, prior art discussed in the "Background Art" or awareness of problems related to such subject matter should not be treated as prior art unless expressly stated to be prior art. Rather, any discussion of subject matter discussed in the "Background Art" should be treated as part of the inventor's approach to a particular problem, which may itself also be inventive.

[0006] The following is a summary of the invention to provide an initial understanding of some features and context. This summary is not intended to identify key or critical elements of the invention or to delineate the scope of the invention. This summary is intended to present certain concepts of the invention in a simplified form as a prelude to the more detailed description that is presented later. [Means for solving the problem]

[0007] According to one aspect of an exemplary embodiment, a method performed by one or more computing devices may determine a location for placing a virtual trip wire. A series of images of at least a portion of a roadway within a field of view of the camera may be acquired or photographed from the camera. A grid point or set of grid points may correspond to a pixel or group of pixels in each image of the series of images. The series of images may be processed to identify a target object. A movement of the target object through the series of images may be determined. A direction of movement of one or more points of each target object through the set of grid points may be identified. A variation in the direction of movement of each of the target objects may be determined. A vector of median directions of movement for each of a plurality of vector points corresponding to the target object in the set of grid points may be determined. A plurality of candidate locations for placing a virtual trip wire on the set of grid points for monitoring the movement of at least a portion of the target object through the series of images may be determined. The virtual trip wire may include a virtual segment extending in a direction approximately perpendicular to the direction of movement along at least a portion of the roadway. A particular candidate location for placing the virtual trip wire may be automatically selected. Here, the particular candidate location may be automatically selected from a plurality of candidate locations based on the plurality of vectors and associated variations in the direction of movement. Automatically selecting a particular candidate location for placing the virtual tripwire may include moving a sliding window across the grid points to identify a location that results in a minimum variation in movement direction at the grid points that overlap the sliding window. In some cases, the placement of the virtual tripwire within the sliding window for the particular candidate location may be based on determining a placement within the sliding window that results in a minimum variation in movement direction at the grid points. The images may be acquired by sampling a video of at least a portion of the path. The target objects may include one or more cars, one or more trucks, one or more bicycles, or one or more pedestrians.In some cases, a speed of at least one target object crossing the virtual tripwire may be estimated for a predetermined time interval based on respective positions of pixels corresponding to at least one of the target objects in successive images. In some cases, a count of the number of vehicles crossing the virtual tripwire for a predetermined time interval may be determined. In some cases, geographic coordinates for a boundary perimeter for each target object may be determined.

[0008] According to another aspect of the exemplary embodiment, the system may include an airborne node having a video camera and a first processor. The first processor may be positioned or operable to acquire a series of images of at least a portion of a roadway, where a grid point or set of grid points corresponds to a pixel or group of pixels in each image of the series of images. The first processor may also be positioned or operable to process the images of the series of images to identify a target object. The first processor may also be positioned or operable to determine a respective boundary perimeter of each target object in at least one image of the series of images, where the respective boundary perimeter includes a geometric object surrounding the respective target object. The first processor may also be positioned or operable to determine, for each target object, geographic coordinates representing at least one point on or within the boundary perimeter. The first processor may also be positioned or operable to transmit a message including the determined geographic coordinates.

[0009] The system may further include a remote computing device having a second processor. The second processor may be positioned or operable to receive at least a portion of the messages transmitted by the airborne node. The second processor may be positioned or operable to determine movement of the object through the series of images. The second processor may be positioned or operable to identify a direction of movement of one or more points of each object through the set of grid points. The second processor may be positioned or operable to determine, for each object, a variation in the direction of movement. The second processor may be positioned or operable to determine a median vector of the direction of movement for each of a plurality of vector points corresponding to the object on the set of grid points. The second processor may be positioned or operable to determine a plurality of candidate locations for placing a virtual tripwire on the set of grid points for monitoring movement of at least a portion of the object through the series of images, where the virtual tripwire includes a virtual segment extending in a direction substantially perpendicular to the direction of movement along at least a portion of the road. The second processor may also be positioned or operable to automatically select a particular candidate location for placing the virtual tripwire, where the particular candidate location is selected from the plurality of candidate locations based on the plurality of vectors and associated variations in movement direction. The remote computing device may also be positioned or operable to move a sliding window across a set of grid points to identify a location that results in a minimization of the variation in movement direction at the grid points that overlap the sliding window. The remote computing device may further be positioned or operable to identify a placement of the virtual tripwire within the sliding window for the particular candidate location based on determining a placement within the sliding window that results in a minimization of the variation in movement direction at the grid points.

[0010] Additionally, the remote computing device may be positioned or operable to determine, for each target object, geographic coordinates for a boundary perimeter. The specific candidate locations may include candidates associated with a determined low variation in direction of movement. The remote computing device may be positioned or operable to estimate a speed of at least one of the target objects crossing the virtual tripwire in a predetermined time interval based on respective positions of pixels corresponding to at least one of the target objects in the successive images. The remote computing device may be positioned or operable to count the number of vehicles crossing the virtual tripwire in a predetermined time interval.

[0011] According to a further aspect of the exemplary embodiment, the article includes a non-transitory storage medium including machine-readable instructions executable by one or more processors to process messages received from one or more airborne nodes, where at least one message includes geographic coordinates representing at least one point on or within a boundary perimeter of each of a plurality of objects in at least one image of a series of images of at least a portion of a roadway acquired by the one or more airborne nodes. The above also allows for determining movement of the objects through the series of images, identifying a direction of movement of one or more points of each object through a set of grid points, determining a variation in the direction of movement for each object, determining a vector of median direction of movement for each of a plurality of vector points corresponding to the object on the set of grid points, and determining a plurality of candidate locations for placing a virtual tripwire on the set of grid points for movement of at least a portion of the objects through the series of images, where the virtual tripwire includes a virtual segment extending in a direction substantially perpendicular to the direction of movement along at least a portion of the roadway. The above also allows for automatic selection of a particular candidate location for placing the virtual tripwire. Here, a particular candidate location is selected from the plurality of candidate locations based on the plurality of vectors and associated variations in the direction of movement.

[0012] Further, the machine-readable instructions may be executable by the one or more processors to automatically select a particular candidate location for placing the virtual tripwire, such as moving a sliding window across the grid points to identify a location that results in a minimum of variation in the direction of movement at the grid points that overlap the sliding window. Further, the machine-readable instructions may be executable by the one or more processors to identify a location of the virtual tripwire within the sliding window for the particular candidate location based on determining a location within the sliding window that results in a minimum of variation in the direction of movement at the grid points. The particular candidate location may include a candidate associated with a determined low variation in the direction of movement. Further, the machine-readable instructions may be executable by the one or more processors to estimate a speed of at least one target object crossing the virtual tripwire at a predetermined time interval based on respective locations of pixels corresponding to at least one of the target objects in the successive images.

[0013] According to one aspect of an exemplary embodiment, a method executed by one or more processors may determine a location for placing a virtual trip wire. A series of images of at least a portion of a roadway within a field of view of a camera may be acquired or photographed. A set of grid points may correspond to a pixel or group of pixels in each image of the series of images. The series of images may be processed to identify a target object. Movement of the target object through the series of images may be determined. A direction of movement of one or more points of each target object may be identified through the set of grid points. A variation in the direction of movement may be determined for each grid point. A plurality of vectors of median directions of movement may be determined for a plurality of vector points corresponding to the set of grid points. A plurality of candidate locations may be determined for placing a virtual trip wire on the set of grid points for monitoring the movement of one or more target objects through the series of images. The virtual trip wire may include a segment extending in a direction approximately perpendicular to the direction of movement along at least a portion of the roadway. A particular candidate location may be selected from the plurality of candidate locations based on the plurality of vectors and associated variation in the direction of movement for placement of the virtual trip wire perpendicular to the direction of movement intersecting the grid point. An alert may be generated based on the selection of a particular candidate location for placing the virtual tripwire.

[0014] According to an aspect of another exemplary embodiment, a system may be provided that can determine a location for placing a virtual trip wire. The system may include one or more video capture devices comprising at least a video camera and at least a first processor to capture a sequence of images of at least a portion of a roadway. The one or more video capture devices may process the images to identify objects of interest. The one or more video capture devices may also determine a respective boundary perimeter for each object of interest in at least one image of the sequence of images. Furthermore, the one or more video capture devices may transmit a message including geographic coordinates for each boundary perimeter of the object of interest in the at least one image.

[0015] The system may also include a network device having at least a second processor for receiving messages from the one or more video capture devices. The network device may also determine motion of the target object through the sequence of images and identify a direction of motion of one or more points of each target object through the set of grid points. The network device may also determine a variation in the direction of motion for the grid points and determine a plurality of vectors of median directions of motion for a plurality of vector points corresponding to the set of grid points. The network device may further determine a plurality of candidate locations for placing a virtual tripwire on the set of grid points for monitoring the motion of the one or more target objects through the sequence of images. The network device may also select a plurality of candidate locations for placement of a virtual tripwire perpendicular to the direction of motion intersecting the grid points based on the plurality of vectors and associated variations in the direction of motion. The network device may also generate an alert based on the selection of a particular candidate location for placing the virtual tripwire.

[0016] According to an aspect of another exemplary embodiment, an article may comprise a non-transitory storage medium including machine-readable instructions executable by one or more processors. The instructions may be executable by the one or more processors to determine a location to place a virtual trip wire. A series of images of at least a portion of a roadway within a field of view of a camera may be acquired or obtained. A set of grid points may correspond to a pixel or group of pixels of each image in the series of images. The series of images may be processed to identify a target object. A movement of the target object through the series of images may be determined. A direction of movement of one or more points of each target object may be identified through the set of grid points. A variation in the direction of movement may be determined for each grid point. A plurality of vectors of median directions of movement may be determined for a plurality of vector points corresponding to the set of grid points. A plurality of candidate locations may be determined for placing a virtual trip wire on the set of grid points for monitoring the movement of one or more target objects through the series of images. A particular candidate location may be selected from the plurality of candidate locations based on the plurality of vectors and associated variation in the direction of movement for placement of a virtual trip wire perpendicular to the direction of movement intersecting the grid point.

[0017] Other features and aspects will become apparent from the following detailed description, taken in conjunction with the accompanying drawings and claims.

[0018] The above Summary is provided to introduce certain concepts in a simplified form that are further described below in the Detailed Description. The above Summary is not intended to limit the claimed subject matter, but rather the content of the claims themselves will determine the scope of that subject matter. [Brief description of the drawings]

[0019] Non-limiting and non-exhaustive embodiments are described with reference to the accompanying drawings. Similar reference numerals refer to similar parts throughout the various drawings unless otherwise specified. Sizes and relative positions of elements in the drawings are not necessarily drawn to scale. For example, shapes of various elements are selected, enlarged, and positioned to improve the readability of the drawings. Particular shapes of elements are selected to facilitate recognition in the drawings. One or more embodiments are described below with reference to the accompanying drawings. [Figure 1] FIG. 1 illustrates an embodiment of a street light assembly having an airborne node positioned to detect near miss conditions. [Diagram 2] FIG. 1 illustrates an embodiment of a system for monitoring traffic along a road. [Diagram 3] FIG. 1 illustrates an embodiment of an image taken from a video acquired by an airborne node mounted on a streetlight assembly. [Figure 4A] 2 is an image showing the movement of a target object along a road; [Figure 4B] 2 is an image showing the movement of a target object along a road; [Figure 4C] 2 is an image showing the movement of a target object along a road; [Figure 4D] FIG. 5 illustrates a set of information-bearing grid vectors on a generated image of a road that represent the location of the approximate center point of the object of interest shown in FIGS. 4A-4C. [Diagram 5] FIG. 13 shows another set of information-bearing grid vectors on a generated image of another road, representing the variations associated with the movement directions of various objects. [Figure 6] FIG. 6 is a diagram of the set of information-bearing grid vectors shown in FIG. 5 and the automatically generated and placed virtual tripwires. [Figure 7] FIG. 1 illustrates an embodiment of an airborne node arranged to automatically generate and place virtual tripwires. [Figure 8] FIG. 1 illustrates an embodiment of a remote computing device. [Figure 9] FIG. 1 illustrates an embodiment of a system for an airborne node to communicate with a remote computing server. [Figure 10] FIG. 1 is a data flow diagram illustrating a process for locating a virtual tripwire on a set of grid points corresponding to a particular video camera's field of view. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] The present invention will be more readily understood with reference to the specification and the accompanying drawings. The terms used in this specification are provided for the purpose of describing certain embodiments, and are not intended to limit the scope of the claims unless a court or competent authority determines that such terms are limiting. Unless otherwise defined in this specification, the terms used in this specification are deemed to have their conventional meanings as known in the relevant art.

[0021] In the following description, specific details are set forth to provide a thorough understanding of various disclosed embodiments. However, one of ordinary skill in the art will recognize that the embodiments can be practiced without one or more of these specific details, or can be practiced with other methods, components, or materials, etc. In other instances, well-known structural details related to computing systems, including client and server computing systems and networks, are omitted to avoid unnecessarily obscuring more detailed descriptions of the embodiments.

[0022] Embodiments of the devices, methods, and systems described herein (i.e., the teachings herein) implement an airborne node (e.g., a node mounted on a streetlight above a roadway) that focuses substantially downward, captures video, and performs or allows automated analysis of the virtual tripwires.

[0023] Conventionally, a traffic engineer may manually select a particular tripwire or line that extends in a direction substantially normal (i.e., perpendicular) to the direction of traffic movement along a roadway to monitor traffic crossing the tripwire line. The engineer may take this action to monitor traffic flow at a particular time of day, at a particular location, or for other reasons. Data collected from such an analysis may lead to the addition or removal of traffic lanes (e.g., right / left turn lanes, "straight through" lanes, bicycle lanes, bus lanes, etc.), changes to traffic light timing or patterns, the installation or modification of pedestrian crossings, or other traffic-related information. However, the inventors have found that manually selecting such tripwires is often arbitrary and not representative of the optimal portions of a roadway for monitoring traffic.

[0024] In certain situations, such as when determining the average speed on a roadway, the number of vehicles turning versus not turning, the average distance or time between vehicles, etc., a trip wire located in a first selected area of ​​the roadway (e.g., location "A") may be better positioned for traffic monitoring than a trip wire located in another, second selected area of ​​the roadway. In certain cases, the objective is to determine the average speed or velocity of vehicles along a roadway. Here, it may be preferable to install the trip wire in a location where the vehicle driver is likely to change speed, e.g., relatively far from locations where the vehicles normally travel at a constant speed, such as intersections and / or traffic lights. In other cases, the objective is to determine the number of vehicles turning in a particular direction at an intersection. In this case, it may be optimal to place the trip wire approximately 20 feet from the intersection, rather than within 10 feet or more than 30 feet.

[0025] One or more embodiments described herein generally provide for automated selection and / or placement of virtual tripwires for monitoring traffic along roads used for the movement of people and vehicles (e.g., automobiles, bicycles, pedestrians, and other moving objects and / or groups). In the field of video analytics, virtual tripwires may be used to monitor traffic volume, density, speed, direction, or other characteristics of moving objects, including people and vehicles. For example, an automated process may determine where to place a virtual tripwire within the field of view of a given camera based on certain criteria, sometimes with no human intervention and in other cases with only minimal human intervention (e.g., identifying the road, identifying the type of data to collect, and other management information).

[0026] A "virtual tripwire," as used herein, refers to a processor-generated virtual boundary within an image or sequence of images. A virtual tripwire represents one or more lines or segments between two or more points on an image, sequence of images, set of lattice points (e.g., a "grid" or grid of lattice points), or other collection of data. For example, if a camera is configured to take multiple images (e.g., multiple images taken based on a schedule, motion, or another criteria) of the same location (e.g., a portion of a road within the camera's field of view) as a streaming video of unique images, either continuously or intermittently (e.g., based on a periodicity), a virtual tripwire may represent a line extending between two pixels in the image or in the corresponding geographic coordinates in the real world that correspond to those pixels.

[0027] The virtual tripwire may be utilized to directly or indirectly initiate an action by a computing device. With respect to traffic monitoring, for example, the virtual tripwire may be utilized to determine when to count vehicles or to monitor the performance of a vehicle action based on vehicles crossing a virtual tripwire threshold.

[0028] In embodiments of the present invention, a series of images of a vehicle roadway may be analyzed to automatically (i.e., without human intervention) determine an optimal location for placing a virtual tripwire. In some embodiments, a single virtual tripwire may be placed within the field of view of a particular camera corresponding to images captured by a single source camera. However, those skilled in the art will also appreciate that in some embodiments, more than one virtual tripwire may be utilized, more than one camera may be utilized, and the process of automatically determining optimal locations for any number of virtual tripwires may be performed locally (e.g., by a streetlight-based computing device), or remotely, or individually by a single computing device, or collectively by a set of computing devices working together, or in any other suitable manner. The virtual tripwire may be utilized to directly or indirectly cause or effectuate, for example, count the number of vehicles traveling on a road during a particular time interval, determine the average speed or velocity of vehicles, count the number of vehicles turning in a particular direction at an intersection, determine or trigger an analysis of congestion on a particular portion of a road, determine vehicle following distances (such as average following distance, minimum following distance, maximum following distance, and sets of all or some of these), determine or trigger an analysis of the effects of various traffic light timing patterns, determine or trigger an analysis of the effects of pedestrians, count the number of large commercial vehicles, count the number of passenger cars, count the number of public transportation vehicles and trigger an analysis of their schedules, count bicycle traffic, count the number of pedestrians, and many other events.

[0029] In one embodiment, one or more still images or videos are taken or captured. The images or videos are captured by one or more cameras, each having one or more fields of view, along the roadway of the vehicle. The captured video may include two-dimensional video, three-dimensional video, infrared or enhanced video, or other forms of video / image streams using techniques known or learned by those skilled in the art. The video may be sampled at various intervals to obtain or capture a series of images. In some cases, for example, the video may be sampled 30 times per second, or at different sampling rates for different implementations. A sampling rate may be utilized based at least in part on available processing power or capacity for analyzing the sampled images, and in at least some cases, the sampling rate may be dynamically adjustable. For example, a streetlight-based node may have one or more cameras along with one or more processors capable of capturing and processing high resolution video or images at 60 frames per second or more. Alternatively, such a streetlight-based node may only capture and process video or images at 15 frames per second or less.

[0030] In one embodiment, the mapping between pixels of the captured image and real-world geographic coordinates, such as latitude / longitude coordinates, may be known in advance or may be determined for each sampled image. For example, if a video camera is configured or positioned to capture video with a field of view that is substantially fixed or pointed in a particular direction, with the camera located in a relatively static position, a mapping may be known between various anchor points in the sampled images of the video and a set of real-world geographic coordinates and / or grid points. For example, roads that include manhole covers, pedestrian crossings, curbs, street lights, or other specific anchor points (e.g., objects, points of interest, markings, relationships between objects, etc.) may utilize such anchor points or points as references for mapping pixels of the sampled images to real-world geographic coordinates. In at least some cases, the airborne node includes positioning circuitry (e.g., Global Positioning System (GPS)) that provides the node's position (e.g., longitude, latitude, altitude, etc.) in a geographic coordinate system, and the node's position may be used to mathematically determine, within acceptable accuracy, the real-world geographic coordinates within the field of view of the node's camera.

[0031] Any suitable number of sampled images may also be processed to identify one or more moving or stationary objects along a road. For example, if a truck is traveling along a road, the truck may be an object located at different positions in the sampled images of the road at different times as the truck travels. If the truck is traveling at a relatively high speed along a relatively straight path, the truck may move quickly from one side of the series of sampled images to the other side of the series of sampled images. On the other hand, if the truck is not moving, such as when stopped at a traffic light or traffic jam, the truck may be in approximately the same position in successive images of the series of captured images. Along these lines, an analysis may possibly be performed to predict the subsequent position of the truck, either alone or with reference to other objects in the scene being analyzed. Any or all of the speed of the truck (e.g., true ground speed, or speed relative to other vehicles, etc.), the trajectory of the truck, the consistency or inconsistency of the truck's movements, or other factors may be considered in the analysis.

[0032] Processing the one or more sampled images may include generating respective boundary perimeters for specific target objects in at least one of the images. The target objects in the images may include one or more identified vehicles (e.g., cars, trucks, bicycles, etc.), pedestrians, animals, fire hydrants, control signals, utility boxes, access panels, manhole covers, curbs, storefronts, or any other target objects. In some cases, the boundary perimeter refers to a virtual object having a geometric shape that encloses the target object in the geometric plane of the image. For example, if the image includes a two-dimensional image, the boundary perimeter may include an object (e.g., a box, rectangle, or other enclosing object) that encloses the target object. In one embodiment, the boundary perimeter may include an object having a geometric shape that completely encases or surrounds the target object.

[0033] According to one or more embodiments, the video cameras may be located in, on, or associated with airborne nodes located on or around various street lights. As used herein, an airborne node may be a "video processing device," "video processing module," or other information capture module that comprises a computing device having at least one video camera. In one particular embodiment, the airborne node includes a housing (e.g., a package or other containment structure) that is fixedly or removably attached to the street light. The housing is positioned such that the video camera can capture video of the road (i.e., data representing still or streaming images that exist independently or as part of a full or partial stream of images). In one example, the video camera is positioned to capture a stream of video, and the computing device is positioned to sample image data of the video stream at a periodic rate. The computing device, alone or in combination with other computing devices located remotely, may implement a neural network to process a sequence of captured video images. Such processing may be implemented to identify a target object and determine a danger area for the object.

[0034] According to certain embodiments, one or more computing devices of or associated with the airborne node may implement a neural network, such as a deep learning neural network. For example, such a neural network may be trained via supervised learning, unsupervised learning, and / or reinforcement learning. According to one embodiment, the neural network described herein may include, for example, a convolutional neural network (CNN) or a long short-term memory (LSTM) neural network.

[0035] One or more processes executed by a computing device of the airborne node or one or more processes executed by a computing device associated with the airborne node may identify geographic coordinates, such as latitude and longitude or other coordinates, that correspond to different pixel locations in the captured images. Geographic coordinates of a boundary perimeter location may be identified for the target object. For each target object (e.g., a boundary perimeter around the target object) in the series of one or more images, an approximate leading point, center point, centroid, or other such designator may be determined. In an exemplary embodiment, if the boundary perimeter has a shape such as a rectangle that encircles the target object, coordinates of the edge of the boundary perimeter may be identified. The coordinates of the boundary perimeter may be utilized to determine an approximate leading edge, center, or other point of interest of the object, such as a point located approximately in the middle of the area defined by the boundary perimeter. If a vehicle or other target object is moving through the series of captured images, the approximate target point of the object will also be moving through the series of captured images. In at least some exemplary embodiments, the location of the target point corresponding to the approximate center point of the target object through the series of captured images may be automatically utilized to determine whether and / or where to place a virtual tripwire.

[0036] In some embodiments, after a sufficient number of images have been processed (e.g., one, tens, hundreds, thousands, tens of thousands, or other number of images that, when processed, show an accurate determination at 50%, 75%, 90%, 98%, or other accuracy measures), the movement of points corresponding to approximate target points of particular target objects through the images may be utilized to determine particular characteristics of each target object. For example, an approximate direction of movement of the target object through the series of images may be determined, an estimate of an approximate speed or velocity of the target object through the series of images may be determined, a degree of consistency or stability of the target object's movement may be determined, or other characteristics may be determined.

[0037] In an exemplary embodiment, each captured image may be associated with a timestamp corresponding to the time zone in which the image was captured or other time base (e.g., Coordinated Universal Time (UTC), Global Positioning System time, etc.). If approximate object points of an object (e.g., a vehicle) traveling along a road are determined in a series of successively captured images, each of which is associated with a timestamp, then the speed or velocity of the object (e.g., a vehicle) can be estimated by determining the distance between the approximate object points in the images and estimating the distance traveled by the object at a particular time threshold given the time difference between the timestamps for the images.

[0038] In some implementations, the geographic coordinates of the corners of the boundary perimeter may be sufficient to identify the boundary perimeter. For example, a video camera of the airborne node or another computing device may take images, and one or more processes of the same may process these images locally to identify the target object and the corresponding boundary perimeter. The one or more processes may generate metadata or other information for the target object that identifies the geographic coordinates corresponding to the boundary perimeter, and the processes may transmit the metadata to a local or remote computing server for subsequent processing. In some implementations, both the metadata and the corresponding images may be transmitted to the computing server. In some implementations where processing and / or transmission bandwidth is limited, for example, only the metadata may be processed by or transmitted to the computing server. The computing server may receive the metadata and may identify the target object's boundary parameters, motion, speed, and other characteristics and conditions.

[0039] In some cases, if the boundary perimeter of the target object has a rectangular shape, the boundary perimeter may be defined by determining the geographic coordinates of four corners of the boundary perimeter, or by determining the geographic coordinates of two of the four corners. Such geographic coordinates may be transmitted from a transmitter of the airborne node or other computing device to a network, such as a cloud-based network, for further processing. In some implementations, some or all of the approximate target points of the boundary perimeter may also be determined by one or more processes, and the geographic coordinates for the approximate target points of the target object may be transmitted to the network for further processing. In another implementation, a remote computing server or other network device may identify the approximate target points of the target object in a particular image, for example, based on the geographic coordinates received from a transmitter of the airborne node or other computing device.

[0040] In implementations in which both the metadata and the corresponding image are transmitted to a local or remote computing server, the metadata may include information describing or characterizing the captured image and / or the boundary perimeter within the captured image. According to some implementations, the metadata may include the timing at which a particular image was captured, location information, an airborne node identifier from which the image data was captured, or other suitable metadata. In some implementations where processing and / or transmission bandwidth is limited, for example, only the metadata may be transmitted to the local or remote computing server. The computing server may receive the metadata and may determine an optimal location for placing one or more virtual tripwires for a particular image.

[0041] After determining the optimal locations of one or more virtual tripwires, information corresponding to the locations of such virtual tripwires may be stored in a local or remote repository (e.g., a memory device) or utilized in future processing of images received from the airborne node's computing device or other related images. The captured images may include information corresponding to the virtual tripwires. In some embodiments, information such as metadata regarding the locations of such virtual tripwires may be transmitted from a remote computing server or other such network device to the airborne node or other device from which the captured images were obtained.

[0042] After the location of the virtual tripwire has been identified or determined, the virtual tripwire may be used, for example, directly or indirectly to cause certain traffic-related analyses to be performed on the captured imagery that is subsequently processed. In one particular embodiment, the virtual tripwire may be used as a threshold where, if an object of interest crosses the plane, one or more processes may be executed or directed to cause the traffic volume, density, speed, direction of movement, or other such characteristics of the selected object of interest to be measured.

[0043] In some embodiments, a report, such as a spreadsheet or other document, may be generated in a user-friendly manner to identify an analysis of the traffic. For example, the report may be utilized to determine locations along roads that may be unsafe (e.g., have an unacceptably high incidence of collisions, near misses, interactions with pedestrians or animals, etc.). This may allow a decision to be made as to whether to lower speed limits, add traffic signs such as traffic lights or stop signs, or take other action.

[0044] In some implementations, an alert may be generated, for example, based on an analysis of the monitored traffic. For example, if a relatively large number of vehicles (e.g., tens, hundreds, or more) are monitored or a relatively slow speed of the vehicles (e.g., less than 2 miles per hour (mph), less than 10 mph, less than 20 mph, or some other speed measurement) is determined, an alert may be generated to notify a traffic engineer that the timing of traffic signals switched between green, yellow, and red should be adjusted. Such an alert may include, for example, generating a short message service (SMS), text message, electronic mail (email) to the traffic engineer, or sounding an audible alarm.

[0045] FIG. 1 illustrates an embodiment of a street light assembly 100 having an airborne mounted node 102. The street light assembly 100 may include a light pole 104 of any suitable material, dimensions, or other characteristics. The light pole 104 may have luminaires 106 and an airborne mounted node 102 permanently or removably coupled or attached thereto. The street light assembly 100 may be positioned along a roadway such that light emitted from the luminaires 106 illuminates at least a portion of the roadway (i.e., the street light assembly is positioned "above" the roadway). The positioning of the street light assembly 100 above the roadway is selected to improve visibility and safety for vehicle drivers, bicyclists, pedestrians, and anyone who may cross, cross, or cross the roadway. In at least some cases, the airborne mounted node 102 is positioned on the street light assembly 100 specifically in proximity to a pedestrian crosswalk. To avoid confusion, along the lines of airborne assembly 102 is airborne assembly 102a, which may be referred to herein, individually or collectively, as airborne assembly 102.

[0046] In one particular embodiment, the street light assembly 100 may have a height of more than 20 feet. For example, the airborne node 102 may be mounted 20 feet above street level. In such an embodiment, the extent of the video of the road may be several times (e.g., 2x, 4x, 8x, or other multiple) the height at which the video is acquired. Thus, in at least one embodiment, if the video is taken from a height of 20 feet, an area of ​​80 feet in width, radius, diameter, or other such dimension may be observed or monitored via the video taken by the airborne node 102. In the embodiment shown in FIG. 1, the airborne node 102 may obtain power from the light pole 104, one or more batteries such as rechargeable batteries, a solar-based circuit, a connector integral with the luminaire 106, or other source. Along these lines, any one or more airborne nodes 102a may be located on the support arm of the street light assembly 100, below the luminaire 106, above the luminaire 106, or other location.

[0047] FIG. 2 illustrates an embodiment of a system for monitoring traffic 120 along a roadway 108. In this embodiment, two street light assemblies 100a and 100b are shown. A first street light assembly 100a and a second street light assembly 100b are positioned along the line of the street light assemblies 100 shown in FIG. 1. Each of the street light assemblies 100a, 100b includes an airborne node 102 positioned to capture video and process images of the roadway 108 in the video. In some embodiments, the airborne node 102 is mounted on or included in some, but not all, of the street light assemblies positioned along the roadway 108. For example, in some embodiments, every other street light assembly, or some other number of street light assemblies, may include an airborne node 102. In some embodiments, the airborne node 102 mounted on the street light assemblies may be utilized to capture video and process images of objects along one side of a two-way roadway or both sides of a two-way roadway. In these cases or other cases, the airborne node 102 of the first street light assembly 100a cooperates with the airborne node 102 of the second street light assembly 100b to generate one or more images for processing information related to a particular target object.

[0048] In the system for monitoring traffic 120 shown in Figure 2, two vehicles (e.g., trucks) 114a, 114b are traveling on a road proximate to a first street light assembly 100a and a second street light assembly 100b. The vehicles may include a first vehicle 114a and a second vehicle 114b. Also shown is a pedestrian 116 crossing the road via a crosswalk 112. According to one embodiment, one or more airborne nodes 102 acquire video and generate, take, or sample images from the video to identify objects of interest and possibly points of interest associated with the objects of interest.

[0049] 3 illustrates one embodiment of an image 130 captured from video captured by a video camera on an airborne node mounted on a street light assembly. Image 130 is generated, captured, or realized by an airborne node mounted on a street light assembly located along roadway 108. In some cases, multiple images along the line of image 130 are captured and processed.

[0050] The road 108 shown in Figure 3 may be the same or a different portion of the road 108 shown in Figure 2. The airborne node, i.e., the remote computing device, is positioned to identify one or more objects of interest in the captured image 130.

[0051] In the image 130 shown in FIG. 3, four different target objects are identified. These four objects include a first vehicle 114c (i.e., a truck), a second vehicle 114d (i.e., a truck), a third vehicle 114e (i.e., a car), and a pedestrian 116. Each object may be identified in a series of consecutive captured images from a video, and in some cases, the image 130 may be one of a series of images in a video stream. In other cases, the image 130 may be a synthesized or generated image from one or more video sources. By processing the series of consecutive captured images, for example, the speed or velocity of the target object may be estimated, and the trajectory of the object's future location may be predicted. For example, if a mapping between geographic coordinates or grid points and pixels of the captured images is known or determined, the speed at which the object is traveling along the road 108 may be estimated.

[0052] A boundary perimeter may be calculated or determined for each of the target objects. In the present invention, a boundary perimeter is a virtual object of a geometric or different shape that partially or completely surrounds or envelops the target object. In the image 130 shown in FIG. 3, the first vehicle 114c is associated with the first boundary perimeter 124c, the second vehicle 114d is associated with the second boundary perimeter 124d, the third vehicle 114e is associated with the third boundary perimeter 124e, and the pedestrian 116 is associated with the fourth boundary perimeter 126. A two-dimensional pixel coordinate system 140 may be used to analyze the individual pixels of the image 130. A global positioning system (GPS) geographic coordinate system 136 may be used to define anchor points of the image 130. This may include any suitable structure determined within the image.

[0053] According to at least one embodiment, for each target object and its associated boundary perimeter, the geographic coordinates of a particular geographic coordinate system 136 may be determined by one or more processors of the airborne node or a neural network implemented by any local or remote computing device associated with one or more airborne nodes. For example, metadata indicative of the geographic coordinates of the boundary perimeter may be determined, and the metadata may be transmitted to a remote computing device, such as a back-end server, for further processing. According to one or more embodiments, the remote computing device may receive, analyze, translate, or process the received geographic coordinates of the boundary perimeter for the target object. In some embodiments, the remote computing device may estimate an approximate center point of the target object based on the received geographic coordinates of the boundary perimeter associated with the target object. In another embodiment, processing in the airborne node or another computing device, such as its neural network, may estimate the approximate center point of the target object and then transmit the geographic coordinates of the approximate center point of the target object to a remote computing device for further processing.

[0054] According to one embodiment, the airborne node or other computing device, such as a remote computing device, may obtain or process the geographic coordinates of each object of interest in the time-stamped metadata associated with the series of images. In these cases or other cases, the metadata of one or more images may include timestamp information, location information, the airborne node identifier, and further other information. For example, if the images are taken for a particular period of time (e.g., 15 minutes, 2 hours, 10 hours, 24 hours, or other longer or shorter period of time), there may be enough information to determine an optimal location for placing the virtual tripwire at a set of grid points corresponding to the field of view of the video camera that generated the image 130. The image 130 shown in FIG. 3 illustrates the placement of the virtual tripwire 138a at an automatically generated location. For example, the placement of the virtual tripwire 138a may be determined based on various analyses determined from geographic coordinate data received by the airborne node or other computing device, such as a remote computing device, from a particular video camera associated with a particular airborne node.

[0055] More specifically, the received geographic coordinate data for the target objects may be processed at the airborne node or other computing device, such as a remote computing device, to identify directions of movement for the various target objects represented by lattice points of a grid associated with the geographic coordinates. For example, each pixel or group of adjacent pixels captured by a video camera associated with a particular airborne node may correspond to one or more particular real-world geographic coordinates or lattice points of a grid of geographic coordinates. In other words, a mapping may be determined between each pixel or group of pixels of the image 130 and the lattice points of the grid.

[0056] In at least one exemplary embodiment, the location of the center point of the target object in the series of captured images may be correlated with the lattice points of the grid. For example, for each captured image, geographic coordinates corresponding to one or more center points of the target object may be identified, and grid points corresponding to these geographic coordinates may be determined. If the target object is moving through the series of images, the geographic coordinates of the approximate center point of the target object are represented by a movement that appears to move across the grid to different grid points at the time points associated with the series of images. The represented movement of these grid points relative to the target object may be utilized to determine specific vector information for the target object. In some cases, for example, the direction of movement of the target object may be determined based on the movement of the determined approximate center point, or other points, or point information, of the target object at different times.

[0057] 4A-4C show images 130a-130c showing the movement of a target object 114 (e.g., a moving vehicle) along a road. FIG. 4D shows a set of information-bearing grid vectors on the image 130d of the road, which represents the location of the approximate center point of the target object shown in FIG. 4A-4C. The images 130a-130c may be generated by a video camera device of one or more airborne nodes 102. The generated image 130d may be generated by one or more processors of one or more airborne nodes 102 or a remote computing device. In the present invention, FIG. 4A-4D may be collectively referred to as FIG. 4. The description of the above-mentioned structure will not be repeated for the sake of brevity.

[0058] 4A, a target object 114, such as a car, is traveling along a road at a particular time, which may be represented by a time stamp, such as time t1. A boundary perimeter 124 surrounds the target object 114.

[0059] An approximate center point 132a may be determined for the target object 114 at time tl in the image 130a. In some cases, the approximate center point 132 may be generated mathematically from the determined length and width of the object. In some cases, the approximate center point 132a may be generated as an approximate geometric center of a group of pixels. In these cases or still other cases, the approximate center point 132a may be generated in other ways. In at least some cases, the approximate center point 132a is determined algorithmically, such as by an artificial intelligence algorithm (e.g., machine learning, machine vision, neural networks, etc.) recognizing the target object and providing metadata about the object, such as the approximate center point.

[0060] 4B, the same target object 114 and boundary perimeter 124 are captured, generated, or determined, as appropriate, but image 130b is associated with a different timestamp, such as t2, where target object 124 may have moved along the road such that the approximate center point 132b of target object 114 at time t2 is in a different location than it was in image 130a at time t1.

[0061] Correspondingly, image 130c shown in Figure 4C shows the same object 114 and boundary perimeter 124, except that image 130c is associated with a different timestamp, such as t3. In image 130c, object 114 has continued to move along the roadway such that the approximate center point 132c of object 114 at time t3 is at a third location different from that in image 130a at time t1 or image 130b at time t2.

[0062] The movement of the approximate center point 132a, 132b, 132c of the target object 114 through the sequence of images 130a, 130b, 130c may be utilized to determine the direction of movement of the target object 114. For simplicity, the approximate center points 132a-132c may be individually or collectively referred to as the approximate center point 132. In at least one particular embodiment, the geographic coordinates associated with the location or associated direction of movement of the approximate center point 132 of the target object 114 may be plotted on a set of grid points, such as a two-dimensional pixel coordinate system 140. In such a case, for example, each pixel or group of adjacent pixels of an image captured by a video camera may correspond to a particular geographic coordinate and grid point on the grid.

[0063] FIG. 4D shows the generated image 130d. Here, grid points are shown for the positions of approximate center points 132a-132c of the target object 114 at times tl, t2, and t3, respectively. Also shown on the grid are approximate center points 132a, 132b, and 132c, and corresponding vectors 134a, 134b, and 134c indicating the direction of movement of the target object 114 are also determined. As shown, approximate center point 132a is associated with vector 134a indicating the approximate direction of movement of the target object 114 at time tl. Approximate center point 132b is associated with vector 134b indicating the approximate direction of movement of the target object 114 at time t2. Approximate center point 132c is associated with vector 134c indicating the approximate direction of movement of the target object 114 at time t3.

[0064] For simplicity, in the embodiment shown in Figures 4A-4C, only three images of the target object 114 and corresponding approximate center points 132a-132c are shown. However, in other contemplated embodiments, tens, hundreds, thousands, tens of thousands, and more approximate center points are identified, generated, or determined for the target object in images captured by one or more video cameras over a particular length of time. Also, in some cases, vectors indicating the direction of movement 134a, 134b, 134c may be determined for each of these approximate center points and applied to a virtual grid or other organizational structure, such as the two-dimensional pixel coordinate system 140 shown in Figure 4D. When a sufficient number of target object movements have been identified over time, the results may be applied to a coordinate system such as a grid, and the results may be further utilized to identify one or more optimal or preferred locations for the virtual tripwires.

[0065] 5 shows another set of information-bearing grid vectors 142 on a generated image of another road, representing the direction of travel and associated variations for various objects of interest. The embodiment shown in FIG. 5 shows the direction of travel and associated variations for various points on the grid. The road shown in FIG. 5 is generally aligned with north-south and east-west travel lanes, as indicated by the compass rose 144. In some cases, the compass rose 144 is not present, and instead longitude and latitude coordinates or other mechanisms for understanding and processing the direction of travel are provided.

[0066] The embodiment shown in FIG. 5 includes a set of information-bearing vectors arranged on a set of grid points corresponding to the field of view of a video camera, such as the video camera of the airborne node 102. A number of approximate center points and travel direction vectors are shown in the embodiment 142. Each of these may be associated with a variation. For example, each point (i.e., each approximate center point of the target object) may be represented by a circle or a dot. Additionally, each point may be associated with a vector indicating a median direction of travel and a variation in the direction of travel. For example, various arrows are shown indicating vectors indicating various directions of travel. As shown in the figure, some points have arrows pointing to the left of the image, such as to indicate a direction of travel from east to west. Other points have arrows pointing down the image, such as to indicate a direction of travel from north to south.

[0067] Additionally, in each of the sets of information-bearing grid vectors 142 shown in FIG. 5, a reference indicator is shown. A minus sign, for example, at one of the points may indicate that the point is associated with a relatively low variation in the direction of travel. Additionally, a cross (or "x"), for example, through one of the points indicates that the point is associated with a relatively high variation in the direction of travel. In the embodiment shown in FIG. 5, points located at or near an intersection may be associated with a relatively high variation in the direction of travel, and points located relatively far from the intersection may be associated with a relatively low variation in the direction of travel. The generated image shown in FIG. 5 as a set of information-bearing grid vectors 142 may be useful to traffic engineers, road planners, and the like. Thus, in some cases, any suitable number of generated images may be formed, superimposed, displayed, or implemented. In these and other cases, each approximation point is virtually represented, for example, in a database, file, memory, or other repository. In this manner, in addition to the vector indicating the direction of travel of the media and an indicator representing the degree of variation in the direction of travel, other suitable metadata may also be associated with one or more approximation center points. A non-limiting and non-exhaustive list of such metadata may include timestamp information, identifying information about the camera, identifying information about the object of interest, and weather or other environmental information collected contemporaneously with the image, etc. Other metadata are contemplated.

[0068] The set of points shown in Figure 5 is shown relatively large for ease of illustration, however, it should be noted that in some embodiments any suitable number of points (e.g., tens, hundreds, thousands, tens of thousands) may be utilized and thus other generated images may render, for example, smaller points in different ways.

[0069] Variations in the direction of movement and / or vectors indicative of the direction of movement may be utilized to determine a location to place the virtual tripwire. For example, as shown in FIG. 6, an automatically generated and placed virtual tripwire 138b is overlaid on top of the information-bearing grid vector 142 shown in FIG. 5. FIG. 6 illustrates an embodiment showing the direction of movement and associated variations for various points on a two-dimensional pixel coordinate system 140 (e.g., a grid). In this exemplary embodiment, a sliding window 146 may be utilized to determine a location to place the virtual tripwire 138b. For example, the sliding window 146 may be moved across the grid, and data associated with one or more points overlying the sliding window 146 may be processed to determine various candidate locations for placing the virtual tripwire 138b. In some embodiments, the sliding window 146 is automatically slid across the grid, such as, for example, the direction of movement along a road. Such "sliding" of the sliding window 146 may be performed computationally, and the generated image may be updated to indicate such movement. In the embodiment shown in FIG. 6, left and right arrows (not shown) indicate the movement of the sliding window 146 .

[0070] In one embodiment, the goal is to count vehicles crossing the virtual tripwire 138b and determine the average speed or velocity of each vehicle. In this case, it may be preferable for the virtual tripwire 138b to be formed at a determined geographic location where the sliding window 146 overlaps a point associated with a relatively low variation in direction of travel. For example, the sliding window 146 may be virtually "moved" across a series of grid points until the sliding window 146 or virtual tripwire 138b overlaps a point in the generated image that has a minimal level of variation in direction of travel.

[0071] On the other hand, if the goal is to count vehicles changing direction, such as an intersection between a north-south road and an east-west road, it may be desirable to place the virtual tripwire 138b at a point where the sliding window 146 overlaps with a point associated with a relatively high variation in direction of travel. In such an embodiment, the sliding window 146 may be computationally, virtually, or otherwise moved until the sliding window 146 or virtual tripwire 138b overlaps with a point having the highest level of variation in direction of travel.

[0072] Once a location relative to the sliding window 146 that overlaps points having a preferred level or value of variation in movement direction is identified, a virtual tripwire 138b may be placed at a location within the sliding window 146.

[0073] In some cases, the virtual tripwire 138b is placed at a predetermined location or at a predetermined position within the sliding window 146. In other cases, the optimal location of the virtual tripwire 138b within the sliding window 605 may further be selected via manual adjustment or other mechanisms that allow for a greater level of precision.

[0074] Information corresponding to the location of the optimal placement of the virtual tripwire 138b may then be logged or stored for further processing.

[0075] Also shown in FIG. 6 is an optional second virtual tripwire 138c. The position of the implemented second virtual tripwire 138c is positioned using a sliding window in the same or similar manner as described for virtual tripwire 138b. In general, the second virtual tripwire 138c is formed with different parameters and positioned to provide different information than the first virtual tripwire 138b. Thus, the optimal position of the first virtual tripwire 138b is different from the optimal position of the second virtual tripwire 138c. Along these lines, any suitable number of virtual tripwires may be formed in accordance with the teachings of the present invention, with their respective optimal positions.

[0076] 7 illustrates an embodiment of an airborne node 102 deployed to automatically generate and place virtual tripwires. In some embodiments, the airborne node 102 illustrated in FIG. 7 may be deployed as a standalone video processing device with local on-board computing resources, while in other embodiments, the airborne node 102 may communicate with one or more remote computing servers. As illustrated in FIG. 2, the airborne node 102 may be deployed on or in close proximity to a street light assembly located along a roadway 108.

[0077] The airborne node 102 may include various components, such as a video camera 150, a memory 152a, a receiver 154a, a processor 156a, a transmitter 158a, and other logic 160a. The video camera 150 is a video capture device positioned to capture video of, for example, traffic and other objects of interest along the road 108. The processor 156a may sample or capture one or more images, such as a series of images, from the video. The processor 156a may also, alone or in conjunction with one or more other local or remote computing devices, identify one or more objects of interest in one or more captured images and identify geographic coordinates of a perimeter boundary associated with the one or more objects of interest. The processor 156a may generate metadata indicative of the geographic coordinates of the perimeter boundary and may store such information in the memory 152a. The memory 152a may additionally or alternatively include computer-executable instructions executable by the processor 156a to perform one or more processes, such as identifying the perimeter boundary, generating the sliding window 146, moving the sliding window, and determining one or more suitable positions for the virtual tripwire 138b. The transmitter 158a may transmit one or more messages, such as a message including metadata indicating the geographic coordinates or other information of the perimeter boundary. For example, the transmitter 158a may transmit such a message to a server for subsequent processing. The receiver 154a may receive one or more messages originating from a remote computing server or from other devices. In at least some cases, the receiver 154a and the transmitter 158a are combined into a single transmitting and receiving device.

[0078] According to one embodiment, the airborne node 102 may acquire a series of images of at least a portion of a road, process the images to identify objects of interest, determine a boundary perimeter for each of the objects of interest in one or more images of the series of images, and determine an approximate center point for each of the objects of interest along with the location of the center point at various times (e.g., t1, t2, tn, etc. corresponding to timestamps in the series of images). In some cases, the respective boundary perimeters are formed as virtual geometric objects surrounding each of two or more objects of interest in at least one geometric plane of the images. The transmitter 158a may transmit a message including the geographic coordinates for the respective boundary perimeters, the approximate center point, or other selected data or metadata related to the objects of interest in at least one of the images, e.g., to a remote computing device for further processing.

[0079] Embodiments of the airborne node 102 include other logic 160a, which may include any suitable circuitry, executable software, data, and other logic. In some cases, for example, the other logic 160a includes positioning circuitry (e.g., Global Positioning System (GPS)) that provides the node's location (e.g., longitude, latitude, altitude, etc.) within a geographic coordinate system. In some cases, the other logic 160a includes identification logic for uniquely identifying each airborne node 102 within a system of airborne nodes 102. In these and other cases, the other logic 160a may include security logic for removing, anonymizing, obfuscating, aggregating, or processing personally identifiable information (PII) from any images captured by the video camera 150. Such security logic may alternatively or additionally encrypt or protect the node's PII and other data and perform certain other security functions.

[0080] 8 illustrates an embodiment of a remote computing device 172. The remote computing device 172 may comprise, for example, a computing server. The remote computing device 172 may be a standalone device or a network of computing devices (e.g., a cloud computing system, a server farm, etc.). In this manner, the remote computing device 172 may be a standalone computing device having a single set of computing resources or may include any suitable number of remote computing devices.

[0081] The remote computing device 172 includes various components, such as a memory 152b, a receiver 154b, a processor 156b, a transmitter 158b, and other logic 160b. The receiver 144b of the remote computing device 172 may receive a single message or multiple messages from the airborne node 102. In some cases, the message includes one or more images (i.e., a single image or a stream of images) from one or more airborne nodes 102. In these cases or other cases, the message includes, for example, geographic coordinates for the perimeter boundary of one or more objects of interest in the sequence of images. In some cases, this information is included in the metadata of the received message. Of course, other messages are contemplated.

[0082] In some cases, the processor 156b may be arranged to process the first, second and third images 130a-130c of Figure 4A, and in some cases, the processor 156b may be arranged to generate the image 130d of Figure 4D. Along these lines, the processor 156b may additionally or alternatively process the image data to generate the set of information-bearing grid vectors 142 shown in Figures 5 and 6. In some cases, for example, the processor 156b described herein with respect to Figure 6 may be arranged to identify suitable locations on the grid for placing the virtual tripwire 138b through the use of a sliding window 146.

[0083] Memory 152b may additionally or alternatively be arranged to include computer-executable instructions executable by processor 156b. The one or more processes executed by processor 156b may identify a location for a virtual tripwire, and in some cases, the one or more processes may determine an approximate center point of the target object.

[0084] Transmitter 158b may transmit one or more messages indicating a location, such as via geographic coordinates and / or grid points, for the virtual tripwire, such as a message to a video processing device, and may also transmit an alert, for example, based on the determination of the location for the virtual tripwire.

[0085] An embodiment of remote computing device 172 includes other logic 160b, which may include any suitable circuitry, executable software, data, and other logic. In some cases, for example, other logic 160a includes, but is not limited to, web server functionality, database functionality, security logic (e.g., hardware or software, or hardware and software arranged to perform encryption / decryption, obfuscation, or other data protection operations), parallel processing, one or more user interfaces, and one or more machine interfaces.

[0086] Those skilled in the art will recognize that features of the other logic 160a described with respect to the airborne node 102 may also be implemented in the other logic 160b of the remote computing device 172, and vice versa.

[0087] In some cases, other logic 160a (FIG. 7) of airborne node 102 and / or other logic 160b of remote computing device 172 may perform geostatistical calculations to predict the location of the object of interest, the direction of movement of the object of interest, its likelihood to change direction, variance, or other similar predictive information. Such geostatistical calculations may be applied to identify an optimal location for the virtual tripwire. A user may select the size, shape, length, width, time of active data collection, and other parameters associated with the virtual tripwire. Some or all of the information is applied as described herein to generate the location of the virtual tripwire.

[0088] Geostatistical calculations as used herein may include Gaussian process regression (e.g., kriging algorithms), simple interpolation, space-time averaging, modeling, and many other techniques. In some embodiments, certain techniques are used to create certain types of predicted conditions (e.g., likelihood of turning, likelihood of accelerating or decelerating, likelihood of a first object of interest overtaking (i.e., passing) a second object of interest, etc.). In other embodiments, different techniques are used to create other types of predicted conditions. For example, in areas with high traffic volumes or roads with many intersections, space-time averaging techniques may affect the calculation of predicted positioning, predicted direction of movement, and predicted variance. In areas where imagery updates are slow or where actual data is limited or too old, more emphasis may be placed on kriging techniques to generate predictive information. In still other cases, when non-periodic events occur (e.g., funeral processions, construction, storms or inclement weather, popular concerts or sporting events, etc.), data mining and modeling techniques may be used to generate predictive information related to one or more objects of interest.

[0089] In at least some cases, the kriging algorithm performs spatial interpolation by applying weighting factors based on the calculated distance between the geographic points where the anchor points or hazard zone boundary data were collected. Kriging techniques model the correlation structure of the data analyzed from the images as a function of distance and may include information about known covariation between the expected separation or lack of separation between objects of interest and other spatial information (co-kriging).

[0090] In the embodiments contemplated by the present invention, the kriging algorithm provides predictions of quantitative measures (e.g., how many predicted turns will actually occur). If a particular mapped grid contains few or no actual objects of interest, or if images showing such objects were taken in the recent past (i.e., the data is not "stale"), predictions of lane changes and turns, etc. with acceptable accuracy can be made of the likelihood of turns, speed changes, etc. This prediction can be derived from a spatio-temporal model of the correlation structure of training data taken across many similar road locations (e.g., kriging or co-kriging in the case of pure spatial interpolation), providing a consistent and quantitative process for data interpolation. Thus, the interpolation algorithm described herein provides analytically based estimates for the prediction of dangerous conditions. This can provide useful data on which an automated decision on the placement of virtual tripwires can be made.

[0091] The spatio-temporal averaging techniques described herein involve averaging a set of data derived from multiple images collected over geographic space and / or time. The averaged data can be used in determining predicted conditions. When such data is derived from images of actual road conditions, the data is known to be accurate to the geographic location at the time the data was collected.

[0092] In some embodiments, data is collected and spatially averaged within an area, such as a grid segment of M×N pixels, where M and N represent integers indicating the pixel count, and M and N represent the “length” and “width” of a pixel in a particular image. In another embodiment, data is collected and spatially averaged across multiple areas. Spatial averaging may include simple averaging, where the sum of values ​​is divided by the number of samples, weighted averaging, or other averaging techniques. In one example of target object behavior, weighted data averaging is used such that data analyzed from pixels in one area of ​​an image is fully weighted, data analyzed from directly adjacent pixel areas is partially weighted at a first level, and data analyzed from nearby, non-adjacent pixel areas is partially weighted at a second level lower than the first level. The weighted data samples may be summed in this embodiment and the sum divided by the number of samples.

[0093] In some embodiments, pixel data representing boundary perimeters, virtual tripwire placements, and / or actual target objects may be collected and averaged over time. It is recognized that based on certain conditions (e.g., midday vs. rush hour, daytime vs. evening or nighttime, winter vs. summer, and other similar conditions), certain data will be less relevant to the calculation of predicted lane changes, turns, and the like. Thus, techniques may be employed to reduce the influence of certain data as the data ages. For example, a weighting factor may be applied to certain image data to modify (e.g., increase or decrease) its influence for "X" percent per hour (e.g., 5 percent, 10 percent, 25 percent, or other percentage) for a determined number of hours, "X" percent per month, "X" percent based on the size or speed of the target object, or any other condition. The weighting factor may then be further decreased as conditions change, and the percentage of increase or decrease in the weighting factor may also be changed. For example, it is recognized in some embodiments that more lane changes occur during rush hour but not during daylight hours. Thus, in this embodiment, the predictive data may be fully weighted during rush hour, and then for each hour thereafter, the weighting of the predictive data may be decreased by a particular amount until the weighting reaches a desired level. In this manner, weighting data for certain times of the day may have little effect on the predictive calculations of lane changes, turns, etc., while weighting data for other times of the day may have a disproportionately large effect. Such time-based target object behavior conditions may enable improved automatic selection of virtual tripwire locations.

[0094] Other techniques applied to generate behavioral data for various target objects include data mining techniques. Data mining techniques may be categorized as classification and / or regression algorithms. These types of classification and regression techniques may be implemented as tree-based algorithms (e.g., classification and regression tree (CART) techniques). Using CART techniques, the analysis at a given airborne node 102 or remote computing device 172 may apply a progressive or recursive sequence of binary (e.g., if-then) decisions. In at least some cases, CART techniques are implemented to prepare data drawn from multiple images for use within a kriging algorithm. These classification techniques iterate over the data to predict categorical variables continuously over a determined time period, while regression techniques iterate over the data to predict continuous variables. Techniques contemplated herein may include "bagging" techniques, "random forest" techniques, and others.

[0095] The exemplary CART technique described in this invention for generating behavioral values ​​of target objects is non-parametric and non-linear. The values ​​obtained from tree-based classification and regression algorithms are derived from a handful (e.g., 2, 5, 10, 25, or other number) of logical if-then conditions (tree nodes). The algorithm does not require implicit assumptions about the underlying relationship between predictor variables and dependent variables, and the actual relationship is not necessarily linear. As an example of nonlinearity, in a selected region (e.g., a grid segment) with a particular characteristic (e.g., an intersection with a crosswalk), a continuous outcome variable (e.g., the likelihood that two objects will simultaneously change lanes, speed up, slow down, drop off, etc.) may be positively related to the input variable (e.g., the relative speed between two objects, a third object observed in close proximity, etc.) if the input variable is less than a particular amount (e.g., 20 kilometers (km) per hour, 10 kilometers per hour, or other speed), but negatively related if the variable is the same under different characteristics or is greater than that amount (faster) under any characteristic. In this way, tree algorithms may reveal two or more branches based on the value of a single input variable, and can serve to illuminate nonlinear relationships between variables.

[0096] Still other techniques can be used to generate values ​​representing lane change, turn, and any other target object behavior predictions. For example, techniques to improve model selection can be used to obtain more relevant data. More specifically, model selection techniques can be used to select or identify data in one set of road characteristics that are similar in other roads. By improving model selection techniques, for example, it can be predicted that a turn is more likely in the vicinity of a detected pothole, or that high speed driving will occur on a Sunday afternoon near a sports stadium, and so on, avoiding overfitting of the model.

[0097] 9 illustrates one embodiment of a system 170 for an airborne node to communicate with a remote computing device 172. Various video camera devices, such as a first video camera device 150a, a second video camera device 150b, and an Nth video camera device 150n, may be integrated with or associated with various airborne nodes 102b, 102c, 102n. The airborne nodes 102b, 102c, 102n of FIG. 5 are along the lines of other airborne nodes 102 of the present invention. In some cases, the airborne node 102c may have multiple cameras associated with it, and in other cases, the airborne node 102n may have only a single video camera 150n associated with it, or may be configured without a video camera.

[0098] Information from any of the video cameras 150a, 150b, 150n may be communicated, for example, via one or more wired or wireless (e.g., fiber, mobile, etc.) connections to a remote computing device 172. For example, each of the airborne nodes 102b, 102c, 102n and their associated video camera devices may be mounted to a street light assembly, such as the street light assembly 100 of FIG.

[0099] 9, it should be noted that while only three video camera devices 150a, 150b, 150n are shown, in some implementations, more or less than three video camera devices may be utilized. Each of the video camera devices may capture images from the video and process the images to determine geographic coordinates of surrounding boundaries associated with objects of interest in the images. Each of the video camera devices 150a, 150b, 150n may also generate metadata describing the geographic coordinates for the approximate center points of the respective objects of interest, as well as other information identifying the objects of interest and / or images in which these objects of interest are identified.

[0100] The airborne nodes 102b, 102c, 102n may communicate metadata corresponding to the perimeter boundaries in one or more images to a remote computing device 172 via transmission over a network 174, such as the Internet or a mobile network. The remote computing device 172 may process or analyze the received metadata to identify an optimal location for placing the virtual tripwire, such as on a set grid point. After a location for the virtual tripwire is identified, information corresponding to the location may be stored in a repository, such as in a data storage device 176. It should be noted that while only one remote computing device 172 is shown in the system 170 of FIG. 9, in some implementations, a cloud-based network device, such as a set of remote computing server devices, may be utilized.

[0101] 10 is a data flow diagram illustrating a process 1000 for identifying a location for placing a virtual trip wire on a set of grid points corresponding to a field of view of a video camera. This embodiment in accordance with claimed subject matter may include all, some, or other modules of modules 1002-1018. Also, the order of modules 1002-1018 is merely an example order. Here, the process begins with module 1002.

[0102] A sequence of images of at least a portion of a road within the field of view of the camera is acquired in module 1004. For example, a set of grid points may correspond to a pixel or group of pixels in each image of the sequence of images.

[0103] In module 1006, the sequence of images may be processed to identify objects of interest.

[0104] In module 1008, the movement of the objects through the sequence of images may be determined and the direction of movement of one or more points of each object through the set of grid points may be identified.

[0105] In module 1010, the variation in movement direction for individual grid points or groups of grid points may be determined.

[0106] In module 1012, vectors of median movement directions may be determined for vector points corresponding to a set of grid points.

[0107] A number of candidate locations for placing a virtual tripwire on the set of grid points for monitoring the movement of one or more objects of interest through the sequence of images may be determined in module 1014. For example, the virtual tripwire may include a segment that extends in a direction approximately perpendicular to the direction of travel along at least a portion of the road.

[0108] A particular candidate location may be selected from a number of candidate locations for placement of a virtual tripwire perpendicular to the direction of movement that intersects with the grid point in module 1016. For example, the selection may be based on a number of vectors and associated variations in the direction of movement.

[0109] At module 1018, an alert may be generated based on the selection of a particular candidate location for placing the virtual tripwire.

[0110] A process proceeds to identify a location to place the virtual tripwire.

[0111] Having described certain embodiments above, further clarifications will be provided for certain terms used herein to provide a more complete understanding of what is deemed to be inventive steps in the present invention.

[0112] In the present invention, terms such as "optimum," "best," "ideal," "perfect," and the like, as used in the context of selecting a location for a virtual tripwire, are terms of relationship and not of absolute superiority. For example, in a selection between location "A" and location "B" for a virtual tripwire, location "A" may be determined as the "optimum" or "best." Such a determination is based on consideration of certain factors that apply to both location "A" and location "B." Although one or more other locations (i.e., location "C," location "D," location "N," etc.) may provide improved results over location "A," in the context of this specification, location "A" may be determined to be similar, if not optimal, best, ideal, superior, top, exceptional, outstanding, superlative, outstanding, unparalleled, unmatched, top, largest, highest, or highest, etc., relative to another location or many other locations. Stated another way, the present invention describes systems, methods, and devices for automatically determining an optimal location (e.g., location "A") for a virtual tripwire over other considered locations for any particular set of one or more reasons, factors, or parameters.

[0113] In general, as used herein, a "target object" refers to an object having a determined position over time (e.g., the object's movement, the object's position relative to other objects that are stationary or moving, etc.) that is used in determining the optimal location of the virtual tripwire. For example, the target object may include a predefined object such as a vehicle, a living body (e.g., a light pole, a control box, a fire hydrant, or other permanently or temporarily stationary object), or other object (e.g., trash or other debris, traffic cones or other work site materials, other detectable or determinable objects). At any given instance, one or more target objects may be imaged, analyzed, tracked, or processed. In some implementations, a target object may be identified based on the relative movement of one or more target objects through a series of captured images. In some implementations, a target object may be identified based on its estimated size. For example, a non-stationary object may be considered a target object if it includes a certain minimum number of neighboring pixels in a series of captured images.

[0114] In the present invention, video has been described with respect to two-dimensional video. This description is not limiting, and those skilled in the art will appreciate that other imaging and information generating technologies may also be used. For example, three-dimensional video, other multi-dimensional video, "heat mapping" video, infrared, microwave, radar, LiDAR, etc. may provide node-collected information for analysis in accordance with the principles of the present invention. Along these lines, the video cameras described herein (e.g., video cameras 150, 150a, 150b, 150c of Figures 4 and 9) may be any device arranged to provide information, such as images or image streams, suitable for determining objects of interest, areas of interest, (i.e., boundary perimeters), and positions relative to virtual tripwires, etc.

[0115] In the present invention, the boundary perimeter has been described as a geometric object that surrounds the target object in the geometric plane of the image. This description is not limiting, and it will be understood by those skilled in the art that any suitable two-dimensional, three-dimensional, or other multi-dimensional shape may be used as the target area, and such target area may be interchangeably understood as the target area as described herein. Exemplary boundary perimeters may be rectangular, square, circular, oval, triangular, or hexagonal. Other exemplary boundary perimeters have irregular shapes. In some cases, the boundary perimeter tracks the shape of the target object by one or more predetermined distances. The target area defined by the boundary perimeter may in some cases substantially enclose the target object, while in other cases, the target area (i.e., the boundary perimeter) may only partially enclose the target object. For example, if the vehicle is moving forward in a particular direction, the target area (i.e., the boundary perimeter) may in some cases only be formed around the front of the vehicle, or only be formed around the front and sides of the vehicle, but not the rear of the vehicle. In a computing environment in the context of this specification, a boundary perimeter may be realized as a boundary perimeter virtual object that is positioned with reference to or according to a determined geographic location.

[0116] The term metadata, as used herein, means "information" pertaining to "data." "Information" may be any suitable information and may be in any suitable form that is represented or can be represented in a computing device. "Data" may be any suitable data related to detection of objects of interest and near-miss conditions of such objects. For example, metadata may include, but is not limited to, geographic coordinates, identifiers to objects of interest, addresses, time of day, day of the week, year, sensor identifiers, camera identifiers, airborne node identifiers, and any other such suitable information.

[0117] The term road as used herein includes any surface on which a vehicle travels. A vehicle may be an automobile, car, truck, bus, van, lorry, cart (e.g., golf cart, minibus), motorcycle, bicycle, scooter, recreational vehicle, wagon, trailer, tractor, sled, snowmobile, construction equipment (e.g., loader, bulldozer, steam roller, etc.), train, trolley, tram, monorail, ground plane, etc. A vehicle may be powered by a gasoline engine, electricity, chemical reaction, or other power source. A vehicle may be manually controlled, partially autonomously controlled, or fully autonomously controlled, either locally or remotely. A road may be formed of asphalt, concrete, gravel, steel, tile, wood, composite material, hard-packed earth, or any other surface suitable for vehicle travel. A road may be any path of any suitable length, width, or other dimensions. A road may be outdoor, indoor, or partially outdoor and partially indoor. Exemplary roads contemplated in the present invention include, but are not limited to, alleys, roads, highways, streets, autobahns, bike paths, boulevards, bridleways, bridleways, thoroughfares, bypasses, side streets, schoolyards, farm roads, causeways, circles, trails, crosswalks, cul-de-sacs, cul-de-sacs, tracks, private roads, private streets, highways, factories, garages, toll highways, groves, lanes, military bases, motorways, overpasses, parking lots, roads, ramps, roads, lines, ring roads, service roads, shoulders, side roads, plazas, stores, terraces, main roads, footpaths, tunnels, turnpikes, underpasses, and warehouses.

[0118] In some cases of the present invention, embodiments have been described in which the airborne node is located above, below, or adjacent to the luminaire (e.g., FIG. 1). In these and other cases, the airborne node may be electromechanically coupled to a standardized powerline interface of the luminaire. The standardized powerline interface is a roadway area lighting standard driven by a standards body such as ANSI C136.41 (e.g., a NEMA-based connector / socket system), although other standardized powerline interfaces are also contemplated (e.g., an interface conforming to ZHAGA CONSORTIUM, an international body that creates industry standards in the LED lighting industry). In at least some aspects of the present disclosure, the standardized powerline interface defines a limited number of electrical / communication conduits through which signals may enter or be passed out of the streetlight luminaire. In some aspects, the interface may be referred to as a NEMA interface, a NEMA socket, an ANSI C136 interface, or the like.

[0119] At least one known NEMA interface implements the power line interface with connectors and receptacles that include seven electrical / communication conduits (e.g., pins, blades, springs, connectors, receptacles, sockets, and other similar "contacts"). A set of three primary contacts carry line voltage signals, load voltage signals, and neutral voltage signals. Four secondary contacts may be used by a streetlight controller to pass power, control information, status information, etc.

[0120] As will be appreciated based on the above specification, one or more aspects of the above-described embodiments of the present disclosure may be implemented using computer programming or engineering techniques, including computer software, firmware, hardware, or any combination or subset thereof. Any such resulting program having computer readable code may be embodied or provided in one or more non-transitory computer readable mediums, thereby making a computer program product, i.e., an article of manufacture, according to the embodiments described herein. For example, the non-transitory computer readable medium may be, but is not limited to, a fixed drive, a semiconductor memory such as a flash memory, a read-only memory (ROM), and / or any transmission / reception medium, such as the Internet, cloud storage, the Internet of Things, or other communication networks or links. An article of manufacture including computer code may be produced and / or used by executing the code directly from a particular medium, or by copying the code from a particular medium to another medium, or by transmitting the code over a network.

[0121] A computer program (also referred to as a program, software, software application, "app", or code) includes machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, cloud storage, Internet of Things, and / or apparatus (e.g., memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. However, "machine-readable medium" and "computer-readable medium" do not include ephemeral signals. The term "machine-readable signal" refers to any signal that may be used to provide machine instructions and / or any other type of data to a programmable processor.

[0122] FIG. 10 includes a data flow diagram illustrating non-limiting processes that may be used by the airborne node and other computing system embodiments described herein. Each of the processes described in this regard may represent a module, segment, or portion of software code that includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some implementations, the functions noted in the process may occur in a different order, may include additional functions, may occur simultaneously, and / or may be omitted. Thus, the description and illustration of the processes herein should not be taken to imply a fixed order for performing the process steps. Rather, the process steps may be performed in any order that is practicable, including the simultaneous execution of at least some steps. Although the present disclosure has been described in conjunction with specific examples, it should be understood that various changes, substitutions, and alterations apparent to those skilled in the art may be made to the disclosed embodiments without departing from the spirit and scope of the present disclosure as defined in the appended claims.

[0123] Some of the detailed descriptions are presented herein in terms of algorithms or symbolic representations of operations on binary digital signals stored within a memory of a specific apparatus or special purpose computing device or platform. In the context of this particular specification, the term specific apparatus or the like includes a general purpose computer once it has been programmed to perform particular functions according to instructions from program software. Algorithmic descriptions or symbolic representations are an example of techniques used by those skilled in the signal processing or related arts to convey the substance of their work to others skilled in the art. An algorithm is generally considered herein to be a self-consistent sequence of operations or similar signal processing that leads to a desired result. In this context, operations or processing include physical manipulations of physical quantities. Typically, though not necessarily, such quantities can take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, or otherwise manipulated.

[0124] The drawings herein illustrate portions of one or more non-limiting computing device embodiments, such as one or more components of airborne node 102. The computing device may include operating hardware found in conventional computing device equipment, such as one or more processors, volatile and non-volatile memory, serial and parallel input / output (I / O) circuitry conforming to various standards and protocols, wired and / or wireless networking circuitry (e.g., communications transceivers), one or more user interface (UI) modules, logic, and other electronic circuitry.

[0125] A processing device or "processor" as described herein includes a central processing unit (CPU), a microcontroller (MCU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a peripheral interface controller (PIC), or a state machine, or the like. Thus, a processor as described herein includes any device, system, or part thereof, that controls at least one operation, and such devices may be implemented in hardware, firmware, or software, or a combination of at least two thereof. The functionality associated with a particular processor may be centralized or variable, local or remote. A processor may interchangeably refer to any type of electronic control circuitry configured to execute programmed software instructions. The programmed instructions may be high-level software instructions, compiled software instructions, assembly language software instructions, object code, binary code, or microcode, or the like. The programmed instructions may reside in an internal or external memory, or may be hard-coded as a state machine or a set of control signals. In accordance with the methods and devices described herein, one or more embodiments embody software executable by a processor, which, when executed, performs the actions of one or more methods.

[0126] The present application describes several embodiments that include or cooperate with one or more computing devices. It is recognized that these computing devices are arranged to execute one or more algorithms for implementing various concepts taught herein. Each of the algorithms is understood to be a finite sequence of steps for solving a logical or mathematical problem or performing a task. Any or all of the algorithms taught in this disclosure may be represented by a mathematical formula, a flowchart, a data flow diagram, a description in the specification, and other such means as are apparent in this disclosure. Along these lines, a structure for executing the algorithms disclosed herein includes at least one processing device that executes at least one software instruction retrieved from at least one memory device. The structure may optionally include suitable input circuitry known to those skilled in the art (e.g., keyboards, buttons, memory devices, communication circuits, touch screen inputs, and any other integrated circuits and peripheral circuitry inputs (e.g., accelerometers, thermometers, light detection circuits, and other such sensors)), suitable output circuitry known to those skilled in the art (e.g., displays, light sources, audio devices, tactile devices, control signals, switches, relays, etc.), and any additional circuits or other structures taught in this disclosure. Thus, all calls to means or step-plus-function elements in any of the claims will, where desired, be explicitly set forth.

[0127] As known by those skilled in the art, a computing device has one or more memories, each memory comprising any combination of volatile and non-volatile computer readable media for reading and writing. Volatile computer readable media include, for example, random access memory (RAM), etc. Non-volatile computer readable media include, for example, read only memory (ROM), hard disks, optical disks, and / or magnetic media such as flash memory devices, etc. In some cases, a particular memory is separated into separate areas, virtually or physically, such as a first memory, a second memory, a third memory, etc. In these cases, it is understood that the different divisions of the memory may be in different devices or may be embodied in a single memory. The memory in some cases is a non-transitory computer medium configured to store software instructions arranged to be executed by a processor. Some or all of the stored contents of the memory may include software instructions executable by a processing device to perform one or more specific actions.

[0128] The computing devices described herein may further include operable software found in conventional computing devices, such as software drivers that direct operation via an operating system or task loop, I / O circuitry, network circuitry, and other peripheral component circuitry. The computing devices may also include operable application software, such as network software for communicating with other computing devices, database software for building and maintaining databases, and task management software, as appropriate, for varying communication and / or operation workloads among various processors. In some cases, the computing device is a single hardware machine having at least some of the hardware and software described herein, and in other cases, the computing device is a networked collection of hardware and software machines that cooperate in a server farm to perform the functionality of one or more embodiments described herein. Some aspects of the conventional hardware and software of a computing device are not shown in the figures for simplicity.

[0129] It has proven convenient at times, primarily for reasons of common usage, to refer to such signals as bits, data, values, elements, symbols, characters, terms, numbers, numeric values, or the like. It should be understood, however, that all of these or similar terms are to be associated with the appropriate physical quantities and are merely convenient labels. Unless otherwise indicated, as will be apparent from the following description, throughout this specification, descriptions utilizing terms such as "processing," "computing," "calculating," "determining," and the like, are understood to refer to the operation or process of a specific apparatus, such as a special purpose computer or similar special purpose electronic computing device. Thus, in the context of this specification, a special purpose computer or similar special purpose electronic computing device can manipulate or transform signals that are typically represented as physical electronic or magnetic quantities in the memory, register, or other information storage, transmission, or display devices of the special purpose computer or similar special purpose electronic computing device. In other words, when arranged as described herein, each computing device can vary from a general, unspecified computing device to a combination device arranged from hardware and software configured for a specific, dedicated purpose such as to provide a determined technical solution. Now, to the extent that any of the inventive concepts described herein are determined by a competent adjudicating body to be encompassed by an abstract idea, the ordered combination of elements and limitations arranged as described herein is expressly presented to provide the requisite inventive concept by translating the abstract idea into a tangible and concrete embodiment of that abstract idea.

[0130] Although the embodiments described herein use computerized technology to improve the art of collision avoidance, other techniques and tools are still available to avoid collisions. Thus, the claimed subject matter embodiments do not exclude the entire or even substantial collision avoidance technology domain. The inventions described herein can use both new and known building blocks combined in new and useful ways along with other structures and limitations to create more than was previously known. These embodiments improve computing systems that, if unprogrammed or programmed differently, cannot perform or provide the specific near-miss detection functionality claimed herein. The embodiments described herein improve known collision avoidance processes and techniques. The computerized actions described in the embodiments herein are not purely conventional or well understood. Rather, the actions are new to the industry. Moreover, the combination of actions as described in connection with the present embodiments provides new information, motivation, and business outcomes that are not already present when the actions are considered individually. There is no commonly accepted definition of what constitutes an abstract idea. To the extent that the concepts described in this disclosure may be considered abstract, the claims provide a highly specific, practical, and concrete application of the abstract concepts, and also provide improvements over conventional computer-based systems that perform collision avoidance operations.

[0131] Software may include a full executable software program, a simple configuration data file, links to additional instructions, or any combination of known software types. When a computing device updates software, the update may be small or large. For example, the computing device may download a small configuration data file as part of the software, or the computing device may completely replace most or all of the current software on itself or another computing device with a new version. In some cases, the software or data, or the software and data, are encrypted, encoded, and / or otherwise compressed for reasons including security, privacy, data transfer speed, data costs, etc.

[0132] If a database structure is present in the near miss detection system described herein, the database structure may be formed of a single database or multiple databases. In some cases, hardware or software storage repositories are shared between various functions of the particular system with which they are associated. The database may be formed as part of a local system or local area network. Alternatively or additionally, the database may be formed at a remote location, such as in a dynamic "cloud" computing system that is accessible via a wide area network or other network.

[0133] Input / output (I / O) circuits and user interface (UI) modules may include serial ports, parallel ports, Universal Serial Bus (USB) ports, IEEE 802.11 transceivers and other transceivers that conform to protocols administered by one or more standards bodies, displays, projectors, printers, keyboards, computer mice, microphones, and MEMS (microelectromechanical) devices such as accelerometers.

[0134] In at least one embodiment, devices such as the airborne node 102 may communicate with other devices via communication over a network. The network may include an Internet connection or other types of local area networks (LANs) or wide area networks (WANs). Non-limiting examples of structures that enable or form part of a network include, but are not limited to, Ethernet, twisted pair Ethernet, digital subscriber loop (DSL) devices, wireless LANs, Wi-Fi, worldwide interoperability for microwave access (WiMAX), or the like.

[0135] In the present invention, memory may be used in one configuration or another. Memory may be configured to store data. Alternatively or additionally, memory may be a non-transitory computer readable medium (CRM). CRM is configured to store computing instructions executable by the processor of the airborne node 102. The computing instructions may be stored individually or as groups of instructions in a file. A file may include functions, services, libraries, etc. A file may include one or more computer programs or may be part of a larger computer program. Alternatively or additionally, each file may include data or other computational support material useful for performing the computing functions of the near miss detection system.

[0136] The terms "real-time" or "actual time" as used herein and in the appended claims are not intended to imply instantaneous processing, transmission, receipt, or otherwise, as the case may be. Instead, the terms "real-time" and "actual time" mean that an activity occurs over an acceptably short period of time (e.g., over a period of microseconds or milliseconds) and that the activity may be performed continuously (e.g., collecting and analyzing video to detect or otherwise determine near-miss conditions). Examples of non-real-time activities include those that occur over an extended period of time (e.g., hours or days) or those that occur based on user intervention or direction or other activity.

[0137] Unless otherwise specified, technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein can also be used in the practice or testing of the present invention, a limited number of exemplary methods and materials are described herein.

[0138] In the present invention, when an element (e.g., a component, circuit, device, apparatus, structure, layer, material, etc.) is described as being "on" another element, "coupled" to another element, or "connected" to another element, the element may be directly disposed on, directly coupled to, or directly connected to, or intervening elements may be present. In contrast, when an element is referred to as being "directly on," "directly coupled to," or "directly connected to" another element, there are no intervening elements present.

[0139] The terms "including" and "comprising," as well as their derivatives and variations, are to be interpreted in an open and inclusive sense without restriction in all syntactic contexts (e.g., "including but not limited to"). The term "or" is inclusive and means "and / or." The term "related to," as well as its derivatives, can be understood as including, contained within, interconnected, coupled, communicable, cooperating, sandwiching, juxtaposing, adjacent, joined together, having, having a characteristic, or having a similar meaning.

[0140] References throughout this specification to "one embodiment" and variations thereof mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment. Thus, the appearances of the phrase "in one embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0141] In the present invention, terms such as first, second, etc. may be used to describe various elements, but these elements are not limited by these terms unless the context clearly requires such a limitation. These terms are used only to distinguish one element from another element. For example, a first machine can be called a second machine, and similarly, a second machine can be called a first machine, without departing from the scope of the concept of the present invention.

[0142] In the present invention, the singular forms "a," "an," and "the" include plural references unless the content and context clearly dictate otherwise. The conjunctions "and" and "or" are generally employed in their broadest sense, including "and / or," unless the content and context clearly dictate otherwise. The "and" and "or" configurations shown herein as "and / or" encompass embodiments including all of the associated elements and at least one more alternative embodiments including less than all of the associated elements.

[0143] In this disclosure, sequential lists utilize a comma, which may also be known as the Oxford comma, Harvard comma, serial comma, or other similar terms. Such lists are intended to connect words, clauses, or sentences such that whatever follows the comma is also included in the list.

[0144] It is understood that the term "based on" and / or similar terms are not intended to necessarily denote an exclusive set of factors, but rather allow for the existence of additional factors not necessarily expressly stated. Of course, for all of the above, the particular context and / or usage of the description will provide useful guidance regarding the inferences to be drawn. It should be noted that the present specification merely provides one or more exemplary examples, and the subject matter of the present disclosure is not limited to these exemplary examples or examples.

[0145] In the description herein, specific details are set forth to provide a thorough understanding of various exemplary embodiments. It should be understood that various modifications to the embodiments will be readily apparent to those skilled in the art, and that the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Furthermore, in the following description, numerous details are set forth for the purposes of explanation. However, those skilled in the art will appreciate that the embodiments may be practiced without the use of these specific details. In other examples, well-known structures and processes are not shown or are omitted to avoid obscuring the description with unnecessary details. Thus, the present invention is not intended to be limited to the embodiments shown, but instead is to be accorded the widest scope consistent with the principles and features described herein. Thus, these and other changes can be made to the embodiments in light of the above detailed description. In general, the terms used in the appended claims should not be construed to limit the claims to the specific embodiments described in the specification, but should be construed to include all possible embodiments along with the full scope of equivalents to which such claims are entitled. The scope of the claims is therefore not limited by the present disclosure.

Claims

1. A method performed by one or more computing devices (102, 172), comprising: identifying a target object (114, 116) in a sequence of images (130a-130d) captured within a field of view of a camera (150), each image (130) in the sequence comprising a set of pixels, one or more pixels of the set of pixels corresponding to grid points of a geographic coordinate grid; determining the motion of the object through the sequence of images; determining a direction of movement of the target object through the grid points of the geographic coordinate grid based on the determined movement; determining the variation in motion direction for each of the target objects to generate a plurality of variations in motion direction; determining a median vector (134) of movement directions for each of a plurality of vector points corresponding to the object on the geographic coordinate grid to generate a plurality of vectors; determining a plurality of candidate locations on grid points of the geographic coordinate grid for placing a virtual tripwire (138) for monitoring movement of at least a portion of the object of interest through the sequence of images, the virtual tripwire including a virtual segment extending in a direction substantially perpendicular to a direction of movement of at least one of the objects of interest; automatically selecting specific candidate locations for placing the virtual tripwire by computationally sliding a sliding window (146) across the grid points of the geographic coordinate grid, identifying locations that result in a minimum level of variation or a maximum level of variation in the direction of movement of at least one of the target objects at the grid points that overlap the sliding window; A method comprising:

2. 2. The method of claim 1, wherein automatically selecting a particular candidate location for placing the virtual tripwire (138) comprises computationally sliding a sliding window (146) across grid points of the geographic coordinate grid to identify a location that results in a minimum of variation in the direction of movement of at least one of the target objects (114, 116) at the grid points that overlap the sliding window.

3. The method of claim 2, further comprising the step of positioning the virtual tripwire (138) at positions within the sliding window (146) across selected lattice points of the geographic coordinate grid.

4. The method of claim 1, wherein the sequence of images (130a-130d) is acquired by sampling video from the camera (150).

5. The method of claim 1 , wherein the target objects (114, 116) include at least one of a vehicle, a bicycle, and a pedestrian.

6. 2. The method of claim 1, further comprising estimating a velocity of at least one of the objects across the virtual tripwire for a predetermined time interval based on respective positions of pixels corresponding to at least one of the objects in successive images.

7. The method of claim 1 , further comprising counting the number of said objects (114, 116) that cross said virtual tripwire (138) for a predetermined time interval.

8. The method of claim 1 , further comprising determining geographic coordinates for a boundary perimeter (124) for each of the objects of interest (114, 116).

9. 1. A system comprising: an airborne node (102) including a video camera (150) and a first processor (156a); and a computing device (172) located remotely from the airborne node and including a second processor (156b), The first processor, Operable to identify a target object (114, 116) in a sequence of images (130a-130d) captured within a field of view of the video camera, each image in the sequence comprising a set of pixels, one or more pixels in the set of pixels corresponding to grid points of a geographic coordinate grid; Operable to determine a boundary perimeter (124) for the object in at least one image of the sequence of images, each of the boundary perimeters surrounding a corresponding object; operative to determine a set of geographic coordinates corresponding to grid points on the geographic coordinate grid, the set of geographic coordinates representing points on or within the boundary perimeter for the object of interest; Operable to transmit at least one message including said set of geographic coordinates; The second processor comprises: Operable to receive the at least one message from the first processor; Operable to determine movement of the object through the sequence of images; Operable to determine a direction of movement of the target object through the grid points based on the determined movement; Operable to determine the variation in movement direction for each of the target objects to generate a plurality of variations in movement direction; Operable to determine a median vector of movement directions (134) for each of a plurality of vector points corresponding to the object on the geographic coordinate grid to generate a plurality of vectors; and determining a plurality of candidate locations for placing a virtual tripwire (138) on at least a portion of the set of grid points for monitoring movement of at least a portion of the target object through the sequence of images, the virtual tripwire including a virtual segment extending in a direction substantially perpendicular to a direction of movement of at least one of the target objects; and operable to automatically select a particular candidate location for placing the virtual trip wire by computationally sliding a sliding window (146) across the grid points of the geographic coordinate grid, identifying a location that results in a minimum level of variation or a maximum level of variation in the direction of movement of at least one of the target objects at the grid points that overlap the sliding window. system.

10. 10. The system of claim 9, wherein the second processor is operable to position the virtual tripwire at locations within the sliding window across selected lattice points of the geographic coordinate grid.

Images