Predictive adjustment of distributed surveillance video data capture using graphics mapping network

Abstract

Systems, cameras, and methods for predictive adjustment of multi-camera surveillance video data capture based on a sub-region graphics mapping network are described. Different groups of networked cameras are deployed throughout the area, and are represented in a sub-area graphic mapping based on the physical locations of the cameras, and are associated with each other by a main graphic mapping. Each group of networked cameras includes a master camera that is connectable to master cameras in other groups directly and / or through a control center to share video capture update messages in response to regional video alarm events.

Description

Technical Field

[0001] The present disclosure relates generally to video surveillance systems, and more particularly to video surveillance systems configured to control video capture parameters based on other networked cameras. Background Art

[0002] Network-based video surveillance systems are a growing computing application in both commercial and personal markets. Some video surveillance systems may include one or more cameras communicatively connected to a server (such as a network video recorder) via a wired interface, a wired or wireless local area network, or a wired or wireless wide area network (such as the Internet). As the cameras record video, the video is forwarded to a server system where it is stored and / or analyzed for subsequent retrieval. In some configurations, the video may be recorded in the camera's onboard memory, transferred to the server, or not. Client or user systems are communicatively connected to the server system and / or the cameras to request, receive, and display streams of recorded video data and / or related alerts and analytics.

[0003] Increasingly, video surveillance applications use object detection and recognition, such as facial recognition, based on discrete objects identified in a video stream. Due to the high storage costs of surveillance applications (including continuous surveillance), cameras may include features to selectively capture high-quality video. For example, upon detection of an object, motion, or similar visual condition, the camera may modify its video capture operation to capture high-quality data. These modifications to the video capture operation may include initiating data capture (such as when a light or motion sensor provides visual conditions), changing the video capture rate from a lower video capture rate to a higher video capture rate, moving the camera's field of view using zoom and / or pan-tilt-zoom (PTZ) capabilities, and / or changing the data storage destination.

[0004] While such trigger conditions reduce video data usage, they may also delay object detection, miss fast-moving objects, miss data objects that are more difficult to detect at lower capture rates, miss objects within the PTZ range that do not pass through the current field of view, and / or miss early portions of video events of interest (which may include critical angles, lighting, or other image features to support object recognition).

[0005] Systems and methods for predictively modifying video capture operations to selectively capture an object of interest based on data from another networked camera in a neighborhood may be advantageous. A reliable and efficient way to use video data from networked cameras in the same area to initiate modifications to video capture operations in different groups of networked cameras before an object is detectable in the fields of view of those cameras may be desired. Summary of the Invention

[0006] Various aspects are described for predictively modifying video capture operations in a multi-camera video surveillance network consisting of multiple networked camera groups, particularly using video data from one camera group to initiate changes to video capture parameters of another camera group based on a graphical mapping of spatial relationships between the camera groups.

[0007] One general aspect includes a system comprising a first camera including a network interface configured to communicate with a first plurality of networked cameras, wherein the first plurality of networked cameras includes the first camera and a second camera of a second plurality of networked cameras. The system also includes at least one controller in communication with the first plurality of networked cameras and the second camera, the at least one controller configured to, alone or in combination: receive a first video capture update message from at least one camera of the first plurality of networked cameras based on a first graphical mapping of the first plurality of networked cameras; determine a zone video alarm event based on the first video capture update message; and send a second video capture update message to the second camera based on the zone video alarm event, wherein the second camera is configured to send a third video capture update message to at least one other camera of the second plurality of networked cameras in response to the second video capture update message and based on a second graphical mapping of the second plurality of networked cameras.

[0008] Specific implementations may include one or more of the following features. The system may include the first plurality of networked cameras, wherein each camera in the first plurality of networked cameras is configured to, in response to receiving the first video capture update message: determine at least one event parameter from the first video capture update message; modify the video capture operation of the camera in the first plurality of networked cameras based on the at least one event parameter; and capture video data using the camera based on the modified video capture operation. The system may include the second plurality of networked cameras, wherein each camera in the second plurality of networked cameras is configured to, in response to receiving the third video capture update message: determine a regional video capture modification from the third video capture update message; modify the video capture operation of the camera in the second plurality of networked cameras based on the regional video capture modification; and capture video data using the camera based on the modified video capture operation. The first graphical map may include: a first coordinate position system; a first set of coordinates and a primary indicator for the first camera; a second set of coordinates for each other camera in the first plurality of networked cameras; and a third set of coordinates for each child node between cameras in the first plurality of networked cameras, wherein each child node is mapped to at least two parent nodes of a corresponding camera in the first plurality of networked cameras. The second graphical mapping may include: a second coordinate location system; a fourth set of coordinates and a primary indicator for the second camera; a fifth set of coordinates for each other camera in the second plurality of networked cameras; and a sixth set of coordinates for each child node between cameras in the second plurality of networked cameras, wherein each child node is mapped to at least two parent nodes of a corresponding camera in the second plurality of networked cameras. The first coordinate location system and the second coordinate location system may be different. At least one camera in the first plurality of networked cameras may include a first object detector configured to detect a first object of interest in video data captured by the camera; at least one camera in the second plurality of networked cameras may include a second object detector configured to detect a second object of interest in video data captured by the camera; the first video capture update message may be responsive to detecting the first object of interest; the second video capture update message may include at least one parameter of the first object of interest; and the at least one camera in the second plurality of networked cameras may be configured to modify at least one operating parameter of the second object detector in response to the third video capture update message to detect the first object of interest in video data captured by the camera. The second video capture update message may include at least one parameter selected from the following: a sample of video data of the area video alarm event; the coordinates of the camera of the first plurality of networked cameras that captured the area video alarm event; and the direction of travel of the object of interest determined for the area video alarm event.The at least one controller may be further configured to determine the second plurality of networked cameras and the second camera based on the selected at least one parameter, either alone or in combination. The system may include: the first plurality of networked cameras corresponding to a first sub-area mapped to a first set of physical locations; the second plurality of networked cameras corresponding to a second sub-area mapped to a second set of physical locations; and a third plurality of networked cameras corresponding to a third sub-area mapped to a third set of physical locations. The first graphical map is based on the first set of physical locations; the second graphical map is based on the second set of physical locations; the third graphical map corresponds to the third plurality of networked cameras and is based on the third set of physical locations; and the at least one controller may be further configured to select between the second plurality of networked cameras and the third plurality of networked cameras to receive the second video capture update message based on the video zone alarm event, either alone or in combination. The at least one controller may be further configured to: determine a priority value for a zone video event alarm based on the zone video alarm event; compare the priority value to at least one event priority threshold; and, in response to the priority value satisfying the at least one event priority threshold, select at least one networked camera from a group of the plurality of networked cameras including the second plurality of networked cameras. The second video capture update message may include the priority value; and the second plurality of networked cameras may be further configured to selectively modify video capture operations based on the priority value. The system may include a computer system remote from the first plurality of networked cameras and the second plurality of networked cameras, wherein the computer system may include: a plurality of graphical maps corresponding to a plurality of sub-regional sets of networked cameras; a network interface configured to communicate with a master camera in each of the plurality of sub-regional sets; and the at least one controller. The first plurality of networked cameras may be a first sub-regional set corresponding to a first set of physical locations monitored by the computer system; the first camera may be a master camera of the first plurality of networked cameras; the second plurality of networked cameras may be a second sub-regional set corresponding to a second set of physical locations monitored by the computer system; and the second camera may be a master camera of the second plurality of networked cameras.

[0009] Another general aspect includes a computer-implemented method comprising: receiving a first video capture update message from a first camera in a first plurality of networked cameras and from at least one camera in the first plurality of networked cameras based on a first graphical mapping of the first plurality of networked cameras; determining a regional video alarm event based on the first video capture update message; sending a second video capture update message to a second camera in a second plurality of networked cameras based on the regional video alarm event; and sending a third video capture update message by the second camera to at least one other camera in the second plurality of networked cameras based on a second graphical mapping of the second plurality of networked cameras and in response to the second video capture update message.

[0010] Specific implementations may include one or more of the following features. The computer-implemented method may include: in response to receiving the first video capture update message and by at least one camera in the first plurality of networked cameras: determining at least one event parameter from the first video capture update message; modifying the video capture operation of the camera in the first plurality of networked cameras based on the at least one event parameter; and capturing video data using the camera based on the modified video capture operation. The computer-implemented method may include: in response to receiving the third video capture update message and by at least one camera in the second plurality of networked cameras: determining a regional video capture modification from the third video capture update message; modifying the video capture operation of the camera in the second plurality of networked cameras based on the regional video capture modification; and capturing video data using the camera based on the modified video capture operation. The first graphical map may include: a first coordinate position system; a first set of coordinates and a primary indicator for the first camera; a second set of coordinates for each other camera in the first plurality of networked cameras; and a third set of coordinates for each child node between cameras in the first plurality of networked cameras, wherein each child node is mapped to at least two parent nodes of a corresponding camera in the first plurality of networked cameras. The second graphical mapping may include: a second coordinate location system; a fourth set of coordinates and a primary indicator for the second camera; a fifth set of coordinates for each other camera in the second plurality of networked cameras; and a sixth set of coordinates for each child node between cameras in the second plurality of networked cameras, wherein each child node is mapped to at least two parent nodes of a corresponding camera in the second plurality of networked cameras. The first coordinate location system and the second coordinate location system may be different. The computer-implemented method may include: detecting, by at least one camera in the first plurality of networked cameras, a first object of interest in video data captured by the camera using a first object detector; and modifying, by at least one camera in the second plurality of networked cameras, in response to the third video capture update message, at least one operating parameter of a second object detector to detect the first object of interest in video data captured by the camera, wherein the first video capture update message is responsive to detecting the first object of interest, and the second video capture update message may include at least one parameter of the first object of interest. The computer-implemented method may include: including in the second video capture update message at least one parameter selected from: a sample of video data of the area video alarm event; the coordinates of a camera among the first plurality of networked cameras that captured the area video alarm event; and a direction of travel of an object of interest determined for the area video alarm event; and determining the second plurality of networked cameras and the second camera based on the at least one selected parameter.The computer-implemented method may include: selecting between the second plurality of networked cameras and a third plurality of networked cameras to receive the second video capture update message based on the video zone alarm event, wherein: the first plurality of networked cameras corresponds to a first sub-area mapped to a first set of physical locations; the second plurality of networked cameras corresponds to a second sub-area mapped to a second set of physical locations; the third plurality of networked cameras corresponds to a third sub-area mapped to a third set of physical locations; the first graphical map is based on the first set of physical locations; the second graphical map is based on the second set of physical locations; and the third graphical map corresponds to the third plurality of networked cameras and is based on the third set of physical locations. The computer-implemented method may include: determining a priority value for a zone video event alarm based on the zone video alarm event; comparing the priority value to at least one event priority threshold; selecting at least one networked camera from a group of the second plurality of networked cameras, in response to the priority value satisfying the at least one event priority threshold, wherein the second video capture update message includes the priority value; and selectively modifying video capture operations by at least one of the second plurality of networked cameras based on the priority value.

[0011] Yet another general aspect includes a system comprising: a first plurality of networked cameras including a first camera; a second plurality of networked cameras including a second camera; at least one processor; at least one memory; a device for receiving a first video capture update message from the first camera and from at least one camera of the first plurality of networked cameras based on a first graphical mapping of the first plurality of networked cameras; a device for determining a regional video alarm event based on the first video capture update message; a device for sending a second video capture update message to the second camera based on the regional video alarm event; and a device for sending a third video capture update message by the second camera based on a second graphical mapping of the second plurality of networked cameras and in response to the second video capture update message to at least one other camera of the second plurality of networked cameras.

[0012] Various embodiments advantageously apply the teachings of computer-based surveillance systems to improve the functionality of such computer systems. Various embodiments include operations that overcome or at least reduce problems previously encountered in surveillance systems, thereby being more efficient and / or more cost-effective than other surveillance systems. Specifically, various embodiments disclosed herein include hardware and / or software that improves the selective capture of surveillance video data by using video data from other camera groups in an area to trigger modifications to video capture operations based on graphical mapping relationships between camera groups. Thus, the embodiments disclosed herein provide various improvements to network-based video surveillance systems.

[0013] It should be understood that the language used in this disclosure has been primarily selected for readability and instructional purposes and does not limit the scope of the subject matter disclosed herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A computer-based monitoring system is schematically illustrated.

[0015] Figure 2 Schematically illustrates a system configured to use predictive adjustments to modify video capture operations (such as may be performed by Figure 1 Example graphical mapping and camera configuration for multiple networked cameras used in a computer-based surveillance system.

[0016] Figure 3 Schematically illustrates Figure 1 Some elements of a computer-based monitoring system.

[0017] Figure 4 Schematically illustrates a graphical map of areas and corresponding sub-areas monitored by multiple groups of networked cameras and a method for Figures 1 to 3 The computer-based monitoring system is used together with some elements of the computer-based monitoring management system.

[0018] Figure 5A is a flow chart of an example method for sending alerts based on a graphical mapping relationship between cameras.

[0019] Figure 5B is a flow chart of an example method for modifying video capture based on event parameters in an alert.

[0020] Figure 6 is a flow chart of an example method for predicting modifications to video capture operations between networked cameras based on video events detected by one camera group that modify the operations of another camera group.

[0021] Figure 7 is a flow chart of an example method for modifying video capture operations based on a video capture update message based on an event detected by another camera group.

[0022] Figure 8 is a flow chart of an example method of using object detector parameters from one camera group to selectively modify the operation of another camera group in response to a detected object of interest.

[0023] Figure 9 is a flow chart of an example method for generating and using regional video capture update messages. DETAILED DESCRIPTION

[0024] Figure 1An embodiment of an example video surveillance system 100 is shown in which a plurality of cameras 110 are interconnected with a network video recorder 130 to display surveillance video on a user device 170. Although some example features are illustrated, various other features are not shown for the sake of brevity and to not obscure relevant aspects of the example embodiments disclosed herein. In some embodiments, the cameras 110, the network video recorder 130, and the user device 170 are computer-based components that can be interconnected via a network 102. Additional components, such as a network video server 160 and / or a network video storage device 162, can also be connected to the network 102. In some embodiments, one or more cameras, such as Figure 1 The cameras 110.5-110.n) in FIG10 can be directly connected to the network video recorder 130 without communicating through the network 102. Similarly, in an alternative embodiment (not shown), the user device 170 can be directly connected to the network video recorder 130.

[0025] In some embodiments, one or more networks 102 can be used to communicatively interconnect the various components of the monitoring system 100. For example, each component (such as the camera 110, the network video recorder 130, the external storage device 140.n, the network video server 160, the network video storage device 162, and / or the user device 170) can include one or more network interfaces and corresponding network protocols for communicating via the network 102. The network 102 can include wired and / or wireless networks (e.g., any number and / or configuration of public and / or private computer networks) that can be coupled in a suitable manner for transmitting data. For example, the network 102 can include any of a variety of conventional data communication networks, such as a local area network (LAN), a wide area network (WAN), a telephone network such as the public switched telephone network (PSTN), an intranet, the Internet, or any other suitable communication network or combination of communication networks. In some embodiments, the network 102 can include multiple different networks, subnetworks, and / or virtual private networks (VPNs), which can be used to restrict communication between specific components. For example, camera 110 may be located on a limited access network such that video data and control data may only be transmitted between camera 110 and network video recorder 130 , thereby enabling network video recorder 130 to control access to camera 110 and its video data.

[0026] The camera 110 may comprise an analog or digital camera connected to an encoder that generates an encoded video stream having a defined resolution, aspect ratio, and video encoding format. In some embodiments, the camera 110 may comprise an Internet Protocol (IP) camera configured to encode its corresponding video stream and stream it to a network video recorder 130 via the network 102. In some embodiments (not shown), the encoder may reside in the network video recorder 130. In some embodiments, the camera 110 may be configured to receive audio data via an integrated or connected microphone (not shown) and include an embedded, synchronized, and / or independent audio stream with its corresponding video stream. In some embodiments, the camera 110 may comprise an image sensor 112, a processor or central processing unit (CPU) 114, a memory 116, an encoder 118, an audio channel 120, a control circuit 122, and / or a network interface 126. In some embodiments, the camera 110 may include onboard analytics, such as a video analytics subsystem 124. In some configurations, networked cameras 110 may be configured into multiple groups, where each group is associated with a set of physical locations in a sub-area of ​​the area being monitored. For example, cameras 110.1-110.4 may be configured as a first group monitoring one sub-area, while cameras 110.5-110.n may be configured as a second group monitoring a different sub-area.

[0027] For example, the image sensor 112 may include a solid-state device configured to capture light waves and / or other electromagnetic waves and convert the light into an image typically composed of colored pixels. The image sensor 112 may determine the base image size, resolution, bandwidth, depth of field, dynamic range, and other parameters of the captured video image frame. The image sensor 112 may include a charge coupled device (CCD), a complementary metal oxide semiconductor (CMOS), and / or other image sensor devices of various sensor sizes and aspect ratios. In some embodiments, the image sensor 112 may be paired with one or more filters such as an infrared (IR) blocking filter to modify the light received by the image sensor 112 and / or processed by the camera 110. For example, the IR blocking filter may be selectively enabled or disabled for different image capture use cases. In some embodiments, one or more cameras 110 may include more than one image sensor and associated video data paths. In some embodiments, multiple image sensors are supported by the same circuit board and / or processing subsystem including CPU 114 , memory 116 , encoder 118 , audio channel 120 , control circuitry 122 , analysis subsystem 124 , and / or network interface 126 .

[0028] Digital video data from image sensor 112 may be received by processor 114 for (temporary) storage and processing in memory 116 and / or encoded by encoder 118. Processor 114 may include any type of conventional processor or microprocessor that interprets and executes instructions. In some embodiments, processor 114 may include a neural network processor, such as one used by analysis subsystem 124 to support object recognition or other onboard analytics. Memory 116 may include random access memory (RAM) or another type of dynamic storage device that stores information and instructions for execution by processor 114, and / or read-only memory (ROM) or another type of static storage device that stores static information and instructions for use by processor 114, and / or any suitable storage element, such as a solid-state memory element. Memory 116 may store basic input / output system (BIOS), firmware, and / or operating system instructions used to initialize and execute instructions and processes for camera 110. Encoder 118 may encode the video stream received by image sensor 112 using a variety of possible digital encoding and / or compression formats. In some embodiments, the encoder 118 can use a compressed video format to reduce the storage size and network bandwidth necessary to store and transmit the original video stream. For example, the encoder 118 can be configured to encode the video data into Joint Photographic Experts Group (JPEG), Moving Picture Experts Group (MPEG)-2, MPEG-4, Advanced Video Coding (AVC) / H.264, and / or other video coding standards or proprietary formats. In some configurations, the settings used by the encoder 118 can be modified in response to video capture update messages received by the camera 110. The camera 110 may include an audio channel 120 that is configured to capture audio data to be processed and encoded with the image data in the resulting video stream and / or as a separate audio data stream.

[0029] The control circuitry 122 may include control circuitry for managing the physical position of the camera 110. In some embodiments, the camera 110 may be a pan-tilt-zoom (PTZ) camera capable of remote orientation and zoom control. The control circuitry 122 may be configured to receive motion commands via the network interface 126 and / or via another interface (such as a dedicated remote control interface, such as a short-range infrared signal, Bluetooth, etc.). For example, the network video recorder 130 and / or the user device 170 may be configured to send PTZ commands to the control circuitry 122, which converts these commands into motor position control signals for a plurality of actuators used to control the position of the camera 110. In some embodiments, the control circuitry 122 may include logic for automatically responding to movement or other triggers detected by the image sensor 112 to redirect the camera 110 toward the source of the movement or other trigger. For example, an auto-tracking feature may be embodied in firmware that enables the camera to estimate the size and position of an object based on changes in pixels in the raw video stream from the image sensor 112 and adjust the camera's position to follow the moving object, returning to a default position when motion is no longer detected. Similarly, an auto-capture feature may be embodied in firmware that enables the camera to determine and bind to an object based on an object detection algorithm, and to center and zoom on the object to improve image size and quality. In some embodiments, the control circuitry 122 may include logic for virtual PTZ or ePTZ that enables a high-resolution camera to digitally zoom and pan to portions of an image collected by the image sensor 112 without the camera physically moving. In some embodiments, the control circuitry 122 may include software and one or more application protocol interfaces (APIs) for enabling remote devices to control additional features and capabilities of the camera 110. For example, the control circuitry 122 can enable the network video recorder 130, another camera 110, and / or user device 170 to configure video formats, enable and disable filters, set motion and / or audio detection, auto-tracking, and similar features, and / or initiate video data streaming. In some embodiments, one or more systems can provide PTZ position control signals (and / or PTZ positioning commands that the control circuitry 122 converts into PTZ position control signals) via an API. In some configurations, the control circuitry 122 can modify the camera position or other features in response to a video capture update message received by the camera 110.

[0030] In some embodiments, camera 110 can include a video analytics subsystem 124 configured for onboard video analytics. For example, video analytics subsystem 124 can be configured to use CPU 114 and memory 116 to perform at least a portion of video analytics on video data captured by camera 110. In some embodiments, video analytics subsystem 124 can be configured to operate similarly to video analytics subsystem 156 in network video recorder 130, as further described below, and embody one or more analytics engines and / or analytics model libraries. In some embodiments, video analytics subsystem 124 can be configured to support object detection, classification, and / or recognition. For example, video analytics subsystem 124 can receive a real-time video data stream from sensor 112 and / or encoder 118, detect video events, and instruct another camera to modify video capture operations before an object of interest enters the other camera's field of view. In some configurations, settings and reference data used by video analytics subsystem 124 can be modified in response to video capture update messages received by camera 110.

[0031] The network interface 126 may include one or more wired or wireless connections to the network 102 and / or a dedicated camera interface of the network video recorder 130. For example, the network interface 126 may include an Ethernet jack and a corresponding protocol for IP communication with the network video recorder 130. In some embodiments, the network interface 126 may include a Power over Ethernet (PoE) connection to the network video recorder 130 or another camera access point. PoE allows power and network data for the camera 110 to travel on the same wires. In some embodiments, the network interface 126 may enable the IP camera to be configured as a network resource with an IP address accessible on a LAN, WAN, or the Internet. For example, the network video recorder 130 and / or the user device 170 may be configured to selectively receive video from the camera 110 from any location connected to the Internet using Internet addressing and security protocols.

[0032] The network video recorder 130 may include a computer system configured as a video storage device to record video streams from the cameras 110. For example, the network video recorder 130 may be configured to receive a video stream from each of the cameras 110 for storage, analysis, and / or display via the user device 170. In some embodiments, the cameras 110 may transmit encoded video streams based on raw image data collected from their respective image sensors 112, with or without video data compression. A single video stream may be received from each camera 110, and the network video recorder 130 may be configured to receive video streams from all connected cameras in parallel, as network bandwidth and processing resources permit.

[0033] The network video recorder 130 may include a housing and a bus that interconnects at least one processor or CPU 132, at least one memory 134, at least one storage device 140, and at least one interface (such as a camera interface 136, a network interface 138, and / or a storage interface 142). The housing (not shown) may include a casing for mounting the various subcomponents of the network video recorder 130, locating any physical connectors used for the interfaces, and protecting the subcomponents. Some housings may be configured for installation within a rack system. The bus (not shown) may include one or more conductors that allow communication between the components of the network video recorder 130. The processor 132 may include any type of processor or microprocessor that interprets and executes instructions or operations. The memory 134 may include random access memory (RAM) or another type of dynamic storage device that stores information and instructions executed by the processor 132 and / or read-only memory (ROM) or another type of static storage device that stores static information and instructions used by the processor 132 and / or any suitable storage element.

[0034] In some embodiments, the network video recorder 130 may include a camera interface 136 configured to connect to one or more cameras 110. For example, the camera interface 136 may include multiple Ethernet ports and support protocols compatible with the PoE standard for connecting to the cameras 110.5-110.n. In some embodiments, the camera interface 136 may include a PoE network switch for providing power to the connected cameras and routing data packets (such as control data and video data) to and from the cameras 110.5-110.n. In some embodiments, the network video recorder 130 may not include a dedicated camera interface 136 and may use a network interface 138 to communicate with the cameras 110 via the network 102.

[0035] Network interface 138 may include one or more wired or wireless network connections to network 102. Network interface 138 may include a physical interface (such as an Ethernet port) and associated hardware and software protocols for communicating over network 102 (such as a network interface card).

[0036] The storage device 140 may include one or more non-volatile memory devices configured to store video data, such as a hard disk drive (HDD) or a solid-state drive (SSD). In some embodiments, the storage device 140 is or includes multiple solid-state drives. In some embodiments, the network video recorder 130 may include an internal storage device 140.1 and an expandable storage device that connects additional storage devices 140.n via a storage interface 142. Each storage device 140 may include a non-volatile memory (NVM) or device controller 144 based on computing resources (processor and memory) and multiple NVM or media devices 146 for data storage (e.g., one or more NVM devices, such as one or more flash memory devices). In some embodiments, a corresponding data storage device 140 in the one or more data storage devices includes one or more NVM controllers, such as a flash memory controller or a channel controller (e.g., for a storage device having NVM devices in multiple memory channels). In some embodiments, the storage devices 140 may each be enclosed in a housing, such as a multi-component sealed housing having a defined form factor and a port and / or connector for interconnecting with the storage interface 142. The storage device 140.1 and each expansion storage device 140.n may be the same storage device type or different storage device types.

[0037] In some embodiments, the corresponding data storage device 140 may include a single non-volatile storage media device, while in other embodiments, the corresponding data storage device 140 includes multiple media devices. In some embodiments, the media device includes NAND-type flash memory or NOR-type flash memory. In some embodiments, the storage device 140 may include one or more hard disk drives. In some embodiments, the storage device 140 may include a flash memory device, which in turn includes one or more flash memory dies, one or more flash memory packages, one or more flash memory channels, etc. However, in some embodiments, one or more data storage devices in the data storage device 140 may have other types of non-volatile data storage media (e.g., phase change random access memory (PCRAM), resistive random access memory (ReRAM), spin transfer torque random access memory (STT-RAM), magnetoresistive random access memory (MRAM), etc.).

[0038] In some embodiments, each storage device 140 includes a device controller 144, which includes one or more processing units (sometimes also referred to as CPUs or processors or microprocessors or microcontrollers) configured to execute instructions in one or more programs. In some embodiments, the one or more processors are shared by one or more components within the functionality of the device controller and in some cases beyond its functionality. The media device 146 is coupled to the device controller 144 via connections, which typically transmit commands in addition to data, and optionally transmit metadata, error correction information, and / or other information in addition to data values ​​to be stored in the media device and data values ​​read from the media device 146. The media device 146 may include any number (i.e., one or more) of memory devices, including but not limited to non-volatile semiconductor memory devices, such as flash memory devices. In some embodiments, the media device 146 may include a NAND or NOR flash memory device composed of single-level cells (SLC), multi-level cells (MLC), or triple-level cells or more.

[0039] The storage interface 142 may include a physical interface for connecting to one or more external storage devices using an interface protocol that supports storage device access. For example, the storage interface 142 may include a Peripheral Component Interconnect Express (PCIe), Serial Advanced Technology Attachment (SATA), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), Universal Serial Bus (USB), Firewire, or a similar storage interface connector that supports storage protocol access to the storage device 140.n. In some embodiments, the storage interface 142 may include a wireless data connection with sufficient bandwidth for video data transmission. Depending on the configuration and protocol used by the storage interface 142, the storage device 140.n may include a corresponding interface adapter, firmware, and / or protocol for receiving, managing, and responding to storage commands from the network video recorder 130.

[0040] The network video recorder 130 may include a plurality of modules or subsystems that are stored and / or instantiated in the memory 134 for execution as instructions or operations by the processor 132. For example, the memory 134 may include a camera control subsystem 150 configured to control the camera 110. The memory 134 may include a video capture subsystem 152 configured to receive a video stream from the camera 110. The memory 134 may include a video storage subsystem 154 configured to store received video data in the storage device 140 and / or the network video storage 162. The memory 134 may include a video analysis subsystem configured to analyze the video stream and / or video data for defined events, such as motion, recognized objects, recognized faces, and combinations thereof. The memory 134 may include a video display subsystem configured to selectively display the video stream on the user device 170, which may be attached to the network video recorder 130 or remotely connected via the network 102.

[0041] In some embodiments, the monitoring system 100 may include one or more remote and / or cloud-based resources to support the functionality of the network video recorder 130 and / or user device 170. For example, the monitoring system 100 may include a network video server 160, such as a cloud-based server system, configured to host some, all, or a selected portion of the functionality of the network video recorder 130. As another example, the monitoring system 100 may include a network video storage device 162, such as a cloud-based network-attached storage system or a distributed storage system, to store live and / or archived video data, supplementing and / or replacing the storage device 140. In some embodiments, most of the functionality described above for the network video recorder 130 may reside within the network video recorder 130, and selected functionality may be configured to utilize additional resources within the network video server 160 and / or network video storage device 162. For example, network video server 160 may be configured to support specialized and / or processing-intensive event detection algorithms to supplement video analytics subsystem 156 , and / or network video storage 162 may be configured to support archiving of inactive video data for long-term storage.

[0042] User device 170 can be any suitable computer device, such as a computer, computer server, laptop, tablet device, netbook, internet kiosk, personal digital assistant, mobile phone, smartphone, gaming device, or any other computing device. User device 170 is sometimes referred to as a host, client, or client system. In some embodiments, user device 170 can host or instantiate one or more applications for interfacing with surveillance system 100. For example, user device 170 can be a personal computer or mobile device running a surveillance monitoring and management application that is configured to provide a user interface for network video recorder 130. In some embodiments, user device 170 can be configured to access camera 110 and / or its corresponding video stream through network video recorder 130 and / or directly through network 102. In some embodiments, one or more functions of network video recorder 130 can be instantiated in user device 170, and / or one or more functions of user device 170 can be instantiated in network video recorder 130.

[0043] User device 170 may include one or more processors or CPUs 172 for executing computing operations or instructions stored in memory 174 to access video data and other functions of network video recorder 130 via network 102. In some embodiments, processor 172 may be associated with memory 174 and input / output devices 176 for performing both video display operations and monitoring system management operations. Processor 172 may include any type of processor or microprocessor that interprets and executes instructions or operations. Memory 174 may include random access memory (RAM) or another type of dynamic storage device that stores information and instructions for execution by processor 172 and / or read-only memory (ROM) or another type of static storage device that stores static information and instructions for use by processor 172 and / or any suitable storage element. In some embodiments, user device 170 may allocate a portion of memory 174 and / or another local storage device (located in or attached to user device 170) for storing selected video data for user device 170. In some embodiments, user device 170 may include one or more input / output (I / O) devices 176. For example, a graphics display (such as a monitor and / or a touch screen display) and / or other user interface components (such as a keyboard, a mouse, function buttons, a speaker, a vibration motor, a touchpad, a pen, voice recognition, a biometric recognition mechanism, and / or any number of supplemental devices for adding functionality to the user device 170). The network interface 178 may include one or more wired or wireless network connections to the network 102. The network interface 178 may include a physical interface (such as an Ethernet port) and / or related hardware and software protocols for communicating via the network 102 (such as a network interface card, a wireless network adapter, and / or a cellular data interface).

[0044] User device 170 may include a plurality of modules or subsystems that are stored and / or instantiated in memory 174 for execution as instructions or operations by processor 172. For example, memory 174 may include a video manager 180 configured to provide a user interface for selectively navigating and displaying live, near-real-time, and / or stored video streams. Memory 174 may include an alarm manager 182 configured to provide a user interface for setting, monitoring, and displaying alarms based on video events. Memory 174 may include a camera manager 184 configured to provide a user interface for identifying, configuring, and managing cameras 110. Memory 174 may include a configuration manager 186 to provide a user interface for setting and managing system settings, user access controls, storage options, and other configuration settings for surveillance system 100. Memory 174 may include a network video recorder manager 188 configured to provide a user interface for identifying, configuring, and managing network video recorder 130 and / or multiple network video recorders. Memory 174 may include an analysis manager 190 configured to provide a user interface for selecting, training, and managing event detection algorithms for monitoring system 100 .

[0045] Figure 2 A computer-based monitoring system 200 (such as Figure 1 Schematic diagram of multiple networked cameras implemented in a surveillance system 100 (see FIG. 1 ) configured to modify video capture operations using predictive adjustments. In the example configuration shown, camera 110 is configured for onboard capture and analysis of video data, but similar functionality can be implemented by a host computer, such as the network video recorder 130 described above. In camera environment 202, three networked cameras 110.1-110.3 are located around multiple locations with interconnected pathways, such as a network of roads and / or pedestrian paths in a city, campus, park, or similar location. In one example configuration, each camera 110 may be placed on the exterior of a building or mounted to a municipal service pole or similar structure to provide video capture near a building, intersection, or other point of interest. When installed, cameras 110 have known locations with known spatial relationships to one another and a defined field of view 206. For example, a general overhead view of camera environment 202 may allow for graphical mapping of cameras 110 and other related attributes of camera environment 202 using a coordinate location system. The coordinate location system may include a two-dimensional mapping based on longitude and latitude or another XY coordinate system and a scale applied to camera environment 202. In some configurations, a three-dimensional mapping may be used, such as adding elevation, altitude, or similar Z coordinates to a two-dimensional coordinate position system. The field of view 206 may be defined based on two or more bounding vectors and, if equipped with a PTZ actuator, may include a default position and maximum range in each adjustment direction.

[0046] like Figure 2 As shown, a graphical map (using a two-dimensional coordinate position system in the example shown) can be overlaid on the camera environment 202. Each camera 110 can be considered a parent node 204. For example, the coordinate position of camera 110.1 can be abstracted as parent node 204.1, the coordinate position of camera 110.2 can be abstracted as parent node 204.2, and the coordinate position of camera 110.3 can be abstracted as parent node 204.3. The paths of travel between or among cameras 110 can be mapped, and the intersections can be abstracted as child nodes 208 (or subnodes). Each child node 208 can be a child node of each parent node to which it is connected, without passing through another parent node or requiring a turn at an intersection (child nodes are connected to parent nodes via a straight path that can pass through another child node, such as node 208.3 being a child node of parent node 204.1 but not parent node 204.2). For example, child nodes 208.1, 208.2, 208.3, and 208.5 can be child nodes of camera 110.1, child nodes 208.1, 208.2, 208.4, and 208.5 can be child nodes of camera 110.2, and child nodes 208.1, 208.3, and 208.4 can be child nodes of camera 110.3. Each path segment between nodes represents an edge in the graph map. For example, edge 210.1 extends from parent node 204.1 to child node 208.1, edge 210.2 extends from child node 208.1 to child node 208.2, and so on for edges 210.3, 210.4, 210.5, 210.6, 210.7, 210.8, 210.9, and 210.10. A possible object path can consist of each set of edges from one parent node to another parent node, typically through one or more child nodes. In some configurations, the presence of one or more child nodes between each pair of parent nodes may indicate that the cameras do not have overlapping fields of view and may not be able to capture the same object of interest simultaneously.

[0047] Cameras 110 can be deployed to monitor their respective fields of view 206 for object detection and related object detection event handling, such as alerting, selective archiving, and the like. For example, cameras 110 can be placed in their respective monitoring locations to detect an object of interest 240 (in this case, a person) as it moves along a movement path 242 into their respective fields of view 206. Other example objects of interest may include vehicles, animals, equipment, and the like. Movement path 242 may indicate the direction of the event from the camera that most recently captured the object of interest (e.g., camera 110.1). However, the presence of intersections and decision points modeled by child nodes 208 may reflect that object of interest 240 may not travel in a straight line toward another camera and may still eventually enter the other camera's field of view 206 based on available object paths. For example, object of interest 240 may enter field of view 206.2 of camera 110.2 by traveling to child node 208.2 and turning right to follow edge 210.3. Alternatively, the object of interest 240 may continue straight at subnode 208.2 and turn right at subnode 208.3 to follow edge 210.5 toward camera 110.3. However, unless camera 110.3 actively changes its actuator position and field of view 206.3, it is unlikely to capture the object of interest 240.

[0048] In some configurations, the camera 110 can be configured for video capture 212 based on different video capture rates. For example, the camera 110 can include a passive video capture rate 212.1 that includes lower quality video (e.g., lower pixel count and / or frame rate) to save storage space and / or network bandwidth during operating periods when there may not be an object of interest in the field of view. The camera 110 can include an active video capture rate 212.2 that includes higher quality video (e.g., higher pixel count and / or frame rate) to provide better video data for analysis, display, and / or storage of possible object detection events. Thus, even if the camera 110.2 is already oriented toward the most likely object travel path, it may be advantageous to proactively and predictively trigger a change in video capture 212.

[0049] Once camera 110.1 has detected an object of interest 240, it may be advantageous to predictively and selectively adjust the video capture operations of other cameras. While all cameras could be alerted to modify their video capture operations each time any of the cameras detects an object of interest, doing so would result in a waste of storage, network bandwidth, and other resources. In some configurations, competition for constrained resources in the system (such as limited network bandwidth and / or storage channels) may prevent all cameras from operating at the active capture rate 212.2 simultaneously and / or for an extended period of time. Therefore, it may be desirable to predict which cameras are most likely to have an object enter their field of view and allow those cameras to modify their video capture operations to maximize the likelihood of capturing the object without wasting resources. If camera 110.3 modifies its video capture operations before the object of interest 240 enters the field of view 206.3, such as by switching to its active capture rate, pivoting to its right-facing actuator position, prioritizing object detectors for the object type of the object of interest, and other changes to video capture operating parameters, it can maximize the chance of capturing a high-quality image of the object to support further detection, identification, surveillance, or other analysis.

[0050] Camera 110 may already be deployed with an object detector 214 configured to detect objects of interest using video data captured at its passive capture rate 212.1 and / or active capture rate 212.2. For example, object detector 214 may be trained on low-quality video to detect one or more objects of interest with a relatively low confidence threshold, thereby triggering active capture rate 212.2 and processing additional video data at higher quality, and using the object detector trained on the higher-quality data to confirm object detection, achieve classification and / or object recognition, or support other analysis. Alternatively, other sensors, such as motion sensors, audio sensors, or detection algorithms, such as video tripwires, may be used to initially trigger the camera's active capture rate 212.2 when an object of interest initially enters camera environment 202. In some configurations, additional processing of the high-quality video data may include classification of the object type and / or determination of the direction of travel of the object of interest.

[0051] Camera 110 may include trigger logic 216 for determining a video event that triggers predictive adjustments to other cameras. For example, trigger logic 216 may be based on one or more parameters of a detected object and / or subsequent processing. In some configurations, a specific type of object (such as a person, vehicle, or animal) may be a trigger condition, and a direction of motion that would cause the object to leave the field of view of the current camera may be another trigger condition. If the trigger condition is met, trigger logic 216 may access a graphical map 218 and path logic 220 to determine which other camera(s) should be modified, and in some configurations, what those modifications should be. For example, graphical map 218 may be represented as a reference table or data structure 250 and may include an entry 258 for each other camera identifier 252, child node identifier 254, and / or child node location 256 (such as coordinates in a coordinate positioning system). In some configurations, a near-edge entry or field for each child node identifier 254 and / or location 256 may indicate the beginning of an object path that can be used to determine, from the direction of the event, the path logic 220 to reach the shared child node of one or more other cameras. For example, if the current field of view, such as field of view 206.1, and the position and / or movement indication of object of interest 240 are detected near edge 210.10 of object of interest 240, camera 110.1 may determine shared child nodes 208.2, 208.1, and 208.3 in graphical map 218 for cameras 110.2 and 110.3 from reference table 250. Path logic component 220 may include a set of logic rules for determining a predicted object path based on graphical map 218 and / or reference table 250. In some configurations, camera 110.1 may select cameras 110.2 and 110.3 to receive an alert message or similar video capture update message based on the shared child node identifiers in the direction of the event.

[0052] Camera 110 may include a communication channel 222 for sending one or more messages to other cameras, such as messages updating video capture parameters to prepare for an incoming object. For example, camera 110 may use the camera identifier 252 determined from reference table 250 to be most likely to see the object next, and include a messaging protocol to address the video capture update message to that camera identifier. In some configurations, camera 110 may be an Internet Protocol (IP) camera on a public network, and communication channel 222 may be configured for peer-to-peer communication over a network interface. In some configurations, camera 110 may include a network or direct connection to a host computer, and communication channel 222 may be configured to communicate only with the host computer. In such configurations, the host computer may be configured as a router for communication between cameras and / or may be configured to process messages from a detected camera and generate corresponding video capture update messages to send to the predicted camera.

[0053] In some configurations, one camera in a camera group can be configured as the master camera, and messages can be routed to the master camera by other cameras. The master camera can also be configured for network communication with hosts and / or master cameras in other camera groups. For example, camera 110.1 can be configured as the master camera for a camera group including cameras 110.1-110.3. In some configurations, the graphical mapping data structure 250 can include a master indicator 260 associated with the camera designated as the master camera. For example, the graphical mapping data structure 250 can include a master flag appended to the camera identifier 252 of the camera designated as the master. In some configurations, except for the camera selected for changing video capture operations, each camera in the group can send any alarm messages to the master camera, and the master camera can evaluate event parameters in the alarm messages to determine whether other camera groups should be notified of a regional video alarm event. For example, the master camera can execute logic or interface with the host to evaluate event parameters and selectively send regional alarm messages, such as regional video capture update messages, to other camera groups via their respective master cameras.

[0054] Camera 110 may include an adjustment logic component 224 configured to receive video capture update messages and adjust video capture operating parameters based on such messages. For example, adjustment logic component 224 may receive and parse messages from a detection camera to determine a shared child node identifier and be able to determine changes in video capture operating parameters corresponding to an object approaching from the shared child node. In some configurations, a set of video capture operating parameters based on object type, predicted direction of entry into the field of view, and / or other parameters may be determined based on the received message and the shared child node identifier to help modify the predicted video capture of the camera. In some configurations, reference table 250 may be configured with a set of operating parameters indexed by object type and child node identifier, and the applicable set of operating parameters may be included in the message, or the object parameters in the message may allow the receiving camera to determine the set of operating parameters from a table. Adjustment logic component 224 may use the modified parameters to initiate operational changes to camera actuators, encoders, analysis, and / or storage functions in response to the message. In some configurations, the adjustment logic 224 in the receiving camera may also be configured to recognize when the camera is already engaged in high priority video capture (such as due to a different object or event of interest) and may reject predictive updates that would negatively impact the current video capture operation.

[0055] The camera 110 may include non-volatile memory 230 configured to store video data captured by the camera. As described above, the on-camera memory may include a limited capacity, and the camera 110 may send the video data via the communication channel 222 for storage in other memory systems, which may include the memory of other cameras, a host computer, and / or a video storage device accessible by the host computer. In some configurations, modification of the video capture operating parameters may include improving the quality of the captured video (in terms of both sampling rate and encoding) and correspondingly increasing the amount of storage capacity used to store such video.

[0056] Figure 3 Schematically illustrates selected modules of a monitoring system 300 configured to predictively modify the video capture operations of selected cameras based on a graphical map, and to coordinate the configuration of alarms across a group of cameras by a master camera. Figures 1 to 2 130. For example, the monitoring system 300 may be configured in a network video recorder similar to the network video recorder 130. In some embodiments, one or more of the selected modules may access or be instantiated in: a processor, memory, and other resources of a camera configured for video capture (similar to the camera 110); and / or a user device configured for video surveillance, similar to the user device 170. For example, a camera and its embedded or attached computing resources may be configured with some or all of the functionality of the monitoring controller 330, and / or those functionality may be shared between the camera controller and the network video recorder or a video surveillance as a service (VSaaS) server. Similarly, some or all of the analytics engine 340 may be instantiated in the camera and / or shared with other monitoring system components.

[0057] The monitoring system 300 and / or any of its components may include a bus 310 that interconnects at least one processor 312, at least one memory 314, and at least one interface, such as a camera interface 316 and a network interface 318. The bus 310 may include one or more conductors that allow communication between the components of the monitoring system 300. The processor 312 may include any type of processor or microprocessor that interprets and executes instructions or operations. The memory 314 may include a random access memory (RAM) or another type of dynamic storage device that stores information and instructions for execution by the processor 312 and / or a read-only memory (ROM) or another type of static storage device that stores static information and instructions for use by the processor 312 and / or any suitable storage element, such as a hard disk or solid-state storage element.

[0058] The camera interface 316 can be configured to connect to one or more cameras. For example, the camera interface 316 can include multiple Ethernet ports and support protocols compatible with the PoE standard for connecting to multiple cameras. In some embodiments, the camera interface 316 can include a PoE network switch for providing power to connected cameras and routing data packets (such as control data and video data) to and from connected cameras.

[0059] Network interface 318 may include one or more wired or wireless network connections to a network, similar to network 102. Network interface 318 may include a physical interface (such as an Ethernet port) and associated hardware and software protocols (such as a network interface card or wireless adapter) for communicating over the network.

[0060] The surveillance system 300 may include one or more non-volatile memory devices 320 configured to store video data. For example, the non-volatile memory device 320 may include multiple flash memory packages organized as an addressable memory array and / or one or more solid-state drives or hard disk drives. In some embodiments, the non-volatile memory device 320 may include multiple storage devices within, attached to, or accessible by a camera and / or network video recorder for storing and accessing video data.

[0061] The monitoring system 300 may include a plurality of modules or subsystems that are stored and / or instantiated in the memory 314 for execution as instructions or operations by the processor 312. For example, the memory 314 may include a monitoring controller 330 configured to control one or more cameras, capture and store video streams from those cameras, and enable user access, such as through a monitoring application 350. The memory 314 may include an analytics engine configured to analyze video data to detect events for use by the monitoring controller 330 and / or the monitoring application 350. The memory 314 may include a monitoring application configured to provide a user interface for monitoring, viewing, and managing surveillance video and / or the monitoring system 300.

[0062] The monitoring controller 330 may include interface protocols, functions, parameters, and data structures for connecting to and controlling cameras, capturing and storing video data from those cameras, and interfacing with the analytics engine 340 and monitoring application 350. For example, the monitoring controller 330 may be an embedded firmware application and corresponding hardware located within a network video recorder (NVR) configured for network communication and / or direct communication with a group of associated cameras. The monitoring controller 330 may be configured as a central acquisition point for video streams from associated cameras, enabling analysis of the captured video data by the analytics engine 340 and presentation of the video streams and video event alerts to a user via the monitoring application 350. In some embodiments, some or all of the functionality of the monitoring controller 330 may be located on each camera, and a NVR may not be required. For example, a group of networked cameras may be configured to use onboard memory and processors for camera control and video capture, including generating and responding to alerts or notifications to update video capture operations, and may include a certain amount of onboard video storage. In such a configuration, the set of networked cameras may be configured to interface with a remote monitoring manager and / or monitoring application operating on a control center or end-user computer system, which may include the functionality of a network video recorder, a network video server, and / or a network video storage device.

[0063] In some embodiments, the monitoring controller 330 may include a plurality of hardware modules and / or software modules configured to process or manage defined operations of the monitoring controller 330 using the processor 312 and the memory 314. For example, the monitoring controller 330 may include a camera control interface 332, a video capture interface 334, a video storage interface 336, and an access and display manager 338. In some configurations, the monitoring controller 330 may include an alternative configuration for a master camera including a master node logic component 360 and / or a control center interface 370.

[0064] The camera control interface 332 may include a camera interface protocol and a set of functions, parameters, and data structures for using, configuring, communicating with, and providing command messages to the cameras via the camera interface 316 and / or the network interface 318. For example, the camera control interface 332 may include an API and command set for interacting with control circuitry in each camera to access one or more camera functions. In some embodiments, the camera control interface 332 may be configured to set video configuration parameters for the camera's image sensor, microphone, and / or video encoder, access pan-tilt-zoom features, set or modify camera-based motion detection, tripwire, object detection, and / or low-light detection parameters, and / or otherwise manage the operation of the cameras. For example, the camera control interface 332 may maintain a camera configuration table, page, or similar data structure that includes an entry for each camera being managed and its corresponding camera-specific configuration parameters, active control features (such as PTZ control), and other configuration and control information for managing the cameras. In some embodiments, each camera may be assigned a unique camera identification that may be used by the monitoring controller 330 , the analytics engine 340 , and / or the monitoring application 350 to associate video data with the camera that received it.

[0065] In some embodiments, the camera control interface 332 may include a message interface 332.1 for sending and / or receiving control messages with one or more cameras. For example, the message interface 332.1 may include a messaging protocol for adjusting the values ​​of operating parameters for each camera. In some configurations, camera-specific configuration parameters may include ranges of operating values ​​and corresponding conditions for applying those operating parameters to video capture operations. For example, different sets of configuration parameters may be used for different times, environmental conditions, or operating modes for each camera. The message interface 332.1 may enable the monitoring controller 330 to modify the current operating configuration parameters of a camera. For example, the message interface 332.1 may be configured to send a video capture update message to change one or more video capture parameters for a selected camera. In some configurations, the message interface 332.1 may use Internet Protocol, master-slave, and / or multi-master messaging via the camera interface 316 and / or the network interface 318 to send messages to the selected camera. In some configurations, application-level messaging may be used to send, parse, and respond to messages, enabling the camera to determine parameter changes from the message content to initiate changes in video capture operations. In some configurations, the message interface 332.1 can enable the monitoring controller 330 to use one or more update messages to directly change the memory location in the camera storing the configuration parameters to be changed. In some configurations, the message interface 332.1 can be integrated into or responsive to the video capture interface 334 to enable the monitoring controller 330 to respond to video events by sending video capture update messages to selected cameras. In some configurations, the message interface 332.1 can include a peer-to-peer messaging interface between cameras in a networked group to enable direct communication of alarm and video capture update messages between cameras.

[0066] In some embodiments, the camera control interface 332 may include a PTZ controller 332.2 for one or more cameras. For example, each camera may be equipped with a PTZ control unit and associated motors and / or digital controllers and a command interface for moving the camera from its current position to pan, zoom, and / or tilt to change the field of view. In some embodiments, the PTZ controller 332.2 may include a remote controller unit that sends PTZ control commands to adjust the camera position and / or zoom in real time, such as in response to detecting an object of interest in the field of view (but not ideally positioned in the field of view). In some embodiments, the PTZ controller 332.2 may include a set of configuration settings for an auto-tracking or auto-capture function within a selected camera. For example, one or more cameras may include an auto-capture feature for detecting an object of interest and then centering and zooming on the detected object. The PTZ controller 332.2 may be used to configure parameters for the auto-capture feature, such as the category of object to be captured (e.g., person, face, vehicle, license plate, etc.), PTZ range or limit, timing, quality or reliability thresholds, etc. In some embodiments, the PTZ controller 332.2 can be configured to respond to update messages (from the message interface 332.1) regarding video events outside the camera's current field of view in order to predictively change video capture operating parameters in anticipation of an object entering the field of view. For example, in response to a video event of an object approaching from the direction of another camera, the PTZ controller 332.2 can be commanded to pan the camera toward an expected interception point of the other camera or an object of interest in the adjusted field of view.

[0067] The video capture interface 334 may include a camera interface protocol and a set of functions, parameters, and data structures for receiving video streams from associated cameras. For example, the video capture interface 334 may include a video data channel and associated data buffers for managing multiple camera video data streams. In some embodiments, each camera may be assigned a dedicated video channel for continuously and / or selectively transmitting its video stream to the video capture interface 334. For example, each camera configured as a primary camera may have a dedicated video channel for its corresponding primary video stream. The video capture interface 334 may be configured to pass each received video stream to the video storage interface 336, the analysis engine 340, and / or the access / display manager 338. For example, the received video stream may be buffered by the video capture interface and then streamed to the video storage interface 336, the analysis engine 340, and the access / display manager 338. In some embodiments, the video capture interface 334 may receive camera video metadata describing the camera video format, time and location information, and event or condition markers based on onboard camera analysis. The video capture interface 334 can generate additional video metadata for video format changes and provide the video metadata to the video storage interface 336 and / or other components. In some embodiments, the video capture interface 334 can support an audio channel or audio track for audio data synchronized with the captured video data. For example, the one or more supported video formats may include one or more audio channels for audio data from one or more microphones associated with a camera. In some embodiments, the video capture interface 334 can use video events detected from one camera to trigger changes in the video capture operations (such as video capture operating mode and video capture rate) of one or more other cameras in the monitoring system 300.

[0068] In some embodiments, the video capture interface 334 may include a video stream manager 334.1 configured to identify and manage multiple video streams received from the camera. For example, the video stream manager 334.1 may manage: video buffer allocation and space; processing of video streams from a camera video format to another video format; flushing buffered video to storage via the video storage interface 336; and / or display via the access / display manager 338. In some embodiments, the video stream manager 336.1 may send video streams to the analysis engine 340 for analysis and / or provide notification to the analysis engine 340 regarding the availability and storage location of video data for analysis in the non-volatile memory 320 (as determined by the video storage interface 336). In some embodiments, the video stream manager 334.1 may include a configurable video path. For example, the storage path (through the video storage interface 336), the display path (through the access / display manager 338), and / or the analysis path (through the analysis engine 340) may each be configured for specific processing, priority, and timing. In some embodiments, one or more selectable storage paths and corresponding storage locations or storage modes may be associated with different video capture operating modes and / or video capture rates and may be selected in response to a trigger such as a video event trigger.

[0069] In some embodiments, the video stream manager 334.1 may be configured to use an encoder / decoder 334.2 to encode the camera video stream in a desired video format. In some embodiments, the encoder / decoder 334.2 may be configured to receive a raw video data stream from an image sensor and determine the video data format to be used, including the capture rate of the raw data from the image sensor. For example, the video capture rate may include the number of pixels or resolution of the image data from the image sensor and the frame rate that determines the frequency at which pixel values ​​are determined. In some embodiments, the encoder / decoder 334.2 may support two or more selectable video capture rates and corresponding video formats. For example, the encoder / decoder 334.2 may support a passive video capture rate 334.3 and an active video capture rate 334.4. The passive video capture rate 334.3 may have a lower rate than the active video capture rate 334.4, such that the passive video capture rate 334.3 captures less video data (e.g., low-quality video 320.3) and requires less processing, network bandwidth, and / or data storage, but has a lower video quality that may be less effective for display and / or analysis. Active video capture rate 334.4 can capture more video data with higher video quality (e.g., high quality video 320.3) for display and analysis, but at the expense of increased processor usage, network bandwidth, and / or data storage. In some embodiments, video encoder / decoder 334.2 can support more than two selectable video capture rates. In some embodiments, an in-camera video encoder can encode video data from an image sensor in a first (camera) video format, and video stream manager 334.1 can use encoder / decoder 334.2 to re-encode them in one or more other formats. For example, video stream manager 334.1 can use encoder / decoder 334.2 to change the resolution, image size, frame rate, codec, compression factor, color / grayscale, or other video format parameters.

[0070] In some embodiments, the video capture interface 334 may include multiple operating modes 334.5. For example, a standby operating mode may include a low power state in which the camera is not actively capturing video, and a normal operating mode may include a normal power state in which the camera may activate any of its resources, including capturing video using its image sensor and processor. In some embodiments, the operating modes 334.5 may include a low light operating mode for low light conditions, a motion / tripwire only mode in which only low-level processing of image sensor data (or a separate motion sensor) is performed without video capture, and other operating modes. In some embodiments, one or more operating modes may be associated with a particular video capture rate and encoding format. For example, a passive video capture mode may use a passive video capture rate 334.3, and an active video capture mode may use an active video capture rate 334.4. The camera may use different operating modes during different operating periods, and the operating modes may be used to modify one or more video capture operations. For example, each operating mode may define whether video capture is paused or in operation, and if in operation, which set of video capture parameters to use, such as video capture rate (resolution and frame rate), encoding codec, filters, etc.

[0071] In some embodiments, the video capture interface 334 may include trigger conditions 334.6 for moving between operating modes 334.5. For example, the video capture interface 334 may have a default operating mode and one or more other operating modes that are triggered when specific conditions are met. For example, a camera may be configured to default to normal operating mode, but change to low-light operating mode when light levels drop below a threshold. In some configurations, due to lower resource usage, a camera may include a passive video capture mode as the default mode for continuous video monitoring and include one or more trigger conditions 334.6 for changing operating modes. For example, when motion, a video tripwire, or an object is detected from passive video data, the video capture interface 334 may change the operating mode of the camera that detected the condition to active video capture mode. However, using passive video data, these video-based triggers may be less reliable, and there may be a lag between an object entering the field of view and the successful triggering of the video condition, including processing time required for monitoring analysis. In some embodiments, the video capture interface 334 may include one or more trigger conditions 334.6 that predictively change operating modes based on video events occurring on other cameras. For example, the video capture interface 334 may include a video event detector 334.7 that determines video events from each camera and uses a graphical map 334.8 and / or a reference table 334.12 to determine which other cameras should change operating modes and what those operating mode changes should be.

[0072] Video event detector 334.7 can support detecting one or more video event types based on video data received by monitoring controller 330. For example, each camera's incoming video stream can be processed by one or more analytical models (such as object detector 334.18) to determine whether an object of interest appears within a frame of video data with a selected confidence threshold. In some configurations, video event detector 334.7 can use video event parameters generated from one or more sources (such as video metadata, time, event detected by the camera (e.g., motion sensor, video tripwire, low-weight vehicle-mounted object detector, etc.)) and output from one or more analytical engines. For example, an object can be detected by a camera's onboard object detector and, based on the type of object detector, assigned an object type parameter along with the time of detection and position in the frame. In some configurations, actuator positions (and corresponding field of view) can be mapped to edges in graphical map 334.8 to determine event direction. For example, actuator positions can define a set of vectors that define the field of view, and object positions in the field of view can correspond to paths represented by edges in graphical map 334.8, which can be referred to as event detection edges. In some configurations, an analysis engine with an object motion tracking model can then process a series of video frames in the video data stream to determine the direction of travel of the detected object. Object detection can trigger a video event detector 334.7 to identify a video event and initiate event response logic 334.17 to evaluate a graphical map 334.8 and / or a reference table 334.12 to determine whether video capture parameters of one or more other cameras should be updated. In some configurations, the video event detector 334.7 can isolate a portion of a video frame or video data corresponding to the object of interest (e.g., a bounding box) to include in the video event parameters along with other object data.

[0073] The graph map 334.8 may include a data structure for representing the surveillance environment and cameras as a set of nodes and edges, such as described above with respect to Figure 2 As described. For example, the graph map 334.8 may include a set of parent nodes corresponding to cameras in the surveillance system and a set of child nodes that map to object travel paths between cameras (and more specifically, the intersections of these paths). In some configurations, the graph map 334.8 may include multiple graph maps configured for different objects and / or event types 334.9. For example, one graph map may be configured for pedestrian traffic and model walking paths between camera locations, and another graph map may be configured for vehicular traffic and model driving paths between locations. Similarly, different graph maps may be configured for different times of day or conditions. For example, some paths may be closed during certain times of the day, and this may be reflected in different sets of child nodes and edges.

[0074] Graphical map 334.8 can be based on a coordinate location system consisting of a coordinate grid and scale 334.10. For example, latitude and longitude coordinates can be used as the coordinate grid. Alternatively, another grid system can be laid out for a campus, building complex, city, or other area of ​​interest based on relevant features to define an alternative coordinate grid. The coordinate location system can include a two-dimensional coordinate layout, such as longitude and latitude or another XY coordinate system. Different camera groups can use different coordinate location systems, which can be based on the scale, configuration, and / or size / precision of the target object of interest of the sub-area. In some configurations, a three-dimensional coordinate layout can be used, such as adding elevation, altitude, or similar Z coordinates to the two-dimensional coordinate location system. Based on the coordinate location system, each camera and interconnected path can be modeled as a node and edge 334.11 in graphical map 334.8. Using the coordinate location system, each camera can be defined as a parent node with a specific coordinate location (e.g., XY value), and each intersection can be defined as a child node with its own coordinate location. Edges can be defined as lines connecting parent and child nodes. In some configurations, the graph map 334.8 can be represented as a data structure, such as a data table or array, consisting of the coordinates of each node and the node pairs that define the edges. In some configurations, the edge data can allow the calculation of the corresponding path length for each edge, and the path lengths of multiple edges traversed by paths between and across nodes can be determined by aggregating those path lengths.

[0075] Reference table 334.12 may include a data structure for using video event parameters to determine which other cameras would benefit from a change in operating mode to increase the likelihood of capturing additional video data associated with the triggered video event. For example, if one camera detects an object of interest, reference table 334.12 may include entries for each other camera, indicating their spatial relationship to the detecting camera based on graphical map 334.8. In some configurations, reference table 334.12 may include a video event index 334.13, which includes one or more index fields corresponding to the associated event parameters. For example, video event index 334.13 may include a combination of the object type and event direction parameters of the detected object of interest. The event detection edge determined from the camera and object positions and graphical map 334.8 may be used as the index value for video event index 334.13. In some configurations, each camera entry 334.14 in reference table 334.12 may include a camera identifier and position parameters defining the camera's field of view, parent node coordinates, and child nodes. For example, the same camera identifier used to manage camera configuration parameters may be used, and the position parameters may include parent node coordinates, child node coordinates, and possible object detection edges based on the actuator / field of view range of the camera. In some configurations, each entry may also include a set of video capture parameters 334.15 that correspond to an increased likelihood of capturing subsequent video events in the field of view of that camera. For example, the video capture parameters 334.15 of a spatially adjacent camera may include active capture rate parameters and PTZ control parameters that are used to move the camera to the closest position of its field of view and the predicted edge to a shared child node. In some configurations, the reference table 334.12 may include multiple entries for each camera relative to each other camera, and indexed by event direction and / or detected event edge parameters to determine whether and which other camera identifiers and video capture parameter sets are selected in response to any given video event (based on shared child nodes in the event direction).

[0076] In some embodiments, event response logic 334.16 may include logic rules for applying video events and their corresponding video event parameters to trigger conditions 334.6 to trigger one or more operating modes 334.5. For example, video response logic 334.16 may include a set of thresholds and logic rules for applying those thresholds to trigger modifications to video capture operations. In some configurations, video response logic 334.16 may include a set of active capture thresholds for initiating active video capture rates 334.3 and / or corresponding active video capture modes. In some configurations, event response logic 334.16 may use graphical mapping 334.8 and / or reference table 334.12 to determine whether and which cameras should change their video capture parameters. For example, video event detector 334.7 may use the video event parameters to search for video event index 334.13 to determine whether the video event should trigger a change in the operating mode of any of the other cameras. If no video event index value matches the corresponding video event parameters, no update message may be initiated. Similarly, if multiple entries are returned in the search for different cameras, update messages can be sent to multiple cameras. In some configurations, event response logic component 334.16 can use graphical mapping 334.8 and / or reference table 334.12 to determine a group of child nodes shared with other cameras relative to the detection camera. This group of shared child nodes can be filtered based on the event direction and / or object detection edge to only include shared child nodes in that direction. Event response logic component 334.16 can generate a video capture update message based on the identified child nodes to send to the cameras with shared child nodes. For example, message interface 332.1 can be used to send a child node alert 334.17 including a child node identifier of a shared child node, and this shared child node identifier can be used by the receiving camera to determine the video capture parameter changes to be performed, such as adjusting the camera position towards the shared child node.

[0077] In some configurations, the thresholds used by the event response logic component 334.16 may include confidence thresholds for object detection or classification and / or motion direction. For example, the video event detector 334.7 may return a video event with an object type, a type confidence value, a motion direction value, and a motion confidence value. If the object type matches the object type of interest, the type confidence value meets the type confidence threshold (e.g., 50%), and the motion direction value meets the motion confidence threshold (e.g., 50%), the video response logic component 334.16 may trigger the message interface 332.1 to send a video capture update message to the selected camera. In some embodiments, additional and / or alternative sets of logic rules and / or entries in the reference table 334.12 may be included in the event response logic component 334.16 for applying video event parameters to initialize video capture, trigger PTZ movement toward a predicted intercept position of the camera's field of view, and / or change data storage and / or processing paths.

[0078] In some embodiments, the video capture interface 334 may be configured with an object detector 334.18 that supports detection of one or more object classes (such as people, animals, motor vehicles, etc.). For example, the object detector 334.18 may operate on captured video data received from a camera to detect whether an object of interest is present in the video data. In some embodiments, the object detector 334.18 may include a lightweight object detection model that can be processed in near real time using the limited processing bandwidth of the camera and / or associated computing resources. In some embodiments, the object detection model may operate on the video data in the video stream and return a flag or class for the detected object type, an object confidence quality metric, the object location, and / or object boundary data (such as two horizontal positions and two vertical positions) to define a bounding box within the video frame. In some embodiments, the object detector 334.18 may have one or more associated object confidence thresholds for evaluating an object confidence value for each object detection event. For example, the object detector 334.18 may include an object detection threshold below which the presence of an object is not considered sufficiently certain to cause an object detection event, such as a 50% confidence level. In some embodiments, object detector 334.18 may be used to identify object detection events and issue corresponding alerts to video event detector 334.16.

[0079] In some embodiments, the video capture interface 334 may include a predictive capture timer 334.19 configured to determine how long a camera that predictively changes its video capture operation should wait to see if an object of interest is detected or return to default operation. For example, the predictive capture timer 334.19 may use a predefined operating period to determine the duration of time to wait for an object of interest to be detected within its field of view after an update message is sent by a selected camera and / or a parameter is changed. In some configurations, the duration of the predictive operating period may be a default value that is predetermined for each spatial relationship (e.g., provided in a corresponding entry in reference table 334.12) and / or dynamically generated based on the distance between the cameras and the velocity from the object motion model. In some configurations, the object path length can be calculated from the edges traversed between two camera parent nodes, where each edge between the nodes traversed by the path is summed together to determine the path length. If there are multiple shared child nodes and / or paths between parent nodes, multiple path lengths can be determined and used, for example, by averaging or using the maximum or minimum path length. Once the duration of the predictive operation period has been determined for a particular video event and camera, the predictive capture timer 334.19 can monitor the elapsed time and return the camera to its default operating mode when the duration is met. For example, when the time has elapsed, the message interface 332.1 can send another update message to return the camera to default operation. Alternatively, the parameters sent in the original update message can include a duration parameter, and the camera can implement its own predictive capture timer to automatically return to default operation when the time has elapsed.

[0080] The video storage interface 336 may include a storage interface protocol and a set of functions, parameters, and data structures for managing the storage of video data 320.1 in non-volatile memory 320 (such as a storage device and / or network video storage) for later retrieval and use by the access / display manager 338 and / or the analysis engine 340. For example, the video storage interface 336 may write camera video stream data from the video data buffer and / or storage path video data from the video capture interface 334 to the non-volatile memory 320. In some embodiments, audio data may be stored in a separate audio data file or object and / or as synchronized audio data in an audio track or channel of the video data 320.1. In some embodiments, the video storage interface 336 may include a storage manager 336.1 configured to manage video storage space in the non-volatile memory 320 according to one or more operating modes 334.5, data retention, and / or data archiving schemes. For example, the surveillance system 300 can support continuous and / or triggered recording of video data from associated cameras, and the storage manager 336.1 can include logic for implementing data retention and overwrite policies whereby fixed storage space in the non-volatile memory 320 is recycled for storing recently captured video, video data that meets specific retention criteria, and / or video data is deleted or archived after one or more time periods defined in the data retention policy. In some embodiments, the storage manager 336.1 can support different storage locations for high-quality video 320.2 captured during active video capture mode and low-quality video 320.3 captured during passive video capture mode. The video storage interface 336 can also include a metadata manager 336.2 for receiving video metadata and storing it as tags or metadata tracks in the video data or in an associated metadata table, file, or similar data structure associated with the corresponding video data object.

[0081] The access / display manager 338 may include an API and a set of functions, parameters, and data structures for displaying video from the video capture interface 334 and / or the video storage interface 336 to a user display application (such as the monitoring application 350). For example, the access / display manager 338 may include a monitoring or display configuration for displaying one or more video streams in real time or near real time on a graphical user display of a user device and / or receive video navigation commands from the user device to selectively display stored video data from the non-volatile memory 320. In some embodiments, the access / display manager 338 may maintain an index of real-time / near real-time video streams and / or stored or archived video streams available for access by the monitoring application 350. In some embodiments, the video index may include a corresponding metadata index that includes video data parameters (e.g., time, location, camera identifier, format, low light / normal light, etc.), detected audio and video event metadata (event time, location, type, parameters, etc.), and / or video management parameters (expiration, active / archived, access control, etc.) for displaying and managing video data. The access / display manager 338 can be configured to support the monitoring application 350 when instantiated in the same computing device as the monitoring controller 330, when directly attached to the computing device hosting the monitoring controller 330, and / or via a network within a LAN, WAN, VPN, or the Internet. In some embodiments, the access / display manager 338 can provide the user video path with selective access to the user video format 336.2 and / or video stream assigned by the video capture interface 334.

[0082] The analysis engine 340 may include interface protocols, functions, parameters, and data structures for analyzing video data to detect video events, add these video events to video metadata, and / or issue alerts (such as via the monitoring application 350). For example, the analysis engine 340 may be an embedded firmware application and corresponding hardware located in a network video recorder configured to perform local analysis of video data captured from associated cameras, and may be integrated into or accessible by the monitoring controller 330. In some embodiments, the analysis engine 340 may run on a computing device separate from the monitoring controller 330 (such as a camera with analysis capabilities, a dedicated analysis device, a data storage system with analysis capabilities, or a cloud-based analysis service). In some embodiments, the analysis engine 340 may operate in real time or near real time on video data received by the video capture interface 334, video data stored by the video storage interface 336, and / or a combination thereof, based on the nature of the video event (and processing requirements), the amount of video to be processed, and other factors. In some embodiments, the monitoring system 200 may include multiple analysis engines configured for specific types of events and corresponding event detection algorithms or models.

[0083] In some embodiments, the analysis engine 340 may include multiple hardware modules and / or software modules configured to process or manage the defined operations of the analysis engine 340 using the processor 312 and the memory 314. For example, the analysis engine 340 may include an event manager 342 and an analysis model library 344. The analysis engine 340 may be configured to run one or more event detection algorithms to determine, mark, and / or initiate alarms or other actions in response to detected video events. In some embodiments, the analysis engine 340 may be configured to mark or construct a metadata structure that maps detected events to time and image location markers of the video stream from which the events were detected. For example, the analysis engine 340 may use motion, tripwire detection, object recognition, facial recognition, audio detection, speech recognition, and / or other algorithms to determine events occurring in a video stream and mark these events in corresponding metadata tracks and / or in separate metadata tables associated with video data objects.

[0084] The event manager 342 may include a storage interface and / or buffer interface protocol and a set of functions, parameters, and data structures for processing target video streams for predefined event types and updating or adding metadata parameters describing detected video events. For example, the event manager 342 may be configured to: process all input video streams for the monitoring controller 330; and / or selectively process video data based on user selection (via the monitoring application 350) or metadata criteria received from the camera or video capture interface 334. In some embodiments, the event manager 342 may include, support, or supplement event detection performed by the monitoring controller 330, such as video events from the video event detector 334.7 and / or object detection events from the object detector 334.18. In some embodiments, the event manager 342 may include a video selector 342.1 configured to select target video streams or video datasets for analysis, including associated audio data. For example, the video selector 342.1 may identify a live video stream or bounded video dataset for near real-time analysis, such as a video with a specified camera identification and a timestamp between start and end time markers and / or a video that includes a defined set of metadata parameters. The event manager 342 may include an event type selector configured to determine one or more event types to be detected from the selected video data. For example, the analysis configuration may be configured to analyze the selected video stream for a predefined set of audio event detection, motion detection, tripwire detection, object recognition, facial recognition, speech recognition, and / or similar video event types. Each event type may be mapped or corresponded to an analysis model type, a set of parameters, and one or more model weights for defining an event detection algorithm stored in the analysis model library 344 for use by the analysis engine 340 in detecting potential video events (and / or predictive audio events).

[0085] The analysis model library 344 may include an API and a set of functions, parameters, and data structures for storing multiple analysis models for use by the analysis engine 340 in processing video data. For example, the analysis model library 344 may include multiple trained analysis models and corresponding event detection algorithms for different event types, target object categories (e.g., cars, license plates, equipment, people, etc.), and / or training conditions. In some embodiments, the analysis model library 344 may also support audio analysis models and / or combined video and audio analysis models. Each analysis model includes a set of basic equations for the analysis model type, a set of target parameters, and one or more weights that determine the event detection algorithm to be used for event detection processing. In some embodiments, at least some analysis models may be machine learning-based models trained based on one or more sets of relevant reference data. For example, a reference data set may be used to train the basic equations to determine the model weights to be used in the resulting analysis model. The trained analysis models may be deployed in the analysis engine 340 and / or the monitoring controller 330. In some embodiments, the analysis engine 340 may include or access a training service for generating (training) or updating (retraining) the analysis models in the analysis model library 344.

[0086] In some embodiments, the analysis model library 344 may include at least one object recognition model 344.1. For example, a motor vehicle recognition model may apply a set of weighted parameter values ​​representing relationships between a set of feature vectors for comparison with reference data (such as a set of feature vectors of known motor vehicles) and determine a probabilistic reliability or correlation factor. The analysis model library 344 may include or access object reference data 344.2 for matching detected objects with previously identified (or recognized) reference objects. For example, the motor vehicle recognition model may be applied to a reference database of relevant motor vehicle images and / or feature sets extracted therefrom to provide vehicle reference data. In some embodiments, for any given detected object, the object recognition model 344.1 may return one or more identified matches and corresponding reliability values. For example, assuming at least one match is found that meets a threshold reliability value, the motor vehicle recognition model may return one or more known individuals from the reference data and corresponding reliability values. In some configurations, the video event parameters of the initial object detection event may be used as reference data for matching similar objects detected in other video data from the camera. For example, once an object of interest is detected, the bounding box and the image data it contains can be designated as reference data for subsequent object recognition computations by other cameras that detect the same type of object in response to receiving an update message.

[0087] In some embodiments, the analytical model library 344 may include at least one object motion model 344.3 for tracking and predicting the movement of an object of interest based on video data. For example, the object motion model 344.3 may include an object motion tracking model that detects the same object frame by frame. In some configurations, the object motion model 344.3 may utilize the object's bounding box from frame to frame to track the object's movement within the video frame. The detected object's movement within the frame can then be converted into a possible direction of travel for the object itself using a spatial model. For example, for a particular camera position, rightward movement within a frame may roughly correlate with northward movement within the surveillance environment, and leftward movement may correlate with southward movement. Changes in object size may also be used to estimate movement into or out of the frame. In some configurations, the object motion model 344.3 may be used to determine a movement direction parameter for the video event detector 334.7. Other models for determining or approximating the direction of travel of an object of interest within video data may also be employed.

[0088] The monitoring application 350 may include interface protocols, functions, parameters, and data structures for providing a user interface for monitoring and viewing surveillance video and / or managing the monitoring system 300, such as through the monitoring controller 330. For example, the monitoring application 350 may be a software application running on a user device that is integrated with, connected to, or in network communication with the monitoring controller 330 and / or a hosted network video recorder. In some embodiments, the monitoring application 350 may run on a computing device separate from the monitoring controller 330, such as a personal computer, mobile device, or other user device. In some embodiments, the monitoring application 350 may be configured to interact with an API presented by the access / display manager 338.

[0089] In some configurations, the monitoring controller 330 may include additional functionality for a selected camera in a camera group to act as a master camera or master node for the group of networked cameras. For example, an instance of the monitoring controller 330 in a master camera may include master node logic 360. The master node logic 360 may include an API and a set of functions, parameters, and data structures for determining regional events based on local event alarms generated by cameras in its group and handling notifications to and from other networked camera groups in the same region. In some configurations, the master node logic 360 may include master node configuration parameters 262. For example, the master node configuration parameters 262 may include one or more master node indicators for the selected camera. In some configurations, the master node indicator may be included as a reference value in one or more graphical maps or related data structures. For example, the master node indicator may be a flag value appended to a parent node or camera identifier in the graphical map 334.8 and / or reference table 334.12, which indicates to any system (other cameras, surveillance management system, etc.) that the corresponding camera is the master camera for the group.

[0090] The master node logic component 360 may include a master graph map 364 that is configured to associate a sub-region graph map for each sub-region with a corresponding group of networked cameras. For example, the master graph map 364 may include a parent node corresponding to the master camera of each group, and the connections and spatial relationships of these master cameras may be represented in the master graph map 364. In some configurations, the master graph map 364 may include additional node data describing other camera nodes and / or edges in the sub-region. However, the master graph map 364 may be based only on the relationships between master nodes, and only master cameras (and their groups) may access the sub-region or local graph map of the camera group to reduce cross-group overhead.

[0091] The master node logic component 360 may include a regional event logic component 366, which is configured to determine when a regional video alarm event has occurred and when notification should be provided to other master cameras and their groups based on local video events within its group. For example, the regional event logic component 366 may process alarms or messages generated by cameras within its group (including from processing its own video) and received via the message interface 332.1 to determine whether a regional alarm or video capture update message should be sent to one or more other camera groups. The regional event logic component 366 may include a set of event conditions 366.1 that are evaluated for each local group video event. For example, the event conditions 366.1 may be configured as logic rules for evaluating one or more video event parameters and / or associated criteria or thresholds for determining whether a particular video event has regional significance. The regional event logic component 366 may determine an event priority value 366.2 for any regional video event that meets the event conditions. For example, the same set of logic rules used to evaluate the video event parameters of the event conditions may also determine corresponding priority values, such as for different types of objects, time of day, location, and / or confidence values. The zone event logic component 366 may include at least one priority threshold 366.3, which selects from a range of possible priority values ​​and determines the priority level at which zone notifications should be sent. For example, a first priority threshold (low) may determine that an event condition should be classified as a zone video alarm event, while a second priority threshold (medium) and a third priority threshold (high) may determine which other groups receive the alarm and how the receiving master camera and camera group handles the incoming alarm and associated video capture updates. The sub-zone logic component 366.4 may include rules for determining which other sub-zones should be notified of zone events. For example, the sub-zone logic component 366.4 may use the event priority value 366.2 and the priority threshold 366.3 to determine which master cameras to notify, such as for low-priority events, only notifying the next adjacent (or overlapping) group in the last known direction of the object of interest, for medium-priority events, notifying all adjacent (or overlapping) groups, and for high-priority events, notifying all groups. The sub-zone logic component 366.4 may use the master graphical map 364 to evaluate proximity and range to other sub-zones for selecting the camera group to alert.

[0092] The master node logic component 360 may include a master message interface 368 for communicating with the master cameras of other camera groups. For example, the master message interface 368 may be configured similarly to the message interface 332.1, but include the network address or similar identifier or channel of each of the other master cameras of the other sub-regional groups. In some configurations, the master message interface 368 may include direct network communication between the master cameras, or a host computer, control center, or similar surveillance management system may be used to route messages between the master cameras. The master message interface 368 may include a regional update logic component 368.1, which is configured to generate alert messages for selected master cameras. For example, the regional update logic component 368.1 may select video event parameters to include in the payload of an alert or notification. These video event parameters (such as the video sample that triggered the video event, object parameters of the detected object of interest, the last known location (e.g., the coordinates of the camera that captured the video event), and / or the direction of travel of the object of interest) may be included in the regional video capture update message for use by the receiving camera group. The event priority, time, and other metadata associated with the event may also be included in the regional video capture update message. The regional update logic component 368.1 can assemble the message payload and send the message to the master camera selected by the sub-region logic component 366.4 via the master message interface 368. The master message interface 368 can also include a local update logic component 368.2, which is configured to receive and parse alarm messages received from other master cameras and corresponding groups. For example, the local update logic component 368.2 can receive regional video capture update messages and determine how to process them for its group. The local update logic component 368.2 can use event priority and / or other video event parameters to determine whether to distribute corresponding local video capture update messages to one or more cameras in its group. For example, the local update logic component 368.2 can compare the received event priority with the operational priority of the group and / or individual cameras in the group to determine whether they should be updated. In some configurations, once the local update logic component 368.2 determines that a received area video event meets the priority threshold of its group, it may treat the video event as if it had detected itself (e.g., similar to the output of the video event detector 334.7) and use its local graphical map 334.8, reference table 334.12, and event response logic component 334.16 to notify some or all other cameras in the group with a local video capture update message. For example, event parameters from the area video event may be sent to all parent nodes in the graphical map and / or selectively based on the video event location, direction of travel, and / or relative position of the receiving group and the originating group. In some configurations, each camera may include its own operational priority, and the local video capture update message priority may be evaluated to determine whether to modify the operation of that camera.For example, if a camera has recently triggered its own high priority video capture operation, it may ignore video capture update messages with low or medium priority.

[0093] In some configurations, the monitoring controller 330 may include additional functionality for the master camera or master node to communicate with the control center hosting the monitoring management controller. For example, an instance of the monitoring controller 330 in the master camera may include a control center interface 370. The control center interface 370 may include an API and a set of functions, parameters, and data structures for communicating with one or more control centers that oversee the operation of the group of networked cameras in the area. The control center interface 370 may include a control message interface 372 for sending and receiving messages with a control center system (such as a monitoring management controller). The control message interface 372 may be configured to be substantially similar to the message interface 332.1 to provide messaging between the master camera and the control center over a network channel. The control center interface 370 may include a control message logic component 374 for generating messages to be sent to the control center and processing messages received from the control center. For example, the control message logic component 347 may include an interface to the master node logic component 360 for receiving and directing regional video event messages and / or video event data to and from the control center. About Figure 4 Further described is an arrangement that uses a control center computer system as an intermediary between camera groups and their corresponding master cameras.

[0094] Figure 4 Schematically illustrates selected modules of a control center computer system 402 configured to monitor a set of networked camera groups 400 in an area 430 using a sub-area graphical map, and configuration of a master camera to coordinate alarms across the camera groups. The control center computer system 402 may be combined with a system similar to Figures 1 to 3 130. For example, the control center computer system 402 may include functionality similar to that of the network video recorder 130. In the example shown, the control center computer system 402 may interface with camera groups 432.1, 432.2, 432.3, and 432.4 in area 430. In some embodiments, one or more of the selected modules may access or be instantiated in: a processor, memory, and other resources of a camera configured for video capture (similar to camera 110); and / or a user device configured for video surveillance, similar to user device 170. For example, a camera and its embedded or attached computing resources may be configured with some or all of the functionality of the surveillance controller 330, and / or those functionality may be shared between the camera controller and a network video recorder or a video surveillance as a service (VSaaS) server.

[0095] Region 430 is shown as a mapping area where multiple camera groups have been deployed in a specific physical location. Each camera group 432 may include multiple networked cameras as described above. Each camera group 432 can also be represented in the corresponding sub-region graphic map of the group and its associated physical location (or sub-region). In the example shown, each camera group 432 includes a main camera 434 (represented as a master node in the graphic map) and multiple other cameras 436 (represented as a parent node in the graphic map--the master node is also a parent node). For example, group 432.1 includes main camera 434.1 and other cameras 436.1.1 and 436.1.2 (and other cameras), and group 432.2 includes main camera 434.2 and other cameras 436.2.1 and 436.2.2 (and other cameras). Each camera group 432 can also include a plurality of sub-nodes 438 corresponding to the turns or connections in the path between the camera positions in its corresponding sub-region graphic map. 1. For example, group 432.1 includes child nodes 438.1.1 and 438.1.2 (and other child nodes), and group 432.2 includes child nodes 438.2.1 and 438.2.2 (and other child nodes). Similar symbols in each group 432 correspond to similar nodes (e.g., camera / parent node 436 and child nodes 438), and for the sake of brevity and to reduce the complexity of the drawings, additional labels have been omitted. Thus, for example, group 432.3 includes seven other cameras 436 in addition to main camera 434.3 and seven child nodes 438, even though they are not labeled, and group 432.4 includes four other cameras 436 and four child nodes 438. Each main camera 434 is communicatively connected to the control center computer system 402, such as via network communication. In some configurations, other cameras 436 may also be communicatively connected to the control center computer system 402.

[0096] The control center computer system 402 may include a bus 410 interconnecting at least one processor 412, at least one memory 414, and at least one interface (such as a network interface 418). The bus 410 may include one or more conductors that allow communication between the components of the control center computer system 402. The processor 412 may include any type of processor or microprocessor that interprets and executes instructions or operations, and may include multiple processors that operate individually or in combination to perform the operations described herein. The memory 314 may include a random access memory (RAM) or another type of dynamic storage device that stores information and instructions for execution by the processor 312 and / or a read-only memory (ROM) or another type of static storage device that stores static information and instructions for use by the processor 312 and / or any suitable storage element such as a hard disk or solid-state storage element. The non-volatile memory 420 may include one or more data storage devices configured to store video data 420.1 (which may include event video data 420.2) and / or event metadata 420.3 (such as video event parameters) in corresponding data structures for use by the control center computer system 402.

[0097] The control center computer system 402 may include a plurality of modules or subsystems that are stored in the memory 414 and / or instantiated in the memory for execution as instructions or operations by the processor 412. For example, the memory 414 may include a monitoring manager controller 480 that is configured to manage the operation of a group of networked cameras. The memory 414 may include an analysis engine 490 configured substantially as described above for the analysis engine 340. The memory 414 may include a monitoring application 492 configured substantially as described above for the monitoring application 350.

[0098] Monitoring manager (or management) controller 480 may include interface protocols, functions, parameters, and data structures for connecting to and controlling a group of cameras, and may include capturing and storing video data from those cameras. For example, monitoring manager controller 480 may be an embedded firmware application and corresponding hardware located in a network video recorder configured for network communication and / or direct communication with a group of associated cameras. Monitoring manager controller 480 may be configured as a central acquisition point for video streams from associated cameras, enabling analysis of the captured video data by analysis engine 490 and presentation of the video streams and video event alerts to a user via monitoring application 492. Monitoring manager controller 480 may include multiple hardware modules and / or software modules configured to process or manage the defined operations of monitoring manager controller 480 using processor 412 and memory 414. For example, monitoring manager controller 480 may include a master camera interface 482, a master graphical map 484, zone event logic 486, and update message logic 488.

[0099] The master camera interface 482 can be configured substantially similar to the camera control interface 332 for establishing communications between the control center computer system 402 and each group's master camera. The message interface 482.1 can be configured substantially similar to the message interface 332.1, and the video capture interface 482.2 can be configured substantially similar to the video capture interface 332.2.

[0100] The master graphical map 484 may include a graphical map that relates the camera groups monitored by the control center computer system 402 to each other. For example, the master graphical map 484 may include a master node 484.1 corresponding to the master cameras and describing their relative positions to each other. In some configurations, the master graphical map 484 may be embodied in a data structure having entries corresponding to the master node 484.1 and coordinate positions (such as longitude and latitude positions) on a shared coordinate system. Node connections may be used to represent adjacency and / or overlap between camera groups corresponding to the master nodes. The master graphical map 484 may include a sub-region graphical map 484.2 that describes parent nodes, child nodes, and edges, as well as the corresponding physical locations of each camera group, as described above for graphical map 334.8. For example, the master graphical map 484 may maintain a master copy of a graphical map that represents each camera group and is used by each camera group to coordinate local alarms and video capture update messages. The data structure of the master graphical map 484 may also include a group configuration 484.3 that describes each camera group. For example, each camera group may include a set of group configuration parameters that may be associated with the master node entry for that group. Example group configuration parameters may include detector types 484.4 describing the object types supported by the object detectors operating in the camera group, camera parameters 484.5 describing the configuration and capabilities of the cameras in the group (e.g., model, memory, operating mode, image capture capabilities, operating schedule, etc.), and / or group priority thresholds 484.6 describing the default priority for responding to zone video alarm events. In some configurations, the master graphical map 484 may be embodied in one or more data structures, such as one or more data tables, lists, databases, or similar data structures for associating a structured set of parameters with camera groups and / or master nodes 484.1 using a graphical map.

[0101] Region event logic 486 can be configured similarly to region event logic 366 to move some or all of the region event decisions to control center computer system 402. Region event logic 486 can include a similar set of event conditions 486.1, event priority values ​​486.2, priority thresholds 486.3, and / or sub-region or group selection logic 486.4. For example, region event logic 486 can receive video event alerts from a primary camera, evaluate video event parameters from those video events against event conditions 486.1, and determine corresponding event priority values. Based on priority threshold 486.3 and / or group priority threshold 484.6, sub-region selection logic 486.4 can determine one or more additional camera groups to receive region video capture update messages or similar video event alerts.

[0102] Update message logic 488 can use master camera interface 482 and message interface 482.1 to send regional video capture update messages for modifying video capture parameters of other camera groups based on regional video alarm events. Update message logic 488 can be configured to generate an alarm message for a selected master camera. For example, update message logic 488 can select video event parameter values ​​488.1 to include in the payload of an alarm or notification message. These video event parameters (such as the video sample that triggered the video event, object parameters of the detected object of interest, the last known location (e.g., the coordinates of the camera that captured the video event), and / or the direction of travel of the object of interest) can be included in the regional video capture update message for use by the receiving camera group. An event priority value 488.2 (based on event priority value 486.2 determined by regional event logic 486) can also be included as a parameter of the regional video capture update message. In some configurations, time and other video event metadata can also be included in the regional video capture update message. The update message logic component 488 can assemble the message payload and send the message to the master camera selected by the sub-region selection logic component 386.4 through the master camera interface 482. Figure 3 As described above, the monitoring manager controller 480 can receive a regional video capture update message generated by one of the master cameras and forward the message to the target other master cameras. Various other configurations of assigning functions between the monitoring manager controller 480 and the monitoring controller 330 of the master camera are also possible.

[0103] like Figure 5A As shown, the monitoring system 300 can operate according to an example method for sending an alarm based on a graphical mapping relationship between cameras, that is, according to Figure 5A The method 500 illustrated in blocks 510-518 operates.

[0104] Video events can be monitored at block 510. For example, each camera can capture video data and analyze it for objects of interest using an object detection model.

[0105] At block 512, a determination may be made as to whether an event has been detected. For example, one of the cameras may detect an object of interest in its video data stream that satisfies the triggering conditions of its video event detector, and method 500 may proceed to block 514. If no event has been detected, method 500 may return to block 510 to continue monitoring for video events.

[0106] Child nodes on the event detection edge may be determined at block 514. For example, a detection camera may determine an event detection edge or event direction from the camera and / or object position, and use the event detection edge to determine child nodes connected to the event detection edge (based on a graph map or a reference table based on the graph map).

[0107] At block 516, parent nodes of child nodes on the shared event detection edge may be determined. For example, the set of child nodes determined at block 514 may be used to determine parent nodes that share an edge connection to one or more of the child nodes.

[0108] At block 518, an alert message with the child node identifiers may be sent to the parent node. For example, the detection camera may send a video capture update message to each camera corresponding to each parent node determined at block 516, including one or more child node identifiers of one or more shared child nodes in the direction of the event. In some configurations, the master camera or master node may be considered to share the child nodes of all other cameras and always receive alert messages.

[0109] like Figure 5B As shown, the monitoring system 300 can operate according to an example method for modifying video capture based on event parameters and / or priorities in an alarm, i.e., according to Figure 5B The method 502 illustrated in blocks 530-540 operates.

[0110] At block 530, alarms may be monitored. For example, each camera may monitor for video capture update messages or similar alarms from other cameras, which may include local alarms based on regional alarms received by the master camera.

[0111] At block 532, a determination can be made as to whether an alert has been received. For example, one of the cameras may have detected an object of interest in its video data stream and sent an alert according to method 500, or the primary camera may have received a zone alert from another camera group. If an alert has been received, method 502 may proceed to block 534. If an alert has not been received, method 502 may return to block 530 to continue monitoring for alerts.

[0112] Event parameters and / or event priority may be determined from the alert message at block 534. For example, a video capture update message may include video event parameters in the message based on the detected video event, and each receiving camera may parse the video event parameters including the event priority from the message.

[0113] At block 536, the event location can be determined based on the graphical map. For example, event parameters such as child node identifiers can be included in locally generated alerts and / or regionally generated alerts can include video event parameters based on the master graphical map, and the receiving camera can determine the event location based on these parameters.

[0114] Modifications to the video capture operation based on the event parameters and / or event priority may be determined at block 538. For example, the receiving camera may be configured with actuator positions and fields of view corresponding to different approach paths oriented in different directions, and the receiving camera may change actuator positions and / or otherwise modify video capture operations for different video capture rates, object detector configurations, etc. based on the determined event location.

[0115] At block 540, the camera may capture video data based on the modified video capture operation. For example, in response to the modification of the video capture operation determined at block 538, the receiving camera may initiate video capture if an object of interest enters its field of view from the direction of a video event detected by another camera (either in the group or from a different group).

[0116] like Figure 6 As shown, the monitoring system 300 can operate according to an example method for predictively modifying video capture operations between networked cameras based on video events detected by one camera group that modify the operation of another camera group, i.e., according to Figure 6 The method 600 illustrated in blocks 610-634 operates.

[0117] At block 610, multiple groups of cameras may be configured in different sub-areas with known physical relationships between them represented in the graphical map. For example, a user may deploy multiple groups of networked cameras in surveillance locations with fixed positions and fixed or movable fields of view based on the orientation, size, and depth of field of the image sensors and associated lenses, and map the cameras and paths for object movement between them on the sub-areas and the main graphical map.

[0118] Network communications may be established between the cameras in each group at block 612. For example, each camera may be configured with a network interface and corresponding network protocols for peer-to-peer communications with at least one master camera and / or networking to a shared host, such as a network video recorder or a control center computer system.

[0119] A master camera or master node may be determined for each group at block 614. For example, when configuring camera groups, a user may select one camera from each group to serve as the master camera for the group.

[0120] Network communications may be established between the master cameras from each group at block 616. For example, each master camera may be configured with its network interface and corresponding network protocol for peer-to-peer communications with master cameras from other groups, either directly and / or networked to a shared host (such as a network video recorder or a control center computer system) for routing communications to the other master cameras.

[0121] At block 618, video data may be received from at least one camera using the current set of video capture operating parameters. For example, a surveillance controller of a camera may continuously receive video data from a video sensor using a passive video capture rate.

[0122] A video event can be determined from the video data at block 620. For example, a surveillance controller of one of the cameras can detect an object of interest in its field of view.

[0123] At block 622, a local video capture update message may be determined and sent by the camera to other cameras in its group. For example, a surveillance controller of a camera that detects a video event may generate an alert message with video event parameters and send it to one or more other cameras in the group, including the master camera.

[0124] The master camera may receive a local video capture update message at block 624. For example, a monitoring controller in a master camera in a group in which a video event is detected may receive an alert message from the camera that detected the event.

[0125] A regional video alarm event may be determined at block 626. For example, the monitoring controller of the master camera may evaluate video event parameters to determine whether they meet one or more event conditions for issuing a regional alarm to one or more other camera groups.

[0126] The zone video capture update message may be sent to the master camera in one or more other camera groups at block 628. For example, the monitoring controller of the master camera may send or route the zone alarm message to other master cameras based on their camera identifiers and / or corresponding network addresses.

[0127] The other master cameras may receive the regional video capture update message at block 630. For example, the other master cameras may receive the message from the originating camera group via their network interface.

[0128] At block 632, other master cameras may determine corresponding local video update messages based on the regional video capture update message. For example, in response to the message, the receiving master camera may evaluate the video event parameters in the message to generate corresponding local video update messages for one or more cameras in its group.

[0129] At block 634, the local video capture update message may be sent to its corresponding camera group. For example, each master camera that receives a regional video capture update may selectively send the local video update message generated at block 632 to one or more cameras in its group.

[0130] like Figure 7 As shown, the monitoring system 300 can operate according to an example method for modifying video capture operations based on a video capture update message based on an event detected by another camera group, i.e., according to Figure 7 In some embodiments, the method 500 may be combined with the method 700 illustrated in blocks 710-732. Figure 6 6. The method 600 of FIG.

[0131] Video capture operating parameters may be determined at block 710. For example, each camera in the camera group may operate in a default operating mode having a corresponding set of video capture operating parameters and / or may operate in one or more video capture modes in response to video events detected by the camera or within the group.

[0132] An operational priority may be determined at block 712. For example, each camera operating in its default operational mode may have the lowest operational priority, or may operate in a higher priority state having a corresponding operational priority value in response to a directly detected video event or a local video event.

[0133] An operating period may be initiated at block 714. For example, each camera may begin operating in its current operating mode, and that mode may continue until interrupted by another event, such as receipt of a video capture update message (or detection of an event in its own environment).

[0134] A video capture update message may be received at block 716. For example, a camera may receive a video capture update message, such as local video capture update information based on a regional video update message received by a master camera for its group.

[0135] The regional video capture priority may be determined from the video capture update message at block 718. For example, the received video capture update message may be based on a regional video capture update message, and the receiving camera may parse the regional video event priority value from a video event parameter in the message.

[0136] At block 720, it may be determined whether the regional video capture priority meets a threshold for overriding the current operating priority. For example, the receiving camera may compare the regional video event priority value with its current operating priority value as a threshold for determining whether to override its current video capture operating mode.

[0137] At block 722, a regional video capture update may be determined based on the regional video event parameters. For example, the receiving camera may parse the additional video event parameters from the message to determine what new operating mode and corresponding video capture parameters (such as actuator direction, capture rate, object detector parameters, etc.) should be used to interrupt the current operating mode.

[0138] At block 724, video capture operations may be modified based on the regional video capture update determined at block 722. For example, the receiving camera may modify its operating mode to support object detection based on the regional video capture update.

[0139] The duration of the modified operating period may be determined at block 726. For example, the camera may receive or determine a duration for monitoring the object of interest and set a predictive capture timer to measure the elapsed time of the duration.

[0140] Video data may be captured at block 728. For example, during the current operating period, the camera may capture video data from the video image sensor using the modified video capture operation.

[0141] The video data may be stored at block 730. For example, during the current operating period, the camera may transmit and store captured video data for use in the modified video data operation.

[0142] At block 732, the duration of the modified video capture operation may be determined to have elapsed, and the camera may return to the default video capture operating parameters. For example, if the predetermined duration is met without detecting an object of interest, the camera may return to block 714 to initiate another operating period using the original set of video capture operating parameters.

[0143] like Figure 8 As shown, the monitoring system 300 can operate according to an example method for using object detector parameters from one camera group to selectively modify the operation of another camera group in response to a detected object of interest, i.e., according to Figure 8 The method 800 illustrated in blocks 810-834 operates.

[0144] The camera groups in each group are determined at block 810. For example, during installation of a video surveillance system, an installer may group cameras installed in various sub-areas into sub-area sets of networked cameras.

[0145] At block 812, one or more object types may be determined as objects of interest for each camera group. For example, each group in the surveillance system may be configured to monitor people, vehicles, animals, inventory, or one or more other objects of interest.

[0146] At block 814, an object detector may be configured for the object of interest. For example, each camera may include an object detector of a selected type that is trained to detect objects of that type that meet specific criteria of interest and determine corresponding object characteristics. The object detector may be trained directly, or a previously trained object detector model for the object of interest may be loaded.

[0147] Video data may be captured at block 816. For example, a camera may operate according to its default video capture mode of operation to capture, analyze, and / or store video data from its image sensor.

[0148] At block 818, objects of interest may be detected. For example, a camera may detect objects of a selected object type from its video data.

[0149] At block 820, object parameters of the detected object of interest may be determined. For example, the camera may determine object type, location, timing, bounding box, size, and other parameters from object detection, as well as one or more object parameters based on further classification, such as color, subtype, activity type, and the like.

[0150] At block 822, it may be determined that the detected object of interest triggers a regional video alarm event. For example, the detection camera, the corresponding master camera, and / or the control center may determine that the detected object satisfies one or more conditions of a regional video alarm event.

[0151] The object parameters of the detected object of interest may be included in a regional video capture update message at block 824. For example, a master camera or a control center computer system may generate a regional video capture update message and include one or more object parameters from a detected video event, such object type and other parameters, extracted video data (such as frame or bounding box contents) of the detected object, and / or object detector parameters used to detect the object.

[0152] At block 826, object parameters of the detected object of interest may be included in a local video capture update message. For example, a master camera that receives a regional video capture update message may generate a local video capture update message and include one or more object parameters from the regional video capture update message.

[0153] The object parameters may be determined from the local video update message at block 828. For example, the cameras in the receiving group may parse the object parameters from the local video capture update message sent by its master camera.

[0154] At block 830, object detector parameters may be modified for the object of interest. For example, cameras in the receiving group may change their object detector parameters to adjust or prioritize the object type and / or other characteristics of the object of interest to increase the likelihood of detecting the object of interest.

[0155] At block 832, the updated object detector parameters may be used to capture video data. For example, the receiving camera may enter an operational mode using the object detector parameters from block 830.

[0156] At block 834, the object of interest may be detected in another group. For example, one or more of the receiving cameras may successfully detect the object of interest using the updated object detection parameters when the object of interest enters its field of view and return a corresponding alert message.

[0157] like Figure 9 As shown, the monitoring system 300 can operate according to an example method for generating and using a regional video capture update message, namely, according to Figure 9 In some configurations, some or all of blocks 910-924 may be performed by a control center computer system rather than the master camera of the camera group that detected the event.

[0158] At block 910, a local video capture update message may be received. For example, a master camera may receive a local video capture update message from one of the cameras in its group based on its sub-region graphical map.

[0159] At block 912, video event parameters may be determined. For example, based on the local video capture update message, the master camera may determine a set of video event parameters for the local video event that generated the local video capture update message.

[0160] A regional video alarm event may be determined at block 914. For example, the master camera may evaluate the set of video event parameters against one or more conditions for triggering a regional video alarm event.

[0161] At block 916, a regional video alarm priority value may be determined. For example, the master camera may use one or more conditions for triggering a regional video alarm event to assign or generate an event priority value for a local video event.

[0162] Video event parameters may be selected for inclusion in the regional video capture update message at block 918. For example, the master camera may select some or all of the video event parameters included in the local video capture update message for inclusion in the regional video capture update message, and may include the regional video alarm priority value.

[0163] The zone video alarm priority value may be compared to a group event priority threshold at block 920. For example, each camera group may be assigned a threshold priority value to receive zone video alarms, and the master camera may compare the zone video alarm priority value to the priority thresholds of each other group.

[0164] At block 922, a camera group may be selected to receive a zone video capture update message. For example, a master camera may select another group to receive a zone alarm based on the comparison at block 920 where the alarm priority value meets the priority threshold for the group.

[0165] At block 924, a regional video capture update message may be sent. For example, the master camera sends the generated regional video capture update message to the master cameras of the selected group.

[0166] At block 926, the regional video capture update message may be received by the other camera groups. For example, the primary camera in the other camera groups selected at block 922 may receive the regional video capture update message.

[0167] At block 928, a regional video alarm priority value may be determined. For example, the receiving master camera may parse the regional video alarm priority value from the regional video capture update message.

[0168] At block 930, cameras in the group may be selected to receive corresponding local video capture update messages. For example, based on the regional video alarm priority value and / or other video event parameters in the regional video capture update message, the receiving master camera may determine which other cameras in the group should receive local video capture update messages based on the regional video capture update message.

[0169] At block 932, a local video capture update message may be sent to the cameras in the group. For example, the receiving master camera may send a local video capture update message to the other cameras in the group selected at block 930.

[0170] Although at least one exemplary embodiment has been presented in the foregoing detailed description of the technology, it will be understood that a large number of variations are possible. It will also be understood that the exemplary embodiment or embodiments are examples and are not intended to limit the scope, applicability, or configuration of the technology in any way. On the contrary, the foregoing detailed description will provide those skilled in the art with a convenient guide for implementing the exemplary embodiments of the technology, and it will be understood that various modifications may be made to the function and / or arrangement of the elements described in the exemplary embodiments without departing from the scope of the technology as set forth in the appended claims and their legal equivalents.

[0171] As will be appreciated by those skilled in the art, various aspects of the present technology may be embodied as systems, methods, or computer program products. Thus, some aspects of the present technology may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or a combination of hardware and software aspects, all of which may generally be referred to herein as circuits, modules, systems, and / or networks. Furthermore, various aspects of the present technology may take the form of a computer program product embodied in one or more computer-readable media, including computer-readable program code embodied thereon.

[0172] Any combination of one or more computer-readable media can be utilized. A computer-readable medium can be a computer-readable signal medium or a physical computer-readable storage medium. For example, a physical computer-readable storage medium can be, but is not limited to, an electronic, magnetic, optical, crystal, polymer, electromagnetic, infrared or semiconductor system, device or equipment, etc., or any suitable combination of the foregoing. Non-limiting examples of physical computer-readable storage media can include, but are not limited to, an electrical connection comprising one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, an optical fiber, a compact disc read-only memory (CD-ROM), an optical processor, a magnetic processor, etc., or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium can be any tangible medium that can contain or store a program or data for use by an instruction execution system, device, and / or equipment or in combination with it.

[0173] Any combination of processors, CPUs, controllers, or similar hardware circuits may be used to execute instructions stored on a computer-readable medium. For example, the processors, CPUs, controllers, and similar hardware circuits described herein may be embodied in or include one or more hardware processor packages and / or processor cores that operate individually or in combination to execute instructions and perform the described functions. In some configurations, these processors or groups of processors may be stand-alone circuits in their own packages, integrated with other hardware elements in a system on a chip (SOC), an application-specific integrated circuit (ASIC), or the like, and / or integrated in a printed circuit board assembly (PCBA) via a communication bus.

[0174] The computer code embodied on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, fiber optic cable, radio frequency (RF), etc., or any suitable combination of the foregoing. The computer code for performing the operations of various aspects of the present technology may be written in any static language, such as the C programming language or other similar programming languages. The computer code may be executed entirely on the user's computing device, partially on the user's computing device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the latter case, the remote computing device may be connected to the user's computing device via any type of network or communication system, including but not limited to, a local area network (LAN) or wide area network (WAN), a unified network or connection to an external computer (e.g., via the Internet using an Internet service provider).

[0175] Aspects of the present technology may be described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus, systems, and computer program products. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, may be implemented by computer program instructions. These computer program instructions may be provided to a processing device (processor) of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, so that instructions executed by the processing device or other programmable data processing apparatus may create a device for implementing the operations / actions specified in the flowchart and / or block diagrams.

[0176] Some computer program instructions may also be stored in a computer-readable medium that can instruct a computer, other programmable data processing apparatus, or other device to operate in a specific manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the operations / actions specified in the flowcharts and / or blocks in the block diagrams. Some computer program instructions may also be loaded onto a computing device, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computing device, other programmable apparatus, or other device to produce a computer-implemented process, such that the instructions executed by the computer or other programmable apparatus provide one or more processes for implementing the operations / actions specified in the flowcharts and / or blocks in the block diagrams.

[0177] The flowcharts and / or block diagrams in the above figures can illustrate the architecture, functions and / or operations of the possible specific implementations of the devices, systems, methods and / or computer program products according to various aspects of the present technology. In this regard, the boxes in the flowchart or block diagram can represent modules, fragments or parts of the code, which may include one or more executable instructions for implementing one or more specified logical functions. It should also be noted that in some alternative aspects, some functions shown in the box may not occur in the order shown in the figure. For example, in fact, the two boxes shown in succession can be executed substantially simultaneously, or these boxes can sometimes be executed in the opposite order, depending on the operations involved. It should also be noted that the boxes illustrated in the block diagram and / or flowchart or the combination of boxes in the block diagram and / or flowchart can be implemented by a system based on dedicated hardware that can perform one or more specified operations or actions, or a combination of dedicated hardware and computer instructions.

[0178] While one or more aspects of the present technology have been illustrated and discussed in detail, those skilled in the art will appreciate that modifications and / or adaptations may be made to the various aspects without departing from the scope of the technology as set forth in the appended claims.

Claims

1. A system, comprising: A first camera comprising a network interface configured to communicate with: a first plurality of networked cameras, wherein said first plurality of networked cameras includes said first camera; and a second camera of a second plurality of networked cameras; and at least one controller in communication with the first plurality of networked cameras and the second camera, the at least one controller being configured, alone or in combination: receiving a first video capture update message from at least one camera of the first plurality of networked cameras based on a first graphical mapping of the first plurality of networked cameras; determining a regional video alarm event based on the first video capture update message; as well as and sending a second video capture update message to the second camera based on the regional video alarm event, wherein the second camera is configured to send a third video capture update message to at least one other camera of the second plurality of networked cameras in response to the second video capture update message and based on a second graphical mapping of the second plurality of networked cameras.

2. The system according to claim 1, further comprising: the first plurality of networked cameras, wherein each camera of the first plurality of networked cameras is configured to, in response to receiving the first video capture update message: determining at least one event parameter from the first video capture update message; modifying a video capture operation of the camera of the first plurality of networked cameras based on the at least one event parameter; as well as Based on the modified video capture operation, video data is captured using the camera.

3. The system according to claim 1, further comprising: the second plurality of networked cameras, wherein each camera in the second plurality of networked cameras is configured to, in response to receiving the third video capture update message: determining a regional video capture modification from the third video capture update message; modifying a video capture operation of the camera in the second plurality of networked cameras based on the regional video capture modification; as well as Based on the modified video capture operation, video data is captured using the camera.

4. The system of claim 1 , wherein: The first graphical mapping includes: First coordinate position system; a first set of coordinates and a primary indicator of the first camera; a second set of coordinates for each other camera in the first plurality of networked cameras; and a third set of coordinates for each child node between cameras in the first plurality of networked cameras, wherein each child node is mapped to at least two parent nodes of a corresponding camera in the first plurality of networked cameras; and The second graphical mapping includes: Second coordinate position system; a fourth set of coordinates and a primary indicator of the second camera; a fifth set of coordinates for each other camera in the second plurality of networked cameras; and A sixth set of coordinates for each child node between cameras in the second plurality of networked cameras, wherein each child node is mapped to at least two parent nodes of a corresponding camera in the second plurality of networked cameras.

5. The system of claim 4, wherein the first coordinate position system and the second coordinate position system are different.

6. The system of claim 1 , wherein: at least one camera of the first plurality of networked cameras includes a first object detector configured to detect a first object of interest in video data captured by the camera; at least one camera of the second plurality of networked cameras comprises a second object detector configured to detect a second object of interest in video data captured by the camera; the first video capture update message being responsive to detecting the first object of interest; The second video capture update message includes at least one parameter of the first object of interest; and The at least one camera of the second plurality of networked cameras is configured to modify at least one operating parameter of the second object detector in response to the third video capture update message to detect the first object of interest in the video data captured by the camera.

7. The system of claim 1 , wherein: The second video capture update message includes at least one parameter selected from the following: a sample of video data of the regional video alarm event; coordinates of the camera of the first plurality of networked cameras that captured the regional video alarm event; and a direction of travel of an object of interest determined for the area video alarm event; and The at least one controller is further configured to determine the second plurality of networked cameras and the second camera based on the selected at least one parameter, alone or in combination.

8. The system according to claim 1, further comprising: the first plurality of networked cameras corresponding to a first sub-area mapped to a first set of physical locations; the second plurality of networked cameras corresponding to a second sub-region mapped to a second set of physical locations; and a third plurality of networked cameras corresponding to a third sub-region mapped to a third set of physical locations, wherein: the first graphical map being based on the first set of physical locations; the second graphical map being based on the second set of physical locations; a third graphical map corresponding to the third plurality of networked cameras and based on the third set of physical locations; and The at least one controller is further configured to select between the second plurality of networked cameras and the third plurality of networked cameras to receive the second video capture update message based on the video zone alarm event, alone or in combination.

9. The system of claim 1 , wherein: The at least one controller is further configured, alone or in combination: determining a priority value of the regional video event alarm based on the regional video alarm event; comparing the priority value to at least one event priority threshold; as well as responsive to the priority value satisfying at least one event priority threshold, selecting at least one plurality of networked cameras from a group of a plurality of networked cameras including the second plurality of networked cameras; The second video capture update message includes the priority value; and The second plurality of networked cameras are further configured to selectively modify video capture operations based on the priority value.

10. The system according to claim 1, further comprising: a computer system remote from the first plurality of networked cameras and the second plurality of networked cameras, wherein: The computing system includes: a plurality of graphical maps corresponding to a plurality of sub-region sets of networked cameras; a network interface configured to communicate with a master camera in each of the plurality of sub-area sets; and said at least one controller; the first plurality of networked cameras being a first sub-regional set corresponding to a first set of physical locations monitored by the computer system; The first camera is a master camera of the first plurality of networked cameras; the second plurality of networked cameras being a second sub-regional set corresponding to a second set of physical locations monitored by the computer system; and The second camera is a master camera of the second plurality of networked cameras.

11. A computer-implemented method, comprising: receiving a first video capture update message from a first camera of a first plurality of networked cameras and from at least one camera of the first plurality of networked cameras based on a first graphical mapping of the first plurality of networked cameras; determining a regional video alarm event based on the first video capture update message; sending a second video capture update message to a second camera in a second plurality of networked cameras based on the regional video alarm event; as well as A third video capture update message is sent by the second camera to at least one other camera of the second plurality of networked cameras based on the second graphical mapping of the second plurality of networked cameras and in response to the second video capture update message.

12. The computer-implemented method of claim 11 , further comprising, in response to receiving the first video capture update message and by at least one camera of the first plurality of networked cameras: determining at least one event parameter from the first video capture update message; modifying a video capture operation of the camera of the first plurality of networked cameras based on the at least one event parameter; as well as Based on the modified video capture operation, video data is captured using the camera.

13. The computer-implemented method of claim 11 , further comprising, in response to receiving the third video capture update message and by at least one camera of the second plurality of network-connected cameras: determining a regional video capture modification from the third video capture update message; modifying a video capture operation of the camera in the second plurality of networked cameras based on the regional video capture modification; as well as Based on the modified video capture operation, video data is captured using the camera.

14. The computer-implemented method of claim 11 , wherein: The first graphical mapping includes: First coordinate position system; a first set of coordinates and a primary indicator of the first camera; a second set of coordinates for each other camera in the first plurality of networked cameras; and a third set of coordinates for each child node between cameras in the first plurality of networked cameras, wherein each child node is mapped to at least two parent nodes of a corresponding camera in the first plurality of networked cameras; and The second graphical mapping includes: Second coordinate position system; a fourth set of coordinates and a primary indicator of the second camera; a fifth set of coordinates for each other camera in the second plurality of networked cameras; and A sixth set of coordinates for each child node between cameras in the second plurality of networked cameras, wherein each child node is mapped to at least two parent nodes of a corresponding camera in the second plurality of networked cameras.

15. The computer-implemented method of claim 14, wherein the first coordinate position system and the second coordinate position system are different.

16. The computer-implemented method of claim 11 , further comprising: detecting, by at least one camera of the first plurality of networked cameras and using a first object detector, a first object of interest in video data captured by the camera; as well as modifying, by at least one camera of the second plurality of networked cameras and in response to the third video capture update message, at least one operating parameter of a second object detector to detect the first object of interest in the video data captured by the camera, wherein: The first video capture update message is responsive to detecting the first object of interest; and The second video capture update message includes at least one parameter of the first object of interest.

17. The computer-implemented method of claim 11 , further comprising: The second video capture update message includes at least one parameter selected from the following: a sample of video data of the regional video alarm event; coordinates of the camera of the first plurality of networked cameras that captured the regional video alarm event; and a direction of travel of an object of interest determined for said area video alarm event; as well as The second plurality of networked cameras and the second camera are determined based on the selected at least one parameter.

18. The computer-implemented method of claim 11 , further comprising: selecting between the second plurality of networked cameras and a third plurality of networked cameras to receive the second video capture update message based on the video zone alarm event, wherein: The first plurality of networked cameras corresponds to a first sub-region mapped to a first set of physical locations; the second plurality of networked cameras corresponding to a second sub-region mapped to a second set of physical locations; the third plurality of networked cameras corresponding to a third sub-region mapped to a third set of physical locations; the first graphical map being based on the first set of physical locations; the second graphical map is based on the second set of physical locations; and A third graphical map corresponds to the third plurality of networked cameras and is based on the third set of physical locations.

19. The computer-implemented method of claim 11 , further comprising: determining a priority value of the regional video event alarm based on the regional video alarm event; comparing the priority value to at least one event priority threshold; selecting at least one plurality of networked cameras from a group of a plurality of networked cameras comprising the second plurality of networked cameras, in response to the priority value satisfying the at least one event priority threshold, wherein the second video capture update message includes the priority value; as well as Video capture operations are selectively modified by at least one camera of the second plurality of network cameras and based on the priority value.

20. A system comprising: a first plurality of networked cameras including a first camera; a second plurality of networked cameras including a second camera; at least one processor; at least one memory; means for receiving a first video capture update message from the first camera and from at least one camera of the first plurality of networked cameras based on a first graphical mapping of the first plurality of networked cameras; means for determining a regional video alarm event based on said first video capture update message; means for sending a second video capture update message to the second camera based on the regional video alarm event; and Means for sending, by the second camera and in response to the second video capture update message, a third video capture update message to at least one other camera of the second plurality of networked cameras based on the second graphical map of the second plurality of networked cameras.

Images