Object detection via region of interest

Abstract

The method includes: identifying a pixel indicative of motion based on a pixel frame; selecting a region of interest within the frame based on the pixels indicative of motion, the region being a subset of the frame of pixels and including the pixels indicative of motion; identifying an object based on pixels within the region; and issuing an alert in response to the region including both the object and the pixel indicative of motion.

Description

Technical Field

[0001] Aspects of the technology described herein relate to security systems and methods. Background Art

[0002] Some monitoring systems use one or more cameras to capture images of the area around or within a residence or business. Such monitoring systems can process the images locally and transmit the captured images to a remote service. If motion is detected, the monitoring system can send an alert to one or more user devices. Summary of the invention

[0003] The present disclosure relates to techniques for detecting an object depicted within an image using a region of interest. In at least one example, a method is provided. The method includes: identifying pixels indicating motion based on a frame of pixels; selecting a region of interest within the frame based on the pixels indicating motion, the region being a subset of the frame of pixels and including pixels indicating motion; identifying an object based on pixels within the region; and issuing an alarm in response to the region including both the object and pixels indicating motion.

[0004] In another example, a computing device is provided. The computing device includes: a memory; and at least one processor, the at least one processor coupled to the memory and configured to: identify pixels indicating motion based on a frame of pixels; select a region of interest within the frame based on the pixels indicating motion, the region being a subset of the frame of pixels and including pixels indicating motion; identify an object based on pixels within the region; and issue an alarm in response to the region including both the object and pixels indicating motion.

[0005] In another example, one or more computer-readable media are provided. One or more computer-readable media store a sequence of instructions that are executable to detect an object in an image data frame using a region of interest. The sequence of instructions includes: instructions for identifying pixels indicating motion based on a pixel frame; instructions for selecting a region of interest within a frame based on the pixels indicating motion, the region being a subset of the pixel frame and including pixels indicating motion; instructions for identifying an object based on pixels within the region; and instructions for issuing an alarm in response to the region including both an object and pixels indicating motion. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Additional examples of the present disclosure and its features and advantages will become more apparent by reference to the description herein taken in conjunction with the accompanying drawings, which are incorporated into and constitute a part of this disclosure. The drawings are not necessarily drawn to scale.

[0007] Figure 1 is a schematic diagram of a security system according to some examples described herein.

[0008] Figure 2 is a schematic diagram of a base station according to some examples described herein.

[0009] Figure 3 is a schematic diagram of a keypad according to some examples described herein.

[0010] Figure 4A is a schematic diagram of a security sensor according to some examples described herein.

[0011] Figure 4B is a schematic diagram of an image capture device according to some examples described herein.

[0012] Figure 4C is a schematic diagram of another image capture device according to some examples described herein.

[0013] Figure 5 is a schematic diagram of a data center environment, a monitoring center environment, and customer equipment according to some examples described herein.

[0014] Figure 6 is a sequence diagram of a monitoring process according to some examples described herein.

[0015] Figure 7 is a schematic diagram of a security device according to some examples described herein.

[0016] Figure 8 Based on some of the examples described in this article Figure 7 A sequence diagram of the operational processes performed by the characteristics of a safety device.

[0017] Fig. 9 is a flow chart of an object detection process according to some examples described herein.

[0018] Fig.10 is a flow chart illustrating a process of generating a region of interest (ROI) according to some examples described herein.

[0019] Fig.11 is a flow chart illustrating a ROI merging process according to some examples described herein.

[0020] Fig. 12A is a diagram illustrating a frame of image data according to some examples described herein.

[0021] Fig. 12B is a diagram showing some examples described herein. Fig. 12A A map of the ROI associated with a frame of image data.

[0022] Fig. 12C is a diagram showing some examples described herein. Fig. 12AA map of the translation of the ROI associated with the image data frame.

[0023] Fig.12D is a diagram showing some examples described herein. Fig. 12A A cropped view of the ROI associated with the image data frame.

[0024] Fig.12E is a diagram showing some examples described herein. Fig.12D A graph of the objects detected within the crop.

[0025] Fig.13 is a schematic diagram of a computing device according to some examples described herein. DETAILED DESCRIPTION

[0026] As outlined above, at least some examples disclosed herein relate to systems and processes for detecting objects represented within image data. In some implementations, these systems and processes are utilized in security systems to effectively monitor a space that is physically proximate to a device that monitors the space. In these implementations, the systems and processes exploit properties of the operating environment to achieve technical advantages over other security systems, as will be further described below. Among other things, these advantages include reduced power consumption, reduced false positive detection of security threats, and / or improved detection of small objects.

[0027] Security systems may include a series of sensors configured to detect various events such as motion, humidity, temperature changes, and sound. For example, a passive infrared (PIR) sensor is a motion sensor that detects temperature changes in the field of view. The PIR sensor may be configured with a threshold so that any change greater than the threshold constitutes motion and causes a motion trigger. An imaging sensor may be able to distinguish between certain objects such as people in a captured image frame, for example. An image sensor may be configured to trigger an object detection alarm if an object of interest is identified within an image frame. An imaging sensor may use any of a variety of techniques to locate and identify objects in a frame. For example, computer vision-based object detection may use specific filters to locate different attributes or features within an image frame, and then combine features to classify whether a particular class of objects is found. For example, an object detector may locate all faces in a frame. Recently, methods based on machine learning (ML) or artificial intelligence (AI) have been used, in which a process or model is trained on a large number of images containing objects of interest to identify similar objects in new or unseen images.

[0028] When deploying AI or ML models within safety systems including battery-powered devices, some acute issues and constraints are memory capacity and model efficiency. A common working practice that favors these constraints is to deploy smaller models in both model size and operating resolution. While a smaller operating resolution for model inference can allow for improved inference rates, it also requires image preprocessing that includes padding the image (for reshaping) or resizing the image to fit the model input dimensions. Both of these preprocessing steps can result in distortion of the original image context such as loss of pixel data under downsampling or reduction in the relative size of objects within the image. These issues can raise questions about how the model can effectively detect objects within an image frame, especially in the case of relatively small objects that may be virtually erased when downsampled.

[0029] In view of these and other problems and constraints, at least some of the examples disclosed herein retain image data and context by cropping a region of interest (ROI) from an image rather than reshaping or resizing the image. Cropping is different from reshaping or resizing because cropping removes unwanted background from an image, thereby increasing the percentage of pixels in the image used to depict an object within the crop. Changing the size of an image (which may involve upsampling or downsampling) changes the resolution of the image while maintaining the proportion of the original content. Reshaping an image changes the aspect ratio of the image and may involve the insertion of padding (added pixels). In some examples, a crop can be created at or near the size of a model input dimension, which allows minimization to no downsampling. In addition, a square crop can be made from the image, which is usually required for the model input to not allow additional padding or image reshaping. Cropping from the image at the original dimensions of the image in this way allows the retention of unadulterated pixel data depicting the object, as well as the retention of the relevant structural context of the object residing in the frame. This is because cropping neither changes the pixels within the crop nor the pixel ratios relative to each other. Passing only the ROI from the image to the model for inference also removes extraneous image context from the original image that may lead to false positive detections.

[0030] To create crops for model inference, the target regions within the image identified by the motion detector can be used to propose ROIs. Fig.12DFurther shown is an example cropping as a subset of pixels from an image. When motion is detected, a bounding box enclosing the motion area is output. These motion areas can be used as targets to create ROIs for cropping images. ROIs are associated with specific images. By using the motion area as the midpoint of the proposed ROI, either side of the target coordinates and above and below are filled (extended) with half of the desired ROI dimension. This creates a set of ROI boundaries useful for creating image cropping. In some examples, image cropping can be created in the model input dimension centered on the target (e.g., an object in motion). This ROI-based object detection method enables the use of a smaller, resource-saving model without sacrificing object detection accuracy. This is because the ROI includes fewer pixels than the original image. The method recognizes and utilizes the following understanding that objects represented in the image can be accurately identified by analyzing only a portion of the image frame rather than the entire image frame. Since security system equipment is often placed indoors and / or near a secure location or object, this close physical proximity often (if not always) exists within the security system context.

[0031] Then, various examples are described herein, but it is apparent to one of ordinary skill in the art that many more examples and implementations are possible. Therefore, the examples described herein are not the only possible examples and implementations. Furthermore, the advantages described above are not necessarily the only advantages, and it is not necessarily expected that all of the advantages described will be achieved by each example.

[0032] For the purpose of promoting an understanding of the principles of the present disclosure, reference will now be made to the examples illustrated in the drawings and specific language will be used to describe the examples illustrated in the drawings. It should be understood, however, that no limitation is intended thereby to the scope of the examples described herein.

[0033] Figure 1 is a schematic diagram of a security system 100 configured to monitor geographically different locations according to some examples. Figure 1 As shown in FIG. 1 , system 100 includes monitoring location 102A, monitoring center environment 120, data center environment 124, one or more customer devices 122, and communication network 118. Each of monitoring location 102A, monitoring center 120, data center 124, one or more customer devices 122, and communication network 118 includes (e.g., as described below with reference to Fig.131 . The one or more client devices 122 are configured to host one or more client interface applications 132. The monitoring center environment 120 is configured to host one or more monitor interface applications 130. The data center environment 124 is configured to host a monitoring service 128 and one or more transmission services 126. Location 102A includes image capture devices 104 and 110, a contact sensor assembly 106, a keypad 108, a motion sensor assembly 112, a base station 114, and a router 116. The base station 114 hosts a monitoring client 136. The image capture device 110 hosts a camera agent 138. The security devices (e.g., devices 104, 106, 108, 110, 112, and 114) disposed at location 102A may be referred to herein as location-based devices.

[0034] In some examples, router 116 is a wireless router that is configured to communicate with location-based devices via communications that conform to a communication standard, such as any of the various Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards. Figure 1 , router 116 is also configured to communicate with network 118. It should be noted that router 116 implements a local area network (LAN) within and near location 102A by way of example only. Other networking technologies involving other computing devices are suitable for use within location 102A. For example, in some examples, base station 114 can receive and forward communication packets transmitted by image capture device 110 via a point-to-point personal area network (PAN) protocol such as Bluetooth. Other wired, wireless, and mesh network technologies and topologies will clearly benefit from the present disclosure and are intended to fall within the scope of the examples disclosed herein.

[0035] continue Figure 1For example, the network 118 may include one or more public networks and / or private networks that support, for example, IP. The network 118 may include, for example, one or more LANs, one or more PANs, and / or one or more wide area networks (WANs). The LAN may include a wired network or a wireless network that supports various LAN standards such as versions of IEEE 802.11, etc. The PAN may include a wired network or a wireless network that supports various PAN standards such as Bluetooth, ZIGBEE, etc. The WAN may include a wired network or a wireless network that supports various WAN standards such as the code division multiple access (CDMA) radio standard, the global system for mobile (GSM) radio standard, etc. The network 118 connects the computing devices, the monitoring center environment 120, the data center environment 124, and the client device 122 in the location 102A and implements data communication between the computing devices, the monitoring center environment 120, the data center environment 124, and the client device 122 in the location 102A. In at least some examples, both monitoring center environment 120 and data center environment 124 include network devices (e.g., similar to router 116) configured to communicate with network 118 and computing devices collocated with or proximate to the network equipment. It should be noted that in some examples, network 118 and existing networks within location 102A support other communication protocols such as MQTT or other IoT protocols.

[0036] continue Figure 1 In one example, data center environment 124 may include physical space, communications, cooling, and power infrastructure to support networking operations of computing devices. For example, such infrastructure may include rack space into which computing devices are mounted, uninterruptible power supplies, cooling plenums and equipment, and networking equipment. Data center environment 124 may be dedicated to security system 100, may be a non-dedicated, commercial cloud computing service (e.g., MICROSOFT AZURE, AMAZON WEBSERVICES, GOOGLE CLOUD, etc.), or may include a hybrid configuration consisting of dedicated resources and non-dedicated resources. Regardless of the physical or logical configuration of data center environment 124, such as Figure 1 As shown in , data center environment 124 is configured to host monitoring service 128 and transport service 126 .

[0037] continue Figure 1 For example, monitoring center environment 120 may include multiple computing devices (e.g., desktop computers) and network devices (e.g., one or more routers) connected to the computing devices and network 118. Client devices 122 may include personal computing devices (e.g., desktop computers, laptop computers, tablet computers, smartphones, etc.) and network devices (e.g., routers, cellular modems, cellular radios, etc.). Figure 1As shown in , monitoring center environment 120 is configured to host monitor interface 130 , and customer device 122 is configured to host customer interface 132 .

[0038] continue Figure 1 In an example of, devices 104, 106, 110, and 112 are configured to acquire analog signals via sensors incorporated into the devices, generate digital sensor data based on the acquired signals, and transmit the sensor data to base station 114 (e.g., via a wireless link with router 116). The type of sensor data generated and transmitted by these devices varies with the type of sensors included in the devices. For example, image capture devices 104 and 110 can acquire ambient light, generate image data frames based on the acquired light, and transmit the frames to base station 114, monitor interface 130, and / or customer interface 132, although pixel resolution and frame rate may vary depending on the capabilities of the devices. Figure 1 As shown in , image capture device 104 has a field of view (FOV) that starts near the front door of location 102A and can capture images of the pedestrian walkway, the highway, and the space between location 102A and the highway. Image capture device 110 has a FOV that starts near the bathroom of location 102A and can capture images of the living room and dining area of ​​location 102A. Image capture device 110 can also capture images of the outdoor area outside of location 102A through windows 117A and 117B on the right side of location 102A.

[0039] In addition, if Figure 1 As shown in , in some examples, the image capture device 110 is configured to communicate with the monitoring service 128, the monitor interface 130, and the customer interface 132 separately from the monitoring client 136 via the execution of the camera agent 138. These communications may include sensor data generated by the image capture device 110 and / or commands sent by the monitoring service 128, the monitor interface 130, and / or the customer interface 132 to be executed by the image capture device 110. The commands may include, for example, a request for an interactive communication session in which monitoring personnel and / or customers interact with the image capture device 110 via the monitor interface 130 and the customer interface 132. These interactions may include requests for the image capture device 110 to transmit additional sensor data and / or requests for the image capture device 110 to communicate via a user interface (e.g., Figure 4B and Figure 4C The user interface 412 of the embodiment of the present invention may present a request for output. The output may include audio and / or video output.

[0040] continue Figure 1In an example of, the contact sensor assembly 106 includes the following sensor, which can detect the presence or absence of a magnetic field generated by a magnet when a magnet is close to the sensor. When the magnetic field is present, the contact sensor assembly 106 generates Boolean sensor data specifying a closed state. When the magnetic field is not present, the contact sensor assembly 106 generates Boolean sensor data specifying an open state. In either case, the contact sensor assembly 106 can transmit sensor data indicating whether the front door of the location 102A is open or closed to the base station 114. The motion sensor assembly 112 may include an audio emitting device that can radiate sound (e.g., ultrasound) waves and an audio sensor that can obtain reflections of the waves. When the audio sensor detects a reflection because no object is in motion in the space monitored by the audio sensor, the motion sensor assembly 112 generates Boolean sensor data specifying a stationary state. When the audio sensor does not detect a reflection because an object is in motion in the monitored space, the motion sensor assembly 112 generates Boolean sensor data specifying an alarm state. In either case, the motion sensor assembly 112 can transmit the sensor data to the base station 114. It should be noted that the specific sensing modalities described above are not limited to the present disclosure. For example, as one of many possible examples, the motion sensor assembly 112 may base its operation on the acquisition of changes in temperature rather than changes in reflected sound waves.

[0041] continue Figure 1 In some examples, the keypad 108 is configured to interact with the user and interoperate with other location-based devices in response to the interaction with the user. For example, in some examples, the keypad 108 is configured to receive input specifying one or more commands from the user and transmit the specified commands to one or more addressing processes. These addressing processes may include processes implemented by one or more of the location-based devices and / or one or more of the monitor interfaces 130 or the monitoring service 128. The command may include, for example, a code to authenticate the user as a resident of the location 102A and / or a code requesting activation or deactivation of one or more of the location-based devices. Alternatively or additionally, in some examples, the keypad 108 includes a user interface (e.g., a tactile interface such as a set of physical buttons or a set of virtual buttons on a touch screen) configured to interact with the user (e.g., receive input from the user and / or present output to the user). Further, in some examples, the keypad 108 can receive and respond to the transmitted commands, and present the response as a visual or audio output via the user interface.

[0042] continue Figure 1In some examples, the base station 114 is configured to interoperate with other location-based devices to provide local command and control and store and forward functionality via execution of the monitoring client 136. In some examples, to implement the store and forward functionality, the base station 114 receives sensor data through execution of the monitoring client 136, packages the data for transmission, and stores the packaged sensor data in a local memory for subsequent communication. Such communication of the packaged sensor data may include, for example, transmitting the packaged sensor data as a payload of a message to one or more of the transmission services 126 when a communication link to the transmission service 126 via the network 118 is operable. In some examples, packaging the sensor data may include filtering the sensor data and / or generating one or more summaries of multiple sensor readings (maximum, minimum, average, change in value since a previous communication of the sensor data, etc.). To implement the local command and control functionality, the base station 114 performs various programmed operations under the control of the monitoring client 136 in response to various events. Examples of these events may include receiving a command from keypad 108 or customer interface application 132, receiving a command from customer interface application 132 or one of monitor interfaces 130 via network 118, or detecting the occurrence of a predetermined event. Programmatic operations performed by base station 114 under the control of monitoring client 136 may include: activating or deactivating one or more of devices 104, 106, 108, 110, and 112; sounding an alarm; reporting an event to monitoring service 128; and transmitting location data to one or more of transmission services 126, to name a few example operations. Location data may include data specifying sensor readings (sensor data), configuration data for any of the location-based devices, commands entered and received from a user (e.g., via keypad 108 or customer interface 132), or data derived from one or more of these data types (e.g., filtered sensor data, a summary of sensor data, event data specifying an event detected at a location via sensor data, etc.).

[0043] continue Figure 1In some examples, the transport service 126 is configured to securely, reliably, and efficiently exchange messages between processes implemented by location-based devices and processes implemented by other devices in the system 100. These other devices may include client devices 122, devices disposed in the data center environment 124, and / or devices disposed in the monitoring center environment 120. In some examples, the transport service 126 is also configured to parse messages from location-based devices to extract payloads included in the messages, and store the payloads and / or data derived from the payloads in one or more data stores hosted in the data center environment 124. The data contained in these data stores can then be accessed by, for example, the monitoring service 128, the monitor interface 130, and the client interface 132.

[0044] In some examples, the transport service 126 exposes and implements one or more application programming interfaces (APIs) configured to receive, process, and respond to calls from processes (e.g., monitoring client 136) implemented by a base station (e.g., base station 114) and / or processes (e.g., camera agent 138) implemented by other devices (e.g., image capture device 110). Individual instances of the transport service within the transport service 126 can be associated with certain manufacturers and models of location-based monitoring devices (e.g., SIMPLISAFE devices, RING devices, etc.) and are specific to the manufacturers and models of location-based monitoring devices (e.g., SIMPLISAFE devices, RING devices, etc.). The API can be implemented using various architectural styles and interoperability standards. For example, in one example, the API is a web service interface implemented using a representational state transfer (REST) ​​architectural style. In this example, the API calls are encoded in a hypertext transfer protocol (HTTP) with JavaScript object notation (JSON) and / or extensible markup language (XML). These API calls are addressed to one or more uniform resource locators (URLs) that are API endpoints monitored by the transport service 126. In some examples, portions of the HTTP communication are encrypted for increased security. Alternatively or additionally, in some examples, the API is implemented as an MQTT agent that receives messages and transmits response messages to an MQTT client hosted by a base station and / or other device. Alternatively or additionally, in some examples, the API is implemented using simple file transfer protocol commands. Therefore, the transport service 126 is not limited to a particular protocol or architectural style. It should be noted that in at least some examples, the transport service 126 can transmit one or more API calls to a location-based device to request data from the location-based device, or to request an interactive communication session with a location-based device.

[0045] continue Figure 1 In some examples, the monitoring service 128 is configured to control the overall logical setup and operation of the system 100. Thus, the monitoring service 128 can interoperate with any of the transport service 126, the monitor interface 130, the customer interface 132, and the location-based devices. In some examples, the monitoring service 128 is configured to monitor data from various sources of reportable events (e.g., a break-in event), and when a reportable event is detected, notify one or more of the monitor interface 130 and / or the customer interface 132 of the reportable event. In some examples, the monitoring service 128 is also configured to maintain status information about the location 102A. The status information may indicate, for example, whether the location 102A is secure or under threat. In some examples, the monitoring service 128 is configured to change the status information to indicate that the location 102A is secure only when a communication indicating a clear event is received (e.g., rather than making such a change in response to an interruption in the reception of a break-in event). This feature can prevent a "crash and smash" robbery from being successfully executed. Referring below to Figure 5 and Figure 6 Additional example processes are described that monitoring service 128 is configured to perform.

[0046] continue Figure 1 In some examples, each monitor interface 130 is configured to control the interaction of a computing device with a monitoring person, and to perform various programmed operations in response to the interaction. For example, in some examples, the monitor interface 130 controls its host device to provide information to the monitoring person about a reportable event detected at a monitoring location (such as, location 102A). Such events may include, for example, movement or an alarm condition generated by one or more of the location-based devices. Alternatively or additionally, in some examples, the monitor interface 130 controls its host device to interact with a user to configure features of the system 100. Referring below to Figure 6 Additional example processes that monitor interface 130 is configured to perform are described.

[0047] continue Figure 1 In some examples, each customer interface 132 is configured to control the interaction of the computing device with the customer and to perform various programmed operations in response to the interaction. For example, in some examples, the customer interface 132 controls its host device to provide information to the customer about reportable events detected at a monitored location (such as, location 102A). Such events may include, for example, an alarm condition generated by one or more of the location-based devices. Alternatively or additionally, in some examples, the customer interface 132 is configured to process input received from the customer to activate or deactivate one or more of the location-based devices. Further, in some examples, the customer interface 132 configures features of the system 100 in response to input from the user. Referring below to Figure 6Additional example processes that client interface 132 is configured to perform are described.

[0048] Now turn to Figure 2 , schematically illustrates an example base station 114. Figure 2 As shown in FIG. 1 , the base station 114 includes at least one processor 200, volatile memory 202, non-volatile memory 206, at least one network interface 204, a user interface 212, a battery component 214, and an interconnect mechanism 216. The non-volatile memory 206 stores executable code 208 and includes data storage 210. Figure 2 In some of the examples shown, the features of the base station 114 listed above are incorporated into or are part of the housing 218 .

[0049] In some examples, non-volatile (non-transitory) memory 206 includes: one or more read-only memory (ROM) chips; one or more hard disk drives or other magnetic or optical storage media; one or more solid-state drives (SSDs) such as flash drives or other solid-state storage media; and / or one or more hybrid magnetic SSDs. In some examples, code 208 stored in non-volatile memory may include an operating system and one or more applications or programs configured to execute under the operating system. Alternatively or additionally, code 208 may include dedicated firmware and embedded software that can execute without relying on a commercial operating system. In any case, execution of code 208 may be implemented Figure 1 of the monitoring client 136 and may generate manipulation data as part of the data store 210 .

[0050] continue Figure 2In some examples, the processor 200 may include one or more programmable processors, which are used to execute one or more executable instructions such as a computer program specified by code 208 to control the operation of base station 114. As used herein, the term "processor" describes a circuit system that performs a function, an operation, or a series of operations. Functions, operations, or a series of operations can be hard-coded into the circuit system, or soft-coded by instructions stored in a memory device (e.g., volatile memory 202) and executed by the circuit system. In some examples, the processor 200 is a digital processor, but the processor 200 can be analog, digital, or mixed. Therefore, the processor 200 can use digital values ​​and / or use analog signals to perform functions, operations, or a series of operations. In some examples, the processor 200 can be implemented in one or more application-specific integrated circuits (ASICs), microprocessors, digital signal processors (DSPs), graphics processing units (GPUs), neural processing units (NPUs), microcontrollers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), or multi-core processors. The example of processor 200 as a multi-core may provide a function of executing instructions in parallel and at the same time, or a function of executing one instruction in parallel and at the same time on more than one piece of data.

[0051] continue Figure 2 In an example, prior to execution of code 208, processor 200 may copy code 208 from non-volatile memory 206 to volatile memory 202. In some examples, volatile memory 202 includes one or more static or dynamic random access memory (RAM) chips and / or cache memory (e.g., memory disposed on the silicon die of processor 200). Volatile memory 202 may provide faster response times than main memory (such as non-volatile memory 206).

[0052] Through the execution of code 208, processor 200 can control the operation of network interface 204. For example, in some examples, network interface 204 includes one or more physical interfaces (e.g., radio, Ethernet port, universal serial bus (USB) port, etc.) and a software stack including drivers and / or other code 208, which is configured to communicate with the one or more physical interfaces to support one or more LAN, PAN and / or WAN standard communication protocols. The communication protocol may include, for example, transmission control protocol (TCP), user datagram protocol (UDP), HTTP, and MQTT, etc. Thus, network interface 204 enables base station 114 to communicate via a computer network (e.g., by Figure 1 The LAN established by router 116, Figure 1The network interface 204 may utilize sub-GHz wireless networking to transmit a wake-up message to other computing devices to request sensor data streams or other operations. Using sub-GHz wireless networking may improve the operable communication distance and / or reduce communication power consumption.

[0053] By executing the code 208, the processor 200 can control the operation of the user interface 212. For example, in some examples, the user interface 212 includes a user input device and / or an output device (e.g., a keyboard, a mouse, a touch screen, a display, a speaker, a camera, an accelerometer, a biometric scanner, an environmental sensor, etc.) and a software stack including a driver and / or other code 208, which is configured to communicate with the user input device and / or the output device. For example, the user interface 212 can be implemented by a client device 122 hosting a mobile application (e.g., a client interface 132). The user interface 212 enables the base station 114 to interact with the user to receive input and / or present output. The presented output may include, for example, one or more graphical user interfaces (GUIs), which include one or more controls configured to display output and / or receive input. The input may specify a value to be stored in the data store 210. The output may indicate a value stored in the data store 210. It should be noted that in some examples, portions of the user interface 212 are accessible and / or visible as part of or through the housing 218. These portions of the user interface 212 may include, for example, one or more light emitting diodes (LEDs). Alternatively or additionally, in some examples, the user interface 212 includes a 95 dB siren that the processor 200 causes to sound to indicate that an intrusion event has been detected.

[0054] continue Figure 2In some examples, the various features of the base station 114 described above can communicate with each other via the interconnect mechanism 216. In some examples, the interconnect mechanism 216 includes a communication bus. In addition, in some examples, the battery assembly 214 is configured to supply operating power to the various features of the base station 114 described above. In some examples, the battery assembly 214 includes at least one rechargeable battery (e.g., one or more NiMH or lithium batteries). In some examples, the rechargeable battery has a running time capacity sufficient to operate the base station 114 for 24 hours or more when the base station 114 is disconnected from the line power or otherwise does not receive line power. Alternatively or additionally, in some examples, the battery assembly 214 includes a power supply circuit system for receiving, controlling and distributing line power to operate the base station 114 and recharging the rechargeable battery. The power supply circuit system may include, for example, a transformer and rectifier that converts AC line power to a DC device and recharges the power, as well as other circuit systems.

[0055] Now turn to Figure 3 , schematically shows an example keypad 108. Figure 3 As shown in FIG. 1 , the keypad 108 includes at least one processor 300, volatile memory 302, non-volatile memory 306, at least one network interface 304, a user interface 312, a battery assembly 314, and an interconnect mechanism 316. The non-volatile memory 306 stores executable code 308 and data storage 310. Figure 3 In some of the examples shown, the features of the keypad 108 listed above are incorporated into or are part of the housing 318 .

[0056] In some examples, corresponding descriptions of the processor 200, volatile memory 202, non-volatile memory 206, interconnect mechanism 216, and battery assembly 214 made with respect to the base station 114 apply to the processor 300, volatile memory 302, non-volatile memory 306, interconnect mechanism 316, and battery assembly 314 with respect to the keypad 108. Therefore, those descriptions will not be repeated.

[0057] continue Figure 3In some examples, through the execution of code 308, the processor 300 can control the operation of the network interface 304. In some examples, the network interface 304 includes one or more physical interfaces (e.g., radio, Ethernet port, USB port, etc.) and a software stack including drivers and / or other code 308, which is configured to communicate with the one or more physical interfaces to support one or more LAN, PAN and / or WAN standard communication protocols. These communication protocols may include, for example, TCP, UDP, HTTP, and MQTT, etc. Therefore, the network interface 304 enables the keypad 108 to access other computing devices (e.g., other location-based devices) and communicate with them via a computer network (e.g., a LAN and / or a point-to-point connection established by the router 116).

[0058] continue Figure 3 In some examples, the processor 300 can control the operation of the user interface 312 through the execution of the code 308. In some examples, the user interface 312 includes a user input device and / or an output device (e.g., a physical key arranged as a keypad, a touch screen, a display, a speaker, a camera, a biometric scanner, an environmental sensor, etc.) and a software stack including a driver and / or other code 308, which is configured to communicate with the user input device and / or the output device. Therefore, the user interface 312 enables the keypad 108 to interact with the user to receive input and / or present output. The presented output may include, for example, one or more GUIs, which include one or more controls configured to display output and / or receive input. The input may specify a value to be stored in the data store 310. The output may indicate a value stored in the data store 310. It should be noted that in some examples, a portion of the user interface 312 (e.g., one or more LEDs) is accessible and / or visible as part of the housing 318 or through the housing 318.

[0059] Now turn to Figure 4A , schematically illustrates an example security sensor 422. The specific configuration of the security sensor 422 (e.g., the image capture devices 104 and 110, the motion sensor assembly 112, and the contact sensor assembly 106) is shown in FIG. Figure 1 As shown in and described above. Figure 4A As shown in FIG. 4 , the security sensor 422 includes at least one processor 400, volatile memory 402, non-volatile memory 406, at least one network interface 404, a battery component 414, an interconnect mechanism 416, and at least one sensor component 420. The non-volatile memory 406 stores executable code 408 and data storage 410. Some examples include a user interface 412. Figure 4AIn some of the examples shown, the features of the safety sensor 422 listed above are incorporated into or are part of the housing 418 .

[0060] In some examples, corresponding descriptions of the processor 200, volatile memory 202, non-volatile memory 206, interconnect mechanism 216, and battery assembly 214 made with respect to the base station 114 apply to the processor 400, volatile memory 402, non-volatile memory 406, interconnect mechanism 416, and battery assembly 414 with respect to the security sensor 422. Therefore, those descriptions will not be repeated.

[0061] continue Figure 4A In some examples, by executing code 408, processor 400 can control the operation of network interface 404. In some examples, network interface 404 includes one or more physical interfaces (e.g., a radio (including an antenna), an Ethernet port, a USB port, etc.) and a software stack including a driver and / or other code 408, which is configured to communicate with one or more physical interfaces to support one or more LAN, PAN and / or WAN standard communication protocols. Communication protocols may include, for example, TCP, UDP, HTTP, and MQTT, etc. Therefore, network interface 404 enables security sensor 422 to access other computing devices (e.g., other location-based devices) and communicate with them via a computer network (e.g., a LAN and / or a point-to-point connection established by router 116). For example, in at least one example, when executing code 408, processor 400 controls the network interface to stream sensor data obtained from sensor component 420 to base station 114 (e.g., via UDP). Alternatively or additionally, in at least one example, through the execution of code 408, the processor 400 can control the network interface 404 to enter a power saving mode by powering off the 2.4 GHz radio and powering on the sub-GHz radio, both of which are included in the network interface 404. In this example, through the execution of code 408, the processor 400 can control the network interface 404 to enter a streaming mode or an interactive mode by powering on the 2.4 GHz radio and powering off the sub-GHz radio, for example, in response to receiving a wake-up signal from a base station via the sub-GHz radio.

[0062] continue Figure 4AIn some examples, the processor 400 can control the operation of the user interface 412 through the execution of the code 408. In some examples, the user interface 412 includes a user input device and / or an output device (e.g., a physical button, a touch screen, a display, a speaker, a camera, an accelerometer, a biometric scanner, an environmental sensor, one or more LEDs, etc.) and a software stack including a driver and / or other code 408, which is configured to communicate with the user input device and / or the output device. Therefore, the user interface 412 enables the security sensor 422 to interact with the user to receive input and / or present output. The output presented may include, for example, one or more GUIs, which include one or more controls configured to display output and / or receive input. The input may specify a value to be stored in the data store 410. The output may indicate a value stored in the data store 410. It should be noted that in some examples, a portion of the user interface 412 is accessible and / or visible as a part of the housing 418 or through the housing 418.

[0063] continue Figure 4A For example, the sensor assembly 420 may include one or more types of sensors, such as those described above with reference to Figure 1 The sensors described in the image capture devices 104 and 110, the motion sensor assembly 112, and the contact sensor assembly 106 or other types of sensors. For example, in at least one example, the sensor assembly 420 includes an image sensor (e.g., a charge coupled device or an active pixel sensor) and / or a temperature sensor or a thermal imaging sensor (e.g., an active and / or passive infrared (PIR) sensor). Regardless of the type of one or more sensors accommodated, the processor 400 can (e.g., via the execution of the code 408) acquire sensor data from the accommodated sensor and stream the acquired sensor data to the processor 400 for communication with the base station.

[0064] It should be noted that in some examples of devices 108 and 422, operations performed by processors 300 and 400 when under the control of corresponding controls of code 308 and 408 may be hard-coded and / or implemented in hardware, rather than as a combination of hardware and software. In addition, execution of code 408 may implement Figure 1 The camera agent 138 may generate manipulation data as part of the data store 410 .

[0065] Now turn to Figure 4B , schematically illustrates an example image capture device 500. The specific configuration of the image capture device 500 (eg, image capture devices 104 and 110) is described in detail in Figure 1 As shown in and described above. Figure 4B, image capture device 500 includes at least one processor 400, volatile memory 402, non-volatile memory 406, at least one network interface 404, a battery assembly 414, and an interconnect mechanism 416. These features of image capture device 500 are shown in dashed lines to indicate that these features of image capture device 500 reside within a housing 418. Non-volatile memory 406 stores executable code 408 and data storage 410.

[0066] Some examples also include an image sensor assembly 450, a light source 452, a speaker 454, a microphone 456, a wall mount 458, and a magnet 460. The image sensor assembly 450 may include a lens and an image sensor (e.g., a charge coupled device or an active pixel sensor) and / or a temperature sensor or a thermal imaging sensor (e.g., an active and / or passive infrared (PIR) sensor). The light source 452 may include a light emitting diode (LED) such as a red, green, and blue emitting LED. In some examples, the light source 452 may also include an infrared light emitting diode. The speaker 454 may include a transducer configured to emit a sound in the range of 60 dB to 80 dB or more. In addition, in some examples, the speaker 454 may include an alarm configured to emit a sound in the range of 70 dB to 90 dB or more. The microphone 456 may include a microelectromechanical system (MEMS) microphone. The wall mount 458 may include a mounting bracket configured to receive screws or other fasteners for attaching the bracket to the wall, and a cover configured to be mechanically coupled to the mounting bracket. In some examples, the cover includes a magnetic material such as aluminum or stainless steel to enable the magnet 460 to magnetically couple to the wall mount 458, thereby holding the image capture device 500 in place.

[0067] In some examples, the corresponding descriptions of the processor 400, volatile memory 402, network interface 404, non-volatile memory 406, code 408 for network interface 404, interconnect mechanism 416, and battery assembly 414 made with respect to the security sensor 422 apply to these same features with respect to the image capture device 500. Therefore, those descriptions will not be repeated here.

[0068] continue Figure 4B In the example of, through the execution of code 408, processor 400 can control the operation of image sensor component 450, light source 452, speaker 454 and microphone 456. For example, in at least one example, when executing code 408, processor 400 controls image sensor component 450 to obtain sensor data in the form of image data to be streamed to base station 114 (or Figure 1Alternatively or additionally, in at least one example, through the execution of code 408, processor 400 controls light source 452 to emit light so that image sensor assembly 450 collects sufficient reflected light to form image data. In addition, in some examples, through the execution of code 408, processor 400 controls speaker 454 to emit a sound. The sound can be generated locally (e.g., via an audible alarm of a siren), or received from base station 114 (or a user or monitoring personnel) via network interface 404 (e.g., speech from a user or monitoring personnel). Figure 1 Furthermore, in some examples, through the execution of code 408, processor 400 controls microphone 456 to obtain sensor data in the form of sound to be streamed to base station 114 (or Figure 1 one of processes 130, 128 or 132).

[0069] It should be understood that in Figure 4B In the example of FIG. 4 , light source 452, speaker 454 and microphone 456 implement Figure 4A It should also be understood that the image sensor assembly 450 and the light source 452 implement Figure 4A An example of the sensor assembly 420. Therefore, Figure 4B The image capture device 500 shown in FIG. Figure 4A 4. For example, the image capture device 500 may be a battery-powered outdoor sensor that is configured to be installed and operate in an outdoor environment, such as outside a home, office, store, or other commercial or residential building.

[0070] Now turn to Figure 4C , schematically illustrates another example image capture device 520. The specific configuration of the image capture device 520 (eg, image capture devices 104 and 110) is described in detail in Figure 1 As shown in and described above. Figure 4C As shown in , the image capture device 520 includes at least one processor 400, volatile memory 402, non-volatile memory 406, at least one network interface 404, a battery assembly 414, and an interconnect mechanism 416. These features of the image capture device 520 are shown in dashed lines to indicate that these features of the image capture device 520 reside within a housing 418. The non-volatile memory 406 stores executable code 408 and data storage 410. As described above with reference to Figure 4BAs described above with respect to image capture device 500 , image capture device 520 also includes image sensor assembly 450 , speaker 454 , and microphone 456 .

[0071] In some examples, image capture device 520 also includes light sources 452A and 452B. Light source 452A may include a light emitting diode (LED) such as a red, green, and blue emitting LED. In some examples, light source 452B may also include an infrared light emitting diode to achieve night vision.

[0072] It should be understood that in Figure 4C In the example of FIG. 4 , light sources 452A and 452B, speaker 454, and microphone 456 implement Figure 4A It should also be understood that the image sensor assembly 450 and the light source 452 implement Figure 4A An example of the sensor assembly 420. Therefore, Figure 4C The image capture device 520 shown in FIG. Figure 4A 4. For example, the image capture device 520 may be a battery-powered indoor sensor that is configured to be installed and operate in an indoor environment, such as within a home, office, store, or other commercial or residential building.

[0073] Now turn to Figure 5 , schematically showing Figure 1 Data center environment 124, Figure 1 Monitoring center environment 120, Figure 1 one of the client devices 122, Figure 1 Network 118 and Figure 1 Aspects of a plurality of monitoring locations 102A to 102N (collectively referred to as locations 102). Figure 5 As shown in , the data center environment 124 hosts a monitoring service 128 and a transport service 126 (referred to as transport services 126A through 126D, respectively). The monitoring service 128 includes a location data store 502, a sensor data store 504, an artificial intelligence (AI) service 508, an event listening service 510, and an identity provider 512. The monitoring center environment 120 includes computing devices 518A through 518M (collectively referred to as computing devices 518) that host monitor interfaces 130A through 130M. Each location 102A through 102N includes a base station (e.g., Figure 1 138) and an image capture device (e.g., a base station 114, not shown) hosting software camera agents 138A to 138N (collectively referred to as camera agents 138). Figure 1 image capture device 110, not shown).

[0074] like Figure 5 As shown in , the transport service 126 is configured to process ingress messages 516B from the customer interface 132A, the monitoring client 136, the camera agent 138, and / or the monitor interface 130. The transport service 126 is also configured to process egress messages 516A sent to the customer interface 132A, the monitoring client 136, the camera agent 138, and the monitor interface 130. The location data store 502 is configured to store location data associated with identifiers of customers whose locations are monitored within multiple records. For example, the location data may be stored in a record with an identifier of the customer and / or an identifier of the location so that the location data is associated with the customer and the location. The sensor data store 504 is configured to store sensor data (e.g., one or more frames of image data) associated with identifiers of locations and timestamps at which the sensor data was acquired within multiple records.

[0075] continue Figure 5 In some examples, the AI ​​service 508 is configured to process sensor data (e.g., images and / or image sequences) to identify movement, faces, and other features within the sensor data. The event monitoring service 510 is configured to scan the location data transmitted via the inlet message 516B for the event data, and in the case of identifying the event data, execute one or more event processing routines to process the event data. In some examples, the event processing routine may include an event reporter that is configured to identify reportable events and transmit messages specifying reportable events to one or more recipient processes (e.g., client interface 132 and / or monitor interface 130). In some examples, the event monitoring service 510 can interoperate with the AI ​​service 508 to identify events from sensor data. The identity provider 512 is configured to receive an authentication request including security credentials from the monitoring client 136 or the camera agent 138 via the transmission service 126. When the identity provider 512 can authenticate the security credentials in the request (e.g., via a verification function, a cross-reference lookup, or some other authentication process), the identity provider 512 can transmit a security token in response to the request. The monitoring client 136 or the camera agent 138 can receive, store, and include the security token in a subsequent ingress message 516B, so that the transport service 126A can securely process (e.g., unpack / parse) the packet included in the ingress message 516B to extract the location data before passing the location data to the monitoring service 128.

[0076] continue Figure 5In the example of FIG. 5 , the transport service 126 is configured to receive the inbound message 516B, verify the authenticity of the message 516B, parse the message 516B, and extract the location data encoded in the message 516B before passing the location data to the monitoring service 128 for processing. The location data may include the location data described above with reference to Figure 1 Any of the location data described. Each transport service 126 may be configured to process ingress messages 516B generated by location-based monitoring devices of a particular manufacturer and / or model. The monitoring client 136 and the camera agent 138 are configured to generate an ingress message 516B including a packet of location data based on sensor information received at the location 102 and transmit it to the monitoring service 128 via the network 118.

[0077] continue Figure 5 In some examples, computing device 518 is configured to host monitor interface 130. In some examples, each monitor interface 130A to 130M is configured to present a GUI including one or more image frames and / or other sensor data. In some examples, client device 122 is configured to host client interface 132. In some examples, client interface 132 is configured to present a GUI including one or more image frames and / or other sensor data. Figure 6 Additional features of the monitor interface 130 and the customer interface 132 are further described.

[0078] Now turn to Figure 6 , the monitoring process 600 is shown as a sequence diagram. In some examples, the process 600 can be performed by a security system (e.g., Figure 1 More specifically, in some examples, at least a portion of process 600 is performed by a location-based device under control of device control system (DCS) code (e.g., code 308 or 408) that is executed by at least one processor (e.g., Figures 3 to 4C The DCS code may include, for example, a camera agent (e.g., Figure 1 At least a portion of process 600 is performed by a base station (e.g., Figure 1 The base station 114) monitors the client (e.g., Figure 1 At least a portion of process 600 is performed by a monitoring center environment (e.g., Figure 1 The monitoring center environment 120) at the monitor interface (e.g., Figure 1 At least a portion of process 600 is performed by a data center environment (e.g., Figure 1 data center environment 124) in monitoring services (e.g., Figure 1 ) or under the control of a transmission service (e.g., Figure 1 At least a portion of process 600 is performed by a client device (e.g., Figure 1 The client device 122) at the client interface (e.g., Figure 1 The system is executed under the control of the client interface 132).

[0079] like Figure 6 As shown in FIG. 6 , process 600 is performed by monitoring client 136 to authenticate an identity provider (e.g., Figure 5 The process begins with the monitoring client 136 authenticating with the identity provider 512 of the authentication request. More specifically, in some examples, the monitoring client 136 transmits an authentication request to the transport service 126 via one or more API calls to the transport service 126. In these examples, the transport service 126 parses the authentication request to extract security credentials therefrom, and passes the security credentials to the identity provider for authentication. In some examples, if the identity provider authenticates the security credentials, the transport service 126 receives a security token from the identity provider and transmits the security token as a payload within an authentication response to the authentication request. In these examples, if the identity provider cannot authenticate the security credentials, the transport service 126 generates an error code and transmits the error code as a payload within the authentication response to the authentication request. After receiving the authentication response, the monitoring client 136 parses the authentication response to extract the payload. If the payload includes an error code, the monitoring client 136 may retry the authentication and / or communicate with a user interface of its host device (e.g., Figure 2 The monitoring client 136 may interoperate with the user interface 212 of the base station 114 to present an output indicating the authentication failure. If the payload includes a security token, the monitoring client 136 stores the security token for subsequent use in the communication of location data via the ingress message. It should be noted that the security token may have a limited validity period (e.g., 1 hour, 1 day, 1 week, 1 month, etc.), after which the monitoring client 136 may need to re-authenticate with the transmission service 126.

[0080] Continuing with process 600, one or more DCSs 602 hosted by one or more location-based devices obtain 606 a data stream describing a location (e.g., Figure 1 The acquired sensor data may be of any of various types, as described above with reference to Figure 14. In some examples, one or more of the DCS 602 continuously acquires sensor data. In some examples, one or more of the DCS 602 acquires sensor data in response to an event such as expiration of a local timer (push event) or receiving an acquisition polling signal (polling event) transmitted by the monitoring client 136. In some examples, one or more of the DCS 602 streams the sensor data to the monitoring client 136 with minimal processing beyond acquisition and digitization. In these examples, the sensor data may constitute a vector sequence in which each vector member includes a sensor reading and a timestamp. Alternatively or additionally, in some examples, one or more of the DCS 602 performs additional processing of the sensor data, such as generating one or more summaries of multiple sensor readings. Further, in some examples, one or more of the DCS 602 performs complex processing of the sensor data. For example, if the security sensor includes an image capture device, the security sensor may perform image processing routines such as edge detection, motion detection, facial recognition, threat assessment, and reportable event generation.

[0081] Continuing with process 600, DCS 602 transmits sensor data 608 to monitoring client 136. As with sensor data acquisition, DCS 602 may transmit sensor data 608 continuously or in response to an event such as a push event (originating from DCS 602) or a polling event (originating from monitoring client 136).

[0082] Continuing with process 600, the monitoring client 136 monitors 610 the location by processing the received sensor data 608. For example, in some examples, the monitoring client 136 performs one or more image processing routines. These image processing routines may include any of the image processing routines described above with reference to operation 606. By distributing at least some of the image processing routines between the DCS 602 and the monitoring client 136, some examples reduce the power consumed by the battery-powered device by offloading the line-powered device. In addition, in some examples, the monitoring client 136 may perform an integrated threat detection process that utilizes sensor data 608 from multiple, different DCSs 602 as input. For example, in at least one example, the monitoring client 136 will attempt to confirm the open state received from the contact sensor through motion and facial recognition processing of a scene image including a window to which the contact sensor is fixed. If two or more of the three processes indicate the presence of an intruder, the threat score is increased, and or the intrusion event is declared, recorded locally and transmitted. Other processing that may be performed by the monitoring client 136 includes outputting local alerts (e.g., in response to detection of certain events and / or satisfaction of other criteria) and detection of maintenance conditions for location-based equipment, such as the need to replace or recharge low batteries and / or replace / maintain equipment hosting the DCS 602. Any of the above-described processes within operation 610 may result in the creation of location data specifying the process results.

[0083] Continuing with process 600, the monitoring client 136 transmits location data 614 to the monitoring service 128 via one or more inlet messages 612 to the transport service 126. As with the sensor data 608 communications, the monitoring client 136 may transmit location data 614 continuously or in response to an event such as a push event (originating from the monitoring client 136) or a polling event (originating from the monitoring service 128).

[0084] Continuing with process 600, the monitoring service 128 processes 616 the received location data. For example, in some examples, the monitoring service 128 performs one or more routines described above with reference to operations 606 and / or 610. Additionally or alternatively, in some examples, the monitoring service 128 uses historical information associated with the location identified in the location data and / or other locations that are geographically proximate to the location (e.g., within the same Zone Improvement Program (ZIP) code) to calculate a threat score or further refine an existing threat score. For example, in some examples, the monitoring service 128 may increase the threat score calculated by the DCS 602 and / or the monitoring client 136 if multiple break-ins have been recorded for the location within the same ZIP code and / or other locations within a configurable time span that includes the current time. In some examples, monitoring service 128 determines whether location data 614 includes any reportable event data by applying a set of rules and criteria to location data 614, and if so, transmits event reports 618A and / or 618B to monitor interface 130 and / or customer interface 132. Reportable events may be a certain type of event (e.g., a break-in) or a certain type of event that meets additional criteria (e.g., movement within a particular area combined with a threat score exceeding a threshold). Event reports 618A and / or 618B may have a priority based on the same criteria used to determine whether an event reported therein is reportable, or may have a priority based on a different set of criteria or rules.

[0085] Continuing with process 600, monitor interface 130 interacts with monitoring personnel through, for example, one or more GUIs 620. These GUIs may provide details and context regarding one or more reportable events.

[0086] Continuing with process 600, customer interface 132 interacts with at least one customer via, for example, one or more GUIs 622. These GUIs may provide details and context regarding one or more reportable events.

[0087] It should be noted that the processing of sensor data and / or location data as described above with reference to operations 606, 610, and 616 can be performed by processors disposed within various portions of the system 100. For example, in some examples, the DCS 602 performs minimal processing of the sensor data (e.g., only acquiring and streaming), and the remainder of the processing described above is performed by the monitoring client 136 and / or the monitoring service 128. This approach may help extend the battery runtime of location-based devices. In other examples, the DCS 602 performs as much sensor data processing as possible so that the monitoring client 136 and the monitoring service 128 perform only the processes that require sensor data across location-based devices and / or locations. This approach may help increase the scalability of the system 100 in terms of adding new locations.

[0088] Now turn to Figure 7 , shows an example of a security device 702 configured to implement the various techniques disclosed herein. As discussed above, the security device 702 can be associated with a security system installed at a monitoring location. The security device 702 includes a motion sensor 704, an image sensor 706, and a memory 710, which are electrically coupled to a controller 700. For example, the controller 700 may include one or more processors such as the processor 400 discussed above or may be implemented by one or more processors such as the processor 400 discussed above. The security device 702 may also include the above reference FIG. 4A to FIG. 4C Any of the components and functions of the security sensor 422 and / or image capture devices 500 and 520 discussed above. Therefore, it should be understood that the security device 702 may include Figure 7 Parts not shown.

[0089] In one example, the motion sensor 704 is a PIR motion sensor. In one example, the image sensor 706 is a digital camera that collects still image frames and / or video image frames that make up the video feed / video stream. The image sensor 706 may include the above-referenced Figure 4B and Figure 4C The controller 700 includes an NPU 708 for efficiently applying a neural network or other artificial intelligence model to perform aspects of motion detection and object detection processes based on image frames captured by the image sensor 706, as discussed in more detail below. In one example, the memory 710 includes non-volatile flash memory and stores code 712 that can be executed by the controller 700. The memory 710 may include the memory described above with reference to FIG. FIG. 4A to FIG. 4CThe non-volatile memory 406 discussed. The code 712 includes, among other software programs, an ROI generator 714, an ROI merging engine 716, and an object detector 718. In some examples, the object detector 718 applies an artificial intelligence (AI) or machine learning (ML) model to a set of image data and generates output data that identifies objects within the image data and specifies a confidence level that the identified objects are correctly identified. Examples of AI models that can be applied by the object detector 718 include any from the YOLO model family, which can be obtained as part of the OpenAl package from the GitHub code repository. The process by which the code 712 is configured to be executed by components of the security device 702 is also described below. The code 712 may include the above referenced FIG. 4A to FIG. 4C The code discussed 408. For example, the security device 702 can be a battery-powered indoor sensor that is configured to be installed and operate in an indoor environment such as within a home, office, store, or other commercial or residential building.

[0090] According to certain examples, the controller 700 and the motion sensor 704 operate in a low power state or mode of operation, wherein the image sensor 706 (and optionally, other components of the security device 702) are deactivated until an event triggers the motion sensor 704. In the low power mode of operation, the motion sensor 704 remains active, but components that typically consume more power, such as the image sensor 706, are powered off. In the low power mode of operation, the controller 700 performs minimal processing sufficient to monitor events that trigger the motion sensor 704 to preserve battery life and thus extend the time between battery refresh services. When the motion sensor 704 indicates motion and issues a signal or notification (e.g., sends motion trigger data to the controller 700), the controller 700 is placed in a normal operating mode in which the image sensor 706 (and other components of the security device 702 that are powered off in the low power state) are enabled. Thus, the motion sensor 704 acts as a mode "switch" that configures the security device 702 to a "full power" or normal operating mode only when necessary. In this manner, power may be conserved by operating the security device 702 in a low power mode with various components powered off until a potential event of interest is detected, and thereby extending the operational lifecycle of the device when using battery power.

[0091] Reference Figure 8 , showing the Figure 7702. In the case where the security device 702 is operating in a low power mode, the motion sensor 704 is active or otherwise powered. At 802, the motion sensor 704 detects an event (e.g., an object in motion) and sends a signal (e.g., a motion trigger signal) to the controller 700 as indicated at 804. As discussed above, in some examples, the motion sensor 704 is a PIR sensor that detects motion based on changes in temperature detected within its field of view. Therefore, in some examples, the motion sensor 704 can be adjusted to detect humans and / or animals based on a known temperature range associated with the body temperature of humans / animals. The signal from the motion sensor 704 causes the code 712 being executed by the controller 700 to configure the security device 702 in a normal operating mode, which includes activating or enabling the image sensor 706, as indicated at 806. Once activated, at 808, the image sensor 706 captures one or more frames of image data. In some examples, the image sensor 706 passes frames of image data ("images" or "image frames") to the controller 700 for processing, as indicated at 810. These frames of image data may be implemented as two-dimensional arrays of pixel data ("pixels"). Other data structures and encoding standards for frames will become apparent in light of this disclosure.

[0092] According to some examples, the controller 700 applies a motion detection process 812 operating on captured frames of image data (e.g., from the image sensor 706) to detect instances of motion. Figure 8 In the example shown in , process 812 is implemented by controller 700. In the example, controller 700 reads a single frame of image data at a time. In some examples, controller 700 converts the frame to grayscale and changes the size of the frame for motion detection. In one example, the size of the grayscale frame is changed (e.g., reduced) to 320×192 pixels for motion detection; however, in other examples, other frame sizes (e.g., 288×288 pixels, 320×192 pixels) can be used. In some examples, reducing the image frame can allow the image to be processed faster (e.g., using an artificial neural network (ANN)) and / or processed with lower power consumption (which may be particularly beneficial for battery-powered sensors) compared to when process 812 is applied to a full-size image captured by image sensor 706.

[0093] It should be noted that in some examples, process 812 is implemented by a combination of image sensor 706 and controller 700. In these examples, image sensor 706 obtains a frame of image data and performs the pre-processing described above (e.g., converting to grayscale and / or changing the size of the frame). Additional frames of image data can be acquired by image sensor 706 as process 812 continues and repeats various actions as discussed below.

[0094] Continuing with process 812, the controller 700 operates on a plurality of image data frames (e.g., consecutive frames) captured by the image sensor 706. In some examples of process 812, the controller 700 locates the position of the moving object in the field of view of the image sensor 706. The field of view of the image sensor 706 corresponds to the range of the observable world "seen" (i.e., detected) by the image sensor 706 at any given moment, which is generally a solid angle relative to the image sensor 706. In some examples, the solid angle defines the area in which the PIR sensor is sensitive to and receives electromagnetic radiation. Computer vision techniques can be used to determine the position of the object within the field of view. For example, there are existing foreground detection processes and / or background subtraction processes that can be used to locate moving objects in image data frames. These processes identify pixels that have changed between frames and generate bounding boxes if the pixels have changed. The motion detection process can be adjusted to reject noise (e.g., by using a threshold to reject small differences between consecutive image frames). Therefore, the output of the motion detection process 812 includes a bounding box describing the position of the motion detected within the scene. Examples of programming libraries that implement and disclose motion detection processes include OpenCV, which can be obtained on the GitHub code repository.

[0095] The controller 700 is also configured to implement an object detection process 814 (optionally in combination with the image sensor 706). In some examples, the motion detection process 812 and the object detection process 814 are executed simultaneously via separate processing threads. In these examples, the motion detection process 812 can operate on the image frames before the object detection process 814, and the object detection process 814 can skip some frames operated by the motion detection process, for example, to save computing resources. For example, in some examples, the object detection process 814 is configured to operate on 1 to 10 frames per second, while the motion detection process 812 is configured to operate on 25 or more frames per second. In some examples, the object detection process 814 is configured to operate on 3 to 6 frames per second. In some examples, the object detection process 814 is configured to operate on 4 or 5 frames per second. Fig. 9 A specific example of the object detection process 814 is shown in FIG.

[0096] like Fig. 9 As shown in FIG. 8 , the object detection process 814 is performed by a controller (e.g., Figure 7 Controller 700) from an image capture device (e.g., Figure 7 The image sensor 706 of the image capture device 702 receives an image data frame. For example, in some examples, the image capture device obtains an image data frame that captures its field of view via a lens and associated circuit system, and stores the frame in a buffer that can be accessed by the controller. The size and format of the frame can vary between examples. For example, in some examples, the frame consists of 1920×1080 pixels encoded in a red, green, and blue (RGB) color format. In other examples, the frame includes 1536×1536 pixels. In one example, the RGB frame is resized (e.g., reduced or enlarged) to 512×288 pixels for object detection; however, in other examples, other frame sizes (e.g., 416×416 pixels, 150×150 pixels, etc.) may be used. Fig. 12A An example image data frame 1200 is shown. Fig. 12A As shown in , frame 1200 has two dimensions—height 1202 and width 1206 .

[0097] Continuing with process 814, the controller defines 904 a ROI within the image data frame. For example, in some examples, the controller executes ROI generator code (e.g., Figure 7 ROI generator 714) to define the ROI. Fig.10 An example of ROI generation process 904 is shown in FIG. Fig.10 As shown in , process 904 is iterative and may loop through two or more target regions before terminating.

[0098] Process 904 begins with the controller selecting 1002 a region within a frame of image data as a target for an ROI. For example, in some examples, the controller selects 1002 a region within a frame of image data as a target for an ROI by a motion detection process (e.g., as described above with reference to Figure 8 The controller selects a target region to be used as the ROI basis by a bounding box defined within the frame by the object detection process (e.g., described above with reference to the motion detection process 812) that has not been processed by the current iteration of process 904. Alternatively or additionally, in some examples, the controller selects a target region to be used as the ROI basis by a bounding box defined within the frame by the object detection process (e.g., described above with reference to the motion detection process 812). Figure 8 The bounding box identified by the previous iteration of the described object detection process 814) is used as the target area. As an illustration, Fig. 12A The frame 1200 has three bounding boxes defined by the motion detection process, rectangles 1208, 1210, and 1212. The controller may select any one of these bounding boxes as the next target area in operation 1002.

[0099] Continuing with process 904, the controller sets 1004 the initial boundaries of the ROI. For example, in some examples, the controller sets the initial boundaries of the ROI with reference to the target area. In some examples, the controller selects an initial ROI boundary that places the target area at the center within the ROI. Additionally or alternatively, in some examples, the controller selects an initial boundary of the ROI that includes pixels that reside outside the target area. As discovered by the applicant through experimental testing, these additional pixels can increase the accuracy of identifying objects within the target area. For example, in at least one example, the controller identifies the larger of the height and width of the target area, multiplies the larger dimension by a factor of two, and uses the product for the height and width of the ROI. Therefore, in this example, the ROI has a square shape and is sized to include a number of pixels equal to the square of the larger dimension of the target area. This approach produces an ROI that is approximately twice the size of the target area. As an illustration, Fig. 12B Frame 1200 has three square ROIs 1220 , 1222 , and 1224 .

[0100] It should be noted that the examples described herein are not limited to ROIs having the shapes and sizes described above. For example, in some examples, the ROI can be shaped as a rectangle, a circle, or other shape, and can be sized to include various numbers of pixels. In these examples, the shape and size of the ROI can vary to better conform to the model input. Additionally or alternatively, in other examples, the controller sets the initial dimensions of the ROI so that the target area is not centered within the ROI. Such operations can allow the controller to skip separate translation operations (e.g., operation 1016 described below). It should be noted that in some examples, the initial boundaries of the ROI can extend beyond the image data frame, for example, where the target area is positioned close to the edge of the frame or is of sufficient size to generate a large ROI. As can be seen from the example of FIG. Fig. 12B As seen in one illustration, ROIs 1220 , 1222 , and 1224 have boundaries that reside both inside and outside of frame 1200 .

[0101] Continuing with process 904, the controller determines 1006 whether the dimensions of the ROI are smaller than the dimensions of the model trained to detect objects within the image. For example, in some examples, an object detector (e.g., Figure 7The object detector 718 of the ANN is configured to receive a two-dimensional array of RBG pixels having a height and a width (e.g., a predefined height and width) as an input to the ANN. In these examples, the controller compares the height and width of the ROI with the height and width of the model in operation 1006. If the controller determines that the height or width of the ROI is less than the height or width of the model, the controller performs operation 1008. If the controller determines that the height and width of the ROI are not less than the height and width of the model, the controller performs operation 1010.

[0102] It should be noted that the height and width of the model can be equal or unequal depending on the implementation of the model. For example, in one example where the model is trained to apply to images with a square aspect ratio, the height is 416 pixels and the width is 416 pixels. In other examples, the height is 512 pixels and the width is 512 pixels. In other examples, the height is 640 pixels and the width is 360 pixels. It is apparent in light of this disclosure that other model sizes and dimensions can be used.

[0103] Continuing with process 904, the controller adjusts 1008 the dimension of the ROI that is less than the corresponding dimension of the model to be equal to the corresponding dimension. For example, in some examples, the controller determines the difference between the dimension of the ROI and the corresponding dimension of the model, and increases the dimension of the ROI by the difference. In some examples, this increase is achieved by adding half of the difference to both ends of the ROI boundary parallel to the dimension, thereby maintaining the position of the target area within the ROI as much as possible.

[0104] Continuing with process 904, the controller determines 1010 whether the dimensions of the ROI are greater than the dimensions of the image data frame. For example, in some examples, the controller compares the height and width of the ROI to the height and width of the image data frame in operation 1010. If the controller determines that the height or width of the ROI exceeds the height or width of the frame, the controller performs operation 1012. If the controller determines that the height and width of the ROI do not exceed the height and width of the frame, the controller performs operation 1014.

[0105] Continuing with process 904, the controller adjusts 1012 the dimension of the ROI that exceeds the corresponding dimension of the frame to be equal to the corresponding dimension. For example, in some examples, the controller determines the difference between the dimension of the ROI and the corresponding dimension of the frame (e.g., via subtraction) and reduces the dimension of the ROI by the difference. In some examples, this reduction is achieved by removing half of the difference from both ends of the ROI boundary that are parallel to the dimension, thereby maintaining the position of the target area within the ROI as much as possible.

[0106] Continuing with process 904, the controller determines 1014 whether any of the boundaries of the ROI are located outside the image data frame. For example, in some examples, the controller implements a pixel-based coordinate system that includes the image data frame and extends beyond the image data frame. In these examples, the controller uses the coordinate system to define the boundaries of the ROI and the boundaries of the frame. In one example, the controller compares the coordinates of the ROI boundaries with the coordinates of the frame boundaries within operation 1014. If the controller determines that any of the coordinates of the ROI boundaries reside outside the coordinates of the frame boundaries, the controller performs operation 1016. If the controller determines that none of the coordinates of the ROI boundaries reside outside the coordinates of the frame boundaries, the controller performs operation 1018. As can be referenced Fig. 12B As seen in one illustration, ROIs 1220 , 1222 , and 1224 have at least one bounding coordinate that resides outside of the bounding coordinates of frame 1200 .

[0107] Continuing with process 904, the controller translates 1016 the ROI to position its boundaries within the frame of image data. For example, in some examples, the controller identifies an endpoint of the ROI boundary that extends outside the frame, determines the difference between the endpoint and the nearest point in the frame, and moves the boundary of the ROI into the frame by an amount equal to the difference while maintaining the dimensions of the ROI. In these examples, the controller repeats the process until none of the endpoints of the ROI boundary are outside the frame. This method maintains the size and shape of the ROI while translating the ROI into the frame. As can be seen from the example of FIG. 1 , the controller translates 1016 the ROI boundary to position its boundaries within the frame. Fig. 12C As can be seen, by the execution of operation 1016 , ROIs 1220 , 1222 , and 1224 have been translated into frame 1200 .

[0108] Continuing with process 904, the controller stores 1018 the data defining the ROI as determined by the previous operation within the data structure for subsequent processing. For example, in some examples, the controller stores the coordinates of the ROI in a local data store (e.g., FIG. 4A to FIG. 4C In data 410), the data structure associates the coordinates of the ROI with the image data frame 1200.

[0109] Continuing with process 904, the controller determines 1020 whether any additional, unprocessed object regions are associated with the image data frame. For example, in some examples, the controller determines whether any bounding boxes defined by the motion detection process and associated with the image data frame 1200 have not been selected by operation 1002 by accessing a data structure that associates object regions with image data frames. If the controller determines that there are no such unprocessed object regions, process 904 can end. If the controller determines that there are at least one such unprocessed object region, the controller returns to operation 1002.

[0110] Back to Fig. 9 In process 814, if the ROI meets the merging criteria, the controller merges the ROI associated with the image data frame 906. For example, in some examples, the controller executes the ROI merging engine code (e.g., Figure 7 ROI merging engine 716) to merge ROIs. Fig.11 An example of the ROI merging process 906 is shown in FIG.

[0111] like Fig.11 As shown in , process 906 begins with the controller determining 1102 whether the number of ROIs generated for the image data frame is greater than two. For example, in some examples, the controller accesses one or more local data structures specifying the association between the ROI and the image data frame, such as the one described above with reference to Fig.10 If the controller determines that there are less than two ROIs for a given frame of image data, process 906 may end. If the controller determines that there are two or more ROIs associated with the frame, the processor performs operation 1104.

[0112] Continuing with process 906, the controller selects 1104 ROI pairs to analyze for potential merging. For example, in some examples, the controller selects ROI combinations that have not been recorded as being processed by the current instance of process 906 from one or more local data structures specifying associations between ROIs and image data frames. In some examples, the controller determines whether any particular ROI is associated with an object that was previously detected and is being tracked. For example, in some examples, the controller accesses a data structure that lists previously detected objects that have not timed out. Object tracking and timeouts are further described below with reference to process 816. In an example, within operation 1104, the controller will not select these particular ROIs to include in the ROI pairs.

[0113] Continuing with process 906, the controller determines 1106 whether each ROI within the ROI pair is greater than a threshold size. For example, in some examples, the controller determines the pixel percentage of the ROI in the ROI pair by dividing the total number of pixels in the ROI by the total number of pixels in the frame. Next, the controller compares each pixel percentage to a threshold percentage (e.g., 50%). If the controller determines that both pixel percentages are greater than or equal to the threshold percentage, the controller performs operation 1108. If the controller determines that one or both of the pixel percentages are less than the threshold percentage, the controller performs operation 1114.

[0114] Continuing with process 906, the controller determines 1108 whether the amount of overlap of the ROIs crosses or otherwise exceeds a threshold amount of overlap. For example, in some examples, the controller calculates an area intersection (IOA) metric for the ROI pair by dividing the cardinality of the pixel sets within the intersection of the pixel sets within the ROI by the cardinality of the pixel sets within the smaller ROI. The cardinality of a set is the number of elements within the set. Next, the controller compares the IOA with a threshold percentage (e.g., 25%, 40% to 60%, 50%, etc.). If the controller determines that the IOA is greater than or equal to the threshold percentage, the controller performs operation 1110. Additionally or alternatively, the controller may calculate and compare an area and ratio (AOU) metric or some other metric indicating the amount of overlap between the ROIs within operation 1108. If the controller determines that the IOA is less than the threshold percentage, the controller performs operation 1114.

[0115] Continuing with process 906, the controller merges the ROI pair 1110. For example, in some examples, the controller creates a new ROI that includes a set of pixels that is the union of the sets of pixels within the ROIs of the ROI pair. Additionally, in some examples, the controller further extends the new ROI to include additional pixels located within the frame region defined by extending the boundaries of the ROI to the intersection between the extended boundaries. As an illustration, Fig. 12C Frame 1200 has an example of an extended region 1226 formed by the following four points—1) a first point A at the initial intersection of the left vertical boundary of ROI 1224 and the bottom horizontal boundary of ROI 1222; 2) a second point B at the bottom left boundary of ROI 1222; 3) a third point C at the intersection of an extension of the bottom horizontal boundary of ROI 1224 until the extension meets the extension of the left vertical boundary of ROI 1222; and 4) a fourth point D at the bottom left boundary of ROI 1224.

[0116] Continuing with process 906, the controller stores 1112 the data defining the merged ROI in a data structure for subsequent processing. For example, in some examples, the controller stores the coordinates of the merged ROI in a local data store (e.g., FIG. 4A to FIG. 4C 410). Fig.12D See an example of merged ROI 1228. Merged ROI 1228 is formed by the union of ROIs 1222, 1224 and extended region 1226.

[0117] Continuing with process 906, the controller determines 1114 whether any additional, unprocessed ROI pairs are associated with the image data frame. For example, in some examples, the controller checks the ordered list described above with reference to operation 1104 and / or one or more local data structures specifying associations between ROIs and image data frames for ROI combinations that have not been processed by process 906. If the controller determines that there are no such unprocessed ROI pairs, process 904 can end. If the controller determines that there are at least one such unprocessed ROI pair, the controller returns to operation 1104.

[0118] It should be noted that some examples of process 906 include certain operations in addition to operations 1102 to 1114 described above. For example, in some examples, before selecting the first ROI pair in operation 1104, the controller sorts the ROIs into a list ranked by size, and then selects only ROIs of a size (e.g., pixel count) that is at least a threshold percentage (e.g., 50%) of the pixel count of the frame. These examples can omit operation 1106 because only appropriately sized ROIs are selected. Additionally or alternatively, some examples can select an ROI pair that includes at least one merged ROI within operation 1104. In the event that a merged ROI substantially overlaps with another ROI, merging the merged ROI with the other ROI can produce additional performance benefits in terms of inference rate without significantly increasing the false positive object recognition rate.

[0119] Back to Fig. 9 In process 814, the controller applies 908 the object detection model to certain ROIs (e.g., ROIs that are preferred for object detection). For example, in some examples, the controller executes object detection code (e.g., Figure 7 718) to apply the object detection model to the ROI. In some examples, application of the model involves the NPU (e.g., Figure 7 708). In addition, in some examples, the controller prioritizes the application of the model for certain ROIs over other ROIs. For example, in some examples, the controller applies the model to a configurable number of ROIs (e.g., two ROIs) based on size, where the ROI with the largest size (e.g., most pixels) is given priority. These examples limit the number of ROIs to which the model is applied to ensure an acceptable rate of power consumption and time consumption. In some examples, the time consumed by operation 908 falls within a range between 100 milliseconds and 350 milliseconds (e.g., 250 milliseconds). As an example, Fig.12DTwo ROIs 1220 and 1228 are shown cropped from the image frame 1200 and to which an object detection model can be applied. Either or both of the ROIs can be scaled to accommodate the object detection model. In some examples, downsampling the ROI can enable the ROI to be processed faster (e.g., using an ANN) and / or with lower power consumption (which may be particularly beneficial for battery-powered sensors) than if the object detection process were applied to the full-size ROI. It should be noted that in the case where no ROI is associated with the image frame, the controller adjusts (resizes, reshapes, pads, etc.) the entire frame to accommodate the object detection model before applying the object detection model to the scaled image data.

[0120] Continuing with process 814, the controller stores 910 the data defining the region depicting the object in a data structure for subsequent processing. For example, in some examples, the controller stores information specifying a bounding box identified by the object detection model in a local data store (e.g., FIG. 4A to FIG. 4C As an example, Fig.12E Three bounding boxes 1250 , 1252 , and 1254 identified within ROIs 1220 and 1228 are shown.

[0121] Back to Figure 8As discussed in more detail below, the controller 700 is also configured to perform an object tracking process 816 on the output of the motion detection process 812 and the output of the object detection process 814 to track and classify the detected objects. Based on applying the motion detection process 812, the object detection process 814, and the object tracking process 816 to the captured image frame 808, the controller 700 generates an output 818 (e.g., different categories of output). According to the output 818, the security device 702 can take different actions. For example, as further described below, the controller 700 can be configured to generate (e.g., via the use of a Kalman filter) a certain category of output 818 for identified moving objects (identified objects paired with motion) and stationary objects (detected objects not paired with motion). In an example, the detection of stationary objects or the detection of identified moving objects that are not considered to represent a threat will not cause the security device 702 to trigger an alarm. Instead, the controller 700 can deactivate the image sensor 706, and the security device 702 can return to a low power state until the motion sensor 704 detects a new instance of motion. Detection of an identified moving object (such as, for example, a person) that may represent a threat may cause the security device 702 to trigger an alarm. After the alarm is triggered, the controller 700 may instruct the image sensor 706 (indicated at 820) to begin recording a video sequence at 822. The video sequence may be reviewed by a human operator (or a device hosting an artificial intelligence process) to determine whether to take additional action.

[0122] Within process 816, the controller 700 applies one or more AI models to the outputs from the motion detection process 812 and the object detection process 814 to track and classify the detected moving objects. For example, in some examples, the controller 700 controls the NPU 708 to apply one or more ANNs trained to produce output 818. In some examples, process 816 includes matching (or pairing) instances of detected motion with detected objects to identify and classify moving objects. For example, if there is sufficient overlap between the first bounding box output from the motion detection process 812 and the second bounding box output from the object detection process 814, the controller 700 links the object associated with the first bounding box and the motion associated with the second bounding box to determine that the object is moving. In the example, process 816 uses an implementation of a linear and distribution algorithm to pair or match the first bounding box output from the motion detection process 812 and the second bounding box output from the object detection process 814, respectively. Thus, as discussed above, the pairing process may link detected motion (discovered during the motion detection process 812 ) with detected objects (identified during the object detection process 814 ) to produce outputs 818 of different categories.

[0123] In the example, a Kalman filter is used to track an object in process 816. A Kalman filter is a control process that uses time-varying measurement observations to predict future values ​​of a measurement. The predictions of the Kalman filter tend to be robust to noise and other inaccuracies present in the measurement. For each frame (acquired at 808), a new object detection result (from 814) is compared with a set of currently tracked objects. If the first bounding box and the second bounding box have any overlap, the tracked object is updated accordingly. If the detected object does not overlap with any currently tracked object, a new tracked object is created. In the example, process 816 includes a timeout feature so that if the tracked object has not been updated within a time period, the tracked object is deleted. In the example, for example, a Kalman filter is also used to track the detected motion (from the motion detection process 812) frame by frame. As discussed above, the bounding box describing the detected motion is compared with the bounding box describing the tracked detected object, and if there is an overlap, the tracked object is marked as a moving object. In contrast, a tracked object whose bounding box does not overlap with any bounding box output from the motion detection process 812 may be identified as a stationary object. Thus, the system can distinguish between moving objects and stationary objects.

[0124] In some examples, the output of the Kalman filter for the tracked object can be used to track the detected motion. However, the shape of the bounding box may be inconsistent frame by frame, and therefore this may generate erroneous motion. Therefore, the example described above uses the Kalman filter to independently track the detected object and the detected motion, and the bounding box is paired as discussed above to associate the object with the motion. Tracking the detected object and detecting motion in multiple image frames using the Kalman filter helps to reduce false positive alarms. In some examples, the object detection process 814 can be adjusted to detect one or more specific types of objects of interest with a relatively low certainty / confidence threshold (e.g., with a low confidence indicator as discussed above), which allows the false positives filtered out during the object tracking process 816. In this way, the system can be configured with an initial bias to the detection to avoid missing the object of interest. However, subsequent filtering can remove the false positive detection before the alarm is triggered, thereby reducing the number of false alarms received by the user of the security system. For example, using a Kalman filter to track objects and motion across multiple image frames allows the system to reject independent instances of motion and / or object detection that are not repeated frame by frame and that may therefore represent noise or simply detection errors. Thus, using a Kalman filter or other process for tracking detections made across multiple frames of image data may increase robustness to false negatives and filter out false positives.

[0125] Now turn to Fig.13, schematically illustrates a computing device 1300. Fig.13 As shown in FIG. 1 , the computing device includes at least one processor 1302, volatile memory 1304, one or more interfaces 1306, non-volatile memory 1308, and an interconnect mechanism 1314. The non-volatile memory 1308 includes code 1310 and at least one data store 1312.

[0126] In some examples, the non-volatile (non-transitory) memory 1308 includes one or more read-only memory (ROM) chips; one or more hard disk drives or other magnetic or optical storage media; one or more solid-state drives (SSDs) such as flash drives or other solid-state storage media; and / or one or more hybrid magnetic SSDs. In some examples, the code 1310 stored in the non-volatile memory may include an operating system and one or more applications or programs configured to execute under the operating system. Alternatively or additionally, the code 1310 may include dedicated firmware and embedded software that can be executed without relying on a commercial operating system. In any case, the execution of the code 1310 may generate manipulated data that can be stored in the data store 1312 as one or more data structures. The data structure may have fields that are associated by co-location in the data structure. Such an association can also be achieved by allocating storage for the field in a location within the memory that passes the association between the fields. However, other mechanisms may be used to establish associations between information in the fields of the data structure, including by using pointers, tags, or other mechanisms.

[0127] continue Fig.13In some examples, processor 1302 may be one or more programmable processors, which are used to execute one or more executable instructions such as a computer program specified by code 1310 to control the operation of computing device 1300. As used herein, the term "processor" describes a circuit system that performs a function, operation, or sequence of operations. Functions, operations, or sequences of operations may be hard-coded into the circuit system, or soft-coded by instructions stored in a memory device (e.g., volatile memory 1304) and executed by the circuit system. In some examples, processor 1302 is a digital processor, but processor 1302 may be analog, digital, or mixed. Therefore, processor 1302 may use digital values ​​and / or use analog signals to perform functions, operations, or sequences of operations. In some examples, processor 1302 may be embodied in one or more application specific integrated circuits (ASICs), microprocessors, digital signal processors (DSPs), graphics processing units (GPUs), NPUs, microcontrollers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), or multi-core processors. The example of processor 1302 as a multi-core may provide the function of executing instructions in parallel or executing one instruction on more than one piece of data in parallel.

[0128] continue Fig.13 In an example, prior to execution of code 1310, processor 1302 may copy code 1310 from non-volatile memory 1308 to volatile memory 1304. In some examples, volatile memory 1304 includes one or more static or dynamic random access memory (RAM) chips and / or cache memory (e.g., memory disposed on the silicon die of processor 1302). Volatile memory 1304 may provide faster response times than main memory such as non-volatile memory 1308.

[0129] Through the execution of code 1310, processor 1302 can control the operation of interface 1306. Interface 1306 may include a network interface. These network interfaces may include one or more physical interfaces (e.g., radio, Ethernet port, USB port, etc.) and a software stack including a driver and / or other code 1310, which is configured to communicate with one or more physical interfaces to support one or more LAN, PAN and / or WAN standard communication protocols. Communication protocols may include, for example, TCP and UDP, etc. Therefore, the network interface enables computing device 1300 to access other computing devices via a computer network and communicate with other computing devices.

[0130] Interface 1306 may include a user interface. For example, in some examples, the user interface includes a user input device and / or output device (e.g., keyboard, mouse, touch screen, display, speaker, camera, accelerometer, biometric scanner, environmental sensor, etc.) and a software stack including a driver and / or other code 1310, which is configured to communicate with the user input device and / or output device. Therefore, the user interface enables computing device 1300 to interact with the user to receive input and / or present output. The output presented may include, for example, one or more GUIs, which include one or more controls configured to display output and / or receive input. The input may specify a value to be stored in data storage 1312. The output may indicate a value stored in data storage 1312.

[0131] continue Fig.13 In some examples, the various features of computing device 1300 described above can communicate with each other via interconnect mechanism 1314. In some examples, interconnect mechanism 1314 includes a communication bus.

[0132] Various inventive concepts may be embodied as one or more methods, examples of which have been provided. The actions performed as part of a method may be ordered in any suitable manner. Thus, even though sequential actions are shown in illustrative examples, examples may be constructed that perform actions in a different order than shown, which may include performing some actions simultaneously.

[0133] A description of additional examples follows.Other variations will be apparent in light of this disclosure.

[0134] Example 1 is a method comprising: identifying pixels indicating motion based on a pixel frame; selecting a region of interest within the frame based on the pixels indicating motion, the region being a subset of the pixel frame and including pixels indicating motion; identifying an object based on the pixels within the region; and issuing an alarm in response to the region including both the object and pixels indicating motion.

[0135] Example 2 includes the subject matter of Example 1, wherein selecting the region of interest comprises setting a boundary of a region containing pixels indicative of motion.

[0136] Example 3 includes the subject matter of Example 2, wherein setting the boundary of the zone comprises identifying a first length of a boundary of pixels indicating motion; calculating a second length equal to the first length multiplied by a factor; and setting the boundary of the zone to have a length equal to the second length.

[0137] Example 4 includes the subject matter of Example 3, wherein calculating the second length comprises multiplying the first length by a factor of two.

[0138] Example 5 includes the subject matter of Example 3 or Example 4, wherein identifying the first length comprises determining that the first length is greater than another length of another boundary of pixels indicative of motion.

[0139] Example 6 includes the subject matter of Example 5, further comprising setting another border of the region to have another length equal to the second length.

[0140] Example 7 includes the subject matter of Example 6, wherein setting a boundary of the region comprises setting the boundary to be a square.

[0141] Example 8 includes the subject matter of any of Examples 2 to 7, wherein setting the boundary of the region comprises establishing a size of the region equal to or greater than a size of an input on which the model is configured to operate.

[0142] Example 9 includes the subject matter of any of Examples 2 to 8, wherein setting the boundary of the region includes establishing a size of the region to be equal to or smaller than a size of the pixel frame.

[0143] Example 10 includes the subject matter of any of Examples 2 to 9, wherein setting the boundary of the region comprises translating the boundary from a first position where a portion of the boundary resides outside the pixel frame to a second position where a portion of the boundary resides inside the pixel frame or on the boundary of the pixel frame.

[0144] Example 11 includes the subject matter of any one of Examples 1 to 10, wherein the zone is a first zone, the object is a first object, and the method further includes: selecting a second zone within a pixel frame; selecting a third zone within the pixel frame; prioritizing the first zone and the third zone over the second zone; and identifying a second object based on the second zone.

[0145] Example 12 includes the subject matter of Example 11, wherein selecting the second region comprises identifying, based on the pixel frame, an object previously identified within another pixel frame.

[0146] Example 13 includes the subject matter of any one of Examples 1 to 12, wherein the region is a first region, the object is a first object, and the method further includes: selecting a second region within a pixel frame; selecting a third region within the pixel frame; merging the second region with the third region to generate a merged region; and identifying a second object based on the merged region.

[0147] Example 14 includes the subject matter of Example 13, wherein merging the second region with the third region comprises determining that the size of the second region is equal to or greater than a threshold percentage of the size of the pixel frame; and determining that the size of the third region is equal to or greater than a threshold percentage of the size of the pixel frame.

[0148] Example 15 includes the subject matter of Example 14, wherein determining that the size of the second region is equal to or greater than a threshold percentage of the size of the pixel frame comprises determining that the size of the second region is equal to or greater than 50% of the size of the pixel frame.

[0149] Example 16 includes the subject matter of Example 14 or Example 15, wherein merging the second region with the third region comprises determining that a size of an intersection between the second region and the third region is greater than a threshold percentage of a size of a smaller of the second region and the third region.

[0150] Example 17 includes the subject matter of Example 16, wherein merging the second region with the third region comprises determining that a size of an intersection between the second region and the third region is greater than 25% of a size of a smaller of the second region and the third region.

[0151] Example 18 includes the subject matter of any of Examples 1 to 17, further comprising scaling the region to conform to the model before applying the model to recognize the object.

[0152] Example 19 is a computing device comprising a memory and at least one processor coupled to the memory. The at least one processor is configured to: identify pixels indicating motion based on a frame of pixels; select a region of interest within the frame based on the pixels indicating motion, the region being a subset of the frame of pixels and including pixels indicating motion; identify an object based on pixels within the region; and issue an alarm in response to the region including both the object and pixels indicating motion.

[0153] Example 20 includes the subject matter of Example 19, wherein selecting the region of interest comprises setting a boundary of a region containing pixels indicative of motion.

[0154] Example 21 includes the subject matter of Example 20, wherein setting the boundary of the zone comprises identifying a first length of a boundary of pixels indicating motion; calculating a second length equal to the first length multiplied by a factor; and setting the boundary of the zone to have a length equal to the second length.

[0155] Example 22 includes the subject matter of Example 21, wherein calculating the second length comprises multiplying the first length by a factor of two.

[0156] Example 23 includes the subject matter of Example 21 or Example 22, wherein identifying the first length comprises determining that the first length is greater than another length of another boundary of pixels indicative of motion.

[0157] Example 24 includes the subject matter of Example 23, wherein the at least one processor is further configured to set another boundary of the region to have another length equal to the second length.

[0158] Example 25 includes the subject matter of Example 24, wherein setting the border of the region comprises setting the border to be a square.

[0159] Example 26 includes the subject matter of any of Examples 20 to 25, wherein setting the boundaries of the region comprises establishing a size of the region equal to or greater than a size of an input on which the model is configured to operate.

[0160] Example 27 includes the subject matter of any of Examples 20 to 26, wherein setting the boundary of the region comprises establishing a size of the region to be equal to or smaller than a size of the pixel frame.

[0161] Example 28 includes the subject matter of any of Examples 20 to 27, wherein setting the boundary of the region comprises translating the boundary from a first position where a portion of the boundary resides outside the pixel frame to a second position where a portion of the boundary resides inside the pixel frame or on the boundary of the pixel frame.

[0162] Example 29 includes the subject matter of any of Examples 19 to 28, wherein the zone is a first zone; the object is a first object; and at least one processor is further configured to select a second zone within a pixel frame; select a third zone within a pixel frame; prioritize the first zone and the third zone over the second zone; and identify a second object based on the second zone.

[0163] Example 30 includes the subject matter of Example 29, wherein selecting the second region comprises identifying, based on the pixel frame, an object previously identified within another pixel frame.

[0164] Example 31 includes the subject matter of any one of Examples 19 to Example 30, wherein the region is a first region; the object is a first object; and at least one processor is further configured to: select a second region within a pixel frame; select a third region within the pixel frame; merge the second region with the third region to generate a merged region; and identify a second object based on the merged region.

[0165] Example 32 includes the subject matter of Example 31, wherein merging the second region with the third region comprises: determining that the size of the second region is equal to or greater than a threshold percentage of the size of the pixel frame; and determining that the size of the third region is equal to or greater than a threshold percentage of the size of the pixel frame.

[0166] Example 33 includes the subject matter of Example 32, wherein determining that the size of the second region is equal to or greater than a threshold percentage of the size of the pixel frame comprises determining that the size of the second region is equal to or greater than 50% of the size of the pixel frame.

[0167] Example 34 includes the subject matter of Example 32 or Example 33, wherein merging the second region with the third region comprises determining that a size of an intersection between the second region and the third region is greater than a threshold percentage of a size of a smaller of the second region and the third region.

[0168] Example 35 includes the subject matter of Example 34, wherein merging the second region with the third region comprises determining that a size of an intersection between the second region and the third region is greater than 25% of a size of a smaller of the second region and the third region.

[0169] Example 36 includes the subject matter of any of Examples 19 to 35, wherein the at least one processor is further configured to scale the region to conform to the model before applying the model to identify the object.

[0170] Example 37 is one or more computer-readable media storing a sequence of instructions executable to detect an object in an image data frame using a region of interest, the sequence of instructions comprising: instructions for identifying pixels indicating motion based on a pixel frame; instructions for selecting a region of interest within a frame based on the pixels indicating motion, the region being a subset of the pixel frame and including pixels indicating motion; instructions for identifying an object based on pixels within the region; and instructions for issuing an alarm in response to the region including both an object and pixels indicating motion.

[0171] Example 38 includes the subject matter of Example 37, wherein the instructions for selecting the region of interest comprise instructions for setting a boundary of a region containing pixels indicative of motion.

[0172] Example 39 includes the subject matter of Example 37 or Example 38, wherein the instructions for setting the boundary of the zone include: instructions for identifying a first length of a boundary of pixels indicating motion; instructions for calculating a second length equal to the first length multiplied by a factor; and instructions for setting the boundary of the zone to have a length equal to the second length.

[0173] Example 40 includes the subject matter of Example 39, wherein the instructions for calculating the second length comprise multiplying the first length by a factor of two.

[0174] Example 41 includes the subject matter of Example 39, wherein the instructions for identifying the first length include instructions for determining that the first length is greater than another length of another boundary of pixels indicative of motion.

[0175] Example 42 includes the subject matter of Example 41, wherein the sequence of instructions further comprises instructions for setting another boundary of the region to have another length equal to the second length.

[0176] Example 43 includes the subject matter of Example 42, wherein the instructions for setting a border of the region include instructions for setting the border to be a square.

[0177] Example 44 includes the subject matter of any of Examples 38 to 43, wherein the instructions for setting the boundaries of the zone include instructions for establishing a size of the zone equal to or greater than a size of an input on which the model is configured to operate.

[0178] Example 45 includes the subject matter of any of Examples 38 to 44, wherein the instructions for setting the boundaries of the region include instructions for establishing the size of the region to be equal to or less than the size of the pixel frame.

[0179] Example 46 includes the subject matter of any of Examples 38 to 45, wherein the instructions for setting a boundary of a zone include instructions for translating the boundary from a first position where a portion of the boundary resides outside a pixel frame to a second position where a portion of the boundary is inside the pixel frame or on a boundary of the pixel frame.

[0180] Example 47 includes the subject matter of any of Examples 37 to 46, wherein the zone is a first zone; the object is a first object; and the instruction sequence further includes: instructions for selecting a second zone within a pixel frame; instructions for selecting a third zone within a pixel frame; instructions for prioritizing the first zone and the third zone over the second zone; and instructions for identifying a second object based on the second zone.

[0181] Example 48 includes the subject matter of Example 47, wherein the instructions for selecting the second region include instructions for identifying, based on the pixel frame, an object previously identified within another pixel frame.

[0182] Example 49 includes the subject matter of any one of Examples 37 to 48, wherein the region is a first region; the object is a first object; and the instruction sequence further includes: instructions for selecting a second region within a pixel frame; instructions for selecting a third region within a pixel frame; instructions for merging the second region with the third region to generate a merged region; and instructions for identifying a second object based on the merged region.

[0183] Example 50 includes the subject matter of Example 49, wherein the instructions for merging the second region with the third region include: instructions for determining that the size of the second region is equal to or greater than a threshold percentage of the size of the pixel frame; and instructions for determining that the size of the third region is equal to or greater than a threshold percentage of the size of the pixel frame.

[0184] Example 51 includes the subject matter of Example 50, wherein the instructions for determining that the size of the second region is equal to or greater than a threshold percentage of the size of the pixel frame include instructions for determining that the size of the second region is equal to or greater than 50% of the size of the pixel frame.

[0185] Example 52 includes the subject matter of Example 50 or Example 51, wherein the instructions for merging the second region with the third region include instructions for determining that the size of the intersection between the second region and the third region is greater than a threshold percentage of the size of the smaller of the second region and the third region.

[0186] Example 53 includes the subject matter of Example 52, wherein the instructions for merging the second region with the third region include instructions for determining that a size of an intersection between the second region and the third region is greater than 25% of a size of a smaller of the second region and the third region.

[0187] Example 54 includes the subject matter of any of Examples 37 to 53, wherein the sequence of instructions further comprises instructions for scaling the region to conform to the model before applying the model to identify the object.

[0188] The use of ordinal terms such as "first," "second," "third," etc. in the claims to modify claim elements does not in itself imply any priority, precedence, or order of one claim element relative to another claim element or a temporal order of actions in performing a method. Such terms are merely used as markers to distinguish one claim element with a particular name from another element with the same name (but for ordinal terms).

[0189] The examples of the methods and systems discussed herein are not limited in application to the construction details and component arrangements set forth in the following description or shown in the accompanying drawings. The methods and systems can be implemented in other examples and can be practiced or executed in various ways. Examples of specific implementations are provided herein for illustrative purposes only and are not intended to be limiting. In particular, the actions, components, elements, and features discussed in conjunction with any one or more examples are not intended to be excluded from similar roles in any other examples.

[0190] In addition, the wording and terms used herein are for descriptive purposes and should not be considered restrictive. Any reference to the examples, parts, elements or actions of the systems and methods cited herein in the singular may also include examples containing multiple examples, and any reference to any example, part, element or action in the plural form herein may also include examples containing only the singular. References in singular or plural form are not intended to limit the currently disclosed system or method, its parts, actions or elements. "Including", "including", "having", "containing", "involving" and its variations used herein are intended to cover the items listed later and their equivalents and additional items. References to "or" may be interpreted as inclusive, so that any term described using "or" may indicate any of the terms described in a single, more than one and all. In addition, in the case of inconsistent usage of terms between this document and the documents incorporated by reference herein, the term usage in the incorporated reference is a supplement to the term usage of this document; for irreconcilable inconsistencies, the term usage in this document shall prevail.

[0191] Having described several examples in detail, various modifications and improvements will readily occur to those skilled in the art. Such modifications and improvements are intended to be within the scope of the present disclosure. Therefore, the foregoing description is only exemplary and is not intended to be limiting.

Claims

1. A method comprising: identifying pixels indicating motion based on the pixel frame; selecting a region of interest within the frame based on the pixels indicative of motion, the region being a subset of the frame of pixels and including the pixels indicative of motion; identifying an object based on pixels within the region; as well as An alarm is issued in response to the region including both the object and the pixels indicative of motion.

2. The method according to claim 1, wherein: Selecting the region of interest includes setting boundaries of the region containing the pixels indicative of motion.

3. The method according to claim 2, wherein: Setting the boundaries of the zone includes: identifying a first length of a boundary of the pixels indicative of motion; calculating a second length equal to the first length multiplied by a factor; and The border of the region is set to have a length equal to the second length.

4. The method according to claim 3, wherein: Calculating the second length includes multiplying the first length by a factor of two.

5. The method according to claim 3 or claim 4, wherein: Identifying the first length includes determining that the first length is greater than another length of another boundary of the pixels indicative of motion. The method of claim 5 , further comprising setting another border of the zone to have another length equal to the second length.

7. The method according to claim 6, wherein: Setting the border of the region includes setting the border into a square.

8. The method according to any one of claims 2 to 7, wherein: Setting the boundaries of the region includes establishing a size of the region equal to or greater than a size of an input on which the model is configured to operate.

9. The method according to any one of claims 2 to 8, wherein: Setting the boundaries of the region includes establishing a size of the region to be equal to or smaller than a size of the pixel frame.

10. The method according to any one of claims 2 to 9, wherein: Setting the border of the region includes translating the border from a first position where a portion of the border resides outside the pixel frame to a second position where the portion of the border resides within the pixel frame or on a border of the pixel frame.

11. The method according to any one of claims 1 to 10, wherein: The zone is a first zone, the object is a first object, and the method further comprises: selecting a second region within the pixel frame; selecting a third region within the pixel frame; giving priority to the first zone and the third zone over the second zone; and A second object is identified based on the second zone.

12. The method according to claim 11, wherein: Selecting the second region includes identifying an object previously identified within another frame of pixels based on the frame of pixels.

13. The method according to any one of claims 1 to 12, wherein: The zone is a first zone, the object is a first object, and the method further comprises: selecting a second region within the pixel frame; selecting a third region within the pixel frame; merging the second region with the third region to generate a merged region; and A second object is identified based on the merged region.

14. The method according to claim 13, wherein: Merging the second region with the third region comprises: determining that the size of the second region is equal to or greater than a threshold percentage of the size of the pixel frame; and A size of the third region is determined to be equal to or greater than a threshold percentage of the size of the pixel frame.

15. The method according to claim 14, wherein: Determining that the size of the second region is equal to or greater than the threshold percentage of the size of the pixel frame includes determining that the size of the second region is equal to or greater than 50% of the size of the pixel frame.

16. The method according to claim 14 or claim 15, wherein: Merging the second region with the third region includes determining that a size of an intersection between the second region and the third region is greater than a threshold percentage of a size of a smaller of the second region and the third region.

17. The method according to claim 16, wherein: Merging the second region with the third region includes determining that a size of an intersection between the second region and the third region is greater than 25% of a size of a smaller of the second region and the third region.

18. The method of any one of claims 1 to 17, further comprising scaling the region to conform to the model before applying the model to identify the object.

19. A computing device comprising a memory and at least one processor coupled to the memory, the at least one processor being configured to: identifying pixels indicating motion based on the pixel frame; selecting a region of interest within the frame based on the pixels indicative of motion, the region being a subset of the frame of pixels and including the pixels indicative of motion; identifying an object based on pixels within the region; as well as An alarm is issued in response to the region including both the object and the pixels indicative of motion.

20. The computing device of claim 19, wherein: Selecting the region of interest includes setting boundaries of the region containing the pixels indicative of motion.

21. The computing device of claim 20, wherein: Setting the boundaries of the zone includes: identifying a first length of a boundary of the pixels indicative of motion; calculating a second length equal to the first length multiplied by a factor; and The border of the region is set to have a length equal to the second length.

22. The computing device of claim 21, wherein: Calculating the second length includes multiplying the first length by a factor of two.

23. A computing device according to claim 21 or claim 22, wherein: Identifying the first length includes determining that the first length is greater than another length of another boundary of the pixels indicative of motion.

24. The computing device of claim 23, wherein: The at least one processor is further configured to set another border of the region to have another length equal to the second length.

25. The computing device of claim 24, wherein: Setting the border of the region includes setting the border into a square.

26. A computing device according to any one of claims 20 to 25, wherein: Setting the boundaries of the region includes establishing a size of the region equal to or greater than a size of an input on which the model is configured to operate.

27. A computing device according to any one of claims 20 to 26, wherein: Setting the boundaries of the region includes establishing a size of the region to be equal to or smaller than a size of the pixel frame.

28. A computing device according to any one of claims 20 to 27, wherein: Setting the border of the region includes translating the border from a first position where a portion of the border resides outside the pixel frame to a second position where the portion of the border resides within the pixel frame or on a border of the pixel frame.

29. A computing device according to any one of claims 19 to 28, wherein: The zone is a first zone; The object is a first object; and The at least one processor is further configured to: selecting a second region within the pixel frame, selecting a third region within the pixel frame, giving priority to the first zone and the third zone over the second zone, and A second object is identified based on the second zone.

30. The computing device of claim 29, wherein: Selecting the second region includes identifying an object previously identified within another frame of pixels based on the frame of pixels.

31. A computing device according to any one of claims 19 to 30, wherein: The zone is a first zone; The object is a first object; and The at least one processor is further configured to: selecting a second region within the pixel frame, selecting a third region within the pixel frame, merging the second region with the third region to generate a merged region, and A second object is identified based on the merged region.

32. The computing device of claim 31 , wherein: Merging the second region with the third region comprises: determining that the size of the second region is equal to or greater than a threshold percentage of the size of the pixel frame; and A size of the third region is determined to be equal to or greater than a threshold percentage of the size of the pixel frame.

33. The computing device of claim 32, wherein: Determining that the size of the second region is equal to or greater than the threshold percentage of the size of the pixel frame includes determining that the size of the second region is equal to or greater than 50% of the size of the pixel frame.

34. A computing device according to claim 32 or claim 33, wherein: Merging the second region with the third region includes determining that a size of an intersection between the second region and the third region is greater than a threshold percentage of a size of a smaller of the second region and the third region.

35. The computing device of claim 34, wherein: Merging the second region with the third region includes determining that a size of an intersection between the second region and the third region is greater than 25% of a size of a smaller of the second region and the third region.

36. A computing device according to any one of claims 19 to 35, wherein: The at least one processor is further configured to scale the region to conform to the model prior to applying the model to identify the object.

37. One or more computer readable media storing a sequence of instructions executable to detect an object in a frame of image data using a region of interest, the sequence of instructions comprising: instructions for identifying pixels indicating motion based on the frame of pixels; instructions for selecting a region of interest within the frame based on the pixels indicative of motion, the region being a subset of the frame of pixels and including the pixels indicative of motion; instructions for identifying an object based on pixels within the region; as well as Instructions for issuing an alarm in response to the region including both the object and the pixels indicative of motion.

38. One or more computer readable media according to claim 37, wherein: The instructions for selecting the region of interest include instructions for setting boundaries of the region containing the pixels indicative of motion.

39. One or more computer readable media according to claim 37 or claim 38, wherein: The instructions for setting the boundaries of the zone include: instructions for identifying a first length of a boundary of the pixels indicative of motion; instructions for calculating a second length equal to the first length multiplied by a factor; and Instructions for setting a boundary of the region to have a length equal to the second length.

40. One or more computer readable media according to claim 39, wherein: The instructions for calculating the second length include multiplying the first length by a factor of two.

41. One or more computer readable media according to claim 39, wherein: The instructions for identifying the first length include instructions for determining that the first length is greater than another length of another boundary of the pixels indicative of motion.

42. One or more computer readable media according to claim 41, wherein: The sequence of instructions also includes instructions for setting another boundary of the region to have another length equal to the second length.

43. One or more computer readable media according to claim 42, wherein: The instructions for setting the border of the region include instructions for setting the border into a square.

44. One or more computer readable media according to any one of claims 38 to 43, wherein: The instructions for setting the boundaries of the region include instructions for establishing a size of the region equal to or greater than a size of an input on which the model is configured to operate.

45. One or more computer readable media according to any one of claims 38 to 44, wherein: The instructions for setting the boundaries of the region include instructions for establishing a size of the region to be equal to or less than a size of the pixel frame.

46. ​​One or more computer readable media according to any one of claims 38 to 45, wherein: The instructions for setting the border of the region include instructions for translating the border from a first position where a portion of the border resides outside the pixel frame to a second position where the portion of the border is inside the pixel frame or on a border of the pixel frame.

47. One or more computer readable media according to any one of claims 37 to 46, wherein: The zone is a first zone; The object is a first object; and The instruction sequence also includes: instructions for selecting a second region within the pixel frame, instructions for selecting a third region within the pixel frame, instructions for giving priority to the first zone and the third zone over the second zone, and Instructions for identifying a second object based on the second zone.

48. One or more computer readable media according to claim 47, wherein: The instructions for selecting the second region include instructions for identifying an object previously identified within another frame of pixels based on the frame of pixels.

49. One or more computer readable media according to any one of claims 37 to 48, wherein: The zone is a first zone; The object is a first object; and The instruction sequence also includes: instructions for selecting a second region within the pixel frame, instructions for selecting a third region within the pixel frame, instructions for merging the second region with the third region to generate a merged region, and Instructions are provided for identifying a second object based on the merging region.

50. One or more computer readable media according to claim 49, wherein: The instruction for merging the second area with the third area includes: instructions for determining that the size of the second region is equal to or greater than a threshold percentage of the size of the pixel frame; and Instructions for determining that a size of the third region is equal to or greater than a threshold percentage of a size of the pixel frame.

51. One or more computer readable media according to claim 50, wherein: The instructions for determining that the size of the second region is equal to or greater than the threshold percentage of the size of the pixel frame include instructions for determining that the size of the second region is equal to or greater than 50% of the size of the pixel frame.

52. One or more computer readable media according to claim 50 or claim 51, wherein: The instructions for merging the second region with the third region include instructions for determining that a size of an intersection between the second region and the third region is greater than a threshold percentage of a size of a smaller of the second region and the third region.

53. One or more computer readable media according to claim 52, wherein: The instructions for merging the second region with the third region include instructions for determining that a size of an intersection between the second region and the third region is greater than 25% of a size of a smaller of the second region and the third region.

54. One or more computer readable media according to any one of claims 37 to 53, wherein: The sequence of instructions also includes instructions for scaling the region to conform to the model prior to applying the model to identify the object.

Images