Managing remote access to image capture devices

Abstract

A disclosed method includes receiving a first request to establish a first connection between a computing device and a camera, wherein the first connection is configured to allow video data to stream from the camera to the computing device, and the camera is configured to concurrently support connections with multiple remote devices. It is determined that a second connection has been established between an endpoint device and the camera, wherein the second connection is configured to allow video data to stream from the camera to the endpoint device. Based at least in part on the first request and the second connection having been established, the second connection is terminated and the first connection is established, thereby providing the computing device with access to one or more functions of the camera without interference from the endpoint device.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 441,593, filed January 27, 2023, entitled “MANAGING REMOTE ACCESS TO IMAGECAPTURE DEVICES,” under 35 U.S.SC §119(e), the entire contents of which are incorporated herein by reference. Background Technology

[0003] Some security systems enable the use of cameras and other devices to remotely monitor locations. Attached Figure Description

[0004] Further examples, features, and advantages of this disclosure will become more apparent from the description taken herein in conjunction with the accompanying drawings, which are incorporated in and constitute a part of this disclosure. The drawings are not necessarily drawn to scale.

[0005] Figure 1 A first example screen is shown according to some embodiments of the present disclosure, which can be displayed by a monitoring device to indicate certain events detected by a security system.

[0006] Figure 2 A second example screen is shown according to some embodiments of the present disclosure, which can be displayed by a monitoring device to present a real-time video feed from a location monitored by a security system.

[0007] Figure 3 A first example screen is shown according to some embodiments of the present disclosure, which can be displayed by a client device to enable access to a real-time video feed from a location monitored by a security system.

[0008] Figure 4 A second example screen is shown according to some embodiments of the present disclosure, which can be displayed by a client device when a monitoring agent gains exclusive access to the same camera while the client views a live feed from the camera.

[0009] Figure 5 A third example screen is shown according to some embodiments of the present disclosure, which can be displayed by a client device when a client attempts to view a live feed from the same camera after a monitoring agent has already obtained exclusive access to the camera.

[0010] Figure 6 Example implementations of a security system according to some embodiments of this disclosure are shown.

[0011] Figure 7 Some embodiments according to this disclosure are shown. Figure 6Example embodiments of a base station of a security system.

[0012] Figure 8 Example embodiments of a base station of a security system are shown. Figure 6 Example embodiments of a keypad of a security system are shown.

[0013] Figure 9 Example embodiments of a base station of a security system are shown. Figure 6 Example embodiments of a security sensor of a security system are shown.

[0014] Figure 10 Example embodiments of a base station of a security system are shown. Figure 6 Example embodiments of a monitoring center environment and a monitoring center environment of a security system are shown.

[0015] Figure 11 is a sequence diagram showing an example signaling process that can be used to establish a peer-to-peer connection between components of a security system to enable streaming of video and / or audio data in accordance with some embodiments of the present disclosure. Figure 6 is a sequence diagram showing a monitoring process that can be performed by components of a security system in accordance with some embodiments of the present disclosure.

[0016] Figure 12 Example embodiments of a base station of a security system are shown. Figure 6 Example embodiments of a process for establishing a peer-to-peer connection between components of a security system to enable streaming of video and / or audio data are shown.

[0017] Figure 13 is a sequence diagram showing an example signaling process that can be used to establish a peer-to-peer connection between components of a security system to enable streaming of video and / or audio data in accordance with some embodiments of the present disclosure. Figure 6 Example embodiments of a process for establishing a peer-to-peer connection between components of a security system to enable streaming of video and / or audio data are shown.

[0018] Figure 14 is a flowchart showing a first example routine that can be performed by a camera of a security system in accordance with some embodiments of the present disclosure. Figure 6 Example embodiments of a process for establishing a peer-to-peer connection between components of a security system to enable streaming of video and / or audio data are shown.

[0019] Figure 15 is a flowchart showing a second example routine that can be performed by a camera of a security system in accordance with some embodiments of the present disclosure. Figure 6 Example embodiments of a process for establishing a peer-to-peer connection between components of a security system to enable streaming of video and / or audio data are shown.

[0020] Figure 16 is a sequence diagram showing interactions that can occur between a client device, a camera, and a monitoring device of a security system when the routines shown in Figure 14 and Figure 15 Example embodiments of a process for establishing a peer-to-peer connection between components of a security system to enable streaming of video and / or audio data are shown. Figure 6 is a sequence diagram showing interactions that can occur between a client device, a camera, and a monitoring device of a security system when the routines shown in

[0021] Figure 17Some embodiments of this disclosure can be used for implementation. Figure 6 A schematic diagram of the client equipment, monitoring equipment, and / or computing devices for one or more services of the security system shown.

[0022] Figure 18 Example tokens that can be adopted by various components of the system disclosed herein, according to some embodiments of this disclosure, are shown. Summary of the Invention

[0023] In some disclosed embodiments, a method includes: receiving a first request to establish a first connection between a computing device and a camera, wherein the first connection is configured to allow video data to be streamed from the camera to the computing device, the computing device being remote from the camera, and the camera being configured to simultaneously support connections to multiple remote devices; determining that a second connection has been established between an endpoint device and the camera, wherein the endpoint device is remote from the camera, and the second connection is configured to allow video data to be streamed from the camera to the endpoint device; and terminating the second connection and establishing the first connection, at least in part based on the establishment of the first request and the second connection, to provide the computing device with access to one or more functions of the camera without interference from the endpoint device.

[0024] In other disclosed embodiments, a method includes: receiving a first request to establish a first connection between an endpoint device and a camera, wherein the endpoint device is remote from the camera and the camera is configured to simultaneously support connections to multiple remote devices; determining that a second connection has been established between a second computing device and the camera, wherein the second computing device is remote from the camera and the second connection is configured to allow video data to be streamed from the camera to the second computing device; and rejecting the first request, at least in part, based on the establishment of the first request and the second connection, such that the second computing device can access one or more functions of the camera without interference from the endpoint device.

[0025] In other disclosed embodiments, a system includes: at least one processor and at least one computer-readable medium encoded with instructions, which, when executed by the at least one processor, cause the system to: receive a first request to establish a first connection between a computing device and a camera, wherein the first connection is configured to allow video data to be streamed from the camera to the computing device, the computing device being remote from the camera, and the camera being configured to simultaneously support connections to multiple remote devices; determine that a second connection has been established between an endpoint device and the camera, wherein the endpoint device is remote from the camera, and the second connection is configured to allow video data to be streamed from the camera to the endpoint device; and, at least in part based on the first request and the establishment of the second connection, terminate the second connection and establish the first connection to provide the computing device with access to one or more functions of the camera without interference from the endpoint device. Detailed Implementation

[0026] Existing security systems use cameras and other sensors to monitor locations for a variety of reasons. Some such systems are configured to detect the occurrence of certain phenomena (e.g., motion and / or sound) within or around a monitored location, and are further configured to send event notifications and associated image data to a remote location for processing and / or review by a human monitoring agent. The monitoring agent can review the event notifications and their associated images to determine whether individual event notifications are causing actual security issues or, conversely, are generated due to innocuous causes (such as a pet or other animal, a visiting neighbor, a tree moving in strong winds, a delivery person, a door-to-door salesperson, etc.).

[0027] A system is provided in which, when a monitoring agent determines that a notification (e.g., an event notification) is causing a potential security issue, the monitoring agent can additionally review live video and / or audio from the location to assess whether the detected event is causing a security issue. For example, in some embodiments, the system can allow a computing device operated by the monitoring agent to establish a peer-to-peer connection with one or more cameras at the location, e.g., to enable the streaming of video data and / or audio data between the monitoring agent’s computing device and the camera(s). Further, in some embodiments, the system can additionally prevent one or more other computing devices (e.g., a computing device operated by a customer) from establishing or maintaining a peer-to-peer connection with such camera(s). Preventing such other computing device(s) from establishing a peer-to-peer connection with the camera(s) can be advantageous because doing so can (A) ensure a high-quality connection with the camera(s), and (B) enable the monitoring agent to take control of the situation without interference from the customer, such as by engaging in two-way communication (e.g., via a microphone and speaker) with one or more suspicious individuals in the vicinity of the camera(s). In some embodiments, the computing device operated by the monitoring agent can be the only computing device permitted to establish a peer-to-peer connection with the camera(s), thereby giving the monitoring agent exclusive access to the hardware resources of the camera(s).

[0028] Without granting the surveillance agent exclusive access to (one or more) cameras, or at least excluding customer access, customers may attempt to stream live video from the cameras and / or communicate with suspicious individuals. This could potentially overload the camera's hardware resources and interfere with the surveillance agent's ability to conduct effective two-way communication with these individuals. Regarding hardware resource overload, some cameras may be unable to reliably stream high-quality video data to multiple endpoints, may favor one peer over another at a given time, or may even freeze or otherwise cease operation for extended periods. This could potentially prevent the surveillance agent from quickly identifying threat situations near the affected (one or more) cameras. Regarding the potential impact on the surveillance agent's ability to conduct two-way communication with (one or more) individuals near the cameras, customers who have observed intruders at the monitored location may experience significant stress and / or extreme anxiety, and therefore may find it difficult to engage in dialogue with (one or more) individuals in a manner that could progressively escalate the situation. By allowing the surveillance agent to control the camera's speaker to exclude customers, the surveillance agent can be in a better position to engage in meaningful dialogue with (one or more) individuals, for example, without having to "negotiate" with the customer. As described below, when a surveillance agent has taken control of one or more cameras to exclude a customer, it can notify the customer so that the customer understands why the customer suddenly lost the camera connection or could not establish such a connection.

[0029] Figure 1 A security system 600 that can be configured according to certain aspects of this disclosure is shown. Figure 6 The computer or other monitoring equipment 1016 (shown below) is located inside the computer or other monitoring equipment. Figure 10 Example screen 102 (described). As shown, the monitoring device 1016 can be operated by the monitoring agent 104, and screen 102 can include a set of event windows 106 corresponding to various events currently in the agent's queue for review. In some implementations, for example, each window 106 can be configured to play back recorded video clips corresponding to the respective events detected at each monitoring location. Figure 1 As shown, in some configurations, screen 102 may include a queue control interface 108, which includes one or more user interface (UI) elements to allow monitoring agent 104 to control various aspects of the agent's queues, such as the maximum number of notifications that can be added to the agent's queue for presentation in the corresponding event window 106. In some implementations, event notifications may be distributed to individual monitoring agents 104 included in a pool of available monitoring agents 104, such that all available monitoring agents 104 have approximately the same number of events in their lookup queues at a given time.

[0030] Camera 604 of security system 600 (seeFigure 6 This can be triggered by a person entering the field of view (FOV) of camera 604, and video signals can be recorded for a period of time, such as until the person leaves the camera's FOV. Video clips of such events can be stored (e.g., in...). Figure 10 The image data is stored in the image data storage 1004 shown and associated with the event. Then, a notification of the event can be added to the query queue of the monitoring agent 104, for example, by... Figure 1 An event window 106 shows a video clip of the detected event being recorded.

[0031] When reviewing an event window 106, for example by viewing a recorded video clip corresponding to a detected motion, the monitoring agent 104 may determine that there is no potential security threat and provide input instructing the monitoring device to review the event notifications in the queue from the agent, thereby releasing the corresponding event window 106 to display another event notification. Alternatively, when reviewing an event window 106, for example by viewing a recorded video clip corresponding to a detected motion, the monitoring agent 104 may determine that there is a potential threat or other security issue and decide that reviewing the live video and / or audio from the monitoring location 602 of the recorded video clip could help resolve the issue. The monitoring agent 104 may access the live video and / or audio from the monitoring location, for example, by selecting the event window 106 that is playing or otherwise displaying the recorded video in question. In response to such a selection, the monitoring device 1016 may begin receiving live video and / or audio streaming from one or more cameras at the monitoring location 602. In some embodiments, for example, the Web Real-Time Communication (WebRTC) function of a browser on the monitoring device 1016 may be used to access the live video and / or audio from one or more cameras 604 at the monitoring location. Figure 6 As shown in the diagram, one or more peer connections are established between the camera(s) and the monitoring device 1016 to enable streaming of video and / or audio data between the camera(s) and the monitoring device 1016. The following is in conjunction with... Figure 12 and Figure 13 This describes an example procedure for securely establishing a peer-to-peer connection between monitoring device 1016 and camera 604.

[0032] Figure 2 This demonstrates that the monitoring device 1016 can respond to the... Figure 1 An example screen 110 is shown presented by selecting an event window 106. In the illustrated example, screen 110 includes three video feed windows 112, configured to display streaming video from three different cameras 604 at a monitoring location 602 corresponding to the selected event window 106. Although Figure 2While not illustrated, controls can additionally or alternatively be provided to allow the monitoring agent 104 to listen to streaming audio from the corresponding camera(s) 604 and speak into the microphone, in order to cause one or more loudspeakers of the camera(s) 604 to output audio representing the voice of the monitoring agent. In the illustrated example, the screen 110 also includes a larger, main viewer window 114 in which the streaming video of one of the video feed windows 112 can optionally be played, thereby making it easier for the monitoring agent 104 to see the content of the video. In some implementations, the monitoring agent 104 can cause the streaming video from a particular camera 604 to be played in the main viewer window 114 by selecting the video feed window 112 for that camera (e.g., by clicking on it).

[0033] The monitoring agent 104 can take appropriate action based on review of the live video and / or audio from the camera(s) 604. If the monitoring agent 104 determines that there can be a threat or other security issue, the monitoring agent 104 can trigger an alarm, notify the police, verbally communicate with one or more individuals at the monitored location 602, e.g., via a loudspeaker on the camera 604, and / or take any of a number of other possible remedial actions. If the monitoring agent 104 determines that there is no security issue, the monitoring agent 104 can instead flag the incident as clear, thereby causing it to be removed from the agent's queue.

[0034] As Figure 3 illustrated, in some implementations the security system 600 can also be configured to allow the customer 302 to operate a smartphone or other client, endpoint, or customer device 624 (illustrated in Figure 6 ) to stream live video and / or audio from one or more of the camera(s) 604 at the monitored location 602. For example, as illustrated, the customer device 624 can present a screen 304 including user interface elements 306 corresponding to respective cameras 604 at the monitored location 602, and can be configured such that selection of one of these user interface elements 306 causes a peer-to-peer connection to be established between the customer device 624 and the corresponding camera 604 (e.g., using WebRTC functionality of a browser on the customer device 624). An example process for securely establishing a peer-to-peer connection between the customer device 624 and a camera 604 is described below in connection with Figure 12 and Figure 13 .

[0035] As Figure 4As shown, in cases where the client 302 is viewing a live feed from the same camera 604 that the monitoring agent 104 has initiated a live stream with using the camera 604, the client device 624 can terminate the connection with the camera 604 and display a message 402 indicating that the monitoring agent 104 has taken over the camera feed. Additionally, as Figure 5 shown, in cases where the client 302 attempts to initiate a live stream with a camera 604 that has already been used by the monitoring agent 104, the client device 624 can refrain from streaming data from the camera 604 and instead display a message 502 indicating that a live stream with the camera 604 cannot be provided due to the monitoring agent 104 currently using the camera 604. The following detailed description of example processes that can be employed by the camera 604 to cause the client device 624 to operate as described above in connection with Figures 14 to 16 Figures 3 to 5 the message 502 is provided below in connection with the detailed description of example processes that can be employed by the camera 604 to cause the client device 624 to operate as described above in connection with the message 402.

[0036] As described above, preventing the client device 624 from establishing and / or maintaining a live stream connection with the camera 604 when the monitoring agent 104 is actively using the camera 604 to address a potential security issue can provide significant advantages, such as allowing the monitoring agent to fully utilize the hardware capabilities of the camera, and preventing the client 302 from interfering with the communication between the monitoring agent 104 and one or more individuals in the vicinity of the camera 604 when the monitoring agent 104 is attempting to assess and / or de-escalate a potential threat situation.

[0037] Figure 6 is a schematic diagram of an example security system 600 that can be employed with various aspects of the present disclosure. As shown, in some implementations, the security system 600 can include a plurality of monitoring locations 602 (only one of which is illustrated in Figure 6 the monitoring locations 602, the monitoring center environment 622, the surveillance center environment 626, the one or more client devices 624, and the communication network(s) 620 can each include one or more computing devices (e.g., as described below in connection with the detailed description of example computing devices that can be employed in connection with the present disclosure). Figure 17 ​(Described). Client device 624 (one or more) may include one or more client applications 634, such as applications hosted on or otherwise accessible to client device 624 (one or more). In some embodiments, client application 634 may be implemented as a web application accessible via a browser on client device 624 (one or more). Monitoring center environment 622 may include one or more monitoring applications 632, such as applications hosted on or otherwise accessible on a computing device within monitoring center environment 622. In some embodiments, monitoring application 632 may be implemented as a web application accessible via a browser on a computing device operated by monitoring agent 104 within monitoring center environment 622. Monitoring center environment 626 may include monitoring service 630 and one or more transport services 628.

[0038] like Figure 6 As shown, monitoring location 602 may include one or more image capture devices (e.g., cameras 604A and 604B), one or more contact sensor components (e.g., contact sensor component 606), one or more keypads (e.g., keypad 608), one or more motion sensor components (e.g., motion sensor component 610), base station 612, and router 614. As illustrated, base station 612 may host monitoring client 616.

[0039] In some implementations, router 614 may be a wireless router configured to communicate with devices (e.g., devices 604A, 604B, 606, 608, 610, and 612) located at monitoring location 602 via communication standards consistent with any of the various Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards. Figure 6 For example, router 614 may also be configured to communicate with network(s) 620(s). In some embodiments, router 614 may implement a local area network (LAN) within or near monitoring location 602. In other embodiments, other types of networking technologies may be used additionally or alternatively within monitoring location 602. For example, in some embodiments, base station 612 may receive and forward communication packets sent by one or both of cameras 604A, 604B via a point-to-point personal area network (PAN) protocol such as Bluetooth. Other suitable wired, wireless, and mesh networking technologies and topologies will become apparent from the benefits of this disclosure and are intended to fall within the scope of the examples disclosed herein.

[0040] The network(s) 620 can include one or more public and / or private networks that support, for example, Internet Protocol (IP) communications. The network(s) 620 can include, for example, one or more LANs, one or more PANs, and / or one or more wide area networks (WANs). LANs that can be employed include wired or wireless networks that support various LAN standards such as IEEE 802.11 versions, etc. PANs that can be employed include wired or wireless networks that support various PAN standards such as Bluetooth, ZIGBEE, etc. WANs that can be employed include wired or wireless networks that support various WAN standards such as code division multiple access (CDMA), global system for mobile (GSM), etc. Regardless of the particular networking technology employed, the network(s) 620 can connect and enable data communications among components within the monitoring location 602, the monitoring center environment 622, the surveillance center environment 626, and the client device(s) 624. In at least some implementations, both the monitoring center environment 622 and the surveillance center environment 626 can include networking components (e.g., similar to the router 614) configured to communicate with the network(s) 620 and various computing devices within these environments.

[0041] The surveillance center environment 626 can include physical space, communications, cooling, and power infrastructure to support the networking operations of a large number of computing devices. For example, the infrastructure of the surveillance center environment 626 can include rack space into which computing devices can be installed, uninterruptible power supplies, cooling plenums and devices, and networking equipment. The surveillance center environment 626 can be dedicated to the security system 600, can be non-dedicated, a commercially available cloud computing service (e.g., MICROSOFT AZURE, AMAZON WEB SERVICES, GOOGLE CLOUD, etc.), or can include a hybrid configuration composed of dedicated and non-dedicated resources. Regardless of its physical or logical configuration, as shown, the surveillance center environment 626 can be configured to host the surveillance service 630 and the transmission service(s) 628. Figure 6

[0042] The monitoring center environment 622 can include multiple computing devices (e.g., desktop computers) and network equipment (e.g., one or more routers) that enable communications between the computing devices and the network(s) 620. The client device(s) 624 can each include a personal computing device (e.g., a desktop computer, a laptop computer, a tablet computer, a smart phone, etc.) and network equipment (e.g., a router, a cellular modem, a cellular radio transceiver, etc.). As shown, the client device(s) 624 can be configured to communicate with the network(s) 620 and, through the network(s) 620, with the monitoring center environment 622 and the surveillance center environment 626. Figure 6 ​For example, monitoring center environment 622 can be configured to host one or more monitoring applications 632, and client devices 624 can be configured to host one or more client applications 634.

[0043] Devices 604A, 604B, 606, and 610 can be configured to acquire analog signals via sensors incorporated into the device, generate digital sensor data based on the acquired signals, and transmit the sensor data to base station 612 (e.g., via a wireless link with router 614). The type of sensor data generated and transmitted by these devices can vary depending on the characteristics of the sensors they include. For example, image capture devices or cameras 604A and 604B can acquire ambient light, generate one or more frames of image data based on the acquired light, and transmit (one or more) frames to base station 612, although pixel resolution and frame rate can vary depending on the capabilities of the device. In some embodiments, cameras 604A and 604B can also receive and store filter zone configuration data and filter (one or more) frames using one or more filter zones (e.g., areas within the camera's field of view, from which image data is edited for various reasons, such as to exclude trees that might generate false positive motion detection results on windy days) before transmitting (one or more) frames to base station 612. Figure 6 In the example shown, camera 604A has a field of view (FOV) starting near the front door of monitoring position 602 and can acquire images of sidewalk 636, road 638, and the space between monitoring position 602 and road 64A0. On the other hand, camera 604B has an FOV starting near the bathroom of monitoring position 602 and can acquire images of the living room and dining area of ​​monitoring position 602. Camera 604B can also acquire images of outdoor areas outside monitoring position 602, for example, through windows 618A and 618B on the right side of monitoring position 602.

[0044] Various sensor components deployed at monitoring location 602 (e.g.) Figure 6 The contact sensor assembly 606 shown may include, for example, a sensor capable of detecting the presence of a magnetic field generated by a magnet when the magnet approaches the sensor. When a magnetic field is present, the contact sensor assembly 606 can generate Boolean sensor data indicating a closed state of a window, door, etc. Conversely, when a magnetic field is absent, the contact sensor assembly 606 can generate Boolean sensor data indicating an open state of a window, door, etc. In either case, Figure 6 The contact sensor assembly 606 shown can transmit sensor data indicating whether the front door of the monitored location 602 is open or closed to the base station 612.

[0045] Various motion sensor components deployed at monitoring location 602 (e.g.) Figure 6 The motion sensor assembly 610 shown may include, for example, components capable of emitting high-frequency pressure waves (e.g., ultrasound) and sensors capable of acquiring reflections of the emitted waves. When the sensor detects a change in the reflected pressure wave, for example because one or more objects are moving within the space monitored by the sensor, the motion sensor assembly 610 may generate Boolean sensor data indicating an alarm state. Conversely, when the sensor does not detect a change in the reflected pressure wave, for example because no object is moving within the monitored space, the motion sensor assembly 610 may generate Boolean sensor data indicating a stationary state. In either case, the motion sensor assembly 610 may transmit the sensor data to the base station 612. It should be noted that the specific sensing modes described above are not limited to this disclosure. For example, as an example of an alternative implementation only, the motion sensor assembly 610 may, conversely (or additionally), operate based on the detection of changes in reflected electromagnetic waves.

[0046] While specific types of sensors have been described above, it should be understood that other types of sensors may be used, either alternatively or additionally, within the monitoring location 602 to detect the presence and / or movement of people, or other conditions of interest, such as smoke, elevated carbon dioxide levels, water accumulation, etc., and to transmit data indicating these conditions to the base station 612. For example, although in Figure 6 Not illustrated, but in some embodiments, one or more sensors may be employed to detect sudden changes in measured temperature, sudden changes in incident infrared radiation, sudden changes in incident pressure waves (e.g., sound waves), etc. Further still, in some embodiments, some such sensors and / or base stations 612 may be additionally or alternatively configured to identify specific signal curves indicating specific conditions, such as sound curves indicating broken glass, footsteps, coughs, etc.

[0047] Figure 6The keypad 608 shown can be configured to interact with a user and, in response to such interaction, interoperate with other devices arranged in the monitored location 602. For example, in some examples, the keypad 608 can be configured to receive input from a user specifying one or more commands and transmit the specified commands to one or more addressed devices and / or processes, e.g., one or more devices arranged in the monitored location 602, the monitoring application(s) 632, and / or the monitoring service 630. The transmitted commands can include, for example, a code authenticating the user as a resident of the monitored location 602 and / or a code requesting activation or deactivation of one or more devices arranged in the monitored location 602. In some implementations, the keypad 608 can include a user interface (e.g., a tactile interface such as a set of physical buttons or a set of "soft" buttons on a touch screen) configured to interact with a user (e.g., receive input from and / or present output to the user). Further, in some implementations, the keypad 608 can receive responses to transmitted commands and present such responses as visual or audio output via the user interface.

[0048] Figure 6 The base station 612 shown can be configured to interoperate with other security system devices arranged at the monitored location 602 to provide local command and control and / or store-and-forward functionality via execution of the monitoring client 616. To implement local command and control functionality, the base station 612 can perform various programmed operations via execution of the monitoring client 616 in response to various events. Examples of such events include receiving a command from the keypad 608, receiving a command from one of the monitoring application(s) 632 or the client application 634 via the network(s) 620, and detecting occurrence of a scheduled event. Programmed operations performed by the base station 612 via execution of the monitoring client 616 in response to events can include, for example, activation or deactivation of one or more of the devices 604A, 604B, 606, 608, and 610; sounding an alarm; reporting an event to the monitoring service 630; and / or transmitting "location data" to one or more of the transmission service(s) 628. Such location data can include, for example, data specifying sensor readings (sensor data), image data acquired by one or more cameras 604, configuration data for one or more devices arranged at the monitored location 602, commands input from and received from a user (e.g., via the keypad 608 or the client application 634), or data derived from one or more of the foregoing data types (e.g., filtered sensor data, filtered image data, aggregations of sensor data, event data specifying events detected at the monitored location 602 via sensor data, etc.).

[0049] In some embodiments, to implement the store-and-forward functionality, the base station 612 can receive sensor data by monitoring the execution of the client 616, package the data for transmission, and store the packaged sensor data in local storage for later transfer. Such communication of packaged sensor data can include, for example, transmitting the packaged sensor data as a payload of a message to one or more of the transmission service(s) 628 when a communication link to the transmission service(s) 628 via the network(s) 620 is operable. In some embodiments, such packaging of sensor data can include filtering the sensor data using one or more filter zones and / or generating one or more aggregates (maximum, average, change in value since a previous transfer value, etc.) of a plurality of sensor readings.

[0050] The transmission service(s) 628 of the monitoring center environment 626 can be configured to receive messages from the monitored locations (e.g., the monitored location 602), parse the messages to extract the payload included therein, and store the payload and / or data derived from the payload in one or more data stores hosted in the monitoring center environment 626. Examples of such data stores are described below in connection with Figure 10 In some embodiments, the transmission service(s) 628 can expose and implement one or more application programming interfaces (APIs) that are configured to receive, process, and respond to calls from base stations (e.g., the base station 612) via the network(s) 620. Various instances of the transmission service(s) 628 can be associated with certain manufacturers and / or models of location-based monitoring devices (e.g., SIMPLISAFE devices, RING devices, etc.) and specific to these manufacturers and / or models.

[0051] The API(s) of the transport service(s) 628 can be implemented using a variety of architectural styles and interoperability standards. For example, in certain implementations, one or more such APIs can include a web service interface implemented using a representational state transfer (REST) architectural style. In such implementations, API calls can be encoded using the hypertext transfer protocol (HTTP) along with JavaScript Object Notation (JSON) and / or extensible markup language. Such API calls can be addressed to one or more uniform resource locators (URLs) corresponding to API endpoints monitored by the transport service(s) 628. In some implementations, portions of the HTTP communications can be encrypted to increase security. Alternatively (or additionally), in some implementations, one or more APIs of the transport service(s) 628 can be implemented as.NET web APIs that are responsive to HTTP data posts to particular URLs. Alternatively (or additionally), in some implementations, one or more APIs of the transport service(s) 628 can be implemented using simple file transfer protocol commands. As such, the API(s) of the transport service(s) 628 are not limited to any particular implementation.

[0052] The monitoring service 630 in the monitoring center environment 626 can be configured to control the overall logical settings and operation of the security system 600. As such, the monitoring service 630 can communicate and interoperate with the transport service(s) 628, the monitoring application(s) 632, the customer application(s) 634, and various devices arranged at the monitoring location 602 via the network(s) 620. In some implementations, the monitoring service 630 can be configured to monitor data from various sources for events (e.g., break-in events) and, when an event is detected, notify one or more of the monitoring application(s) 632 and / or the customer application(s) 634 of the event.

[0053] In some embodiments, the monitoring service 630 can additionally be configured to maintain state information regarding the monitored location 602. Such state information can indicate, for example, whether the monitored location 602 is secure or under threat. In some embodiments, the monitoring service 630 can be configured to change the state information to indicate that the monitored location 602 is secure only upon receiving a communication indicating a clear event (e.g., rather than making such a change merely due to no additional events being detected). This feature can prevent a “crash and smash” theft (e.g., where an intruder quickly destroys or disables the monitoring equipment) from being successfully executed. Additionally, in some embodiments, the monitoring service 630 can be configured to monitor one or more specific areas within the monitored location 602, such as one or more specific rooms or other distinct areas within and / or around the monitored location 602 and / or one or more defined areas within the FOV of respective image capture devices (e.g., cameras 604A and 604B) deployed in the monitored location. Figure 6

[0054] The individual monitoring application(s) 632 of the monitoring center environment 622 can be configured to enable a monitoring personnel to interact with the respective computing device to provide monitoring services for a respective location (e.g., the monitored location 602) and perform various programmed operations in response to such interactions. For example, in some embodiments, the monitoring application 632 can control its host computing device to provide information to a personnel operating the computing device regarding events detected at a monitored location, such as the monitored location 602. Such events can include, for example, detected movement within a specific area of the monitored location 602. As described above in connection with FIGS. 1-3, in some embodiments, the monitoring application 632 can cause the monitoring device 1016 to present a video clip of the event within the respective event window 106 of the screen 102, and can also establish a streaming connection with one or more cameras 604 at the monitored location and cause the monitoring device 1016 to provide streaming video from such camera(s) 604 within the video feed window 112 and / or the main viewer window 114 of the screen 110, as well as allow for audio communication between the monitoring device 1016 and the camera(s) 604. Figure 1 Figure 2

[0055] ​​​The client application(s) 634 of the client device(s) 624 can be configured to enable clients to interact with their computing devices (e.g., their smartphones or personal computers) to access various services provided by the security system 600 for their individual homes or other locations (e.g., the monitored location 602) and to perform various programmed operations in response to such interactions. For example, in some implementations, the client application 634 can control the client device 624 (e.g., a smartphone or personal computer) to provide information to a client operating the client device 624 regarding events detected at a monitored location, such as the monitored location 602. Such events can include, for example, movement detected within a particular area of the monitored location 602. In some implementations, the client application 634 can additionally or alternatively be configured to process input received from a client to enable or disable one or more devices arranged within the monitored location 602. Further, as described above in connection with Figure 3 the monitored location, the client application 634 can additionally or alternatively be configured to establish a streaming connection with one or more cameras 604 at the monitored location and cause the client device 624 to display streaming video from such camera(s) 604 and to allow audio communication between the client device 624 and the camera(s) 604.

[0056] Turning now to Figure 7 , an example base station 612 is schematically illustrated. As Figure 7 indicated, the base station 612 can include at least one processor 702, volatile memory 704, non-volatile memory 708, at least one network interface 706, a user interface 714, a battery component 716, and an interconnection mechanism 718. The non-volatile memory 708 can store executable code 710 and, as illustrated, can also include a data store 712. In some implementations, the above-listed features of the base station 612 can be incorporated within a housing 720 or can otherwise be supported thereby. In some implementations, the user interface 714 of the base station 612 can include only one or more speakers that provide audio output to a user regarding changes in the operating status of the security system 600, detected threats, etc., and / or one or more visual indicators (e.g., light emitting diode (LED) indicators) that indicate when the base station 612 is operable in response to user input (e.g., via the keypad 608), etc. In other implementations, the user interface can additionally or alternatively include more sophisticated output components (e.g., a display screen) and / or can include one or more user input components, such as one or more microphones (e.g., to receive voice commands) and / or a keypad (e.g., to receive tactile input).

[0057] In some embodiments, the non-volatile (non-transitory) memory 708 can include one or more read-only memory (ROM) chips; one or more hard disk drives or other magnetic or optical storage media; one or more solid state drives (SSDs), such as flash drives or other solid state storage media; and / or one or more hybrid magnetic and SSDs. In some embodiments, the code 710 stored in the non-volatile memory can include an operating system and one or more applications or programs configured to execute under control of the operating system. In some embodiments, the code 710 can additionally or alternatively include special-purpose firmware and embedded software that is executable without reliance on a commercially-available operating system. In any case, execution of the code 710 can implement the functionality of the base station 612 regardless of how the code 710 is specifically embodied. Figure 6 The monitoring client 616 is illustrated and enabled to store and manipulate data of the monitoring client 616 within the data store 712.

[0058] The processor 702 of the base station 612 can include one or more processors configured to execute instructions (such as a computer program embodied by the code 710) encoded in a computer readable medium to control the operation of the base station 612. As used herein, the term “processor” describes circuitry that performs a function, an operation, or a sequence of operations. The function, operation, or sequence of operations can be hard coded into the circuitry or soft coded by way of instructions held in a memory device (e.g., the volatile memory 704) and executed by the circuitry. In some embodiments, the processor 702 can be embodied by one or more application specific integrated circuits (ASICs), microprocessors, digital signal processors (DSPs), graphics processing units (GPUs), neural processing units (NPUs), microcontrollers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and / or multi-core processors.

[0059] Prior to execution of the code 710, the processor 702 can copy at least a portion of the code 710 from the non-volatile memory 708 to the volatile memory 704. In some embodiments, the volatile memory 704 can include one or more static or dynamic random access memory (RAM) chips and / or cache memory (e.g., memory disposed on a silicon die of the processor 702). The volatile memory 704 can provide faster response times than the primary memory, such as the non-volatile memory 708.

[0060] By executing code 710, processor 702 can control the operation of network interface 706. For example, in some embodiments, network interface 706 can include one or more physical interfaces (e.g., radio transceivers, Ethernet ports, Universal Serial Bus (USB) ports, etc.) as well as a software stack including drivers and / or other code 710 configured to communicate with the one or more physical interfaces to support one or more LAN, PAN, and / or WAN standard communication protocols. Such communication protocols can include, for example, Transmission Control Protocol (TCP) and User Datagram Protocol (UDP), among others. As such, network interface 706 can enable base station 612 to access and communicate with other computing devices (e.g., other devices arranged in monitoring location 602) via a computer network (e.g., a LAN established by routers 614 of Figure 6 Figure 6 Figure 6 For example, in some embodiments, network interface 706 can utilize sub- GHz wireless networking to send wake-up messages to other computing devices to request sensor data streams.

[0061] By executing code 710, processor 702 can additionally control the operation of hardware and software stacks including drivers and / or other code 710 configured to communicate with other system devices. As such, base station 612 can interact with other system components in response to received input. Such input can specify, for example, values to be stored in data storage 712. Base station 612 can also provide output representative of values stored in data storage 712. In some embodiments, base station 612 can additionally include one or more light-emitting diodes (LEDs) or other visual indicators that visually convey information, such as system status or alarm events. Further, in some embodiments, base station 612 can additionally or alternatively include a siren (e.g., a 95 decibel (dB) siren) or other audio output device that can be controlled by processor 702 to output an audio indication that an intrusion event has been detected.

[0062] ​​The various components of the base station 612 can communicate with one another via an interconnection mechanism 718. In some implementations, the interconnection mechanism 718 can include a communication bus. Further, in some implementations, a battery assembly 716 can be configured to supply operating power to the various features of the base station 612. In some implementations, the battery assembly 716 can include at least one rechargeable battery (e.g., one or more nickel-metal hydride (NiMH) or lithium batteries). In some implementations, such a rechargeable battery (or rechargeable batteries) can have a runtime capacity sufficient to operate the base station 612 for twenty-four hours or more when the base station 612 is disconnected from or otherwise does not receive line power. In some implementations, the battery assembly 716 can additionally or alternatively include a power supply circuit that receives, conditions, and distributes line power to operate the base station 612 and / or to recharge the one or more rechargeable batteries. Such a power supply circuit can include, for example, transformers and rectifiers, among other circuitry, that convert AC line power to DC device and / or recharging power.

[0063] Turning now to Figure 8 , an example keypad 608 is schematically illustrated. As Figure 8 indicated, the keypad 608 can include at least one processor 802, volatile memory 804, non-volatile memory 808, at least one network interface 806, a user interface 814, a battery assembly 816, and an interconnection mechanism 818. The non-volatile memory 808 can store executable code 810, and as illustrated, can also include a data store 812. In some implementations, the features of the keypad 608 listed above can be incorporated within or otherwise supported by a housing 820.

[0064] In some implementations, the respective descriptions of the processor 702, the volatile memory 704, the non-volatile memory 708, the interconnection mechanism 718, and the battery assembly 716 with reference to the base station 612 apply to the respective descriptions of the processor 802, the volatile memory 804, the non-volatile memory 808, the interconnection mechanism 818, and the battery assembly 816 with reference to the keypad 608. Accordingly, these descriptions will not be repeated here.

[0065] By executing code 810, processor 802 of keypad 608 can control operation of network interface 806. In some implementations, network interface 806 can include one or more physical interfaces (e.g., radio transceivers, Ethernet ports, USB ports, etc.) and a software stack including drivers and / or other code 810 configured to communicate with the one or more physical interfaces to support one or more LAN, PAN, and / or WAN standard communication protocols. Such communication protocols can include, for example, TCP and UDP, among others. As such, network interface 806 can enable keypad 608 to access and communicate with other computing devices (e.g., other devices arranged in monitoring location 602) via a computer network (e.g., a LAN established by router 614). Figure 6

[0066] By executing code 810, processor 802 can additionally control operation of user interface 814. In some implementations, user interface 814 can include user input and / or output devices (e.g., physical keys arranged as a keypad, a touchscreen, a display, a speaker, a camera, a biometric scanner, an environmental sensor, etc.) and a software stack including drivers and / or other code 810 configured to communicate with the user input and / or output devices. As such, user interface 814 can enable keypad 608 to interact with a user to receive input and / or present output. Examples of output that can be presented by user interface 814 include one or more GUIs including one or more controls configured to display output and / or receive input. Input received by user interface 814 can specify, for example, values to be stored in data store 812. Output provided by user interface 814 can also indicate values stored in data store 812. In some implementations, portions of user interface 814 (e.g., one or more LEDs) can be accessible and / or visible as part of or through housing 820.

[0067] Turning now to Figure 9 , an example sensor assembly 924 is schematically illustrated. Several example implementations of sensor assembly 924 (e.g., cameras 604 and 604B, motion sensor assembly 610, and contact sensor assembly 606) are illustrated in Figure 6 and described above. As Figure 9 ​As shown, the sensor assembly 924 can include the at least one processor 902, the volatile memory 904, the non-volatile memory 908, the at least one network interface 906, the battery assembly 916, the interconnection mechanism 918, and the at least one sensor 922. The non-volatile memory 908 can store the executable code 910, and as illustrated, can also include a data store 912. In some embodiments, the above-listed features of the sensor assembly 924 can be incorporated within or included as part of the housing 920. Further, in some embodiments, the sensor assembly 924 can additionally include a user interface 914.

[0068] In some embodiments, the respective descriptions of the processor 702, the volatile memory 704, the non-volatile memory 708, the interconnection mechanism 718, and the battery assembly 716 with reference to the base station 612 apply to the respective descriptions of the processor 902, the volatile memory 904, the non-volatile memory 908, the interconnection mechanism 918, and the battery assembly 916 with reference to the sensor assembly 924. Accordingly, these descriptions will not be repeated here.

[0069] By executing the code 910, the processor 902 can control the operation of the network interface 906 and the user interface 914 (if present). In some embodiments, the network interface 906 can include one or more physical interfaces (e.g., radio transceivers, Ethernet ports, USB ports, etc.) as well as a software stack including drivers and / or other code 910 configured to communicate with the one or more physical interfaces to support one or more LAN, PAN, and / or WAN standard communication protocols. Such communication protocols can include, for example, TCP and UDP, among others. As such, the network interface 906 can enable the sensor assembly 924 to access and communicate with other computing devices (e.g., other devices arranged in the monitoring location 602 by the router 614) via a computer network (e.g., the LAN established by the router 614). Figure 6 For example, in some embodiments, when executing the code 910, the processor 902 can control the network interface to stream sensor data acquired from the sensor assembly 922 (e.g., via UDP) to the base station 612. Further, in some embodiments, by executing the code 910, the processor 902 can additionally or alternatively control the network interface 906 to enter a power saving mode, for example, by powering down the 2.4 GHz radio transceiver and powering up the sub- GHz radio transceiver both included in the network interface 906. In such embodiments, by executing the code 910, the processor 902 can additionally control the network interface 906 to enter a streaming mode, for example, by powering up the 2.4 GHz radio transceiver and powering down the sub- GHz radio transceiver, for example, in response to receiving a wake-up signal from the base station via the sub- GHz radio transceiver.

[0070] By executing code 910, processor 902 can additionally or alternatively control other operations of sensor assembly 924. In some embodiments, for example, user interface 914 of sensor assembly 924 may include user input and / or output devices (e.g., physical buttons, touchscreens, displays, speakers, cameras, accelerometers, biometric scanners, environmental sensors, one or more LEDs, etc.) and a software stack including drivers and / or other code 910 configured to communicate with user input and / or output devices. Thus, sensor assembly 924 can enable user interface 914 to interact with a user to receive input and / or present output. Output presented by user interface 914 may include, for example, one or more GUIs including one or more controls configured to display output and / or receive input. Input received by user interface 914 may, for example, specify a value to be stored in data storage 912. Output provided by user interface 914 may also indicate a value stored in data storage 912. In some embodiments, portions of sensor assembly 924 may be part of housing 920 or accessible and / or visible through it.

[0071] like Figure 9 As shown, sensor assembly 924 may include one or more types of sensors 922, such as those referenced above. Figure 6 The sensors described include one or more of the cameras 604 and 604B, motion sensor assembly 610, and contact sensor assembly 606, or other types of sensors. In some embodiments, for example, sensor(s)922 may include a camera and a temperature sensor. Regardless of the type of sensor(s)922(s)922(s)922(s) used, processor 902 may (e.g., via execution of code 910) acquire sensor data from sensor(s)922(s)922(s)922 and stream the acquired sensor data to processor 902 for transmission to base station 612.

[0072] It should be noted that in some embodiments of devices 802 and 902, operations performed by processors 802 and 902 under the control of the respective controls of codes 810 and 910 may be hard-coded and / or implemented using hardware rather than as a combination of hardware and software.

[0073] Turn now Figure 10 This is an illustrative example. Figure 6 The aspects shown include monitoring center environment 626, monitoring center environment 622, one of the client devices 624, one or more networks 620, and multiple monitoring locations 602A to 602N (collectively referred to as monitoring locations 602). Figure 10As shown, in some embodiments, the monitoring service 630 can include a location data store 1002, an image data store 1004, an artificial intelligence (AI) service 1008, an event listener service 1010, an identity provider service 1012, a customer service 1038, a surveillance service 1040, and a camera streaming service 1042. Also as shown, Figure 10 As shown, the surveillance center environment 622 can include a plurality of surveillance devices 1016A-1016M (collectively, surveillance devices 1016) that host or are otherwise configured to access respective surveillance applications 632A-632M, and each of the surveillance locations 602A-602N can include respective monitoring clients 616A-616N (collectively, monitoring clients 616), e.g., at each of the surveillance locations 602A-602N within the base station 612 Figure 10 (not shown in FIG. 6B). As described above in connection with Figure 1 and Figure 2 As shown, in some embodiments, the surveillance applications 632 can be configured to cause the surveillance devices 1016 to display the screens 102, 110 that enable the surveillance agents 104 to visually monitor activity at one or more of the surveillance locations 602, as well as to engage in audio conversations with one or more individuals at such locations (e.g., via microphones and speakers of the cameras 604 at the surveillance locations 602). Further, as additionally shown in Figure 10 As shown, in some embodiments, the transmission service(s) 628 can include a plurality of different transmission services 628A-628D that are configured to receive location data packets, e.g., location data packets 1014A-1014D, from the monitoring clients 616A-616N that are deployed at respective ones of the surveillance locations 602A-602N.

[0074] The location data store 1002 of the monitoring service 630 can be configured to store location data in a plurality of records in association with an identifier of a customer that is being monitored at the surveillance location 602. For example, the location data can be stored in a record with an identifier of the customer and / or an identifier of the surveillance location 602 to associate the location data with the customer and the surveillance location 602. The image data store 1004 of the monitoring service 630 can be configured to store one or more frames of image data in a plurality of records in association with an identifier of a location and a timestamp at which the image data was acquired.

[0075] The AI ​​service 1008 of the surveillance service 630 can be configured to process images and / or image sequences to identify semantic regions, movement, faces, and other features within the images or image sequences. The event listening service 1010 of the surveillance service 630 can be configured to scan received location data to look for events and, upon identifying an event, execute one or more event handlers to handle that event. In some embodiments, such event handlers can be configured to identify events and transmit messages about those events to one or more receiving services (e.g., customer service 1038 and / or monitoring service 1040). The operations that customer service 1038 and / or monitoring service 1040 can perform based on events identified by the event listening service 1010 are further described below. In some embodiments, the event listening service 1010 can interoperate with the AI ​​service 1008 to identify events within image data.

[0076] Identity provider service 1012 can be configured to receive an authentication request, including a security certificate, from monitoring client 616. When identity provider 1012 can authenticate the security certificate in the request (e.g., via an authentication function, cross-reference lookup, or some other authentication process), identity provider 1012 can respond to the request by transmitting a security token. Monitoring client 616 can receive, store, and include the security token in subsequent location data (e.g., location data 1014A) packets, enabling receiving transport services (e.g., transport service 628A) to securely process (e.g., unpack / parse) the packets to retrieve the location data before passing it to monitoring service 630. In some implementations, the security token may, for example, be a JSON network token (JWT), such as those described below. Figure 18 The token described is 1802.

[0077] One or more transmission services 628 of the monitoring center environment 626 can be configured to receive location data packets 1014, verify the authenticity of packets 1014, parse packets 1014, and extract the location data encoded therein before passing location data to the monitoring service 630 for processing. The location data thus processed may include the information referenced above. Figure 6 Any location data type described. In some implementations, each transmission service 628 may be configured to process location data packets 1014 generated by location-based monitoring devices of a particular manufacturer and / or model. Monitoring client 616 may be configured to generate location data packets (e.g., location data packet 1014) based on sensor information received at monitoring location 602 and transmit them, for example, via network(s) 620 to monitoring service 630.

[0078] Monitoring service 1040 can maintain a record of events identified by event listening service 1010 and can assign each event to a monitoring agent 104 currently online using monitoring application 632. Monitoring application 632, operated by a given monitoring agent, can then add events assigned to that monitoring agent 104 to, for example... Figure 1 The event queue within the event window 106 is shown for review by the monitoring agent 104. In some implementations, a given monitoring application 632 may use data describing events within its queue to retrieve location data and / or image data (from location data storage 1002 and / or image data storage 1004, respectively) for presentation or associated presentation within or in connection with the event window 106.

[0079] In response to monitoring agent 104 identifying a specific event to be viewed (e.g., by clicking an event window 106), monitoring service 1040 can interact with camera streaming service 1042 to obtain access credentials, thereby enabling the establishment of a peering connection with one or more cameras 604 at the monitoring location 602 corresponding to that event, and to view, for example, in Figure 2 The live video and / or audio streaming from these cameras, as well as real-time verbal communication with one or more individuals near camera(s) 604, are shown within the video feed window 112 and / or main viewer window 114. (The following is in conjunction with...) Figure 12 This describes an example interaction between components of security system 600, which enables streaming of video and / or audio data between one or more cameras 604 at monitoring location 602 and a monitoring application 632 operated by monitoring agent 104.

[0080] Turn now Figure 11 An example monitoring process 1100, which can be adopted by security system 600, is illustrated as a sequence diagram. Specifically, in some embodiments, various parts of process 1100 may be controlled by (A) at least one processor (e.g., Figure 8 or Figure 9 One or more location-based devices (e.g., [missing information]) are controlled by a device control system (DCS) code (e.g., code 810 or 910) implemented by either processor 802 or 902. Figure 6 (a) Devices 604 to 610); (b) On the monitoring client (e.g., Figure 6 The base station (e.g., under the control of the monitoring client 616) Figure 6 (C) In monitoring applications (e.g., base station 612); Figure 6 The monitoring center environment under the control of the monitoring application 632) (e.g., Figure 6 (d) In the monitoring center environment 622); (e.g., in the monitoring service, Figure 6The monitoring center environment (e.g., under the control of the monitoring service 630) Figure 6 (e) the monitoring center environment 626); and (E) in customer applications (e.g., Figure 6 Client equipment under the control of client application 634) (e.g., Figure 6 (Execute on customer equipment 624).

[0081] like Figure 11 As shown, process 1100 may begin with monitoring client 616 authenticating to monitoring service 630 by exchanging one or more authentication requests and responses 1104 with monitoring service 630. More specifically, in some embodiments, monitoring client 616 may transmit the authentication request to monitoring service 630 via one or more API calls to monitoring service 630. In such an embodiment, monitoring service 630 may parse the authentication request to extract a security certificate from it and pass such a security certificate to an identity provider (e.g., Figure 10 The monitoring service 630 uses the identity provider service 1012 for authentication. In some implementations, when the identity provider authenticates a security certificate, the monitoring service 630 may generate a security token and convey that security token as a payload within the authentication response to the authentication request. In such an implementation, if the identity provider cannot authenticate the security certificate, the monitoring service 630 may instead generate an error (e.g., an error code) and transmit that error as a payload within the authentication response to the authentication request. Upon receiving the authentication response, the monitoring client 616 may parse the authentication response to extract the payload. If the payload includes an error code, the monitoring client 616 may retry authentication and / or its user interface with its host device (e.g., ...). Figure 7 The monitoring client 616 interoperates with the user interface 714 of the base station 612 to present an output indicating authentication failure. If the payload includes a security token, the monitoring client 616 may store the security token for subsequent use in the transmission of location data. It should be noted that in some embodiments, the security token may have a limited lifetime (e.g., one hour, one day, one week, one month, etc.), after which the monitoring client 616 may be required to re-authenticate with the monitoring service 630. In some embodiments, for example, the security token (e.g., in conjunction with the following...) Figure 18 The lifetime of the token (of the type described, 1802) can be defined in the header 1804 and / or payload 1806 of the token 1802 (e.g., as one or more statements).

[0082] Continuing process 1100, one or more device control systems 1102, hosted by one or more location-based devices, can acquire (1106) a description of the location (e.g., Figure 6sensor data of the monitored location 602). The sensor data thus acquired can be any of a variety of types, as discussed above with reference to Figures 6 to 10 In some implementations, the one or more device control systems 1102 can continuously acquire sensor data. In other implementations, the one or more DCS 1102 can additionally or alternatively acquire sensor data in response to an event, such as a timer expiring (a push event) or receiving an acquisition poll signal transmitted by the monitoring client 616 (a pull event). In some implementations, the one or more device control systems 1102 can stream sensor data to the monitoring client 616 with minimal processing beyond acquisition and digitization. In such implementations, the sensor data can constitute a sequence of vectors with individual vector members including, for example, sensor readings and timestamps. In some implementations, the one or more device control systems 1102 can perform additional processing of the sensor data, such as generating one or more aggregations of multiple sensor readings. Still further, in some implementations, the one or more device control systems 1102 can perform complex processing of the sensor data. For example, if the sensor component 924 (one or more) sensors 922 of which are shown in FIG. 9 include an image capture device, the device control system 1102 can perform image processing routines, such as edge detection, motion detection, facial recognition, threat assessment, event generation, and the like. Figure 9

[0083] Continuing the process 1100, the device control component(s) 1102 can transmit the sensor data 1108 to the monitoring client 616. As with sensor data acquisition, the device control system(s) 1102 can transmit the sensor data 1108 continuously or in response to an event, such as a push event (originating from the device control system(s) 1102) or a pull event (originating from the monitoring client 616).

[0084] ​Continuing with process 1100, monitoring client 616 can monitor (1110) monitored location 602 by processing received sensor data 1108. In some embodiments, for example, monitoring client 616 can execute one or more image processing routines. Such image processing routines can include any of the image processing routines described above with reference to operation 1106. By distributing at least some image processing routines between device control system(s) 1102 and monitoring client 616, the amount of power consumed by battery-powered devices can be reduced by offloading processing from line-powered devices. Also, in some embodiments, monitoring client 616 can execute an overall threat detection process that utilizes sensor data 1108 from multiple different device control systems 1102 as input. For example, in some embodiments, monitoring client 616 can attempt to corroborate an open state received from a contact sensor with motion and facial recognition processing on images of the scene that include the window or door to which the contact sensor is affixed. If two or more of the three processes indicate the presence of an intruder, a score (e.g., a threat score) can be increased and / or a break-in event can be declared, locally logged, and communicated. Other processing that monitoring client 616 can perform includes outputting local alerts (e.g., in response to detecting particular events and / or satisfying other criteria) and detecting maintenance conditions for location-based devices, such as the need to replace or recharge low power batteries and / or replace / maintain devices hosting device control system(s) 1102. Any of the above processes within operation 1110 can result in the creation of location data specifying the results of these processes.

[0085] Continuing with process 1100, monitoring client 616 can communicate location data 1112 to monitoring service 630 (via transport service(s) 628). As with the communication of sensor data 1108, monitoring client 616 can communicate location data 1112 continuously or in response to events, such as push events (originating from monitoring client 616) or polling events (originating from monitoring service 630).

[0086] Continuing process 1100, monitoring service 630 may process (1114) the received location data. In some embodiments, for example, monitoring service 630 may perform one or more of the processes described above with reference to operations 1106 and / or 1110. In some embodiments, monitoring service 630 may additionally or alternatively calculate a score (e.g., a threat score) or further refine an existing score using historical information associated with the monitored location 602 identified in the location data and / or other locations geographically close to monitored location 602 (e.g., within the same Zone Improvement Program (ZIP) code). For example, in some embodiments, if multiple intrusions have already been recorded for monitored location 602 and / or other locations within the same ZIP code, monitoring service 630 may increment the score calculated by device control system 1102 and / or monitoring client 616.

[0087] In some implementations, monitoring service 630 may apply a set of rules and criteria to location data 1112 to determine whether location data 1112 includes any events, and if so, to transmit event reports 1116A and / or 1116B to monitoring application 632 and / or client application 634. In some implementations, for example, monitoring service 1040 may assign one or more events to a specific monitoring agent 104, such that these events will be forwarded to the monitoring agent 104 in the operating monitoring application 632, for example, in the corresponding event window 106 (…). Figure 1 (As shown in the image) The event can be, for example, a specific type of event (e.g., intrusion) or a specific type of event that meets additional criteria (e.g., movement within a specific area combined with a threat score exceeding a threshold). Event reports 1116A and / or 1116B can have priorities based on the same criteria used to determine whether an event reported therein is reportable, or they can have priorities based on different sets of criteria or rules.

[0088] Continuing with process 1100, the monitoring application 632 in the monitoring center environment 622 can, for example, use one or more GUIs (such as...) Figure 1 and Figure 2 The screens shown (102 and 110) interact with the monitoring agent 104 (1118). Such a GUI can provide details and context about one or more events.

[0089] like Figure 11 As shown, client application 634 of client device 624 (e.g., smartphone, personal computer, or other endpoint device) can also interact with at least one client via, for example, one or more GUIs (1120). Such a GUI can provide details and context about one or more events.

[0090] It should be noted that the processing of sensor data and / or location data as described above with reference to operations 1106, 1110, and 1114 can be performed by processors arranged within various parts of the security system 600. In some embodiments, the device control system(s) 1102 can perform minimal processing of the sensor data (e.g., only acquisition and streaming), and the remainder of the processing can be performed by the monitoring client 616 and / or the monitoring service 630. This approach can help extend the battery life of location-based devices. In other embodiments, the device control system(s) 1102 can perform as much sensor data processing as possible, such that the monitoring client 616 and the monitoring service 630 only perform the processes required to cross the location-based device and / or location sensor data. This approach can help improve the scalability of the security system 600 when adding new locations.

[0091] Figure 12 and Figure 13 Example techniques are illustrated for establishing a peer-to-peer connection (e.g., for video and / or audio streaming) between a camera 604 at monitoring location 602 and either (A) a monitoring application 632 hosted on or otherwise accessible by monitoring device 1016 and a client application 634 hosted on or otherwise accessible by client device 624. In some embodiments, monitoring application 632 and client application 634 may be web applications accessed using browsers hosted on monitoring device 1016 and client device 624, respectively, and the WebRTC functionality of these browsers may be used to establish a peer-to-peer connection with camera 604. As described below... Figure 13 The camera streaming service 1042 can provide a signaling channel for establishing a peer-to-peer connection between the camera 604 and the corresponding browser. As an example, the camera streaming service 1042 can be implemented using the Amazon Kinesis video streaming service provided by Amazon Web Services (AWS).

[0092] like Figure 12 As indicated by arrow 1202, monitoring application 632 can provide a user token to monitoring service 1040. The user token may correspond to a monitoring agent 104 that has been authenticated to monitoring application 632 and may be included in a request for live streaming access to one or more cameras 604 at monitoring location 602. In some implementations, for example, in response to monitoring agent 104 selecting an event window 106 corresponding to a specific camera 604, such a camera access request can be sent from monitoring application 632 to monitoring service 1040, as described above. Figure 1 and Figure 2The user token can be a JWT in some implementations, such as the token 1802 described below in connection with Figure 18 The token 1802 described below.

[0093] The monitoring service 1040 can evaluate the user token received from the monitoring application 632 (e.g., by verifying the signature 1808 of the token, as described below in connection with Figure 18 The monitoring service 1040 can evaluate the user token received from the monitoring application 632 (e.g., by verifying the signature 1808 of the token, as described below in connection with Figure 13 An example process by which signaling information can be exchanged between the monitoring application 632 and the camera 604 via a signaling channel established by the camera streaming service 1042 to determine configuration information for a peer-to-peer connection between the monitoring application 632 and the camera 604 is described below in connection with Figure 18 The token 1802 described below.

[0094] As indicated by the arrow 1204 in Figure 12 In some implementations, the monitoring service 1040 can authenticate to the camera streaming service 1042 on behalf of the monitoring application 632 that provided the user token, and request access to the camera streaming service 1042 (per arrow 1202), as indicated. In some implementations, the access request sent by the monitoring service 1040 to the camera streaming service 1042 can specify one or more parameters that identify the particular monitoring location 602 in question, the particular camera(s) 604 to which access is to be granted, a particular time window during which access to such camera(s) 604 is to be granted, and / or any of a number of other limitations or constraints on whether and / or how access to the camera(s) 604 is to be allowed. The use of these parameters can help ensure that the camera(s) 604 are accessed only by authorized personnel and only when needed to evaluate a particular event.

[0095] In authenticating the access request received from the monitoring service 1040, the camera streaming service 1042 can establish a signaling channel between the monitoring application 632 and the camera 604, and generate an access token (e.g., the token 1802 described below in connection with Figure 18The monitoring application 632 can then use the access token (type 1802 described) to access the signaling channel (e.g., via a network API call to the API endpoint of the camera streaming service 1042). In some implementations, the monitoring service 1040 can configure the access token to include one or more parameters specified in the access request. For example, in some implementations, such parameters can be defined (e.g., as one or more statements) in the header 1804 and / or payload 1806 of the access token.

[0096] like Figure 12 As indicated by arrows 1206 and 1208, camera streaming service 1042 can send the generated access token to monitoring service 1040, which can then pass the access token to monitoring application 632. In some embodiments, camera streaming service 1042 may also send additional information along with the access token, such as the network address (e.g., network API endpoint) of the signaling channel established by camera streaming service 1042, thereby allowing monitoring application 632 to make network API calls to camera streaming service 1042 for signaling purposes. As previously described, the access token generated by camera streaming service 1042 can be configured based on parameters included in the access request sent by monitoring service 1040 to camera streaming service 1042 to restrict the ability of receiving monitoring application 632 to access the established signaling channel in a manner defined by those parameters. For example, the access token generated by camera streaming service 1042 can be set to expire after a specific time period based on a time limit parameter included in the access request.

[0097] As shown below Figure 13 As described above, upon receiving an access token from the monitoring service 1040, the monitoring application 632 can send a Session Description Protocol (SDP) proposal to the network address of the signaling channel, and the signaling channel can forward the SDP proposal to the camera 604, thereby initiating a signaling process to establish a peer-to-peer connection between the monitoring application 632 and the identified camera 604. Finally, also in conjunction with the following... Figure 13 As described above, Figure 12 As indicated by arrows 1210 and 1212, when a suitable interactive connection establishment (ICE) candidate is identified, one or more peer connections can be established between the monitoring application 632 and the camera 604, thereby enabling the streaming of video data from the camera 604 to the monitoring application 632 and / or the exchange of audio data between the monitoring application 632 and the camera 604.

[0098] A similar process can be employed to establish one or more peer-to-peer connections between the client application 634 and one or more cameras 604 at the monitored location, thereby enabling video data to be streamed from the camera(s) 604 to the client application 634 and / or audio data to be exchanged between the client application 634 and the cameras 604. Accordingly, this process is not described again herein. However, it should be appreciated that the scope of permissions provided in the access request sent from the client service 1038 to the camera streaming service 1042 can be different (e.g., less restrictive) than the scope of permissions provided by the access request sent from the monitoring service 1040 to the camera streaming service 1042, as it can not be desirable to restrict the ability of a client to live stream with a camera in the same manner as a monitoring agent 104.

[0099] Figure 13 is a sequence diagram 1300 that illustrates how signaling information (e.g., WebRTC signaling information) can be exchanged between the monitoring application 632 (or, alternatively, the client application 634) and a camera 604 via the camera streaming service 1042 to establish a peer-to-peer connection between the monitoring application 632 (or, alternatively, the client application 634) and the camera 604. Although Figure 13 The exchange of signaling information between the monitoring application 632 and the camera 604 is depicted, and the exchange of signaling information between these two components is described in the following section, it should be appreciated that the same process can equally be used to exchange signaling information between the client application 634 and the camera 604.

[0100] As noted above, in some implementations, in response to providing a user token (see arrow 1202 in Figure 12 the monitoring application 632 can have received an access token for the camera streaming service 1042 from the monitoring service 1040 (see arrow 1208 in Figure 12 Such an access token can enable the monitoring application 632 to access the signaling channel established by the camera streaming service 1042, thereby allowing the monitoring application 632 to make network API calls to the camera streaming service 1042 for signaling purposes.

[0101] As Figure 13As shown, the signaling process can begin with the monitoring application 632 using the received access token to send (1302A, 1302B) an SDP offer to the camera 604 (via the camera streaming service 1042). The monitoring application 632 can create the SDP offer, for example, by calling the CreateOffer() function of a Web Real-Time Communication (WebRTC) application programming interface (API) of a browser or other WebRTC-enabled component of the monitoring device 1016. The SDP offer can include information about the kind of media to be sent by the monitoring device 1016, its format, the transport protocol used, the Internet Protocol (IP) address and port of the monitoring device 1016, and / or other information describing the media to be transmitted and / or required by the monitoring device 1016.

[0102] Upon receiving the SDP offer from the monitoring application 632, the camera 604 can send (1304A, 1304B) an SDP answer to the monitoring application 632 via the camera streaming service 1042. The camera 604 can create the SDP answer, for example, by calling the CreateAnswer() function of a Web Real-Time Communication (WebRTC) API of a browser or other WebRTC-enabled component of the camera 604. The SDP answer can include information about the kind of media to be sent by the camera 604, its format, the transport protocol used, the Internet Protocol (IP) address and port of the camera 604, and / or other information describing the media to be transmitted and / or required by the camera 604.

[0103] In addition to sharing information about the media to be exchanged and the respective devices that will exchange it, the monitoring application 632 and the camera 604 can share information about the network connections they can use to exchange the media. Specifically, the monitoring application 632 can share one or more ICE candidates with the camera 604, and vice versa, where each ICE candidate is sent by a device describing available methods that the device can use to communicate (either directly or by using a Traversal Using Relays around NAT (TURN) server). The monitoring application 632 and the camera 604 can collect ICE candidates, for example, by creating an ICE candidate event listener using the WebRTC API (e.g., by calling the function peerConnection.addEventListener('icecandidate', event => {...}).

[0104] In some implementations, the respective devices can first offer their best ICE candidate, causing it to move in the direction of its worse candidates. Ideally, ICE candidates employ User Datagram Protocol (UDP) (as it is faster, and media streams can be relatively easily recovered from interruptions), but the ICE standard also allows Transmission Control Protocol (TCP) candidates.

[0105] Possible UDP candidate types include host, peer reflexive (prflx), server reflexive (srflx), and relay. A "host" candidate is one whose IP address is the actual direct IP address of the remote peer. A "peer reflexive" candidate is one whose IP address comes from a symmetric network address translation (NAT) between two peers. A "server reflexive" candidate is generated by a UDP Session Traversal Utilities for NAT (STUN) server. A relay candidate is generated by a TURN server. Possible TCP candidate types include active, passive, and so. An "active" transport will attempt to open an outgoing connection, but will not receive incoming connection requests. A "passive" transport will receive incoming connection attempts, but will not itself attempt a connection. A "so" transport will attempt to open a connection to its peer simultaneously.

[0106] As an example, Figure 13 It is illustrated how the monitoring application 632 can send (1306A, 1306B) ICE candidate "A" to the camera 604 and how the camera 604 can send (1308A, 1308B) ICE candidate "B" to the monitoring application 632. Different pairs of the identified ICE candidates can be tested, and one of the identified ICE candidate pairs can be selected by the endpoint that has been designated as the "controlling agent" to use for establishing (1310) a peer-to-peer connection between the monitoring application 632 and the camera 604.

[0107] Additional information regarding the use of WebRTC to establish a peer-to-peer connection can be found on the web page accessible via the uniform resource locator (URL) "webrtc.org", the entire contents of which are incorporated herein by reference.

[0108] As noted above, it can be advantageous to provide the monitoring agent 104 with exclusive access to one or more cameras 604 at the monitoring location 602 (or at least to exclude the client from access to such camera(s)), as doing so can (A) ensure a high quality connection with the camera(s) 604, and (B) enable the monitoring agent 104 to control the situation without interference from the client 302, such as by engaging in two-way communication (e.g., via a microphone and speaker) with one or more suspicious individuals in the vicinity of the camera(s) 604. Figure 14 and Figure 15 Two example routines 1400 and 1500 are shown that can be performed by the camera 604 to provide the monitoring agent 104 with exclusive access to the camera 604, and to notify the client (via the client application 634) that the monitoring agent 104 has been given such exclusive access (e.g., as described above in connection with Figure 4 and Figure 5Specifically, routine 1400 can be executed by camera 604 in response to receiving a peer-to-peer connection request initiated by monitoring application 632, and routine 1500 can be executed by camera 604 in response to receiving a peer-to-peer connection request initiated by client application 634. In some embodiments, the processor 902 of camera 604 (see...) Figure 9 It can execute instructions specifically implemented by code 910 so that camera 604 performs the operations of routines 1400 and 1500.

[0109] Now will describe Figure 14 The example 1400 is shown. As shown, example 1400 can begin at step 1402, in which camera 604 can receive a request to access camera 604 from monitoring device 1016. In some embodiments, for example, step 1402 can correspond to camera 604 receiving an SDP proposal from the WebRTC signaling channel provided by camera streaming service 1042, wherein such a proposal is initiated by monitoring application 632 using an access certificate provided by camera streaming service 1042, as described above. Figure 12 and Figure 13 As stated above.

[0110] In step 1404 of routine 1400, camera 604 can establish a peer-to-peer connection with monitoring device 1016. For example, as described above... Figure 13 When a suitable ICE candidate is identified, one or more peer connections can be established between the monitoring application 632 and one or more cameras 604, thereby enabling the streaming of video data from one or more cameras 604 to the monitoring application 632 and / or the exchange of audio data between the monitoring application 632 and one or more cameras 604.

[0111] At decision 1406 in routine 1400, camera 604 can determine whether a peer connection also exists between client device 624 and camera 604. In some implementations, for example, camera 604 can use the above-described combination. Figure 12 and Figure 13 The described process determines whether another WebRTC peer connection has also been established between camera 604 and client application 634 accessed by client device 624.

[0112] When camera 604 determines at decision 1406 that no other peer connection exists between client device 624 and camera 604, routine 1400 may terminate. Conversely, when camera 604 determines (at decision 1406) that another peer connection does exist between client device 624 and camera 604, routine 1400 may proceed to step 1408, where camera 604 may use the existing peer connection between camera 604 and client device 624 to send a message to client device 624 (e.g., to client application 634) indicating that the existing peer connection is about to be terminated. In some embodiments, in response to receiving such a message, client application 634 may cause client device 624 to display a message 402 indicating that monitoring agent 104 has taken over camera feeds, for example, as described above. Figure 4 As stated above.

[0113] In step 1410 of routine 1400, camera 604 can disconnect from peering with client device 624, thereby providing monitoring agent 104 with exclusive access to camera 604 via monitoring application 632 on monitoring device 1016 operated by monitoring agent 104.

[0114] Now will describe Figure 15 The example 1500 is shown. As shown, example 1500 can begin at step 1502, in which camera 604 receives an access request from client device 624. In some implementations, for example, step 1502 may correspond to camera 604 receiving an SDP proposal from the WebRTC signaling channel provided by camera streaming service 1042, wherein such a proposal is initiated by client application 634 using an access certificate provided by camera streaming service 1042, as described above. Figure 12 and Figure 13 As stated above.

[0115] In the decision 1504 of routine 1500, camera 604 can determine whether a peer connection already exists between monitoring device 1016 and camera 604. In some implementations, for example, camera 604 may use the above-described combination... Figure 12 and Figure 13 The described process determines whether another WebRTC peer connection has been established between camera 604 and monitoring application 632 accessed by monitoring device 1016.

[0116] When camera 604 determines at decision 1504 that there is no peer connection between monitoring device 1016 and camera 604, routine 1400 can proceed to step 1506, in which camera 604 can establish a peer connection with client device 624. For example, as described above... Figure 13As noted, upon identifying a suitable ICE candidate, one or more peer-to-peer connections can be established between the client application 634 and the camera(s) 604, thereby enabling video data to be streamed from the camera(s) 604 to the client application 634 and / or audio data to be exchanged between the client application 634 and the camera(s) 604.

[0117] When the camera 604 determines (at decision 1504) that a peer-to-peer connection already exists between the monitoring device 1016 and the camera 604, the routine 1500 can instead proceed to step 1508, at which the camera 604 can establish a peer-to-peer connection with the client device 624 to provide a minimal set of channels over which the camera 604 can send one or more messages to the client device 624. For example, as noted above in connection with Figure 12 As noted, upon identifying a suitable interactive connection establishment (ICE) candidate, a minimal WebRTC connection can be established between the client application 634 and the camera(s) 604.

[0118] At step 1510 of the routine 1500, the camera 604 can use the data channel established between the camera 604 and the client device 624 to send a message 502 to the client device 624 (e.g., to the client application 634) indicating that the request to access the camera 604 has been denied. In some implementations, in response to receiving such a message, the client application 634 can cause the client device 624 to display the message 502 indicating that live streaming with the camera 604 cannot be provided due to the monitoring agent 104 currently using the camera 604, e.g., as noted above in connection with Figure 5 As noted.

[0119] At step 1512 of the routine 1500, the camera 604 can tear down the peer-to-peer connection providing the minimal set of data channels with the client device 624.

[0120] Figure 16 is a sequence diagram 1600 illustrating interactions that can occur between the client device 624, the camera 604, and the monitoring device 1016 when the routines 1400 and 1500 are performed by the camera 604, as described below in connection with Figure 14 and 15 respectively. In particular, the top of the sequence diagram 1600 illustrates component interactions in scenarios corresponding to the routine 1400, in which the client device 624 is already connected to the camera 604 when the monitoring device 1016 makes a request to access the camera 604, and the bottom of the sequence diagram 1600 illustrates component interactions in scenarios corresponding to the routine 1500, in which the client device 624 makes a request to access the camera 604 when the monitoring device 1016 is already connected to the camera 604.

[0121] Regarding the top of sequence diagram 1600, operations 1602 and 1604 can correspond to steps 1502 and 1506 of routine 1500, respectively. Operations 1606 and 1608 of sequence diagram 1600 can correspond to steps 1402 and 1404 of routine 1400, respectively. Operations 1610 and 1612 of sequence diagram 1600 can correspond to steps 1408 and 1410 of routine 1400, respectively.

[0122] Regarding the bottom of sequence diagram 1600, operations 1614 and 1616 of sequence diagram 1600 can correspond to steps 1402 and 1404 of routine 1400, respectively. Operations 1618, 1620, and 1622 of sequence diagram 1600 can correspond to steps 1502, 1508, and 1510 of routine 1500, respectively.

[0123] Turn now Figure 17 The diagram illustrates the computing device 1700. For example... Figure 17 As shown, computing device 1700 may include at least one processor 1702, volatile memory 1704, one or more interfaces 1706, non-volatile memory 1708, and interconnect mechanism 1714. Non-volatile memory 1708 may include executable code 1710, and as illustrated, may additionally include at least one data storage 1712.

[0124] In some embodiments, the non-volatile (non-transient) memory 1708 may include one or more read-only memory (ROM) chips; one or more hard disk drives or other magnetic or optical storage media; one or more solid-state drives (SSDs), such as flash drives or other solid-state storage media; and / or one or more hybrid magnetic and SSDs. Further, in some embodiments, code 1710 stored in the non-volatile memory may include an operating system and one or more applications or programs configured to execute under the control of the operating system. In some embodiments, code 1710 may additionally or alternatively include dedicated firmware and embedded software that is executable and independent of a commercially available operating system. Regardless of its configuration, execution of code 1710 may produce manipulated data that can be stored as one or more data structures in data storage 1712. The data structures may have fields associated by location within the data structure. This association can also be achieved by allocating storage for fields in locations associated with transported fields within memory. However, other mechanisms can be used to establish associations between information in fields of a data structure, including by using pointers, tags, or other mechanisms.

[0125] The processor 1702 of the computing device 1700 can be embodied by one or more processors configured to execute one or more executable instructions, such as a computer program specified by the code 1710, to control the operation of the computing device 1700. The functions, operations, or sequences of operations can be hard coded into the circuitry or soft coded by instructions held in a memory device, such as the volatile memory 1704, and executed by circuitry. In some implementations, the processor 1702 can be embodied by one or more application specific integrated circuits (ASICs), microprocessors, digital signal processors (DSPs), graphics processing units (GPUs), neural processing units (NPUs), microcontrollers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), or multi-core processors.

[0126] Prior to executing the code 1710, the processor 1702 can copy the code 1710 from the non-volatile memory 1708 to the volatile memory 1704. In some implementations, the volatile memory 1704 can include one or more static or dynamic random access memory (RAM) chips and / or cache memory (e.g., memory arranged on a silicon die with the processor 1702). The volatile memory 1704 can provide faster response times than the primary memory, such as the non-volatile memory 1708.

[0127] By executing the code 1710, the processor 1702 can control the operation of the interface 1706. The interface 1706 can include a network interface. Such a network interface can include one or more physical interfaces (e.g., radio transceiver devices, Ethernet ports, USB ports, etc.) as well as a software stack including drivers and / or other code 1710 configured to communicate with the one or more physical interfaces to support one or more LAN, PAN, and / or WAN standard communication protocols. Such communication protocols can include, for example, TCP and UDP, among others. As such, the network interface can enable the computing device 1700 to access and communicate with other computing devices via a computer network.

[0128] The interface(s) 1706 can include one or more user interfaces. For example, in some embodiments, the user interface(s) 1706 can include user input and / or output devices (e.g., a keyboard, a mouse, a touchscreen, a display, a speaker, a camera, an accelerometer, a biometric scanner, an environmental sensor, etc.) and a software stack including drivers and / or other code 1710 configured to communicate with the user input and / or output devices. As such, the user interface(s) 1706 can enable the computing device 1700 to interact with a user to receive input and / or present output. The presented output can include, for example, one or more GUIs including one or more controls configured to display output and / or receive input. The received input can specify values to be stored in the data storage 1712. The displayed output can indicate values stored in the data storage 1712.

[0129] The various features of the computing device 1700 described above can communicate with one another via an interconnection mechanism 1714. In some embodiments, the interconnection mechanism 1714 can include a communication bus.

[0130] Figure 18 An example token 1802, such as a JSON web token (JWT), is shown that can be employed by various system components as described above. As illustrated, the token 1802 can include a header 1804, a payload 1806, and a signature 1808. In some embodiments, the header 1804 can specify a signing technique used to generate the signature 1808 based on the contents of the header 1804 and / or the payload 1806 and a private key. In some embodiments, for example, the specified signing technique can involve (A) combining a base64url encoded header and a base64url encoded payload, (B) hashing the combined base64url value using a hashing technique (e.g., SHA256), and (C) encrypting the determined hash using a private key. As such, by verifying the signature 1808 using the private key and the specified signing technique, a recipient device is able to confirm that the contents of a token 1802 it receives from another device have not been altered or otherwise compromised. In some embodiments, the header 1804 or the payload 1806 of the token 1802 can additionally include an identifier of the device for which it was generated, thereby enabling a recipient device to confirm that a received token 1802 is from the same device for which the token 1802 was originally generated, thereby limiting the use of the token 1802 by other devices to which it can have been transmitted.

[0131] The following paragraphs (M1) through (M14) describe examples of methods that can be performed in accordance with the present disclosure.

[0132] (M1) can perform a method comprising: receiving a first request to establish a first connection between a computing device and a camera, wherein the first connection is configured to allow video data to stream from the camera to the computing device, the computing device is remote from the camera, and the camera is configured to support connections to multiple remote devices simultaneously; determining that a second connection has been established between an endpoint device and the camera, wherein the endpoint device is remote from the camera, and the second connection is configured to allow video data to stream from the camera to the endpoint device; and based at least in part on the first request and the second connection having been established, terminating the second connection and establishing the first connection to provide the computing device with access to one or more functions of the camera without interference from the endpoint device.

[0133] (M2) can perform the method as recited in paragraph (M1), and the method can further comprise sending a message to the endpoint device indicating that the second connection will be terminated prior to terminating the second connection.

[0134] (M3) can perform the method as recited in paragraph (M1) or paragraph (M2), wherein the method can be performed by an application hosted on the camera.

[0135] (M4) can perform the method as recited in any of paragraphs (M1) through (M3), wherein the first connection can comprise a first peer-to-peer connection between the computing device and the camera, and the second connection can comprise a second peer-to-peer connection between the endpoint device and the camera.

[0136] (M5) can perform the method as recited in any of paragraphs (M1) through (M4), wherein the first connection can be further configured to allow audio data to stream from the computing device to the camera.

[0137] (M6) can perform the method as recited in any of paragraphs (M1) through (M5), wherein the second connection can be further configured to allow audio data to stream from the endpoint device to the camera.

[0138] (M7) can perform the method as recited in any of paragraphs (M1) through (M6), wherein the first request can comprise a Session Description Protocol (SDP) offer received from the computing device via a Web Real-Time Communication (WebRTC) signaling server, and the method can further comprise: establishing the first connection comprises sending an SDP answer to the computing device via the WebRTC signaling server.

[0139] (M8) can perform a method comprising: receiving a first request to establish a first connection between an endpoint device and a camera, wherein the endpoint device is remote from the camera and the camera is configured to simultaneously support connections with multiple remote devices; determining that a second connection has been established between a second computing device and the camera, wherein the second computing device is remote from the camera and the second connection is configured to allow video data to stream from the camera to the second computing device; and based at least in part on the first request and the second connection having been established, rejecting the first request such that the second computing device is able to access one or more functions of the camera without interference from the endpoint device.

[0140] (M9) can perform the method as described in paragraph (M8), and the method can further comprise: sending a message to the endpoint device that causes the endpoint device to output an indication that the first request has been rejected.

[0141] (M10) can perform the method as described in paragraph (M8) or paragraph (M9), wherein the message can be sent via a data channel between the camera and the endpoint device, and the method can further comprise: establishing the data channel prior to sending the message, and tearing down the data channel after sending the message.

[0142] (M11) can perform the method as described in any of paragraphs (M8) through (M10), wherein the method can be performed by an application hosted on the camera.

[0143] (M12) can perform the method as described in any of paragraphs (M8) through (M11), wherein the first connection can comprise a first peer-to-peer connection between the endpoint device and the camera, and the second connection can comprise a second peer-to-peer connection between the second computing device and the camera.

[0144] (M13) can perform the method as described in any of paragraphs (M8) through (M12), wherein the second connection can be further configured to allow audio data to stream from the second computing device to the camera.

[0145] (M14) can perform the method as described in any of paragraphs (M8) through (M13), wherein the first request can comprise a Session Description Protocol (SDP) offer received from the endpoint device via a Web Real-Time Communication (WebRTC) signaling server.

[0146] The following paragraphs (S1) through (S14) describe examples of apparatuses and / or systems that can be configured in accordance with the present disclosure.

[0147] (S1) A system can include at least one processor and at least one computer- readable medium encoded with instructions that, when executed by the at least one processor, cause the system to: receive a first request to establish a first connection between a computing device and a camera, wherein the first connection is configured to allow video data to stream from the camera to the computing device, the computing device is remote from the camera, and the camera is configured to simultaneously support connections with multiple remote devices; determine that a second connection has been established between an endpoint device and the camera, wherein the endpoint device is remote from the camera and the second connection is configured to allow video data to stream from the camera to the endpoint device; and based at least in part on the first request and the second connection having been established, terminate the second connection and establish the first connection to provide the computing device with access to one or more functions of the camera without interference from the endpoint device.

[0148] (S2) The system as described in paragraph (S1) can be configured wherein the at least one computer-readable medium further encodes additional instructions that, when executed by the at least one processor, further cause the system to send a message to the endpoint device indicating that the second connection will be terminated prior to terminating the second connection.

[0149] (S3) The system as described in paragraph (S1) or paragraph (S2) can be configured wherein the instructions can be included in an application hosted on the camera.

[0150] (S4) The system as described in any of paragraphs (S1) through (S3) can be configured wherein the first connection can comprise a first peer-to-peer connection between the computing device and the camera, and the second connection can comprise a second peer-to-peer connection between the endpoint device and the camera.

[0151] (S5) The system as described in any of paragraphs (S1) through (S4) can be configured wherein the first connection can be further configured to allow audio data to stream from the computing device to the camera.

[0152] (S6) The system as described in any of paragraphs (S1) through (S5) can be configured wherein the second connection can be further configured to allow audio data to stream from the endpoint device to the camera.

[0153] (S7) The system as described in any of paragraphs (S1) through (S6) can be configured wherein the first request can comprise a Session Description Protocol (SDP) offer received from the computing device via a Web Real-Time Communication (WebRTC) signaling server, and the at least one computer-readable medium can further encode additional instructions that, when executed by the at least one processor, further cause the system to establish the first connection at least in part by sending an SDP answer to the computing device via the WebRTC signaling server.

[0154] (S8) A system can include at least one processor and at least one computer- readable medium encoded with instructions that, when executed by the at least one processor, cause the system to: receive a first request to establish a first connection between an endpoint device and a camera, wherein the endpoint device is remote from the camera and the camera is configured to simultaneously support connections with multiple remote devices; determine that a second connection has been established between a second computing device and the camera, wherein the second computing device is remote from the camera and the second connection is configured to allow video data to stream from the camera to the second computing device; and based at least in part on the first request and the second connection having been established, reject the first request, enabling the second computing device to access one or more functions of the camera without interference from the endpoint device.

[0155] (S9) The system as described in paragraph (S8) can be configured and the at least one computer-readable medium can be further encoded with additional instructions that, when executed by the at least one processor, further cause the system to send a message to the endpoint device that causes the endpoint device to output an indication that the first request has been rejected.

[0156] (S10) The system as described in paragraph (S8) or paragraph (S9) can be configured, wherein the message can be sent via a data channel between the camera and the endpoint device, and the at least one computer-readable medium can be further encoded with additional instructions that, when executed by the at least one processor, further cause the system to establish the data channel prior to sending the message and to disestablish the data channel after sending the message.

[0157] (S11) The system as described in any of paragraphs (S8) through (S10) can be configured, wherein the instructions can be included in an application hosted on the camera.

[0158] (S12) The system as described in any of paragraphs (S8) through (S11) can be configured, wherein the first connection can comprise a first peer-to-peer connection between the endpoint device and the camera, and the second connection can comprise a second peer-to-peer connection between the second computing device and the camera.

[0159] (S13) The system as described in any of paragraphs (S8) through (S12) can be configured, wherein the second connection can be further configured to allow audio data to stream from the second computing device to the camera.

[0160] (S14) The system as described in any of paragraphs (S8) through (S13) can be configured, wherein the first request can comprise a Session Description Protocol (SDP) offer received from the endpoint device via a Web Real-Time Communication (WebRTC) signaling server.

[0161] The following paragraphs (CRM1) through (CRM14) describe examples of computer-readable media that can be configured in accordance with the present disclosure.

[0162] (CRM1) At least one non-transitory computer-readable medium can be encoded with instructions that, when executed by at least one processor of a system, cause the system to: receive a first request to establish a first connection between a computing device and a camera, wherein the first connection is configured to allow video data to stream from the camera to the computing device, the computing device is remote from the camera, and the camera is configured to support connections to multiple remote devices simultaneously; determine that a second connection has been established between an endpoint device and the camera, wherein the endpoint device is remote from the camera and the second connection is configured to allow video data to stream from the camera to the endpoint device; and based at least in part on the first request and the second connection having been established, terminate the second connection and establish the first connection to provide the computing device with access to one or more functions of the camera without interference from the endpoint device.

[0163] (CRM2) At least one non-transitory computer-readable medium can be configured as described in paragraph (CRM1), and can further be encoded with additional instructions that, when executed by the at least one processor, further cause the system to, prior to terminating the second connection, send a message to the endpoint device indicating that the second connection will be terminated.

[0164] (CRM3) At least one non-transitory computer-readable medium can be configured as described in paragraph (CRM1) or paragraph (CRM2), wherein the instructions can be included in an application hosted on the camera.

[0165] (CRM4) At least one non-transitory computer-readable medium can be configured as described in any of paragraphs (CRM1) through (CRM3), wherein the first connection can comprise a first peer-to-peer connection between the computing device and the camera, and the second connection can comprise a second peer-to-peer connection between the endpoint device and the camera.

[0166] (CRM5) At least one non-transitory computer-readable medium can be configured as described in any of paragraphs (CRM1) through (CRM4), wherein the first connection can be further configured to allow audio data to stream from the computing device to the camera.

[0167] (CRM6) At least one non-transitory computer-readable medium can be configured as described in any of paragraphs (CRM1) through (CRM5), wherein the second connection can be further configured to allow audio data to stream from the endpoint device to the camera.

[0168] (CRM7) can be configured as recited in any of paragraphs (CRM1) through (CRM6), wherein the first request can include a Session Description Protocol (SDP) offer received from the computing device via a Web Real-Time Communication (WebRTC) signaling server, and the at least one non-transitory computer-readable medium can further encode additional instructions that, when executed by the at least one processor, further cause the system to establish the first connection at least in part by sending an SDP answer to the computing device via the WebRTC signaling server.

[0169] (CRM8) At least one non-transitory computer-readable medium can encode instructions that, when executed by at least one processor of a system, cause the system to: receive a first request to establish a first connection between an endpoint device and a camera, wherein the endpoint device is remote from the camera and the camera is configured to concurrently support connections with multiple remote devices; determine that a second connection has been established between a second computing device and the camera, wherein the second computing device is remote from the camera and the second connection is configured to allow video data to stream from the camera to the second computing device; and deny the first request based at least in part on the first request and the second connection having been established, enabling the second computing device to access one or more functions of the camera without interference from the endpoint device.

[0170] (CRM9) can be configured as recited in paragraph (CRM8) and can further encode additional instructions that, when executed by the at least one processor, further cause the system to send a message to the endpoint device that causes the endpoint device to output an indication that the first request has been denied.

[0171] (CRM10) can be configured as recited in paragraph (CRM8) or paragraph (CRM9), wherein the message can be sent via a data channel between the camera and the endpoint device, and the at least one non-transitory computer-readable medium can further encode additional instructions that, when executed by the at least one processor, further cause the system to establish the data channel prior to sending the message and to disestablish the data channel after sending the message.

[0172] (CRM11) can be configured as recited in any of paragraphs (CRM8) through (CRM10), wherein the instructions can be included in an application hosted on the camera.

[0173] (CRM12) can be configured as recited in any of paragraphs (CRM8) through (CRM11), wherein the first connection can include a first peer-to-peer connection between the endpoint device and the camera, and the second connection can include a second peer-to-peer connection between the second computing device and the camera.

[0174] (CRM13) can configure the at least one non-transitory computer-readable medium as in any of paragraphs (CRM8) through (CRM12), wherein the second connection is further configurable to allow streaming of audio data from the second computing device to the camera.

[0175] (CRM14) can configure the at least one non-transitory computer-readable medium as in any of paragraphs (CRM8) through (CRM13), wherein the first request can include a Session Description Protocol (SDP) offer received from the endpoint device via a Web Real-Time Communication (WebRTC) signaling server.

[0176] Various inventive concepts can be embodied as one or more methods, of which an example has been provided. The acts performed as part of the method can be ordered in any suitable way. Accordingly, an example can be constructed in which acts are performed in an order different than illustrated in the illustrative examples, which can include performing some acts simultaneously, even though shown as sequential acts in illustrative examples.

[0177] The use of ordinal terms such as "first," "second," "third," etc. to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another, or temporal sequence of acts of a method. The terms are used merely as labels to distinguish between two importantly distinct cycles of a claim element.

[0178] Examples of the methods and systems discussed herein are not limited in application to the details of construction and the arrangement of components set forth in the following description or illustrated in the accompanying drawings. The methods and systems are capable of implementation in other examples and of being practiced or being carried out in various ways. Examples of specific implementations are provided herein for illustrative purposes only and are not intended to be limiting. In particular, acts, components, elements and features discussed in connection with any one or more examples are not intended to be excluded from a similar role in any other examples.

[0179] Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. Any references to examples, acts, means, or elements of the systems and methods herein using a singular form are not intended to exclude plural forms of the example, act, means, or element. Any references to examples, acts, means, or elements of the systems and methods herein using a singular form are intended to include both the singular and the plural form. References to examples, acts, means, or elements of the systems and methods herein using a plural form are intended to include both the singular and the plural form.

[0180] The use of “including,” “comprising,” “having,” “containing,” “involving,” and variations thereof herein are meant to be open-ended, and do not limit the item, composition, process, method, or apparatus to the items, compositions, processes, methods, or apparatus that follow the respective terms. References to “or” can be construed as inclusive so that any terms described using “or” can indicate any of a single, more than one, and all of the described terms. In addition, in cases where there is a conflict between what is described in this document and what is described in a document that is incorporated by reference, the term usage in this document controls. The term “consisting” is intended to be construed as meaning “comprising” but not “including” or “comprising” or “including” more than one element, component, or step.

[0181] Having described several examples, one skilled in the art will be aware that modifications and variations are possible within the scope of the present disclosure. Such modifications and variations are intended to be within the scope of the present disclosure. Accordingly, the foregoing description is by way of example only, and is not intended to be limiting.

Claims

1. A method comprising: The camera receives a request to establish a first connection between a first application hosted by a computing device and the camera, wherein the first connection will be configured to allow video data to be streamed from the camera to the computing device, which is remote from the camera, the camera is configured to simultaneously support connections to multiple remote devices so that the camera can simultaneously stream video data to the multiple remote devices, and the first application is of a first type; In response to the camera receiving the request, the camera establishes the first connection and determines that a second connection has been established between the camera and a second application hosted by an endpoint device, wherein the endpoint device is remote from the camera, the second connection is configured to allow video data to be streamed from the camera to the endpoint device, and the second application is a second type different from the first type; and In response to the camera determining that the second connection has been established and at least in part based on the first application being of the first type and the second application being of the second type, the camera terminates the second connection to provide the computing device with access to one or more functions of the camera via the first connection without interference from the endpoint device via the second connection; The first type of application is configured to be operated by a monitoring agent associated with a security company, and the second type of application is configured to be operated by a person associated with the property where the camera is located.

2. The method according to claim 1, further comprising: Before terminating the second connection, the camera sends a message to the endpoint device indicating that the second connection will be terminated.

3. The method according to claim 1, wherein, The first connection includes a first peer-to-peer connection between the computing device and the camera, and the second connection includes a second peer-to-peer connection between the endpoint device and the camera.

4. The method according to claim 1, wherein, The first connection is also configured to allow audio data to be streamed from the computing device to the camera.

5. The method according to claim 4, wherein, The second connection is also configured to allow audio data to be streamed from the endpoint device to the camera.

6. The method according to claim 1, wherein, The request includes a Session Description Protocol (SDP) proposal received from the computing device via a WebRTC signaling server; and Establishing the first connection includes sending an SDP response to the computing device via the WebRTC signaling server.

7. A method comprising: The camera receives a request to establish a first connection between a first application hosted by an endpoint device and the camera, wherein the endpoint device is remote from the camera, the camera is configured to simultaneously support connections to multiple remote devices so that the camera can simultaneously stream video data to the multiple remote devices, and the first application is of a first type; In response to the camera receiving the request, the camera determines that a second connection has been established between the camera and a second application hosted by a computing device, wherein the computing device is remote from the camera, the second connection is configured to allow video data to be streamed from the camera to the computing device, and the second application is a second type different from the first type; and In response to the camera determining that the second connection has been established and at least in part based on the first application being of the first type and the second application being of the second type, the camera rejects the request, enabling the computing device to access one or more functions of the camera via the second connection without interference from the endpoint device via the first connection; The second type of application is configured to be operated by a monitoring agent associated with a security company, while the first type of application is configured to be operated by a person associated with the property where the camera is located.

8. The method according to claim 7, further comprising: The camera sends a message to the endpoint device, causing the endpoint device to output an indication that the request has been rejected.

9. The method according to claim 8, wherein, The message is sent via a data channel between the camera and the endpoint device, and the method further includes: The data channel is established before the message is sent; and The data channel is closed after the message is sent.

10. The method according to claim 7, wherein, The first connection includes a first peer-to-peer connection between the endpoint device and the camera, and the second connection includes a second peer-to-peer connection between the computing device and the camera.

11. The method according to claim 7, wherein, The second connection is also configured to allow audio data to be streamed from the computing device to the camera.

12. The method according to claim 7, wherein, The request includes a Session Description Protocol (SDP) proposal received from the endpoint device via a WebRTC signaling server.

13. A camera, comprising: At least one processor; and At least one computer-readable medium encoded with instructions that, when executed by the at least one processor, cause the camera to: A request is received to establish a first connection between a first application hosted by a computing device and the camera, wherein the first connection will be configured to allow video data to be streamed from the camera to the computing device, which is remote from the camera, the camera is configured to simultaneously support connections to multiple remote devices so that the camera can simultaneously stream video data to the multiple remote devices, and the first application is of a first type. In response to receiving the request, the first connection is established and it is determined that a second connection has been established between a second application hosted by an endpoint device and the camera, wherein the endpoint device is remote from the camera, the second connection is configured to allow video data to be streamed from the camera to the endpoint device, and the second application is a second type different from the first type; and In response to determining that the second connection has been established and at least in part based on the first application being of the first type and the second application being of the second type, the second connection is terminated to provide the computing device with access to one or more functions of the camera via the first connection without interference from the endpoint device via the second connection; The first type of application is configured to be operated by a monitoring agent associated with a security company, and the second type of application is configured to be operated by a person associated with the property where the camera is located.

14. The camera according to claim 13, wherein, The at least one computer-readable medium further encodes additional instructions that, when executed by the at least one processor, also cause the camera to: Before terminating the second connection, a message indicating that the second connection will be terminated is sent to the endpoint device.

15. The camera according to claim 13, wherein, The first connection includes a first peer-to-peer connection between the computing device and the camera, and the second connection includes a second peer-to-peer connection between the endpoint device and the camera.

16. The camera according to claim 13, wherein: The first connection is also configured to allow audio data to be streamed from the computing device to the camera.

17. The camera according to claim 13, wherein, The request includes a Session Description Protocol (SDP) proposal received from the computing device via a WebRTC signaling server, and the at least one computer-readable medium further encodes additional instructions that, when executed by the at least one processor, also cause the camera to: The first connection is established at least in part by sending an SDP response to the computing device via the WebRTC signaling server.

Images