Modifying operation of a sensor using collected sensor data

Abstract

Techniques are provided herein for modifying operation of a sensor using collected sensor data. Sensor data is collected according to an initial mode of operation, the sensor data describing a physical environment of a user authorized to access a controlled resource. The initial mode of operation defines an initial performance of at least one sensor used to collect the sensor data. Responsive to detecting, based on the sensor data, a presence of a different user in the physical environment of the authorized user, the at least one sensor is switched from the initial mode of operation to a modified mode of operation. The modified mode of operation is different from the initial mode of operation and defines a revised performance of the at least one sensor used to collect the sensor data.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims priority to Luxembourg application number 101928, filed on July 17, 2020, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0003] Embodiments of the subject matter relate generally to sensors, and more particularly, to modifying operation of a sensor using collected sensor data. BACKGROUND

[0004] Sensors are often used to monitor a physical environment. For example, security cameras are often installed around a building to monitor for intruders. Captured video can be used to alert a user (e.g., a homeowner or security guard) of a current issue, such as an attempted break-in, and / or provide evidence of a past issue. For example, a store owner who suspects an item was stolen can review video to determine who or what caused the item to be lost. In either case, there can be issues with managing resource usage related to collecting sensor data. For example, continuously capturing video results in a large amount of video data that is difficult to store and / or transmit to other locations. Current techniques capture low resolution video data to reduce the size of the collected data, but this results in lower quality video. Accordingly, improvements are needed. BRIEF DESCRIPTION OF DRAWINGS

[0005] In the drawings, which are not necessarily drawn to scale, like numerals can describe similar components in different views. Like numerals having different letter suffixes can represent different instances of similar components. Some embodiments are illustrated by way of example, not limitation, in the figures of the accompanying drawings in which:

[0006] Figure 1 FIG. 1 is a block diagram illustrating an example system for modifying operation of a sensor using collected sensor data, in accordance with some example embodiments.

[0007] Figure 2 FIG. 2 is a block diagram of a monitoring system, in accordance with some example embodiments.

[0008] Figure 3 FIG. 3 is a flow diagram illustrating a method for modifying operation of a sensor using collected sensor data, in accordance with certain example embodiments.

[0009] Figure 4 FIG. 4 is a flow diagram illustrating a method for restoring operation of a sensor using collected sensor data, in accordance with certain example embodiments.

[0010] Figure 5 FIG. 5 is a flow diagram illustrating a method for performing an action using collected sensor data, in accordance with certain example embodiments.

[0011] Figure 6 is a block diagram illustrating a representative software architecture, which can be used in conjunction with various hardware architectures varying from highly integrated to more modular architectures. For example, different portions of the architecture can be implemented on various hardware architectures.

[0012] Figure 7 is a block diagram illustrating machine components capable of reading instructions from a machine-readable medium (e.g., a machine-readable storage medium) and performing any one or more of the methodologies discussed herein in accordance with some example embodiments. DETAILED DESCRIPTION

[0013] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of some example embodiments. It will be apparent, however, to one skilled in the art that the present subject matter can be practiced without these specific details or with an equivalent set of details.

[0014] Reference in the specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the subject matter. The appearances of the phrase “in one embodiment” or “in an embodiment” in various places in the specification are not necessarily all referring to the same embodiment.

[0015] For purposes of explanation, specific configurations and details are set forth in order to provide a thorough understanding of the subject matter. However, it will be apparent to one skilled in the art that embodiments of the subject matter described can be practiced without the specific details presented herein or in a manner consistent with the various combinations of the features presented. Moreover, well-known features can be omitted or simplified in order not to obscure the embodiments being described. Various examples can be given throughout this description. These are only examples and do not limit the scope of the claims to the specific examples given. For instance, the examples can be modified if necessary to conform to the spirit of the description.

[0016] Systems, methods, and non-transitory computer-readable media for modifying operation of a sensor using collected sensor data are disclosed. An authentication system is used to restrict access to an access-controlled resource. An access-controlled resource is any type of physical or digital object, account, area, service, application, portion of an application, digital service, digital media, etc. that is intended to be limited to a set of authorized users. An access-controlled resource can be associated with an authentication requirement that limits access to the access-controlled resource to the set of authorized users. That is, the authentication requirement is used to allow authorized users to access the access-controlled resource while restricting access to the access-controlled resource to unauthorized users.

[0017] An example of an access-controlled resource is an email account facilitated by an online service. Authentication requirements, such as requiring a user to provide authentication credentials associated with the email account (e.g., a username and password associated with the email account), can be used to limit access to the email account to authorized users. For example, an authorized user can provide a username and password to access the email account, while an unauthorized user who does not know the username and password cannot access the email account. As another example, the access-controlled resource can be a digital file or set of digital files. Authentication requirements, such as prompting a user for a username and password, can be used to limit access to the digital file or set of digital files to authorized users, such as a specified group of employees of a company. As another example, the access-controlled resource can be a server room located within a building. Authentication requirements, such as requiring a user to enter a code, can be used to limit access to the server room to authorized users. For example, an authorized user can enter the room using the code, while an unauthorized user who does not know the code cannot enter the room. As another example, the access-controlled resource can be an in-person or online administered exam. Authentication requirements, such as requiring a user to provide a username / password and / or a government-issued identification, can be used to ensure that the appropriate user is taking the test.

[0018] To further secure the access-controlled resource, sensor data describing a physical environment surrounding an authorized user can be collected to detect the presence of an unauthorized user. For example, sensor data can be collected while an authorized user is using an automated teller machine (ATM) to detect the presence of an unauthorized user who poses a potential danger to the authorized user. As another example, sensor data can be collected while an authorized user is taking an online exam to ensure that the authorized user (e.g., test-taker) does not seek the assistance of other unauthorized users during the exam.

[0019] The sensor data can be collected according to various operational modes. An operational mode defines the performance of available sensors when collecting sensor data, as well as how the collected sensor data is stored and / or otherwise processed. For example, an operational mode can define the individual sensors used to collect sensor data, the positioning and / or view (e.g., zoom level) of the sensors, the quality or resolution of the sensor data to be collected, whether the sensor data is to be stored, the duration of sensor data to be stored, whether the sensor data is to be transmitted to another device, and the like.

[0020] The operational mode for collecting sensor data can be automatically modified based on the collected sensor data, such as when an unauthorized user is detected in the physical environment surrounding an authorized user. For example, the initial operational mode for collecting sensor data can be modified to a modified operational mode to increase the resolution of collecting sensor data, adjust the position and / or view of the sensors to better capture the unauthorized user, use additional sensors to capture sensor data, store the captured sensor data in memory, transmit the captured sensor data to a designated network location, and so on. Similarly, the modified operational mode can be modified in response to determining that the unauthorized user is no longer in the physical environment of the authorized user. For example, the modified operational mode can revert back to the initial operational mode used prior to detecting the unauthorized user.

[0021] Adjusting the operational mode for collecting sensor data provides reduced resource usage without sacrificing performance. For example, sensor data can be collected according to an initial operational mode that conserves computational resources, and the initial operational mode is adjusted to a modified operational mode that provides a higher level of performance when an unauthorized user is detected in the physical environment of an authorized user, such as by increasing resolution, using additional sensors, and so on. This allows performance to be increased when valuable sensor data is being collected (e.g., when an unauthorized user is detected in the physical environment of an authorized user), while providing resource-conserving performance when less valuable sensor data is being collected (e.g., when no unauthorized user is detected in the physical environment of an authorized user).

[0022] Any of a variety of types of sensors can be used to collect sensor data, such as optical sensors (e.g., cameras), audio sensors (e.g., microphones), infrared sensors, motion sensors, and so on. Thus, each operational mode can also use any number of sensors and / or combinations of sensors. For example, one operational mode can use optical, audio, and infrared sensors. As another example, an operational mode can use only optical and infrared sensors.

[0023] In addition to adjusting the operational mode for collecting sensor data, additional actions can be performed based on the collected sensor data. For example, when an unauthorized user is detected in the physical environment of an authorized user, access to an access-controlled resource can be limited. Limiting access to an access-controlled resource can include closing user interface windows that present sensitive information, obfuscating sensitive data, stopping access to an access-controlled resource (e.g., locking a door), etc. As another example, an alert can be presented to the authorized user indicating that an unauthorized has been detected. As another example, a notification can be sent to alert a designated third-party user that an unauthorized user has been detected. As another example, a distress button can be provided to the authorized user that is configured to alert the appropriate authorities, limit access to an access-controlled resource, sound an alarm, etc.

[0024] Adjusting the operational mode of the sensors provides a technical solution to the technical problem of managing resource usage when collecting sensor data and when collecting high-quality sensor data. In particular, when no unauthorized user is detected in the physical environment around an authorized user, an initial operational mode can be used that is configured to reduce resource usage. For example, sensor data can be captured at a relatively lower resolution, using fewer sensors, etc. Similarly, the sensor data can be processed in a manner that reduces resource usage, such as storing less sensor data, not storing sensor data, retaining sensor data for a shorter period of time, not generating metadata, etc. Using the initial mode allows for reduced resource usage when no unauthorized user is detected and the primary goal of the sensor data can simply be to detect an unauthorized user. However, when an unauthorized user is detected, the operational mode can be modified to a modified operational mode that provides higher quality performance, such as by capturing sensor data using additional sensors, at a higher resolution, etc. Thus, when the collected sensor data is most valuable (e.g., when an unauthorized user is detected), higher performance that can be resource intensive is used, while when the collected sensor data can be less valuable (e.g., when no unauthorized user is detected) resource-conserving performance is used.

[0025] Figure 1is a block diagram illustrating an example system 100 for modifying operation of a sensor 108 using collected sensor data, in accordance with some example embodiments. As shown, the system 100 includes a client device 102, an authentication system 104, a sensor 108, and a monitoring system 110, which are connected to a communication network 106 and configured to communicate with each other by using the communication network 106. The communication network 106 is any type of network, including a local area network (LAN), such as an intranet, a wide area network (WAN), such as the Internet, or any combination thereof. Moreover, the communication network 106 can be a public network, a private network, or a combination thereof. The communication network 106 is implemented using any number of communication links associated with one or more service providers, including one or more wired communication links, one or more wireless communication links, or any combination thereof. Moreover, the communication network 106 is configured to support the transmission of data formatted using any number of protocols.

[0026] A plurality of computing devices can be connected to the communication network 106. A computing device is any type of general purpose computing device capable of network communication with other computing devices. For example, a computing device can be a personal computing device, such as a desktop or workstation, a business server, or a portable computing device, such as a laptop computer, a smartphone, or a tablet personal computer (PC), or an Internet of Things (IoT) device, such as a sensor, a wearable device, a smart meter, etc. The computing device can include some or all of the features, components, and peripherals of the machine 700 illustrated in FIG. 7. Figure 7

[0027] To facilitate communication with other computing devices, the computing device can include a communication interface configured to receive communications, such as requests, data, etc., from another computing device with which the computing device is in network communication and to pass the communications to an appropriate module running on the computing device. The communication interface also transmits communications to another computing device with which the computing device is in network communication.

[0028] The authentication system 104 manages access to access-controlled resources. An access-controlled resource is any type of physical or digital object, account, area, service, etc., that is intended to be limited to a set of authorized users. An access-controlled resource can be associated with an authentication requirement that limits access to the access-controlled resource to the set of authorized users. The authentication system 104 enforces the authentication requirement to allow authorized users to access the access-controlled resource while restricting unauthorized users from accessing the access-controlled resource. The authentication requirement is satisfied by providing the correct authentication credentials. For example, the authentication requirement can be providing the correct authentication credentials, such as a code, a password, or a combination of a username and password. As another example, the authentication requirement can be providing the correct authentication credentials, such as specified identity information (e.g., a social security number, a mother’s maiden name, a bank account balance, etc.), an answer to a personal question, etc.​

[0029] One example of an access-controlled resource is an email account, which is associated with an authentication requirement to provide a correct username and password associated with the email account. Another example of an access-controlled resource is a digital file that is associated with an authentication requirement to provide a correct password. Other examples of access-controlled resources include a physical room that is associated with an authentication requirement to provide a correct access code, and an exam that is associated with an authentication requirement to provide a username / password and / or an acceptable form of identification.

[0030] A user communicates with the authentication system 104 to request access to an access-controlled resource by using a client device 102 that is connected to the communication network 106 through direct and / or indirect communication. For example, the user uses the client device 102 to provide authentication credentials (e.g., a password, a code, a username / password, etc.) to the authentication system 104. The authentication system 104 determines whether the received authentication credentials satisfy the authentication requirement, and, if so, approves the authentication request. For example, the authentication system 104 provides the authorized user with access to the access-controlled resource, or utilizes functionality of the authentication system 104 to notify an online service (e.g., an email service, a banking service, etc.) that the authorized user has satisfied the authentication requirement.

[0031] Although the illustrated system 100 includes only one client device 102, this is for ease of explanation and not for limitation. Those skilled in the art will appreciate that the system 100 can include any number of client devices 102. Moreover, the authentication system 104 can accept connections from and interact with any number of client devices 102 simultaneously. The authentication system 104 can support connections from various different types of client devices 102, such as desktop computers; mobile computers; mobile communication devices, such as mobile phones, smartphones, tablet computers; smart televisions; set-top boxes; keyboards, and / or any other network-enabled computing device. Thus, the client devices 102 can be of different types, capabilities, operating systems, and so on.

[0032] Further, while the client device 102 and the authentication system 104 are shown as separate entities, this is merely one possible embodiment. In other embodiments, the client device 102 and the authentication system 104 are combined into a single device. For example, a keypad placed near a door can perform the functions of both the client device 102 and the authentication system 104. For example, a user can use the keypad to enter authentication credentials, such as a password, to gain access to a room protected by a door. The keypad can perform the functions of the authentication system 104 by determining whether to approve an authentication request (e.g., determining whether the entered authentication credentials satisfy the authentication requirements). These examples, as well as others, do not require the use of the communication network 106, but can use direct communication between the client device 102 and the authentication system 104 (or, when contained in a single device, can communicate as components of the single device).

[0033] In some embodiments, a user can interact with the authentication system 104 via a client-side application installed on the client device 102. In some embodiments, the client-side application includes a component specific to the authentication system 104 or an online service (not shown) that utilizes the functionality of the authentication system 104 to manage access to access-controlled resources. For example, the component can be a standalone application, one or more application plugins, and / or a browser extension. However, a user can also interact with the authentication system 104 via a third-party application (e.g., a web browser) that resides on the client device 102 and is configured to communicate with the authentication system 104 or an online service (not shown) that utilizes the functionality of the authentication system 104. In either case, the client-side application presents a user interface (UI) for the user to interact with the authentication system 104. For example, the user interacts with the authentication system 104 via a client-side application integrated with a file system or via a webpage displayed using a web browser application.

[0034] As part of an authentication request (e.g., a user attempts to access an access-controlled resource), the user provides authentication credentials (e.g., a username / password, a code, etc.) to the authentication system 104 to satisfy authentication requirements enforced by the authentication system 104. The authentication requirements are enforced to restrict access to the access-controlled resource. The authentication system 104 approves or denies the authentication request based on whether the received authentication credentials satisfy the authentication requirements. For example, the authentication system 104 compares the received authentication credentials to stored authentication credentials associated with the access-controlled resource to determine whether the received authentication credentials match the stored authentication credentials or satisfy a similarity threshold to the stored authentication credentials.

[0035] If the authentication credentials provided by the user satisfy the authentication requirements, the authentication system 104 approves the authentication request. Alternatively, if the authentication credentials provided by the user do not satisfy the authentication requirements, the authentication system 104 denies the authentication request.

[0036] The authentication requirements implemented by the authentication system 104 initially restrict access to the access-controlled resource by unauthorized users, however, do not protect the access-controlled resource after the access-controlled resource has been accessed by an authorized user. Thus, an unauthorized user can attempt to access the access-controlled resource after an authorized user (e.g., an authorized user that has satisfied the authentication requirements) has accessed the access-controlled resource. For example, an unauthorized user can attempt to view sensitive account data presented on a screen of the authorized user, attack an authorized user that has entered a password on an ATM, provide assistance to an authorized user during an exam, and so on.

[0037] The monitoring system 110 prevents such scenarios by collecting sensor data that describes a physical environment of the authorized user to detect the presence of an unauthorized user. For example, the monitoring system 110 can collect sensor data when providing the authorized user with access to the access-controlled resource, such as when the authorized user is using an ATM, logging into an email account, taking an exam, within a secure physical location, and so on.

[0038] The monitoring system 110 can collect the sensor data using one or more sensors 108 connected to the communication network 106. Although the sensors 108, the monitoring system 110, the client device 102, and the authentication system 104 are illustrated as separate entities, this is merely an example and is not meant to be limiting. The sensors 108, the monitoring system 110, the client device 102, and the authentication system 104 can be embodied in one or more devices. For example, the monitoring system 110 and / or the sensors 108 can be partially or entirely incorporated within the client device 102. For example, the client device 102 can be a laptop computer or a smartphone equipped with multiple sensors 108 such as a camera, a microphone, and so on. As another example, the sensors 108 can include a combination of sensors 108 incorporated in the client device 102 and sensors 108 independent of the client device 102. For example, the sensors 108 can include a microphone included in the client device 102 such as a laptop computer, and a camera independent of the client device 102.

[0039] The monitoring system 110 can collect sensor data according to various operational modes. An operational mode defines the performance of the available sensors 108 in collecting sensor data, and how the collected sensor data is to be stored and / or otherwise processed by the monitoring system 110. For example, an operational mode can define the individual sensors 108 used to collect sensor data, the positioning and / or view (e.g., zoom level) of each sensor 108, the quality or resolution at which each sensor 108 collects sensor data, whether to store the sensor data, the duration for which the sensor data is stored, whether to transmit the sensor data to another device, and so on.

[0040] The monitoring system 110 modifies the operational mode for collecting sensor data based on the collected sensor data. For example, the monitoring system 110 can modify the operational mode in response to detecting the presence of an unauthorized user in the authorized user's physical environment. This allows the monitoring system 110 to increase the resolution at which sensor data is collected, adjust the position and / or view of the sensors 108 to better capture the unauthorized user, use additional sensors 108 to capture sensor data, store the captured sensor data in memory, transmit the captured sensor data to a designated network location, and so on. Similarly, the monitoring system 110 can modify the operational mode in response to determining that the unauthorized user is no longer in the authorized user's physical environment. For example, the monitoring system 110 can modify the operational mode to a previous operational mode used prior to detecting the unauthorized user.

[0041] By adjusting the operational mode for collecting sensor data, the monitoring system 110 provides reduced resource usage and high quality performance. For example, the monitoring system 110 can collect sensor data according to an initial operational mode that conserves computing resources, and adjust the operational mode to a higher level of performance upon detecting the presence of an unauthorized user in the authorized user's physical environment, such as by increasing the resolution, using additional sensors 108, and so on. This allows the monitoring system 110 to provide higher performance when collecting valuable sensor data (e.g., when an unauthorized user is detected in the authorized user's physical environment), and resource-conserving performance when collecting less valuable sensor data (e.g., when no unauthorized user is detected in the authorized user's physical environment).

[0042] Any of various types of sensors 108 can be used to collect sensor data, such as optical sensors (e.g., cameras), audio sensors (e.g., microphones), infrared sensors, and so on. Thus, any number of sensors 108 and / or combinations of sensors 108 can also be used for each operational mode. For example, one operational mode can use optical, audio, and infrared sensors 108, while another operational mode can use only optical and infrared sensors 108.

[0043] In addition to adjusting the operational mode for collecting sensor data, the monitoring system 110 can perform additional actions based on the collected sensor data. For example, when an unauthorized user is detected in the physical environment of an authorized user, the monitoring system 110 can restrict access to an access-controlled resource. Restricting access to an access-controlled resource can include closing user interface windows that present sensitive information, obfuscating sensitive data, stopping access to an access-controlled resource (e.g., locking a door), and the like. As another example, the monitoring system 110 can present an alert to the authorized user indicating that an unauthorized user has been detected. As another example, the monitoring system 110 can send a notification to a designated third-party user indicating that an unauthorized user has been detected. As another example, the monitoring system 110 can provide an authorized user with a distress button that enables the authorized user to alert the appropriate authorities, restrict access to an access-controlled resource, sound an alarm, and the like.

[0044] Figure 2 is a block diagram of a monitoring system 110 in accordance with some example embodiments. To avoid obscuring the subject matter of the present invention with unnecessary detail, various functional components (e.g., modules) that are not germane to an understanding of the subject matter of the present invention have been omitted from Figure 2 However, those skilled in the art will readily recognize that the monitoring system 110 can support various additional functional components to facilitate additional functionality not specifically described herein. Moreover, Figure 2 The various functional modules depicted in

[0045] As shown, the monitoring system 110 includes a sensor management component 202, a sensor data reception component 204, a sensor data processing component 206, a baseline state determination component 208, an unauthorized user detection component 210, an operational mode modification component 212, an alert component 214, a resource restriction component 216, and a data store 218.

[0046] The sensor management component 202 manages the performance of the sensors 108. For example, the sensor management component 202 can communicate with the plurality of sensors 108 to initiate, modify, and / or stop the use of the sensors 108. The sensor management component 202 transmits commands to individual sensors 108 to elicit a specified action of the sensor 108. For example, the sensor management component 202 can send a command to a sensor 108 to cause the sensor 108 to begin collecting sensor data. As another example, the sensor management component 202 can send a command to a sensor 108 to cause the sensor 108 to modify the sensor's configuration while collecting sensor data, such as by increasing or decreasing the resolution at which the sensor 108 captures sensor data, changing the view or position of the sensor 108 (e.g., zoom in / out, move left / right), etc. As another example, the sensor management component 202 can send a command to a sensor 108 to cause the sensor 108 to stop collecting sensor data.

[0047] The sensor management component 202 can cause the various sensors 108 to operate based on a specified operational mode of the monitoring system 110. The operational mode defines the performance of the monitoring system 110 in collecting, storing, and / or processing sensor data. For example, the operational mode can define the specified sensors 108 used to collect sensor data, as well as the configuration of the sensors 108, such as resolution level, position, view, etc.

[0048] The sensor management component 202 can initially cause the sensors 108 to operate according to an initial operational mode and thereafter modify the performance of the sensors 108 based on a modified operational mode. The sensor management component 202 can receive a command from another component of the monitoring system 110 (e.g., the operational mode modification component 212) to modify the performance of the sensors 108 based on a modified operational mode. In turn, the sensor management component 202 sends a command to individual sensors 108 to cause the sensors 108 to operate according to the modified operational mode. The sensor management component 202 can initially cause the sensors 108 to operate according to an initial operational mode that provides reduced resource usage, such as using a limited number of sensors to capture sensor data, capturing sensor data at a lower resolution, etc. When an unauthorized user is detected, the sensor management component 202 can cause the sensors 108 to modify their performance based on a modified operational mode. For example, the sensor management component 202 can cause the sensors 108 to operate according to a modified operational mode that provides higher quality (e.g., using more sensors 108, higher resolution) and / or better captures the detected unauthorized user (e.g., directs the position of the sensors 108 toward the unauthorized user, zooms in or out to better capture the unauthorized user).

[0049] The sensor data receiving component 204 receives sensor data collected by various sensors 108. The sensors 108 can be any of a variety of types of sensors 108 used to collect sensor data. For example, the sensors can include optical sensors that capture images and / or video, audio sensors that capture sound, infrared sensors that capture infrared radiation and / or heat, motion sensors that detect motion, and so forth. Thus, the sensor data receiving component 204 can receive any of a variety of types of sensor data.

[0050] The sensor data processing component 206 processes sensor data received by the sensor data receiving component 204. Processing sensor data can include storing the sensor data, transmitting the sensor data to a designated network destination, generating metadata based on the sensor data, and so forth. For example, the sensor data processing component 206 can store some or all of the sensor data in the data store 218. As another example, the sensor data processing component 206 can generate metadata describing the sensor data, such as timestamp data describing a time at which the sensor data was collected, a type of sensor data collected, data identifying an authorized user, geographic data, a sensor 108 that collected the sensor data, and so forth.

[0051] The sensor data processing component 206 can process sensor data according to different operating modes of the monitoring system 110. For example, the sensor data processing component 206 can process sensor data according to an initial operating mode, and upon receiving a command to modify the operating mode, process sensor data according to a modified operating mode.

[0052] Processing sensor data according to different operating modes allows the sensor data processing component 206 to achieve different goals, such as conserving computing resources and / or providing higher quality performance. For example, the sensor data processing component 206 can process data according to an initial operating mode intended to reduce resource usage when no unauthorized users are detected in a physical environment around an authorized user. If an unauthorized user is detected in the physical environment, the sensor data processing component 206 can be instructed to process sensor data according to a modified operating mode that provides higher quality performance. For example, the sensor data processing component 206 can begin generating metadata or increase an amount of metadata generated based on the sensor data, increase an amount of metadata stored, transmit sensor data and / or metadata to a designated network destination, and so forth.

[0053] The baseline status determination component 208 determines a baseline status of the authorized user's physical environment. The baseline status of the physical environment describes the authorized user's physical environment at or proximate to the time when the authorized user is provided access to the access-controlled resource. The determined baseline status of the physical environment can be used as a basis for detecting the presence of an unauthorized user. For example, the baseline status of the physical environment can identify the presence of other users present in the authorized user's physical environment when the authorized user is provided access to the access-controlled resource. These users are likely not unauthorized users that pose a threat to the authorized user. For example, an authorized user using an ATM can be with their children, friends, or spouse. Accordingly, the monitoring system 110 can be configured to ignore the presence of these users when determining whether an unauthorized user is present. That is, the presence of the other users identified in the baseline status does not trigger detection of an unauthorized user.

[0054] The baseline status determination component 208 can use sensor data collected by the sensors 108 to determine the baseline status of the authorized user's physical environment. For example, the baseline status determination component 208 can use video, image, sound, and / or infrared data to determine the baseline status of the physical environment. The baseline status determination component 208 can also elicit information from the authorized user describing the physical environment, such as by prompting the user to enter information regarding the number of users present.

[0055] The unauthorized user detection component 210 determines whether an unauthorized user is present in the authorized user's physical environment based on sensor data collected by the sensors 108. The unauthorized user detection component 210 can use any of a variety of types and combinations of types of sensor data, such as video, image, sound, motion data, and / or infrared data. Further, the unauthorized user detection component 210 can use any of a variety of techniques to detect the presence of an unauthorized user.

[0056] In some embodiments, the unauthorized user detection component 210 can determine, based on the sensor data, a value indicative of a degree or level of detected change (e.g., movement, sound, heat, etc.) in the physical environment and compare the value to a threshold to determine whether an unauthorized user is present. For example, the unauthorized user detection component 210 can use sensor data collected by a motion sensor to determine a motion value indicative of a level or degree of detected motion in the physical environment. Similarly, the unauthorized user detection component 210 can use sensor data collected by an infrared sensor to determine an infrared radiation value indicative of a level or degree of detected change in infrared radiation in the physical environment. As another example, the unauthorized user detection component 210 can use sensor data collected by multiple sensors 108 (e.g., video cameras, microphones, motion sensors, infrared sensors, etc.) to determine an aggregate value indicative of a degree or level of detected change in the physical environment.

[0057] In any case, the unauthorized user detection component 210 compares the value determined based on the sensor data to a threshold to determine whether the detected change is indicative of the presence of an unauthorized user. For example, the unauthorized user detection component 210 determines that an unauthorized user is present when the value exceeds the threshold and determines that an unauthorized user is not present when the value does not exceed the threshold.

[0058] The unauthorized user detection component 210 can use other techniques and / or be based on other considerations to detect the presence of an unauthorized user, such as using facial and / or object recognition techniques, voice recognition, shadow recognition, or movement, determining a location of the detected change, etc.

[0059] The unauthorized user detection component 210 can notify other components of the monitoring system 110 when an unauthorized user is detected. Similarly, the unauthorized user detection component 210 can notify other components of the monitoring system 110 when it determines that an unauthorized user is no longer present in the physical environment. For example, the unauthorized user detection component 210 can notify the operational mode modification component 212, the alert component 214, and / or the resource limitation component 216 when the presence of an unauthorized user is detected and / or when the physical environment no longer detects an unauthorized user.

[0060] The operation mode modification component 212 causes changes in the operation mode of the monitoring system 110. For example, the operation mode modification component 212 sends commands to the sensor management component 202 and / or the sensor data processing component 206 to cause the respective components to modify the operation mode of collecting and processing sensor data. For example, the operation mode modification component 212 can send a command to the sensor management component 202 and / or the sensor data processing component 206 to operate according to a modified operation mode upon detecting an unauthorized user in the physical environment. As another example, the operation mode modification component 212 can send a command to the sensor management component 202 and / or the sensor data processing component 206 to restore operation to an initial operation mode upon no longer detecting an unauthorized user in the physical environment.

[0061] The alert component 214 performs specified alert actions in response to detecting an unauthorized user. For example, the alert component 214 can cause an alert notification to be presented to an authorized user, such as by presenting an alert notification on a display of the client device 102, sounding an alarm, etc. The alert component 214 can also transmit a notification to a specified network destination, such as a computing device associated with a security personnel, a police station, a test official, etc. The notification can include data indicating that an unauthorized user has been detected as well as collected sensor data and / or metadata generated from the sensor data used to detect the unauthorized user. For example, the notification can include image, video, and / or audio data as well as metadata, such as a time at which the sensor data was collected.

[0062] In some embodiments, the alert component 214 can enable an authorized user to manually cause any of the above-described functions. For example, the alert component 214 can present an alert button to an authorized user that the authorized user can select to cause any of the described functions, such as sounding an alarm, notifying security or police, etc. The alert button can be presented as a user interface element, such as a button, on a display of the client device 102. The authorized user can actuate the alert button, such as by clicking or tapping the alert button, to cause the associated function.

[0063] The resource restriction component 216 imposes restrictions on accessing controlled resources in response to detecting an unauthorized user. For example, the resource restriction component 216 can cause a window that includes sensitive information to close, cause an application to terminate, cause a user to be logged out of an account, obfuscate sensitive data, cause a door to lock, etc. In response to a determination that the unauthorized user is no longer in the physical environment, the resource restriction component 216 can remove the restrictions on accessing the controlled resources. For example, the resource restriction component 216 can log an authorized user into an account, re-open a window that presents sensitive data, unobfuscate obfuscated data, unlock a door, etc.

[0064] Figure 3is a flowchart illustrating a method 300 for modifying operation of a sensor using collected sensor data, in accordance with certain example embodiments. The method 300 can be embodied in computer-readable instructions for execution by one or more processors, such that the operations of the method 300 can be performed, in part or in whole, using Figure 1 the system 100 shown; accordingly, the method 300 will be described hereinafter in the context of the system 100 shown by way of example only. It should be appreciated, however, that at least some of the operations of the method 300 can be deployed in various other hardware configurations and the method 300 is not intended to be limited to the Figure 1 system 100 shown.

[0065] At operation 302, the authentication system 104 determines that the authorization requirements associated with accessing the controlled resource have been satisfied. A user communicates with the authentication system 104 to request access to the controlled resource. For example, the user uses the client device 102 to provide authentication credentials (e.g., a password, a code, a username / password, etc.) to the authentication system 104. The authentication system 104 determines whether the received authentication credentials satisfy the authentication requirements, and, if so, approves the authentication request.

[0066] At operation 304, the authentication system 104 grants the authorized user access to the controlled resource. For example, the authentication system 104 provides the requesting user with access to the controlled resource, or notifies an online service (e.g., an email service, a banking service, etc.) that the user has satisfied the authentication requirements using functionality of the authentication system 104.

[0067] At operation 306, the sensor data receiving component 204 collects sensor data describing the physical environment of the authorized user according to an initial operating mode. The sensor data receiving component 204 receives sensor data collected by various sensors 108. The sensors 108 can be any of a variety of types of sensors 108 for collecting sensor data. For example, the sensors can include optical sensors that capture images and / or video, audio sensors that capture sound, infrared sensors that capture infrared radiation and / or heat, etc. The initial operating mode can be configured to conserve computing resources, for example, by collecting sensor data at a lower resolution, storing and / or otherwise processing the sensor data at a reduced level, etc. In some embodiments, the initial operating mode can include using optical sensors to capture video and / or images and using infrared sensors to capture infrared radiation and / or heat.

[0068] At operation 308, the unauthorized user detection component 210 detects the presence of an unauthorized user in the physical environment based on the sensor data. The unauthorized user detection component 210 can use any of a variety of types and combinations of sensor data, such as video, images, sound, motion data, and / or infrared data. Further, the unauthorized user detection component 210 can use any of a variety of techniques to detect the presence of an unauthorized user, such as CNNs, r-CNNs, full-CNNs, Mask R-CNNs; single-step multi-box detector, edge finding (such as Gaussian difference); VGG-16, etc.

[0069] In some embodiments, the unauthorized user detection component 210 can determine values indicative of a degree or level of change detected in the physical environment based on the sensor data and compare the one or more values to a threshold to determine whether an unauthorized user is present. For example, the unauthorized user detection component 210 can use sensor data collected by a motion sensor to determine a motion value indicative of a level or degree of motion detected in the physical environment. Similarly, the unauthorized user detection component 210 can use sensor data collected by an infrared to determine an infrared radiation value indicative of a level or degree of infrared radiation change detected in the physical environment. As another example, the unauthorized user detection component 210 can use sensor data collected by a plurality of sensors 108 (e.g., video cameras, microphones, motion sensors, infrared sensors, etc.) to determine an aggregate value indicative of a degree or level of change detected in the physical environment.

[0070] In any case, the unauthorized user detection component 210 compares the value determined based on the sensor data to a threshold to determine whether the detected change is indicative of the presence of an unauthorized user. For example, the unauthorized user detection component 210 determines that an unauthorized user is present when the value exceeds the threshold and determines that an unauthorized user is not present when the value does not exceed the threshold.

[0071] At operation 310, the operational mode modification component 212 causes the sensors 108 to switch from the initial operational mode to the modified operational mode to collect sensor data. For example, the sensors can be adjusted to the modified operational mode that provides a higher level of performance, such as by increasing resolution, using additional sensors, etc. This allows for increased performance when valuable sensor data is being collected (e.g., when an unauthorized user is detected in the physical environment of an authorized user) while providing resource-conservative performance when less valuable sensor data is being collected (e.g., when no unauthorized user is detected in the physical environment of an authorized user).

[0072] Figure 4is a flowchart illustrating a method 400 for recovering sensor operation using collected sensor data, in accordance with certain example embodiments. The method 400 can be embodied in computer-readable instructions executed by one or more processors, such that the operations of the method 400 can be performed, in part or in whole, by Figure 1 the system 100 shown; accordingly, the method 400 will be described hereinafter in the context of the system 100 shown by way of example only. It should be appreciated, however, that at least some of the operations of the method 400 can be deployed in various other hardware configurations and the method 400 is not intended to be limited to the Figure 1 system 100 shown.

[0073] At operation 402, the unauthorized user detection component 210 detects the presence of an unauthorized user in the physical environment based on the sensor data. The unauthorized user detection component 210 can use any of a variety of types and combinations of sensor data, such as video, images, sound, motion data, and / or infrared data. Moreover, the unauthorized user detection component 210 can use any of a variety of techniques to detect the presence of an unauthorized user.

[0074] In some embodiments, the unauthorized user detection component 210 can determine one or more values indicative of a degree or level of change detected in the physical environment based on the sensor data and compare the one or more values to a threshold to determine whether an unauthorized user is present. For example, the unauthorized user detection component 210 can use sensor data collected by a motion sensor to determine a motion value indicative of a level or degree of motion detected in the physical environment. Similarly, the unauthorized user detection component 210 can use sensor data collected by an infrared sensor to determine an infrared radiation value indicative of a level or degree of infrared radiation change detected in the physical environment. As another example, the unauthorized user detection component 210 can use sensor data collected by a plurality of sensors 108 (e.g., video cameras, microphones, motion sensors, infrared sensors, etc.) to determine an aggregate value indicative of a degree or level of change detected in the physical environment.

[0075] In any case, the unauthorized user detection component 210 compares the value determined based on the sensor data to a threshold to determine whether the detected change is indicative of the presence of an unauthorized user. For example, the unauthorized user detection component 210 determines that an unauthorized user is present when the value exceeds the threshold and determines that an unauthorized user is not present when the value does not exceed the threshold.

[0076] At operation 404, the operating mode modification component 212 causes the sensor to switch from the initial operating mode to the modified operating mode to collect sensor data. For example, the sensor can be adjusted to the modified operating mode that provides a higher level of performance, such as by increasing resolution, using additional sensors, etc. This allows for increased performance when valuable sensor data is being collected (e.g., when an unauthorized user is detected in the authorized user's physical environment) while providing resource-conservative performance when less valuable sensor data is being collected (e.g., when an unauthorized user is not detected in the authorized user's physical environment).

[0077] At operation 406, the unauthorized user detection component 210 determines that the unauthorized user is no longer in the physical environment based on the sensor data. For example, the unauthorized user detection component 210 can determine that the detected change in the physical environment is no longer present and / or that a value determined based on the updated sensor data is below a threshold value.

[0078] At operation 408, the operating mode modification component 212 causes the sensor to revert from the modified operating mode to the initial operating mode to collect sensor data. For example, the sensor 108 can be adjusted to the initial operating mode that provides resource-conservative performance.

[0079] Figure 5 FIG. 5 is a flowchart illustrating a method 500 for performing an action using collected sensor data, in accordance with certain example embodiments. The method 500 can be embodied as computer-readable instructions executed by one or more processors, such that the operations of the method 500 can be performed, in part or in whole, by the monitoring system 110; thus, the method 400 is described below by way of example with reference thereto. However, it should be appreciated that at least some of the operations of the method 500 can be deployed on a variety of other hardware configurations and the method 500 is not intended to be limited to the monitoring system 110.

[0080] At operation 502, the unauthorized user detection component 210 detects the presence of an unauthorized user in the physical environment based on the sensor data. The unauthorized user detection component 210 can use any of a variety of types and combinations of sensor data, such as video, images, sound, motion data, and / or infrared data. Further, the unauthorized user detection component 210 can use any of a variety of techniques to detect the presence of an unauthorized user.

[0081] In some embodiments, the unauthorized user detection component 210 can determine, based on the sensor data, a value indicative of a degree or level of detected change in the physical environment and compare the one or more values to a threshold to determine whether an unauthorized user is present. For example, the unauthorized user detection component 210 can use sensor data collected by a motion sensor to determine a motion value indicative of a level or degree of detected motion in the physical environment. Similarly, the unauthorized user detection component 210 can use sensor data collected by an infrared sensor to determine a value indicative of a level or degree of detected change in infrared radiation in the physical environment. As another example, the unauthorized user detection component 210 can use sensor data collected by multiple sensors 108 (e.g., a video camera, a microphone, a motion sensor, an infrared sensor, etc.) to determine an aggregate value indicative of a degree or level of detected change in the physical environment.

[0082] In any case, the unauthorized user detection component 210 compares the value determined based on the sensor data to a threshold to determine whether the detected change is indicative of the presence of an unauthorized user. For example, the unauthorized user detection component 210 determines that an unauthorized user is present when the value exceeds the threshold and determines that an unauthorized user is not present when the value does not exceed the threshold.

[0083] At operation 504, the operating mode modification component 212 causes the sensor to switch from the initial operating mode to the modified operating mode to collect sensor data. For example, the sensor can be adjusted to the modified operating mode that provides a higher level of performance, such as by increasing resolution, using additional sensors, etc. This allows for increased performance when valuable sensor data is being collected (e.g., when an unauthorized user is detected in the physical environment of an authorized user) while providing resource-conservative performance when less valuable sensor data is being collected (e.g., when no unauthorized user is detected in the physical environment of an authorized user).

[0084] At operation 506, the resource restriction component 216 restricts access to the controlled resource. For example, the resource restriction component 216 can cause a window that includes sensitive information to close, cause an application to terminate, cause a user to be logged out of an account, obfuscate sensitive data, cause a door to lock, etc.

[0085] At operation 508, the unauthorized user detection component 210 determines, based on the sensor data, that the unauthorized user is no longer in the physical environment. For example, the unauthorized user detection component 210 can determine that the detected change in the physical environment is no longer present and / or that a value determined based on updated sensor data is below a threshold.

[0086] At operation 510, the operating mode modification component 212 causes the sensor to recover from the modified operating mode to the initial operating mode to collect sensor data. For example, the sensor 108 can be adjusted to the initial operating mode that provides resource-conservative performance.

[0087] At operation 512, the resource restriction component 216 restores access to the controlled resource. For example, the resource restriction component 216 can log in an authorized user to an account, reopen a window presenting sensitive data, clarify obfuscated data, unlock a door, and so forth.

[0088] Software Architecture

[0089] Figure 6 is a block diagram that illustrates an example software architecture 606, which can be used in conjunction with the various hardware architectures described herein. Figure 6 is a non-limiting example of software architecture 606, and it will be appreciated that many other architectures can be implemented to facilitate the functionality described herein. The software architecture 606 can be executing on machines such as Figure 7 machine 700, including processors 704, memory 714, and (input / output) I / O components 718, among other components. The representative hardware layer 652 is illustrated and can represent, for example, Figure 7 representative hardware layer 652 includes a processing unit 654 having associated executable instructions 604. Executable instructions 604 represent the executable instructions of the software architecture 606, including implementation of the methods, components, and so on described herein. The hardware layer 652 also includes memory and / or storage modules 656, which also have executable instructions 604. The hardware layer 652 can also include other hardware 658.

[0090] In Figure 6 example architecture, the software architecture 606 can be conceptualized as a stack of layers, where each layer provides particular functionality. For example, the software architecture 606 can include layers such as an operating system 602, libraries 620, frameworks / middleware 618, applications 616, and a presentation layer 614. Operationally, the applications 616 and / or other components within the layers can invoke application programming interface (API) calls 608 through the software stack and receive a response, such as a message 612 in response to the API call 608. The layers illustrated are representative in nature and not all software architectures have all layers. For example, some mobile or special purpose operating systems can not provide a frameworks / middleware 618, while others can provide such a layer. Other software architectures can include additional or different layers.

[0091] The operating system 602 can manage hardware resources and provide common services. The operating system 602 can include, for example, a kernel 622, services 624 and drivers 626. The kernel 622 can act as an abstraction layer between the hardware and the other software layers. For example, the kernel 622 can be responsible for memory management, processor management (for example, scheduling), component management, networking, security settings, and so on. The services 624 can provide other common services for the other software layers. The drivers 626 are responsible for controlling or interfacing with the underlying hardware, depending on the implementation. For instance, the drivers 626 can include display drivers, camera drivers,, flash memory drivers, serial communication drivers (for example, Universal Serial Bus (USB) drivers), audio drivers, power management drivers, and so on, depending on the hardware configuration.

[0092] The libraries 620 provide a common infrastructure that can be used by the applications 616 and / or other software components and / or layers. The libraries 620 provide functionality that allows other software components to perform tasks without having to perform implementation-specific tasks themselves. The libraries 620 can include system libraries 644, for example, that can provide functions for handling basic system and / or

[0093] The frameworks / middleware 618 (also sometimes referred to as middleware) provide a higher-level common infrastructure that can be used by the applications 616 and / or other software components and modules. For example, the frameworks / middleware 618 can provide various graphical user interface (GUI) functions, high-level resource management, high-level location services, and so on. The frameworks / middleware 618 can provide a broad spectrum of other APIs that can be used by the applications 616 and / or other software components / modules, some of which can be specific to a particular operating system 602 or platform.

[0094] ​The applications 616 include built-in applications 638 and / or third-party applications 640. Examples of representative built-in applications 638 can include, but are not limited to, a contacts application, a browser application, a book reader application, a location application, a media application, a messaging application, and / or a game application. Third-party applications 640 can include the applications that are developed using the ANDROID TM or IOS TM software development kit (SDK) by an entity other than the vendor of the particular platform, and can be mobile software running on a mobile operating system such as IOS TM , ANDROID TM , , WINDOWS PHONE, or other mobile operating systems. The third-party applications 640 can invoke the API calls 608 provided by the mobile operating system (such as operating system 602) to facilitate functionality described herein.

[0095] The applications 616 can use built-in operating system functions (such as kernel 622, services 624 and / or drivers 626), libraries 620, and frameworks / middleware 618 to create the UI for interaction with a user of the system. Alternatively, or additionally, in some systems, interaction with a user can occur through a representation layer, such as representation layer 614. In these systems, the application / component "logic" can be separated from the aspects of the applications / components that interact with a user.

[0096] Figure 7 is a block diagram illustrating components of a machine 700, according to some example embodiments, able to read instructions 604 from a machine-readable medium (for example, a machine-readable storage medium) and perform any one or more of the methodologies discussed herein. Specifically, the machine 700 can be a mobile device (for example, a cellular phone, a smart phone, a tablet, a smart watch, a wearable, a gaming console, a media player, a navigation device, an Internet of Things (IoT) device, a gaming device, a television, a computer, and so forth), a desktop computer, a laptop computer, an Internet server, a network appliance, a computer system, and so forth. The machine 700 can be Figure 7The illustrative depiction of machine 700 is in the form of a computer system, within which instructions 710 (e.g., software, programs, applications, applets, application software, or other executable code) for causing the machine 700 to perform any one or more of the methodologies discussed herein, can be executed. Thus, the instructions 710 can be used to implement units or components described herein. The instructions 710 transform the general, non-programmed machine 700 into a particular machine 700 programmed to carry out the described and illustrated functions. In alternative embodiments, the machine 700 operates as a standalone device or can be coupled (e.g., networked) to other machines. In a networked deployment, the machine 700 can operate in the capacity of a server machine or a client machine in server-client network environments, or it can act as a peer machine in peer-to-peer (or distributed) network environments. The machine 700 can comprise, but not be limited to, a server computer, a client computer, PC, a tablet computer, a notebook computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular telephone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions 710, sequentially or otherwise, that specify actions to be taken by the machine 700. Further, while only a single machine 700 is illustrated, the term “machine” shall also be taken to include a collection of machines 700 that individually or jointly execute the instructions 710 to perform any one or more of the methodologies discussed herein.

[0097] The machine 700 can include processors 704, memory / storage 706, and I / O components 718, which can be configured to communicate with each other such as via a bus 702. The memory / storage 706 can include a memory 714, such as a main memory, or other memory storage, and a storage unit 716, both accessible to the processors 704 such as via the bus 702. The storage unit 716 and memory 714 store the instructions 710 embodying any one or more of the methodologies or functions described herein. The instructions 710 can also reside completely, or partially, within at least one of the memories 714, within the storage unit 716, within at least one of the processors 704 (e.g., within the cache memory of the processors), or any suitable combination thereof, during execution thereof by the machine 700. Accordingly, memories 714, storage unit 716, and the memories of the processors 704 are examples of machine-readable media.

[0098] I / O component 718 may include a wide variety of components to receive input, provide output, generate output, transmit information, exchange information, capture measurement results, etc. The specific I / O component 718 included in a particular machine 700 will depend on the type of machine. For example, a portable machine such as a mobile phone may include a touch input device or other such input mechanism, while a headless server machine may not include such a touch input device. It should be understood that I / O component 718 may include... Figure 7 Many other components are not shown. The grouping of I / O components 718 according to function is merely for the purpose of simplifying the following discussion and is by no means limiting. In various example embodiments, I / O components 718 may include output components 726 and input components 728. Output components 726 may include visual components (e.g., displays such as plasma display panels (PDPs), light-emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tubes (CRTs)), acoustic components (e.g., speakers), haptic components (e.g., vibration motors, resistance mechanisms), other signal generators, etc. Input components 728 may include alphanumeric input components (e.g., keyboards, touchscreens configured to receive alphanumeric input, photoelectric keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touchpads, trackballs, joysticks, motion sensors, or other pointing instruments), haptic input components (e.g., physical buttons, touchscreens providing position and / or force for touch or touch gestures, or other haptic input components), audio input components (e.g., microphones), etc.

[0099] In further example embodiments, I / O component 718 may include biometric component 730, motion component 734, environmental component 736, or positioning component 738, as well as a wide range of other components. For example, biometric component 730 may include components for detecting facial expressions (e.g., hand expressions, facial expressions, vocal expressions, body posture, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), and recognizing a person (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition). Motion component 734 may include accelerometer components (e.g., accelerometers), gravity sensor components, rotation sensor components (e.g., gyroscopes), etc. Environmental component 736 may include, for example, an illuminance sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers for detecting ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones for detecting background noise), a proximity sensor component (e.g., an infrared sensor for detecting nearby objects), a gas sensor (e.g., a gas detection sensor for detecting the concentration of hazardous gases to ensure safety or to measure pollutants in the atmosphere), or other components that may provide indications, measurements, or signals corresponding to the surrounding physical environment. Positioning component 738 may include a position sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer for detecting air pressure from which altitude can be derived), an orientation sensor component (e.g., a magnetometer), and so on.

[0100] A wide variety of technologies can be used to implement communication. I / O component 718 may include communication component 740, operable to couple machine 700 to network 732 or device 720 via coupling 724 and coupling 722, respectively. For example, communication component 740 may include a network interface component or other suitable device interfaced with network 732. In further examples, communication component 740 may include wired communication component, wireless communication component, cellular communication component, near field communication (NFC) component, etc. Components (e.g., Low Energy) Components and other communication components to provide communication in other ways. Device 720 can be another machine or any of a variety of peripheral devices (e.g., a peripheral device coupled via USB).

[0101] Moreover, the communication components 740 can detect identifiers or include components operable to detect identifiers. For example, the communication components 740 can include radio frequency identification (RFID) tag reader components, NFC smart tag detection components, optical reader components (e.g., an optical sensor to detect one-dimensional bar codes such as Universal Product Code (UPC) bar codes, multi-dimensional bar codes such as Quick Response (QR) codes, Aztec codes, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D bar codes, and other optical codes), or acoustic detection components (e.g., microphones to identify tagged audio signals). In addition, a variety of information can be derived via the communication components 740, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi® signal triangulation, location via cellular signal triangulation, location via detection of an NFC beacon signal that can indicate a specific location, and so forth.

[0102] “Carrier signal” herein refers to any intangible medium that is capable of storing, encoding, or carrying the instructions 710 for execution by the machine 700, and includes digital or analog communications signals or other intangible media to facilitate communication of such instructions 710. Instructions 710 can be transmitted or received by the machine 700 via the network interface device using any one of a number of well-known transfer protocols (e.g., HTTP).

[0103] “Client device” in this context refers to any machine 700 that interfaces to a communications network 732 to access resources on the server system or other client devices 102. A client device 102 can be, but is not limited to, a mobile phone, desktop computer, laptop computer, PDAs, smart phones, tablets, ultrabooks, netbooks, laptops, multi-processor systems, microprocessor-based or programmable consumer electronics, game

[0104] “Communication network” herein refers to one or more portions of the network 732 that can be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a LAN, a wireless LAN (WLAN), a WAN, a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of the Public Switched Telephone Network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, a Wi-Fi® network, another type of network, or a combination of two or more such networks. The network 732, or a portion thereof, can include a wireless or cellular network, and the coupling can be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or other types of cellular or wireless couplings. In this example, the coupling can implement any of a variety of types of data transfer technology, such as Single Carrier Radio Transmission Technology (lxRTT), Evolution-Data Optimized (EVDO) technology, General Packet Radio Service (GPRS) technology, Enhanced Data Rates for GSM Evolution (EDGE) technology, third Generation Partnership Project (3GPP) including 3G, fourth generation wireless (4G), Universal Mobile Telecommunications System (UMTS), High Speed Packet Access (HSPA), Worldwide Interoperability for Microwave Access (WiMAX), Long Term Evolution (LTE) standards, other standards defined by various standard setting organizations, other radio

[0105] "machine-readable medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) able to store instructions 710 that are executed by the machine 700, such that the instructions 710, when executed by one or more processors 704 of the machine 700, enable the machine 700 to perform any one or more of the methodologies discussed herein. Accordingly, a "machine-readable medium" is taken to include single storage devices or devices spanning multiple storage devices, and storage media corresponding to both memory or storage that is centralized or distributed, remote or in situ, and tangible or non-tangible. Thus, "machine-readable medium" is taken to include one or more physical devices configured to store the instructions 710, use of which by the machine 700 enables the machine 700 to implement the one or more methods discussed herein. The term "machine-readable medium" is taken to include a non- transitory machine-readable medium, in addition to other types of tangible machine-readable media.

[0106] "Component" in the present document refers to a device, physical entity, or logic, whose boundaries are defined by a function or a sub-routine, a branch point, an API, or a partitioning, or modularizing of a larger function, depending on the context in which it is used. The component can be combined with other components in a machine to perform a machine process. The component can be a hardware unit that is packaged functional hardware, designed for use with other components, and a part of a program that performs a specific function, typically related to a certain function. The component can constitute a software component (e.g., code embodied on a machine-readable medium) or a hardware component. A "hardware component" is a tangible unit capable of performing certain operations and can be configured or arranged in a certain physical manner. In various example embodiments, one or more computer systems (e.g., a standalone computer system, a client computer system, or a server computer system) or one or more hardware components of a computer system (e.g., a processor or a group of processors 704) can be configured by software (e.g., an application or application portion) as a hardware component that operates to perform certain operations as described herein. A hardware component can also be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware component can include dedicated circuitry or logic that is permanently configured to perform certain operations. A hardware component can be a special-purpose processor, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). A hardware component can also include programmable logic or circuitry that is temporarily configured by software to perform certain operations. For example, a hardware component can include software executed by a general-purpose processor 704 or other programmable processor 704 to configure the hardware component in a particular manner. Once configured, hardware components can cease to be programmable or they can be reconfigured to perform other tasks. A hardware component can be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware component can include dedicated circuitry or logic that is permanently configured to perform certain operations. A hardware component can also include programmable logic or circuitry that is temporarily configured by software to perform certain operations. Aspects of hardware components can be "hard-coded" or "fixed" in that they are not temporarily configurable (for example, by software) but are either hardwired or otherwise permanently configured. Aspects of hardware components can be "soft-coded" or "flexible" in that they are temporarily configurable (for example, by software) to perform one or more particular operations.Where there are multiple hardware components present at the same time, communication can be achieved through signal transmission (e.g., over appropriate circuits and buses 702) between two or more of the hardware components. In embodiments where multiple hardware components are configured or instantiated at different times, communication between such hardware components can for example be achieved through the storage and retrieval of information in memory structures to which the multiple hardware components have access. For example, one hardware component can perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. A further hardware component can then access the memory device at a later time to retrieve and process the stored output. Hardware components can also initiate communications with input or output devices, and can operate on resources (e.g., information collections). The various operations of example methods described herein can be performed, at least partially, by one or more processors 704 that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors 704 can constitute processor-implemented components that operate to perform one or more operations or functions described herein. As used herein,“processor- implemented component” refers to a hardware component implemented with one or more processors 704. Similarly, the methods described herein can be at least partially processor- implemented, with a processor or processors 704 being an example of hardware. For example, at least some of the operations of a method can be performed by one or more processors 704 or processor-implemented components. Moreover, a processor or processors 704 can also operate to support performance of the relevant operations in a“cloud computing” environment or as a“software as a service” (SaaS). For example, at least some of the operations can be performed by a group of computers (as examples of machines 700 including processors 704), with these operations being accessible via a network 732 (e.g., the Internet) and via one or more appropriate interfaces (e.g., an API). The performance of certain of the operations can be distributed among the processors 704, not only residing in a single machine 700, but deployed across a number of machines 700. In some example embodiments, the processors 704 or processor-implemented components can be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other example embodiments, the processors 704 or processor-implemented components can be distributed across multiple geographic locations.

[0107] A "processor" herein refers to any circuit or virtual circuit (physical circuit emulated by logic executing on the actual processor 704) that manipulates data values according to control signals (e.g., "commands," "op codes," "machine code," etc.) and that produces resulting output signals that are used by the machine 700. A processor 704 can be, for example, a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, a radio-frequency integrated circuit (RFIC), or any combination thereof. A processor 704 can also be a multi-core processor having two or more independent processors 704 (sometimes referred to as "cores") that can execute instructions 710 contemporaneously.

[0108] Non-limiting examples

[0109] Example 1 is a system that modifies an operational mode of one or more sensors using collected sensor data, the modification based on detection of an additional user in a physical environment, the system comprising: means for providing a first user with access to an access-controlled resource in response to determining that an authorization requirement associated with the access-controlled resource has been satisfied; means for collecting sensor data describing a physical environment of the first user according to a first operational mode, the first operational mode defining an initial performance of at least one sensor used to collect the sensor data; means for detecting a presence of a second user in the physical environment of the first user that is different from the first user based on the sensor data collected according to the first operational mode; and means for causing the at least one sensor to switch from the first operational mode to a second operational mode in response to detecting the presence of the second user, wherein the second operational mode is different from the first operational mode and the second operational mode defines a revised performance of the at least one sensor used to collect the sensor data.

[0110] In Example 2, the subject matter of Example 1 includes, wherein the sensor data is thermal imaging data collected using a thermal imaging sensor, and detecting the presence of the second user comprises: determining, based on the thermal imaging data, that thermal energy in excess of a thermal energy threshold level is present.

[0111] In Example 3, the subject matter of Examples 1-2 includes, wherein causing the at least one sensor to switch from the first operational mode to the second operational mode comprises: modifying a resolution level of an optical sensor from a first resolution level to a second resolution level, the second resolution level being higher than the first resolution level.

[0112] In Example 4, the subject matter of Examples 1-3 includes, wherein causing the at least one sensor to switch from the first operational mode to the second operational mode includes: modifying an optical view of an optical sensor from a first optical view that captures a first view of the physical environment to a second optical view that captures a second view of the physical environment, the second view being different from the first view.

[0113] In Example 5, the subject matter of Examples 1-4 includes, wherein the second view includes a first portion of the physical environment that is not included in the first view of the physical environment.

[0114] In Example 6, the subject matter of Examples 1-5 includes, wherein the second view of the physical environment is a subset of the first view of the physical environment.

[0115] In Example 7, the subject matter of Examples 1-6 includes, wherein causing the at least one sensor to switch from the first operational mode to the second operational mode includes: starting to collect video data using an optical sensor that was not already in use in the first operational mode.

[0116] In Example 8, the subject matter of Examples 1-7 includes: means for determining that the second user is not present in the physical environment of the first user based on the sensor data collected according to the second operational mode; and means for causing the at least one sensor to revert from the second operational mode to the first operational mode in response to determining that the second user is not present in the physical environment of the first user.

[0117] In Example 9, the subject matter of Examples 1-8 includes: means for determining a baseline state of the physical environment of the first user based on sensor data collected according to the first operational mode, wherein detecting a presence of a second user different from the first user in the physical environment is based on a detected change in the baseline state of the physical environment.

[0118] In Example 10, the subject matter of Examples 1-9 includes: means for restricting access to the access-controlled resource in response to detecting the presence of the second user.

[0119] In Example 11, the subject matter of Examples 1-10 includes: means for causing presentation of an alert button that, when activated, causes an alert notification to be sent to a designated recipient.

[0120] Example 12 is a method of modifying an operational mode of one or more sensors using collected sensor data, the modification based on detection of an additional user in a physical environment, the method comprising: providing a first user with access to an access-controlled resource in response to determining that an authorization requirement associated with the access-controlled resource has been satisfied; collecting sensor data describing a physical environment of the first user in accordance with a first operational mode, the first operational mode defining an initial performance of at least one sensor used to collect the sensor data; detecting a presence of a second user, different from the first user, in the physical environment of the first user based on sensor data collected in accordance with the first operational mode; and causing the at least one sensor to switch from the first operational mode to a second operational mode in response to detecting the presence of the second user, wherein the second operational mode is different from the first operational mode and the second operational mode defines a revised performance of the at least one sensor used to collect the sensor data.

[0121] In Example 13, the subject matter of Example 12 includes, wherein the sensor data is thermal imaging data collected using a thermal imaging sensor, and detecting the presence of the second user comprises: determining, based on the thermal imaging data, that thermal energy in excess of a thermal energy threshold level is present.

[0122] In Example 14, the subject matter of Examples 12-13 includes, wherein causing the at least one sensor to switch from the first operational mode to the second operational mode comprises: modifying a resolution level of an optical sensor from a first resolution level to a second resolution level that is higher than the first resolution level.

[0123] In Example 15, the subject matter of Examples 12-14 includes, wherein causing the at least one sensor to switch from the first operational mode to the second operational mode comprises: modifying an optical view of an optical sensor from a first optical view that captures a first view of the physical environment to a second optical view that captures a second view of the physical environment, the second view being different from the first view.

[0124] In Example 16, the subject matter of Examples 12-15 includes, wherein the second view comprises a first portion of the physical environment that is not included in the first view of the physical environment.

[0125] In Example 17, the subject matter of Examples 12-16 includes, wherein the second view of the physical environment is a subset of the first view of the physical environment.

[0126] In Example 18, the subject matter of Examples 12-17 includes, wherein causing the at least one sensor to switch from the first operational mode to the second operational mode comprises: beginning to collect video data using an optical sensor that was not being used in the first operational mode.

[0127] In Example 19, the subject matter of Examples 12-18 includes, determining that the second user is not present in the physical environment of the first user based on the sensor data collected according to the second operational mode; and in response to determining that the second user is not present in the physical environment of the first user, causing the at least one sensor to revert from the second operational mode to the first operational mode.

[0128] In Example 20, the subject matter of Examples 12-19 includes: determining a baseline state of the physical environment of the first user based on the sensor data collected according to the first operational mode, wherein detecting the presence of a second user in the physical environment different from the first user is based on a detected change in the baseline state of the physical environment.

[0129] In Example 21, the subject matter of Examples 12-20 includes, in response to detecting the presence of the second user, restricting access to the access-controlled resource.

[0130] In Example 22, the subject matter of Examples 12-21 includes causing presentation of an alert button that, when activated, causes an alert notification to be sent to a designated recipient.

[0131] Example 22 is a computing device for modifying an operational mode of one or more sensors using collected sensor data, the modification based on detection of an additional user in a physical environment, the computing device comprising: one or more hardware processors; memory storing instructions that, when executed, cause the one or more hardware processors to perform operations comprising: in response to determining that an authorization requirement associated with an access-controlled resource has been satisfied, providing a first user with access to the access-controlled resource; collecting sensor data describing a physical environment of the first user according to a first operational mode, the first operational mode defining an initial performance of at least one sensor used to collect the sensor data; detecting a presence of a second user in the physical environment of the first user different from the first user based on the sensor data collected according to the first operational mode; and in response to detecting the presence of the second user, causing the at least one sensor to switch from the first operational mode to a second operational mode, wherein the second operational mode is different from the first operational mode and the second operational mode defines a revised performance of the at least one sensor used to collect the sensor data.

[0132] In Example 23, the subject matter of Example 22 includes, wherein the sensor data is thermal imaging data collected using a thermal imaging sensor, and detecting the presence of the second user includes: determining, based on the thermal imaging data, an occurrence of thermal energy that exceeds a threshold level of thermal energy.

[0133] In Example 24, the subject matter of Examples 22-23 includes, wherein causing the at least one sensor to switch from the first operational mode to the second operational mode includes: modifying a resolution level of an optical sensor from a first resolution level to a second resolution level that is higher than the first resolution level.

[0134] In Example 25, the subject matter of Examples 22-24 includes, wherein causing the at least one sensor to switch from the first operational mode to the second operational mode includes: modifying an optical view of an optical sensor from a first optical view that captures a first view of the physical environment to a second optical view that captures a second view of the physical environment, the second view being different from the first view.

[0135] In Example 26, the subject matter of Examples 22-25 includes, wherein the second view includes a first portion of the physical environment that is not included in the first view of the physical environment.

[0136] In Example 27, the subject matter of Examples 22-26 includes, wherein the second view of the physical environment is a subset of the first view of the physical environment.

[0137] In Example 28, the subject matter of Examples 22-27 includes, wherein causing the at least one sensor to switch from the first operational mode to the second operational mode includes: beginning to collect video data using an optical sensor that was not being used in the first operational mode.

[0138] In Example 29, the subject matter of Examples 22-28 includes, based on the sensor data collected according to the second operational mode, determining that the second user is not present in the physical environment of the first user; and in response to determining that the second user is not present in the physical environment of the first user, causing the at least one sensor to revert from the second operational mode to the first operational mode.

[0139] In Example 30, the subject matter of Examples 22-29 includes, based on the sensor data collected according to the first operational mode determining a baseline state of the physical environment of the first user, wherein detecting the presence of the second user in the physical environment different from the first user is based on a detected change in the baseline state of the physical environment.

[0140] In Example 31, the subject matter of Examples 22-30 includes, in response to detecting the presence of the second user, restricting access to access a controlled resource.

[0141] In Example 32, the subject-matter of Examples 22-31 includes causing presentation of an alert button that, when activated, causes sending of an alert notification to a designated recipient.

[0142] Example 33 is at least one machine readable medium including instructions that, when executed by processing circuitry, cause the processing circuitry to perform operations to implement any of Examples 1-32.

[0143] The above description includes references to the accompanying drawings, which form a part of the detailed description. The drawings show, by way of illustration, specific embodiments in which the application can be practiced. These embodiments are also referred to as "examples." Such examples can include elements in addition to those shown or described. However, the present inventors also contemplate examples in which only those elements shown or described are provided. Moreover, the present inventors also contemplate examples using any combination or permutation of those elements shown or described (or one or more aspects thereof), either with respect to a particular example (or one or more aspects thereof), or with respect to other examples (or one or more aspects thereof) shown or described herein.

[0144] In this document, the terms "a" or "an" are used, as is common in patent documents, to include one or more, independent of any other instances or usages of "at least one" or "one or more." In this document, the term "or" is used to refer to a nonexclusive or, such that "A or B" includes "A but not B," "B but not A," and "A and B," unless otherwise indicated. In this document, the terms "including" and "in which" are used as the plain-English equivalents of the respective terms "comprising" and "wherein." Also, in the following claims, the terms "including" and "comprising" are open-ended, that is, a system, device, article, composition, formulation, or process that includes elements in addition to those listed after such a term in a claim are still deemed to fall within the scope of that claim. Moreover, in the following claims, the terms "first," "second," and "third," etc. are used merely as labels, and are not intended to impose numerical requirements on their objects.

[0145] The above description is illustrative and not restrictive. For example, the above examples (or one or more aspects thereof) can be used in combination with each other. Other embodiments can be used as well, e.g., by one of ordinary skill in the art upon reading the above description. The Abstract is provided to allow a quick determination of the disclosure's purpose. The submission of the Abstract is intended to facilitate expedited access to that summary information for the patent drawing and the claims. It is not intended to limit the scope or the meaning of the claims or to predispose the patentability of the claimed embodiments. Furthermore, in the above Detailed Description, various features are grouped together in examples for purposes of streamlining the disclosure. This should not be interpreted as a requirement that the claimed subject matter necessarily have more features than are explicitly described. Rather, inventive subject matter can lie in less than all features of a particular disclosed embodiment. Thus, the following claims are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment. The scope of the application should be determined, not by the Abstract, but by the appended claims and their full scope of equivalents.

Claims

1. A system for modifying the operating mode of one or more sensors using collected sensor data, the modification being based on the detection of an additional user in a physical environment, the system comprising: A unit for providing access to the controlled resource to a first user in response to determining that the authorization requirements associated with accessing the controlled resource have been met; A unit for collecting sensor data describing the physical environment of the first user according to a first operating mode, wherein the first operating mode defines the initial performance of at least one sensor for collecting the sensor data. A unit for detecting the presence of a second user, different from the first user, in the physical environment of the first user based on the sensor data collected according to the first operating mode; as well as A unit for switching at least one sensor from a first operating mode to a second operating mode in response to detecting the presence of the second user, wherein the second operating mode is different from the first operating mode, and the second operating mode defines the correction performance of the at least one sensor for collecting the sensor data.

2. The system according to claim 1, wherein, The sensor data is thermal imaging data collected using a thermal imaging sensor, and detecting the presence of the second user includes: Based on the thermal imaging data, thermal energy exceeding the thermal energy threshold level is determined.

3. The system according to claim 1, wherein, Switching the at least one sensor from the first operating mode to the second operating mode includes: The resolution level of the optical sensor is modified from a first resolution level to a second resolution level, where the second resolution level is higher than the first resolution level.

4. The system according to claim 1, wherein, Switching the at least one sensor from the first operating mode to the second operating mode includes: The optical view of the optical sensor is modified from a first optical view that captures a first view of the physical environment to a second optical view that captures a second view of the physical environment, the second view being different from the first view.

5. The system according to claim 4, wherein, The second view includes a first portion of the physical environment that is not included in the first view of the physical environment.

6. The system according to claim 4, wherein, The second view of the physical environment is a subset of the first view of the physical environment.

7. The system according to claim 1, wherein, Switching the at least one sensor from the first operating mode to the second operating mode includes: An optical sensor is used to collect video data, which has not yet been used in the first operating mode.

8. The system according to claim 1, further comprising: A unit for determining, based on the sensor data collected according to the second operating mode, that the second user does not exist in the physical environment of the first user; as well as A unit for reverting at least one sensor from the second operating mode to the first operating mode in response to determining that the second user does not exist in the physical environment of the first user.

9. The system according to claim 1, further comprising: A unit for determining the baseline state of the physical environment of the first user based on the sensor data collected according to the first operating mode, wherein detecting the presence of a second user in the physical environment that is different from the first user is based on the detected change in the baseline state of the physical environment.

10. The system according to claim 1, further comprising: A unit that restricts access to the access-controlled resource in response to detecting the presence of the second user.

11. The system according to claim 1, further comprising: A unit for causing an alarm button to be displayed, which, when activated, causes an alarm notification to be sent to a designated recipient.

12. A method for modifying the operating mode of one or more sensors using collected sensor data, the modification being based on the detection of an additional user in a physical environment, the method comprising: In response to determining that the authorization requirements associated with accessing the controlled resource have been met, access to the controlled resource is granted to the first user; Sensor data describing the physical environment of the first user is collected according to a first operating mode, wherein the first operating mode defines the initial performance of at least one sensor for collecting the sensor data. The presence of a second user, different from the first user, in the first user's physical environment is detected based on the sensor data collected according to the first operating mode. as well as In response to detecting the presence of the second user, the at least one sensor switches from the first operating mode to a second operating mode, wherein the second operating mode is different from the first operating mode, and the second operating mode defines the correction performance of the at least one sensor for collecting the sensor data.

13. The method according to claim 12, wherein, The sensor data is thermal imaging data collected using a thermal imaging sensor, and detecting the presence of the second user includes: Based on the thermal imaging data, thermal energy exceeding the thermal energy threshold level is determined.

14. The method according to claim 12, wherein, Switching the at least one sensor from the first operating mode to the second operating mode includes: The resolution level of the optical sensor is modified from a first resolution level to a second resolution level that is higher than the first resolution level.

15. A computing device, comprising: One or more hardware processors; as well as A memory that stores instructions, when executed, to cause the one or more hardware processors to perform operations including the method described in any one of claims 12-14.

Images