Determining presence and absence of user using WiFi connection

Abstract

The invention discloses determining the presence and absence of a user using a WiFi connection. Systems and techniques are provided for determining the presence and absence of a user using a WiFi connection. A report may be received from a WiFi access point in an environment. The report may include an identifier of the WiFi device, an indication of connection or disconnection with the WiFi access point, a time of connection or disconnection, and an identifier of the WiFi access point. A connection sequence for the WiFi device may be generated from the report. Whether the WiFi device is present or not present in the environment up to a specified time may be determined based on the connection sequence. If the WiFi device is present in the environment, an indication of the presence of a user associated with the WiFi device may be generated. If the WiFi device is not present in the environment, an indication of absence of a user associated with the WiFi device may be generated.

Description

Description of the case

[0001] This application is a divisional application of Chinese invention patent application 202180034786.7, with a filing date of March 10, 2021. Technical Field

[0002] The present disclosure relates to determining user presence and absence using a WiFi connection. Background Art

[0003] Being able to determine the presence or absence of people in an environment such as a home, office, or other structure may be useful. The environment may include devices and systems that may be automated and may operate based on the presence or absence of individuals within the environment. Summary of the invention

[0004] According to an embodiment of the disclosed subject matter, a report may be received from a WiFi access point in an environment. The report may include an identifier of a WiFi device, an indication of connection or disconnection with one of the WiFi access points, the time of the connection or disconnection, and an identifier of the one of the WiFi access points. A connection sequence of the WiFi device may be generated from the report. The connection sequence may include a time of connection to the WiFi access point by the WiFi device and a time of disconnection from the WiFi access point. It may be determined whether the WiFi device exists in the environment or not in the environment as of a specified time based on a time period of connection to any one WiFi access point and a time period of disconnection from all WiFi access points in the connection sequence. If it is determined that the WiFi device exists in the environment, an indication of existence may be generated for a user associated with the WiFi device, or if it is determined that the WiFi device does not exist in the environment, an indication of non-existence may be generated for the user associated with the WiFi device.

[0005] A control signal for a controllable device in the environment may be generated based on the indication of presence or the indication of absence. The control signal may be sent to the device to be implemented by the device.

[0006] By determining that the WiFi device exists in the environment if an amount of time between a start of one of time periods connected to any one of the WiFi access points and the designated time is greater than a first threshold amount of time and the one of the time periods of connection includes the designated time, or if a total amount of time between a start of a first time period of two or more consecutive time periods connected to any one of the WiFi access points and the designated time is greater than the first threshold and a last time period of the two or more consecutive time periods includes the designated time, determining whether the WiFi device exists in the environment as of the designated time based on a time period of connection to the WiFi access point and a time period of disconnection from the WiFi access point in the connection sequence.

[0007] By determining that the WiFi device is not present in the environment if an amount of time between starts of one of the time periods disconnected from all the WiFi access points is greater than a second threshold amount of time and the one of the time periods disconnected includes the designated time, determining whether the WiFi device is present in the environment or not as of the designated time based on a time period of connection to the WiFi access point and a time period of disconnection from the WiFi access point in the connection sequence.

[0008] The identifier of the WiFi device may be a Salted Hashed Media Access Control address (SHMAC).

[0009] Prior to determining whether the WiFi device is present or absent in the environment as of a specified time based on a time period of connection to any one WiFi access point and a time period of disconnection from all WiFi access points in the connection sequence: connection time data, connection / disconnection count data, conversion data, and connection / disconnection length data of the WiFi device may be generated from the report; sensor and device data may be received from sensors or devices in the environment; and a WiFi device indication may be generated using a machine learning system, the WiFi device indication indicating that the WiFi device should be used to determine the presence or absence of the user associated with the WiFi device in the environment, wherein the connection time data, the connection / disconnection count data, the conversion data, and the connection / disconnection length data of the WiFi device and the sensor and device data are input into the machine learning system.

[0010] Additional reports may be received from the WiFi access points in the environment. The additional reports may include an identifier of a second WiFi device, an indication of connection or disconnection with one of the WiFi access points, a time of the connection or disconnection, and an identifier of the one of the WiFi access points. Second connection time data, second connection / disconnection count data, second conversion data, and second connection / disconnection length data for the second WiFi device may be generated from the additional reports; and a second WiFi device indication may be generated using the machine learning system. The second WiFi device indication indicates that the second WiFi device should not be used to determine the presence or absence of a second user associated with the second WiFi device in the environment, wherein the second connection time data, the second connection / disconnection count data, the second conversion data, and the second connection / disconnection length data of the second WiFi device and the sensor and device data are input to the machine learning system.

[0011] Device data may be received from the WiFi device. The device data may be input into the machine learning system.

[0012] The device data may include geographic location data of the WiFi device and / or geo-fence data of the WiFi device.

[0013] According to an embodiment of the disclosed subject matter, there is included: means for receiving reports from WiFi access points in an environment, wherein each of the reports includes an identifier of a WiFi device, an indication of connection or disconnection with one of the WiFi access points, a time of the connection or disconnection, and an identifier of the one of the WiFi access points; means for generating a connection sequence of the WiFi device from the reports, wherein the connection sequence includes a time of connection to the WiFi access point by the WiFi device and a time of disconnection from the WiFi access point; means for determining whether the WiFi device is present or absent in the environment as of a specified time based on a time period of connection to any one WiFi access point and a time period of disconnection from all WiFi access points in the connection sequence; means for generating an indication of presence for a user associated with the WiFi device if it is determined that the WiFi device is present in the environment, or generating an indication of absence for the user associated with the WiFi device if it is determined that the WiFi device is not present in the environment; means for generating a control signal for a controllable device in the environment based on the indication of presence or the indication of absence; means for sending the control signal to the device for implementation by the device; means for determining whether the WiFi device is present in the environment as of a specified time between the start of one of the time periods of connection to any one WiFi access point and the specified time means for determining that the WiFi device is present in the environment if the amount of time between the start of one of the time periods disconnected from all of the WiFi access points is greater than a second threshold amount of time and the one of the time periods disconnected includes the specified time; means for generating connection time data, connection / disconnection count data, transition data, and connection / disconnection length data for the WiFi device from the report; means for receiving sensor and device data from one or more sensors or devices in the environment; means for generating a WiFi device indication using a machine learning system, the WiFi device indication indicating that the WiFi device should be used to determine the presence or absence of the user associated with the WiFi device in the environment, wherein the connection time data, the connection / disconnection count data, the transition data, and the connection / disconnection length data and the sensor and device data of the WiFi device are input to the machine learning system;means for receiving additional reports from the WiFi access points in the environment, wherein each of the additional reports includes an identifier of a second WiFi device, an indication of a connection or disconnection with one of the WiFi access points, a time of the connection or disconnection, and an identifier of the one of the WiFi access points; means for generating second connection time data, second connection / disconnection count data, second transition data, and second connection / disconnection length data for the second WiFi device from the additional reports; means for generating, using the machine learning system, a second WiFi device indication indicating that the second WiFi device should not be used to determine the presence or absence of a second user associated with the second WiFi device in the environment, wherein the second connection time data, the second connection / disconnection count data, the second transition data, and the second connection / disconnection length data of the second WiFi device and the sensor and device data are input to the machine learning system; and means for receiving device data from the WiFi device, wherein the device data is input to the machine learning system.;

[0014] Additional features, advantages, and embodiments of the disclosed subject matter may be set forth or become apparent by consideration of the following detailed description, drawings, and claims. In addition, it should be understood that the foregoing summary and the following detailed description are illustrative and are intended to provide further explanation without limiting the scope of the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which are included to provide a further understanding of the disclosed subject matter, are incorporated into and constitute a part of this specification. The accompanying drawings also illustrate embodiments of the disclosed subject matter and, together with the detailed description, serve to explain the principles of the embodiments of the disclosed subject matter. No attempt is made to show structural details in more detail than is necessary for a basic understanding of the disclosed subject matter and the various ways in which it may be practiced.

[0016] Figure 1A Example systems and arrangements suitable for determining user presence and absence using a WiFi connection in accordance with implementations of the disclosed subject matter are shown.

[0017] Figure 1B Example systems and arrangements suitable for determining user presence and absence using a WiFi connection in accordance with implementations of the disclosed subject matter are shown.

[0018] Figure 2 An example time-flow diagram suitable for determining user presence and absence using a WiFi connection in accordance with an implementation of the disclosed subject matter is shown.

[0019] Figure 3An example process suitable for determining user presence and absence using a WiFi connection in accordance with an implementation of the disclosed subject matter is shown.

[0020] Figure 4 An example process suitable for determining user presence and absence using a WiFi connection in accordance with an implementation of the disclosed subject matter is shown.

[0021] Figure 5 Example systems and arrangements suitable for determining user presence and absence using a WiFi connection in accordance with implementations of the disclosed subject matter are shown.

[0022] Figure 6 A computing device in accordance with an embodiment of the disclosed subject matter is shown.

[0023] Figure 7 A system according to an embodiment of the disclosed subject matter is shown.

[0024] Figure 8 A system according to an embodiment of the disclosed subject matter is shown.

[0025] Fig. 9 A computer according to an embodiment of the disclosed subject matter is shown.

[0026] Fig.10 A network configuration according to an embodiment of the disclosed subject matter is shown. DETAILED DESCRIPTION

[0027] According to the embodiments disclosed herein, using WiFi connection to determine user presence and absence can allow the connection and disconnection of a WiFi device to a WiFi access point in the same environment to determine whether a user associated with the WiFi device is present in the environment or not in the environment. The WiFi access point in the environment can send a report including the time of connection and disconnection of the WiFi device to the cloud computing system. The cloud computing system can collate the report of the WiFi device using the time of connection and disconnection with the WiFi access point to determine the time period when the WiFi device is connected to and disconnected from the WiFi access point. The cloud computing system can apply rules to the time period of connection and disconnection to determine whether the time period indicates whether the WiFi device and its associated user are present in the environment or not in the environment as of the time when the rule is applied. The connection and disconnection of the WiFi device to the WiFi access point in the same environment can be used in combination with signals from other devices in the environment to determine whether to use the connection and disconnection of the WiFi device to the WiFi access point to determine whether the user associated with the WiFi device is present in the environment or not in the environment.

[0028] An environment may include multiple WiFi access points. The environment may be, for example, a structure such as, for example, a home, office, apartment, or other structure, and may include a combination of enclosed and open spaces. WiFi access points may be distributed throughout the environment, and may, for example, form a mesh network or a hub-and-spoke network. WiFi access points may provide access to a local area network (LAN) and a wide area network (WAN) such as the Internet for WiFi devices connected to the WiFi access points. Access to the WAN may be provided through any suitable wired or wireless WAN connection, which the WiFi access point may access, for example, through a connection to a wired or wireless modem.

[0029] The range in which a WiFi access point can broadcast and receive WiFi signals may include overlapping areas, resulting in an area of ​​the environment in which a WiFi device can connect to more than one WiFi access point. For example, a house may have three floors and may include a single WiFi access point located on each floor. A WiFi device in a specific area on the first floor may be able to connect to any of the WiFi access points on the first and second floors of the house.

[0030] A WiFi device can connect to and disconnect from a WiFi access point in an environment. A WiFi device can be any suitable device that includes a WiFi radio that allows the WiFi device to connect to a WiFi network, such as, for example, a phone, tablet, laptop, watch or other wearable device, or a WiFi-enabled tracking tag. As the WiFi device moves within the environment and enters and exits the environment, the WiFi device can connect to and disconnect from different WiFi access points throughout the environment depending on the location of the WiFi device and the range of the WiFi access point. For example, a WiFi device can start on the third floor of a house and connect to a WiFi access point on that floor. The WiFi device can move to the second floor of the house, disconnect from the WiFi access point on the third floor, and connect to a WiFi access point on the second floor. The WiFi device can move to the first floor of the house, disconnect from the WiFi access point on the second floor, and connect to a WiFi access point on the first floor. The WiFi device can leave the house and disconnect from the WiFi access point on the first floor. The WiFi device can re-enter the house later and connect to a WiFi access point on the first floor.

[0031] WiFi access points in the environment may report connections and disconnections to the WiFi access points by WiFi devices. Connections and disconnections may be reported to, for example, a cloud computing system that may be remote from the environment via an Internet connection. WiFi access points may report connections and disconnections of WiFi devices in real time. For example, a WiFi access point on the third floor of a house may report connections of WiFi devices to a cloud computing system when a connection is successfully established, and may report disconnections of WiFi devices when disconnections are detected. Detection of disconnections of WiFi devices by WiFi access points may be delayed from the actual disconnection that occurs on the WiFi device.

[0032] The reports of connections and disconnections of the WiFi device sent to the cloud computing system may include any suitable data, including an identifier of the WiFi device, an identifier of the WiFi access point, an indication of whether the report is for a connection or a disconnection, and the time when the WiFi access point detected the connection or disconnection. The identifier of the WiFi access point may be any suitable identifier that may allow the cloud computing system to distinguish between reports from different WiFi access points in the same environment. The identifier of the WiFi access point may be based on a MAC address of a component of the WiFi access point, for example, or may be an identifier assigned to the WiFi access point by a user.

[0033] The identifier of the WiFi device may be a privacy-preserving identifier that may allow the cloud computing system to distinguish reports of the WiFi device from reports of other WiFi devices, but may not allow positive identification of the WiFi device itself or a user of the WiFi device. For example, the identifier may be based on a media access control (MAC) address of the WiFi device, such as a salted hash MAC (SHMAC) generated when a user opts in to allow the cloud computing system to receive reports from WiFi access points in the environment. The SHMAC for the WiFi device may be generated by, for example, a WiFi access point and may be sent to the cloud computing system in a report for the WiFi device. The user may also enter the SHMAC or any other suitable identifier of the WiFi device directly into the WiFi access point or the cloud computing system. The SHMAC may allow all reports received by the cloud computing system that include the same SHMAC to be considered as reports for the same WiFi device, but may not allow the cloud computing system to identify the WiFi device, such as by determining its MAC address or other such identifier that can be used to identify the physical WiFi device.

[0034] The user can specify the WiFi device that the cloud computing system should use when determining that the user is present in the environment or not present in the environment. The specified WiFi device can be considered to act as an indicator of the presence or absence of the user of the WiFi device in the environment. The WiFi access point can only send reports for specified WiFi devices to the cloud computing system, or can send reports for all WiFi devices to the cloud computing system. The cloud computing system can receive an identifier for each specified WiFi device, which can allow the cloud computing system to determine which reports belong to which WiFi devices, but does not identify any physical WiFi device. Similarly, the cloud computing system may not be able to identify individual users of the WiFi device. Each separate WiFi device can be considered to be associated with a separate user, but there may be no clear identification of the user for the cloud computing system. The presence or absence of a WiFi device can be considered to indicate the presence or absence of a user associated with the WiFi device.

[0035] The WiFi access point may also update previously sent reports of connections and disconnections. For example, when a WiFi device has not sent data to the WiFi access point for a period of time, the WiFi access point may detect that the WiFi device has been disconnected. In some cases, the WiFi device may still be connected to the WiFi access point and may begin sending data to the WiFi access point after the WiFi access point has reported the disconnection to the cloud computing system. The WiFi access point may update the report to indicate that the WiFi device was not disconnected during the period between the report of the disconnection and the update to the report.

[0036] The cloud computing system can use reports of connections and disconnections received from WiFi access points within the environment by WiFi devices to determine the presence or absence of a user of the WiFi device in the environment. The cloud computing system can collate reports with identifiers of the same WiFi device. The cloud computing system can use the time of connection and disconnection in the report to determine the time period during which the WiFi device is connected to each WiFi access point in the environment, and any time period during which the WiFi device is not connected to any WiFi access point in the environment. This can generate a connection sequence for the WiFi device. The cloud computing system can then apply rules to the time periods in the connection sequence to determine whether the user of the WiFi device is present in the environment or not in the environment as of the time the rule is applied to the connection sequence. For example, if the WiFi device is connected to a single WiFi access point in the environment for a time period exceeding a first threshold time period, the cloud computing system can determine that the user associated with the WiFi device is present in the environment during the time period. If the WiFi device is not connected to a single WiFi access point in the environment for a time period exceeding the first threshold time period, the combined amount of time that the WiFi device has been connected to any WiFi access point in the environment can be determined. If the combined amount of time exceeds the first threshold time period, the cloud computing system can determine that the user associated with the WiFi device is present in the environment during the time period. If the WiFi device does not connect to any of the WiFi access points in the environment for an amount of time exceeding a second threshold time period after disconnecting from one of the WiFi access points, the cloud computing system can determine that the user associated with the WiFi device is not present in the environment. The first threshold time period and the second threshold time period can be any suitable length of time determined in any suitable manner.

[0037] Debouncing may be used if a WiFi device rapidly connects to and disconnects from a WiFi access point. Debouncing may remove multiple reports generated in rapid succession, which may indicate that the WiFi device is at the edge of the range of the WiFi access point, bouncing back and forth between connection and disconnection. For example, when a WiFi device is at the edge of the range of a WiFi access point, the WiFi device may repeatedly connect to and disconnect from the WiFi access point, causing the WiFi access point to repeatedly generate reports of connection and disconnection. The cloud computing system may debouncing by, for example, ignoring reports from the WiFi access point when the reports have an identifier for the same WiFi device, and the times of disconnection and reconnection are too close, such as separated by a time period below a small threshold, which may indicate that the WiFi device is at the edge of the range of the WiFi access point.

[0038] The cloud computing system can make separate determinations of presence or absence based on each separate WiFi device for which the cloud computing system receives reports. For example, if the cloud computing system receives reports of two different WiFi devices having two different SHMACs from WiFi access points in the environment, the cloud computing system can determine the presence or absence of the two separate WiFi devices and their associated users. The determination for the first WiFi device can be made based on a report that includes a first SHMAC of the two different SHMACs, and the determination for the second WiFi device can be made based on a report that includes a second SHMAC of the two different SHMACs.

[0039] The cloud computing system may update the determination of presence or absence at any suitable time. For example, the cloud computing system may update the determination of presence or absence made based on the report of a WiFi device whenever a new report of the WiFi device is received from a WiFi access point, or retroactively update the determination of presence or absence made based on the report of the WiFi device whenever a previous report of the WiFi device is updated. This may allow the cloud computing system to maintain current determinations of presence or absence and retroactively adjust previously made determinations so that the record of presence and absence maintained over time may be more accurate.

[0040] The cloud computing system may use the determined presence or absence of the user based on the user's associated WiFi device in any suitable manner. For example, the environment may include controllable devices such as lights, sensors, security devices, locks, A / V devices, HVAC systems, and electromechanical devices such as blinds, which may be controlled from the cloud computing system directly or through a computing device (such as a hub computing device) located in the environment. The cloud computing system may generate control signals to control the controllable devices based on determining the presence and absence of the user in the environment, or may send the determination of presence and absence to a computing device in the environment, which may use them to generate control signals for the controllable devices.

[0041] In some embodiments, instead of the user telling the cloud computing system which WiFi devices to use to determine the presence or absence of the user, the cloud computing system can determine which WiFi devices to use based on the connection and disconnection with the WiFi access point in the same environment and the signals from other devices in the environment. The connection and disconnection of some WiFi devices in the environment may not be a good indicator of whether the user is present in the environment or not in the environment. For example, some WiFi devices may be mainly fixed, such as desktop computers, TVs with built-in WiFi, game consoles, and other WiFi-enabled appliances and A / V electronic devices and appliances. These WiFi devices can remain connected to a single WiFi access point most of the time, and may not be useful in determining whether the user associated with these WiFi devices is present in the environment or not in the environment. Some WiFi devices, such as laptops and tablet computers, can move in the environment, connect to different WiFi access points within the environment and disconnect from different WiFi access points within the environment, but may only occasionally or never leave the environment, which may make them less useful in determining whether the user is not present in the environment. In addition, some WiFi devices can enter and leave the environment, but may do so occasionally because they may belong to guests of the environment.

[0042] The WiFi access point may send reports to the cloud computing system for all WiFi devices connected to and disconnected from the WiFi access point. The WiFi device may be identified using a privacy-preserving identifier, such as, for example, a SHMAC determined by the WiFi access point for the WiFi device. The cloud computing system may only know the SHMAC of the WiFi device and may not receive any other identification data of the WiFi device from the report of the WiFi access point.

[0043] The cloud computing system can use reports from WiFi access points to determine data about the connection and disconnection of various WiFi devices to WiFi access points. For example, the cloud computing system can determine, for each WiFi device whose report is received, the number of connections and disconnections of the WiFi device within a set time period (such as a 24-hour period). The cloud computing system can, for example, determine the time span that the WiFi device is connected to any WiFi access point in the environment for each WiFi device whose report is received, and similarly determine the time span that the WiFi device is not connected to any WiFi access point in the environment. For example, the cloud computing system can determine, for each WiFi device whose report is received, the number of different WiFi access points in the environment to which the WiFi device is connected within a set time period, and the number of transitions made by the WiFi device between WiFi access points in the environment. The cloud computing system can, for example, determine the amount of time that the WiFi device is connected to each WiFi access point for each WiFi device whose report is received.

[0044] The cloud computing system can also receive data from other devices within the environment. The environment can, for example, include a hub computing device. The hub computing device can be any suitable computing device for managing sensors and other systems such as automation systems within the environment. The hub computing device can be, for example, a controller for the environment. For example, the hub computing device can be or include a thermostat, a security hub, or other computing devices located within the environment. The hub computing device can also be another device within the environment, or it can be a separate computing device dedicated to managing the environment, which can be connected to the devices in the environment through, for example, the Internet. The hub computing device can be connected to multiple sensors and controllable devices distributed throughout the environment or structure through any suitable wired, wireless, local and wide area connections. For example, the hub computing device, sensors, and other components of the environment can be connected in a mesh network. For example, some of the sensors can be motion sensors, including passive infrared sensors for motion detection, light sensors, cameras, microphones, entrance sensors, light switches, and mobile device scanners that can use Bluetooth, WiFi, RFID or other wireless devices as sensors to detect the presence of devices such as phones, tablets, laptops, or FOBs. Sensors can be distributed separately or combined with other sensors in the sensor device. For example, the sensor devices may include a low-power motion sensor and a light sensor, or a microphone and a camera, or any other combination of available sensors.

[0045] The hub computing device can receive signals including data from sensors and other devices throughout the environment and send data from the signals to the cloud computing system. The data can include, for example, open / close events detected by sensors monitoring external doors of the environment and motion detected by motion sensors monitoring the area around the external doors of the environment, data indicating when devices (including lights, appliances, and A / V equipment) are turned on or off based on input from a user rather than through automatic control of the hub computing device or cloud computing system, and any other suitable data that can indicate whether a user is present in the environment, absent from the environment, entering the environment, or leaving the environment. In some embodiments, data from sensors and other devices can be sent directly to the cloud computing system via a WiFi access point, as the hub computing device can be entirely part of a cloud computing system with computing capabilities located outside the environment.

[0046] The cloud computing system can receive other data directly from the WiFi device. For example, the WiFi device can be permitted by its user to send geographic location data and data determined using the geographic location data to the cloud computing system. The geographic location data can be obtained by the WiFi device in any suitable manner, including by using a global positioning system (GPS) radio, or by cellular or WiFi triangulation. The geographic location data can be used for, for example, geo-fencing. The WiFi device or the cloud computing system can use the geolocation data of the WiFi device to determine when the WiFi device has passed through the geo-fence, entered or left the area surrounded by the geo-fence. The geo-fence can be, for example, a geo-fence around the environment, so that the exit of the WiFi device through the geo-fence can indicate that the WiFi device and the user associated with the WiFi device have left the environment. The data received directly from the WiFi device can be sent through a WiFi access point in the environment, a WiFi access point outside the environment, or through other data connections such as a cellular data connection.

[0047] In some embodiments, the cloud computing system can also receive reports of connection and disconnection of a device's Bluetooth radio from a WiFi access point having a Bluetooth radio. For example, a tracking tag can include a Bluetooth radio that can connect to and disconnect from a Bluetooth radio of a WiFi access point in the environment as the tracking tag moves through the environment.

[0048] The cloud computing system can use data determined about connections and disconnections to WiFi access points through various WiFi devices, data received from other devices and sensors in the environment, and data received directly from WiFi devices, such as geolocation data, to determine which WiFi devices to use to determine the presence or absence of a user in the environment. For example, the cloud computing system may include a machine learning system. The machine learning system may be any suitable machine learning system, such as, for example, an artificial neural network, such as a deep learning neural network, a Bayesian network, a support vector machine, any type of classifier, or any other suitable statistical or heuristic machine learning system type. The machine learning system may be a trained machine learning system that may have undergone supervised or unsupervised offline training, or may be a machine learning system that uses supervised or unsupervised online learning.

[0049] Data determined regarding the connection and disconnection of various WiFi devices to WiFi access points, data received from other devices and sensors in the environment, and data received directly from WiFi devices, such as geolocation data, can be used by the cloud computing system as input data to the machine learning system. The machine learning system can generate and output data that may include an indication of the WiFi device, which the cloud computing system should use to determine the presence or absence of a user of the WiFi device in the environment.

[0050] The input data may be input to the machine learning system in any suitable manner. For example, the input data may be partitioned by WiFi device such that a first set of input data includes data about a first WiFi device but not any other WiFi device and results in output data only about the first WiFi device, while a second set of input data may include data about a second WiFi device but not any other WiFi device and results in output data only about the second WiFi device. For example, the output data may be a probability that a WiFi device will be useful to a cloud computing system in determining the presence or absence of any user of the WiFi device, e.g., a particular user of the WiFi device or a group of users who may be possible users of the WiFi device, in the environment, or a binary indication of whether the cloud computing should use the WiFi device to determine the presence or absence of any user in the environment.

[0051] For example, the cloud computing system may receive reports of two WiFi devices identified by SHMAC from a WiFi access point in the environment. The first WiFi device may be, for example, a phone, and the second WiFi device may be, for example, a laptop, but the cloud computing system may not know the type of each of the WiFi devices because the report may identify the WiFi device only by SHMAC. The cloud computing system may determine data about connection and disconnection with the WiFi access point by the phone and the laptop. The cloud computing system may also receive data from other devices and sensors in the environment, and may receive data directly from the phone and the laptop that may be identified only by the SHMAC of the device. The cloud computing system may first input the data for the phone determined from the WiFi access point report for the phone into the machine learning system together with any data received directly from the phone and data from other devices and sensors in the environment. The machine learning system may output an indication of whether the cloud computing system should use the phone to determine whether the user associated with the phone is present or absent in the environment. The indication may be, for example, a probability that may indicate a confidence level that the phone should be used, or a binary yes / no indication of whether the phone should be used. The cloud computing system can then input the data determined for the laptop, along with any data received directly from the laptop and data from other devices and sensors in the environment, into a machine learning system, which can output an indication of whether the cloud computing system should use the laptop to determine the presence or absence of a user associated with the laptop in the environment.

[0052] The input data may also include data for any number of WiFi devices. This may cause the machine learning system to generate output data, which may be, for example, a vector of probabilities or binary indications about whether any WiFi device is used to determine whether a user is present or absent in the environment. The output data from the machine learning system may identify any WiFi device through the SHMAC. For example, the input data may include data for both a phone and a laptop, and the machine learning system may output a vector including two values, one indicating whether the cloud computing system should use the phone to determine whether a user associated with the phone is present or absent in the environment, and the other indicating whether the cloud computing system should use the laptop to determine whether a user associated with the laptop is present or absent in the environment. The values ​​in the vector may be, for example, probabilities or binary indicators.

[0053] The input data may be input to the machine learning system at any suitable time. For example, the cloud computing system may update the input data at any time when a new report of any WiFi device is received from any WiFi access point, new data is received from any sensor or device in the environment, or new data is received directly from any WiFi device. The cloud computing system may use the updated input data as the input of the machine learning system immediately after any update, or use the updated input data as the input of the machine learning system after a certain set number of updates to the input data. The cloud computing system may also input the input data to the machine learning system based on a timed interval, for example, once an hour. Output data from the machine learning system based on the updated input data may be used to update which WiFi devices the cloud computing system uses to determine whether a user is present or absent in the environment. For example, if a user replaces their old phone with a new phone, an update to the input data may display reports of connections and disconnections through the new phone, while the old phone never reconnects after its last disconnected report. Output data from the machine learning system based on the updated input data may indicate that a new phone should be used to determine whether a user is present or absent in the environment, and that the old phone should no longer be used.

[0054] In some embodiments, the WiFi access point may transmit a signal to the cloud computing system to indicate that the WiFi access point is turned on and active. The signal may be, for example, a heartbeat report, which may be sent at specified intervals when the WiFi access point is turned on and active. The WiFi access point may be considered to be closed or inactive when no heartbeat report is received from the WiFi access point for a period of time. The connection sequence of WiFi devices connected to the WiFi access point when the WiFi access point was first considered closed or inactive may be updated to indicate that the machine learning system should not use the connection to the WiFi access point. The machine learning system may output a presence or absence indication based on any other available data. If too many WiFi access points are considered closed or inactive, the machine learning system may output an "unknown" presence or absence indication indicating that there is not enough data available to determine the presence or absence of a WiFi device.

[0055] In some embodiments, the machine learning system can be part of a hub computing device for the environment. Reports from WiFi access points, data from devices and sensors in the environment, and data received directly from WiFi devices can all be received by the hub computing device. Only reports from WiFi access points of WiFi devices that have been used to determine the presence or absence of a user in the environment can be sent to the cloud computing system. The hub computing device can determine data about connections and disconnections to WiFi access points through various WiFi devices, and can use the determined data and data received from other devices and sensors in the environment and data received directly from WiFi devices as input data for the machine learning system. Output data from the machine learning system, which indicates which WiFi devices are used to determine the presence or absence of a user in the environment, can be used by the hub computing device to control which reports the WiFi access points send to the cloud computing system.

[0056] Figure 1A An example system suitable for determining user presence and absence using a WiFi connection in accordance with an implementation of the disclosed subject matter is shown. Cloud computing system 100 may include report processor 110, rule engine 120, and storage 140. Cloud computing system 100 may be any suitable computing device or system for implementing report processor 110, rule engine 120, and storage 140, such as, for example, Fig. 9 The computer 20 described in the above. The cloud computing system 100 may be, for example, a server system that provides cloud computing services using any suitable combination of computing devices distributed over any area and connected in any suitable manner. The report processor 110 may be any suitable combination of hardware or software for receiving reports from WiFi access points and processing the reports to generate a connection sequence 145. The rule engine 120 may be any suitable combination of hardware and software for applying rules to the connection sequence 145 to generate a presence / absence indication. The storage device 140 may be any suitable combination of hardware and software for implementing volatile and non-volatile storage, and may store the connection sequence 145.

[0057] Environment 150 may include WiFi access points 171, 172, and 173. Environment 150 may be, for example, a structure such as a home or office, and may include a combination of indoor and outdoor spaces. WiFi access points 171, 172, and 173 may be any suitable device for creating a WiFi LAN to which devices with WiFi may connect. WiFi access points 171, 172, and 173 may form a mesh network, or may be part of a hub-and-spoke network, and may be connected to a WAN such as the Internet, for example, via a wired or wireless modem that is connected to one or more of WiFi access points 171, 172, and 173 via any suitable wired or wireless connection.

[0058] WiFi devices 191 and 192 may be, for example, smart phones, tablets, wearable devices, or other portable WiFi-equipped devices. A user may have indicated to cloud computing system 100 that WiFi devices 191 and 192 should be used to determine the presence or absence of users of WiFi devices 191 and 192 in environment 150. WiFi device 191 may be connected to WiFi access point 171. WiFi device 192 may be connected to WiFi access point 172. WiFi device 191 may move in environment 150. When WiFi device 191 moves away from WiFi access point 171 and toward WiFi access point 172, WiFi device 191 may connect to and disconnect from WiFi access point 172. After WiFi device 191 has connected to WiFi access point 172, WiFi access point 171 may detect that WiFi device 191 has disconnected. WiFi device 191 may continue to move away from WiFi access point 172 and toward WiFi access point 173 in environment 150. When WiFi device 191 moves away from WiFi access point 172 and toward WiFi access point 173, WiFi device 191 may connect to WiFi access point 173 and disconnect from WiFi access point 172. After WiFi device 191 has connected to WiFi access point 173, WiFi access point 172 may detect that WiFi device 191 has disconnected. WiFi device 191 may continue to move away from WiFi access point 173, disconnect from WiFi access point 173 and not connect to either of WiFi access points 171 and 172. When WiFi device 191 moves, WiFi device 192 may remain connected to WiFi access point 172.

[0059] WiFi access points 171, 172, and 173 may send reports to cloud computing system 100. Reports may include the time of connection and disconnection with WiFi access points 171, 172, and 173 by WiFi devices in environment 150, such as WiFi devices 191 and 192. Reports may be sent at any suitable time or interval or based on any suitable event. For example, WiFi access points 171, 172, and 173 may send a new report at any time they detect the connection or disconnection of a WiFi device, or may send a new report at intervals of any suitable length. Reports may use privacy protection identifiers, such as, for example, SHMACs determined for WiFi devices, to identify WiFi devices. For example, WiFi access point 171 may send a report to cloud computing system 100 when WiFi device 191 initially connects to WiFi access point 171. The report may include the SHMAC for WiFi device 191, an identifier for WiFi access point 171, an indication that a connection is being reported, and the time of the connection. WiFi access point 171 may similarly send a report when it is determined that WiFi device 191 has been disconnected from WiFi access point 171. WiFi access point 172 may send a report to cloud computing system 100 reporting the connection of WiFi device 192, the connection of WiFi device 191, and the subsequent disconnection of WiFi device 191. WiFi access point 173 may send a report to cloud computing system 100 reporting the connection of WiFi device 191 and the subsequent disconnection of WiFi device 191.

[0060] The cloud computing system 100 may include a report processor 110. The report processor 110 may be any suitable combination of hardware and software for receiving reports from WiFi access points, such as WiFi access points 171, 172, and 173, and processing the reports to generate a connection sequence 145. The report processor 110 may receive reports generated based on the connection and disconnection of WiFi devices 191 and 192 from WiFi access points 171, 172, and 173. The report processor 110 may use the reports to generate and update the connection sequence of WiFi devices 191 and 192. The connection sequence of the WiFi devices may be updated each time a report about a WiFi device is received by the report processor 110. For example, upon receiving a report indicating that WiFi device 191 has connected to WiFi access point 171, the report processor 110 may update the connection sequence of WiFi device 191 by adding an indication of the connection of WiFi access point 171 at the time indicated in the report. When the report processor 110 later receives a report from the WiFi access point 172 indicating a connection from the WiFi device 191, the report processor 110 can update the connection sequence of the WiFi device 191 by adding an indication of the connection to the WiFi access point 172 at the time indicated in the report. When a report of a disconnection is received, the disconnection can be similarly added to the connection sequence. The connection sequence 145 generated and updated by the report processor 110 can be stored in the storage device 140 in any suitable format.

[0061] The cloud computing system 100 may include a rules engine 120. The rules engine 120 may be any suitable combination of hardware and software for applying rules to the connection sequence 145 to determine and output a presence / absence indication indicating whether a user should be considered present or absent in the environment 150. If a WiFi device, such as WiFi device 191, is connected to a single WiFi access point in the environment 150, such as WiFi access point 171, for a time period exceeding a first threshold time period, the rules engine 120 may determine that a user associated with the WiFi device was present in the environment 150 during the time period. If the WiFi device is not connected to a single WiFi access point in the environment 150 for a time period exceeding the first threshold time period, a combined amount of time that the WiFi device has been connected to any WiFi access point in the environment 150 may be determined. If the combined amount of time exceeds the first threshold time period, the rules engine 120 may determine that the user associated with the WiFi device was present in the environment 150 during the time period. If the WiFi device does not connect to any WiFi access point in environment 150 for an amount of time that exceeds a second threshold time period after disconnecting from one of the WiFi access points, rules engine 120 may determine that the user associated with the WiFi device is not present in environment 150. The first threshold time period and the second threshold time period may be any suitable lengths of time determined in any suitable manner.

[0062] For example, the rule engine 120 may apply the rule to the connection sequence of WiFi device 191 at a point in time after WiFi device 191 has been connected to WiFi access point 172 and disconnected from WiFi access point 171. The amount of time that WiFi device 191 has been connected to WiFi access point 172 may be less than a first threshold. The combined amount of time that WiFi device 191 has been connected to WiFi access points 171 and 172 may be greater than the first threshold. For example, the first threshold may be five minutes, and when the rule engine 120 applies the rule to the connection sequence of WiFi device 191, WiFi device 191 may have been connected to WiFi access point 171 for two minutes and connected to WiFi access point 172 for 3.5 minutes. Therefore, the rule engine 120 may output an indication that the user of WiFi device 191 is present in environment 150. Similarly, the rule engine 120 may apply the rule to the connection sequence of WiFi device 192 while applying the rule to the connection sequence of WiFi device 191. The amount of time that WiFi device 192 has been connected to WiFi access point 172 may be greater than the first threshold. Therefore, the rule engine 120 may output an indication that the user of WiFi device 192 is present in environment 150. Later, after WiFi device 191 has been disconnected from WiFi access point 173, rules engine 120 may apply the rule to the connection sequence of WiFi device 191. If WiFi device 191 has been disconnected from WiFi access point 173 for a length of time greater than a second threshold, rules engine 120 may output an indication that the user of WiFi device 191 is not present in environment 150. For example, the second threshold may be 3 minutes, and when rules engine 120 applies the rule to the connection sequence of WiFi device 191, WiFi device 191 may have been disconnected from WiFi access point 173 without reconnecting to WiFi access point 171 or 172 for 4 minutes.

[0063] The presence / absence indication output by the rule engine 120 may be in any suitable format and may use any suitable identifier. For example, the cloud computing system 100 may only know the SHMACs of the WiFi devices 191 and 192, and may not know any identification data of any user of the WiFi devices 191 and 192, or may otherwise be unable to associate a particular user with a particular WiFi device. The presence / absence indication output by the rule engine 120 may include a SHMAC with an indication of whether the SHMAC belongs to a WiFi device whose connection sequence indicates that the user of the WiFi device may be present in the environment 150 or not in the environment 150. Presence / absence indications for different WiFi devices may be treated as if they were presence / absence indications for separate users. For example, the presence / absence indication of the WiFi device 191 may be treated as a presence / absence indication of a user unique to the user of the WiFi device 192. In some embodiments, the cloud computing system 100 may be able to associate the SHMAC with a particular user, for example, based on the user's choice to allow their identity or the user's privacy protection identifier to be associated with the SHMAC. A single identified user may be associated with more than one SHMAC, such that presence / absence indications output by the rules engine 120 for SHMACs associated with the same user may be aggregated to make a presence / absence determination for that user.

[0064] Rules engine 120 can apply rules to connection sequence 145 at any suitable time, for example, at any suitable interval or based on any suitable event. For example, rules engine 120 can apply rules to connection sequence 145 every 30 seconds or based on an indication from another component of cloud computing system 100 that a user is present or absent in environment 150, for example, to determine how to control various controllable devices in environment 150.

[0065] Figure 1B An exemplary system suitable for determining user presence and absence using WiFi connections in accordance with an implementation of the disclosed subject matter is shown. WiFi access points in environment 150, such as WiFi access points 171, 172, and 173, can send reports for all WiFi devices connected to and disconnected from the WiFi access points to a report processor 110 of cloud computing system 100. For example, when any of WiFi devices 191, 192, 491, and 492 connect to or disconnect from any of WiFi access points 171, 172, or 173, WiFi access points 171, 172, and 173 can send a report to cloud computing system 100. The report can use a privacy protection identifier, such as, for example, SHMAC, to identify the WiFi device.

[0066] The report processor 110 may process reports received from WiFi access points in the environment 150 to generate and update data that may be stored as part of the machine learning input data 445. The report processor 110 may, for example, use the reports to determine connection time 451, which may be a separate amount of time that the WiFi device is connected to each WiFi access point in the environment 150, connections / disconnections 452, which may be a count of the number of times the WiFi device is connected to and disconnected from WiFi access points in the environment 150 within a set time period, connection / disconnection length 453, which may be a span of time it takes for the WiFi device to connect to any WiFi access point in the environment 150 and a span of time it takes for the WiFi device to disconnect from any WiFi access point in the environment 150, and transitions 454, which may be a number of WiFi access points that the WiFi device is connected to within a set time period and a number of transitions between WiFi access points that the WiFi device makes within the set time period.

[0067] The cloud computing system 100 may also receive device data directly from WiFi devices. For example, the cloud computing system 100 may receive geographic location data and data determined using the geographic location data, such as geo-fence intersections, directly from WiFi devices 191, 192, 491, and 492. Data received directly from WiFi devices may be stored in the machine learning input data 445 as WiFi device data 456. The cloud computing system 100 may also receive signals from sensors and devices of the environment 150. The cloud computing system 100 may store the signals in the storage device 140 as sensor and device data 455. The sensor and device data 455 may be stored as part of the machine learning input data 445.

[0068] The cloud computing system may include a machine learning system 420. The machine learning system 420 may be any suitable combination of hardware and software for implementing a machine learning system that may generate an indication of a WiFi device for determining the presence or absence of a user of the WiFi device in the environment 150. The machine learning system 420 may be, for example, an artificial neural network, such as a deep learning neural network, a Bayesian network, a support vector machine, any type of classifier, or any other suitable statistical or heuristic machine learning system type. The machine learning system 420 may receive machine learning input data 445 as input, and may output an indication of a WiFi device for determining the presence or absence of a user of the WiFi device in the environment 150. The machine learning system 420 may be implemented using any suitable type of learning, including, for example, supervised or unsupervised online learning or offline learning.

[0069] The machine learning input data 445 may be input to the machine learning system 420 in any suitable manner. For example, the machine learning input data 445 may be divided into a plurality of input data sets, wherein each separate input data set includes data about a separate WiFi device. The first input data set may, for example, include data about WiFi device 191, while the second input data set may include data about WiFi device 491. Each input data set may be input to the machine learning system 420 separately. For example, a first input data set including data about WiFi device 191 may be input to the machine learning system 420. The machine learning system 420 may output an indication of whether WiFi device 191 should be used to determine whether a user associated with WiFi device 191 is present or absent in environment 150. The indication may be, for example, a probability or a binary result. All machine learning input data 445 may be input to the machine learning system 420 at the same time, including data for all WiFi devices for which reports were received, such as WiFi devices 191, 192, 491, and 492. Machine learning system 420 may output multiple indications, for example, one indication for each of WiFi devices 191 , 192 , 491 , and 492 , indicating whether each WiFi device should be used to determine the presence or absence of a user associated with the WiFi device in environment 150 .

[0070] The indication of a WiFi device used to determine the presence or absence of a user associated with the WiFi device in environment 150 can be based on, for example, correlations between different types of data in machine learning input data 445 and patterns in data regarding connections and disconnections to WiFi access points of environment 150. For example, WiFi device 491 can be stationary, such as a desktop computer or a game console. WiFi device 491 can have a long connection time to WiFi access point 171, but no disconnections or transitions, and its connection to WiFi access point 171 can not show correlation with signals from sensors in environment 150 that correspond to a user entering or leaving environment 150, such as door opening / closing events. This can cause machine learning system 420 to output an indication that WiFi device 491 should not be used to determine the presence or absence of any user in environment 150. The indication can be, for example, a low probability or binary "no" indication. WiFi device 492 can be mobile, for example, a phone belonging to a user in environment 150. WiFi device 492 may have long connection and disconnection times with all WiFi access points 171, 173, and 174, may frequently connect, disconnect, and switch, and may have connections and disconnections corresponding to signals from entry sensors showing doors of environment 150 being opened and closed. Geolocation data from WiFi device 492 may indicate that some disconnections of WiFi device 492 from WiFi access points correspond to geofence exit events of a geofence around environment 150, and some connections of WiFi device 492 correspond to geofence entry events of a geofence around environment 150. This may cause the machine learning system to output an indication that WiFi device 492 should be used to determine the presence or absence of a user associated with WiFi device 492 in environment 150. This indication may be used by, for example, cloud computing system 100 in determining which connection sequence 145 to input into rule engine 120.

[0071] The machine learning input data 445 may be continuously updated as new data is received from, for example, the report processor 110, the signal receiver 410, and the WiFi devices. The updated machine learning input data 445 may be used as input to the machine learning system 420, which may generate new WiFi device indications.

[0072] The WiFi device indications output by the machine learning system 420 may be used in any suitable manner. The cloud computing system 100 may, for example, use the WiFi device indications to determine which connection sequence 145 the rule engine 120 should apply the rule to in order to generate a presence / absence indication for a user in the environment 150. For example, if the WiFi device indications output by the machine learning system 420 indicate that WiFi device 191 and WiFi device 492 should be used to determine the presence or absence of their associated users in the environment 150, the rule engine 120 may apply the rule to the connection sequence of WiFi device 191 and WiFi device 492. The rule engine 120 may not apply the rule to the connection sequence of WiFi device 192 and WiFi device 491 because these WiFi devices may not be used as indicators of the presence or absence of their associated users in the environment 150. This may allow the rule engine 120 to generate useful presence / absence indications without requiring the user to explicitly tell the cloud computing system 100 which WiFi devices to use for this purpose.

[0073] Figure 2An example time flow diagram suitable for determining user presence and absence using WiFi connections according to an implementation of the disclosed subject matter is shown. WiFi access point 171 may send a report to report processor 110 indicating that a connection from WiFi device 191 occurred at 8:30 AM. WiFi access point 172 may send a report to report processor 110 indicating that a connection from WiFi device 191 occurred at 8:33 AM. Time period 210 may be a time period during which WiFi device 191 was connected to WiFi access point 171 before connecting to WiFi access point 172. WiFi access point 171 may send a report to report processor 110 indicating that a disconnection of WiFi device 191 occurred at 8:34 AM. Time period 211 may be a time period during which WiFi device 191 was connected to WiFi access point 172 before WiFi access point 171 determined that WiFi device 191 had been disconnected therefrom. WiFi access point 173 may send a report to report processor 110 indicating that a connection from WiFi device 191 occurred at 8:36 AM. The time period 212 combined with the time period 211 may be a time period during which the WiFi device 191 was connected to the WiFi access point 172 before being connected to the WiFi access point 173. The WiFi access point 172 may send a report to the report processor 110 indicating that the disconnection of the WiFi device 191 occurred at 8:37 AM. The time period 213 may be a time period during which the WiFi device 191 was connected to the WiFi access point 173 before the WiFi access point 172 determined that the WiFi device 191 had been disconnected therefrom. The WiFi access point 173 may send a report to the report processor 110 indicating that the disconnection of the WiFi device 191 occurred at 8:40 AM. The time period 214 may be a time period during which the WiFi device 191 was connected to the WiFi access point 173 before the WiFi access point 173 determined that the WiFi device 191 had been disconnected therefrom. The time period 215 may be a time period during which the WiFi device 191 was disconnected from all WiFi access points in the environment 150. The rule engine 120 may always consider the most recent time period (e.g., the time period 215) in the connection sequence to be extended to the current time. For example, if there is no report indicating that WiFi device 191 has connected to any of WiFi devices 171, 172, and 173 at 8:50 AM, and the connection sequence of WiFi device 191 is input to rules engine 120 at 8:50 AM, disconnected time period 215 may be extended to 8:50 AM.

[0074] When determining an indication of the presence or absence of a user in environment 150 at a given time, rule engine 120 may apply a rule to connection sequence 145 up to that time. For example, if rule engine 120 determines a presence / absence indication at 8:36 a.m., rule engine 120 may test time periods 210 and 211 and the portion of time period 212 between 8:34 a.m. and 8:36 a.m. against the first and second thresholds of the rule. If the first threshold is five minutes, rule engine 120 may determine that the connection sequence of WiFi device 191 indicates the presence of a user because WiFi device 191 was connected to WiFi access points 171 and 172 for a combined six minutes across time period 210, time period 211, and the first two minutes of time period 212. If rule engine 120 determines a presence / absence indication at 8:45 a.m., rule engine 120 may test time period 215 against the second threshold. If the second threshold is four minutes, rule engine 120 may determine that the connection sequence of WiFi device 191 indicates the absence of a user because WiFi device 191 has been disconnected from all WiFi access points for five minutes. If the second threshold is six minutes, the rules engine 120 may test the time periods 210, 211, 212, 213, and 214 against the first threshold, and may determine that the connection sequence of the WiFi device 191 indicates the presence of the user because the WiFi device 191 is connected to the WiFi access points 171, 172, and 173 for a combined ten minutes across the time periods 210, 211, 212, 213, and 214. Thus, the second threshold may serve as a delay time period for determining the absence of the user based on the disconnection of the WiFi device.

[0075] Figure 3 An example of a process suitable for determining user presence and absence using WiFi connections in accordance with an implementation of the disclosed subject matter is shown.At 300, reports can be received from WiFi access points at a cloud computing system regarding connections and disconnections by WiFi devices to WiFi access points in an environment.

[0076] At 302, a connection sequence can be updated using connection and disconnection times of WiFi devices in reports received from WiFi access points in the environment.

[0077] At 304, rules may be applied to the connection sequence to determine time periods when the WiFi device is present or absent from the environment based on time periods when the WiFi device is connected to or disconnected from WiFi access points in the environment.

[0078] At 306, a presence and absence indication of the user can be generated based on the determination of the presence or absence of the WiFi device.

[0079] Figure 4 An example of a process suitable for determining user presence and absence using WiFi connections in accordance with an implementation of the disclosed subject matter is shown.At 400, reports can be received from WiFi access points at a cloud computing system regarding connections and disconnections by WiFi devices to WiFi access points in an environment.

[0080] At 402, machine learning input data can be generated from the report, including connection time data, connection / disconnection count data, connection / disconnection length data, and conversion data.

[0081] At 404, sensor and device data may be received at a cloud computing system and stored along with machine learning input data.

[0082] At 406, device data, including geo-location data and geo-fence entry and exit data, may be received directly from the WiFi device and stored with the machine learning input data.

[0083] At 408, machine learning input data may be input into a machine learning system.

[0084] At 410 , a WiFi device indication may be generated by a machine learning system that indicates which WiFi devices should be used to determine the presence or absence of a user in an environment.

[0085] Figure 5 An example system suitable for determining user presence and absence using a WiFi connection in accordance with an implementation of the disclosed subject matter is shown. Hub computing device 400 may include signal receiver 410. Hub computing device 400 may be any suitable device for implementing signal receiver 410, such as, for example, Fig. 9 The central computing device 400 may be, for example, a computer 20 as described in Figure 7 The hub computing device 400 may be a single computing device, or may include multiple connected computing devices, and may be, for example, a thermostat, other sensor, phone, tablet, laptop, desktop, television, watch, or other computing device that can act as a hub for the environment 150, which may include security systems and automation functions.

[0086] Environment 150 may be controlled from hub computing device 400. Hub computing device 400 may be connected to various sensors throughout the environment as well as various systems within environment 150, such as an HVAC system. Hub computing device 400 may include any suitable hardware and software interfaces through which a user may interact with hub computing device 400. Hub computing device 400 may be located within environment 150, may be located off-site, or may include computing devices both within and off-site of environment 150. An on-site hub computing device 400 may use computing resources from other computing devices throughout environment 150 or connected remotely, such as, for example, as part of a cloud computing platform.

[0087] Signal receiver 410 may be any suitable combination of hardware or software for receiving signals generated by sensors and other electronic devices that may be part of environment 150 and that may be connected to hub computing device 400. For example, signal receiver 410 may receive signals from sensors and devices 470 that may be distributed throughout environment 150. Sensors and devices 470 may be, for example, any combination of motion sensors, entry sensors, cameras, microphones, light sensors, contact sensors, tilt sensors, WiFi or Bluetooth detectors, lights, appliances, A / V equipment, HVAC systems, security systems, or any other suitable sensor and device types in environment 150. The signals received by the signal receiver 410 from the sensors and devices 470 may include, for example, open / close events detected by sensors monitoring exterior doors of the environment and motion detected by motion sensors monitoring areas around exterior doors of the environment 150, data indicating when devices (including lights, appliances, and A / V devices) are turned on or off based on input from a user rather than automatic control by the hub computing device 400 or the cloud computing system 100, and any other suitable data that may indicate whether a user is present in the environment 150, absent from the environment 150, entering the environment 150, or leaving the environment 150. The signals may include, for example, signals and other data generated by the sensors and devices 470 based on active outputs from the sensors or the lack of active outputs from the sensors. For example, a motion sensor may generate an active output when it detects motion, and may lack an active output when it does not detect motion.

[0088] Signal receiver 410 may transmit signals received from sensors and devices 470 of environment 150 to cloud computing system 100. Cloud computing system 100 may store the signals in storage 140 as sensor and device data 455. Sensor and device data 455 may be stored as part of machine learning input data 445.

[0089] In some embodiments, the machine learning system 420 may be run on the hub computing device 400. The machine learning input data 445 may be stored in the storage of the hub computing device 400 instead of the storage 140 of the cloud computing system 100, or the storage 140 may be connected to the hub computing device 400. If the machine learning input data 445 is stored on the hub computing device 400, the hub computing device 400 may receive reports from the WiFi access points of the environment 150 and device data from the WiFi devices, and may include a report processor similar to the report processor 110 to generate connection time 451, connection / disconnection 452, connection / disconnection length 453, and conversion 454 from the reports. The WiFi device indication output by the machine learning system 420 may be transmitted to the cloud computing system 100, or may be used by the hub computing device 400 to control which reports are sent to the cloud computing system 100. For example, the hub computing device 400 may only cause reports of WiFi devices indicating that the WiFi device should be used to determine the presence or absence of the user in the environment to be sent to the cloud computing system 100.

[0090] A report may be received from a WiFi access point in the environment. The report may include an identifier of the WiFi device, an indication of connection or disconnection with one of the WiFi access points, the time of connection or disconnection, and an identifier of one of the WiFi access points. A connection sequence of the WiFi device may be generated from the report. The connection sequence may include the time of connection to the WiFi access point by the WiFi device or the time of disconnection from the WiFi access point. Whether the WiFi device exists in the environment or not in the environment as of a specified time may be determined based on the time period of connection to any WiFi access point and the time period of disconnection from all WiFi access points in the connection sequence. If it is determined that the WiFi device exists in the environment, an indication of the presence of a user associated with the WiFi device may be generated, or if it is determined that the WiFi device does not exist in the environment, an indication of the absence of a user associated with the WiFi device may be generated.

[0091] A control signal may be generated for a controllable device in the environment based on the indication of presence or absence. The control signal may be sent to the device to be implemented by the device.

[0092] By determining that: if an amount of time between the start of one of the time periods of connection to any WiFi access point and the specified time is greater than a first threshold amount of time and one of the connection time periods includes the specified time, or if a total amount of time between the start of a first time period of two or more consecutive time periods of connection to any WiFi access point and the specified time is greater than the first threshold and a last time period of the two or more consecutive time periods includes the specified time, the WiFi device is present in the environment, based on the time periods of connection to the WiFi access point and the time periods of disconnection from the WiFi access point in the connection sequence, it can be determined whether the WiFi device is present in the environment as of the specified time.

[0093] By determining that the WiFi device is not present in the environment if an amount of time between the start of one of the time periods disconnected from all WiFi access points is greater than a second threshold amount of time and one of the time periods disconnected includes a specific time, it can be determined whether the WiFi device is present in the environment as of the specified time based on the time periods connected to the WiFi access point and the time periods disconnected from the WiFi access point in the connection sequence.

[0094] The identifier of the WiFi device may be a Salted Hashed Media Access Control Address (SHMAC).

[0095] Prior to determining whether the WiFi device is present in the environment or not present in the environment as of a specified time based on a time period of connection to any WiFi access point and a time period of disconnection from all WiFi access points in a connection sequence, connection time data, connection / disconnection count data, conversion data, and connection / disconnection length data of the WiFi device may be generated from a report, sensor and device data may be received from sensors or devices in the environment, and a machine learning system may be used to generate a WiFi device indication indicating that the WiFi device should be used to determine the presence or absence of a user associated with the WiFi device in the environment, wherein the connection time data, connection / disconnection count data, conversion data, and connection / disconnection length data of the WiFi device and the sensor and device data are input into the machine learning system.

[0096] Additional reports may be received from WiFi access points in the environment. The additional reports may include an identifier of the second WiFi device, an indication of connection or disconnection with one of the WiFi access points, the time of connection or disconnection, and an identifier of one of the WiFi access points. Second connection time data, second connection / disconnection count data, second conversion data, and second connection / disconnection length data for the second WiFi device may be generated from the additional report. A second WiFi device indication indicating that the second WiFi device should not be used to determine the presence or absence of a second user associated with the second WiFi device in the environment may be generated using a machine learning system, wherein the second connection time data, second connection / disconnection count data, second conversion data, and second connection / disconnection length data of the second WiFi device and sensor and device data are input into the machine learning system.

[0097] Device data may be received from a WiFi device. The device data may be input into a machine learning system.

[0098] The device data may include geographic location data of the WiFi device and / or geo-fence data of the WiFi device.

[0099] A system may include WiFi access points in an environment and a computing device of a cloud computing system, the computing device of the cloud computing system receiving reports from the WiFi access points in the environment, wherein each of the reports includes an identifier of the WiFi device, an indication of connection or disconnection with one of the WiFi access points, a time of connection or disconnection, and an identifier of one of the WiFi access points, the computing device of the cloud computing system generating a connection sequence of the WiFi device from the reports, wherein the connection sequence includes a time of connection to the WiFi access point and a time of disconnection from the WiFi access point by the WiFi device, based on a time period of connection to any one of the WiFi access points and a time period of disconnection from any one of the WiFi access points in the connection sequence, the computing device of the cloud computing system determining whether the WiFi device is present in the environment or not present in the environment as of a specified time, and if it is determined that the WiFi device is present in the environment, the computing device of the cloud computing system generating an indication of presence for a user associated with the WiFi device, or if it is determined that the WiFi device is not present in the environment, the computing device of the cloud computing system generating an indication of absence for a user associated with the WiFi device.

[0100] The computing device of the cloud computing system may generate a control signal for a controllable device in the environment based on the indication of presence or absence, and send the control signal to the device for implementation by the device.

[0101] By determining that: if an amount of time between the start of one of the time periods of connection to any one of the WiFi access points and the specified time is greater than a first threshold amount of time and one of the time periods of connection includes the specified time, or if a total amount of time between the start of a first time period of two or more consecutive time periods of connection to any one of the WiFi access points and the specified time is greater than a first threshold and a last time period of the two or more consecutive time periods includes the specified time, the WiFi device is present in the environment, the computing device of the cloud computing system may determine whether the WiFi device is present or absent in the environment as of the specified time based on the time periods of connection to the WiFi access point and the time periods of disconnection from the WiFi access point in the connection sequence.

[0102] By determining that the WiFi device is not present in the environment if the amount of time between the start of one of the time periods disconnected from all WiFi access points is greater than a second threshold amount of time and one of the time periods disconnected includes the specified time, the computing device of the cloud computing system may determine whether the WiFi device is present in the environment as of the specified time based on the time periods connected to the WiFi access point and the time periods disconnected from the WiFi access point in the connection sequence.

[0103] The identifier of the WiFi device may include a Salted Hash Media Access Control Address (SHMAC).

[0104] Before determining whether the WiFi device is present in the environment or not present in the environment as of a specified time based on a time period of connection to any one WiFi access point and a time period of disconnection from all WiFi access points in a connection sequence, a computing device of the cloud computing system may generate connection time data, connection / disconnection count data, conversion data, and connection / disconnection length data of the WiFi device from a report; receive sensor and device data from sensors or devices in the environment; and the computing device of the cloud computing system generates a WiFi device indication using a machine learning system to indicate that the WiFi device should be used to determine the presence or absence of a user associated with the WiFi device in the environment, wherein the connection time data, connection / disconnection count data, conversion data, and connection / disconnection length data of the WiFi device and the sensor and device data may be input into the machine learning system.

[0105] The computing device of the cloud computing system may receive additional reports from WiFi access points in the environment, wherein each of the additional reports may include an identifier of a second WiFi device, an indication of connection or disconnection with one of the WiFi access points, a time of connection or disconnection, and an identifier of one of the WiFi access points; the computing device of the cloud computing system may generate second connection time data, second connection / disconnection count data, second conversion data, and second connection / disconnection length data for the second WiFi device from the additional reports; and generate a second WiFi device indication using a machine learning system indicating that the second WiFi device should not be used to determine the presence or absence of a second user associated with the second WiFi device in the environment, wherein the second connection time data, second connection / disconnection count data, second conversion data, and second connection / disconnection length data of the second WiFi device and sensor and device data may be input into the machine learning system.

[0106] The computing device of the cloud computing system can receive device data from the WiFi device. The device data can be input into the machine learning system.

[0107] The device data may include geographic location data of the WiFi device and geo-fence data of the WiFi device.

[0108] The invention comprises the following means: means for receiving reports from WiFi access points in the environment, wherein each of the reports includes an identifier of the WiFi device, an indication of connection or disconnection with one of the WiFi access points, a time of connection or disconnection, and an identifier of one of the WiFi access points; means for generating a connection sequence of the WiFi device from the reports, wherein the connection sequence includes a time of connection to the WiFi access point by the WiFi device and a time of disconnection from the WiFi access point; means for determining whether the WiFi device is present or absent in the environment as of a specified time based on a time period of connection to any one of the WiFi access points and a time period of disconnection from all of the WiFi access points in the connection sequence; means for generating an indication of presence for a user associated with the WiFi device if it is determined that the WiFi device is present in the environment, or generating an indication of absence for a user associated with the WiFi device if it is determined that the WiFi device is not present in the environment; means for generating a control signal for a controllable device in the environment based on the indication of presence or the indication of absence; means for sending the control signal to the device for implementation by the device; means for determining whether the amount of time between the start of one of the time periods of connection to any one of the WiFi access points and the specified time is greater than a first threshold amount of time and one of the time periods of connection includes the specified time, or if two or more of the time periods of connection to any one of the WiFi access points are connected to the environment. means for determining that the WiFi device is present in the environment if the total amount of time between the start of a first of the plurality of consecutive time periods and the specified time is greater than a first threshold, and a last of the two or more consecutive time periods includes the specified time; means for determining that the WiFi device is not present in the environment if the amount of time between the start of one of the time periods disconnected from all WiFi access points is greater than a second threshold amount of time and one of the disconnected time periods includes the specified time; means for generating connection time data, connection / disconnection count data, transition data, and connection / disconnection length data for the WiFi device from the report; means for receiving sensor and device data from one or more sensors or devices in the environment; means for generating, using a machine learning system, a WiFi device indication indicating that the WiFi device should be used to determine the presence or absence of a user associated with the WiFi device in the environment, wherein the connection time data, connection / disconnection count data, transition data, and connection / disconnection length data of the WiFi device and the sensor and device data are input to the machine learning system; means for receiving additional reports from WiFi access points in the environment, wherein each of the additional reports includes an identifier of a second WiFi device, an indication of connection or disconnection to one of the WiFi access points, a time of connection or disconnection, and an identifier of one of the WiFi access points;Means for generating second connection time data, second connection / disconnection count data, second conversion data, and second connection / disconnection length data of a second WiFi device from the additional report; means for generating a second WiFi device indication using a machine learning system, the second WiFi device indication indicating that the second WiFi device should not be used to determine the presence or absence of a second user associated with the second WiFi device in the environment, wherein the second connection time data, second connection / disconnection count data, second conversion data, and second connection / disconnection length data of the second WiFi device and the sensor and device data are input to the machine learning system; and means for receiving device data from the WiFi device, wherein the device data is input to the machine learning system.;

[0109] The embodiments disclosed herein may use one or more sensors. In general, a "sensor" may refer to any device that can obtain information about its environment. Sensors may be described by the type of information they collect. For example, sensor types as disclosed herein may include motion, smoke, carbon monoxide, proximity, temperature, time, physical orientation, acceleration, position, etc. Sensors may also be described in terms of specific physical devices that obtain environmental information. For example, an accelerometer may obtain acceleration information and may therefore be used as a general motion sensor and / or acceleration sensor. Sensors may also be described in terms of specific hardware components used to implement the sensor. For example, a temperature sensor may include a thermistor, a thermocouple, a resistance temperature detector, an integrated circuit temperature detector, or a combination thereof. In some cases, a sensor may operate as multiple sensor types sequentially or simultaneously, such as where a temperature sensor is used to detect temperature changes and the presence of a person or animal.

[0110] In general, a "sensor" as disclosed herein may include multiple sensors or sub-sensors, such as where a location sensor includes both a global positioning sensor (GPS) and a wireless network sensor that provides data that can be associated with a known wireless network to obtain location information. Multiple sensors may be arranged in a single physical housing, such as where a single device includes a motion sensor, a temperature sensor, a magnetic sensor, and / or other sensors. Such a housing may also be referred to as a sensor or sensor device. For clarity, when such a description is needed to understand the embodiments disclosed herein, the sensors are described with reference to the specific functions performed by the sensors and / or the specific physical hardware used.

[0111] Sensors may also include hardware other than specific physical sensors that obtain information about the environment. Figure 6An exemplary sensor as disclosed herein is shown. The sensor 60 may include an environmental sensor 61, such as a temperature sensor, a smoke sensor, a carbon monoxide sensor, a motion sensor, an accelerometer, a proximity sensor, a passive infrared (PIR) sensor, a magnetic field sensor, a radio frequency (RF) sensor, a light sensor, a humidity sensor, or any other suitable environmental sensor, which obtains information of a corresponding type about the environment in which the sensor 60 is located. The processor 64 may receive and analyze data obtained by the sensor 61, control the operation of other components of the sensor 60, and handle communications between the sensor and other devices. The processor 64 may execute instructions stored on a computer-readable memory 65. The memory 65 or another memory in the sensor 60 may also store environmental data obtained by the sensor 61. A communication interface 63, such as a Wi-Fi or other wireless interface, an Ethernet or other local network interface, etc., may allow communication with other devices through the sensor 60. A user interface (UI) 62 may provide information and / or receive input from a user of the sensor. The UI 62 may include, for example, a speaker to output an audible alarm when an event is detected by the sensor 60. Alternatively or additionally, the UI 62 may include a light to be activated when an event is detected by the sensor 60. The user interface may be relatively minimal, such as a limited output display, or it may be a full-featured interface, such as a touch screen. As will be readily appreciated by those skilled in the art, the components within the sensor 60 may send and receive information to and from each other via an internal bus or other mechanism. One or more components may be implemented in a single physical arrangement, such as where multiple components are implemented on a single integrated circuit. A sensor as disclosed herein may include other components, and / or may not include all of the illustrative components shown.

[0112] Sensors as disclosed herein can operate within a communication network, such as a conventional wireless network and / or a sensor-specific network through which sensors can communicate with each other and / or with dedicated other devices. In some configurations, one or more sensors can provide information to one or more other sensors, a central controller, or any other device capable of communicating with one or more sensors on a network. The central controller can be general or dedicated. For example, one type of central controller is a home automation network that collects and analyzes data from one or more sensors within a home. Another example of a central controller is a dedicated controller dedicated to a subset of functions, such as a security controller that primarily or exclusively collects and analyzes sensor data because it involves various security considerations of a location. The central controller can be located locally relative to the sensor with which it communicates and from which sensor data is obtained, such as in the case where it is located within a home that includes a home automation and / or sensor network. Alternatively or additionally, the central controller as disclosed herein can be remote from the sensor, such as where the central controller is implemented as a cloud-based system that communicates with multiple sensors, which can be located in multiple locations and can be local or remote relative to each other.

[0113] Figure 7 An example of a sensor network as disclosed herein is shown, which can be implemented by any suitable wired and / or wireless communication network. One or more sensors 71, 72 can communicate with each other and / or with a controller 73 via a local network 70, such as Wi-Fi or other suitable network. The controller can be a general or special purpose computer. The controller can, for example, receive, aggregate and / or analyze environmental information received from sensors 71, 72. The sensors 71, 72 and the controller 73 can be located locally to each other, such as within a single residence, office space, building, room, etc., or they can be remote from each other, such as where the controller 73 is implemented in a remote system 74 such as a cloud-based reporting and / or analysis system. Alternatively or additionally, the sensor can communicate directly with the remote system 74. The remote system 74 can, for example, aggregate data from multiple locations, provide instructions, software updates and / or aggregate data to the controller 73 and / or sensors 71, 72.

[0114] For example, hub computing device 400 may be an example of controller 73 , and sensor 210 may be an example of sensors 71 and 72 , as shown and described in further detail with reference to FIGS. 1-10 .

[0115] The security system of the disclosed subject matter and the devices of the smart home environment can be communicatively connected via a network 70, which can be a mesh network such as Thread, which provides a network architecture and / or protocol for devices to communicate with each other. A typical home network can have a single device communication point. Such a network may be prone to failure, so that when a single device point is not operating properly, the devices of the network cannot communicate with each other. The mesh Thread network that can be used in the security system of the disclosed subject matter can avoid the use of a single device for communication. That is, in a mesh network such as network 70, there is no single communication point that may fail and thus prohibit the devices coupled to the network from communicating with each other.

[0116] The communication and network protocols used by devices communicatively coupled to the network 70 can provide secure communications, minimize the amount of power used (i.e., be power efficient), and support a wide variety of devices and / or products in the home, such as appliances, access control, climate control, energy management, lighting, safety, and security. For example, the protocols supported by the network and devices connected thereto can be open protocols that can natively carry IPv6.

[0117] Thread networks, such as network 70, can be easy to set up and safe to use. Network 70 can use authentication schemes, AES (Advanced Encryption Standard) encryption, etc. to reduce and / or minimize security vulnerabilities that exist in other wireless protocols. Thread networks can be scalable to connect devices (e.g., 2, 5, 10, 20, 40, 100, 150, 200, or more devices) into a single network that supports multiple hops (e.g., to provide communication between devices when one or more nodes of the network are not operating normally). Network 70, which can be a Thread network, can provide security at the network and application layers. One or more devices (e.g., controller 73, remote system 74, etc.) communicatively coupled to network 70 can store a product installation code to ensure that only authorized devices can join network 70. One or more operations and communications of network 70 can use cryptography, such as public key cryptography.

[0118] Devices communicatively coupled to the network 70 of the smart home environment and / or security system disclosed herein can reduce power consumption and / or reduce power consumption. That is, the devices effectively communicate with each other and operate to provide functions to the user, wherein the devices can have a reduced battery size and increased battery life compared to conventional devices. The device may include a sleep mode to increase battery life and reduce power requirements. For example, communications between devices coupled to the network 70 may use the power-efficient IEEE 802.15.4 MAC / PHY protocol. In an embodiment of the disclosed subject matter, short message transmission between devices on the network 70 can save bandwidth and power. The routing protocol of the network 70 can reduce network overhead and latency. The communication interface of the device coupled to the smart home environment may include a wireless system-on-chip to support a low-power, secure, stable and / or scalable communication network 70.

[0119] Figure 7 The sensor network shown in can be an example of a smart home environment. The depicted smart home environment can include a structure, a house, an office building, a garage, a mobile home, etc. The devices of the smart environment, such as sensors 71, 72, controller 73, and network 70, can be integrated into a smart home environment that does not include an entire structure, such as an apartment, an apartment building, or an office space.

[0120] The smart environment can control and / or couple to devices external to the structure. For example, one or more of the sensors 71, 72 can be located external to the structure, for example, at one or more distances from the structure (e.g., the sensors 71, 72 can be disposed external to the structure, at points along the perimeter of the land where the structure is located, etc.). One or more of the devices in the smart environment need not be physically within the structure. For example, a controller 73 that can receive input from the sensors 71, 72 can be located external to the structure.

[0121] The structure of the smart home environment may include a plurality of rooms at least partially separated from each other via walls. The walls may include inner walls or outer walls. Each room may further include a floor and a ceiling. The devices of the smart home environment, such as sensors 71, 72, may be mounted on, integrated with and / or supported by the walls, floors or ceilings of the structure.

[0122] include Figure 7The illustrated sensor network's smart home environment may include a plurality of devices, including intelligent, multi-sensing, networked devices that can be seamlessly integrated with each other and / or with a central server or cloud computing system (e.g., controller 73 and / or remote system 74) to provide home security and smart home features. The smart home environment may include one or more intelligent, multi-sensing, networked thermostats (e.g., "smart thermostats"), one or more intelligent, networked multi-sensing hazard detection units (e.g., "smart hazard detectors"), and one or more intelligent, multi-sensing, networked entry interface devices (e.g., "smart doorbells"). The smart hazard detectors, smart thermostats, and smart doorbells may be Figure 7 Sensors 71, 72 shown in FIG.

[0123] According to embodiments of the disclosed subject matter, a smart thermostat can detect ambient climate characteristics (e.g., temperature and / or humidity) and can control a structure's HVAC (heating, ventilation, and air conditioning) system accordingly. For example, the ambient client characteristics can be determined by Figure 7 Sensors 71 , 72 shown in FIG. 7 detect, and a controller 73 can control the structure's HVAC system (not shown).

[0124] Smart hazard detectors can detect the presence of hazardous materials or materials that are indicative of hazardous materials (e.g., smoke, fire, or carbon monoxide). For example, smoke, fire, and / or carbon monoxide can be detected by Figure 7 The sensors 71 , 72 shown in the figure detect, and the controller 73 can control the alarm system to provide visual and / or audible alarms to users of the smart home environment.

[0125] The smart doorbell can control doorbell functions, detect a person approaching or leaving a location (e.g., an exterior door of a building), and announce the person approaching or leaving the building via an audible and / or visual message output by a speaker and / or display coupled to, for example, the controller 73.

[0126] In some embodiments, Figure 7 The smart home environment of the sensor network shown in the figure may include one or more intelligent, multi-sensing, networked wall switches (e.g., "smart wall switches"), one or more intelligent, multi-sensing, networked wall plug interfaces (e.g., "smart wall plugs"). The smart wall switch and / or smart wall plug may be Figure 7Sensors 71, 72 shown in . The smart wall switch can detect ambient lighting conditions and control the power and / or dimming state of one or more lights. For example, sensors 71, 72 can detect ambient lighting conditions, and controller 73 can control the power of one or more lights (not shown) in the smart home environment. The smart wall switch can also control the power state or speed of a fan, such as a ceiling fan. For example, sensors 72, 72 can detect the power and / or speed of the fan, and controller 73 can adjust the power and / or speed of the fan accordingly. The smart wall plug can control the supply of power to one or more wall plugs (e.g., so that if no person is detected within the smart home environment, power is not supplied to the plug). For example, one of the smart wall plugs can control the supply of power to a lamp (not shown).

[0127] In an embodiment of the disclosed subject matter, a smart home environment may include one or more intelligent, multi-sensing, networked entry detectors (eg, "smart entry detectors"). Figure 7 The sensors 71, 72 shown in the figure can be smart entry detectors. The illustrated smart entry detectors (e.g., sensors 71, 72) can be set at one or more windows, doors and other entry points of the smart home environment to detect when the window, door or other entry point is opened, broken, destroyed and / or damaged. When the window or door is opened, closed, broken and / or damaged, the smart entry detector can generate a corresponding signal to be provided to the controller 73 and / or the remote system 74. In some embodiments of the disclosed subject matter, unless all smart entry detectors (e.g., sensors 71, 72) indicate that all doors, windows, entryways, etc. are closed and / or all smart entry detectors are armed, an alarm system that can be included with the controller 73 and / or coupled to the network 70 may not be armed.

[0128] Figure 7 The illustrated sensor network smart home environment can include one or more intelligent, multi-sensing, networked door handles (e.g., "smart door handles"). For example, sensors 71, 72 can be coupled to a door handle (e.g., door handle 122 located on an exterior door of a structure in the smart home environment). However, it should be appreciated that the smart door handle can be provided on an exterior door and / or interior door of the smart home environment.

[0129] Smart thermostats, smart hazard detectors, smart doorbells, smart wall switches, smart wall plugs, smart entry detectors, smart door handles, keypads, and other devices in a smart home environment (e.g. Figure 7 ), can be communicatively coupled to each other via a network 70, and coupled to a controller 73 and / or a remote system 74 to provide safety, security and / or comfort for the smart environment.

[0130] A user can interact with one or more networked smart devices (e.g., via network 70). For example, a user can communicate with one or more networked smart devices using a computer (e.g., a desktop computer, a laptop computer, a tablet computer, etc.) or other portable electronic device (e.g., a smart phone, a tablet computer, a key fob, etc.). A web page or application can be configured to receive communications from a user and control one or more networked smart devices based on the communications and / or present information about device operation to the user. For example, a user can view a security system that can arm or disarm a home.

[0131] One or more users can use a networked computer or portable electronic device to control one or more networked smart devices in a smart home environment. In some examples, some or all users (e.g., individuals living at home) can register their mobile devices and / or key FOBs with the smart home environment (e.g., with controller 73). Such registration can be performed at a central server (e.g., controller 73 and / or remote system 74) to authenticate users and / or electronic devices as being associated with the smart home environment, and to provide users with permission to use electronic devices to control networked smart devices and security systems of the smart home environment. Users can use their registered electronic devices to remotely control networked smart devices and security systems of the smart home environment, such as when the resident is at work or on vacation. When the user is within the smart home environment, the user can also use their registered electronic devices to control networked smart devices.

[0132] Alternatively or in addition to registering electronic devices, the smart home environment can infer which individuals live in the home and are therefore users and which electronic devices are associated with those individuals. Thus, the smart home environment "learns" who the users are (e.g., authorized users) and permits electronic devices associated with those individuals to control networked smart devices of the smart home environment (e.g., devices communicatively coupled to the network 70). Various types of notifications and other information can be provided to users via messages sent to one or more user electronic devices. For example, messages can be sent via email, short message service (SMS), multimedia message service (MMS), unstructured supplementary service data (USSD), and any other type of messaging service and / or communication protocol.

[0133] The smart home environment may include communications with devices outside the smart home environment but within the immediate geographic range of the home. For example, the smart home environment may include an outdoor lighting system (not shown) that transmits information about the movement and / or presence of detected people, animals, and any other objects to a central server or cloud computing system (e.g., controller 73 and / or remote system 74) via the communication network 70 or directly, and receives back commands for controlling the lighting accordingly.

[0134] The controller 73 and / or remote system 74 can control the outdoor lighting system based on information received from other networked smart devices in the smart home environment. For example, if any networked smart device (such as a smart wall plug located outdoors) detects movement at night, the controller 73 and / or remote system 74 can activate the outdoor lighting system and / or other lights in the smart home environment.

[0135] In some configurations, the remote system 74 may aggregate data from multiple locations, such as multiple buildings, multiple resident buildings, individual residences within a neighborhood, multiple neighborhoods, etc. In general, as previously described with reference to Figure 8 The described multiple sensor / controller systems 81, 82 can provide information to the remote system 74. The systems 81, 82 can provide data directly from one or more sensors as previously described, or the data can be aggregated and / or analyzed by a local controller such as the controller 73, which then communicates with the remote system 74. The remote system can aggregate and analyze data from multiple locations, and can provide aggregated results to each location. For example, the remote system 74 can examine common sensor data or trends in sensor data for a large area, and provide information about the identified commonality or environmental data trends to each local system 81, 82.

[0136] Where the systems discussed herein collect personal information about a user or may utilize personal information, the user may be provided with an opportunity to control whether a program or feature collects user information (e.g., information about the user's social network, social actions or activities, occupation, the user's preferences, or the user's current location) or to control whether and / or how content that may be more relevant to the user is received from a content server. Additionally, certain data may be processed in one or more ways before it is stored or used so that personally identifiable information is removed. Thus, the user may control how information about the user is collected and used by the systems as disclosed herein.

[0137] Embodiments of the presently disclosed subject matter can be implemented in and used with a variety of computing devices. Fig. 92 is an example computing device 20 suitable for implementing embodiments of the presently disclosed subject matter. For example, the device 20 may be used to implement a controller, a device including a sensor as disclosed herein, etc. Alternatively or additionally, the device 20 may be, for example, a desktop or laptop computer, or a mobile computing device such as a smart phone, a tablet computer, etc. The device 20 may include a bus 21 that interconnects the main components of the computer 20, such as a central processor 24, a memory 27, such as a random access memory (RAM), a read-only memory (ROM), flash RAM, etc., a user display 22, such as a display screen), a user input interface 26, which may include one or more controllers and associated user input devices, such as a keyboard, a mouse, a touch screen, etc., a fixed storage 23, such as a hard drive, flash memory, etc., a removable media component 25 operable to control and receive an optical disk, a flash drive, etc., and a network interface 29 operable to communicate with one or more remote devices via a suitable network connection.

[0138] Bus 21 allows data communication between central processor 24 and one or more memory components 25, 27, which may include RAM, ROM and other memory as previously described. Applications resident in computer 20 are typically stored on and accessed via computer readable storage media.

[0139] Fixed storage 23 may be integrated with computer 20, or may be separate and accessed through other interfaces. Network interface 29 may provide a direct connection to a remote server via a wired or wireless connection. Network interface 29 may provide such connection using any suitable technology and protocol that will be readily understood by those skilled in the art, including digital cellular telephone, WiFi, Bluetooth(R), near field, etc. For example, network interface 29 may allow the device to communicate with other computers via one or more local area networks, wide area networks, or other communication networks, as described in further detail herein.

[0140] Fig.10An example network arrangement according to an embodiment of the disclosed subject matter is shown. One or more clients 10, 11, such as local computers, smart phones, tablet computing devices, etc., can be connected to other devices via one or more networks 7. The network can be a local area network, a wide area network, the Internet, or any other suitable communication network, and can be implemented on any suitable platform including wired and / or wireless networks. The client can communicate with one or more servers 13 and / or databases 15. The device can be directly accessed by the client 10, 11, or one or more other devices can provide intermediate access, such as where the server 13 provides access to resources stored in the database 15. The client 10, 11 can also access a remote platform 17 or a service provided by the remote platform 17, such as a cloud computing arrangement and service. The remote platform 17 can include one or more servers 13 and / or databases 15. One or more processing units 14 can be, for example, part of a distributed system, such as a cloud-based computing system, a search engine, a content distribution system, etc., which can also include a database 15 and / or a user interface 13 or communicate with a database 15 and / or a user interface 13. In some arrangements, the analysis system 5 may provide backend processing, such as where the stored or acquired data is pre-processed by the analysis system 5 before delivery to the processing unit 14 , database 15 , and / or user interface 13 .

[0141] Various embodiments of the presently disclosed subject matter may include or be embodied as computer-implemented processes and apparatus for practicing those processes. Embodiments may also be embodied in the form of a computer program product having a computer program code containing instructions embodied in a non-transitory and / or tangible medium, such as a hard drive, a USB (Universal Serial Bus) drive, or any other machine-readable storage medium, such that when the computer program code is loaded into and executed by a computer, the computer becomes an apparatus for practicing embodiments of the disclosed subject matter. When implemented on a general-purpose microprocessor, the computer program code may configure the microprocessor to become a special-purpose device, such as by creating specific logic circuits specified by the instructions.

[0142] Embodiments may be implemented using hardware that may include a processor, such as a general purpose microprocessor and / or an application specific integrated circuit (ASIC), that embodies all or part of the techniques according to embodiments of the disclosed subject matter in hardware and / or firmware. The processor may be coupled to a memory, such as a RAM, ROM, flash memory, a hard disk, or any other device capable of storing electronic information. The memory may store instructions suitable for execution by the processor to perform techniques according to embodiments of the disclosed subject matter.

[0143] For purposes of explanation, the foregoing description has been described with reference to specific embodiments. However, the illustrative discussion above is not intended to be exhaustive or to limit the embodiments of the disclosed subject matter to the precise forms disclosed. In view of the above teachings, many improvements and variations are possible. The embodiments are selected and described in order to explain the principles of the embodiments of the disclosed subject matter and their practical applications, so that others skilled in the art can utilize those embodiments and various embodiments with various improvements that may be suitable for the specific purposes contemplated.

Claims

1. A computer-implemented method executed by a data processing device, the method comprising: receiving data from wireless access points in an environment, the data providing an indication of a connection or disconnection by a device to one of the wireless access points and a time of the connection or disconnection; Generate a connection sequence of the device according to the data, wherein the connection sequence indicates one or more of: a time when the device is connected to the wireless access point or a time when the device is disconnected from the wireless access point; and A determination is made based at least in part on the connection sequence of the device whether the device is present in the environment or absent from the environment as of a specified time.

2. The computer-implemented method of claim 1 , further comprising: if it is determined that the device is present in the environment, generating an indication of presence for a user associated with the device, or if it is determined that the device is not present in the environment, generating an indication of absence for the user associated with the device; generating a control signal for a controllable device in the environment based on the indication of presence or the indication of absence; as well as The control signal is sent to the device to be implemented by the device.

3. The computer-implemented method of claim 1 , wherein: determining whether the device is present or absent from the environment as of a specified time based at least in part on the connection sequence of the device, further comprising: The device is determined to be present in the environment if an amount of time between the start of a time period of connection to any one of the wireless access points and the specified time is greater than a first threshold amount of time and the time period of connection includes the specified time, or if a total amount of time between the start of a first time period of two or more consecutive time periods of connection to any one of the wireless access points and the specified time is greater than the first threshold amount of time and a last time period of the two or more consecutive time periods includes the specified time.

4. The computer-implemented method of claim 1 , wherein: determining whether the device is present or absent from the environment as of a specified time based at least in part on the connection sequence of the device, further comprising: If the amount of time between the start of a time period in which all of the wireless access points are disconnected and the designated time is greater than a second threshold amount of time and the one of the disconnected time periods includes the designated time, it is determined that the device is not present in the environment.

5. The computer-implemented method of claim 1 , further comprising: Additional data is received from the wireless access point in the environment, wherein the additional data includes an identifier of the device, wherein the identifier of the device includes a salted hashed media access control address (SHMAC).

6. The computer-implemented method of claim 1 , further comprising: Prior to determining whether the device is present or absent from the environment as of a specified time based at least in part on the connection sequence of the device: generating connection time data, connection / disconnection count data, conversion data, and connection / disconnection length data of the device based on the data; receiving sensor and device data from one or more sensors or devices in the environment; as well as A device indication is generated using a machine learning system, wherein the device indication indicates that the device should be used to determine whether a user associated with the device is present or absent from the environment, wherein the connection time data, the connection / disconnection count data, the conversion data and the connection / disconnection length data of the device and the sensor and device data are input into the machine learning system.

7. The computer-implemented method of claim 6, further comprising: receiving additional data from the wireless access points in the environment, wherein the additional data includes an identifier of a second device, an indication of a connection or disconnection with one of the wireless access points, a time of the connection or disconnection, and an identifier of the one of the wireless access points; generating second connection time data, second connection / disconnection count data, second conversion data, and second connection / disconnection length data of the second device from the additional data; and A second device indication is generated using the machine learning system, wherein the second device indication indicates that the second device should not be used to determine whether a second user associated with the second device is present or absent from the environment, wherein the second connection time data, the second connection / disconnection count data, the second conversion data, and the second connection / disconnection length data of the second device and the sensor and device data are input into the machine learning system.

8. The computer-implemented method of claim 6, further comprising: Device data is received from the device, wherein the device data is input to the machine learning system.

9. The computer-implemented method of claim 8, wherein: The device data includes at least one of: geographic location data of the device and geo-fence data of the device.

10. A computer-implemented system for determining presence and absence of a user using a wireless connection, the system comprising: a computing device of a cloud computing system, the computing device of the cloud computing system receiving data from wireless access points in an environment, wherein the data includes an indication of connection or disconnection by a device to one of the wireless access points and a time of the connection or disconnection; Generate a connection sequence of the device according to the data, wherein the connection sequence includes one or more of the following: a time when the device is connected to the wireless access point or a time when the device is disconnected from the wireless access point; and A determination is made based on the connection sequence of the device whether the device is present in the environment or absent from the environment as of a specified time.

11. The computer-implemented system of claim 10, wherein: The computing device of the cloud computing system: if it is determined that the device exists in the environment, an indication of existence is generated for a user associated with the device, or if it is determined that the device does not exist in the environment, an indication of non-existence is generated for the user associated with the device, a control signal for a controllable device in the environment is generated based on the indication of existence or the indication of non-existence, and the control signal is sent to the device to be implemented by the device.

12. The computer-implemented system of claim 10, wherein: The computing device of the cloud computing system determines whether the device exists in the environment or does not exist in the environment as of a specified time based on the connection sequence of the device by the following operation: if an amount of time between the start of a time period of connection to any one of the wireless access points and the specified time is greater than a first threshold time amount and the time period of the connection includes the specified time, or if a total amount of time between the start of a first time period of two or more consecutive time periods of connection to any one of the wireless access points and the specified time is greater than the first threshold time amount and a last time period of the two or more consecutive time periods includes the specified time, it is determined that the device exists in the environment.

13. The computer-implemented system of claim 10, wherein: The computing device of the cloud computing system determines whether the device exists in the environment or not in the environment as of a specified time based on the connection sequence of the device by the following operation: if the amount of time between the start of a time period in which all the wireless access points are disconnected and the specified time is greater than a second threshold time amount and the one of the disconnected time periods includes the specified time, determining that the device does not exist in the environment.

14. The computer-implemented system of claim 10, wherein: The computing device of the cloud computing system further receives additional data from the wireless access point, the additional data comprising an identifier of the device, wherein the identifier of the device comprises a salted hashed media access control address (SHMAC).

15. The computer-implemented system of claim 10, wherein: The computing device of the cloud computing system, before determining whether the device is present in the environment or not present in the environment as of a specified time based on the connection sequence of the device: generates connection time data, connection / disconnection count data, conversion data and connection / disconnection length data of the device based on the data, receives sensor and device data from one or more sensors or devices in the environment, and generates a device indication using a machine learning system, wherein the device indication indicates that the device should be used to determine whether a user associated with the device is present in the environment or not present in the environment, wherein the connection time data, the connection / disconnection count data, the conversion data and the connection / disconnection length data of the device and the sensor and device data are input into the machine learning system.

16. The computer-implemented system of claim 15, wherein: The computing device of the cloud computing system further receives additional reports from the wireless access points in the environment, wherein each of the additional reports includes an identifier of a second device, an indication of connection or disconnection with one of the wireless access points, the time of the connection or disconnection, and an identifier of the one of the wireless access points, and generates second connection time data, second connection / disconnection count data, second conversion data, and second connection / disconnection length data for the second device from the additional reports, and generates a second device indication using the machine learning system, the second device indication indicating that the second device should not be used to determine whether a second user associated with the second device is present or absent in the environment, wherein the second connection time data, the second connection / disconnection count data, the second conversion data, and the second connection / disconnection length data for the second device and the sensor and device data are input into the machine learning system.

17. The computer-implemented system of claim 15, wherein: The computing device of the cloud computing system further receives device data from the device, wherein the device data is input into the machine learning system.

18. The computer-implemented system of claim 17, wherein: The device data includes at least one of: geographic location data of the device and geo-fence data of the device.

19. A system, comprising: One or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, are operable to cause the one or more computers to perform operations comprising: receiving data from one or more wireless access points in an environment, the data providing an indication of a connection or disconnection by a device to one of the wireless access points and a time of the connection or disconnection; Generate a connection sequence for the device based on the data, wherein the connection sequence includes one or more of the following: a time when the device is connected to the one or more wireless access points or a time when the device is disconnected from the wireless access points; and A determination is made based on the connection sequence of the device whether the device is present in the environment or absent from the environment as of a specified time.

20. The system of claim 19, wherein: The instructions further cause the one or more computers to perform operations including: before determining whether the device is present or absent from the environment as of a specified time based on a time period of connection to any one of the one or more wireless access points and a time period of disconnection from all of the one or more wireless access points in the connection sequence: generating connection time data, connection / disconnection count data, conversion data, and connection / disconnection length data of the device based on the data; receiving sensor and device data from one or more sensors or devices in the environment; and A device indication is generated using a machine learning system, wherein the device indication indicates that the device should be used to determine whether the user associated with the device is present in the environment or not present in the environment, wherein the connection time data, the connection / disconnection count data, the conversion data and the connection / disconnection length data of the device and the sensor and device data are input into the machine learning system.

Images