Use power and network data to identify device state changes

Abstract

Device state changes in a building may be determined using a combination of power monitoring and network monitoring. Power monitoring may be performed by obtaining a power monitoring signal and processing the power monitoring signal using a model to determine information about a change in state of one or more devices in the building. Network monitoring may be performed by receiving information about network packets transmitted by devices in the building and processing the information about the network packets to determine information about a change in state of one or more devices in the building. For some devices and some embodiments, information about a change in state of a device may be determined by using both power monitoring and network monitoring.

Description

[0001] This application is a divisional application of patent application number 2018800266529, application date October 22, 2019, and titled “Using power data and network data to identify device state changes.”

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims priority to and is a continuation-of U.S. patent application No. 15 / 439,788 (U.S. Patent No. 9,699,529) filed on February 22, 2017 (SAGE-0003-U01); U.S. patent application No. 15 / 439,796 (U.S. Patent No. 9,800,958) filed on February 22, 2017 (SAGE-0003-U02); and U.S. patent application No. 15 / 599,466 (SAGE-0003-U01-C01) filed on May 19, 2017. Background Art

[0004] Reducing electricity or power usage provides, among other things, the benefits of saving money by lowering payments to electric utilities and also protecting the environment by reducing the amount of resources required to generate electricity. Therefore, electricity users, such as consumers, businesses, and other entities, may desire to reduce their electricity usage to achieve these benefits. If users have information about what appliances (e.g., refrigerators, ovens, dishwashers, stoves, and light bulbs) in their homes and buildings use the most electricity and what actions can be taken to reduce electricity usage, users may be able to reduce their electricity usage more effectively.

[0005] Power monitors for individual devices can be used to measure the power usage of a single device. For example, a device can be plugged into a power monitor, and the monitor can in turn be plugged into a wall outlet. These monitors can provide information about the power usage of the device to which they are attached, but using these monitors to monitor all or even many devices in a house or building may not be practical because it would require a large amount of equipment, which can be expensive and require significant manpower to install.

[0006] Instead of power monitors for individual devices, power monitors can be installed at distribution panels to obtain information about the power used simultaneously by many devices. Power monitors at distribution panels are more convenient because a single monitor can provide aggregated usage information for many devices. However, extracting more specific information about the power used by a single device is more difficult because the monitors typically measure signals reflecting the collective operation of many devices, which may overlap in complex ways. The process of obtaining information about the power usage of a single device from electrical signals corresponding to usage by many devices is called disaggregation.

[0007] In order to provide maximum benefit to end users, a need exists for more accurate disaggregation techniques so that end users receive accurate information about the electrical usage of individual devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The following detailed description of the invention and certain embodiments thereof may be understood by reference to the following drawings:

[0009] Figure 1 is an example of a system for using power monitoring signals to identify devices and state changes of devices.

[0010] Figure 2 is an example of a power monitoring signal.

[0011] Figure 3 is an example of a system for using network data to identify devices and state changes of devices.

[0012] Figure 4A 、 4B 4C illustrate techniques for using network data to identify devices and state changes of devices.

[0013] Figure 5 is an example of a system that identifies devices and changes in their state.

[0014] Figure 6 is an example list of devices.

[0015] Figure 7 is an example of a system for using power monitoring signals and network data to identify devices and state changes of devices.

[0016] Figure 8A and 8B is an example timeline of power monitoring signals and network data.

[0017] Figure 9 is a flow chart illustrating an example implementation of using power monitoring signals and network data to identify devices.

[0018] Figure 10is a flow chart illustrating an example implementation of using power monitoring signals and network data to identify state changes of a device.

[0019] Figure 11 is a flow diagram illustrating an example implementation of training a power model using network data.

[0020] Figure 12 is an example of a device for identifying devices and device status changes using power monitoring signals or network data. DETAILED DESCRIPTION

[0021] Techniques for identifying devices and determining information about changes in the state of devices in a building are described herein. One data source for determining information about devices in a building is the electrical wiring that provides power to the devices in the building. Electrical sensors can be placed on the electrical wiring (or wires) that provide power to the building, and disaggregation techniques can be used to determine information about individual devices in the building. For example, any of the techniques described in U.S. Patent No. 9,443,195 (hereby incorporated by reference in its entirety for all purposes) can be used to determine information about devices using electrical measurements.

[0022] Another data source for identifying devices or determining information about changes in device status in a building is the building's computer network. The building may have a network (such as a local area network), and devices may be connected to the network via cables (e.g., Ethernet cables) or wirelessly (e.g., Wi-Fi). A building may have multiple networks, such as a local area network coordinated by a wireless router, a mesh network created by other devices working together (e.g., Sonos speakers), and a personal area network (e.g., Bluetooth connections between devices). Information about devices on these networks can be obtained, for example, by receiving service broadcasts from the device after it is turned on or by polling the device for information.

[0023] Some devices may be powered by the electrical wiring in a building but may not have a network connection (e.g., a conventional refrigerator). Some devices may be powered by the electrical wiring in a building and have a network connection (e.g., a "smart" refrigerator). Some devices may have a network connection but may not be powered by the electrical wiring (e.g., mobile devices such as smartphones), or may only be powered by the electrical wiring sometimes (e.g., when charging). The techniques described herein can provide more information and / or more accurate information about devices in a building by using a combination of information from the electrical wiring and information from the network in the building.

[0024] Information from the power lines and from the network in the building can be used to provide services to users to inform them about the status of the equipment in the building. A company can provide a power monitoring device that can be installed in the building and connected to both the building's power lines and the computer network. The power monitoring device can use both the power line and the network data to determine information about what equipment is present in the building and also the status of the equipment (e.g., on or off). This information can then be made available to the user, such as by presenting it in a dedicated app or web page (e.g., on a smartphone). The service can provide users with information about the equipment in the building, such as real-time information about the status of the equipment, the real-time power used by the equipment, and historical information about the activity of the equipment.

[0025] For clarity of presentation, the technology described herein will use a house or home as an example of a building to which the technology may be applied, but the technology described herein is equally applicable to any environment in which electricity is used, including but not limited to businesses and commercial buildings, government buildings, and other locations. References to homes throughout should be understood to encompass such other locations.

[0026] Power Monitoring

[0027] Figure 1 An example of a system 100 is illustrated in which information obtained from electrical wiring can be used to determine information about devices in a home. Figure 1 In the example, an electric company 120 provides electric power to a home 110. The electric power is transmitted to a distribution panel 140 where it can then be distributed to different circuits in the house.

[0028] The distribution panel 140 can be any distribution panel that can be found in a building. For example, the distribution panel 140 can implement split-phase electrical power, where a 240-volt AC electrical signal is converted using a split-phase transformer into a three-wire distribution having a single ground and two mains (or branches) each providing 120 volts. Some devices in the house can use one of the two mains to obtain 120 volts; other devices in the house can use the other main to obtain 120 volts; and other devices can use both mains to obtain 240 volts.

[0029] Any type of switchboard can be used, and the technology described herein is not limited to split-phase switchboards. For example, switchboard 140 can be single-phase, two-phase, or three-phase. The technology is also not limited to the number of mains provided by switchboard 140. In the following discussion, switchboard 140 will be described as having two mains, but any number of mains can be used, including only a single main. As will be understood by those skilled in the art, other voltage standards, such as those used in other countries or continents, are intended to be encompassed herein.

[0030] Figure 1 Illustrated are devices that consume power provided by power distribution board 140. For example, power monitor 150, refrigerator 160, and stove 165 may consume power provided via power distribution board 140.

[0031] Power monitor 150 can be connected to sensor 130 to measure electrical properties of electrical wiring connected to house 110. For example, sensor 130 can measure the voltage and / or current level of electrical wiring that provides power to distribution board 140. The measurements can be obtained using any available sensor, and the technology is not limited to any particular sensor or any particular type of value that can be obtained from the sensor. Sensor 130 can include multiple sensors, such as one or more sensors for each main line.

[0032] Sensor 130 may provide one or more power monitoring signals to power monitor 150, such as measurements of the current and / or voltage of each mains connected to distribution board 140. Power monitor 150 may process the power monitoring signals to disaggregate them or to obtain information about individual devices in the home. For example, power monitor 150 may determine that the state of a device has changed, such as a television turning on at 8:30 PM, or a refrigerator's compressor starting at 10:35 AM and 11:01 AM.

[0033] Power monitor 150 may be a device that is obtained separately from distribution panel 140 and installed by a user or an electrician to connect to distribution panel 140. Power monitor 150 may be part of distribution panel 140 and installed by the manufacturer of distribution panel 140. Power monitor 150 may also be part of (e.g., integrated with or into) an electric meter, such as one provided by an electric utility company and sometimes referred to as a smart meter.

[0034] Power monitor 150 may use any suitable technique for performing disaggregation or determining information about power usage or status of individual devices from power monitoring signals. For example, power monitor 150 may use any of the techniques described in US Pat. No. 9,443,195.

[0035] Figure 2 An example of a hypothetical power monitor signal 200 that may be processed by the power monitor 150 is illustrated. Figure 2 The diagram illustrates changes in the power monitoring signal caused by changes in the states of two appliances, a stove and an incandescent light bulb. Each of the changes in the power signal may be referred to as a power event. Figure 2In FIG, a power event labeled HE1 corresponds to the heating element of the stove's burner being turned on, and a power event labeled HE0 corresponds to the heating element of the burner being turned off (for an electric stove, the burner may appear to be on to the user, but the stove may turn the heating element on and off on a periodic basis to maintain a desired temperature). A power event labeled I1 corresponds to an incandescent light bulb being turned on, and a power event labeled I0 corresponds to an incandescent light bulb being turned off.

[0036] At power event 210, the stove's burner is turned on, and the burner's heating element consumes power. Consequently, power usage increases in the power monitoring signal. While the heating element is on, at power event 220, the incandescent light bulb is turned on, further increasing power usage. At power events 230, 240, 250, and 260, the heating element is turned off, then turned on, then off, and then turned back on. At power event 270, the incandescent light bulb is turned off, and at power event 280, the stove's burner is turned off, and the heating element stops consuming power.

[0037] The power monitor 150 can process the power monitoring signal 200 to identify the device and / or determine a change in the state of the device. Some state changes may correspond to a person turning the device on or off, and some state changes may correspond to a change in the function of the device (e.g., cycling a heating element or changing a washing machine from a wash mode to a spin mode). The power monitor 150 can use any suitable technique for identifying the device and determining a change in the state of the device, such as any of the techniques described in U.S. Patent No. 9,443,195.

[0038] In some embodiments, the power monitor 150 can utilize a mathematical model to process the power monitoring signal to identify the device and determine the state change of the device. Such a model will be referred to as a power model. For example, the power monitor 150 can have power models for different types of devices (e.g., stoves, dishwashers, refrigerators, etc.), devices of different manufacturers (e.g., Kenmore dishwashers, Maytag dishwashers, etc.), different versions of devices (e.g., Kenmore 1000 dishwashers), and specific devices (e.g., dishwashers at 100 Main Street). The power monitor 150 can also have power models for special state changes, such as when the device is turned on, the device is turned off, or the device changes operation in another way (the water pump of the dishwasher is turned on or off). In common use, the "1000" in the Kenmore 1000 dishwasher can be referred to as the "model" of the dishwasher, but in order to avoid confusion with the mathematical model, the "model" of the dishwasher will be referred to as a "version" in this article instead.

[0039] When a power monitoring signal is processed using a power model (such as any of the power models mentioned above), the power model may generate a score (e.g., probability, likelihood, confidence, etc.) indicating a match between the power model and the power monitoring signal. Figure 2 A portion of the power monitoring signal, including any event in the power event, can be processed by multiple power models, and a score can be generated by each of the power models. The power model score can be used to identify devices in the home and / or determine a change in state of devices in the home.

[0040] In some embodiments, the power monitoring signal processing can be performed as follows. The power monitoring signal can be continuously processed to identify power events in the power monitoring signal. When the power monitoring signal has a stable value or slight fluctuations, there may not be any device state changes in the house. The power event detection component can detect changes in the electrical signal that may correspond to device state changes, and cause these parts of the power monitoring signal to be further processed. The power event detection component can be implemented using any appropriate technology (such as a classifier).

[0041] After detecting a power event, a signature generating component may be used to generate a signature from the portion of the power monitoring signal that includes the power event.Any suitable signature may be calculated, such as any of the signatures described in US Patent 9,443,195.

[0042] The signature can then be processed by one or more power models to identify the device or determine a device state change corresponding to the power event. Any suitable power model can be used, such as a transition model, a device model, a wattage model, and a prior model described in U.S. Patent 9,443,195. For example, each power model can generate a score, and the device state change can be determined by selecting the power model with the highest score.

[0043] Now, we'll describe techniques for using power models to identify devices in a home. In some implementations, the power monitor 150 can be installed with an initial set of power models corresponding to devices that may be in the home. When processing power monitoring signals, the power models can be used to generate scores for power events occurring in the power monitoring signals. These scores can be used to identify which devices are in the home.

[0044] In some embodiments, if the score generated by the power model exceeds a threshold, the power monitor 150 may identify the device as being present in the house. For example, when a dishwasher model processes a portion of the power monitoring signal corresponding to the dishwasher being started, the dishwasher model may generate a score exceeding a threshold. The threshold may be specific to the dishwasher model, or the threshold may be the same for all power models. In some embodiments, a confidence level may be calculated in addition to or instead of the score, and the device may be identified when the confidence level exceeds the threshold.

[0045] In some embodiments, additional criteria may be considered before identifying a device. For example, the power monitor 150 may have multiple dishwasher power models corresponding to different types of dishwashers (e.g., conventional dishwashers and energy-efficient dishwashers), or there may be power models for dishwashers of different makes. When processing a power monitoring signal corresponding to the start of a dishwasher, multiple dishwasher models may generate scores (or confidence levels) that exceed a threshold. Instead of identifying multiple dishwashers, a single dishwasher corresponding to the model with the highest score that exceeds the threshold may be identified. For example, the power monitor 150 may have power models for a Kenmore dishwasher and a Bosch dishwasher. Each model may generate a score that exceeds the threshold, but the score for the Kenmore model may be higher than the score for the Bosch model. Therefore, the Kenmore dishwasher may be identified.

[0046] In some embodiments, the power monitor 150 can be updated with additional power models after a device has been identified. For example, after determining that a house has a dishwasher, a power model can be added to the power monitor 150 corresponding to the most common make of dishwasher. The power monitoring signal can then be processed using power models for dishwashers of different makes, and the highest-scoring power model can be used to identify the make of the dishwasher in the house. This process can be repeated to determine additional information, such as the make of the dishwasher (e.g., Kenmore 1000 dishwasher).

[0047] After having identified the devices in the house, the power monitor 150 can process the power monitoring signals to determine changes in state of the identified devices. In addition to having power models for specific devices, the power monitor 150 can also have models for specific state changes of the devices, and these models can be used to determine when the devices change state.

[0048] As described above, a power model for a device's state change can be used to process the power monitoring signal to generate a score for a possible state change. When the score (or confidence level) exceeds a threshold, the power monitor 150 can determine that a state change corresponding to the model has occurred. For example, the power monitor 150 can have power models for the start and end operations of a dishwasher. When the power model for the start of the dishwasher generates a score that exceeds a threshold, the power monitor 150 can determine that the dishwasher has started. Similarly, when the power model for the end of the dishwasher generates a score that exceeds a threshold, the power monitor 150 can determine that the dishwasher has ended its cycle.

[0049] The power model used to determine a state change need not be the same as the power model used to identify the device, and the threshold used to determine a state change need not be the same as the threshold used to identify the device. The model and threshold can be adjusted to obtain a desired tradeoff between accuracy and error rate.

[0050] Various aspects of the operations described above for identifying a device and determining a change in state of the device can be performed by another computer instead of the power monitor 150 (e.g., a server computer operating in conjunction with the power monitor). For example, in some embodiments, the power monitor 150 can provide a power monitoring signal to another computer (e.g., a server), and the other computer can identify the device and the change in state. In some embodiments, the other computer can identify the device, and the power monitor 150 can determine the change in state of the device.

[0051] Network Monitoring

[0052] In some embodiments, the power monitor 150 can be connected to a computer network in a house. For example, the power monitor 150 can have a wired connection to a router in the house (e.g., LAN Ethernet), can have a wireless connection to the network (e.g., Wi-Fi), or can have a direct network connection to other devices (e.g., Bluetooth). In these embodiments, the power monitor 150 is also a network monitor, but for clarity of presentation, the following description will continue to use the term power monitor.

[0053] Figure 3An example system 300 is illustrated in which a power monitor 150 is connected to a network in a house. In this example, the power monitor 150 has a network connection to a network device 310, which can be any device that facilitates a network in the house, such as a modem, router, or hub. Other devices in the house can also be connected to the network device 310. For example, a television 320 (e.g., a smart TV), a computer 330 (e.g., a personal computer), a smart switch 340 (e.g., a Phillips Hue or Belkin Wemo switch), and a phone 360 ​​(e.g., an Android phone or iPhone) can also be connected to the home network. The power monitor 150 can connect to the other devices in the house using any suitable wired or wireless network configuration (such as a LAN Ethernet or a direct connection to other devices). In some embodiments, the power monitor 150 can communicate on multiple networks simultaneously (e.g., a Wi-Fi connection to a home router and a Bluetooth connection to a specific device).

[0054] The power monitor 150 can use data transmitted over a computer network to learn about devices in the house. For example, the power monitor 150 can listen for broadcast messages from other devices, can poll other devices, or can listen for network data generated by other devices. In some embodiments, the power monitor 150 can learn about other devices in the house indirectly via a network external to the house. For example, a device in the house can transmit information to a third-party server operating in conjunction with the device, and the third-party server can transmit information to a server operating in conjunction with the power monitor 150. Each of these techniques can be used to identify devices in the house and determine a change in the state of the device (e.g., whether the device is turned on or off).

[0055] In some embodiments, devices in a premises may transmit data that includes information about the device itself (e.g., broadcast messages or responses to polls). For example, network transmissions may include any of the following: status (e.g., a device that has just been turned on or is about to be turned off), services provided by the device, a user-assigned name (e.g., "John's Mac"), make, hardware version, software version, network address (e.g., IP address), identification number (e.g., MAC address, device serial number, universally unique identifier, globally unique identifier, or temporary identifier), or other information (e.g., protocol-specific identification (such as a ZeroConf service name) or a resource locator used with the SSDP protocol).

[0056] Figure 4AThe diagram shows an example timeline of a power monitor 150 learning information about another device by listening to a broadcast message from the other device (the broadcasting device). Some devices can be configured to broadcast information across a network to inform other devices on the network about services or capabilities provided by the device. For example, a television can provide a service to allow other devices (e.g., a phone or personal computer) to stream video content to the television for display on the television, and the television can broadcast a message to the network so that other devices know that this service is available.

[0057] exist Figure 4A In , the broadcasting device transmits a message to announce the availability of a service and later transmits a message to announce the withdrawal of the service. Figure 4A , time progresses from top to bottom, and at 410, the broadcasting device is turned on, such as by a person. Then, at 411, the broadcasting device can announce the services it provides by broadcasting one or more messages on the network, and the message can be received by other devices on the network (including power monitor 150). Later, the broadcasting device can receive a power outage event at 412, such as input (e.g., from a person) to turn itself off. After receiving the input, the broadcasting device can broadcast a message to other devices on the network that the service is revoked at 413, and the message can be received by power monitor 150. After broadcasting the message, the broadcasting device can be turned off at 414.

[0058] Broadcasts are not limited to after a device is turned on and before a device is turned off, and broadcasts can be used at any time to announce any changes in service or to alert other devices on the network that a service is available. For example, a device can broadcast a message before going to sleep or after waking up from sleep mode to indicate the status of the device, to indicate that the device is in use (e.g., a song is playing or information about the song being played), or to indicate that the type of media being presented has changed. In some embodiments, a device can broadcast messages using Simple Service Discovery Protocol (SSDP), zero configuration networking (zeroconf), or NetBIOS. For example, in the case of utilizing SSDP, a device can use multicast addressing to broadcast messages to all devices on the network.

[0059] Information from the broadcast message can be used to identify devices in the house. For example, the message may include data that can be used to identify the device, such as text or an identifier that describes the device. Information from the broadcast message can also be used to determine the status of the device. For example, announcing the availability of a service may indicate that the device is turned on and announcing the withdrawal of a service may indicate that the device is turned off. In some embodiments, the broadcast message may provide some information about the device, and after receiving the broadcast message, the power monitor 150 may poll the device to obtain additional information about the status of the device. For example, a network speaker system may broadcast that the service is available, and the power monitor 150 may then poll the network system to discover that it is currently playing music and information about the music being played (e.g., song title, volume, etc.).

[0060] Figure 4B An example timeline is illustrated of power monitor 150 learning information about another device using a polling technique. Power monitor 150 can poll other devices one at a time, or can poll multiple devices simultaneously, but for clarity of presentation, only one polling device is shown.

[0061] exist Figure 4B At the start of the timeline in , the polling device is in the off state. At 420, the power monitor 150 polls the polling device, but because the polling device is in the off state, the polling device does not respond to the polling request. The power monitor 150 can determine that the polling device is in the off state when it does not receive a response to the polling request (or may not receive a response to multiple polling requests).

[0062] At 421, the polling device is turned on, for example, by a person. At 422, the power monitor 150 polls the polling device again. Since the polling device is now turned on, the polling device responds at 423. After receiving the response from the polling device, the power monitor 150 can determine that the polling device has transitioned from the off state to the on state. The polling response can include information that can be used to determine the state of the device, as described in more detail below. The polling response can include any of the information described above for the broadcast message.

[0063] At 424, the power monitor 150 polls the polling device again, and at 425, the polling device responds to the polling request, so that the power monitor 150 can determine that the polling device is still in the on state. At 426, the polling device is turned off. At 427, the power monitor 150 polls the polling device again and does not receive a response, so the power monitor 150 can determine that the polling device is now in the off state.

[0064] Any suitable polling technique may be used to poll devices. In some embodiments, the broadcast techniques described above may also allow for polling. For example, SSDP may allow a device to poll other devices on a network to determine what services those devices provide, and those devices may respond with messages similar to the broadcast messages described above. In some embodiments, lower-level protocols may be used for polling, such as Internet Control Message Protocol (ICMP) sending echo messages (pings) or Address Resolution Protocol (ARP) sending echo messages.

[0065] Information from the poll response can be used to identify devices in the house. For example, the poll response can include data that can be used to identify the device, such as text or an identifier describing the device. The poll response can also be used to determine the state of the device. For example, the lack of a response to the poll request can indicate that the device is off, a response can indicate that the device is on, and information in the response can provide additional information (e.g., the song currently playing).

[0066] Figure 4C 1 illustrates an example timeline of power monitor 150 learning information about another device by monitoring network data sent by the device to the other device. In some embodiments, individual device monitoring may only be used with explicit selection by the user, or only network packet headers (rather than packet bodies) may be used to avoid collecting too much information or sensitive information.

[0067] exist Figure 4C , the monitored device is turned on at 430. At 431, 432, 433, and 434, the monitored device transmits data over the network to another device, which may be a different device than the power monitor 150. At 435, the monitored device is turned off and stops sending network transmissions. The power monitor 150 may passively receive the network data over the network, such as by passively receiving network data packets.

[0068] The power monitor 150 may process the received data to determine information about the monitored device. Information in the received data may be used to identify the device, such as text or an identifier describing the device. The received data may also be used to determine the status of the device. For example, the fact that the device is transmitting data indicates that the device is on, while if the device does not transmit any data for a period of time, it can be determined that the device is off.

[0069] In some embodiments, the power monitor 150 may have information about publicly available APIs for third-party devices and use these APIs to determine whether such third-party devices are present in the house. For example, the power monitor 150 may periodically issue requests using the Nest thermostat API to determine whether a Nest thermostat is present in the house. After determining that a Nest thermostat is in the house, it may be polled more frequently to determine the status of the Nest thermostat or the status of the heating / cooling system. The API may allow, for example, other devices to determine the temperature of the house, a desired temperature set by a user, or the status of a heating or cooling system (e.g., whether the furnace is currently active or the start and stop times of furnace activity). Querying a device using a specialized API may allow the power monitor 150 to determine the status of both the device itself (e.g., a Nest thermostat) and other devices connected to it (e.g., a furnace).

[0070] In some embodiments, the power monitor 150 can subscribe to notifications transmitted by a third-party device, which is referred to herein as a notification device. The notification device can be configured to transmit notifications to other devices (e.g., periodically or when a state changes) and allow other devices to register for notifications using an API to receive notifications. The power monitor 150 can determine that the notification device is in the house and then subscribe to receive notifications from the device. The power monitor 150 can transmit a request to receive notifications from the notification device without knowing that the notification device exists and be subscribed to receive notifications if the notification device exists. Some notification devices may have a pairing process that requires user assistance, and the power monitor 150 can be configured to receive input from the user to assist in the pairing process, such as receiving a username and password to pair with the notification device.

[0071] Other third-party devices may also provide information about the device to which they are connected, such as a smart switch 340 (e.g., Phillips Hue or Belkin Wemo). The smart switch may have an API that allows other devices to interact with it, and the power monitor 150 may use this API to determine the state of the switch and, accordingly, whether the device connected to the smart switch is consuming power (e.g., device on, device off, or a dimmer-type switch at 40%).

[0072] In some embodiments, the power monitor 150 can receive information about devices in the house via a server outside the house (such as a third-party server operated in conjunction with the third-party devices in the house). The power monitor 150 can operate in conjunction with the power monitor server, and the third-party devices in the house can operate in conjunction with the third-party server. For example, a Nest thermostat can send information about the status of the thermostat or the house's heating / cooling system to a server operated by Nest. The third-party server can allow a user to provide configuration information so that the third-party server transmits information to the power monitor server using, for example, an API of the power monitor server or the third-party server. For example, the Nest server can transmit information received from the Nest thermostat in the house to the power monitor server, such as on a periodic basis or when the status of the thermostat changes (or the status of the heating / cooling system changes). In some embodiments, the power monitor server can send a request for information about the third-party device to the third-party server instead of receiving a notification from the third-party server. In some embodiments, the third-party device can communicate directly with the power monitor server, or the power monitor 150 can communicate directly with the third-party server.

[0073] In some embodiments, information received in a network transmission from a device can be used to obtain further information about the device. For example, the network transmission can include a unique identifier, such as a media access control (MAC) address. The unique identifier can be used to determine additional information about the device using a data store or information repository about the device that uses the unique identifier. The data store can provide information about the type, make, and / or version of the device based on the unique identifier. For example, for a MAC address, a block of consecutive MAC addresses can be specific to the type of device (e.g., a television), the type of manufacturer, or the version of the device (e.g., a particular model of television). In some embodiments, a data store of MAC address information can be available, such as through a third-party service, and information about the device can be obtained using the MAC address. A data store of the identifier can be created, purchased, or accessed (e.g., using a third-party server) to obtain information about the device from the identifier.

[0074] In some embodiments, a user device (such as a smartphone) can be used to determine information about devices in the house, and the smartphone can relay this information to the power monitor 150. For example, a speaker 370 (e.g., a portable Bluetooth speaker) may not have a network connection with the network device 310 and may instead use another network (such as a Bluetooth network) to communicate with other devices. The speaker 370 may be too far from the power monitor 150 so that the power monitor may not be able to detect the speaker 370's network. A user device such as a phone 360 ​​can be brought into the same room as the speaker 370, and the phone 360 ​​may be able to use a direct network connection to determine the presence of the speaker 370 and / or the operating status of the speaker 370. The phone 360 ​​can then relay information about the presence and / or operating status of the speaker 370 to the power monitor 150 or a server working in conjunction with the power monitor 150.

[0075] In some embodiments, a network model can be used to identify a device or determine the state of a device by processing network data. The network model can include any appropriate mathematical model, such as a classifier, a neural network, a self-organizing map, a support vector machine, a decision tree, a random forest, a logistic regression, a Bayesian model, linear and nonlinear regression, and a Gaussian mixture model. The network model can receive input data obtained from a network transmission and output an identification of a device or device state change with an optional score. For example, the network model can receive broadcast messages, information about poll responses (or the lack thereof) received from a device, or information about network data generated by the device. In some embodiments, the network model can receive only the header of a network transmission, or can receive all data from the network transmission.

[0076] In some embodiments, network models can be used to identify devices or to determine the state of a device using messages broadcast by the device. For example, a device may transmit a certain number and / or type of messages before shutting down and a different number and / or type of messages when entering sleep mode. When the device is about to shut down, a first network model can be created to describe the expected messages, and when the device is about to enter sleep mode, a second network model can be created to describe the expected messages. When a broadcast message is received from a device, the message can be processed using both network models to generate a score for each network model, and the state transition can be determined by the highest-scoring network model.

[0077] Network models can also be created to describe other aspects of the aforementioned network transmissions. For example, a network model can be created to describe how often a device responds to polling requests and the expected time delay between transmitting a polling request and receiving a response. In another example, a network model can be created to describe the expected network transmissions of a device in a particular state and applied when passively monitoring the device's network transmissions.

[0078] In some embodiments, a rule-based approach can be used to identify devices in a premises or determine the state of a device. The power monitor 150 (or a server operating in conjunction with the power monitor) can have rules created to identify the type of device, the make of the device, or the version of the device; rules to determine when a device's state has changed; and rules to determine other aspects of a device. Rules can be created for any of the techniques described above, such as processing broadcast messages from devices, polling devices, or monitoring network data generated by devices.

[0079] Some rules may output a Boolean value to indicate whether the conditions of the rule are met. For example, a rule may be created to determine whether a device transmitting network data is a television, and if the conditions of the rule are met, the device is determined to be a television. Some rules may output a score, such as on a scale of 1 to 100, to indicate a match between the network data and the rule. For example, a rule may be created to determine whether a device transmitting network data is a television, and the score generated by the rule may be used to make that determination, such as by comparing the score to a threshold or combining the score with other scores as described herein.

[0080] There may be multiple rules for making a determination. For example, there may be multiple rules for determining whether a device transmitting network data is a television, and if any one of the rules is met, then the device may be determined to be a television.

[0081] Any data in or related to a network transmission can be used as input to a rule. For example, information extracted from a network transmission (such as network addresses, identifiers, headers, strings) can be used as input to a rule. Information not in the network transmission but related to the network transmission can also be used, such as the time the network transmission was received.

[0082] In some embodiments, a rule may include one or more conditions that need to be met for the rule to be satisfied. For example, a condition may include any comparison of data, such as inequality, equality, greater than, or less than. A rule may employ any combination of conditions, including, but not limited to, combinations using Boolean algebra. Examples of rules include the following: a device broadcasting ZerConf services including AFP, HTTP, and SSH is an Apple MAC OS computer; the structured data returned as part of a NetBIOS status poll request provides the user-visible name of the host; a device with a previously unseen MAC address making a DHCP request is a new device joining the network for the first time; and a known MAC address vendor prefix is ​​used to determine the vendor of a device.

[0083] In some embodiments, device fingerprinting techniques can be used to identify devices on a network. Device fingerprints can be created for known devices such as Mac computers, Windows computers, and Linux computers, and possibly different operating system versions of each. Fingerprints can be created by sending multiple requests to each device using, for example, different protocols and port numbers (e.g., using a program such as Nmap). The responses of each device can be recorded to create a fingerprint for each device. When creating a device fingerprint, any appropriate data can be used, such as the number and type of network protocols broadcast by the device, the broadcast behavior, the number or frequency of different types of broadcasts, or parameters used for network transmissions (e.g., TCP window size).

[0084] For an unknown device in the house, a similar request can be sent to the device, and the response can be used to create a fingerprint of the unknown device. The fingerprint of the unknown device can be compared with the fingerprint of the known device to determine information about the unknown device. For example, if the fingerprint of the unknown device has the closest match to the fingerprint of a MAC laptop, the unknown device can be identified as a MAC laptop. In some embodiments, the Address Resolution Protocol (ARP) can be used to identify devices on the network, and the above-mentioned techniques can then be used to fingerprint each of the identified devices.

[0085] Any of the techniques described above for using network data to identify a device or determine a device state can generate a score that indicates a match between the data and the identified device or state change. For example, applying a rule to a broadcast message can result in determining that the device is a Toshiba TV with a score of 80% or a Sony TV with a score of 60%. The score generated by the network model can be combined with the score generated by the power model, as described in more detail below.

[0086] Combined power and network monitoring

[0087] Figure 5A system 500 is illustrated for providing users with information about devices in their home using one or both of power monitoring and network monitoring. Figure 5 In the embodiment of the present invention, power monitor 150 can have any of the functionality described above. For example, power monitor 150 can identify the presence of devices in the house, can determine the status of devices in the house, and can determine the power consumption of devices in the house. Power monitor 150 can transmit information about devices in the home to server 510 using any known networking technology. For example, power monitor 150 can have a wireless connection to a router, which in turn is connected to server 510.

[0088] The server 510 can process the information received from the power monitor 150 and present the information to the user, such as through the user device 540. The server 510 can maintain a device list for the devices in the home and update the device list with newly identified devices or updated status of devices. The server 510 can also log changes in device status, record the history of power consumption of the house and individual devices, and perform any other operations described in U.S. Patent 9,443,195. To perform some operations, the server 510 can access other resources, such as a third-party server 530 and a device information data store 520.

[0089] A user can use user device 540 to obtain information about devices in the house. User device 540 can be any device that provides information to the user, including but not limited to a phone, tablet, desktop computer, and wearable device. User device 540 can present information to the user, such as changes in device status and real-time power usage. For example, user device 540 can present a web page to the user, or a special purpose application can be installed on user device 540. The information presented by user device 540 can include any information described in U.S. Patent No. 9,443,195.

[0090] In order to provide information to the user about devices in the home, a list of devices in the home may be maintained. Figure 6 An example device list 600 is illustrated. The list of devices may include any information related to the devices in the home, including, but not limited to, a device ID (which may be specific to the home or all devices known to the company providing the service), a name (e.g., a user-provided name), a type, a make, a version, one or more power models for identifying a change in state of the device, a network ID (e.g., a network address or other identifier such as a MAC address), and one or more network models for identifying a change in state of the device and the state of the device.

[0091] The device list 600 may be stored in one or more locations. For example, the device list 600 may be stored in one or more of the power monitor 150, the server 510, the device information data store 520, or on the user device 540. Different versions of the device list may be stored in different locations corresponding to the processing at that location. For example, the power monitor 150 may not store the name, type, make, or version of the device because this information may not be needed to determine a device state change. The user device 540 may not store information about the power model and network model because the user device 540 may not determine any device state changes.

[0092] Figure 7 The diagram shows a system 700 having a power monitor 150 that processes power monitoring signals and network data to identify devices in the home and identify changes in the state of the devices in the home. As described above, home 110 is supplied with power from electric company 120. Distribution board 140 receives power and distributes it to devices in the home. Some devices receive power from distribution board 140 and are not connected to the network (e.g., refrigerator 160, stove 165, and light bulb 350). Some devices receive power from the distribution board and are connected to the network (e.g., power monitor 150, network device 310, smart TV 320, computer 330, and smart switch 340). Some devices can be connected to the network but not receive power from distribution board 140, or only receive power from the distribution board from time to time (e.g., phone 360 ​​that can be charged using phone charger 710).

[0093] As described above, power monitor 150 may receive one or more power monitoring signals from sensor 130 that measure electrical properties of power provided to premises 110. Power monitor 150 is also connected to a network in premises 110 via network device 310. Power monitor 150 may receive network data from other devices via network device 310 (as indicated by the dashed line), or may receive network data directly from other devices ( Figure 7 The power monitoring and network monitoring system (not shown) receives network data. For some devices in a home, one or both of power monitoring and network monitoring may be inaccurate or have a higher error rate than desired when identifying devices or device state changes. By using both power monitoring and network monitoring in a collaborative or simultaneous manner, the performance of identifying devices or device state changes can be improved. Simultaneous power monitoring and network monitoring can be implemented using any suitable technique, such as a combined score, rule, or voting technique, as described in more detail below.

[0094] In some embodiments, the operation of power monitor 150 may change depending on whether a user is interacting with user device 540 to view information about devices in the home. In situations where a user is viewing information about devices in the home using user device 540, it may be desirable to identify new devices or changes in the state of devices more quickly than when no user is viewing information about devices in the home.

[0095] When a user begins to view information about devices in the home using user device 540, user device 540 can establish a network connection to server 510, and server 510 can transmit information about the devices to user device 540. Server 510 can also have a network connection (directly or indirectly) to power monitor 150. Therefore, server 510 can transmit information to power monitor 150 to change the operation of power monitor 150. Any operation of power monitor 150 can be changed to improve the user experience of the end user. In some embodiments, power monitor 150 can change its operation to more quickly identify new devices and state changes of new devices. For example, the power monitor can increase the polling frequency for polling other devices in the house. In the case where no user is viewing information about the device, a polling frequency of 5 minutes can be sufficient to determine updates. When a user begins to view information about the device, the polling frequency can be increased (e.g., 10 seconds) to more quickly determine updates.

[0096] Figure 5 and Figure 7 The arrangement of the components and specific functions of the server 510 only provides one example of how the techniques described herein may be implemented, but other configurations are possible. For example, power monitor 150 may perform some or all of the operations of server 510, and power monitor 150 may provide information directly to user device 540. In another example, power monitor 150 may include some or all of the functionality of user device 540, and a user may interact directly with power monitor 150 to obtain information about device events and power consumption.

[0097] Identify device

[0098] When the power monitor 150 is first installed in a home, a device list can be created for the devices in the home. In some embodiments, an empty device list can be created. In some embodiments, the device list can be initialized with devices based on input from one or more people in the home. For example, a user can specify any type, make, and / or version of devices in the home.

[0099] The power monitor 150 can use one or both of power monitoring and network monitoring to add or update devices in the device list using any of the techniques described herein. Devices can be added to the device list after being identified by the power monitor, and devices on the device list can be further updated as additional information about them is determined. For example, the power monitor 150 may initially determine that a house has a dishwasher, and it may later determine that the house has a Kenmore 1000 dishwasher.

[0100] In some embodiments, the power monitor 150 may add multiple devices to the device list in response to processing a power event in the power monitoring signal. For example, if two power models generate scores above a threshold, the device corresponding to each power model may be added to the device list. In some embodiments, the scores generated by the power models may also be included in the device list.

[0101] The power monitor 150 can also process the network data to identify devices in the house and add them to the device list. For example, the power monitor 150 can use any of the above-mentioned techniques to obtain information about devices connected to the network in the house and then add the devices to the device list of the house.

[0102] In some embodiments, the power monitor 150 can add multiple devices to a device list (optionally with scores) in response to processing a network data transmission. For example, in response to processing a broadcast message, a Toshiba TV can be added to a device list with a first score, and a Sony TV can be added to a device list with a second score.

[0103] The scores of the devices on the device list can be updated over time. For example, processing a first power event can cause the first device and the second device to be added to the device list with corresponding scores. At a later time, the power monitor 150 can process a second power event to generate a second set of scores for the first and second devices. The second set of scores generated using the second power event can be used to update the overall scores of the first and second devices on the device list.

[0104] Similarly, a network data transmission at a later time can be used to update the scores of the devices on the device list. For example, based on processing the first network transmission, the first and second devices can be added to the device list with corresponding scores. At a later time, the power monitor 150 can process the second network transmission and generate second scores for the first and second devices. As above, the second set of scores generated using the second network transmission can be used to update the overall scores of the first and second devices on the device list.

[0105] In some embodiments, devices on the device list may have scores generated using both power monitoring and network monitoring. After processing a power event, the first device may be added to the device list with the corresponding score. Later, the score of the first device may be updated by processing a first network transmission. An overall score for the devices on the device list may be generated by processing any combination of power events and network transmission events.

[0106] Scores can be combined using any suitable technique. In some embodiments, the overall score for a device can be the average of all individual scores generated for that device. In some embodiments, a variance can be determined for each score, and the scores can be normalized (e.g., Z-scored) before combining.

[0107] In some embodiments, the power monitor can use both power monitor monitoring and network monitoring to identify devices. For devices that are connected to power lines and have a network connection, the device can cause a power event in the power monitor signal around the time of the state change and transmit data (network event) over the network. The power monitor 150 can process both power events and network events simultaneously to identify devices.

[0108] Power events and network events can occur in either order. Figure 8A An example is shown of a power event 810 at a first time followed by a network event 820 at a second time. For example, power event 810 may correspond to a television being turned on, and network event 820 may correspond to a broadcast message over the television announcing the availability of a service. Figure 8B An example is shown in which a network event 850 at a first time is followed by a power event 860 at a second time. For example, a user may turn off a television, and the network event 850 may be a broadcast message revoking service, and the power event 860 may correspond to the television turning off.

[0109] When processing a power event, the power monitor 150 may search for network events that are close in time to the power event (e.g., within a time interval, such as within 1 second) and process them simultaneously. For example, the power monitor 150 may process a network event where the difference between the time of the power event and the time of the network event is less than a threshold. Similarly, when processing a network event, the power monitor 150 may search for power events that are close in time to the network event.

[0110] In some embodiments, the power monitor 150 can process the power event and generate a score for the corresponding power model. The power monitor 150 can then look for network events that are close in time to the power event and use information from the network events to adjust the scores generated by the power models. For example, the first two scoring power models may be for a Toshiba TV and a Sony TV, and the scores may be close to each other. The network event may occur shortly after the power event, and the network event may correspond to the Toshiba TV. The power monitor 150 can use the network event to increase the score for identifying the Toshiba TV, and then identify the Toshiba TV and add it to the device list.

[0111] The time difference between the power event and the network event can also be used to identify the device. The device can issue a network event with a consistent delay after causing the power event, or vice versa. For example, after causing the power event, turning on a TV can consistently issue a network event (e.g., a broadcast service) at approximately 3.5 seconds. The timing of the power event and the network event can be a feature that can be used with any of the identification techniques described herein.

[0112] In some embodiments, a device identification model may be used that receives both information about power events and information about network events as inputs and then generates a score indicating a match between the inputs and the device. For example, a device identification model may be created for the type of device, the make of the device, or the version of the device. In some embodiments, the power monitor 150 may calculate a score for each device identification model, select the device identification model with the highest score, and use the highest scoring device identification model to identify the device. The device identification model may be created using any suitable classification technique, such as a neural network, a self-organizing map, a support vector machine, a decision tree, a random forest, logistic regression, a Bayesian model, linear and nonlinear regression, and a Gaussian mixture model.

[0113] In some embodiments, a confidence level may be calculated in addition to the score, and the confidence level may be used to determine whether to add a device to a device list or update information about a device in a device list. For example, if only one score is above a threshold and the score is much higher than all other scores, then the confidence level may be high. If multiple scores are above a threshold, no score is above the threshold, or the highest score is close to the second highest score, then the confidence level may be low. Any appropriate technique may be used to determine the confidence level for a change in the state of the highest scoring device. If the highest scoring model (e.g., a power model, a network model, or a device identification model) has a low confidence level, the device list may not be updated.

[0114] Devices can also be removed from the device list. For example, a user can review the device list and remove devices that are not in the house. Devices can also be automatically removed from the device list. For example, a device can be removed if it has not been recognized for longer than a period of time, if the device's overall score falls below a threshold, or if enough data has been collected to distinguish the device from a replacement device (e.g., enough data has been collected to confidently determine that the TV is a Toshiba TV and not a Sony TV).

[0115] Figure 9 is a flow chart illustrating an example implementation of identifying devices in a building. Figure 9 In the embodiment of the present invention, the order of steps is exemplary, and other orders are possible, not all steps are required, and in some embodiments, some steps can be omitted or other steps can be added. The process of the flowchart can be implemented by any computer or system described in this article, for example.

[0116] At step 910, a power monitoring signal is obtained. For example, the power monitoring signal may be obtained by receiving measurements from a sensor that measures electrical properties of a power line in a building, such as a current or voltage sensor. The power line may provide power to some or all devices in the building. The power monitoring signal may also be obtained by receiving a power monitoring signal from another device. For example, a server may receive the power monitoring signal transmitted by a power monitor.

[0117] At step 920, the power monitoring signal is processed using a plurality of models to generate a score for each model. Each model may have been created to identify information about a device, such as the type of device, the make of the device, or the version of the device. Any suitable model may be used, such as any of the models described herein. The score may be any value indicating a match between the power monitoring signal and the model, such as a likelihood or log likelihood. Processing the power monitoring signal using the model may be performed by extracting features from the power monitoring signal and then processing the features using the model.

[0118] At step 930, first information about a first device in the building is determined using the scores. In some embodiments, the highest-scoring model may be selected, and the first information may be associated with the highest-scoring model. For example, the highest-scoring model may indicate that the first device is a dishwasher, and the first information may include: the device is a dishwasher. In some embodiments, information about network transmissions may be combined with the scores from the power model to determine the first information about the first device, as described herein.

[0119] In step 940, the device list is updated with the first information about the first device. The device list may be stored in any suitable location, such as on the power monitor or on a server operating in conjunction with the power monitor. In the event that the device list does not yet include any information about the first device, a new entry with the first information may be added to the device list. In the event that the first device is already on the device list, the entry for the first device may be updated with the first information. For example, in the event that the first information is that the first device is a Kenmore dishwasher, and the device list has an entry for the first device as a dishwasher, the device list may be updated to indicate that the make of the dishwasher is Kenmore. The power monitor may cause the device list to be updated locally, or cause the device list to be updated by transmitting the first information to the server so that the server updates the device list.

[0120] At step 950, a network data transmission is received from the second device via the network in the building. The network data transmission may be received directly from the second device (e.g., a point-to-point Bluetooth network) or indirectly from the second device (e.g., via a network router). The network data transmission may be in any suitable format, such as a network data packet or an Ethernet data packet. The network data transmission may be received using any of the techniques described herein, such as receiving a broadcast message in a broadcast network data packet, receiving a response to a polling request, or monitoring the network transmission on the network.

[0121] In step 960, second information about the second device is determined using the network data transmission. The second information can be determined using data or information in the network data packet or other information related to the network data transmission (such as the time when the network transmission was received). In some embodiments, multiple network transmissions can be used to determine the second information. The second information about the second device can be determined using any of the techniques described herein, such as processing information related to network data transmissions having a rule or network model. In some embodiments, information in or about the network data transmission can be combined with information obtained from power monitoring (e.g., a score from a power model) to determine the second information.

[0122] In step 970 , similar to step 940 , the device list is updated with second information about the second device.

[0123] At step 980, a request for information about devices in the building is received. For example, a user may use a user device (such as a smartphone or tablet) to view information about devices in the building. Software running on the user device (e.g., a web page viewed in a browser or a special-purpose application) may request information about devices in the building and send the request using, for example, a REST API. The request may be received by a server operating in conjunction with the power monitor.

[0124] At step 990, first information and second information are transmitted to the user device. For example, the first information may be that the first device is a dishwasher, and the second information may be that the second device is a Toshiba TV. The information may be transmitted by a server operating in conjunction with the power monitor, and the information may be presented by software running on the user device.

[0125] In some embodiments, equipment in a building may be identified as described in the following clauses, combinations of any two or more of them, or in combination with other clauses presented herein.

[0126] 1. A system for identifying devices in a building, the system comprising at least one computer, the at least one computer comprising at least one processor and at least one memory, the at least one computer being configured to: obtain a power monitoring signal by measuring electrical properties of a power line in the building, wherein the power line provides power to the devices in the building; process a first power event in the power monitoring signal using a plurality of power models to generate a score for each of the plurality of power models, wherein the first power event corresponds to a state change of the first device; use the score to determine first information about the first device; cause the first device to be updated in a device list using the first information about the first device; receive a first broadcast network data packet, wherein the first broadcast network data packet is sent by a second device; use information in the first broadcast network data packet to determine second information about the second device; cause the second device to be updated in a device list using the second information about the second device; receive a request for information about the devices in the building from a user device; and transmit the first information about the first device and the second information about the second device to the user device.

[0127] 2. The system of clause 1, wherein the first information about the first device comprises a type of the first device, a make of the first device, or a version of the first device.

[0128] 3. The system of claim 1, wherein at least one computer is configured to receive a second broadcast network data packet, wherein the second broadcast network data packet is sent by the first device; and determine first information about the first device using the plurality of scores and information in the second broadcast network data packet.

[0129] 4. The system of clause 3, wherein the plurality of scores are used to determine the type of the first device, and information in the second broadcast network packet is used to determine the make of the first device.

[0130] 5. The system of clause 3, wherein the first power event in the power monitor signal occurs before receiving the second broadcast network data packet.

[0131] 6. The system of clause 3, wherein the at least one computer is configured to select the second broadcast network packet by comparing a time of the first power event and a time of the second broadcast network packet.

[0132] 7. The system of clause 1, wherein the at least one computer comprises a power monitor installed in the building and a server computer having a network connection to the power monitor.

[0133] 8. The system of clause 1, wherein the information in the first broadcast network packet includes an identifier of the second device, and wherein the at least one computer is configured to obtain second information about the second device from the data store using the identifier of the second device.

[0134] 9. The system of claim 1, wherein at least one computer is configured to determine second information about a second device by processing information in a first broadcast network packet using a plurality of rules, wherein each of the plurality of rules includes comparing the information in the network packet to at least one condition.

[0135] 10. A computer-implemented method for identifying devices in a building, the method comprising: obtaining a power monitoring signal by measuring electrical properties of a power line in the building, wherein the power line provides power to the devices in the building; processing a first power event in the power monitoring signal using a plurality of power models to generate a score for each of the plurality of power models, wherein the first power event corresponds to a state change of the first device; determining first information about the first device using the score; causing the first device to be updated in a device list using the first information about the first device; receiving a first broadcast network data packet, wherein the first broadcast network data packet is sent by a second device; determining second information about the second device using information in the first broadcast network data packet; and causing the second device to be updated in a device list using the second information about the second device.

[0136] 11. The computer-implemented method of clause 10, wherein the power monitoring signal is indicative of a current or voltage of the power transmission line.

[0137] 12. The computer-implemented method of claim 10, comprising: receiving a plurality of broadcast network packets, wherein each of the plurality of broadcast network packets is sent by a first device; and determining first information about the first device using the plurality of scores and information in the plurality of broadcast network packets.

[0138] 13. The computer-implemented method of claim 10, wherein determining second information about the second device comprises: processing information in a first broadcast network data packet using a plurality of network models to generate a second score for each of the plurality of network models; and determining second information about the second device using the second scores of the plurality of network models.

[0139] 14. A computer-implemented method of claim 10, wherein determining second information about the second device comprises generating a second score for each of a plurality of recognition models, wherein each recognition model processes a second power event in the power monitoring signal and information in the first broadcast network data packet; and determining second information about the second device using the second scores of the plurality of recognition models.

[0140] 15. The computer-implemented method of clause 10, comprising: receiving a plurality of broadcast network data packets, wherein each of the plurality of broadcast network data packets is sent by a second device; and determining second information about the second device using information in the plurality of broadcast network data packets.

[0141] 16. The computer-implemented method of clause 15, wherein determining the second information comprises using a first time corresponding to the first broadcast network data packet and a second time corresponding to the second broadcast network data packet.

[0142] 17. One or more non-transitory computer-readable media, comprising computer-executable instructions that, when executed, cause at least one processor to perform actions including: obtaining a power monitoring signal by measuring electrical properties of a power line in a building, wherein the power line provides power to equipment in the building; processing a first power event in the power monitoring signal using a plurality of power models to generate a score for each of the plurality of power models, wherein the first power event corresponds to a state change of a first device; determining first information about the first device using the score; updating the first device in a device list using the first information about the first device; receiving a first broadcast network data packet, wherein the first broadcast network data packet is sent by a second device; determining second information about the second device using information in the first broadcast network data packet; and updating the second device in a device list using the second information about the second device.

[0143] 18. The one or more non-transitory computer-readable media of clause 17, wherein determining the first information about the first device comprises selecting a highest scoring power model using a plurality of scores, and wherein the first information corresponds to the highest scoring power model.

[0144] 19. The one or more non-transitory computer-readable media of clause 17, wherein the second information about the second device comprises a type of the second device, a make of the second device, or a version of the second device.

[0145] 20. The one or more non-transitory computer-readable media of clause 17, wherein the at least one computer is configured to determine second information about the second device by processing information in the first broadcast network packet using a plurality of rules.

[0146] The techniques described above use a combination of power monitoring and network monitoring to provide improved identification of devices in a building relative to techniques based solely on power monitoring. With power monitoring alone, some devices may not be identified at all or may be identified but with lower accuracy. The combination of network monitoring and power monitoring allows for the identification of a greater number of devices (such as networked devices without mechanical components) and also allows for the identification of devices with greater accuracy. Using information from both power monitoring and network monitoring allows for the creation of models and / or classifiers with lower error rates than models and / or classifiers that use information from power monitoring alone.

[0147] Identifying device state changes

[0148] In addition to identifying devices in a house, the power monitor 150 can also use one or both of power monitoring and network monitoring to determine the state of the devices in the house. Some devices (e.g., light bulbs) may have only two states: on and off. Other devices may have multiple states. For example, a washing machine may have a wash cycle, a rinse cycle, and a spin cycle. A television may have an on state, an off state, and a sleep state.

[0149] Power monitoring can be used to identify state changes using any of the techniques described herein and in U.S. Patent 9,443,195. For example, power monitor 150 can process the power monitoring signal, select the power model that produces the highest score, determine the device state change based on the model, and update the state of the device in the device list.

[0150] Network monitoring can also be used to identify state changes using any of the techniques described herein. For example, a power monitor can process broadcast messages, poll devices, and monitor device network data to determine the state of the devices based on their network data as described above.

[0151] In some embodiments, the power monitor can use both power monitor monitoring and network monitoring to identify device state changes. For devices connected to both power lines and network connections, the device can cause a power event in the power monitor signal near the time of the state change and transmit data (network event) over the network. The power monitor can process both power events and network events to identify state changes. Power events and network events can occur in either order.

[0152] As described above, when processing a power event, the power monitor can search for network events that are temporally close to the power event and process them simultaneously. For example, the power monitor can process a network event where the difference between the time of the power event and the time of the network event is less than a threshold. Similarly, when processing a network event, the power monitor can search for power events that are temporally close to the network event.

[0153] In some embodiments, a power monitor can process a power event and generate a score for a corresponding power model. The power monitor can then look for network events that are close in time to the power event and use information from the network events to adjust the scores generated by the power models. For example, the top two scoring power models can be a Toshiba TV turned on and a computer turned on, and the scores can be close to each other. The network event can occur shortly after the power event, and the network event can correspond to the Toshiba TV being turned on. The power monitor can use the network event to potentially increase the score for the Toshiba TV being turned on. For example, the score can be increased by a fixed amount, a percentage, or can be combined with the score generated by the network event processing to generate an overall score for a device state change. The power monitor can then identify the state change as corresponding to the Toshiba TV being turned on. The timing of power events and network events can also be used as a feature of any of the techniques described herein for determining a device state change.

[0154] For some devices, the timing between device state changes and network events may not be precisely known. For example, in some embodiments, a device may be determined to be off when it has not transmitted any network data for a period of time. Thus, it may be determined that the device was turned off at some time after its last network transmission, but the time at which it was turned off may be unknown. In some embodiments, the power monitor may process network events to determine a time-varying score. For example, in a case where the device's last network transmission was at a first time, the power monitor may output a higher score for a time period shortly after the first time and a lower score at later times, such as a score with an exponential decay.

[0155] Some devices may have a known rebroadcast window, where the device will rebroadcast the services provided by the device at set time intervals. In the event that a device has issued a broadcast but has not issued another broadcast within the rebroadcast window, it may be determined that the device has shut down.

[0156] In some embodiments, a state change model can be used that receives both information about power events and information about network events as inputs, and then generates a score indicating the match between the inputs and the device state change. For example, a state change model can be created for each state change of a device (e.g., a state change of the type of device, the manufacture of the device, or the version of the device). In some embodiments, the power monitor can calculate a score for each state change model, select the state change model with the highest score, and use the highest scoring state change model to identify the state change. The state change model can be created using any appropriate classification technique, such as a neural network, a self-organizing map, a support vector machine, a decision tree, a random forest, logistic regression, a Bayesian model, linear and nonlinear regression, and a Gaussian mixture model.

[0157] As above, in addition to the score, a confidence level may be calculated and used when determining whether a state change has occurred. In the event that the highest scoring model (e.g., power model, network model, or state change model) has a low confidence level, no state change may be identified.

[0158] When determining a device state change, the power monitor 150 can use a list of known states of the device and the possible transitions between the states of the device. For example, a television may have three states: an on state, an off state, and a sleep state. However, due to how the television operates, not all state transitions are possible. For three states, there are six possible state transitions (on to off, on to sleep, off to on, off to sleep, sleep to on, and sleep to off). From the "off" state, it is only possible to transition to the "on" state (off to sleep is not possible). From the "sleep" state, it is only possible to transition to the "on" state (sleep to off is not possible). Therefore, the power monitor can implement a technique (e.g., a model or rule) for identifying all possible state changes of the device. When determining a device state change, the power monitor can select a state change from the list of possible state changes of the device and can determine the current state from the end state of the state change (e.g., when selecting an off to on state change, it is determined that the device is now on). When determining the state of the device, the power monitor can select a state from the list of possible states of the device. The power monitor can select a state change or state depending on the implementation and / or the device being monitored.

[0159] Any of the techniques described above for identifying a device may also be used to determine a change in state of a device. Similarly, any of the techniques described for determining a change in state of a device may also be used to identify a device.

[0160] Figure 10is a flow chart illustrating an example implementation for determining the state of a device. Figure 10 In the embodiment of the present invention, the order of steps is exemplary, and other orders are possible, not all steps are required, and in some embodiments, some steps can be omitted or other steps can be added. The process of the flowchart can be implemented by any computer or system described in this article, for example.

[0161] At step 1010, information about devices in the building is accessed from a data store. The information can be stored in any suitable manner and in any suitable location or locations. For example, the power monitor 150 can obtain information about devices in the building from a locally stored device list that includes information about techniques for determining device state changes (e.g., information about models or rules). The power monitor can also access information from a device list stored on a server. The stored information can include information about devices in the building that are powered by the power line and / or connected to a network in the building.

[0162] In step 1020 , a power monitoring signal is obtained, similar to step 910 .

[0163] In step 1030, the state of the first device is determined using the power monitoring signal. Any of the techniques described herein can be used to determine the state of the first device, such as using multiple power models to calculate multiple scores and selecting the state or state change corresponding to the highest scoring power model. The state or state change of the first device can be selected from a list of possible states or state changes of the first device. In some embodiments, network monitoring technology may not be used to determine the state (even if the first device has a network connection). In some embodiments, power monitoring technology can be combined with network monitoring technology to determine the state of the first device, for example, by using any of the techniques described herein. In some embodiments, the state of the first device can be determined by first determining a state change and then determining the state of the first device from the state change.

[0164] At step 1040, the status of the first device is updated in the stored information. Any of the techniques described in step 940 for updating the device list may also be used to update the status of the first device in the device list. For example, the power monitor may cause the status to be updated in the locally stored device list, or transmit the information to the server so that the server updates the device list with the status of the first device.

[0165] At step 1050, a network data transmission is received. Any of the techniques described in step 950 may be used to receive the network transmission. In some embodiments, a power monitor server operating in conjunction with the power monitor may receive the network data transmission from another server operating in conjunction with a device in the house (e.g., a Nest server operating in conjunction with a Nest thermostat), the power monitor may receive the network data transmission directly from a server operating in conjunction with a device in the house, or the power monitor server may receive the network transmission directly from a device in the house.

[0166] At step 1060, it is determined that the second device transmitted the network data. Any suitable technique may be used to determine that the second device transmitted the network data, such as using a network address or identifier in the network data to identify the second device as the sender of the network data. In the event that the power monitor receives the network data from the second device via another device (such as a router), the second device may still be considered the sender or transmitter of the network data. In the event that the network data transmission is received via another server, it may be determined that the network data transmission corresponds to the second device rather than being transmitted by the second device.

[0167] In step 1070, the state of the second device is determined using the network data. Any of the techniques described herein may be used to determine the state of the second device using the network data. The state or state change of the second device may be selected from a list of possible states or state changes of the second device. In some embodiments, power monitoring technology may not be used to determine the state (even if the second device receives power from the power lines in the building). In some embodiments, network monitoring technology may be combined with power monitoring technology to determine the state of the second device, for example, by using any of the techniques described herein. In some embodiments, the state of the second device may be determined by first determining a state change and then determining the state of the second device from the state change.

[0168] The status of the second device is updated in the stored information at step 1080. Any of the techniques described in steps 940 or 1040 for updating the device list may also be used to update the status of the second device in the device list.

[0169] At step 1090, a request for information about devices in the building is received. Any of the techniques described in step 980 may be used to request information about devices in the building.

[0170] In step 1095, information about the state of the first device and the state of the second device is transmitted to the user device. For example, information indicating that the first device is turned off and the second device is in a sleep state may be transmitted to the user device.

[0171] In some implementations, the state of a device may be determined as described in the following clauses, combinations of any two or more thereof, or in conjunction with other clauses presented herein.

[0172] 1. A computer-implemented method for determining a state of a device in a building, the method comprising: accessing stored information about the devices in the building, wherein the devices in the building include a first device and a second device, and wherein the information about the devices includes a state of the first device and a state of the second device; obtaining a power monitoring signal by measuring electrical properties of a power line in the building, wherein the power line provides power to the devices in the building; determining the state of the first device by: (1) processing a first power event in the power monitoring signal using a plurality of power models to generate a score for each of the plurality of power models, and (2) using the score to select a state of the first device from a plurality of possible states; updating the state of the first device in the stored information using the selected state; receiving a first broadcast network packet, wherein the first broadcast network is transmitted by a second device; determining using information in the first broadcast network packet that the first broadcast network packet was sent by the second device; selecting the state of the second device from a plurality of possible states of the second device using information in the first broadcast network packet; and updating the state of the second device in the stored information using the selected state.

[0173] 2. The computer-implemented method of clause 1, wherein the first broadcast network packet comprises a broadcast using Simple Service Discovery Protocol, Zero Configuration Networking, or NetBIOS.

[0174] 3. The computer-implemented method of clause 1, the method comprising: transmitting a request for information to the second device; and wherein receiving the first broadcast network data packet comprises receiving a response to the request for information.

[0175] 4. The computer-implemented method of clause 3, wherein transmitting the request for information comprises: polling the second device, requesting information using an API of the second device, or requesting information about a service provided by the second device.

[0176] 5. The computer-implemented method of claim 1, wherein selecting the state of the second device comprises: (1) processing information in the first broadcast network packet using a plurality of network models, and (2) selecting a highest-scoring network model, wherein the highest-scoring network model corresponds to the selected state.

[0177] 6. The computer-implemented method of clause 1, wherein selecting the state of the second device comprises processing information in the first broadcast network packet using a plurality of rules, wherein each rule of the plurality of rules comprises comparing the information in the network packet to at least one condition.

[0178] 7. A computer-implemented method of claim 1, wherein selecting a state of the second device comprises: processing a second power event in the power monitoring signal using a plurality of power models to generate a second score for each of the plurality of power models; processing information about a first broadcast network data packet using a plurality of network models to generate a third score for each of the plurality of network models; and selecting a state using the second score and the third score.

[0179] 8. A computer-implemented method of claim 1, wherein selecting a state of the second device comprises generating a second score for each state change model in a plurality of state change models, wherein each state change model processes a second power event in the power monitoring signal and information in the first broadcast network data packet; and selecting the state using the second score.

[0180] 9. The computer-implemented method of claim 1, comprising: receiving a plurality of broadcast network data packets, wherein each of the plurality of broadcast network data packets is transmitted by a second device; and using information in the plurality of broadcast network data packets to select a state of the second device.

[0181] 10. A system for determining a state of a device in a building, the system comprising at least one computer, the at least one computer comprising at least one processor and at least one memory, the at least one computer being configured to: store information about the devices in the building, wherein the devices in the building include a first device and a second device, and wherein the information about the devices includes a state of the first device and a state of the second device; obtain a power monitoring signal by measuring electrical properties of a power line in the building, wherein the power line provides power to the devices in the building; determine the state of the first device by: (1) processing a first power event power monitoring signal using a plurality of power models to generate a score for each of the plurality of power models, and (2) using the score to select a state of the first device from a plurality of possible states; using the selected state to update the state of the first device in the stored information; receive first network data from a server computer, wherein the server computer transmits the network data based on the information received from the second device; using the first network data to determine that the first network data corresponds to the second device; using the first network data to select a state of the second device from a plurality of possible states of the second device; and using the selected state to update the state of the second device in the stored information.

[0182] 11. The system of clause 10, wherein the at least one computer is configured to cause the status of the first device to be updated in the stored information by transmitting the status of the first device to the server computer.

[0183] 12. The system of clause 10, wherein the power monitoring signal is indicative of a current or voltage of the power transmission line.

[0184] 13. The system of clause 10, wherein the at least one computer comprises a power monitor installed in the building and a server computer having a network connection to the power monitor.

[0185] 14. The system of claim 13, wherein the server computer is configured to determine that the user device has a network connection to the server computer; the server computer is configured to send an indication to the power monitor that the user device has a network connection to the server computer; and the power monitor is configured to modify its processing in response to receiving the indication.

[0186] 15. The system of clause 14, wherein the power monitor is configured to modify its processing by changing the frequency with which the device is polled.

[0187] 16. The system of claim 10, wherein at least one computer is configured to select a state of the second device by processing a second power event in the power monitoring signal using a plurality of power models to generate a second score for each power model in the plurality of power models; and to select the state using the second score and information in the first network data.

[0188] 17. The system of clause 10, wherein the at least one computer is configured to obtain information about the power consumption of the second device from the data store of device information using the status of the second device; and transmit the information about the power consumption of the second device to the user device.

[0189] 18. The system of clause 10, wherein the at least one computer is configured to transmit the state of the first device and the state of the second device to the user device.

[0190] 19. One or more non-transitory computer-readable media comprising computer-executable instructions that, when executed, cause at least one processor to perform actions including: obtaining a power monitoring signal by measuring electrical properties of a power line in a building, wherein the power line provides power to equipment in the building; determining a state of a first device by: (1) processing a first power event in the power monitoring signal using a plurality of power models to generate a score for each of the plurality of power models, and (2) using the score to select a state of the first device from a plurality of possible states; using the selected state to update the state of the first device in a device list; receiving a first broadcast network packet, wherein the first broadcast network packet is transmitted by a second device; using information in the first broadcast network packet to determine that the first broadcast network packet is transmitted by the second device; using information in the first network event to select a state of the second device from a plurality of possible states of the second device; and using the selected state to update the state of the second device in the device list.

[0191] 20. The one or more non-transitory computer-readable media of clause 19, wherein causing the state of the first device to be updated in the stored information comprises changing the state from an off state to an on state.

[0192] 21. The one or more non-transitory computer-readable media of clause 19, wherein the plurality of power models comprises one or more of a device model, a transition model, or a wattage model.

[0193] 22. The one or more non-transitory computer-readable media of clause 19, wherein determining that the first broadcast network packet was transmitted by a second device comprises using a network address or identifier in the first broadcast network packet.

[0194] 23. The one or more non-transitory computer-readable media of clause 19, wherein processing a first power event in a power monitoring signal using multiple power models comprises: calculating multiple features using a portion of the power monitoring signal including the first power event; and processing the multiple features using the multiple power models.

[0195] The techniques described above use a combination of power monitoring and network monitoring to provide improved identification of device state changes in a building relative to techniques based on power monitoring alone. With power monitoring alone, some device state changes may not be identified at all or may be identified, but with lower accuracy. The combination of network monitoring and power monitoring allows a greater number of device state changes to be identified (such as changes in the state of networked devices without mechanical components), and device state changes may also be identified with higher accuracy. Using information from both power monitoring and network monitoring allows the creation of models and / or classifiers with lower error rates than models and / or classifiers that use information from power monitoring alone.

[0196] Reporting power usage

[0197] In some embodiments, the power monitor can provide information to the user about the power consumption of individual devices in the house. For example, for any device that uses power monitoring and / or network monitoring to determine changes in device status, the techniques described in U.S. Patent 9,443,195 can be used to determine the power consumption of the device over time and / or present real-time information about the power consumption of the device to the user.

[0198] Some devices in a house may have relatively constant power usage or power usage that falls within a range for each state of the device. For example, a television and a light bulb may consume a relatively fixed amount of power when they are on and consume no power when they are off. For such devices, the expected power consumption or power range can be determined by measuring the power usage of several instances of the device in each state and calculating the average power consumption for that state. A list of the expected power usage or power range of the device can be created for each state and made available to the power monitor.

[0199] When a device with relatively constant power usage is determined to be in a particular state (e.g., by processing network events), the power monitor (or a server operating in conjunction with the power monitor) can obtain the expected power usage or power range for the device state (e.g., from its own storage or a third-party server) and report to the user that the device is consuming power within the expected amount or range. When the device is turned off, the power monitor can report to the user that the device has stopped consuming power.

[0200] Some devices in a house may be "always on," for example, cable modems, wireless routers, smart thermostats, and IoT devices. For always-on devices, expected power usage or a power range can be determined as described above. After an always-on device is identified as being in the house, the power monitor can report to the user that the device is always on and provide the expected power usage or a power range.

[0201] The power monitor (or a server operating in conjunction with the power monitor) can also calculate the total amount of energy consumed by devices that have relatively constant power usage or devices that are always on. For these devices, the total amount of energy consumed by the device can be calculated by (1) determining the amount of time the device has been in each of its possible states, (2) determining the expected power usage in each state, (3) determining the total amount of energy consumed in each state by multiplying the amount of time in the state by the expected power consumption of the state, and (4) determining the total amount of energy consumed by the device by adding the energy consumption of each state.

[0202] Using network events to train power models

[0203] Training power models for some devices may be more challenging than for others. Devices with mechanical components, such as pumps and motors, may have a more easily identifiable power signature due to the relationship between the operation of the mechanical components and power usage. Devices with few or no mechanical components may have a more difficult to identify power signature. For example, many electronic devices, such as computers and televisions, can convert AC power to DC power and then use the DC power to power their internal electronics.

[0204] The power model can be trained using a corpus of training data as described in U.S. Patent 9,443,195. For example, tens, hundreds, or thousands of examples of power events in a power signal corresponding to a television being turned on can be used to train a power model for television on. Because the power events corresponding to television on can be similar to the power events for other electronic devices turning on, collecting sufficient training data for all electronic devices in a house can be challenging.

[0205] Network events can be used to help collect training data for training power models for devices connected to both the power line and the network. Assume that it is desired to train a power model for a first device (e.g., a television) that is turned on. An instance of the first device being turned on can be identified by processing the network event using any of the techniques described above. Next, a portion of the power monitoring signal corresponding to the first device being turned on is stored so that it can be used later for training purposes. The portion of the power monitoring signal can include portions before and / or after the network event. By performing these steps over an extended period of time and / or for multiple households, a training corpus of training data can be collected for training a power model for the first device being turned on.

[0206] Any portion of the power monitoring signal may include multiple power events that are close to or even overlap each other. Therefore, in addition to the power event for the first device to turn on, the collected training data may also include power events for other device state changes. Power events for other devices may be referred to as extraneous power events.

[0207] Different methods can be used to handle exogenous power events in the training data. In some embodiments, no special action may be taken, and all data may be used as is to train the power model. Although the training data will include exogenous power events, if their number is small compared to the power events corresponding to the first device being turned on, they may not significantly affect model training. In some embodiments, training data with more than one power event may be discarded so that only the portion of the power monitoring signal containing a single power event is used to train the power model for the first device. In some embodiments, exogenous power events can be identified so that only power events that are likely to correspond to the first device being turned on are used to train the power model. For example, features can be calculated for all power events in the training data, and clustering techniques (e.g., k-means) can be used to identify exogenous power events. In the event that a portion of the power monitoring signal contains a power event for the first device being turned on at a first time and an exogenous power event at a second time, only the portion of the power monitoring signal that includes the power event at the first time can be used to train the power event for the first device being turned on.

[0208] The training data can then be used to train a power model for the first device to be turned on. The power model can be trained using any suitable technique, such as any of the techniques described in U.S. Patent 9,443,195. For example, the power model can include any of a transition model, a device model, or a wattage model. Training techniques can include, but are not limited to, training neural networks, self-organizing maps, support vector machines, decision trees, random forests, logistic regression, Bayesian models, linear and nonlinear regression, and Gaussian mixture models.

[0209] The power model may also be trained similarly for other state changes of the first device and state changes of other devices.

[0210] Figure 11 is a flow chart illustrating an example implementation of using network data to train a power model for a device. Figure 11 In the embodiment of the present invention, the order of steps is exemplary, and other orders are possible, not all steps are required, and in some embodiments, some steps can be omitted or other steps can be added. The process of the flowchart can be implemented by any computer or system described in this article, for example.

[0211] At step 1110, information about a first network event in a building is received. The network event may correspond to any data transmission, such as a network data packet transmitted from a device in the building to a power monitor in the building. In some embodiments, the server may receive information about the network event from a power monitor, where the power monitor receives a network data transmission from another device in the building. The information about the network event may include information in or about the transmitted network data, such as a time of receipt. The network event may be associated with a time, such as the time the network data was received by the power monitor.

[0212] In step 1120, the information about the network event is used to determine that the first device has changed state. Any of the techniques described herein may be used to determine that the first device has changed state using the network event. The determination may be performed by the power monitor or by a server operating in conjunction with the power monitor. In some embodiments, other steps in the process may depend on determining that the first device has changed state. For example, the power monitor may process the information about the network event to determine that the first device has changed state and transmit information about the network event and / or the power monitoring signal only after determining that the first device has changed state (in step 1130).

[0213] At step 1130, a power monitoring signal is received, wherein the received power monitoring signal is a portion of a power monitoring signal used by a power monitor. For example, a server may receive the power monitoring signal from a power monitor. The received power monitoring signal may include measurements taken at a time corresponding to a network event (e.g., a time of transmission or reception).

[0214] Steps 1110, 1120, and 1130 can be repeated any number of times, and the corresponding network events and power monitoring signals can originate from the same building or from multiple buildings. The collected power monitoring signals can correspond to devices that share a common characteristic. For example, the power monitoring signals can all originate from televisions, can all originate from Toshiba televisions, or can originate from a specific version of a Toshiba television. Because the power monitoring signals share a common characteristic, they can be used to train a power model to detect that common characteristic. In step 1140, a determination can be made as to whether additional data is available or desired for training the power model. If more data is available or desired, processing can proceed to step 1110. If more data is not available or desired, processing can proceed to step 1150.

[0215] At step 1150, a power model is trained using the power monitoring signals received at one or more instances of step 1130. The power model can be any power model described herein or in U.S. Patent 9,443,195 and can be trained using any of the techniques described herein or in U.S. Patent 9,443,195. In some embodiments, the power monitoring signals can be processed to identify and remove or ignore extraneous power events corresponding to devices other than the first device, as described herein.

[0216] The trained power model may be deployed to power monitors in the building at step 1160. For example, the power model may be used by the power monitors to identify devices in the building, or to identify device state changes in the building using any of the techniques described herein.

[0217] In some embodiments, the power model may be trained as described in the following clauses, combinations of any two or more thereof, or in combination with other clauses presented herein.

[0218] 1. A system for training a power model for identifying a device or a change in a state of a device, the system comprising at least one computer, the at least one computer comprising at least one processor and at least one memory, the at least one computer being configured to: receive information about a first network event, wherein the first network event corresponds to a first device in a first building transmitting data at a first time; determine using the information about the first network event that the first device changed state; obtain a first power monitoring signal, wherein the first power monitoring signal corresponds to a measurement of an electrical property of a power line in the first building, and wherein the first power monitoring signal includes the measurement at the first time; receive information about a second network event, wherein the second network event corresponds to a second device in a second building transmitting data at a second time; determine using the information about the second network event that the second device changed state; obtain a second power monitoring signal, wherein the second power monitoring signal corresponds to a measurement of an electrical property of a power line in the second building, and wherein the second power monitoring signal includes the measurement at the second time; train a power model using the first power monitoring signal and the second power monitoring signal, wherein the power model is configured to use the power monitoring signals to identify a device or a change in state of a device; and transmit the power model to a power monitor in a third building.

[0219] 2. The system of clause 1, wherein the first device and the second device are devices of the same type, have the same manufacturer, or have the same version.

[0220] 3. The system of clause 1, wherein the at least one computer comprises a first power monitor in the first building, a second power monitor in the second building, and a server computer.

[0221] 4. The system of claim 1 , wherein at least one computer is configured to obtain a third power monitoring signal, wherein the third power monitoring signal corresponds to a measurement of an electrical property of a power transmission line in a third building; and process the third power monitoring signal using a power model to determine that the third device changes state.

[0222] 5. The system of clause 1, wherein the at least one computer is configured to transmit a request for information to the second device; and wherein the network data is received in response to the request for information.

[0223] 6. The system of clause 1, wherein the at least one computer is configured to receive information about a plurality of network events, wherein each network event corresponds to the first device transmitting data; and use the information about the plurality of network events to determine that the first device changes state.

[0224] 7. The system of clause 1, wherein the at least one computer is configured to determine that the first device changed state by (1) processing information about the first network event using a plurality of network models, and (2) selecting a highest scoring network model.

[0225] 8. A method for training a power model for identifying a device or a change in a state of a device, the method comprising: receiving information about a first network event, wherein the first network event corresponds to a first device transmitting data at a first time; obtaining a first power monitoring signal, wherein the first power monitoring signal corresponds to a measurement result of an electrical property of a transmission line, and wherein the first power monitoring signal includes the measurement result at the first time; receiving information about a second network event, wherein the second network event corresponds to the first device transmitting data at a second time; obtaining a second power monitoring signal, wherein the second power monitoring signal corresponds to a measurement result of an electrical property of the transmission line, and wherein the second power monitoring signal includes the measurement result at the second time; training a power model using the first power monitoring signal and the second power monitoring signal, wherein the power model is configured to use the power monitoring signal to identify the first device or a change in the state of the first device.

[0226] 9. The method of clause 8, comprising: using information about the first network event to determine that the first device changed state; and using information about the second network event to determine that the second device changed state.

[0227] 10. The method of clause 8, comprising transmitting the power model to a power monitor in the building.

[0228] 11. The method of clause 8, wherein the first network event comprises a broadcast network data packet.

[0229] 12. The method of clause 8, wherein the information about the first network event includes a time of the first network event.

[0230] 13. The method of clause 8, comprising: determining that the first power monitor signal includes a first power event and a second power event; determining that the first power event corresponds to the first device changing state; and training the power model using the first power event.

[0231] 14. The method of clause 13, wherein determining that the first power event corresponds to the first device comprises: calculating a first characteristic of the first power event and a second characteristic of the second power event; and processing the first characteristic and the second characteristic using a mathematical model, or clustering the first characteristic and the second characteristic.

[0232] 15. The method of clause 13, wherein the first device is in a first building, and the method comprises: receiving information about a third network event, wherein the third network event corresponds to a second device in a second building transmitting data at a third time; obtaining a third power monitoring signal, wherein the third power monitoring signal corresponds to a measurement result of an electrical property of a power line in the second building, and wherein the second power monitoring signal includes the measurement result at the third time; and training a power model using the third power monitoring signal.

[0233] 16. One or more non-transitory computer-readable media, comprising computer-executable instructions that, when executed, cause at least one processor to perform actions including: receiving information about a first network event, wherein the first network event corresponds to a first device transmitting data at a first time; obtaining a first power monitoring signal, wherein the first power monitoring signal corresponds to a measurement result of an electrical property of a transmission line, and wherein the first power monitoring signal includes the measurement result at the first time; receiving information about a second network event, wherein the second network event corresponds to the first device transmitting data at a second time; obtaining a second power monitoring signal, wherein the second power monitoring signal corresponds to a measurement result of an electrical property of the transmission line, and wherein the second power monitoring signal includes the measurement result at the second time; and training a power model using the first power monitoring signal and the second power monitoring signal, wherein the power model is configured to use the power monitoring signal to identify the first device or a change in state of the first device.

[0234] 17. The one or more non-transitory computer-readable media of clause 16, wherein the power monitoring signal indicates a current or voltage of the power line.

[0235] 18. The one or more non-transitory computer-readable media of clause 16, wherein the power model comprises a transition model, a device model, or a wattage model.

[0236] 19. The one or more non-transitory computer-readable media of clause 16, wherein the first network event comprises a broadcast network data packet.

[0237] 20. The one or more non-transitory computer-readable media of clause 16, comprising transmitting the power model to a power monitor in a building.

[0238] The above techniques allow for the creation of power models for devices where it may otherwise be difficult to obtain sufficient training data to train the power model. For devices with few or no mechanical components (e.g., televisions and computers), there may not be a large amount of variability in the power signatures of the devices. To obtain an example power signature for such a device, a person can manually change the state of the device and save the power monitoring signal in a window around the manual state change, but this process does not scale to obtain a large amount of training data or to obtain data for a large number of devices. By applying the techniques described herein, training data for a device can be obtained through network monitoring. Because this process may not require human action, it can therefore scale to obtain a large amount of training data for a large number of devices and / or training data for a large number of devices. By using network monitoring to obtain training data, power models can be created more quickly (because training data can be obtained more quickly) and the power model can be more accurate (because more training data can be available).

[0239] Implementation Method

[0240] Figure 12 Components of some embodiments of a computing device 1200 that may be used with any power monitor or server operating in conjunction with a power monitor are illustrated. Figure 12 In the , components are shown as being on a single computing device, but the components may be distributed among multiple computing devices, such as among any of the devices described above or among several server computing devices.

[0241] The computing device 1200 may include any typical components of a computing device, such as one or more processors 1211, volatile or non-volatile memory 1210, and one or more network interfaces 1212 for connecting to a computer network. The computing device 1200 may also include any input and output components, such as a display, a keyboard, and a touch screen. The computing device 1200 may also include various components or modules that provide specific functionality, and these components or modules may be implemented in software, hardware, or a combination thereof. Hereinafter, several examples of components are described for an example embodiment, and other embodiments may include additional components or exclude some of the components described below.

[0242] The computing device 1200 may include a power event processing component 1220 that can be used to process power monitoring signals and determine information about devices from the power monitoring signals, such as identifying devices and device state changes. The computing device 1200 may include a network event processing component 1221 that can be used to process data from a computer network and determine information about devices from the network data, such as identifying devices and device state changes. The computing device 1200 may include a device identification component that can be used to identify information about a device (such as the type of device or the manufacturer of the device) by using information obtained from power monitoring and / or network monitoring. The computing device 1200 may include a device state change component that can be used to identify a state change of a device by using information obtained from power monitoring and / or network monitoring. The computing device 1200 may have a device update component 1224 that can be used to update a device list after a device has been identified or a state change of a device has been identified.

[0243] Computing device 1200 may include or have access to various data stores, such as data stores 1230, 1231, and 1232. The data stores may use any known storage technology, such as files or relational or non-relational databases. For example, computing device 1200 may have a power model data store 1230 to store power models or information about power models. Computing device 1200 may have a network model data store 1231 to store network models or information about network models. Computing device 1200 may have a device information data store 1232, which may be used to store information about devices, such as a list of devices for one or more buildings.

[0244] Although only a few embodiments of the present invention are shown and described, it will be apparent to those skilled in the art that many changes and modifications may be made thereto without departing from the spirit and scope of the present disclosure as described in the appended claims. All patent applications and patents (both foreign and domestic) and all other publications cited herein are hereby incorporated in their entirety to the full extent permitted by law.

[0245] The methods and systems described herein may be deployed, in part or in whole, by a machine that executes computer software, program code, and / or instructions on a processor. The processor may be part of a server, cloud server, client, network infrastructure, mobile computing platform, fixed computing platform, or other computing platform. The processor may be any type of computing or processing device capable of executing program instructions, codes, binary instructions, and the like. The processor may be or may include a signal processor, a digital processor, an embedded processor, a microprocessor, or any variant thereof, such as a coprocessor (mathematical coprocessor, graphics coprocessor, communication coprocessor, etc.), which may directly or indirectly facilitate the execution of program code or program instructions stored thereon. In addition, the processor may implement the execution of multiple programs, threads, and codes. Threads may be executed simultaneously to enhance the performance of the processor and to facilitate the simultaneous operation of applications. By way of embodiment, the methods, program codes, program instructions, and the like described herein may be implemented in one or more threads. A thread may generate other threads that may have assigned priorities associated therewith; the processor may execute these threads based on priority or in any other order based on the instructions provided in the program code. The processor may include a memory for storing methods, codes, instructions, and programs as described herein and elsewhere. The processor can access a storage medium through an interface that can store methods, codes, and instructions as described herein and elsewhere. The storage medium associated with the processor for storing methods, programs, codes, program instructions, or other types of instructions that can be executed by a computing or processing device can include, but is not limited to, one or more of a CD-ROM, a DVD, a memory, a hard disk, a flash drive, RAM, ROM, a cache, and the like.

[0246] The processor may include one or more cores that can enhance the speed and performance of the multiprocessor. In an embodiment, the process may be a dual-core processor, a quad-core processor, other chip-level multiprocessors, etc., that combines two or more independent cores (referred to as dies).

[0247] Method and system described herein can be deployed in part or in its entirety by executing a machine of computer software on a server, cloud server, client, firewall, gateway, hub, router or other such computer and / or networking hardware. A software program can be associated with a server, which can include a file server, a print server, a domain server, an Internet server, an intranet server and other variations such as auxiliary servers, host servers, distributed servers, etc. The server can include one or more of a memory, a processor, a computer-readable medium, a storage medium, a port (physical and virtual), a communication device and an interface that can access other servers, clients, machines and equipment by wired or wireless media, etc. Method, program or code as described herein and elsewhere can be performed by a server. In addition, other equipment required for the execution method described in this application can be considered as a part of the infrastructure associated with the server.

[0248] Server can provide interface to other equipment, and described equipment includes but not limited to client, other server, printer, database server, print server, file server, communication server, distributed server etc.Additionally, this coupling and / or connection can promote the remote execution of the program across network.Without departing from the scope of the present disclosure, some or all of networking in these equipment can promote the parallel processing of the program or method at one or more locations.In addition, any equipment that is attached to server by interface can comprise at least one storage medium that can store method, program, code and / or instruction.Central repository can provide the program instruction that will be carried out on different equipment.In this embodiment, remote storage can serve as the storage medium that is used for program code, instruction and program.

[0249] The software program may be associated with a client, which may include a file client, a print client, a domain client, an Internet client, an intranet client, and other variations such as an auxiliary client, a host client, a distributed client, and the like. The client may include one or more of a memory, a processor, a computer-readable medium, a storage medium, a port (physical and virtual), a communication device, and an interface capable of accessing other clients, servers, machines, and devices via wired or wireless media. Methods, programs, or codes as described herein and elsewhere may be executed by the client. Additionally, other devices required to perform the methods as described in this application may be considered part of the infrastructure associated with the client.

[0250] The client can provide an interface to other equipment, and the equipment includes but is not limited to server, other clients, printer, database server, print server, file server, communication server, distributed server etc. Additionally, this coupling and / or connection can promote the remote execution of the program across the network. Without departing from the scope of the present disclosure, some or all of the networking in these equipment can promote the parallel processing of the program or method at one or more locations. In addition, any equipment attached to the client by the interface can comprise at least one storage medium that can store method, program, application, code and / or instruction. The central repository can provide the program instruction to be performed on different devices. In this embodiment, the remote storage repository can serve as the storage medium for program code, instruction and program.

[0251] The methods and systems described herein can be deployed in part or in whole through a network infrastructure. The network infrastructure can include elements such as computing devices, servers, routers, hubs, firewalls, clients, personal computers, communication devices, routing devices, and other active and passive devices, modules, and / or components known in the art. The computing and / or non-computing devices associated with the network infrastructure can include, in addition to other components, storage media such as flash memory, buffers, stacks, RAM, ROM, etc. The processes, methods, program codes, instructions described herein and elsewhere can be executed by one or more of the network infrastructure elements.

[0252] The methods, program codes, and instructions described herein and elsewhere can be implemented on a cellular network having multiple cells. The cellular network can be a frequency division multiple access (FDMA) network or a code division multiple access (CDMA) network. The cellular network can include mobile devices, cell sites, base stations, repeaters, antennas, towers, etc. The cell network can be GSM, GPRS, 3G, EVDO, mesh, or other network types.

[0253] The methods, program codes, and instructions described herein and elsewhere can be implemented on or through a mobile device. Mobile devices may include navigation devices, cellular phones, mobile phones, mobile personal digital assistants, laptop computers, palmtop computers, netbooks, pagers, e-book readers, music players, and the like. These devices may include, among other components, storage media such as flash memory, buffers, RAM, ROM, and one or more computing devices. A computing device associated with the mobile device may be enabled to execute the program codes, methods, and instructions stored thereon. Alternatively, the mobile device may be configured to execute instructions in collaboration with other devices. The mobile device may communicate with a base station that is coupled to a server and configured to execute the program code. The mobile device may communicate over a peer-to-peer network, a mesh network, or other communication network. The program code may be stored on a storage medium associated with the server and executed by a computing device embedded within the server. The base station may include a computing device and a storage medium. The storage device may store the program codes and instructions executed by the computing device associated with the base station.

[0254] Computer software, program code and / or instructions may be stored and / or accessed on machine-readable media, which may include: computer components, devices and recording media that retain digital data used to calculate a certain time interval; semiconductor storage known as random access memory (RAM); mass storage commonly used for more permanent storage, such as optical disks, forms such as hard disks, tapes, drums, cards and other types of magnetic storage; processor registers, cache memory, volatile memory, non-volatile memory; optical storage such as CDs, DVDs; removable media such as flash memory (e.g., USB sticks or keys), floppy disks, magnetic tape, paper tape, punch cards, stand-alone RAM disks, Zip drives, removable mass storage, offline, etc.; other computer memory such as dynamic memory, static memory, read / write storage, variable storage, read-only, random access, sequential access, location addressable, file addressable, content addressable, network attached storage, storage area networks, bar codes, magnetic ink, etc.

[0255] The methods and systems described herein can transform physical and / or intangible items from one state to another. The methods and systems described herein can also transform data representing physical and / or intangible items from one state to another, such as from usage data to a normalized usage data set.

[0256] The elements described and depicted herein (included in the flowcharts and block diagrams throughout the accompanying drawings) imply the logical boundaries between elements. However, according to software or hardware engineering practices, the depicted elements and their functions can be implemented on a machine by a computer-executable medium, and the computer-executable medium has a processor capable of executing program instructions stored thereon as a single-chip software structure, as an independent software module, or as a module using external routines, codes, services, etc. or any combination of these, and all such embodiments can be within the scope of the present disclosure. Examples of such machines may include, but are not limited to, personal digital assistants, laptop computers, personal computers, mobile phones, other handheld computing devices, medical equipment, wired or wireless communication equipment, transducers, chips, calculators, satellites, tablet PCs, e-books, gadgets, electronic devices, devices with artificial intelligence, computing devices, network equipment, servers, routers, etc. In addition, the elements or any other logical components depicted in the flowcharts and block diagrams can be implemented on a machine capable of executing program instructions. Therefore, although the aforementioned figures and descriptions set forth the functional aspects of the disclosed system, the specific arrangement of the software for implementing these functional aspects should not be inferred from these descriptions unless explicitly stated or otherwise clear according to the context. Similarly, it will be appreciated that the various steps identified and described above may vary, and the order of the steps may be adapted to the particular application of the technology disclosed herein. All such variations and modifications are intended to fall within the scope of this disclosure. Therefore, the depiction and / or description of the order of various steps should not be construed as requiring a particular order of execution of those steps, unless required by a particular application, or explicitly stated or otherwise clear from the context.

[0257] The methods and / or processes described above and their steps may be implemented in hardware, software, or any combination of hardware and software suitable for a particular application. The hardware may include a general-purpose computer and / or a special-purpose computing device or a specific computing device or special aspects or components of a specific computing device. The processes may be implemented in one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors, or other programmable devices together with internal and / or external memory. The processes may also or alternatively be embodied in an application-specific integrated circuit, a programmable gate array, programmable array logic, or any other device or combination of devices that can be configured to process electronic signals. It should also be appreciated that one or more processes may be implemented as computer executable code that can be executed on a machine-readable medium.

[0258] Computer executable code may be created using a structured programming language such as C, an object-oriented programming language such as C++, or any other high-level or low-level programming language (including assembly language, hardware description languages, and database programming languages ​​and techniques) that can be stored, compiled, or interpreted to run on one of the aforementioned devices and processors, a heterogeneous combination of processor architectures, or a combination of different hardware and software, or any other machine capable of executing program instructions.

[0259] Thus, in one aspect, each of the methods described above and combinations thereof may be embodied in computer executable code that, when executed on one or more computing devices, performs its steps. In another aspect, the method may be embodied in a system that performs its steps and may be distributed across devices in a variety of ways, or all functionality may be integrated into a dedicated, stand-alone device or other hardware. In another aspect, the apparatus for performing the steps associated with the processes described above may include any of the hardware and / or software described above. All such permutations and combinations are intended to fall within the scope of this disclosure.

[0260] All documents cited herein are hereby incorporated by reference.

Claims

1. A system for determining a status of a device in a building, the system comprising: At least one computer comprising at least one processor and at least one memory, the at least one computer being configured to: obtaining a power monitoring signal by measuring an electrical property of a power line in the building, wherein the power line provides power to the devices in the building, the devices in the building comprising a first device and a second device; determining a state of the first device by: (1) processing a first power event in the power monitoring signal using a plurality of power models to generate a score for each of the plurality of power models, and (2) selecting a state of the first device from a plurality of possible states using the scores; receiving a first network data packet from a server computer, wherein the server computer transmits the first network data packet based on information received from the second device; using the first network data packet to determine that the first network data packet corresponds to the second device; as well as The first network data packet is used to select a state of the second device from a plurality of possible states of the second device.

2. The system of claim 1, wherein the power monitoring signal indicates a current or a voltage of the transmission line. The system according to claim 1 , wherein the first network data packet is a broadcast network data packet.

4. The system of claim 1, wherein the first network data packet comprises a broadcast using Simple Service Discovery Protocol, Zero Configuration Networking, or NetBIOS.

5. The system of claim 1, wherein the first network data packet is received in response to a request for information transmitted to the second device.

6. The system of claim 1, wherein the first network data packet is received in response to polling the second device, requesting information using an API of the second device, or requesting information about a service provided by the second device.

7. The system of claim 1 , wherein the state of the second device is selected by: (1) processing information in the first network data packet using a plurality of network models, and (2) selecting a highest scoring network model, wherein the highest scoring network model corresponds to the selected state.

8. The system of claim 1, wherein the state of the second device is selected by processing information in the first network packet using a plurality of rules, wherein each rule of the plurality of rules includes comparing the information in the first network packet to at least one condition.

9. The system of claim 1 , wherein the state of the second device is selected by: processing a second power event in the power monitoring signal using a plurality of power models to generate a second score for each power model in the plurality of power models; processing information about the first network data packet using a plurality of network models to generate a third score for each of the plurality of network models; as well as The state is selected using the second score and the third score.

10. A system according to claim 1, wherein the state of the second device is selected by generating a second score for each state change model in a plurality of state change models, wherein each state change model processes a second power event in the power monitoring signal and information in the first network data packet; and using the second score to select the state.

11. The system of claim 1 , wherein the state of the second device is selected by: receiving a plurality of network packets, wherein each of the plurality of network packets is transmitted by the second device; and The state of the second device is selected using information in the plurality of network packets.

12. A computer-implemented method for determining a status of a device in a building, the method comprising: obtaining a power monitoring signal by measuring an electrical property of a power line in the building, wherein the power line provides power to the devices in the building, the devices in the building comprising a first device and a second device; determining a state of the first device by: (1) processing a first power event in the power monitoring signal using a plurality of power models to generate a score for each of the plurality of power models, and (2) selecting a state of the first device from a plurality of possible states using the scores; receiving a first network data packet, wherein the first network data packet is transmitted by the second device; using information in the first network data packet to determine that the first network data packet is sent by the second device; as well as The information in the first network packet is used to select a state of the second device from a plurality of possible states of the second device.

13. The computer-implemented method of claim 12, wherein the first network data packet comprises a broadcast using Simple Service Discovery Protocol, Zero Configuration Networking, or NetBIOS.

14. The computer-implemented method of claim 12, the method comprising: transmitting a request for information to the second device; Wherein receiving the first network data packet includes receiving a response to the request for information.

15. The computer-implemented method of claim 14, wherein transmitting the request for information comprises polling the second device, requesting information using an API of the second device, or requesting information about a service provided by the second device.

16. The computer-implemented method of claim 12, wherein selecting a state of the second device comprises: (1) processing information in the first network data packet using a plurality of network models, and (2) selecting a highest-scoring network model, wherein the highest-scoring network model corresponds to the selected state.

17. The computer-implemented method of claim 12, wherein selecting the state of the second device comprises processing information in the first network packet using a plurality of rules, wherein each rule of the plurality of rules comprises comparing the information in the first network packet to at least one condition.

18. The computer-implemented method of claim 12, wherein selecting a state of the second device comprises: processing a second power event in the power monitoring signal using a plurality of power models to generate a second score for each power model in the plurality of power models; as well as processing information about the first network data packet using a plurality of network models to generate a third score for each of the plurality of network models; as well as The state is selected using the second score and the third score.

19. The computer-implemented method of claim 12, wherein selecting a state of the second device comprises: generating a second score for each state change model of a plurality of state change models, wherein each state change model processes a second power event in the power monitoring signal and information in the first network data packet; as well as The state is selected using the second score.

20. The computer-implemented method of claim 12, comprising: receiving a plurality of broadcast network data packets, wherein each of the plurality of broadcast network data packets is transmitted by the second device; as well as The state of the second device is selected using information in the plurality of broadcast network packets.

21. An apparatus for determining a status of an apparatus in a building, the apparatus comprising: At least one computer comprising at least one processor and at least one memory, the at least one computer being configured to: obtaining a power monitoring signal by measuring an electrical property of a power line in the building, wherein the power line provides power to the devices in the building, the devices in the building comprising a first device and a second device; determining a state of the first device by: (1) processing a first power event in the power monitoring signal using a plurality of power models to generate a score for each of the plurality of power models, and (2) selecting a state of the first device from a plurality of possible states using the scores; receiving a first network data packet from a server computer, wherein the server computer transmits the first network data packet based on information received from the second device; using the first network data packet to determine that the first network data packet corresponds to the second device; as well as The first network data packet is used to select a state of the second device from a plurality of possible states of the second device.

22. The device of claim 21, wherein the first network data packet comprises a broadcast using Simple Service Discovery Protocol, Zero Configuration Networking, or NetBIOS.

23. The apparatus of claim 21, wherein the at least one computer is further configured to: transmitting a request for information to the second device; Wherein receiving the first network data packet includes receiving a response to the request for information.

24. The device of claim 23, wherein the at least one computer transmits the request for information via polling the second device, requesting information using an API of the second device, or requesting information about a service provided by the second device.

25. The device of claim 21, wherein the at least one computer selects the state of the second device by: (1) processing information in the first network data packet using a plurality of network models, and (2) selecting a highest scoring network model, wherein the highest scoring network model corresponds to the selected state.

26. The device of claim 21, wherein selecting the state of the second device comprises processing information in the first network packet using a plurality of rules, wherein each rule of the plurality of rules comprises comparing the information in the first network packet to at least one condition.

27. The device of claim 21, wherein selecting the state of the second device comprises: processing a second power event in the power monitoring signal using a plurality of power models to generate a second score for each power model in the plurality of power models; processing information about the first network data packet using a plurality of network models to generate a third score for each of the plurality of network models; as well as The state is selected using the second score and the third score.

28. The device of claim 21, wherein selecting the state of the second device comprises: generating a second score for each state change model of a plurality of state change models, wherein each state change model processes a second power event in the power monitoring signal and information in the first network data packet; as well as The state is selected using the second score.

29. The apparatus of claim 21, comprising: receiving a plurality of broadcast network data packets, wherein each of the plurality of broadcast network data packets is transmitted by the second device; as well as The state of the second device is selected using information in the plurality of broadcast network packets.

30. A system for identifying equipment in a building, the system comprising: At least one computer comprising at least one processor and at least one memory, the at least one computer being configured to: obtaining a power monitoring signal by measuring an electrical property of a power line in the building, wherein the power line provides power to the device in the building; processing a first power event in the power monitoring signal using a plurality of power models to generate a score for each power model in the plurality of power models, wherein the first power event corresponds to a state change of a first device; determining first information about the first device using the score; causing the first device to be updated in a device list using the first information about the first device; receiving a first network data packet, wherein the first network data packet is sent by a second device; using information in the first network data packet to determine second information about the second device; as well as The second device is caused to be updated in the device list using the second information about the second device.

31. The system of claim 30, wherein the first information about the first device includes a type of the first device, a manufacture of the first device, or a version of the first device.

32. The system of claim 30, wherein the at least one computer is configured to: receiving a second network data packet, wherein the second network data packet is sent by the first device; and The first information about the first device is determined using the plurality of scores and information in the second network data packet.

33. The system of claim 32, wherein the plurality of scores are used to determine a type of the first device, and the information in the second network data packet is used to determine a make of the first device.

34. The system of claim 32, wherein the first power event in the power monitor signal occurs prior to receiving the second network data packet.

35. The system of claim 32, wherein the at least one computer is configured to select the second network packet by comparing a time of the first power event to a time of the second network packet.

36. The system of claim 30, wherein the information in the first network data packet includes an identifier of the second device, and wherein the at least one computer is configured to obtain the second information about the second device from a data store using the identifier of the second device.

37. A system according to claim 30, wherein the at least one computer is configured to determine second information about the second device by processing the information in the first network data packet using multiple rules, wherein each of the multiple rules includes comparing the information in the first network data packet with at least one condition.

38. The system of claim 30, wherein the at least one computer is configured to: receiving a plurality of network data packets, wherein each of the plurality of network data packets is sent by the first device; and The first information about the first device is determined using a plurality of scores and information in the plurality of network packets.

39. The system of claim 30, wherein the at least one computer is configured to: determining the second information about the second device by processing the information in the first network packet using a plurality of network models to generate a second score for each of the plurality of network models; and The second information about the second device is determined using the second scores of the plurality of network models.

40. The system of claim 30, wherein the at least one computer is configured to: determining the second information about the second device by generating a second score for each of a plurality of recognition models, wherein each recognition model processes a second power event in the power monitoring signal and the information in the first network data packet; and The second information about the second device is determined using the second scores of the plurality of recognition models.

41. The system of claim 30, wherein the at least one computer is configured to: receiving a plurality of network data packets, wherein each of the plurality of network data packets is sent by the second device; and The second information about the second device is determined using information in the plurality of network packets.

42. The system of claim 41, wherein the at least one computer is configured to determine the second information including using a first time corresponding to the first network data packet and a second time corresponding to the second network data packet.

43. A computer-implemented method for identifying a device in a building, the method comprising: obtaining a power monitoring signal by measuring an electrical property of a power line in the building, wherein the power line provides power to the device in the building; processing a first power event in the power monitoring signal using a plurality of power models to generate a score for each power model in the plurality of power models, wherein the first power event corresponds to a state change of a first device; determining first information about the first device using the score; causing the first device to be updated in a device list using the first information about the first device; receiving a first network data packet, wherein the first network data packet is sent by a second device; using information in the first network data packet to determine second information about the second device; as well as The second device is caused to be updated in the device list using the second information about the second device.

44. The computer-implemented method of claim 43, wherein the power monitoring signal is indicative of a current or a voltage of the transmission line.

45. The computer-implemented method of claim 43, comprising receiving a plurality of broadcast network data packets, each of the plurality of broadcast network data packets being transmitted by the first device; and determining the first information about the first device using a plurality of scores and information in the plurality of broadcast network data packets.

46. ​​A computer-implemented method according to claim 43, wherein determining the second information about the second device includes processing the information in the first network data packet using multiple network models to generate a second score for each network model in the multiple network models; and using the second scores of the multiple network models to determine the second information about the second device.

47. A computer-implemented method according to claim 43, wherein determining the second information about the second device includes generating a second score for each recognition model of a plurality of recognition models, wherein each recognition model processes a second power event in the power monitoring signal and information in the first network data packet; and determining the second information about the second device using the second scores of the plurality of recognition models.

48. The computer-implemented method of claim 43, comprising receiving a plurality of broadcast network data packets, wherein each of the plurality of broadcast network data packets is transmitted by the second device; and using information in the plurality of broadcast network data packets to determine the second information about the second device.

49. The computer-implemented method of claim 48, wherein determining the second information comprises using a first time corresponding to a first broadcast network data packet and a second time corresponding to a second broadcast network data packet.

50. A device for identifying a device in a building, the device comprising: At least one computer comprising at least one processor and at least one memory, the at least one computer being configured to: obtaining a power monitoring signal by measuring an electrical property of a power line in the building, wherein the power line provides power to the device in the building; processing a first power event in the power monitoring signal using a plurality of power models to generate a score for each power model in the plurality of power models, wherein the first power event corresponds to a state change of a first device; determining first information about the first device using the score; causing the first device to be updated in a device list using the first information about the first device; receiving a first network data packet, wherein the first network data packet is sent by a second device; using information in the first network data packet to determine second information about the second device; as well as The second device is caused to be updated in the device list using the second information about the second device.

51. The device of claim 50, wherein the first information about the first device comprises a type of the first device, a manufacture of the first device, or a version of the first device.

52. The apparatus of claim 50, wherein the at least one computer is configured to: receiving a second network data packet, wherein the second network data packet is sent by the first device; and The first information about the first device is determined using the score and information in the second network data packet.

53. The device of claim 52, wherein the score is used to determine a type of the first device and the information in the second network data packet is used to determine a make of the first device.

54. The apparatus of claim 52, wherein the first power event in the power monitor signal occurs prior to receiving the second network data packet.

55. The apparatus of claim 52, wherein the at least one computer is configured to select the second network data packet by comparing a time of the first power event to a time of the second network data packet.

56. The device of claim 50, wherein the information in the first network data packet includes an identifier of the second device, and wherein the at least one computer is configured to obtain the second information about the second device from a data store using the identifier of the second device.

57. A device according to claim 50, wherein the at least one computer is configured to determine second information about the second device by processing the information in the first network data packet using multiple rules, wherein each of the multiple rules includes comparing the information in the first network data packet with at least one condition.

58. The apparatus of claim 50, wherein the at least one computer is configured to: receiving a plurality of network data packets, wherein each of the plurality of network data packets is sent by the first device; and The first information about the first device is determined using the score and information in the plurality of network packets.

59. The apparatus of claim 50, wherein the at least one computer is configured to: determining the second information about the second device by processing the information in the first network packet using a plurality of network models to generate a second score for each of the plurality of network models; and The second information about the second device is determined using the second scores of the plurality of network models.

60. The apparatus of claim 50, wherein the at least one computer is configured to: determining the second information about the second device by generating a second score for each of a plurality of recognition models, wherein each recognition model processes a second power event in the power monitoring signal and information in the first network data packet; and The second information about the second device is determined using the second scores of the plurality of recognition models.

61. The apparatus of claim 50, wherein the at least one computer is configured to: receiving a plurality of network data packets, wherein each of the plurality of network data packets is sent by the second device; and The second information about the second device is determined using information in the plurality of network packets.

62. The apparatus of claim 61, wherein the at least one computer is configured to determine the second information comprises using a first time corresponding to a first network data packet and a second time corresponding to a second network data packet.

Images