In-home event talkback and notification

Abstract

Techniques for in-home health talkback and notification are disclosed. In one example, a first device receives, from a first user device using a first transport protocol, a message including a description of an event of a first person in an environment. The first device detects that the message is an event message. The first device transmits the event message to a second computing device in the environment using a second transport protocol, wherein the first device is in operable communication with the second device using a local area network. The first device receives a response from the second device using the second transport protocol that the second computing device has transmitted the event message to a second user device.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority under 35 U.S.C. §119(e) to U.S. Provisional Application No. 63 / 337,952, filed on May 3, 2022, and U.S. Non-provisional Application No. 18 / 142,451, filed on May 2, 2023, the disclosures of which are incorporated herein by reference in their entirety for all purposes. Technical Field

[0003] The present application relates to the field of networks, and in particular to intercom and notification of events within a residence in the network. Background Art

[0004] Modern electronic devices are seamlessly integrated into surrounding environments, such as residential or office environments. These electronic devices are becoming increasingly popular for users to access a range of healthcare-based applications. Users can connect to the Internet via electronic devices in their homes or offices and access healthcare-based applications via their mobile phones, tablet computers, home entertainment systems, etc. Summary of the invention

[0005] In some specific implementations, a first computing device receives an event message (e.g., a health event message) from a first user device via a short-range transmission protocol. The message includes a description of an event (e.g., a health event) experienced by a first person in a residential environment. The first computing device may be operable to communicate with a second computing device via a smart home platform that manages the residential environment. The first computing device transmits the message to the second computing device via an Internet-based transmission protocol. The second computing device may detect a second person in the residential environment who may provide assistance to the first person. The first device receives a response from the second device via the Internet-based transmission protocol that the second computing device has detected the second person and has transmitted the message to a second user device associated with the second person.

[0006] In some implementations, the first computing device transmits the message to multiple computing devices throughout the environment, including the second computing device. The first computing device transmits instructions along with the message to each other computing device in the residential environment to determine whether another person in the residential environment can be detected. Each of the multiple computing devices can determine whether they are near a user device associated with a person. Upon detecting the device, each of the multiple computing devices can determine the likelihood that the person associated with the user device is still nearby. Each of the multiple computing devices can then transmit a response back to the first computing device as to whether they have detected a person who can provide assistance.

[0007] In some implementations, the first computing device may launch a virtual assistant in response to receiving the message. The virtual assistant may be integrated with the first computing device. The virtual assistant may ask the first person if they need help. The virtual assistant may receive and record a response from the first person. The virtual assistant may further analyze the response to determine whether the first person needs help. The first computing device may transmit the message to the second computing device based on whether the first person needs help.

[0008] Each specific implementation provides at least the following advantages. The first user device that detects the event and transmits the message to the first computing device may be a battery-powered device. By transmitting the message via a short-range transmission protocol, the first user device can save battery power. The saved battery power can be used to power an alarm signal (such as a flash or sound) to help people find the first person. The first computing device can communicate with other devices in the residential environment to find someone who can provide assistance to the first person. By finding someone in the residential environment, the residential health intercom and notification system reduces the response time to provide assistance to the first person. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is an illustration of an in-home health intercom and notification system according to one or more embodiments.

[0010] Figure 2 is an illustration of an in-home intercom and notification system in a residential environment according to one or more embodiments.

[0011] Figure 3 is an illustration of an in-home intercom and notification system in a residential environment according to one or more embodiments.

[0012] Figure 4 is an illustration of an in-home intercom and notification system in a residential environment according to one or more embodiments.

[0013] Figure 5 is an illustration of a computing device for receiving intercom messages and notifications according to one or more embodiments.

[0014] Figure 6 is a diagram of a resident device according to one or more embodiments.

[0015] Figure 7 is a signaling diagram for an in-home intercom and notification system according to one or more embodiments.

[0016] Figure 8 is a process flow for an in-home intercom and notification system according to one or more embodiments.

[0017] Fig. 9 is a process flow for an in-home intercom and notification system according to one or more embodiments.

[0018] Fig.10 is a process flow for transmitting event messages via an in-home intercom and notification system according to one or more embodiments.

[0019] Fig.11 is a process flow for providing incident-related mitigation via an in-home intercom and notification system according to one or more embodiments.

[0020] Fig.12 is a process flow for an in-home intercom and notification system according to one or more embodiments.

[0021] Fig.13 is a process flow for detecting critical events via an in-home intercom and notification system according to one or more embodiments.

[0022] Fig.14 A process flow for detecting a device via an in-home intercom and notification system according to one or more embodiments. DETAILED DESCRIPTION

[0023] In the following description, various examples will be described. For the purpose of explanation, specific configurations and details are set forth in order to provide a thorough understanding of the examples. However, it will also be apparent to those skilled in the art that some examples may be implemented without these specific details. In addition, well-known features may be omitted or simplified to avoid confusing the described examples.

[0024] Computing devices are integrated throughout modern residential and office environments. These computing devices can execute software applications that improve the user's daily life and help them stay healthy. For example, a wearable device may include a biometric scanner for monitoring physiological characteristics, and a fall detection system may detect whether the user has fallen. In some instances, if an adverse event (e.g., an adverse health event) such as an irregular heartbeat or a fall is detected, the wearable device may further alert a healthcare provider or emergency medical service. One question is whether the person needs to automatically communicate with the emergency medical service or healthcare provider. In many instances, people can simply recover on their own. For example, not every fall requires emergency medical services to respond. Another issue is power conservation. In many instances, these automatic alerts are issued via Wi-Fi or cellular networks, which draw more power than short-range transmission protocols. People who need help may need their device power to last until help can arrive. Transmitting messages via a more power-dependent transmission protocol may waste precious battery life.

[0025] Embodiments described herein solve the above problems by utilizing device connectivity so that a computing device can detect whether a person needs help and convey a message to another person in the same area. A user device can detect that a first person in a residential environment needs help. For example, a user device may be equipped with a positioning sensor that detects that a first person has fallen in a specific room / area (e.g., a kitchen of a residence). A user device can generate an event message (e.g., a health event message) and transmit the message to another device in a residence. The event message may include an identifier indicating that the message is an event message. The message may be transmitted via short-range wireless technology to save power. A first resident device may receive the message and then transmit the message to other resident devices in the entire location to search for someone who can help. The resident device may be connected to an external power source, and therefore communicates via a transmission protocol (Wi-Fi and cellular) that is more dependent on power. Transmission protocols that are more dependent on power have a larger signal range, which increases the probability of finding someone who can help in a residence.

[0026] In some embodiments, each resident device in a residential environment may be configured to perform an embodiment of the present disclosure. In some embodiments, one or more resident devices may perform operations to synchronize with each other (e.g., and / or coordinate operations). For example, a specific resident device of an environment may be selected among other resident devices to perform one or more embodiments in the embodiments described herein. In this example, other resident devices (e.g., in different rooms of the environment) may route messages to the selected resident device for performing these operations (e.g., identifying a recipient device, and / or conveying announcements to other devices of the residential environment). Each resident device is typically configured for a specific location in an environment. For example, a smart speaker may be dedicated to a kitchen, while another smart speaker may be dedicated to a bedroom. Each resident device is typically connected to a wall socket and is therefore configured to be infrequently moved, which is contrary to a smart watch or smart phone that moves around with a user. Additionally, each resident device that receives a message from a first resident device may locally determine whether it is near someone who can provide assistance. In some embodiments, the determination may be based on a confidence score near a resident device. In the instance that the resident device determines that it is near a person, the resident device can transmit a message to the user device (e.g., mobile device, smart watch) of the person. The person can receive the message and provide assistance to the first person who is experiencing the event.

[0027] Figure 1 is an illustration of a residential health intercom and notification system according to one or more embodiments. The residential health intercom and notification system 100 can be implemented in an environment such as a residential environment or an office environment, or at any location with a network of connected devices, such as a rental property, a park, or other public place. Each environment can correspond to a defined (e.g., limited) area and / or structure (e.g., a living unit). Figure 2 Describing the residential environment in more detail, the in-home health intercom and notification system 100 enables a resident device to determine that someone in the environment is experiencing an event, to find another person in the same environment and to send a message to the other person.

[0028] In the event that there is more than one person in the environment, the resident device may send a message to each person in the environment. For example, if there are two or more people in the environment, the resident device may send a message to each of those people's devices. Figure 4 and Fig.14 This feature is described in more detail.

[0029] As illustrated, the first person 102 may experience an event such as a heart attack or a fall within the environment. A device such as the first user device 104 may detect that the first person 102 has experienced an event and may need help. For example, the first user device 104 may be a smart watch with location detection hardware, other sensors, and accompanying software. The smart watch may include sensors that detect, for example, a fall motion of the first person 102. For example, the smart watch may also include sensors that detect the heart rate, blood pressure, or other biometric data of the first person. The smart watch may also include software that interprets the sensor data to determine whether the first person 102 has experienced an event.

[0030] In other embodiments, the first user device 104 may be an image capture device with image recognition software. The image capture device may capture images as time series data of the first person 102. The image capture device may access software that implements a neural network (e.g., a gated recurrent unit (GRU) network or a long short-term memory (LTSM) network) that analyzes the images and determines whether the first person 102 has experienced an event.

[0031] The first user device 104 may generate an event message (e.g., a health event message) in response to determining that the first person 102 has experienced or is experiencing an event. The message may include the identity of the first person 102. In many instances, the first user device 104 is associated with a user account. For example, the first user device 104 includes security features for the operating system of the first user device. These security features may include the identity of the first person 102. In other instances, the identity of the first person may be confirmed by other services associated with the first person 102 and the first user device 104. The message may also include an alert that the first person 102 may need help. The message may also include the location of the first user device 104. For example, the first user device 104 may include a location service for determining a location. The first user device 104 may use the service to determine its location and include the location in the message. The message may also include an identifier indicating that the message is an event message.

[0032] The first user device 104 can locate the nearest resident device. In many instances, the first user device 104 is connected to the resident device via a local network. In other instances, the first user device 104 can be connected to the resident device via a smart home platform (e.g., Apple ) is connected to the resident device. In instances where the first user device 104 is connected to the resident device via a local area network (LAN), the first user device 104 can discover other devices connected to the network via the LAN. For example, as the first user device 104 moves around in the environment, the first user device can discover other devices as it passes by them. In instances where the first user device 104 is connected to the resident device through a smart home platform, the first user device 104 can discover other devices via the platform. In each instance where the first user device 104 discovers another computing device, the first user device 104 can store the discovery in its local memory.

[0033] In response to detecting an event, the first user device 104 may scan its local memory and determine the last discovered device. For example, the first user device 104 may store an identifier and a network address (e.g., a media access control (MAC) address) of each discovered device in a memory. The first user device may then compare the timestamps associated with discovering these devices to determine the last discovered device. The last discovered device may be the device closest to the first user device 104. Therefore, the last discovered device may be reachable via a short-range transmission protocol.

[0034] As illustrated, the last device found in the environment is the first resident device 106. The first user device 104 can transmit a message to the first resident device 106 via a first transmission protocol, which can be a short-range transmission protocol, such as a wireless technology (such as Bluetooth, Zigbee, etc.). If a person needs help, saving the battery life of the person's device can provide more time for the person to contact and communicate with another person. Compared with other transmission protocols (such as Wi-Fi and cellular), short-range transmission protocols draw less power from the device. Therefore, the first user device 10 saves battery life by communicating with the first resident device 106 via the first transmission protocol.

[0035] The first resident device 106 can receive the message from the first user device 104. In some examples, the first resident device 106 communicates with a virtual assistant (e.g., ) integration. For example, the first resident device 106 is a smart speaker with an integrated virtual assistant. In these instances, the first resident device 106 may transmit computer-readable instructions to the virtual assistant to guide the assistant to verify whether the first person 102 needs help. The virtual assistant may ask the first person 102 whether he needs help. As illustrated, the virtual assistant may use the speaker of the first resident device to ask the person "Do you need help?" The virtual assistant may further record the response from the first person 102. As illustrated, the first person 102 says "I need help" in response to the virtual assistant's question. The first resident device 106 may record the response, for example, through a microphone integrated with the smart speaker. In other instances, the first resident device 106 may include other recording capabilities, such as image capture hardware. In these instances, the first resident device may capture an image or a collection of images of the response.

[0036] Based on the response, the first resident device 106 may determine whether the first person 102 needs help. For example, the virtual assistant may access natural language processing (NLP) capabilities, such as NLP services. The virtual assistant may also access image processing capabilities, such as neural networks. The virtual assistant may transmit its determination to the first resident device 106. If the virtual assistant determines that the first person 102 does not need help, the first resident device 106 may stop processing the message. If the first resident device determines that the first person 102 needs help, the first resident device may continue to process the message. In some cases, the first person 102 may be incapacitated or otherwise unable to respond to queries from the virtual assistant. Therefore, the virtual assistant may start a counter or countdown timer from the moment the question is asked. If the first person 102 does not respond within a threshold time, the virtual assistant may determine that the first person needs help.

[0037] The first resident device 106 may be connected to the second resident device 110 and the third resident device 112 via the network 108. Each of the second resident device 110 and the third resident device 112 may be, for example, a smart speaker. The network 108 may be a local network that communicates via a second transmission protocol (such as Wi-Fi or cellular). The first resident device 106 may transmit the message to each other resident device in the environment via the second transmission protocol. In some embodiments, the first resident device 106 may select the second transmission protocol based on receiving the message. For example, the first resident device 106 may include logic that causes the first resident device 106 to use the second transmission protocol based on receiving the message. In an instance where the virtual assistant obtains a recording of the first person 102, the first resident device may further append the message to include the recording.

[0038] The resident devices in the environment are typically connected to a power source via an outlet and not to a battery power source. Therefore, power considerations for the resident devices are different from power considerations for the first user device 104. Therefore, the first resident device 106 can transmit messages to the second resident device 110 and the third resident device 112 via a second transmission protocol (such as Wi-Fi or cellular). It should be understood that although only the first resident device 106, the second resident device 110, and the third resident device 112 are shown, the environment can include as many resident devices as desired or as few as one resident device.

[0039] Each of the second resident device 110 and the third resident device 112 may determine whether it can detect a person. Each of the second resident device 110 and the third resident device 112 may determine whether it can detect a user device associated with an account of a person. In some embodiments, the second resident device 110 and the third resident device 112 may further make a determination of whether there is a second person associated with the user device based on a confidence score.

[0040] The confidence score may be based on various factors, and each resident device may calculate a confidence score based on sensory input. The sensory input may be, for example, accelerometer-based data, gyroscope-based data, touch-based sensor data, audio data, device charging information, and application data. The sensory input may be collected locally by the resident device, or one or more other devices may collect the input data and transmit the input data to the resident device. For example, the other device may be any type of device capable of performing sensing capabilities, such as a microphone, a motion sensing device, a positioning sensing device, a position sensing device, an image capture device, or a temperature sensing device paired with the resident device.

[0041] Each resident device may calculate a confidence score that describes the likelihood that a person is present. For example, the resident device may be a tablet computer, and the person may interact with the tablet computer (e.g., using a word processing application, streaming video or audio, etc.). The resident device may calculate a confidence score based on the degree of interaction of the person with the tablet computer (e.g., with a word processing or other application). For example, if the person is actively typing via a word processing application, the resident device may calculate a confidence score that indicates that the person is present. However, if the person has stopped typing, the resident device may look at the time that has passed since the last instance of typing. If the time that has passed is greater than a threshold time, the confidence score may indicate that the person has moved away from the resident device.

[0042] The resident device may evaluate multiple factors when calculating the confidence score. These factors may include historical data associated with the application. For example, in addition to the length of time that has passed, the resident device may also access historical data about the average length of time that the person pauses when typing via a word processing application. In this example, the resident device may calculate the confidence score based on the elapsed time and the average length of time. The resident device may further use a set of heuristics to determine whether to weight one factor more than another factor.

[0043] As illustrated, second resident device 110 may detect second user device 114 and determine that second person 116 is present. Second resident device 110 may further transmit a message to second user device 114. Second user device 114 may display the message for second person 116. Second resident device 110 may transmit a message back to first resident device that it has detected second person 116 and has transmitted the message.

[0044] As described above, in some embodiments, the message may include a recording of the first person 102. For example, the second user device 114 may play a recording of "I need help" for the second person 116. The message may further convey the identity and location of the first person 102 to the second person 116. The second person 116 may then go to the location of the first person 102 in the environment to provide assistance.

[0045] Figure 2 is an illustration of an in-home intercom and notification system in a residential environment according to one or more embodiments. In some embodiments, Figure 1 The residential in-home health intercom and notification system 100 can be implemented in a residential environment 202. Turning to the elements of the residential environment 202 in further detail, multiple elements are depicted. The residential environment 202 may include multiple rooms. As illustrated, the residential environment 202 includes a first room 204, a second room 206, and a third room 208. The first room 204 may be, for example, an upstairs bedroom. The second room 206 may be, for example, a kitchen, and the third room 208 may be, for example, a family room. As illustrated, a first resident device 210 (e.g., a smart speaker) may be positioned in the first room 204 and associated with the bedroom. For example, the first resident device 210 may be associated with an account holder having the first room 204 in the residential environment 202. Similarly, the second resident device 212 may be positioned in the second room 206 and associated with the kitchen. The third resident device 214 may be positioned in the third room 208 and associated with the family room. In some embodiments, one or more associations (eg, between a resident device and a particular room) may be stored by one or more resident devices of the residential environment 202 .

[0046] Devices located in the residential environment 202 can be managed by a smart home platform. The smart home platform can be software that enables authorized users to coordinate and control home automation accessories from a graphical user interface (GUI) regardless of the supplier. Each of the first resident device 210, the second resident device 212, and the third resident device 214 can be associated with the residential environment 202. As illustrated, each resident device is located in the residence. However, it should be understood that one or more resident devices may be located outside the physical residence, such as in a separate garage, a separate guest suite, or a shed. Each resident device in the resident device can be associated with a user account registered on the smart home platform. The user account can be an authorized user directly associated with the residential environment 202. For example, the authorized user may include a mother and a father living in the residence. The user account can also be associated with an authorized user indirectly associated with the residential environment 202. For example, the user account can be associated with a nanny, a house custodian, or a long-term guest.

[0047] In addition to these resident devices, the residential environment 202 may also be associated with a plurality of accessory devices such as smart lights, door locks, and appliances. Authorized users may access the smart home GUI using their smart phones to access the smart home platform. The user may further manipulate the GUI to control accessories such as smart lights. The residential environment 202 may be a home containing resident devices and accessory devices; and managed by the smart home platform.

[0048] A first user device 216 (e.g., a smartwatch) in the first room 204 and associated with a first person 218 may also be located within the residential environment 202. A second user device 220 (e.g., a mobile phone) may be located within the second room 206 and may be associated with a second person 222. One or both of the user devices may be associated with a user account associated with the residential environment 202. For example, the first user device 216 may be associated with a user account of a grandfather registered on a smart home platform. In other instances, one or both of the user devices may be devices that are not registered on a smart home platform. For example, the second user device 220 may be associated with a handyman performing repairs in the residence.

[0049] like Figure 2 As illustrated, the proximity of the first user device 216 to the first resident device 210 may be greater than the proximity of the first user device 216 to the second resident device 212 or the third resident device 214. Figure 2 A residential environment 202 is depicted in FIG. 2 , but a similar layout is contemplated in an office environment.

[0050] In some embodiments, event messages (e.g., health event messages) may be received or transmitted by any of the depicted devices. In some embodiments, a sending device may receive an instruction corresponding to the selection of a transmission protocol for transmitting a message. In some embodiments, a sending device may also (or alternatively) receive an instruction corresponding to the selection of one or more devices (e.g., a resident device) for transmitting a message. As described herein, any of the resident devices (e.g., the first resident device 210, the second resident device 212, and the third resident device 214) may be used to transmit event messages to other devices and / or locations within the residential environment 202. Therefore, although the resident device may be primarily described as transmitting announcements to other devices in the embodiments herein, the embodiments should not be construed as being limited thereto (e.g., suitable user devices and / or server devices may also perform the operations of the embodiments described herein).

[0051] In some embodiments, a sending device (e.g., a resident device or a suitable user device) may be configured to transmit a healthcare event to a specific device (e.g., a specific user device and / or a specific resident device) based on a detected proximity between one or more devices. In some embodiments, one or more devices associated with the residential environment 202 may transmit location information (e.g., GPS coordinates, speed and / or acceleration information, etc.) to a resident device of the residential environment. In some embodiments, any suitable real-time location service (RTLS) may be used to determine the location information (e.g., location data) of the device. Based on the location information determined relative to each device, the resident device may determine the location of each device and / or the relative proximity between the devices. As further illustrated in the representative scenarios described below, the location and / or proximity information may be used to select a specific one or more devices for receiving healthcare event messages.

[0052] Consider a scenario in which a first person 218 experiences an event (e.g., a fall, an accelerated heartbeat) detected by a sensor of a first user device 216. The first user device 216 may determine which other device it last detected. For example, each other device may be configured to periodically advertise its device identifier across a transmission protocol (such as a Wi-Fi network), and the first user device 216 may search its memory to determine which device identifier was last received. In another example, the first user device 216 may transmit a signal via a short-range transmission protocol (such as Bluetooth or Zigbee) to discover another device. The first user device 216 may determine the distance between itself and the other device based on the strength of the response signal from the other device.

[0053] As illustrated, the first user device 216 can discover the first resident device 210. The first resident device 210 can be in the first room 204, and the first user device 216 and the first person 218 can also be in the first room. As illustrated, the first resident device 210 has the maximum proximity with the first user device 216. The first resident device 210 can initiate a session with the first user device 216 via a first transmission protocol (such as a short-range transmission protocol). The first resident device 210 and the first user device 216 can actively form a connection, and each can enter a connected state. The connection does not require an authentication process in which the first person 218 needs to authorize the connection. As described above, the purpose of the short-range transmission protocol is to save the battery life of the first user device 216.

[0054] The first user device 216 can use a location service to retrieve the location. The first user device 216 can generate a message that can include the location of the first user device 216, the identity of the account holder associated with the first user device 216, and computer-readable instructions for the first resident device 210.

[0055] The first resident device 210 may receive a message from the first user device 216 via a short-range transmission protocol. In some embodiments, the first resident device 210 may initialize a virtual assistant in response to receiving the message. The virtual assistant may be a software agent executed on the first resident device 210 and integrated with the smart home platform. The virtual assistant may respond and perform tasks based on commands or questions. The virtual assistant may ask the first person 218 if he needs help. The virtual assistant may communicate with the first person 218 via the first resident device 210. For example, if the first resident device 210 is a smart speaker, the virtual assistant may talk to the first person 218 through the speaker. The virtual assistant may further record the response via software and hardware (such as a microphone and / or image capture device of the first resident device 210).

[0056] The first resident device 210 may transmit the message to each other resident device in the residential environment 202 via a second transmission protocol (such as Wi-Fi). Each resident device in the residential environment 202 is connected to a local network (such as a Wi-Fi network). Each resident device also includes identification information, such as the address and name of each other resident device in memory. As illustrated, the first resident device 210 may identify each other device and transmit the message to the second resident device 212 and the third resident device 214 via the second transmission protocol. In an embodiment in which the first resident device 210 initializes the virtual assistant, the first resident device 210 transmits a message with a record from the first person 218.

[0057] Resident devices, such as smart speakers, may draw power via a wall outlet rather than a battery. Therefore, for the first user device 216, which may be battery-powered, the power consumption issue in choosing to use a short-range transmission protocol or an Internet-based transmission protocol may be more significant. Additionally, messages transmitted via an Internet-based transmission may have a greater range than messages transmitted via a short-range transmission protocol. Therefore, the first resident device 210 may communicate with the second resident device 212 and the third resident device 214 via a second transmission protocol. By transmitting the message to each resident device via the second transmission protocol, the first resident device 210 increases the probability that the message reaches someone who can provide assistance.

[0058] Each of the second resident device 212 and the third resident device 214 may receive a message from the first resident device 210 via a second transmission protocol. As illustrated, the first resident device 210 transmits the message from the first room 204 to the second resident device 212 in the second room 206 of the residential environment 202 and the third resident device 214 in the third room 208. The message may include computer-readable instructions for determining whether a person is detectable by either resident device.

[0059] Each resident device in the second resident device 212 and the third resident device 214 can determine whether it can detect the person in its corresponding room. In some embodiments, if the second resident device 212 or the third resident device 214 detects the second user device, it can be assumed that the second person exists with the device. As described above, in some other embodiments, the confidence score of the person in the environment is determined based on the existence of the person. Each resident device in the second resident device 212 and the third resident device 214 can perform various techniques to detect the user device. For example, the second resident device 212 and the third resident device 214 can search its memory to determine whether the user device is found. In another example, each other device can transmit a signal via a short-range transmission protocol (such as Bluetooth or Zigbee) to find another device. If the second resident device 212 or the third resident device 214 detects another user device, the second resident device or the third resident device can send an event message to the detected user device.

[0060] The second resident device 212 and the third resident device 214 may then make a determination based on the confidence score whether a person is present. As described above, the determination may be based on various factors. For example, a person presses a volume button on a smart speaker within a threshold time period. Another example may be receiving an audio signal within a threshold time period. Another example may be detecting a device associated with a person within a threshold time period. For example, detecting a smart phone or smart watch registered with the user. If a person is present based on the confidence score, the resident device may transmit a message that a person is present to the first resident device 210.

[0061] As illustrated, no person or user device is located in the third room 208 with the third resident device 214. As illustrated, the second resident device 212 is located in the second room with the second person 222. The second resident device 212 can detect the second user device 220 and determine whether a person is present. The second resident device 212 can transmit a message back to the first resident device 210 that it has detected a person.

[0062] The second resident device 212 can transmit the message to the second user device 220. The message can be presented on the second user device 220 in a manner that attracts the attention of the second person. For example, if the second user device 220 is a smart watch or a smart phone, the message can be presented on the display as a widget. The widget can be an extension of an application executed on the second user device 220. For example, the widget can be an extension of an SMS application or a smart home platform application. The widget can also occupy a larger space on the device display than a standard application icon. In some embodiments, the message includes a note to the first person 218. For example, Figure 1 As illustrated, the message includes the recording "I need help." The second user device 220 may include a prompt via an icon that allows the second person 222 to listen to the recording. Figure 4 Aspects of the message on the second user device 220 are described in more detail. Upon receiving the message, the second person 222 can determine the identity of the first person 218, determine the location of the first person 218 in the residential environment 202, and receive a record of the first person 218. The second person 222 can then find the first person 218 and provide assistance to the first person.

[0063] In some examples, the first resident device 210 does not receive any responses from any other resident devices in the residential environment 202. The first resident device 210 may, for example, include time tracking software that monitors the time that has passed after receiving the message from the first user device 216. The first resident device 210 may transmit the message to emergency services in the event that a threshold time has passed after receiving the message from the first user device 216. For example, the first resident device 210 may transmit the message to a dedicated emergency services address via the Internet.

[0064] The first resident device 210 may further detect one or more individuals entering the environment. These individuals may be, for example, law enforcement officers, emergency medical technicians, ambulance drivers, or neighbors. The first resident device 210 may use various technologies (such as device detection, motion sensing, audio sensing, and image sensing) to detect individuals. The first resident device 210 may employ these technologies alone or in combination. For example, the first resident device 210 may be connected to a network, such as a local area network (LAN). In the case where an individual enters the environment using a device, the device may be connected to the network. In addition, a connection including a device identifier may be broadcast to other devices in the network. In response to receiving the broadcast, the first resident device 210 may check its memory to determine whether it has previously detected the device identifier. Based on contact with emergency services, the first resident device may determine that the individual has entered the environment in response to the event. In another example, the first resident device 210 may employ motion sensing and detect motion within the environment. For example, the first resident device may detect a door opening, a window opening, or a person moving around the environment.

[0065] In yet another example, the first resident device 210 may include hardware and software for enabling sensing capabilities (such as audio or visual sensing capabilities). The first resident device 210 may use sensing capabilities to record its surroundings. For example, in response to transmitting a message to an emergency service, the first resident device may begin to record audio and / or visually. The first resident device 210 may further analyze the recording or transmit the recording to a service for analyzing the recording to detect individuals in the environment. The analysis may include determining that the individual has entered the environment and the characteristics of the individual. For example, if the first resident device 210 uses audio sensing capabilities, the first resident device may determine that the individual has entered the environment based on detecting various audio inputs (such as door opening sounds or voices). The first resident device 210 may further determine the characteristics of the individual based on the audio input. For example, the first resident device 210 may directly or through a service use natural language processing (NLP) technology and determine the characteristics of the individual. For example, NLP techniques can be used to detect identification phrases such as "Law enforcement, entering residence," "Emergency medical services, can you hear me?" or "I'm your neighbor, are you okay?" These phrases can be mapped to characteristics of individuals, such as "Law enforcement, entering residence" can be mapped to law enforcement.

[0066] In yet another example, if the first resident device 210 employs visual sensing capabilities, the first resident device may determine that an individual has entered the environment based on detecting various visual inputs, such as a door opening or an individual in the environment. It should be understood that the visual sensing capability may be a substitute for or in combination with the audio sensing capability. The first resident device 210 may further determine the characteristics of the individual based on the visual input. For example, the first resident device 210 may employ image processing techniques directly or through a service and determine the characteristics of the individual. For example, image processing techniques may be used to detect identification objects, such as law enforcement badges, life star badges, and faces of neighbors. These objects may be mapped to the characteristics of the individual, such as the life star badge may be mapped to an emergency medical technician. It should be understood that in some instances, the first resident device 210 includes hardware and software for providing sensing capabilities. In other instances, the first resident device 210 may receive sensor data from one or more other devices. For example, the first resident device 210 may receive sensor data from audio and image capture devices in the environment. As described above, the first resident device 210 may be connected to a network, and other devices may also be connected to the network. If the first resident device 210 lacks one or more sensing capabilities, the first resident device may contact another device connected to the network and having these capabilities to begin recording. For example, after transmitting the message to the emergency services, the first resident device may detect each other device connected to the network, send a message describing the response of the sensing capabilities to the other devices, and transmit a message to begin recording based on the response.

[0067] In some instances, the residential environment 202 may include a fourth resident device 224 capable of presenting an audio or visual message. The fourth resident device 224 may be, for example, a television or a laptop computer. As illustrated, the first room 204 includes a fourth resident device 224 in the form of a smart television. The fourth resident device 224 may be managed by a smart home platform that manages the residential environment 202. The first resident device 210 may determine the characteristics of the first person's event. The first resident device 210 may further transmit instructions for presenting mitigation steps to the fourth resident device 224. For example, the first resident device 210 may receive information that the first person is choking from the first user device 216. In response, the first resident device 210 may search online resources to obtain cardiopulmonary resuscitation (CPR) instructions. The first resident device 210 may further transmit computer-readable instructions that instruct the fourth resident device 224 to access online resources to display cardiopulmonary resuscitation (CPR) instructions. In some instances, the first resident device 210 may present mitigation steps in the absence of the fourth resident device 224.

[0068] The first resident device 210 may further transmit to the first user device 216 a message that the second resident device 212 detected the second person 222. In response, the first user device 216 may present an alert signal. The alert signal may be any signal that draws attention to the first user device 216 and, therefore, to the first person 218. For example, if the first user device 216 is a smartwatch or a smart phone, the first user device may present an auditory noise, such as a siren or beep. The first user device 216 may also present a visual effect, such as a strobe light effect or other light-based effect.

[0069] It should be understood that a single resident device rather than multiple resident devices may be used to perform the above functionality. Figure 3 is an illustration of an in-home intercom and notification system in a residential environment according to one or more embodiments. In some embodiments, Figure 1 The in-home health intercom and notification system 100 can be implemented in a residential environment 302. Turning to the elements of the residential environment 302 in further detail, multiple elements are depicted. The residential environment 302 may include multiple rooms. As illustrated, the residential environment 302 includes a first room 304 and a second room 306. The first room 304 can be, for example, an upstairs bedroom. The second room 306 can be, for example, a kitchen. As illustrated, a resident device 310 (e.g., a smart speaker) can be positioned within the first room 304 and associated with the bedroom. For example, the resident device 310 can be associated with an account holder who is staying in the first room 304 in the residential environment 302.

[0070] Resident device 310 may be managed by a smart home platform. A smart home platform may be software that enables authorized users to coordinate and control home automation accessories from a graphical user interface (GUI), regardless of vendor.

[0071] A first user device 312 (e.g., a smartwatch) in the first room 304 and associated with a first person 314 may also be located within the residential environment 302. A second user device 316 (e.g., a mobile phone) may be located within the second room 306 and may be associated with a second person 318.

[0072] The first person 314 experiences an event (e.g., a fall, an accelerated heartbeat, etc.) detected by a sensor associated with the first user device 312. The first user device 312 may determine which other device it last detected (e.g., one of a plurality of different other devices that the first user device 312 recently communicated with). For example, each other device (e.g., a resident device 310, a second user device 316, etc.) may be configured to periodically advertise its device identifier across a transmission protocol (such as a Wi-Fi network), and the first user device 312 may search a local memory to determine which device identifier corresponding to another device was last received. In another example, the first user device 312 may transmit a signal via a short-range transmission protocol (such as Bluetooth or Zigbee) to discover another device. The first user device 312 may determine the distance between itself and the other device based on the strength of the response signal from the other device.

[0073] As illustrated, a first user device 312 may discover a resident device 310. The resident device 310 may be in a first room 304, where the first user device 312 and a first person 314 may also be. The resident device 310 may initiate a session with the first user device 312 via a first transmission protocol, such as a short-range transmission protocol. In some examples, the session may not require an authentication process in which the first person 314 needs to authorize the connection. As described above, one use of the short-range transmission protocol is to save battery life of the first user device 312.

[0074] The first user device 312 can use the location service to retrieve a location corresponding to its own location. The first user device 312 can generate a message including sensor data, which can include the location of the first user device 312, the identity of the account holder associated with the first user device 312, and computer-readable instructions for the resident device 310.

[0075] The resident device 310 may receive a message from the first user device 312 via a short-range transmission protocol. In some embodiments, the resident device 310 may initialize the virtual assistant in response to receiving the message. The virtual assistant may ask the first person 314 if he needs help. For example, if the resident device 310 is a smart speaker, the virtual assistant may talk to the first person 314 through one or more speakers of the resident device 310. The virtual assistant may further record the response (e.g., using a microphone and / or image capture device of the resident device 310).

[0076] The resident device 310 may determine whether it can detect a second user device in the residential environment 302. If the resident device 310 can detect the second user device, it may be assumed that the second person is present. In some embodiments, determining the second person may be based on a confidence score. The resident device 310 may perform various techniques to detect a user device (e.g., the second user device 316). For example, the resident device 310 may search its memory to determine whether the second user device was previously discovered.

[0077] The resident device 310 can transmit the message to the second user device 316. The message can be presented on the second user device 316 in a manner that attracts the attention of the second person. For example, if the second user device 316 is a smart watch or a smart phone, the message can be presented on the display as a widget or notification (e.g., a pop-up window). In some cases, the widget can be an extension of an application executed on the second user device 316. For example, the widget can be an extension of an SMS application or an intelligent residential platform application. The widget can also occupy a larger space than a standard application icon on the device display. Alternatively, the message can be presented as a pop-up notification, thereby covering other parts of the screen, including user interface elements that the user is interacting with. In some embodiments, the message includes a record of the first person 314 or includes a record of the surrounding environment of the first resident device in an instance where the first person 314 cannot communicate. In addition to the record, the event message can also include the identity of the first person 314, the location of the first person 314 in the residential environment 302. The second person 318 can then find the first person 314 and provide help to the first person.

[0078] In some instances, the resident device 310 may not detect any other user devices. The resident device 310 may, for example, include time tracking software that monitors the time that has passed after receiving the message from the first user device 312. If a threshold time has passed after receiving the message from the first user device 312, the resident device 310 may contact emergency medical services. The emergency medical services may include calling 911 or calling an emergency contact if an emergency contact has been provided.

[0079] The resident device 310 may further transmit to the first user device 312 a message that it has detected the second person 318. In response, the first user device 312 may present an alert signal. The alert signal may be any signal that draws attention to the first user device 312 and, therefore, to the first person 314. For example, if the first user device 312 is a smart watch or a smart phone, the first user device may present an auditory noise, such as a siren or beep. The first user device 314 may also present a visual effect, such as a strobe light effect or other light-based effect.

[0080] Figure 4 402 is a diagram of an intra-residential intercom and notification system in a residential environment according to one or more embodiments. Turning to the elements of the residential environment 402 in further detail, a plurality of elements are depicted. The residential environment 402 may include a plurality of rooms. As illustrated, the residential environment 402 includes a first room 404, a second room 406, and a third room 408. The first room 404 may be, for example, an upstairs bedroom. The second room 406 may be, for example, a kitchen. The third room 408 may be, for example, a study. As illustrated, a resident device 410 (e.g., a smart speaker) may be positioned within the first room 404 and associated with the bedroom. For example, the resident device 410 may be associated with an account holder who is staying in the first room 404 in the residential environment 402.

[0081] Resident device 410 may be managed by a smart home platform. A smart home platform may be software that enables authorized users to coordinate and control home automation accessories from a graphical user interface (GUI), regardless of vendor.

[0082] A first user device 412 (e.g., a smartwatch) in the first room 404 and associated with a first person 414 may also be located within the residential environment 402. A second user device 416 (e.g., a mobile phone) may be located within the second room 406 and may be associated with a second person 418. A third user device 420 (e.g., a mobile phone) may be located within the second room 406 and may be associated with a third person 422. Other variations of user devices in different rooms, partitions, etc. are contemplated, and the present disclosure is not limited to reference. Figure 4 Examples provided.

[0083] In some instances, the first person 414 may experience an event (e.g., a fall, an accelerated heartbeat, etc.) detected by a sensor associated with the first user device 412. The first user device 412 may determine which other device(s) it can detect (e.g., a plurality of different other devices that are nearby or that the first user device 412 has recently communicated with). For example, each other device (e.g., a resident device 410, a second user device 416, a third user device 420, etc.) may be configured to periodically advertise its device identifier across a transmission protocol (such as a Wi-Fi network), and the first user device 412 may search a local memory to determine which device identifier corresponding to another device was last received. In another example, the first user device 412 may transmit a signal via a short-range transmission protocol (such as Bluetooth or Zigbee) to discover another device. The first user device 412 may determine the distance between itself and the other device based on the strength of the response signal from the other device.

[0084] As described above, residential device 410 can be managed by a smart home platform (e.g., Apple HomeKit). In some instances, a person (e.g., first person 414, second person 418, or third person 422) with management privileges to the smart home platform may have added a fourth person 424 to the smart home platform. As illustrated, fourth person 424 may be located outside residential environment 402. In some examples, fourth person 424 may have a relationship (e.g., marital relationship or other relationship) with first person 414 and may have gone out to buy groceries. As a result of registering on the smart home application, resident device 410 may know fourth person 424. Resident device 410 may be configured to communicate with fourth user device 426 associated with fourth person 424. Since fourth person 424 may be outside residential environment 402, resident device 410 may be configured to communicate with fourth user device 426 using various transmission protocols (such as cellular, Wi-Fi, or Internet-based communications).

[0085] As illustrated, a first user device 412 may discover a resident device 410. The resident device 410 may be in a first room 404, where the first user device 412 and a first person 414 may also be. The resident device 410 may initiate a session with the first user device 412 via a first transmission protocol, such as a short-range transmission protocol. In some examples, the session may not require an authentication process in which the first person 414 needs to authorize the connection. As described above, one use of the short-range transmission protocol is to save battery life of the first user device 412.

[0086] The first user device 412 can use the location service to retrieve a location corresponding to its own location. The first user device 412 can generate an event message including sensor data, which can include the location of the first user device 412, the identity of the account holder associated with the first user device 412, and computer-readable instructions for the resident device 410.

[0087] The resident device 410 may receive an event message from the first user device 412 via a short-range transmission protocol. In some embodiments, the resident device 410 may initialize the virtual assistant in response to receiving the message. The virtual assistant may ask the first person 414 if he needs help. For example, if the resident device 410 is a smart speaker, the virtual assistant may talk to the first person 414 through one or more speakers of the resident device 410. The virtual assistant may further record the response (e.g., using a microphone and / or image capture device of the resident device 410).

[0088] The resident device 410 may determine whether it can detect another user device in the residential environment 402. If the resident device 410 detects another user device, it may be assumed that there is a person associated with the user device. In some embodiments, determining that there is another person in the residential environment 402 may be based on a confidence score. The resident device 410 may perform various techniques to detect other user devices (e.g., the second user device 416 and the third user device 420). For example, the resident device 410 may search its memory to determine whether another user device has been previously discovered. The discovery of other user devices may be based on notifications that other user devices have been connected to the local network used by the resident device 410. In other instances, the resident device 410 may discover the other user device in the residential environment 402 based on a signal transmitted to the resident device 410 by another device.

[0089] The resident device 410 may further determine whether it can detect another user device based on the user device associated with the smart house platform. As described above, people can add one or more user accounts to the smart house platform. The smart house platform can receive the user identification and user device information of each account associated with the smart house platform. For example, a person can add the account of a spouse or another sibling to their smart house platform. Consider the case where person A has a residence with one or more devices that can be controlled by a smart house application. Person A can add the account of person B to their smart house platform. Person B can be, for example, a person who also lives in the residence, or person B can also be, for example, a person who lives in another residence. In any case, the smart house platform can be configured with the information, identification, device identifier, contact information of each added person. The smart house platform can be operated as an application on the resident device 410, and the resident device can communicate with the smart house platform to identify the person added to the smart house platform.

[0090] The resident device 410 may generate an event message and transmit the event message to each identified device (e.g., the second user device 416, the third user device 40, and the fourth user device 426). The resident device 410 may also send the identity of each detected user device together with the event message. Therefore, the event message may include a record of the first person 414 or include a record of the surrounding environment of the resident device in an instance where the first person 414 cannot communicate. The event message may also include the location of the source of the message. For example, if the first person is located in a residential environment 402, the message may include an indication that the message originates from a residence. In addition to the record and location, the resident device 410 may also include the identity of the first person 414 and / or the location of the first person 414 in the residential environment 402. The resident device 410 may also send the identities of other detected devices. For example, the resident device 410 may send a transmission to the second user device 416, the transmission including the event message and the indication that the resident device 410 has transmitted the event message to the second user device 416, the third user device 420, and the fourth user device 426. The event message and the identities of the other user devices may be sent as a single message or as a group of messages.

[0091] The resident device 410 may further transmit a message to the first user device 412 that it has detected another person. In response to receiving the message from the resident device 410, the first user device 412 may present an alarm signal. The alarm signal may be any signal that draws attention to the first user device 412 and, therefore, to the first person 414. For example, if the first user device 412 is a smart watch or a smart phone, the first user device may present an auditory noise, such as a siren or beep. The first user device 416 may also present a visual effect, such as a strobe light effect or other light-based effect.

[0092] Figure 5 is an illustration of a computing device for receiving intercom messages and notifications according to one or more embodiments. User device 500 may be, for example, any suitable computing device, such as a smartwatch or a smart phone. For example, user device 500 may be Figure 1 The first user device 104 or the second user device 114. The user device 400 may also be Figure 2 The user device 400 may include a display 402 for displaying a message (e.g., a health event message). The display 402 may include touch sensor technology for converting touches into electronic signals. The display 402 may be configured to display one or more application icons 404. In some embodiments, the user device 400 may include an image capture device 406, a microphone 408, and a speaker 410.

[0093] The user device 400 may be configured to display a widget 412 on the display 402. The widget 412 may be an extension of an application executing on the user device 400. The widget may also be larger than one or more application icons 404. The message may be configured to be displayed on the widget 412. For example, the resident device may include computer-readable instructions and a message to display the event as a widget instead of an SMS message, an email message, or other message type. Since the person experiencing the event needs someone's help, the message may be displayed in a way that prevents the message from being lost in other message types. As in Figure 4 As seen in FIG, if the SMS application icon 414 includes a visual push notification 416 that an SMS message has been received, the message will be hidden behind the SMS application 414. The user will still have to access the message through the SMS application and may have to sift through multiple SMS messages to see the message.

[0094] Widget 412 can display one or more messages. Widget 412 can further provide a prompt to access the record of the person in need of help. As illustrated, widget 412 is displaying message 418 for the person in need of help. Message 418 may include the location of the person in need of help in a residential environment. Message 418 may also include the identity of the person in need of help. As illustrated, the person in need of help is a father and he is in the living room. Widget 412 can also present a queue icon 420, which displays multiple messages from the person in need of help. In some embodiments, due to the urgency characteristics of the person in need of help, widget 412 displays message 418 as the first message in multiple messages. Widget 412 can also be configured to rearrange the presentation order of multiple messages based on the message being a higher priority. For example, the widget can save one or more messages that are not related to the event before receiving message 418. Messages that are not related to the event can be sorted based on various parameters (such as arrival time, sender, or message type). Upon receiving message 418, widget 412 can be configured to override the sorting parameters so that message 418 is displayed at the beginning of any order. In other words, widget 412 can be configured to implement a priority sorting parameter, where message 418 is displayed as the first message in the message queue and is displayed above any other messages. As an example, refer to Figure 4, widget 412 is displaying a message from the father above the message from the mother. The message from the mother may be a message unrelated to an event, and the message from the father may be an event message. Whenever a message from both the mother and the father is received, and regardless of any other parameters, widget 412 may be configured to position the message from the father above the message from the mother based on the message from the father being an event message. Additionally, if the messages from the mother and the father are ordered in a queue, the message from the father may be ordered as the first message, regardless of the ordering parameters that order the message from the mother above the message from the father.

[0095] Widget 412 may further present a recording icon 422 for presenting a recording of a person in need of help. A person may click on the icon and listen to or view a recording of a person in need of help. Recording icon 422 may also include a location in a residential environment where the recording is made. Widget 412 may further provide an emergency medical services icon. A user may click on the emergency services icon, and user device 400 may contact emergency medical services.

[0096] Figure 6 600 includes one or more user devices 602, one or more resident devices 604, one or more networks 606, and a representative resident device 608 (the representative resident device may include one of the one or more resident devices 604). Figure 6 Each of these elements depicted in the diagram 600 may be similar to one or more elements depicted in other diagrams described herein. For example, user device 602 may be similar to any of the other user devices described herein, and so on. In some embodiments, at least some elements (e.g., and / or all elements) of diagram 600 may be in a residential environment (e.g., Figure 2 Operating within the context of a residential environment 202).

[0097] Turning to each element in further detail, the user device in user device 502 can be any suitable computing device (e.g., a mobile phone, a tablet computer, a personal computer (PC), smart glasses, a smart watch, etc.). In some embodiments, the user device may perform any one or more of the operations of the user device described herein. Depending on the type of user device and / or the location of the user device (e.g., inside or outside a residential environment), the user device may enable communication through a network 606 (e.g., including a LAN and / or WAN) using one or more network protocols (e.g., Bluetooth connection, Thread connection, ZigBee connection, infrared connection, Wi-Fi connection, etc.) and / or network paths, as further described herein. In some embodiments, the user device will be connected to another device (e.g., a TV), and the user device may provide data (e.g., notification messages, instructions, user interfaces) through the other device for presentation.

[0098] In some embodiments, the resident device 604 may correspond to any one or more of the resident devices described herein. For example, the resident device 604 may correspond to Figure 2 5. The resident devices 504 may be one or more of the resident devices of the residential environment 202 of the embodiment of the present invention. Each resident device 504 may be any suitable computing device (e.g., a mobile phone, a tablet computer, a smart speaker device, a smart media player communicatively connected to a TV, etc.). In some embodiments, the resident device may be located in a specific location (e.g., a room) of the residential environment.

[0099] In some embodiments, one or more networks 606 may include WAN (e.g., the Internet) and / or LAN. As described herein, a residential environment may be associated with a LAN, so that the devices present in the residential environment can communicate with each other through the LAN. As described herein, a WAN may be outside a residential environment. For example, a router associated with a LAN (and therefore a residential environment) may enable the traffic from the LAN to be transmitted to the WAN, and vice versa. In some embodiments, a resident device 604 may typically reside in a residential environment and communicate with other devices of the residential environment through a LAN. In some embodiments, a user device 602 may be transient. For example, as described herein, a user device may be present in a residential environment (e.g., and communicate through a LAN), and in another example, a user device may be outside a residential environment and communicate through the Internet (or other suitable networks).

[0100] As described herein, resident device 668 may represent one or more of resident devices in resident device 604. Resident device 608 has at least one memory 610, one or more processing units (or processors) 612, storage unit 614, communication interface 616, and input / output (I / O) device 618.

[0101] Turning to each element of the resident device 608 in further detail, the processor 512 may be implemented in hardware, computer executable instructions, firmware, or a combination thereof as appropriate. The computer executable instructions or firmware implementation of the processor 612 may include computer executable instructions or machine executable instructions written in any suitable programming language to perform the various functions described.

[0102] The memory 610 can store program instructions that can be loaded and executed on the processor 612, as well as data generated during the execution of these programs. Depending on the configuration and type of the resident device 608, the memory 610 can be a volatile memory (such as a random access memory (RAM)) and / or a non-volatile memory (such as a read-only memory (ROM), a flash memory, etc.). In some specific implementations, the memory 510 may include a variety of different types of memory, such as a static random access memory (SRAM), a dynamic random access memory (DRAM) or a ROM. The resident device 508 may also include an additional storage device 614, such as a removable storage device or a non-removable storage device, including but not limited to a magnetic storage device, an optical disk and / or a tape storage device. The disk drive and its associated computer-readable medium can provide a non-volatile storage device for computer-readable instructions, data structures, program modules and other data for the computing device. In some embodiments, the storage device 614 can be used to store data content received from one or more other devices (e.g., other resident devices 604 and / or user devices 602). For example, the storage device 614 can store user profiles of users associated with a residential environment. Storage device 614 may also store configuration settings, such as instructions for transmitting messages to devices associated with a residential environment.

[0103] The resident device 608 may also include a communication interface 616 that allows the resident device 608 to communicate with a stored database, another computing device or server, a user terminal, and / or other devices over the network 606. The resident device 608 may also include an I / O device 618 such as for enabling connections to a keyboard, mouse, pen, voice input device, touch input device, display, speaker, printer, etc. In some embodiments, the I / O device 618 may be used to output information related to the event message. This may include, but is not limited to, recording or mitigation steps for people who need help.

[0104] Turning to the contents of memory 610 in further detail, memory 610 may include an operating system 620 and one or more applications or services for implementing the features disclosed herein, including a communication module 622, a message analysis module 624, a device identification module 626, a broadcast module 628, and a device location analysis module 630. In some embodiments, any one or more of the applications or services resident on device 608 (or any other features of the resident device described herein) may be used to implement an event message service.

[0105] The communication module 622 may include code that causes the processor 612 to generate messages, forward messages, reformat messages, and / or otherwise communicate with other entities. For example, as described herein, the communication module 622 may transmit messages to and / or receive messages from other user devices 602 and other resident devices 604. As described herein, the communication module 622 may transmit messages via one or more transmission protocols.

[0106] The message analysis module 624 may include code that causes the processor 612 to receive and process the message. In some embodiments, one or more of the operations of the message analysis module 624 may be similar to those described herein. For example, the message analysis module 624 may also determine from the message, for example, the identity of the person who needs help, the location of the person, and other appropriate information.

[0107] The device identification module 626 may include code that causes the processor 612 to identify one or more devices in the residential environment. For example, the device identification module 626 may retrieve one or more user profiles (e.g., from the storage device 614) and identify one or more devices associated with each user profile. In another example, the device identification module 626 may further identify one or more candidate recipient devices based on device location and / or proximity data, as described below with respect to the device location analysis module 630.

[0108] Broadcast module 628 may include code that causes processor 636 to transmit a message throughout a residential environment. In some embodiments, the presentation of the message may be similar to that described in reference to Figure 1 and Figure 2 In some embodiments, the transmission of the message may depend in part on the type of recipient device (e.g., mobile phone, smart watch, smart speaker, etc.). In some embodiments, notifications and / or announcements may be presented (or blocked) based in part on settings stored on resident device 608 (and / or settings of the recipient device).

[0109] The device location analysis module 630 may include code that causes the processor 612 to determine the location of a particular device. In some embodiments, one or more of the operations of the device location analysis module 630 may be similar to the operations described in reference to FIG. Figure 1 and Figure 2 For example, the device location analysis module 630 can receive location information from one or more user devices 602 and / or resident devices 604, which is used to determine the location of each device.

[0110] Figure 7 FIG. 7 is a signaling diagram 700 for an in-home intercom and notification system according to one or more embodiments. Figure 7 As shown, the first user device 702, the first resident device 704, the second resident device 706 and the second user device 708 can interact with each other. Although the operation of processes 700, 800, 900, 1000, 1100, 1200, 1300 and 1400 is described as being performed by a computing device, it should be understood that any suitable device (e.g., user device, resident device) can be used to perform one or more operations of these processes. Processes 700, 800, 900, 1000, 1100, 1200, 1300 and 1400 (described below) are respectively illustrated as logic flow charts, and each operation of these logic flow charts represents a series of operations that can be implemented in hardware, computer instructions or their combination. In the context of computer instructions, operation represents a computer executable instruction stored on one or more computer-readable storage media, and these computer executable instructions perform the operation when executed by one or more processors. Generally speaking, computer executable instructions include routines, programs, objects, components, data structures, etc. that perform specific functions or implement specific data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and / or in parallel to implement the described processes.

[0111] At 710, the first user device 702 may detect an event. The event may be a health-related experience of a person in a residential environment. For example, a person in a kitchen may experience symptoms similar to a stroke. The first user device 702 may be a computing device located in a residential environment. In response to detecting the event, the first user device 702 may generate an event message (e.g., a health event message). The message may include the identity of a user account associated with the first user device 702, a description of the event, and the location of the first user device 702.

[0112] At 712, the first user device 702 may detect the nearest resident device. In some embodiments, the first user device 702 is connected to a network (such as a Wi-Fi network) and stores the address and identifier of each other network device it discovers. The first user device 702 may search its memory to find the last resident device discovered. In other embodiments, the first user device may transmit a signal requesting a return signal from any other device. The first user device 702 may determine the nearest resident device based on the strength of the response signal.

[0113] At 714, the first user device 702 may transmit the message to the first resident device 704. The first resident device 704 may be the device closest to the first user device 702. The first user device 702 may transmit the message via a first transmission protocol. The first transmission protocol may be a short-range transmission protocol. The first user device 702 may save battery life by transmitting the message via the first transmission protocol.

[0114] At 716, the first resident device 704 may transmit the message to the second resident device 706. In addition, the first resident device 704 may include computer readable instructions for the second resident device 706 to determine whether it can detect a person. It should be understood that the first resident device 704 may transmit the message and instructions to each of the other resident devices in the residential environment.

[0115] At 718, in some embodiments, the second resident device 706 may determine whether it can detect the user device and further detect a person. The second resident device 706 may make these determinations based on various factors. For example, the second resident device 706 may scan its memory to determine the time when it last discovered another device in the residential environment. If the time elapsed since discovery is less than the threshold time and the device is associated with a person, the second resident device 706 may determine that there is a user device. The second resident device 706 may determine the time when it last received a signal from the user device. If the time elapsed since discovery is less than the threshold time and the device is associated with a person, the second resident device 706 may determine that there is a person based on a confidence score. If the time elapsed since discovery is greater than the threshold time and the device is associated with a person, the second resident device 706 may determine that there is no person based on a confidence score.

[0116] At 720, the second resident device 706 may transmit the message to the second user device 708. At 722, the second resident device 706 may transmit a message that it detected a person and transmitted the message. At 724, the second user device 708 may present the message. For example, the second user device 708 may include a display and may display the message.

[0117] At 726, the first resident device 704 may transmit a message to the first user device 702 to present an alert signal. In some embodiments, the first user device 702 may project an audio or visual alert signal in response to the message.

[0118] Figure 8 800 is a process flow for an in-home intercom and notification system according to one or more embodiments. At 802, a computing device may receive an event message (e.g., a health event message) from a user device. The computing device may be, for example, a resident device that communicates with other devices in a residential environment (e.g., Figure 1 The first resident device 106 or Figure 2 The message may describe an event experienced by someone in a residential environment. The message may include the identity of the person experiencing the event and the location of the user device.

[0119] At 804, the computing device may transmit the message to other computing devices in the residential environment via the second transmission protocol. The other computing devices may also be resident devices located at various locations in the residential environment (e.g., Figure 1 The second resident device 110 and the third resident device 112 and Figure 2 In some embodiments, the computing device further transmits instructions to each of the other computing devices to determine whether they can detect a person.

[0120] The computing device may receive a response from at least one of the other computing devices at 806. The response may include a notification that the event message has been transmitted to the user device.

[0121] At 808, the computing device may transmit a message that at least one computing device detects a person to a user device associated with the person experiencing the event. In some embodiments, the message may include instructions for the user device to present an alert signal to help people find the person experiencing the event.

[0122] Fig. 9 900 is a process flow for an in-home intercom and notification system according to one or more embodiments. At 902, a computing device may detect that a person is experiencing an event. The computing device may be a first user device (e.g., Figure 1 The first user equipment 104 or Figure 2The computing device may be equipped with sensors, such as biometric sensors, audio sensors, image capture sensors. Based on the data collected by the sensors, the computing device may determine that a person is experiencing an event. The person may be a user with a user account associated with the computing device. The computing device may further determine the characteristics of the event. For example, if the person is sleeping and breathing slows, the computing device may determine that the person may be experiencing sleep apnea.

[0123] At 904, in response to detecting the event, the computing device may create an event message (eg, a health event message). The message may include the identity of the person associated with the user account, a description of the event, and the location of the computing device in the residential environment.

[0124] At 906, the computing device may determine whether it can detect a receiving device within a short range. The search for devices within a short range may be based on determining that an event has occurred. In other words, an indication that a person may be injured may trigger a search for a device that can notify another person within a shorter distance to provide assistance. A short range may be a range within which a computing device can communicate with a receiving device via a short-range transmission protocol. A receiving device may be any device connected to a network of a residential environment. For example, a receiving computing device may be a resident device in a residential environment (e.g., Figure 1 The first resident device 116 or Figure 2 The computing device may use various techniques to determine whether it can detect the receiving device. For example, the computing device may send a signal via a first transmission protocol such as a short-range transmission protocol (e.g., Bluetooth). The computing device may determine the distance between the computing device and the receiving device based on the strength of the signal response. The computing device may also include the identity of each computing device it has discovered in the residential environment. For example, if the computing device is a smart watch worn by a person, the smart watch may store the identity of each other computing device it discovers while the person walks around the residential environment. The computing device may determine that the last detected device is the closest device.

[0125] If the computing device detects a receiving device, process 900 proceeds to step 908. At 908, the computing device may transmit the message to the receiving device via a first transmission protocol. The first transmission protocol may be a short-range transmission protocol, such as Bluetooth. The computing device may save power by transmitting the message via the first transmission protocol instead of a second transmission protocol, such as Wi-Fi or cellular.

[0126] At 910, the computing device may receive a notification that a user device associated with a second person has been detected. The notification may be received from a receiving device. The notification may also include instructions to present an alert signal such as an audio, visual, or vibration signal. At 912, the computing device may activate a device alert system and present an alert signal.

[0127] If the computing device does not detect a receiving device at 906, the computing device may contact emergency medical services at 914. The computing device may also activate an alarm system at 912 and present an alarm signal.

[0128] Fig.10 1000 is a process flow for an in-home intercom and notification system according to one or more embodiments. At 1002, a computing device may receive an event message (e.g., a health event message). The computing device may be a resident device in a residential environment (e.g., Figure 1 The second resident device 110 or Figure 2 from another resident device (e.g., Figure 1 The first resident device 116 or Figure 2 The message may include instructions for determining whether a person can be detected.

[0129] At 1004, the computing device may determine whether a person is detectable. The determination may be based on whether the computing device detects a user device and whether the data indicates that the possibility of the person being present is greater than the possibility of the person not being present. The computing device may initiate a search for the user device based on determining that the received message is an event message. Therefore, someone may need help. For example, the computing device may detect an identifier indicating that the message is an event message. Based on the detection, the computing service may perform a search for the user device. The computing device may use various techniques to detect whether a user device is present. In instances where the computing device is connected to a network, the computing device may discover other devices. In each instance where the computing device discovers another computing device, the computing device may store the discovery in its local memory. Therefore, the computing device may scan its local memory and determine whether it has discovered another device.

[0130] If the computing device detects another user device at 1004, process 1000 proceeds to 1006. At 1006, the computing device transmits the event message to the other user device. The event message may be sent using a transmission protocol different from the transmission protocol for receiving the event message of the device of the person in need of help. For example, if the transmission protocol for receiving the message may be Bluetooth. Bluetooth may be advantageous because it consumes less power than, for example, Wi-Fi. The device of the person in need of help may be battery-powered, and may need to save as much power as possible in order to be able to continuously transmit event messages. On the other hand, the computing device may draw power from an electrical outlet, and therefore be able to use a more power-consuming transmission protocol. Compared with the transmission protocol (e.g., Bluetooth) for receiving messages, a more power-consuming protocol may allow the computing device to detect the user device at a greater distance. The computing device may switch to a more power-consuming transmission protocol (e.g., Wi-Fi) based on determining that the message is an event message. For example, the computing device may detect an identifier indicating that the message is an event message. Based on this detection, the computing device may select a more power-consuming transmission protocol (e.g., Wi-Fi).

[0131] At 1008, the computing device may transmit a message back to the other resident device that the user device has been detected and the message has been transmitted. The message may be transmitted using a transmission protocol (eg, Bluetooth) used to receive the message.

[0132] If the computing device does not detect the user device at 1004, the computing device may transmit a message that no person was detected back to another computing device at 1010. This may occur if the computing device was recently introduced to the network and there is no traffic passing through the device.

[0133] Fig.11 1100 is a process flow for an in-home intercom and notification system according to one or more embodiments. At 1102, a computing device may detect a device near a person experiencing an event. For example, the computing device may transmit a signal via a first transmission protocol and then determine a distance based on the strength of the signal response. If the determined distance is less than a threshold distance, the computing device may determine that the device is near the person experiencing the event. The computing device may be, for example, a first resident device (e.g., Figure 1 The first resident device 116 or Figure 2 The device may be, for example, another resident device (e.g., Figure 2 The fourth resident device 224).

[0134] At 1104, the computing device may communicate with the device and determine whether it has audio or visual capabilities. If the computing device determines at 1104 that the device does have audio or visual capabilities, process 1100 proceeds to 1106. At 1106, the computing device may determine whether the device is turned on. Specifically, whether the device is in sleep mode or presentation mode.

[0135] If the computing device determines that the device is in presentation mode, process 1100 proceeds to 1108. At 1108, the computing device may transmit computer-readable instructions to the device to present event-related mitigation information. For example, if a person has fallen, the device may present a video or audio recording describing how to help someone stand up without hurting them. However, if the computing device determines that the device is in sleep mode, the computing device may send computer-readable instructions at 1110 to turn the device to presentation mode. The computing device may then transmit computer-readable instructions to the device to present event-related mitigation information. However, if the computing device determines at 1104 that the device does not have audio or visual capabilities, process 1000 ends at 1112.

[0136] Fig.12 1200 is a process flow for an in-home intercom and notification system according to one or more embodiments. At 1202, a computing device may receive an event message (e.g., a health event message). The event message may include an identifier indicating that the message is an event message. The computing device may be, for example, a first resident device (e.g., Figure 1 The first resident device 116 or Figure 2 In some embodiments, the computing device is integrated with a virtual assistant. Thus, at 1204, the computing device may transmit computer readable instructions to the virtual assistant to contact the person experiencing the event and ask if they need help. For example, Figure 1 As can be seen in FIG. 1 , the virtual assistant asks the first person 102 “Do you need help?”

[0137] At 1206, the computing device may receive a response from the virtual assistant regarding whether the person needs help. In addition to the response, the computing device may also receive a record of the person needing help. The record may be, for example, an audio record or a visual record.

[0138] At 1208, the computing device may determine whether the person needs help. In some embodiments, the virtual assistant has access to software that interprets the person's response as a determination of whether the person needs help. If the computing device determines at 1208 that the person needs help based on the virtual assistant's analysis, process 1200 proceeds to 1210.

[0139] At 1210, the computing device may append a message to include the record. However, if the computing device determines based on the virtual assistant's analysis that the person does not need help, process 1200 ends at 1212.

[0140] Fig.13 1300 is a process flow for an in-home intercom and notification system according to one or more embodiments. At 1302, a computing device may detect that a person is experiencing an event. The computing device may be a first user device (e.g., Figure 1 The first user equipment 104 or Figure 2 The computing device may be equipped with sensors, such as biometric sensors, audio sensors, image capture sensors. Based on the data collected by the sensors, the computing device may determine that the person is experiencing an event.

[0141] At 1304, the computing device may further detect the severity of the event. For example, the computing device may receive signal data from one or more sensors. The computing device may further access a mapping between sensor-based signals and a severity table. The computing device may perform a lookup on the table and determine the severity of the event. If the event is not a severe event, the computing device may perform a lookup as described in the example above. Figure 8 For example, process 1300 may proceed to Figure 8 804. For purposes of illustrating this diagram, assume that the device detects a serious event. At this point, the computing device contacts emergency services (e.g., law enforcement, healthcare professionals, hospitals) at 1306. The computing device may contact emergency services, for example, via an Internet-based service. After or in parallel with contacting emergency services, the computing device determines at 1308 whether it can detect the receiving device.

[0142] If the computing device cannot detect the receiving device, the computing device may initiate a notification of a device alert system of the computing device at 1312. The notification may also include instructions to present an alert signal such as an audio, visual, or vibration signal.

[0143] However, if the computing device does detect a receiving device (e.g., a user device), process 1300 proceeds to 1310. At 1310, the computing device may create an event message (e.g., a health event message). The message may include the identity of a person associated with the user account, a description of the event, and the location of the computing device in the residential environment. The computing device may transmit the message to the receiving device via a first transmission protocol.

[0144] The receiving device may or may not detect another person in the residential environment. However, considering the severity of the event, the computing device proceeds to 1312 regardless of whether the receiving device detects another person. At 1312, 1312, the computing device may initiate a notification of the device alarm system of the computing device.

[0145] Fig.14 It is a process flow for detecting a device through an intercom and notification system in a residence according to one or more embodiments. At 1402, a computing device may receive an event message (e.g., a health event message). The computing device may be a resident device in an environment (e.g., a residence or an office). The event message may include a record of a first person requesting help or include a record of the surrounding environment of the computing device in an instance where the person is unable to communicate. The message may also include the location of the source of the message. In addition to the record and location, the message may also include the identity of the first person, the location of the first person in the residential environment. The event message may also include instructions for determining whether any other person (e.g., a second person, a third person, a fourth person) can be detected.

[0146] At 1404, the computing device may determine whether any other person may be detected. The determination may be based on whether the computing device detects the user device and whether the data indicates that a person is more likely to be present than not to be present based on detecting the user device.

[0147] The computing device may use various techniques to detect the presence of other user devices in an environment (e.g., a home or office). In instances where the computing device is connected to a network, the computing device may discover other devices connected to the network. In each instance where the computing device discovers another computing device, the computing device may store the discovery in its local memory. Thus, the computing device may scan its local memory and identify each discovered device. The computing device may further determine whether a person may be present based on detecting the user device. For example, the computing device.

[0148] In addition to detecting user devices connected to the network, the computing device can also detect user devices based on association with a smart home platform (e.g., Apple HomeKit). The smart home platform can be operated as an application on the computing device. The smart home platform can include multiple accounts, and the computing device can use the smart home platform to identify user devices associated with each account.

[0149] If the computing device detects other user devices at 1404, the process may proceed to 1406. At 1406, the computing device may transmit the event message to each identified device. The computing device may also send the identification of each other identified device with the event message. It should be understood that if the computing device detects only one other user device, the computing device does not include the identity of the user device.

[0150] The computing device may transmit the event message and other device identifications to each identified user device. Thus, each device receives the event message and the identity of each other device that received the event message.

[0151] The computing device may send the event message using a different transport protocol than the transport protocol used to receive the event message of the device of the person in need of help. The computing device may use various more power-consuming transport protocols (e.g., cellular, Wi-Fi, Bluetooth, Internet) based on determining that the message is an event message. The computing device may use different transport protocols for different devices. For example, for devices in the same environment as the computing device, a local network may be used to transmit the event message. In the case where the user device is outside the environment, the computing device may use a remote transport protocol such as cellular or the Internet.

[0152] If the computing device does not detect a user device at 1404, the process may proceed to step 1408. At 1408, the computing device may transmit a message back to another computing device (eg, a user device of a person in need of assistance) that no person was detected at 1404.

[0153] In some embodiments, the computing device may further directly provide healthcare related information or cause another device to provide the information. For example, the computing device may communicate with another device such as a smart TV, a media streaming device, or other smart speaker. The computing device may directly or indirectly provide healthcare related information related to the person. The information may be configured to provide assistance to a person. For example, the information may be user specific, such as a medication to which the person is allergic. The information may be situation specific, such as CPR techniques in an instance where the event is, for example, choking.

[0154] Although specific embodiments have been described, those skilled in the art will recognize that numerous modifications are possible. A single controller can use the processes described herein to establish pairings with any number of accessories and selectively communicate with different accessories at different times. Similarly, a single accessory can be controlled by multiple controllers with which pairings have been established. Any function of an accessory can be controlled by modeling the function as a service with one or more characteristics and allowing the controller to interact with the service and / or the characteristics of the service (e.g., read, modify, receive updates). Therefore, regardless of the accessory function or the controller form factor or specific interface, the protocols and communication processes described herein can be "universal", meaning that they can be applied in any scenario with one or more controllers and one or more accessories.

[0155] Therefore, although specific embodiments have been described, it should be understood that the embodiments may include all modifications and equivalents within the scope of the following claims.

[0156] As described above, one aspect of the present technology is to collect and use data available from specific and legitimate sources to improve message delivery from one device to one or more devices (e.g., delivering messages for residential health intercom). The present disclosure contemplates that, in some instances, the collected data may include personal information data that uniquely identifies or can be used to identify a specific person. Such personal information data may include demographic data, location-based data, online identifiers, phone numbers, email addresses, residential addresses, data or records related to the user's health (e.g., vital sign measurements, medication information), date of birth, or any other personal information.

[0157] The present disclosure recognizes that the use of such personal information data in the present technology can be used to benefit the user. For example, the personal information data can be used to deliver commands from a user profile on a computing device to one or more computing devices. In addition, the present disclosure also contemplates other uses of the personal information data that benefit the user. For example, a specific state of a device associated with the user (e.g., a healthcare-related device, a wearable device, etc.) can be transmitted from the device back to the user profile.

[0158] The present disclosure envisions that entities responsible for collecting, analyzing, disclosing, transmitting, storing or otherwise using such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities will be expected to implement and consistently apply privacy practices that are generally recognized as meeting or exceeding the requirements of the industry or government that maintain user privacy. Such information about the use of personal data should be highlighted and easily accessible to users, and should be updated as the collection and / or use of data changes. The user's personal information should be collected only for legal use. In addition, such collection / sharing should only occur after receiving user consent or other legal basis specified in applicable law. In addition, such entities should consider taking any necessary steps to protect and safeguard access to such personal information data and ensure that other entities with access to personal information data comply with the privacy policies and procedures of other entities. In addition, such entities may subject themselves to third-party assessments to prove that they comply with widely accepted privacy policies and practices. In addition, policies and practices should be adjusted for specific types of personal information data collected and / or accessed, and applied to applicable laws and standards, including considerations that are exclusive to jurisdictions that can be used to impose higher standards. For example, in the United States, collection or access to certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while health data in other countries may be subject to other regulations and policies and should be handled accordingly.

[0159] Notwithstanding the foregoing, the present disclosure also contemplates implementations in which users selectively block the use or access of personal information data. That is, the present disclosure contemplates providing hardware elements and / or software elements to prevent or block access to such personal information data. In addition to providing "opt-in" and "opt-out" options, the present disclosure also contemplates providing notifications related to access or use of personal information. For example, a user may be notified that the user's personal information data will be accessed when downloading an application, and then the user may be reminded again just before the personal information data is accessed by the application.

[0160] In addition, it is the intention of the present disclosure that personal information data should be managed and processed in a manner that minimizes the risk of unintentional or unauthorized access or use. Once the data is no longer needed, the risk can be minimized by limiting the collection of data and deleting the data. In addition, and when applicable, including in certain healthcare-related applications, data de-identification can be used to protect the privacy of users. De-identification can be promoted by removing identifiers, controlling the amount or specificity of stored data, controlling the storage method of data (e.g., pooling data among users), and / or other methods (such as differential privacy) when appropriate.

[0161] Thus, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also contemplates that various embodiments may also be implemented without access to such personal information data. That is, various embodiments of the present technology will not fail to function properly due to the lack of all or a portion of such personal information data. For example, content may be selected and delivered to a user based on aggregated non-personal information data or an absolute minimum amount of personal information, such as content that is processed only on the user's device or other non-personal information that may be used for content delivery services.

[0162] Although specific embodiments have been described, various modifications, changes, alternative constructions, and equivalents are also encompassed within the scope of this disclosure. Embodiments are not limited to operating within certain specific data processing environments, but are free to operate within multiple data processing environments. Additionally, although embodiments have been described using a specific series of transactions and steps, it should be apparent to those skilled in the art that the scope of this disclosure is not limited to the described series of transactions and steps. The various features and aspects of the above embodiments may be used alone or in combination.

[0163] In addition, although embodiments have been described using specific combinations of hardware and software, it should be appreciated that other combinations of hardware and software are also within the scope of the present disclosure. Embodiments may be implemented in hardware only, or in software only, or using a combination thereof. The various processes described herein may be implemented on the same processor or on different processors in any combination. Therefore, where a component or module is described as being configured to perform certain operations, such configuration may be implemented, for example, by designing an electronic circuit to perform the operation, programming a programmable electronic circuit (such as a microprocessor) to perform the operation, or any combination thereof. Each process may communicate using various techniques, including but not limited to conventional techniques for inter-process communication, and different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0164] Accordingly, the description and drawings should be regarded as illustrative rather than restrictive. However, it is apparent that additions, subtractions, deletions and other modifications and changes may be made thereto without departing from the broader spirit and scope set forth in the claims. Therefore, although specific disclosed embodiments have been described, these embodiments are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims.

[0165] The use of the terms "one" and "an" and "the" and similar indicators in the context of describing the disclosed embodiments (especially in the context of the following claims) will be interpreted as covering the singular and plural, unless otherwise stated or clearly contradicted by the context. Unless otherwise stated, the terms "comprising", "having", "including" and "containing" should be interpreted as open terms (i.e., meaning "including but not limited to"). The term "connected" is interpreted as being partially or completely included, attached or joined together even if there are intervening things. Unless otherwise stated herein, the description of the numerical range herein is only intended to be used as a simple method of individually referring to each individual value falling within the range, and each individual value is incorporated into the specification as if quoted separately in this article. All methods described herein can be performed in any suitable order, unless otherwise stated herein or otherwise clearly contradicted by the context. Unless otherwise stated, the use of any and all examples or exemplary languages ​​(e.g., "such as") provided herein is only intended to better illustrate the embodiments and will not limit the scope of the present disclosure. Any language in the specification should not be interpreted as indicating that any undeclared element is essential to the practice of the present disclosure.

[0166] Unless specifically stated otherwise, disjunctive language such as the phrase "at least one of X, Y, or Z" is intended to be understood in context as generally used to present an item, term, etc., which may be X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Thus, such disjunctive language is generally not intended to, and should not, imply that certain embodiments require that at least one of X, at least one of Y, or at least one of Z each be present.

[0167] Preferred embodiments of the present disclosure are described herein, including the best mode known for carrying out the present disclosure. After reading the foregoing description, variations of those preferred embodiments may become apparent to those of ordinary skill in the art. Those of ordinary skill in the art should be able to appropriately adopt such variations, and may practice the present disclosure in a manner different from that specifically described herein. Therefore, as permitted by applicable law, the present disclosure includes all modifications and equivalents of the subject matter recorded in the claims attached to the present disclosure. In addition, unless otherwise specified herein, the present disclosure encompasses any combination of all possible variations of the above-mentioned elements.

[0168] All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.

[0169] In the foregoing description, various aspects of the disclosure are described with reference to specific embodiments of the disclosure, but those skilled in the art will recognize that the disclosure is not limited thereto. Each feature and aspect of the above disclosure may be used alone or in combination. In addition, without departing from the broader spirit and scope of the present specification, embodiments may be utilized in any number of environments and applications other than those described herein. Therefore, the description and accompanying drawings should be considered as illustrative rather than restrictive.

Claims

1. A computer-implemented method, the method comprising: receiving, by the first computing device from the first user device and via the first transmission protocol, a message including a description of an event for a first person in the environment; detecting, by the first computing device, that the message is an event message; transmitting, by the first computing device, the event message to a second computing device in the environment using a second transmission protocol, the second transmission protocol being used based at least in part on detecting that the message is the event message, the first computing device being in operable communication with the second computing device using a local area network; as well as A response is received, by the first computing device and from the second computing device using the second transmission protocol, indicating that the second computing device has transmitted the event message to a second user device.

2. The computer-implemented method of claim 1 , wherein the method further comprises: In response to receiving the event message, launching a virtual assistant to ask the first person whether he needs help; receiving a message from the virtual assistant including a response as to whether the first person needs assistance; as well as The event message is transmitted to the second computing device based at least in part on the response as to whether the first person needs assistance.

3. A computer-implemented method according to any one of claims 1 and 2, wherein the event message also includes an identity of a user account associated with the first user device and a location of the first user device in the environment.

4. A computer-implemented method according to any one of claims 1 to 3, wherein the response from the second computing device includes a confidence score calculated by the second computing device, and wherein the confidence score describes the probability that a second person is present near the second user device. 5 . The computer-implemented method of claim 1 , wherein detecting that the message is the event message comprises detecting an event message identifier. 6 .

6. The computer-implemented method of any one of claims 1 to 5, wherein the method further comprises: Detecting the second computing device and a third computing device in the environment; wherein transmitting the event message to the second computing device comprises transmitting the event message and identities of the second computing device and the third computing device to the second computing device and the third computing device.

7. The computer-implemented method of any one of claims 1 to 6, wherein the method further comprises: detecting a fourth computing device based at least in part on an account of a smart home platform, wherein the fourth computing device is outside of the environment; and Wherein transmitting the event message to the second computing device comprises transmitting the event message and the identity of the second computing device and the identity of the fourth computing device to the second computing device and the fourth computing device.

8. The computer-implemented method of any one of claims 1 to 7, wherein the first transmission protocol comprises a short-range wireless technology.

9. The computer-implemented method of any one of claims 1 to 8, wherein the second transmission protocol comprises an Internet-based transmission technology.

10. A computing device, comprising: a memory configured to store computer-executable instructions; and A processor configured to access the memory and execute the computer executable instructions to perform an element according to any one of claims 1 to 9.

11. A non-transitory computer readable medium having stored thereon sequences of instructions which, when executed by a processor, cause the processor to perform an element according to any one of claims 1 to 9.

12. A first computing device, the first computing device comprising: a memory configured to store computer-executable instructions; and A processor configured to access the memory and execute the computer executable instructions to at least perform the following operations: receiving, from a first user device and via a first transmission protocol, an event message including a description of an event for a first person in an environment; transmitting the event message to a second computing device in the environment via a second transmission protocol, the first computing device and the second computing device being in operable communication via a local area network; as well as A response is received from the second computing device via the second transmission protocol that the second computing device has transmitted the event message to a second user device.

13. The first computing device of claim 12, wherein the instructions further cause the processor to: In response to receiving the event message, launching a virtual assistant to ask the first person whether he needs help; receiving a message from the virtual assistant including a response as to whether the first person needs help; as well as receiving a message from the virtual assistant including a response as to whether the first person needs assistance; as well as The event message is transmitted to the second computing device based at least in part on the response as to whether the first person needs assistance.

14. The first computing device of any one of claims 12 and 13, wherein the event message further includes an identity of a user account associated with the first user device and a location of the first user device in the environment.

15. A first computing device according to any one of claims 12 to 14, wherein the response from the second computing device includes a confidence score calculated by the second computing device, and wherein the confidence score describes the probability that a second person is present near the second user device.

16. The first computing device of any one of claims 12 to 15, wherein the environment is a residential environment or an office environment.

17. The first computing device of any one of claims 12 to 16, wherein the first transmission protocol comprises a short-range wireless technology.

18. The first computing device according to any one of claims 12 to 17, wherein the second transmission protocol comprises an Internet-based transmission technology.

19. A computer implemented method comprising executing one or more elements of any one of claims 12 to 18.

20. A non-transitory computer readable medium having stored thereon sequences of instructions which, when executed by a processor, cause the processor to perform an element according to any one of claims 12 to 18.

21. A non-transitory computer readable medium having stored thereon sequences of instructions, which, when executed by a processor, cause the processor to perform operations comprising: receiving, from a first user device and via a first transmission protocol, an event message including a description of an event for a first person in an environment; transmitting the event message to a second computing device in the environment via a second transmission protocol, the first computing device being in operable communication with the second computing device via a local area network; and A response is received from the second computing device via the second transmission protocol indicating that the second computing device has determined a probability that a second person is in the environment and has transmitted the event message to a second user device associated with the second person.

22. The non-transitory computer readable medium of claim 21, wherein the instructions further cause the processor to: In response to receiving the event message, launching a virtual assistant to ask the first person whether he needs help; receiving a message from the virtual assistant including a response as to whether the first person needs help; as well as receiving a message from the virtual assistant including a response as to whether the first person needs assistance; as well as The event message is transmitted to the second computing device based at least in part on the response as to whether the first person needs assistance.

23. A non-transitory computer-readable medium according to any one of claims 21 and 22, wherein the response from the second computing device includes a confidence score calculated by the second computing device, and wherein the confidence score describes the probability that the second person is present near the second user device.

24. A non-transitory computer-readable medium according to any one of claims 21 to 23, wherein the response from the second computing device includes a confidence score calculated by the second computing device, and wherein the confidence score describes the probability that the second person is present near the second user device.

25. The non-transitory computer readable medium of any one of claims 21 to 24, wherein the environment is a residential environment or an office environment.

26. The non-transitory computer-readable medium of any one of claims 21 to 25, wherein the first transmission protocol comprises a short-range wireless technology.

27. A computing device, the computing device comprising: a memory configured to store computer-executable instructions; and A processor configured to access the memory and execute the computer executable instructions to perform an element according to any one of claims 21 to 26.

28. A computer implemented method comprising executing one or more elements of any one of claims 21 to 26.

29. A computer-implemented method, the method comprising: receiving, by the first computing device from the first user device and via the first transmission protocol, an event message including a description of an event for a first person in the environment; transmitting, by the first computing device via a second transmission protocol, the event message to a second user device in the environment; as well as A response is received, by the first computing device, from the second user device indicating that the second user device has presented the event message.

30. The computer-implemented method of claim 29, wherein the method further comprises: determining a severity of the event based on the event message; as well as Emergency services are contacted based at least in part on the severity being above a severity threshold.

31. The computer-implemented method of any one of claims 29 and 30, wherein determining the severity of the event comprises: receiving sensor-based data from the first user device; as well as Performs a lookup into a table that maps sensor-based data to event severity.

32. The computer-implemented method of any one of claims 29 to 31, wherein the method further comprises presenting event mitigation information in response to determining the severity of the event.

33. The computer-implemented method of any one of claims 29 to 32, wherein the method further comprises presenting an alert signal in response to determining the severity of the event.

34. A computer-implemented method according to any one of claims 29 to 33, wherein the event message also includes an identity of a user account associated with the first user device and a location of the first user device in the environment.

35. The computer-implemented method of any one of claims 29 to 34, wherein the first transmission protocol comprises a short-range wireless technology.

36. A computer system, the computer system comprising: a memory configured to store computer-executable instructions; and A processor configured to access the memory and execute the computer executable instructions to perform an element according to any one of claims 29 to 35.

37. A non-transitory computer readable medium having stored thereon sequences of instructions which, when executed by a processor, cause the processor to perform an element according to any one of claims 29 to 35.

Images