Technology for tracking objects within a defined area

Abstract

To provide technologies for tracking objects within various defined areas.SOLUTION: The technologies are for tracking various objects (e.g., mammals, animals, humans, pets) within various defined areas (e.g., rooms, apartments, residences, vehicles, tents) to determine whether those objects satisfy or do not satisfy various criteria, signatures or thresholds, which may relate to health, safety or security of those objects within those defined areas. Tracking the objects in the defined areas may be enabled via various radars (e.g., time-of-flight radars, Doppler radars) positioned within the defined areas. Some of such radars may operate in a Ku-band between about 12 GHz and about 18 GHz inclusive, a K-band between about 18 GHz and about 27 GHz inclusive, or a Ka-band between about 26.5 GHz and about 40 GHz inclusive, where each of the bands tracks those objects within the defined areas.SELECTED DRAWING: Figure 1

Description

Technical Field

[0004] , , ,

[0005] , ,

[0001] (Cross - reference to related patent applications) This patent application claims the benefit of priority of U.S. Provisional Patent Application 63 / 153,795, filed on February 25, 2021, which is incorporated herein by reference for all purposes.

[0002] This patent application claims the benefit of priority of U.S. Provisional Patent Application No. 63 / 162,476, filed on March 17, 2021, which is incorporated herein by reference for all purposes.

[0003] (Technical Field) This disclosure relates to tracking an object within a defined area.

Background Art

[0004] A first person (e.g., caregiver, doctor, family member, social worker, home caregiver) may wish to track a second person (e.g., care recipient, patient) within a defined area (e.g., room, apartment) to confirm that the second person is safe, healthy, or secure within the defined area. However, doing so can be technically problematic for various reasons. For example, the second person may wish to maintain privacy regarding such tracking or ensure that such tracking is secure. Similarly, whatever technology (e.g., video camera, proximity sensor) the first person decides to use for such tracking may have various technical drawbacks (e.g., coverage gaps, insufficient accuracy).

Summary of the Invention

[0005] This disclosure enables various techniques for tracking various objects (e.g., animals, humans, pets) within various defined areas (e.g., rooms, apartments, houses, offices, tents, barracks, vehicles, aircraft, spacecraft, clinics, field clinics, hospitals, field hospitals) to determine whether these objects meet or fail to meet various criteria, signatures, or thresholds that may relate to the health, safety, or security of these objects within these defined areas or the environment in which they reside. These techniques can track objects within defined areas through various radars (e.g., time-of-flight radar, Doppler radar) positioned within the defined areas. For example, some of such radars can operate in the Ku band with these values ​​between approximately 12 GHz and approximately 18 GHz, the K band with these values ​​between approximately 18 GHz and approximately 27 GHz, or the Ka band with these values ​​between approximately 26.5 GHz and approximately 40 GHz, each of which has unexpectedly been found to be technically beneficial for tracking these objects within defined areas.

[0006] One embodiment may include a method for providing a device to a user, wherein the device comprises a processor and a time-of-flight radar, the processor being coupled to the time-of-flight radar and the time-of-flight radar being configured to operate in the K-band; and positioning the device within a defined area having living objects therein, activating the time-of-flight radar to operate in the K-band within the defined area, the time-of-flight radar operating in the K-band within the defined area tracking living objects within the defined area, generating a set of data based on tracking living objects within the defined area, sending the set of data to the processor, and instructing the user to cause the processor to determine, based on the set of data, whether an object is experiencing an event within the defined area, and to take action in response to any event determined to be occurring within the defined area.

[0007] One embodiment may include a method comprising the steps of: a processor receiving a set of data from a time-of-flight radar operating in the K-band within a defined area having living objects therein; the time-of-flight radar generating a set of data based on the K-band operation within the defined area and tracking living objects within the defined area; the processor determining, based on the set of data, whether an object is experiencing an event within the defined area; and the processor taking action in response to an event determined to be occurring within the defined area.

[0008] One embodiment of the system may include a device comprising a processor and a time-of-flight radar, wherein the processor is coupled to the time-of-flight radar, the time-of-flight radar is configured to operate in the K-band, the device is configured to be located within a defined area having living objects therein, the time-of-flight radar operating in the K-band within the defined area tracks living objects within the defined area, generates a set of data based on tracking living objects within the defined area, transmits the set of data to the processor, the processor determines based on the set of data whether an object is experiencing an event within the defined area, and takes action in response to an event determined to be occurring within the defined area.

[0009] One embodiment may include a method that provides a device to a user, the device comprising a processor and a radar, the processor being coupled to the radar; positioning the device within a defined area having an object thereon; activating the radar to operate within the defined area so that the radar operating within the defined area tracks an object within the defined area, generates a set of data based on the tracking of the object within the defined area, transmits the set of data to the processor, and instructs the user to allow the processor to determine whether or not to take an action based on the set of data, and to take an action based on the set of data.

[0010] One embodiment may include a method comprising: a processor receiving a set of data from a radar operating within a defined area having living objects therein, and generating a set of data based on the radar operating within the defined area and tracking objects within the defined area; a processor determining whether an action should be taken based on the set of data; and a processor taking an action based on the set of data.

[0011] One embodiment of the system may include a device comprising a processor and a radar, wherein the processor is coupled to the radar, and the device is configured to be located within a defined area having living objects therein, and the radar within the defined area tracks objects within the defined area, generates a set of data based on the tracking of objects within the defined area, transmits the set of data to the processor, which then determines whether or not to take action based on the set of data, and takes action based on the set of data.

[0012] One embodiment may include a method that positions a device within a defined area having living objects therein, wherein the device includes a processor and a radar, the processor being coupled to the radar; and activates the radar to operate within the defined area, the radar operating within the defined area tracking objects within the defined area, generating a set of data based on tracking objects within the defined area, and transmitting the set of data to the processor so that the processor determines whether or not to take action based on the set of data, and takes action based on the set of data.

[0013] One embodiment of the device may include a vehicle comprising a processor, radar, and a zone, wherein the processor is coupled to the radar, the zone is configured to include a driver or passenger, and the processor is programmed to activate the radar to track a driver or passenger within the zone, generate a set of data based on the tracking of the driver or passenger within the zone, transmit the set of data to the processor, determine whether or not to take action based on the set of data, and take action based on the set of data. [Brief explanation of the drawing]

[0014] [Figure 1] This is a top view of one embodiment of a region including a device including radar according to the present disclosure. [Figure 2] This is a side view of Figure 1 in this disclosure. [Figure 3] This figure shows one embodiment of a device including radar according to the present disclosure. [Figure 4] This figure shows a set of embodiments of a set of devices, each containing a radar or sensor, according to the present disclosure. [Figure 5] This is a logic diagram of one embodiment of a device including radar as disclosed herein. [Figure 6] This is a schematic internal diagram of Figure 3 in this disclosure. [Figure 7] This is a logical diagram of the radar as disclosed in this document. [Figure 8] This disclosure shows a photograph of the internal cavity of Figure 3, based on Figure 6. [Figure 9] This figure shows a set of embodiments of the set of form factors that embody the radar according to this disclosure. [Figure 10] This figure shows one embodiment of a circuit board equipped with a set of radar antennas according to the present disclosure. [Figure 11] This figure shows one embodiment of the coverage range of the device shown in Figure 3 according to this disclosure. [Figure 12]This figure shows one embodiment of the microphone set of the device shown in Figure 7 according to this disclosure. [Figure 13] This figure shows one embodiment of the microphone of the device shown in Figure 7 according to this disclosure. [Figure 14] Figures 1 to 7 show raw readings from the devices and an embodiment of a virtual skeleton formed by the devices from the raw readings according to this disclosure. [Modes for carrying out the invention]

[0015] This disclosure enables various techniques for tracking various objects (e.g., mammals, animals, humans, pets) within various defined areas (e.g., rooms, apartments, houses, vehicles, tents) and determining whether these objects meet or do not meet various criteria, signatures, or thresholds that may relate to the health, safety, or security of objects within these defined areas. These techniques can track objects within defined areas through various radars (e.g., time-of-flight radar, Doppler radar) positioned within the defined areas. For example, some of these radars can operate in the Ku band with these values ​​between approximately 12 GHz and approximately 18 GHz, the K band with these values ​​between approximately 18 GHz and approximately 27 GHz, or the Ka band with these values ​​between approximately 26.5 GHz and approximately 40 GHz, each of which has been unexpectedly found to be technically beneficial for tracking these objects within defined areas, as will be further described below.

[0016] Next, the present disclosure will be described more fully with reference to all of the accompanying drawings, which illustrate several embodiments of the present disclosure. However, the present disclosure can be embodied in many different forms and should not necessarily be construed as being limited to the various embodiments disclosed herein. Rather, these embodiments are provided to ensure that the present disclosure is thorough and complete and to fully convey the various concepts of the present disclosure to those skilled in the art. Note that the same or similar numbering may refer to the same or similar elements throughout.

[0017] The various terms used in this book can mean direct or indirect, complete or partial, temporary or permanent, acts or omissions. For example, when an element is said to be "above", "connected" or "coupled" to another element, that element can be directly above, connected or coupled to the other element, or there can be intervening elements including indirect or direct deformation forms. In contrast, when an element is called "directly connected" or "directly coupled" to another element, there are no intervening elements.

[0018] In this specification, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X uses A or B" is intended to mean any of the natural inclusive arrangements. That is, if X uses A, if X uses B, or if X uses both A and B, in any of these cases, "X uses A or B" is satisfied. For example, "X includes A or B" means, unless otherwise specified or clear from the context, "X can include A", "X can include B", "X can include A and B".

[0019] In this specification, each of the singular terms "a", "an" and "the" is intended to include the plural (e.g., 2, 3, 4, 5, 6, 7, 8, 9, 10, 10, 100, hundreds, thousands, millions), including intermediate integer or decimal forms (e.g., 0.0, 0.00, 0.000), unless the context clearly indicates otherwise. Similarly, each of the singular terms "a", "an" and "the" is taken to mean "one or more than two", although the expression "one or more than two" may also be used in this specification.

[0020] As used herein, each of the terms “comprises,” “includes,” or “comprising” identifies the presence of a described feature, integer, step, operation, element, or component, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or groups thereof.

[0021] Where used herein, if this disclosure states that something is “based on” something else, such statement refers to a basis that may also be based on one or more other things. In other words, unless expressly indicated otherwise, “based on” as used herein means inclusively “based on at least part” or “based on at least part.”

[0022] The terms “next,” “next,” or other similar forms used herein are not intended to limit the order of steps. Rather, these terms are used simply to guide the reader throughout this disclosure. While some operations can be described as sequential processes in a process flow diagram, many of these operations can be performed in parallel or concurrently. Furthermore, the order of operations can also be rearranged.

[0023] As used herein, the terms “response” or “reaction” are intended to include machine-sourced actions or omissions, such as input (e.g., local, remote), or user-sourced actions or omissions, such as input (e.g., user input device via).

[0024] In this specification, the terms “approximately” or “substantially” mean a variation of ±10% from the nominal value / term.

[0025] In this specification, various terms such as the first term, second term, third term, etc., may be used to describe various elements, components, areas, layers, or sections, but it should be noted that these elements, components, areas, layers, or sections should not necessarily be limited by such terms. Rather, these terms are used to distinguish one element, component, area, layer, or section from another element, component, area, layer, or section. Thus, the first element, component, area, layer, or section described below may be referred to as the second element, component, area, layer, or section without departing from this disclosure.

[0026] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as those generally understood by those skilled in the art to which this disclosure belongs. These terms, as defined in commonly used dictionaries, should be interpreted as having the meaning consistent with their meaning in the context of the relevant art, and should not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0027] Features or functions described in relation to a particular embodiment can be performed in, in combination with, or subcombined with, various other embodiments. Similarly, different aspects, components, or elements of embodiments disclosed herein can be combined and subcombined. Furthermore, some embodiments, individually or collectively, may constitute a larger system, and other procedures may take precedence over or otherwise modify their application. Moreover, as disclosed herein, many steps may be required before, after, or concurrently with an embodiment. It should be noted that any or all methods or processes disclosed herein can be performed in any way, at least in part, through at least one entity or party.

[0028] Accordingly, all issued patents, published patent applications, and non-patent literature referenced or cited in this disclosure are incorporated herein by reference in whole for all purposes to the same extent as when individual issued patents, published patent applications, or non-patent publications are specifically and individually indicated to be incorporated by reference. For further clarity, all incorporated by reference specifically include these incorporated publications as if they were originally included in this disclosure, as if they had been copied and pasted into this disclosure, for all purposes of this disclosure. Accordingly, any reference to anything disclosed herein includes all subject matter incorporated by reference as described above. However, if a disclosure is incorporated into this disclosure by reference and such disclosure is in whole or in part with this disclosure, this disclosure shall govern in terms of the extent of the conflict or the broader disclosure or the definition of broader terms. If any part or all of such disclosures are in conflict with each other, the later disclosure shall govern to the extent of the conflict.

[0029] Figure 1 is a top view of one embodiment of an area containing a device including the radar according to the Disclosure. Figure 2 is a side view of Figure 1 according to the Disclosure. Figure 7 is a logical diagram of the radar according to the Disclosure. Figure 9 shows a set of embodiments of a set of form factors that embody the radar according to the Disclosure. In particular, area 100 includes a device 102, an object 104, a stand 106, a sofa 108, a table 110, a chair 112, an oven 114, a refrigerator 116, a bathroom 118, a toilet 120, a bathtub 122, a sink 124, an electrical outlet 126, a wall 128, a field of view 130, a wall 132, a door 134, and a door 138.

[0030] Object 104 may include any suitable object, male or female, such as a mammal, animal, human, pet, or any other suitable object that may reside or be present in Area 100. Mammals may include animals, humans, pets, or any other suitable animal. Animals may include zoo animals, humans, pets, or any other suitable animal. Humans may be infants, toddlers, preschoolers, elementary school children, teenagers, young adults, adults, or elderly people. Pets may include dogs, cats, rabbits, birds, or any other suitable pet. Note that an object may not reside in Area 100, but may be present in Area 100. For example, this applies to visitors, workers, maintenance personnel, cleaning personnel, medical personnel, emergency personnel, or any other object (e.g., mammals, animals, humans, pets) that may be present in or movable within Area 100, whether living or inanimate, and whether living within Area 100.

[0031] Area 100 is embodied as a residence (e.g., a studio apartment) of object 104. As shown in Figure 1 (top view), area 100 is defined by walls 132, doors 134, and windows 136, resulting in a rectangular shape. However, it should be noted that area 100 can be defined by walls 132, doors 134, or windows 136 as other suitable shapes (e.g., square, ellipse, polygon, open shape, closed shape, teardrop, cornerless area). Wall 132 includes a stud (e.g., wood, metal) frame with a drywall / siding configuration (e.g., exterior wall) or a drywall / drywall configuration (e.g., interior wall). However, this configuration is not mandatory, and wall 132 may have a different configuration (e.g., brick wall, fabric wall, glass wall, plastic wall, metal wall, lattice, lattice wall, cage wall, log wall). Door 134 or window 136 may be omitted.

[0032] As shown in Figure 1, area 100 includes a bathroom 118 defined by walls 128, 132, and a door 138, and is formed as a rectangle. However, it should be noted that the bathroom 118 can be defined by walls 128, 132, or the door 138 as other suitable shapes (e.g., square, ellipse, polygon, open shape, closed shape, teardrop, cornerless area). Wall 128 includes a stud frame with a drywall / siding configuration (exterior wall) or a drywall / drywall configuration (interior wall). However, this configuration is not mandatory, and wall 128 may have a different configuration (e.g., brick wall, fabric wall, glass wall, plastic wall, metal wall, lattice, grate wall, cage wall, log wall). The door 138 may be omitted.

[0033] Area 100 includes a bathroom 118, but this is not mandatory. For example, the bathroom 118 can be omitted, or Area 100 can consist of a bathroom 118. Similarly, Area 100 can be embodied as a dwelling for the object 104, and the dwelling has a bathroom 118, a living area, and a kitchen area, but this is not mandatory. Thus, Area 100 can be embodied as any residential area suitable for the object 104 to live in. For example, Area 100 can be embodied as a living room or living area, a dining room or dining area, a bedroom or sleeping area, a bathroom or bathroom area, a shower room or shower area, a playroom or play area, a home office or home office area, a basement or basement area, a garage or garage area, a storage room or storage area, an attic or attic area, a gymnasium or gymnasium, an earthen floor or earthen floor area, a closet or closet area, or any other suitable residential room or area, but non-residential areas can also be used. Similarly, although Area 100 is shown as the residential area of ​​Object 104, this is not mandatory. Thus, Area 100 can be embodied in a variety of ways. For example, Area 100 can be embodied in, or may be embodied in, a building, an apartment building, a detached house, an annex, a warehouse, a lobby, an office space, a private room, a corridor, an entrance, a hotel, a tent, a shed, a cage, a medical facility, a nursing home, a hospice, an assisted living facility, a hospital, a vehicle occupant area, a vehicle driver area, a vehicle control area, an elevator, an airplane or helicopter cockpit, an airplane or helicopter passenger cabin, a boat room, a boat cockpit or cabin, or any other suitable area.

[0034] Area 100 and the bathroom 118 contain various everyday objects, whether fixed (e.g., electrical appliances, plumbing fixtures) or movable (e.g., floor lamps, vases). These objects include a stand 106, a sofa 108, a table 110, a chair 112, an oven 114, a refrigerator 116, a bathroom 118, a toilet 120, a bathtub 122, and a sink 124, any or all of which can be omitted from Area 100. Note that the arrangement of these everyday objects within Area 100 is illustrative, and other layouts of these everyday objects are also possible.

[0035] Area 100 includes the floor, ceiling, and corners, although the ceiling or corners are optional. Near the corners, there are stands 106 (e.g., tables, coffee tables, nightstands, chairs, shelves) on which the device 102 is stationary, placed, or positioned (e.g., fixed, secured). However, the stands 106 may or may not be included, and it should be noted that the device 102 can be placed on the floor or attached to the floor (e.g., fastened, fitted, and secured), attached to the ceiling (e.g., fastened, fitted, and secured), suspended from the ceiling (e.g., via cables or chains), attached to a wall 132 or wall 128 (e.g., fastened, fitted, and secured), or suspended from a wall 132 or wall 128 (e.g., fastened, fitted, and secured).

[0036] Device 102 includes a processor (e.g., a controller, edge processor, single-core processor, multi-core processor, system-on-a-chip, graphics processing unit, hardware accelerator, neural network accelerator, machine learning accelerator) and radar (e.g., time-of-flight radar, Doppler radar), the processor being coupled to the radar (e.g., electrically, logically, mechanically) to control the radar (e.g., receive tracking data). For example, the processor may include a controller and hardware accelerator. For example, the processor may enable local computing or edge computing to improve processing speed or to provide data privacy or data security. The radar may have the set of components shown in Figure 7 and a field of view 130. For example, the field of view may be about 120 degrees (or less or 2 or more angles) horizontally and about 90 degrees (or less or more angles) vertically, or may include such angles. Similarly, for example, the radar may be such as those disclosed in U.S. Patent No. 9,019,150.

[0037] Device 102 is shown in Figures 1 to 9 to have a housing (e.g., container, enclosure, box, cube, truncated pyramid, truncated cone, sphere, egg, television set, soundbar, speaker, bookend, flower pot, planter pot, vase, furniture, table, chair, sofa, bed, crib, shelf, bookshelf, TV stand, appliance, dishwasher, refrigerator, overcoat, stove, toy, exercise equipment, treadmill, rowing machine, musical instrument, fixtures) that hosts the processor and radar (e.g., internally and externally), but this is optional, and the housing can be omitted or changed. For example, several form factors of the housing are shown in Figure 9. Similarly, for example, one or both of the processor and the radar may not be housed in a housing at all, or may be housed in different housings (e.g., the processor is housed in a first housing and the radar in a second housing), regardless of whether these different housings are attached to each other, separated from each other, spaced apart from each other, facing each other, or in any other suitable configuration, and whether these housings are structurally or functionally identical or non-identical to each other.

[0038] As shown in Figure 1, device 102 includes power lines (e.g., wires, cords, cables) that supply power to the processor and radar. If device 102 includes other components as disclosed herein, these components can also be powered via the power lines. Thus, the power lines include an electrical plug, and device 102 is positioned in area 100 near outlet 126 such that the electrical plug is slender or flexible enough to be plugged into outlet 126 and thereby power device 102. Outlet 126 can be 110 volts, 220 volts, or other voltage suitable for operating device 102. However, if device 102 includes a battery, which may be rechargeable, that is energized to supply power to the processor, radar, and optionally any other components of device 102 for a predetermined time period (e.g., 30 minutes, 90 minutes, 120 minutes, 24 hours, 72 hours), as disclosed herein, the power lines may be omitted or an alternative power option may be used.

[0039] The processor can activate the radar to operate within area 100 so that it operates within area 100 and tracks living objects 104 within area 100 when object 104 is located within the field of view 130 within area 100. When object 104 is located within the field of view 130 within area 100, the radar generates a set of data based on tracking living objects 104 within area 100, and the processor determines based on the set of data whether an event (e.g., a medical emergency, fall, death, heart attack, or seizure, based on the set of data, is occurring within area 100) is occurring at object 104, and takes action corresponding to the event determined to be occurring within area 100 (e.g., initiating communication with a remote telephone unit or server). For example, the processor can distinguish between fast falls and slow falls, each associated with its own signature for medical purposes. For example, the event could be a medical event and could be a diagnostic prediction or diagnostic forecast. For example, an action can be triggered by a threshold based on one or more criteria relating to the object 104 being tracked by the radar or its environment. For example, an action could be (a) the device 102 making a call to a pre-configured telephone number (e.g., family, caregiver, social worker, medical professional, nurse, personal physician, medical facility, emergency service) (e.g., via its SIM module), (b) sending a message to a server located away from area 100, device 102, and object 104 (e.g., via a Wi-Fi interface), (c) performing a set of pre-programmed escalation actions (e.g., if there is no response from object 104, sounding a "Are you OK?" message and calling a pre-configured telephone number), or (d) other appropriate action. As mentioned above, object 104 does not need to reside in area 100 to be in the field of view 130. Thus, object 104 can be present in area 100 (e.g., in a recurring pattern, a single visit, for a relatively long period or temporarily) and be within the field of view 130.

[0040] The radar may operate in the Ku band, including values ​​between approximately 12 GHz and approximately 18 GHz, the K band, including values ​​between approximately 18 GHz and approximately 27 GHz, or the Ka band, including values ​​between approximately 26.5 GHz and approximately 40 GHz, without radiating to other objects in its vicinity (e.g., stationary or mobile medical devices, wearable medical devices, pacemakers, insulin pumps, infusion pumps, microwave ovens, televisions, radios, Wi-Fi, mobile phones, printers, network equipment), while complying with local radiation laws (e.g., regulated by the Federal Communications Commission). If the radar operates in at least two of such bands, the radar may be a single radar unit operating in at least two of such bands, or the radar may have at least two radar units, each dedicated to at least two of such bands. For example, the radar may operate in the Ku band, including values ​​between approximately 12 GHz and approximately 18 GHz. For example, the radar may operate in the K band, including values ​​between approximately 18 GHz and approximately 27 GHz. For example, radar can operate in the Ka-band, which includes these values ​​between approximately 26.5 GHz and 40 GHz.For example, the radar can operate in at least two of the following bands: the Ku band containing these values ​​between approximately 12 GHz and approximately 18 GHz, the K band containing these values ​​between approximately 18 GHz and approximately 27 GHz, or the Ka band containing these values ​​between approximately 26.5 GHz and approximately 40 GHz, which can operate continuously (for example, to object 104 or area 100 or its contents) as required by the processor, based on the processor determining whether various criteria, signatures, or thresholds are met or not (for example, to increase the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation), as disclosed herein. The radar may switch between at least two of these bands to increase the resolution of area 100 or its contents, or to manage power or heat dissipation, or, as disclosed herein, based on the processor's determination of whether various criteria, signatures, or thresholds are met or not (e.g., to increase the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation), or in parallel (e.g., operating simultaneously on two of these bands without self-interference for complementarity or validation or validation).For example, the radar can operate in the Ku band, which includes these values ​​between approximately 12 GHz and approximately 18 GHz; the K band, which includes these values ​​between approximately 18 GHz and approximately 27 GHz; or the Ka band, which includes these values ​​between approximately 26.5 GHz and approximately 40 GHz, and these can operate continuously (for example, to object 104 or area 100 or its contents) as required by the processor, based on the processor determining whether various criteria, signatures, or thresholds are met or not (for example, to increase the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation), as disclosed herein. The radar may switch between these bands to increase the resolution of area 100 or its contents, or to manage power or heat dissipation, or in parallel (e.g., operating on two of these bands simultaneously without self-interference for complementation or validation, or for validation) as requested by the processor, based on the processor's determination of whether various criteria, signatures, or thresholds are met or not (e.g., to increase the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation). For example, as disclosed herein, the radar may switch between Ku bands containing these values ​​between approximately 12 GHz and approximately 18 GHz, as requested by the processor, based on the processor's determination of whether various criteria, signatures, or thresholds are met or not (e.g., to increase the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation). For example, the radar may switch frequencies within the K-band between approximately 18 GHz and approximately 27 GHz, including values ​​such as when required by the processor, based on the processor determining whether various criteria, signatures, or thresholds have been met or not (e.g., to increase the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation).For example, the radar may switch frequencies within the Ka-band, including values ​​between approximately 26.5 GHz and approximately 40 GHz, when requested by the processor based on a processor that determines whether various criteria, signatures, or thresholds are met or not (e.g., to improve the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation), as disclosed herein. These bands have unexpectedly proven to be technically beneficial for a variety of reasons, as disclosed herein.

[0041] With respect to the Ku-band, the radar may operate within area 100 at radio frequencies (wavelengths between approximately 24.00 mm and approximately 16.65 mm) including values ​​between approximately 12 GHz and approximately 18 GHz, or the radar may switch frequencies within the Ku-band including values ​​between approximately 12 GHz and approximately 18 GHz, for example, when required by the processor based on a processor that determines whether various criteria, signatures, or thresholds are met or not, as disclosed herein (for example, to increase the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation). It was unexpectedly found to be technically beneficial that this band penetrates walls / objects within the field of view 130 better than higher frequencies and allows the radar to reach further, which is useful for tracking the position of object 104 within area 100 and detecting the attitude of object 104 within area 100.

[0042] With respect to the K-band, the radar operates within area 100 in a radio frequency band containing these values ​​between approximately 18 GHz and approximately 27 GHz (wavelengths between approximately 16.65 mm and approximately 11.10 mm), or the radar may switch frequencies within the K-band containing these values ​​between approximately 18 GHz and approximately 27 GHz, for example, when required by a processor based on a processor that determines whether various criteria, signatures, or thresholds are met or not (for example, to increase the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation, as disclosed herein). This band has unexpectedly been found to be technically beneficial because it corresponds to the peak of the water absorption spectrum. This is because, in some situations, conventional radar may not be set to operate at frequencies above 22 GHz, as those frequencies are easily absorbed by water. Thus, it is usually desirable for conventional radar's signal to penetrate water vapor and reach another target with higher reflectivity (e.g., metal). Thus, if the object 104 is a mammal such as a human, which may have a water content of up to 60%, a radar tracking the object 104 within the field of view 130 can produce a large measurable change in the amount of reflected signal within area 100, thereby improving the accuracy or precision of a radar operating within area 100 and tracking the object 104 within the field of view 130 within area 100. Within the K-band, as described above, the radio frequency range between approximately 23 GHz and approximately 25 GHz, and especially between approximately 24 GHz, has been unexpectedly beneficial.

[0043] With respect to the Ka band, the radar operates within area 100 within a radio frequency band containing these values ​​between approximately 26.5 GHz and approximately 40 GHz (wavelengths between approximately 11.31 mm and approximately 7.49 mm), or the radar may switch frequencies within the Ka band containing these values ​​between approximately 26.5 GHz and approximately 40 GHz, for example, when requested by the processor based on the processor determining whether various criteria, signatures, or thresholds are met or not (for example, to increase the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation, as disclosed herein). This band has unexpectedly been found to be technically beneficial in facilitating the detection of vital signs (e.g., heart rate, respiratory rate) located within the field of view 130 when object 104 is a mammal such as a human. Since object 104 may repeatedly deflect within the field of view 130 due to oxygen inhalation / exhalation or blood pulsation, this condition results in a larger percentage change in the wavelength of the carrier frequency detected by the processor.

[0044] The radar may switch modality between Doppler mode (or another radar modality) and time-of-flight mode (or another radar modality) when requested by the processor, based on the processor's determination of whether various criteria, signatures, or thresholds are met or not (for example, to increase the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation), as disclosed herein. Note that such switching may or may not operate in series or in parallel, may or may not interfere with each other, and may or may not be accompanied by frequency switching or band switching, regardless of whether the radar is operating in the Ku-band, K-band, Ka-band, or other bands, as disclosed herein. For example, the radar may have a first radar unit operating in Doppler mode and a second radar unit operating in time-of-flight mode, and the processor may request that the first radar unit operate in Doppler mode and then switch to the second radar unit to operate in time-of-flight mode, or vice versa, based on the processor determining whether or not various criteria, signatures, or thresholds are met (for example, parallel or serial radar mode operation is also possible, as disclosed herein, to increase the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation). Note that the first and second radar units may be hosted by a common housing (e.g., internally and externally) as disclosed herein, or each may have its own housing that is spaced apart from one another (e.g., within about 5, 4, 3, 2, or 1 foot or meter).Similarly, for example, the radar may operate in parallel or serial radar mode, but may also operate in Doppler mode or time-of-flight mode, as disclosed herein, based on the processor determining whether various criteria, signatures, or thresholds are met or not (e.g., to increase the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation), the processor may require the radar to operate in Doppler mode and then switch to time-of-flight mode, or vice versa.

[0045] This radar is designed with experience from numerous prototypes, including expertise in radar, signal processing, and artificial intelligence, at operating frequencies of approximately 5 GHz (and within each band), approximately 24 GHz (and within each band), approximately 60 GHz (and within each band), and other frequencies (all and each band function, but some bands function better than others in certain use cases). For example, with respect to approximately 5 GHz (and within each band) or approximately 60 GHz (and within each band), the radar can operate at these frequencies or within their corresponding bands, switch frequencies within them, or switch bands with the Ku band, K band, Ka band, or other bands disclosed herein. In some embodiments, some design parameters relate to the field of view (the left and right limits of what the radar can see). For example, the field of view may be approximately 120 degrees horizontally (or downward or upward), approximately 90 degrees vertically (or downward or upward), or other fields of view. In some embodiments, design parameters relate to the resolution (the granularity at which the radar can identify details within the field of view). Resolution can be implemented via voxels (3D pixels) having a width of approximately 15 degrees, a height of approximately 15 degrees, and a depth of approximately 25 cm, which are useful for position detection and fall detection, although the resolution can be high or low individually. It is useful for measuring heart rate, respiratory rate, or other vital signs. In some embodiments, among the design parameters are penetrability (the balance between the radar's ability to penetrate common objects (walls, furniture, etc.) and reflections from objects being monitored (humans, etc.)). For example, some embodiments allow penetration of two layers of US standard studded drywall or drywall / siding for approximately 20 meters with good or sufficient reflection from human targets. Note that this distance is illustrative and can be increased or decreased based on other parameters (e.g., supplemental data sources, supplemental radar, type of material used in the manufacture of walls or appliances or furniture, radar height relative to the ground floor or physical area or monitoring floor or physical area, power limits set by government authorities).In some embodiments, the design parameter relates to the transmit power (if the field of view defines at least several directions in which the radar scans, the transmit power affects how far the radar can see).

[0046] As described above, radars operating in the K-band, including these values ​​between approximately 18 GHz and 27 GHz (wavelengths between approximately 16.65 mm and 11.10 mm), particularly between approximately 23 GHz and 25 GHz, and more particularly between approximately 24 GHz, have been unexpectedly beneficial due to a good balance between signal penetration, tracking distance, and human detection, while being hampered by regulated power limits that are themselves set to fit well within human safety parameters across a very wide range of applications or use cases, as disclosed herein. By operating radar in the K-band, including these values ​​between approximately 18 GHz and 27 GHz (wavelengths between approximately 16.65 mm and 11.10 mm), particularly within the K-band, including these values ​​between approximately 23 GHz and 25 GHz, and more particularly at approximately 24 GHz, the radar can be further tuned to field of view and resolution by changing some characteristics of some antenna arrays or by adding additional radar subsystem boards that may be required for future applications. For example, this mode of operation allows for relatively high frequencies that enable accurate distance measurement, while also enabling miniaturization of some antennas, making the overall antenna array more compact, and facilitating integration into various form factors, as disclosed herein.

[0047] The material penetration characteristics of radars operating in the K-band at approximately 24 GHz, including these values ​​between approximately 18 GHz and 27 GHz (wavelengths between approximately 16.65 mm and 11.10 mm), particularly between approximately 23 GHz and 25 GHz, and more particularly at approximately 24 GHz, are far better for indoor tracking (e.g., within Area 100) than those operating at approximately 60 GHz or approximately 76-78 GHz. However, operation at approximately 60 GHz (or within its corresponding band, or at a switching frequency therein, or at a switching band with the Ku-band, K-band, Ka-band, or other bands disclosed herein) or approximately 76-78 GHz (or within its corresponding band, or at a switching frequency therein, or at a switching band with the Ku-band, K-band, Ka-band, or other bands) may be sufficient for some indoor use cases (e.g., within Area 100). Specifically, radars operating in the K-band, including these values ​​between approximately 18 GHz and 27 GHz (wavelengths between approximately 16.65 mm and 11.10 mm), particularly between approximately 23 GHz and 25 GHz, and more particularly in the approximately 24 GHz range, can operate through several layers of standard wall structures and see through various types of clothing. For example, some radars above 60 GHz have limitations in indoor operation because walls are substantially opaque or they may be strongly affected by noise-generating clothing. For instance, a radar operating at approximately 60 GHz can detect the heart rate and respiratory rate of a relatively stationary object 104 standing within approximately 7 meters in front of the radar. However, some embodiments of radar operating at approximately 60 GHz do not penetrate solid objects, including clothing and human bodies, sufficiently. Therefore, if the object 104 moves and, if any, clothing flutters, noise may be introduced into the measurement. In contrast, radar operating in the K-band at approximately 24 GHz, including these values ​​between approximately 18 GHz and 27 GHz (wavelengths between approximately 16.65 mm and 11.10 mm), particularly between approximately 23 GHz and 25 GHz, and more particularly at approximately 24 GHz, can penetrate the human body, thereby minimizing operation-related noise. Furthermore, if the radar operates on voxel-based tracking in the K-band at approximately 24 GHz, including these values ​​between approximately 18 GHz and 27 GHz (wavelengths between approximately 16.65 mm and 11.10 mm), particularly between approximately 23 GHz and 25 GHz, and more particularly at approximately 24 GHz, the processor can discard, remove, delete, or ignore any or specific voxels that do not intersect with objects 104 such as the human body, or alternatively, can simultaneously measure the vital signs of multiple objects 104 such as the human body within area 100.For example, when using voxel-based tracking, the processor receives a set of data from the radar, accesses a set of voxels formed based on the set of data, discards, removes, deletes, or ignores a first subset of voxels from the set of voxels based on the fact that the first subset of voxels does not represent a living object 104 in area 100, so that a second subset of voxels is identified from the set of voxels, and takes actions as disclosed herein in response to events determined to be occurring in the defined area based on the second subset of voxels. Thus, this form of voxel filtering enables higher processing efficiency. Furthermore, radars operating in the K-band at about 24 GHz, more particularly at about 23 GHz to about 25 GHz, including these values ​​between about 18 GHz and about 27 GHz (wavelengths between about 16.65 mm and about 11.10 mm), enable millimeter-level resolution within voxels, allowing observation of the movement of an individual's tissues and skin as a function of heart rate and respiration.

[0048] When device 102 is provided to a user who may be the object 104 (e.g., DIY) or not (e.g., an agent, child, or caregiver acting on behalf of a parent or person receiving care), the user can receive instructions regarding device 102 (e.g., configuration and use). For example, the user may be the object (e.g., DIY), or the user may not be the object (e.g., an agent, child, parent, or caregiver acting on behalf of a person receiving care). Device 102 can be provided to the user in a variety of ways. For example, device 102 can be mailed (e.g., US Mail), delivered by courier (e.g., FedEx), shipped (e.g., in a package), sent, handed over, delivered, present in a region, residence, or vehicle, installed, or otherwise suitably made available to the user. The user can be instructed in a variety of ways. For example, the user may be instructed through media such as written documents, drawings, or printed manuals, computer files (e.g., portable document format files, presentation files, word processing files, image files, video files, audio files), websites, mobile applications, audio or pictorial guides, auditory or visual wizards, call centers, or any other appropriate instructions regarding the device (e.g., how to use it). The user may be instructed on how and where to position the device 102 within the area 100 in which the object 104 is present (e.g., relatively temporarily). The user may be instructed on the direction of power supply, how to turn on, and how to start the radar within area 100. If initial setup or configuration is required, the user may be instructed accordingly.

[0049] Once the radar is set up and activated, it tracks objects 104 that are alive (or located) within area 100, generates a set of data based on tracking objects 104 that are alive (or located) within area 100, and sends the set of data to a processor so that the processor can decide whether or not to take action based on the set of data (e.g., individually or in combination with other knowledge, predictions, estimates, inferences, or data from data sources about objects 104 or area 100), and take action based on the set of data. For example, based on the set of data, the processor can decide whether objects 104 are experiencing an event within area 100 (e.g., a medical emergency, a fall, death, a heart attack, a seizure, a diagnostic prediction, a diagnostic estimate, a diagnostic prediction), and can take action in response to the event that it has been determined to be occurring within area 100 (e.g., initiating communication with a remote telephone unit or remote server). For example, the processor can determine whether object 100 is experiencing an event by forming a signature for an object based on a set of data that may span a certain period of time, comparing the signature to a set of signature templates corresponding to a set of events (e.g., medical emergency, fall, death, heart attack, seizure, diagnostic prediction, diagnostic prediction), and then determining whether a matching threshold between the signature and the set of signature templates is met or not. For example, the matching threshold may or may not be met to estimate that the health status of object 104 (e.g., daily living activities, exercise level, movement speed, reaction time) may be deteriorating.

[0050] The event may relate to the daily living activities of object 104 within area 100 (e.g., eating, drinking, sleeping, washing, bathing, excretion, reading, sitting, exercising, cooking, cleaning). For example, the event may identify that object 104 is performing daily living activities adequately or inadequately, or that there is a decrease, increase, or maintenance in the number, frequency, or quality of daily living activities. Similarly, this event may relate to object 104 falling within area 100. Similarly, this event may relate to object 104 being stationary within area 100 for a predetermined period of time (e.g., death, paralysis, injury, unconsciousness, sleep). Furthermore, this event may relate to object 104 being absent from area 100 for a predetermined period of time (e.g., loss, disappearance, injury, death, unconsciousness, seizure, obstruction). Furthermore, this event may be associated with the fact that, while object 104 is within area 100 (for example, dead, paralyzed, injured, unconscious, sleeping in an obstructed area or coverage gap), object 104 is not tracked within area 100 for a predetermined period of time.

[0051] It should be noted that the processor does not need to determine whether object 104 within area 100 is experiencing an event. Thus, whether additional or alternative, the processor determines whether an action should be taken based on a set of data, whether it is independent of the event, dependent on the event, or without the event, and takes action based on the set of data, which may be independent of the event or unrelated to the event. For example, the processor can determine, as disclosed herein, that there are no events being experienced by object 104 within area 100, or that a frequency change within a band (e.g., Ku-band, K-band, Ka-band) may be necessary independently of or agnostically related to an event (e.g., to increase the resolution of object 104 or area 100 or its content, or to manage power or heat dissipation), or that a band change (e.g., between at least two of the Ku-band, K-band, or Ka-band) may be necessary independently of or unrelated to an event (e.g., to increase the resolution of object 104 or area 100 or its content, or to manage power or heat dissipation), or that a set of data may need to be validated or verified by another data source (e.g., a set of microphones, cameras, sensors) independently of or unrelated to an event.

[0052] As described herein, device 102 can be embodied in or as a wide variety of everyday household objects, such as televisions, dumb speakers and smart speakers, bookends, flower pots, planter pots, vases (or other containers), furniture (e.g., tables, chairs, sofas, wall-mounted or tabletop or shelf-mounted picture frames, bookshelves, shelves, dressers, beds, cabinets, shutters), and household appliances (dishwashers, ovens / stoves, microwave ovens, refrigerators, washing machines, dryers, etc.). Such implementations provide a clear technological enhancement to the various sensing capabilities of the current offerings, as disclosed herein. However, it should be noted that additional or alternative non-household or non-residential use cases are also possible. These may include applications for measuring eye movements, facial expressions, mouth movements, head movements, joint movements, heart rate, respiratory rate, or other vital signs in vehicles (buses, ships, airplanes, train cars, etc.) while people are driving, commuting, or traveling to other destinations, and while people are living, working, or playing. For example, if device 102 is installed in or included in a vehicle (e.g., a car, truck, bus, or cockpit), as disclosed herein, the radar may track whether the vehicle operator (e.g., the driver of the vehicle) or passengers are tired, drowsy, asleep, unconscious, dead, having a seizure, or intoxicated, and then take appropriate programmed actions (e.g., not start the vehicle, slow down or stop the vehicle if safe, notify an emergency phone number or radio contact from the vehicle, notify a preset phone number or radio contact from the vehicle, send geolocation from the vehicle to a remote server, open the vehicle windows, sound the vehicle horn or security alarm, open the doors if safe or too cold or the vehicle is not moving, post to social media from the vehicle).Similarly, if device 102 is installed in or included in a vehicle (e.g., a passenger car, automobile, van, minivan, or sports utility vehicle), as disclosed herein, the radar may, if no one else is being tracked in the vehicle (e.g., if no one else is being tracked in the vehicle, the radar may, if an infant (or infant, toddler, disabled, frail, or elderly person) is in a child seat, and may take programmed appropriate actions (e.g., start the vehicle, turn on the vehicle's air conditioning or heating or climate control, notify an emergency number from the vehicle, notify a pre-set number from the vehicle, send location information from the vehicle to a remote server, open the vehicle's windows, sound the vehicle's horn or vehicle's security alarm, open the vehicle's doors if it is safe or not too cold and the vehicle is not moving, post to social media from the vehicle). Tracking of these situations may be supplemented by data from other input devices (e.g., microphones, cameras, sensors) and powered by an onboard battery, as disclosed herein.

[0053] As described herein, a radar operating in the K-band at about 24 GHz, including these values ​​between about 18 GHz and about 27 GHz (wavelengths between about 16.65 mm and about 11.10 mm), particularly including these values ​​between about 23 GHz and about 25 GHz, and more particularly at about 24 GHz, can be a time-of-flight radar that supplies a set of data to a processor for the processor to convert into a set of voxels for simulating an object 104, which may include simulating area 100. The radar can switch bands (e.g., between Ku-band, K-band, or Ka-band) or frequencies (e.g., within Ku-band, K-band, or Ka-band) to improve the resolution of the set of voxels, which is incomplete, inaccurate, or does not meet or fail to meet a preset resolution threshold, as disclosed herein. Thus, the radar can track the position, posture, or vital signs of objects 104 (e.g., people, the elderly, people with disabilities, and people with illnesses) within its field of view 130, whether above ground or underground, indoors or outdoors, in homes; hospitals or nursing homes or rehabilitation facilities; elderly care facilities; care homes and shelters, or other appropriate medical or non-medical facilities.

[0054] As further described herein, a set of microphones included in device 102 and controlled by the processor can optionally provide high-quality 3D audio mapping for localizing or locating sound sources, and optionally provide bidirectional audio functionality. For example, the set of microphones can supply audio data (and data to other sensors) to the processor to enable a flexible computing platform that combines hardware-accelerated artificial intelligence (AI) and processing at the edge, i.e., edge computing. As further described herein, device 102 may include single or various communication options (e.g., wired, wireless, waveguide) so that device 102 can function as a hub for other devices, transferring data, alerts, and critical information to and from virtual computing instances (e.g., servers, applications, databases) or telephones running within cloud computing services (e.g., Amazon Web Services, Google Cloud), regardless of whether the method is wired, wireless, or waveguide. To maximize efficiency and security, device 102 can be securely and regularly updated wirelessly to support an evolving ecosystem with various processor / radar algorithm improvements and additions, as well as optional device and service updates.

[0055] As described herein, the radar transmits a set of data (e.g., raw data) to a processor (e.g., an edge processor), which processes it with onboard algorithms to reconstruct high-resolution 3D information (e.g., voxel information) about the area 100 within the field of view 130. The device 102 then applies various techniques to identify or infer various information about the object 104. For example, if the object 104 is a person, the processor can identify the object 104's posture (e.g., standing, lying down, supine, prone, prone) or the object 104's daily living activities (e.g., sitting, standing, walking, falling, sleeping, eating, exercising, showering, bathing, cooking). The processor uses this information, along with data from other sensing subsystems (e.g., sound, vision, temperature, motion, distance, proximity, moisture) and, where available, additional data sources—ranging from fitness apps (e.g., tracking object 104) to social media (e.g., profile of object 104) and electronic health records (e.g., profile of object 104)—to know, where available, what state people are in (e.g., deterministically, rather than inferring probabilistically), allowing the electronic device 102 to enable people's safety, well-being, and supported self-care while minimizing false alarms and missed signs of increased risk.

[0056] As described herein, device 102 benefits from a virtuous cycle of increasing data sets. These data sets drive quality improvements, enhance user value, resulting in more usage and more data generation. In some embodiments, device 102 has a multi-sensor ambient sensing platform, and cloud computing services provide valuable context. For example, device 102 is further developed / enhanced through continuous field A / B testing (e.g., by wireless firmware updates), resulting in improvements to sensor processing, health monitoring, and user experience, all based on continuously improving the AI, which over time catalyzes additional functionality, use cases, and resulting offerings. For example, device 102 may have processor-accessible memory (e.g., persistent memory, flash memory) that stores an AI model (e.g., based on a convolutional neural network, recurrent neural network, or enhancement algorithm) trained to determine whether an object 104 meets or fails to meet various criteria, signatures, or thresholds that may relate to the health, safety, or security of object 104 within area 100. Thus, if the object 104 is a human being, the processor can access the AI ​​model to analyze the baseline over a long period of time and observe early indicators of changes in the human's safety, well-being, and health. Therefore, tracking the object 104 over time using device 104 should improve monitoring as the AI ​​model gains a better understanding of the object 104.

[0057] As described herein, device 102 has many exemplary use cases. For example, if the object 104 is a person, observing (and the processor acting on) routine biometric measurements, exercise, and daily living activities (e.g., via radar) can help preventively detect the emergence of potential health and safety risks well in advance of episodic physical examinations, laboratory tests, and diagnostic tests, and sometimes well in advance of the person becoming aware of a deterioration in their health. Through observation (e.g., by the processor via radar) and analysis (e.g., by the processor via radar) of daily behavioral patterns such as eating, drinking, sleeping, exercising, washing, bathing, showering, defecating, socializing, getting dressed, and taking medication, a digital image of the person's health status and well-being emerges (the processor can act on this). Furthermore, by having the processor (for example, via radar) move, observe and analyze the time spent in bed and the frequency of toilet use, and take action based on these (for example, electronically notifying the individual, family, or physician via sound prompts, warning messages, or a mobile app), it is possible to identify signs of illness that may be difficult for the individual, family, or even clinicians to notice. Thus, not only can device 102 increase awareness of such undetected disease states, but device 102 can also identify deterioration of mental or physical function (for example, by the processor) and help monitor (and the processor take action based on) signs of worsening chronic conditions that a person is already experiencing. Device 102 enables the detection (for example, by the processor) of such signs of risk to people's safety, well-being, and health well before they become more acute symptoms and lead to ambulance dispatch, admission to the emergency room, and hospitalization. Thus, the desired end result is the reduction of suffering for people supported by device 102, cost reduction through proactive intervention, and efficient use of valuable medical resources.

[0058] One use case for device 102 is detecting falls of object 104 via the processor. For example, if object 104 is an elderly person, a fall could be dangerous to their health and safety. While falls are common, this common term hides significant variations and nuances in the events that 104 experiences. The characteristics and nuances of a fall, such as location, circumstances, direction, speed, orientation, angle, or other, tracked by the radar and transmitted to the processor, can provide clues to the processor about its underlying cause, and these details have significant clinical implications. Unfortunately, many people who fall sustain injuries that complicate efforts to determine what actually happened, and this often presents a diagnostic challenge for physicians. The speed or direction of a fall, as determined by the processor based on the set of data from the radar, may influence head injuries, fractures, and the likelihood of recovery. Thus, the processor can consider or detect various factors contributing to a fall based on the set of data from the radar. These factors may include location (which room, its position within the room, type, and intensity). These factors include situations such as taking a shower or entering a room with a threshold. These factors may include consequences—movements that result in being unable to move, stumbling, or being unable to stand. These factors may include falling out of bed, falling from a standing position, or falling from a chair. These factors may include direction—a simple stumble or a possible correlation with an event affecting the brain. These factors may include speed (speed increases the risk of injury, and trajectory indicates a fall or tumble). Such factors may include disability, and asymmetrical movement may indicate a fracture. Therefore, some of these factors may be related to the severity of the injury or the presence or absence of an underlying medical condition.

[0059] One use case for device 102 is to detect functional limitations of object 104 via the processor. For example, if object 104 is an elderly person, their functional limitations may pose a health and safety risk. A frail person may experience various changes in their mental and physical functions over time, whether cognitive or physical limitations, before they become impaired to the point where they require care or are noticed by others (e.g., tracked by radar and detected by the processor). Detectable evidence of a decline in mental function (e.g., tracked by radar and detected by the processor) may include changes in self-care patterns (e.g., tracked by radar and detected by the processor) or the ability to maintain sleep (e.g., tracked by radar and detected by the processor). Evidence of a decline in physical function (e.g., tracked by radar and detected by a processor) may include a decrease in overall physical activity (e.g., tracked by radar and detected by a processor), a decrease in walking speed (e.g., tracked by radar and detected by a processor), or evidence of tripping (e.g., tracked by radar and detected by a processor).

[0060] One use case for device 102 is to detect changes in daily living activities via the processor. For example, if the object 104 is an elderly person, such changes may be dangerous to health and safety. Elderly individuals are regularly assessed on their ability to perform activities of daily living (e.g., tracked by radar and detected by the processor), and these changes (e.g., tracked by radar and detected by the processor) are part of the aging process that may correlate with future problems in self-care, coping, and independence. Along with disease conditions (e.g., tracked by radar and detected by the processor), these changes can be an important part of the Frailty Index (e.g., tracked by the processor), which is a surrogate of vulnerability to poor outcomes. Thus, device 102 can detect human engagement in several key activities of daily living, including sleep, eating, drinking, excretion, socializing, or others (e.g., tracked by radar and detected by the processor). For example, based on a set of data received from radar, the processor can track sleep and detect sleep interruptions, schedule changes, time to fall asleep, or early awakenings. For example, based on a set of data received from radar, the processor can track timing, frequency, preparation time, eating and drinking time, or whether cooking is involved. For example, based on a set of data received from radar, the processor can track the timing and frequency of toilet use. For example, based on a set of data received from radar, the processor can track socialization—measuring time spent alone, time spent on the phone, frequency and duration of visits from others, and the number of visitors present. As described herein, the processor can distinguish object 104 from others. For example, based on a set of data received from radar, the processor can track minor behaviors (e.g., medication use, changing clothes, grooming).For example, based on a set of data from radar which can be enhanced or supplemented (e.g., augmented) by data from other data sources disclosed herein, the processor can distinguish, based on the set of data, an object 104 located in or living within area 100 from another object 104 located in or living within area 100, before tracking object 104 and another object 104 (e.g., by learning object 104's habits and signatures over time) and determining whether object 104 is experiencing an event involving object 104 within area 100. For example, as time progresses, the processor becomes able to distinguish between object 104 and another object 104 within area 100, and based on the set of data, the processor can determine whether an event is custom (or unique) to object 104, based on distinguishing between object 104 and another object 104.

[0061] One use case for device 102 is detecting evidence of disease via the processor. For example, if object 104 is an elderly person, there may be a disease that poses a risk to their health and safety. Many diseases are evidenced by anomalies that the processor can identify and draw attention to based on a set of data from radar (e.g., electronically notifying the person, their family, or a doctor via sound prompts, warning messages, or a mobile app). The processor (e.g., running various AI algorithms) and cloud analysis logic (e.g., running within a cloud computing instance) enable the correlation of various factors (transmitted by device 102 via a communication unit) that are useful in recommending or suggesting at least some medical assessments, via output devices of device 102 (e.g., speakers, displays) requested by the processor. Examples of these disorders include cardiopulmonary disorders—irregular heartbeat (detected by heart rate analysis by the processor from a set of data transmitted by radar), insufficient cardiac reserve (increased heart rate above baseline during normal activity, detected by the processor from a set of data transmitted by radar), or insufficient respiratory reserve (increased respiratory rate above baseline during normal activity, detected by the processor from a set of data transmitted by radar). Examples of these disorders may include sleep disorders—nocturnal awakenings (via atypical nocturnal awakenings detected by the processor from a set of data transmitted by radar), snoring patterns (via microphone detection by the processor), and sleep apnea (via analysis of respiratory rate and pauses detected by the processor from a set of data transmitted by radar or microphone).Examples of these disorders include neurocognitive disorders—cognitive impairment (abnormal repetition of behavior detected by the processor from a set of data transmitted by radar), turnaround patterns (detected by the processor from a set of data transmitted by radar), omission of cooking (detected by the processor from a set of data transmitted by radar), or impaired self-care (detected by the processor from a set of data transmitted by radar). Examples of these disorders include neuromuscular disorders—motor disorders—mediated by gait analysis (detected by the processor from a set of data transmitted by radar), rise time analysis (detected by the processor from a set of data transmitted by radar), and gait speed analysis (detected by the processor from a set of data transmitted by radar), or seizure detection (abnormal movements, abnormal movements preceding a fall, post-seizure conditions detected by the processor from a set of data transmitted by radar). Examples of these illnesses include infectious diseases; the detection of a cough (detected by the processor from a set of data transmitted from the radar or microphone) may indicate the onset of influenza, pneumonia, upper respiratory infection, or COVID; the frequency of toileting or defecation (detected by the processor from a set of data transmitted from the radar) may indicate a urinary tract infection or the presence of diarrhea or constipation. Examples of such illnesses include metabolic diseases—the frequency of toileting (detected by the processor from a set of data transmitted from the radar) may indicate uncontrolled diabetes or signs of urinary or gastric problems; or thyroid conditions may subtly warn of a resting heart rate over time (detected by the processor from a set of data transmitted from the radar). For example, with respect to the various diseases disclosed herein, the processor can be programmed to identify potential signs, onset, or signs of a specific disease (e.g., neurological disorders, neurodegenerative diseases, Parkinson's disease, Alzheimer's disease, facioscapulohumeral muscular dystrophy, Crohn's disease, chronic obstructive pulmonary disease, atopic dermatitis) based on a set of data from the radar.

[0062] One use case for device 102 is monitoring chronic conditions (by the processor from a set of data transmitted by the radar). For example, if the object 104 is an elderly person, a chronic condition may be dangerous to their health and safety. Various factors for successful management of chronic diseases are related to the individual's behavior. In addition to understanding whether the individual's activities are promoting their health, device 102 can detect changes (by the processor from a set of data transmitted by the radar) that suggest the individual is suffering from a worsening condition. There are various chronic conditions that can be monitored by device 102 (processor). For example, some heart failures and rhythm disorders can be monitored by heart rate and respiratory rate (by the processor from a set of data transmitted by the radar), movement speed (by the processor from a set of data transmitted by the radar), physical activity time (by the processor from a set of data transmitted by the radar), or medication use (by the processor from a set of data transmitted by the radar). Some asthma and chronic obstructive pulmonary disease (COPD) may be monitored by heart rate and respiratory rate (processed from a set of data transmitted by radar), cough and wheezing measurement (processed from a set of data transmitted by radar), movement speed (processed from a set of data transmitted by radar), or medication use (processed from a set of data transmitted by radar). Some sleep disorders may be monitored by sleep duration (processed from a set of data transmitted by radar), interruptions (processed from a set of data transmitted by radar), snoring detection (processed from a set of data transmitted by radar or microphone), or heart rate and respiratory rate (processed from a set of data transmitted by radar).Certain neurobehavioral states can be monitored by tracking potential declines in daily living (by the processor from a set of data transmitted by radar), movement speed (by the processor from a set of data transmitted by radar), medication use (by the processor from a set of data transmitted by radar), or near-falls or falls (by the processor from a set of data transmitted by radar). For example, the processor, as disclosed herein, can fuse such data based on a set of data from radar or augmented by data from other sensors to not only detect falls (or other events), but also estimate or predict at least some emotional health, mood, and stress levels of the object 104.

[0063] Figure 3 shows one embodiment of a radar-inclusive device according to the present disclosure. Figure 4 shows embodiments of a set of radar or sensor-inclusive devices according to the present disclosure. In particular, device 102 is positioned on a stand 106. The device includes a housing having an upper, lower, and intermediate portion spanning between the upper and lower parts. The upper or central portion may be omitted.

[0064] The top surface may include wood, but may also include other suitable materials (e.g., plastic, metal, rubber, fabric, glass). The bottom surface may include wood, but may also include other suitable materials (e.g., plastic, metal, rubber, fabric, glass). The middle section may include fabric, but may also include other suitable materials (e.g., plastic, metal, wood, rubber, glass). Note that, as shown in Figure 4, the top, bottom, and middle sections can be selectively swapped to create different appearances as desired.

[0065] The housing has an elliptical shape when viewed from above, but this is not mandatory, and the housing (or support in another form) can be in other forms (e.g., container, housing, box, frame, base, cube, pyramid, cone, sphere, egg shape, television equipment, soundbar, speaker, bookend, flower pot, planter pot, vase, furniture, table, chair, sofa, bed, crib, shelf, bookshelf, TV stand, home appliance, dishwasher, refrigerator, overcoat, stove, toy, exercise equipment, treadmill, rowing machine, musical instrument, fixtures, electrical equipment, plumbing equipment).

[0066] Figure 5 is a logic diagram of one embodiment of a device including radar according to the present disclosure. In particular, device 102 is powered by a power supply unit, which is powered via the power supply unit. logic unit, logic A communication unit (optionally, itself or any of its components) that is controlled via a unit and powered via a power supply unit The system includes a housing (e.g., an enclosure) that hosts (e.g., internally or externally) a speaker unit (optionally itself or any of its components) controlled via a logic unit and powered via a power supply unit, a connector unit (optionally itself or any of its components) controlled via a logic unit and powered via a power supply unit, a user interface unit (optionally itself or any of its components) controlled via a logic unit and powered via a power supply unit, a microphone unit (optionally itself or any of its components) controlled via a logic unit and powered via a power supply unit, and a sensor unit (optionally itself or any of its components) controlled via a logic unit and powered via a power supply unit.

[0067] The logic unit includes a processor, random access memory (RAM), persistent storage (e.g., embedded multimedia cards, solid-state drives, flash memory), and hardware accelerators (e.g., neural network accelerators, machine learning accelerators), although the hardware accelerators are optional. The communication unit (either itself or any of its components) includes a cellular SIM unit (e.g., a SIM card receiver and corresponding circuitry), a cellular modem, a Wi-Fi unit (e.g., a Wi-Fi chip), a Bluetooth (or another near-field communication standard) unit (e.g., a Bluetooth chip), a Zigbee (or another near-field automation communication standard) unit (e.g., a Zigbee chip), a Z-wave (or another near-field automation communication standard) unit (e.g., a Z-wave chip), a near-field communication (NFC) unit (e.g., an NFC chip), and a registration jack (RJ). The communication unit is used for various communications as disclosed herein. For example, if device 102 needs to be paired (for example, for setup or configuration) with a desktop or mobile device (e.g., a phone, laptop, tablet, or wearable computer) or another device 102 or sensor as disclosed herein, a Bluetooth or Wi-Fi unit can be used. If device 102 needs to be connected to Wi-Fi (e.g., to send data formed by the processor to a cloud computing service or to receive data from a cloud computing service), or to a desktop or mobile device (e.g., a phone, laptop, tablet, or wearable computer), or another device 102 or sensor as disclosed herein, a Wi-Fi unit or RJ can be used.For example, if device 102 has a lamp (e.g., hosted internally or externally by its housing), needs to illuminate an indicator light (either itself or on another device), needs to sound a general alarm (either itself or on another device), or needs to use another device to control the characteristics of object 104 (e.g., using another device, the Zigbee or Z-wave protocol (or other suitable automation communication protocol) can be used by the lamp (either itself or on another device), the alarm (either itself or on another device), the health monitoring device (either itself or on another device), and the general smart home automation (either itself or on another device)). If device 102 needs to be configured for various operations as disclosed herein, such configuration can be done via a mobile app running on a mobile phone, tablet computer, wearable computer, or another computing unit (e.g., laptop, desktop) operated by object 104 or the user and communicating with device 102, as disclosed herein. This arrangement simplifies the collection of configuration information (such as Wi-Fi passwords and authentication codes). Next, whatever computing unit is used, its NFC unit can be used to transmit its configuration information to device 102 (for example, wirelessly), thereby enabling device 102 to use other communication standards. Note that Bluetooth communication technology (or other short-range communication protocols) may be used additionally or alternatively to transmit such configuration information (and the computing unit and device 102 may be configured accordingly). If device 102 needs to synchronize its operation or exchange data with another device 102 (for example, if two radars cooperate to scan area 100), any communication component of the communication unit (e.g., Bluetooth unit, Z-wave unit, Zigbee unit, Wi-Fi unit, RJ unit) can be used. The speaker unit (or any component thereof) may include an amplifier or a speaker.The speaker unit is used for various sound, vocal, or audio outputs, as disclosed herein. For example, settings, health, safety, suggestions, recommendations, or wellness content may be output by the speaker unit, as disclosed herein. The power unit may include a rear panel connector (optional), a barrel jack (optional), power distribution / power supply, a charge controller (optional), and a battery backup (optional). The user interface unit (optional, itself or its components) includes physical interfaces (e.g., buttons, dials, levers, switches) and virtual interfaces (e.g., touch or non-touch displays presenting a graphical user interface). The user interface unit is used for various tactile or visual outputs, as disclosed herein. For example, settings, health, safety, suggestions, recommendations, or wellness content may be output by the user interface unit, as disclosed herein. The microphone unit (optional, itself or its components) may include a set of microphones. The microphone unit is used for various sound, vocal, or audio input captures, as disclosed herein. For example, as disclosed herein, voice commands or sounds identified within area 100, whether from object 104 or elsewhere, can be input by the microphone unit. The sensor unit may include radar, accelerator (optional), gyroscope (optional), inertial measurement unit (optional), photodetector (optional), local temperature sensor (optional), local humidity sensor (optional), and particulate sensor (optional). The sensor unit is used for sensory capture of various inputs, as disclosed herein. For example, as disclosed herein, sensory inputs identified in area 100, whether from object 104 or elsewhere, can be input by the sensor unit. For example, a photodetector, local temperature detector, local humidity detector, or particulate sensor can sense its surrounding environment and its data so that the processor can act accordingly or decide to act accordingly.For example, a photodetector, local temperature detector, local humidity detector, or particulate sensor may detect a fire, smoke, carbon monoxide, pollution, or other event so that the processor issues an alert via a communication unit (e.g., calling emergency services or a designated telephone number, turning off or shutting down a gas supply, electrical panel, or water valve), a speaker unit (e.g., directing object 104 out of area 100), or a user interface unit. Thus, if device 102 includes a communication unit and the processor is coupled to the communication unit (e.g., mechanically, electrically, or logically), the processor may take actions including instructing the communication unit to send a message to another device 102 (e.g., local) or a computer (e.g., desktop, laptop, or mobile phone) located away from area 100, the message containing content related to object 104, area 100, or the event.

[0068] Figure 6 shows an internal view of Figure 3 according to this disclosure. Figure 8 shows a photograph of the internal cavity of Figure 3 based on Figure 6 according to this disclosure. In particular, device 102 includes a housing 202, a logic unit 204, a power supply unit 206, a vent 208, a frame 210, a heat sink 212, a speaker 212, a window 214, a radar 216, and a microphone 218, some of which are shown in Figure 5. The housing 202 hosts the frame 210. The logic unit 204 is fixed or mounted on the frame 210 such that the logic unit 204 extends between the power supply unit 206 and the window 214 (e.g., fastened, fitted, interlocked, bonded), but this positioning can vary. The power supply unit 206 supplies power to the logic unit 204. The power supply unit 206 is fixed or attached to the housing 202 (e.g., fastened, fitted, interlocked, bonded) such that the logic unit 204 extends between the power supply unit 206 and the heat sink 212 or the radar or window 214. The power supply unit 206 faces the logic unit 204, but this positioning can vary. The vent 208 is defined by the housing 202 such that the logic unit 204 extends between the vent 208 and the window 214 of the speaker 212 (left side), but this positioning can vary. The vent 208 is formed by a set of symmetrical openings, but this is not mandatory and can be changed as needed (e.g., asymmetrical openings). The heat sink 212 is positioned between the radar 216 or window 214 and the logic unit 204 or power supply unit 206, but this positioning can vary.

[0069] The vents 208 and heatsink 212 are used collectively as a group of passive cooling components, and the design (e.g., size, shape) or placement of the heatsink 212, or the design (e.g., size, shape) or placement of the housing 202, or the design (e.g., size, shape) or placement of the vents 208, can each influence how convection is optimized (e.g., to keep rising heat away). This is technically advantageous because the vents 208 and heatsink 212 operate with minimal or no noise, and may be desirable in certain environments (e.g., elderly people receiving care). However, the group of passive cooling components may be technically disadvantageous because the heatsink 212 may be larger than desired, and may be economically expensive, space may be limited, or they may not be readily available due to supply shortages. Similarly, passive cooling components can be technically disadvantageous because the amount of heat removed by the vent 208 or heat sink 212 depends heavily on the ambient temperature of the air surrounding the heat sink 212 or housing 202 (e.g., next to a radiator, heater, or vent in winter), potentially causing the logic unit 204 or radar 216 to operate at higher temperatures. Therefore, in situations where passive cooling components are technically disadvantageous, active coolants can be used. Active coolants can be cooling fans, coolant circulation systems, or other suitable coolants. When embodied as a cooling fan, an active coolant can force the ambient air over the radar 216 or logic unit 204 to disperse, enabling more effective cooling than passive cooling components. However, active coolants (e.g., cooling fans) have moving or balancing parts, which can generate noise due to the movement of ambient air. Similarly, active coolants can also wear out or rattle, which may be undesirable in certain environments (e.g., elderly people receiving care). Similarly, the active coolant may reduce the long-term reliability of device 102, increase dust collection rates, increase the likelihood of mechanical failure or component imbalance, increase connector wear, and add noise to radar measurements, potentially requiring more filtering and correction if possible.Thus, depending on the use case, radar may be actively cooled or passively cooled.

[0070] As shown in Figure 6, if the housing 202 includes two speakers 212, the logic unit 204, heatsink 212, or radar 216 may extend between these speakers 212, although these speakers 212 may be located in other positions, whether inside or outside the housing 202. Note that there may be two or more speakers 212, one speaker 212, or no speakers 212 at all. The speakers are powered via the power supply unit 206 and controlled via the logic unit 204.

[0071] The radar 216 (e.g., a receiver with an antenna and a circuit board having a receiver with an antenna, or a transceiver having a transmitting antenna and a receiving antenna) extends between the window 214 and the heatsink 212 or logic unit 204 or power supply unit 206, although this positional relationship may differ. Since the radar 216 tracks (e.g., transmits and receives) through the window 214, the window 214 may be absent, but blockage, interference, and noise can be minimized. As shown in Figure 6, the housing 202 includes two symmetrical windows 214, but there may be more than two windows 214, or the windows 214 may be asymmetrical.

[0072] The microphone 218 is positioned adjacent to the window 214 such that the radar 216, heatsink 212, or logic unit 204 extends between the microphone 218 and the power supply unit 206, although the microphone 218 may be omitted or positioned elsewhere. The microphone 218 is powered via the power supply unit 206 via and controlled by the logic unit 204, as disclosed herein, to supplement, enhance, reinforce, or receive user input from the radar 216.

[0073] Figure 10 shows one embodiment of a circuit board with a radar antenna set according to the present disclosure. Figure 11 shows an embodiment of the scope of application of the device of Figure 3 according to the present disclosure. In particular, as described herein, radar can operate on the digital coding time-of-flight principle, which can be embodied in various ways. Radar can be used to track or monitor the safety, health, welfare, or care of an individual managed by a processor. For example, radar can transmit pulses and measure the time it takes for a reflection (e.g., an echo) to return from an object (e.g., a person, furniture, appliance, wall, floor, ceiling) (e.g., a receiver, transceiver, sensor, distance sensor). For example, radar can operate through a phased array of transmit and receive antennas to perform highly directional measurements. This is shown in Figure 10, where the radar includes a circuit board having two antenna arrays (small gold rectangles, but other colors or shapes are possible) and a heatsink (small black rectangles, but other colors or shapes are possible) covering a main high-frequency processing chip. For example, the radar may include a set of phased arrays, each consisting of a set of patch antennas, enabling the radar to track an object 104 that lives or is located within area 100.

[0074] Device 102 includes a processor coupled (e.g., electrically, logically, or mechanically) to the radar to achieve various design objectives or performances. Some of these objectives or performances can enable edge computing and hardware-accelerated artificial intelligence of neural networks (ANNs), which may include various corresponding models to enable various raw sensing and processing capabilities for building various software-based applications that support various types of use cases, as described herein. For example, some of the ANNs may include convolutional neural networks, recurrent neural networks, long-term short-term memory neural networks, or other suitable forms of ANNs. These forms of edge computing and hardware-accelerated ANNs may enable the processor to handle various use cases based on a set of data from the radar. These uses may include enabling the processor to track the heart rate or pulse of an object 104 based on a set of data from the radar by using various transmission properties of the radar, capturing at least some of the "backscatter" (also known as reflected signals) reflected by the object 104, and tracking the heart rate or pulse of the object 104. These use cases may include the processor tracking respiratory rate based on a set of data from radar, which can be tracked as well as heart rate as described herein, or processing signals to identify several slow respiratory variations based on a set of data from radar. These use cases may include the processor tracking the position of objects 104 within area 100 based on a set of data from radar, i.e., persistent knowledge of the position of each individual moving within area 100 (within field of view 130). These use cases may include the processor tracking fall detection based on a set of data from radar, i.e., the ability to detect the orientation of objects 104 and the rate of change of that orientation.Such use cases may include the processor being programmed for a specific room shape based on a set of data from radar, i.e., the positions of large planes (e.g., floor, walls, ceiling, furniture, appliances). These use cases may include the processor tracking the orientation of an object 104 based on a set of data from radar (the orientation of the object 104). These use cases may include the processor performing object classification based on a set of data from radar. Identifying furniture or appliances can provide context, such as whether someone is sleeping in the bed or lying on the table. These use cases may include the processor tracking various daily living activities based on a set of data from radar, i.e., the ability to accurately determine the timing and frequency of lifestyle patterns, including microbehaviors such as sleeping, drinking, eating, defecating, socializing, and taking medication.

[0075] Given the ongoing chip shortage, device 102 could be designed to leverage 5G chips to mitigate or reduce this risk. For example, since radar is an all-digital design (a non-all-digital or hybrid design is also possible), it can leverage existing communication chips (e.g., radio RF receivers, radio RF transmitters, radio high-frequency transceivers, wired interface cards, waveguides). For instance, 5G (or 3G, 4G, 6G) cellular technology could operate in the same or similar bands as radar operating in the Ku-band, K-band, and Ka-band, leading to enormous and ongoing investments in components for radio systems in these frequency bands. Similarly, further performance improvements driven by competition in the cellular market could impact radar. In contrast, some radars rely on complex analog designs and only a small portion would benefit from these external market developments.

[0076] As described herein, device 102 can be used to track where an object 104 is located within area 100 based on a set of data received from the radar by the processor and processed by the processor, and to detect the fall of the object 104 within area 100 based on a set of data received from the radar by the processor and processed by the processor. For example, if the object 104 is a person, the radar can detect the person's position and orientation through at least one or two standard US-style walls (e.g., drywalls), and the radar can “see” adjacent rooms, bedrooms, bathrooms, corridors, pantries, or other areas 100 within the field of view 130, as shown in Figure 11. However, since the radar may have cover gaps (e.g., occlusion), device 102 may be complemented by another device 102 ("satellite" unit) located within or tracking area 100, or by sensors (e.g., cameras, microphones, motion sensors, proximity sensors, distance sensors) located within or tracking area 100. Devices 102 (e.g., housings) or sensors can be spaced apart from each other to improve the resolution or coverage of objects 104 or areas 100. This allows devices 102 to validate, corroborate, or confirm its data, analysis, or conclusions, or to infer something about objects 102 or areas 100 that devices 102 cannot track or identify. For example, if object 104 is a person, the bathroom is a common place for falls in the home. The radar may generate a set of data with slightly reduced quality or resolution while object 104 is showering, but this does not prevent the detection of a fall. Some bathtubs may cast shadows on the sensor, but this occurs when the radar's view of object 104 is not obstructed or blocked, so the radar can still detect a fall.However, if the radar line of sight of object 104 is obstructed or blocked, device 102 may communicate (e.g., wired, wireless, waveguide) with another device 102 located within or capable of tracking area 100, or with a sensor located within or capable of tracking area 100, to determine, individually or collectively, whether object 104 is experiencing an event such as a fall, as disclosed herein. For example, the second device 102 may track in the Ku-band, K-band, or Ka-band, as disclosed herein. Thus, this configuration provides complementary tracking or sensing capabilities, which may include using low-resolution infrared sensing (or other sensing modalities, whether low-resolution or high-resolution, line-of-sight or non-line-of-sight, including ultrasonic sensors, LiDAR, radar, motion sensors, proximity sensors, distance sensors, or others). The "satellite" device 102 can be deployed in a distant room and provide coarse or fine sensing of the location of people within area 100, extending coverage to adjacent spaces that may be blocked (e.g., by metal walls). The "satellite" device 102 may also have a microphone or set of microphones which can help detect specific falls and provide bidirectional communication (e.g., via a Bluetooth unit, Wi-Fi unit, RJ). Some, many, most, or all calculations are performed on the main unit 102, but these calculations may be distributed or shared with other devices 102 (e.g., via a communication unit). Furthermore, by deploying multiple main units or "satellite" units, the tracking range can be expanded and "rear" coverage can be achieved.

[0077] Thus, if device 102 is a first device 102 located within area 100 to track a house or object 104 located therein (having a first radar that transmits a first set of first tracking data to a first processor), then there may be a second device 102 located within area 100 to track or sense a house or object 104 located therein (having a second radar or sensor that transmits a second set of second tracking data to a second processor). The first device 102 and the second device 102 may be spaced apart from each other. For example, the first device and the second device may face each other. Accordingly, the second device 102 can operate within area 100 without interfering with the first device 102 (e.g., radar interference) operating within area 100, provided that the second radar (e.g., time of flight) or sensor (e.g., sensor unit, audio unit, vision unit) operating within area 100 is shielded or unshielded from the first radar tracking an object 104 that is living or located within area 100. The second radar or sensor generates a second set of data based on tracking or sensing an object 104 that is living or located within area 100, provided that the first radar operating within area 100 is shielded or unshielded from the first radar tracking an object 104 that is living or located within area 100. The second radar or sensor transmits the second set of data to the second processor when the first processor operating in area 100 is shielded from or unshielded from tracking the object 104 that is housed or located in area 100, so that the second processor may, based on the second set of data, determine whether the object 104 is experiencing a second event in area 100 and take a second action in response to the second event that is determined to be occurring in area 100.The first radar and the second radar can both operate in the same band (e.g., K-band, Ku-band, Ka-band) or in the same range within the same band (e.g., between approximately 23 GHz and approximately 25 GHz, including these values). The first event and the second event may or may not be the same event, and the first action and the second action may or may not be the same action. The fields of view of the first radar and the second radar or sensor may or may not overlap. The first device 102 and the second device 102 can be configured such that the second processor communicates with the first processor (e.g., via a communication unit, Bluetooth unit, Wi-Fi unit) so that the first processor can verify or validate the first set of data based on the second set of data, determine whether object 104 is experiencing an event in area 100 based on the second set of data that verifies or validates the first set of data, and take action in response to the event determined to be occurring in area 100. The second processor can be configured to communicate directly (for example, housed in a common housing or having a common controller) or indirectly with the first processor.

[0078] As described herein, a radar can operate in the Ku-band, K-band, or Ka-band to track an object 104 within area 100 by emitting waves. These radio waves are a type of non-ionizing radiation, and exposure to high-frequency non-ionizing radio waves is regulated by the Federal Communications Commission and other international regulatory bodies by limiting "dose" in terms of the amount of energy absorbed by a body (e.g., object 104). Accordingly, in some embodiments, the radar's transmit power is programmable, and the radar can be made to comply with or configured to comply with all appropriate regulations in accordance with guidance on uncontrolled exposure to the general public (i.e., everyday use), which can occur via radio updates.

[0079] As described herein, radar can sense through typical walls found in ordinary houses (e.g., wooden or metal frames with drywall and siding, or wooden or metal frames with a pair of drywalls). The radar can be time-of-flight radar, Doppler radar, or other suitable type of radar. Time-of-flight radar can measure the time it takes for a pulse to travel to and from the reflecting object (e.g., echo), or it can enable direct observation of stationary objects. For example, typical household items (e.g., furniture, housing) are static, and if object 104 is an elderly or frail person, object 104 may also be slow-moving. Doppler radar can measure the movement of a reflecting object toward or away from the Doppler radar. When using Doppler radar, the faster the object is moving, the more something can be seen (e.g., more data can be obtained). In general, observation of humans relies on the argument that since humans are not always motionless, something about their presence is always detectable. However, this means that flapping fabric will be detected, which can be distracting. Stationary objects may be missed. Of these two radar modalities, time-of-flight radar is far better suited for observing the relatively slow-moving environment of a home. Naturally, with time-of-flight systems, motion can be seen by creating a “video” from still frames. Hybrid designs are also possible, as disclosed herein, that use Doppler radar to augment, supplement, substitute, enhance, or replace time-of-flight radar in detecting tremors, skin movements due to heartbeats, chest flexion due to breathing, or other vibrational motions (e.g., radar mode switching). Generally, because such vibrational motions are slow relative to the wavelength of the radar, the Doppler shift is often small, making detection difficult, but possible. However, high-frequency harmonics resulting from vibrational motion can be detected more easily. For example, if object 104 is a human, a heart beating at 60 beats per minute can generate characteristic harmonic vibrations of several hundred Hz.Doppler radar can detect these harmonics, and device 102 (e.g., a processor) can distinguish some or various vibrational movements from intentional movements. For example, this disambiguation may arise from detecting respiration, where a person's natural breathing rate is on the order of a few Hz. However, human movements such as swinging one's arms while walking occur with similar frequency. By detecting the high-frequency harmonics produced by respiration rather than arm swing, Doppler radar can gain a technical advantage by being able to detect respiration even when the person is moving. It should be noted that Doppler radar (and Doppler information) can be used for human surveillance as disclosed herein. It should be noted that some versions of existing Doppler radar products tend to have fairly limited performance in human surveillance tasks. These products may lose the ability to detect activity within about 5 feet (about 1.5 meters) of the source. Similarly, measuring heart rate and respiration rate requires standing still at a distance of about 1-2 feet (about 1.5-2.5 meters). Time-of-flight radar, Doppler radar, or any other suitable radar may be used.

[0080] As described herein, device 102 can be supplemented, enhanced, or augmented by data from line-of-sight sensing (e.g., cameras, LiDAR, radio frequency radar). For example, many technologies rely on a direct line of sight between the sensor and the object 104 being detected. Some optical camera-based, infrared-based, acoustic-based, or LiDAR-based solutions rely on line of sight. Some radio frequency radars (specifically, some versions of about 60 GHz or about 76 GHz to about 81 GHz) have very poor material penetration and are similarly substantially limited to line of sight, but can be used to supplement device 102. Some line-of-sight based systems have difficulty uniformly covering a typical indoor environment. Rooms are not rectangular, and furniture creates occlusion, posing a significant challenge. For example, a person who has fallen behind a table or counter is unlikely to be visible to a single indoor sensor. In contrast, the radars disclosed herein can directly track a person behind an obstacle. Furthermore, some line-of-sight technologies only work in one room, requiring the installation of units in each room and living space to provide adequate coverage. In addition to the various financial costs associated with these devices, such setups also incur professional installation costs, making the business model unsustainable. In contrast, device 102 can cover living spaces of up to approximately 2000 square feet (potentially more or more depending on various factors), operates through walls, can be extended with additional base units (e.g., another device 102 or sensor) for wider coverage (e.g., larger homes, multiple floors), or can provide coverage to enclosed adjacent spaces using "satellite" devices 102. Moreover, computer vision-based systems require people to accept the installation of cameras in their most private spaces, which is difficult to achieve (for example, a US user survey showed that less than 42% of older adults would accept having cameras installed in their bedrooms, and even lower in Europe).

[0081] Figure 12 shows an embodiment of the microphone set of the device of Figure 7 according to this disclosure. Figure 13 shows an embodiment of the microphone of the device of Figure 7 according to this disclosure. Figure 14 shows a raw reading from the devices of Figures 1 to 7, and an embodiment of a virtual skeleton formed by the devices of Figures 1 to 7 from the raw reading according to this disclosure. In particular, device 102 may include a microphone unit, as shown in Figure 5, and be controlled by a processor for various purposes. For example, one of these purposes is to supplement a set of data generated by radar and sent to the processor for processing (e.g., validation, verification, corroboration), as disclosed herein. For example, the microphone unit can be used to receive voice commands from object 104 in order to control a voice assistant (e.g., Siri, Google Assistant) running on the processor. The microphone unit may include a microphone or a set of microphones. The microphone unit is configured to be highly sensitive and robust, enabling accurate and satisfactory use, and is cost-effective while being scalable to support increasing functionality over time. The microphone unit provides a "voice-first" conversational user experience, enabling multi-party communication (e.g., from device 102 to device 102, or from device 102 to a remote telephone or computer), and, with user consent, provides rich context about people and their activities. In such a scenario, device 102 includes a corresponding speaker unit controlled by the processor.

[0082] Device 102 uses a microphone unit and a speaker unit to converse with object 104 (or its agent) for easy setup without requiring device 102 to download a dedicated app, respond to conversational voice commands, enable communication with loved ones, caregivers, and care providers (e.g., pre-programmed call numbers, chat names), and call object 104 for help when device 102 detects an imminent danger or object 102 requests help. However, it should be noted that other embodiments may include apps downloaded from app stores (e.g., Google Play, iTunes).

[0083] The microphone unit is used to capture sounds indicating a fall as determined by the processor, helping to confirm or validate sounds detected by the radar and reduce false positives. For example, if data, information, or instructions from the radar conflict with the microphone unit, the radar can be controlled as determined by the processor (however, this can be reversed, or the control default is not currently selected or available). The microphone unit can capture loud sounds such as collisions or glass breaking, which may indicate other safety-critical situations, intrusions, fires, or weather events as determined by the processor. The microphone unit can capture sounds from the kitchen and bathroom indicating various daily activities and self-care as determined by the processor. The microphone unit can capture voices that may indicate social activities or media consumption as determined by the processor. The microphone unit can capture airway sounds such as coughing, wheezing, shortness of breath, and snoring, which may indicate the onset or worsening of a lung disease as determined by the processor. The microphone unit can capture vocal tones that indicate heightened emotions (sadness, anger, frustration, happiness) or depressive tendencies, as determined by the processor. Over time, the microphone unit can learn from object 104 and capture additional vocal context associated with specific behaviors or symptoms of object 104.

[0084] The microphone unit may include an array of three (but two or more) digital microphones (but analog microphones are also possible). These microphones have a good signal-to-noise ratio and help device 102 hear commands and other sounds from as far away as possible (e.g., within approximately 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1 feet or meters inside a room). In addition to these microphones, the microphone unit subsystem may include a signal processing chip that provides 3D direction of arrival information for sound sourcing.

[0085] The microphone unit may include two different microphone processing chains, either of which can be performed by the microphone unit's processor. The first processor performs echo cancellation and speech separation in preparation for processing by a voice assistant. The second processor is a direction-of-arrival analysis processor that uses all three microphones to separate multiple sound sources and locate where these sounds originated within the monitoring area 100. Thus, the processor can localize sound sources (e.g., voices, falls) so that the processor (e.g., its edge AI algorithm) can correlate these sound sources or sounds with radar data or visual data during training. Similarly, the processor can use this sound information as location information to augment activity recognition. Likewise, the processor can also separate separate sounds. For example, by isolating background noise from a television or vacuum cleaner from human sounds, the processor's ability to hear commands and recognize sounds can be improved. Thus, if device 102 includes a set of microphones and a processor is coupled to and controls the set of microphones (e.g., receives data), the processor can be programmed to activate the set of microphones so that the set of microphones receives a set of acoustic inputs generated from an object 104 located within area 100, and transmits the set of acoustic inputs to the processor so that the processor isolates the set of acoustic inputs, locates where within area 100 the set of acoustic inputs originated from based on the isolation of the set of acoustic inputs, verifies or validates a set of data based on the verification or validation of the set of data by the set of acoustic inputs, and takes action in response to events determined to be occurring within area 100. Note that the microphone unit may have its circuit board mounted on top, as shown in Figure 12, but this is a requirement and the circuit board may be located elsewhere.

[0086] As described herein, device 102 may include a vision unit (e.g., an optical camera, an infrared camera) controlled by the processor for various purposes. For example, one of these purposes is to supplement a set of data generated by the radar and sent to the processor for processing (e.g., validation, verification, corroboration), as disclosed herein. For example, the vision unit can be used to receive gesture commands from object 104 to supplement the control of a voice assistant operating on the processor. For example, the vision unit may include a high-resolution (e.g., 4K, 8K) video camera (e.g., red, green, blue) which the processor can optionally enable as needed, and which can be enabled with the user's consent. This video camera may operate under various lighting conditions, and the processor may augment various sets of available data to enhance the training of various edge AI algorithms, particularly for identifying people via the radar subsystem. Device 102 may include, but is not required, the resulting edge AI algorithms, and object 104 (or its agent) may exclude or disable the vision unit as desired.

[0087] The video camera uses a wide-angle lens (a fisheye lens is also acceptable) to ensure a good view of the room in which the video camera is installed. Such a camera can be an integrated module including a sensor, lens, and supporting electronics. Note that imaging data can be preprocessed (e.g., dewarped) before image processing. The video camera can provide imaging data to the processor to complement the set of data the processor receives from the radar. The imaging data can be used to support training and validation regarding the position, posture, or identity of a person within the video camera's field of view. The processor can compare this imaging data with the set of data received from the radar or data received from the audio unit, at least within the area 100 in which the device 100 is positioned. Furthermore, the imaging data is also useful for recognizing everyday activities and training the radar before the radar is used by the object 104, or while the radar is tracking the object 104. There are also techniques to directly extract vital signs such as heart rate from the imaging data, which can be integrated for training various edge AI algorithms. Because the vision unit may be used under a variety of conditions (e.g., daytime, nighttime), the video camera may need to operate in lighting conditions ranging from full daylight to complete darkness. To meet this need with a single camera (although multiple cameras could also be supplied to the processor), the video camera may use a dual-band filter that can observe both the visible spectrum (red, green, blue) and near-infrared light. Device 102 may also include its own near-infrared light source for nighttime operation. There may be drawbacks to using this dual-band filter (e.g., mild distortion of red-green-blue colors), but as disclosed herein, this may be compensable and may not affect the various applications.Thus, if device 102 includes a camera (e.g., optical or infrared) and a processor is coupled to the camera (e.g., mechanically, electrically, or logically) to control the camera, the processor may be programmed (or instructed by the user) to activate the camera so that it receives images generated from living objects 104 within area 100, the processor verifies or validates a set of data, and based on the verification or validation of the set of data, takes action in response to events determined to be occurring within area 200. The camera may include a dual-band filter configured to observe in the visible spectrum (RGB) and near-infrared light.

[0088] As described herein, device 102 includes a processor (e.g., a controller, edge processor, single-core processor, multi-core processor, system-on-a-chip, graphics processing unit, hardware accelerator, neural network accelerator, machine learning accelerator) and radar (e.g., time-of-flight radar, Doppler radar), the processor being coupled (e.g., electrically, logically, mechanically) to the radar to control it (e.g., receive tracking data). For example, the processor can enable local computing or edge computing to improve processing speed or to provide data privacy or data security. For example, sensors (radar, acoustic, vision, etc.) can supply data (radar, acoustic, vision, etc.) to the edge processor. There are several reasons why this is technically advantageous. Firstly, since some, many, most, or all of the collected data (radar, acoustic, visual, etc.) hardly ever leaves device 102 (as is possible with cloud computing), it is possible to respect people's privacy. Secondly, this minimizes bandwidth usage and power consumption, thereby reducing costs. Thirdly, onboard processing minimizes latency, improves user experience and sensor platform responsiveness, and identifies events when they occur.

[0089] This processor enables an edge computing platform and may include single-core or multi-core processors, regardless of whether these cores are local or remote to each other within the monitored physical area. A multi-core processor may include multiple independent cores. For example, a multi-core processor is a computing component having two or more independent processing units, which are units that read and execute program instructions, such as those for a front-end application, via multiprocessing or multithreading. Program instructions are processing instructions such as addition, data movement, or branching, and the cores can execute multiple instructions simultaneously, thereby improving the overall operating speed of a front-end application suitable for parallel computing. The cores can perform parallel processing while adhering to the principles of indivisibility, consistency, independence, and persistence (ACID) when simultaneously accessing files or other data structures, as disclosed herein, ensuring that the operations / transactions (read, write, erase, etc.) of such data structures are processed reliably. For example, a data structure can be accessed simultaneously through at least two cores without locking the data structure. For example, figures and text can be processed simultaneously, as disclosed herein. It should be noted that a system can have 2, 3, 4, 5, 6, 7, 8, 9, 10, 12 cores, tens, hundreds, thousands, millions of cores, or more than 2 cores. Cores may or may not share a cache, and cores may or may not implement inter-core communication via message passing or shared memory. Common network topologies for interconnecting cores include buses, rings, 2D meshes, and crossbars. A homogeneous multicore system contains only identical cores, while a heterogeneous multicore system can have non-identical cores. The cores of a multicore system can implement architectures such as very long-instruction-word (VLIW), superscalar, vector, or multithreading.Whether additional or alternative, edge computing platforms may include graphics cards, graphics processing units (GPUs), programming logic controllers (PLCs), tensor core units, tensor processing units (TPUs), application-specific integrated circuits (ASICs), or other processing circuits, whether they are stationary or mobile platforms.

[0090] This processor can utilize the latest generation of edge processing chips to power advanced AI at the edge while balancing cost and performance. As shown in Figure 5, the processor can include both a high-power central processing unit (or another form of processing logic unit) and discrete ANN accelerators, enabling a variety of workloads. The processor may be paired with sufficient memory (RAM) and persistent storage to provide a balanced platform. For example, it may have 4, 6, 8, 12, 16, 32, 64, or 128 gigabytes of RAM or persistent memory (e.g., flash memory) or two or more memory modules.

[0091] As shown in Figure 5, device 102 may include a communication unit for partial or complete connectivity options (e.g., wired, wireless, waveguide). Internet (or LAN or WAN or network) connectivity can be provided via Wi-Fi or a cellular modem or other suitable communication device. The communication unit allows device 102 to function as a hub for multiple devices connected via either Bluetooth (e.g., low-power or other suitable personal network) or Wi-Fi (or Li-Fi or other suitable connection). To keep device 102 up-to-date, the communication unit supports over-the-air updates via either Wi-Fi or a cellular data connection (or other suitable connection). Similar to how Tesla updates its vehicles, device 102 is configured to support evolving and expanding use cases, unlocking the potential capabilities of each version or device 102 as various edge AI algorithms and software evolve, and delivering them transparently to the user. For example, this radar can cover a typical US drywall house of approximately 2,000 square feet (or less or more) and detect falls inside and through walls, detect respiratory rate and heart rate in a relatively still room at a distance of up to approximately 20 feet (or less or more), and continuously detect an individual's posture, such as sitting, standing, or lying down. Similarly, device 102 could include wearable biosensor patches (e.g., arms, torso, neck, legs) to provide tagged data (e.g., wirelessly) to further train and validate various edge AI algorithms for detecting heart rate, respiration, posture, and falls. Likewise, device 102 could include training AI to account for encountering various home configurations that may not have been anticipated in testing (e.g., for future proof against unforeseen scenarios)—this training could continue indefinitely into the future.

[0092] As described herein, the processor can use geofencing for a variety of purposes. Because radar can see through walls, it may see areas it should not see, such as a neighbor's property. To prevent this, the processor can be programmed to allow the user to set geofences over areas of interest or define areas of interest where data should be ignored. Object 104 (or its agent or user) can define the boundaries of the geofence. One way to do this is to notify the radar when it should measure the user's position when the user walks around the inner perimeter of area 100. Notification can occur via a computing device (e.g., a handheld unit, a wearable unit) having a button that the computing unit communicates with (e.g., a clicker) via wired, wireless, or waveguide, a mobile app that communicates with the radar (e.g., via a mobile phone or tablet), sound picked up by a microphone unit, or gestures captured by a vision unit. For example, device 102 may include a clicker (e.g., a handheld unit, a wearable unit) that communicates with device 102 (e.g., via a communication unit, a Bluetooth unit, a Wi-Fi unit) (e.g., via wired, wireless, or waveguide pairing), or a mobile application that can run on a mobile device (e.g., a mobile phone, a tablet computer) that communicates with a processor (e.g., via a communication unit, a Bluetooth unit, a Wi-Fi unit) (e.g., via wired, wireless, or waveguide pairing).Thus, the user can be instructed, or the processor can be programmed, to define areas based on (a) moving a clicker inside or outside area 100 (e.g., walking around area 100) and activating the clicker inside or outside area 100 (e.g., stopping and starting, or walking and starting, or walking when active), or (b) interacting with a mobile application located inside or outside area 100 (e.g., a mobile phone, a tablet computer) before the processor takes action in response to an event determined to be occurring within area 100.

[0093] Another method of geofencing is to record the shape and identification of rooms. Various rooms / locations within Area 100 are shown as part of setup or re-setup, calibration or recalibration, or onboarding or re-onboarding, through a mobile application running on a mobile phone or tablet computer communicating with the communication unit of device 102. Objects 104 can be guided by the mobile app to walk around using pressed buttons on the mobile phone or tablet computer and mark areas in Area 100. It should be noted that clickers may be used additionally or alternatively, and setup or re-setup, calibration or recalibration, or onboarding or re-onboarding may be guided by the user interface unit or speaker unit of device 102. Furthermore, similar approaches (e.g., walking using clickers or push-down buttons on mobile phones or tablets, or hand signals captured by a visual system or voice commands captured by a microphone unit) can also be used to mark boundaries for geofencing or blocking from surveillance or observation. Marked areas can be displayed as the shape or plan of a room as seen from radar and can indicate the field of view. Furthermore, there may be installer-based setups that include beacon devices (e.g., wireless beacons) for better tracking of room boundaries and fences. Device 102 may support a setup mode in which device 102 (e.g., a communication unit) listens for beacon transmissions of codes that match the setup during setup mode for better fencing accuracy.Thus, device 102 includes a beacon (e.g., housing) having a transmitter (e.g., radio) configured to transmit a signal (e.g., radio), device 102 includes a receiver (e.g., radio) configured to receive a signal, and a processor is coupled (e.g., mechanically, electrically, logically) to the receiver, and the transmitter can transmit a signal so that the radar and beacon are spaced apart from each other (e.g., within approximately 5, 4, 3, 2, 1 feet or meters), the receiver receives the signal, and the signal can be transmitted to the processor so that the processor determines, based on a set of data and the signal, whether an object 104 is experiencing an event in area 100, and takes action in response to an event that is determined to be occurring in area 100.

[0094] Another way to do this is to put the processor into geofencing mode during the setup process, and the processor may also assume that the user is moving only within the desired recording area 100. The user then moves freely within area 100, which is tracked by the radar. The processor records the extreme edges or corners of this movement (e.g., voxel-based processing). When the movement is complete, the user returns to device 102 and notifies device 102 that the movement is complete. The processor then performs additional processing on the data to determine the geofence extrema. Yet another way is to have the user carry a mobile app (e.g., a cell phone, tablet, or wearable) – which walks around area 100 – and this mobile app communicates with the processor via a communication unit, instructing the user (e.g., via a speaker, display, or via) how the user should walk around area 100 when the user is within and outside the radar range. This allows the radar to use the user as an active probe, and the mobile app can notify the user (e.g., via speaker, display, vibrator) if the user is in the shadow of the sensor or out of range of the radar (so that no movement within area 100 is detected), or to stay in a location within area 100 longer so that the radar can collect additional measurements to reduce sensing noise. Alternatively or in addition to the above, the user can walk inside or outside area 100 where the radar should not see. This could be outside area 100 (e.g., a house, home), as shown in Figure 1-2, and could help reinforce the boundaries that the radar should see, or it could help define specific areas that the user does not want the radar to see (i.e., for privacy reasons). This allows the user to define geofence-defined areas with holes (i.e., "polygons with holes").Alternatively, or in addition to the above, the user may be instructed to carry a large reflective object (e.g., a metal object, a mirror) while walking around area 100 to simplify the radar tracking the user. Furthermore, the user may be instructed to input the radar heights above the floor and below the ceiling within area 100, or the processor may assume these heights as default. The user may also provide or draw a map of area 100 on a computing device (e.g., a mobile phone or tablet touchscreen) and place the radar within the map for the processor of device 102 to access. Thus, the user may be instructed to define area 100, or the processor may be programmed to do so, before the processor takes action in response to an event determined to be occurring within area 100. This may occur based on the user moving within area 100 before the processor takes action in response to an event determined to be occurring within area 100. This may occur based on the user moving outside area 100 before the processor takes action in response to an event determined to be occurring within area 100. This may occur based on the user hosting a radar-trackable reflector before the processor takes action in response to an event determined to be occurring within area 100. This may occur based on the processor accessing a value corresponding to the height from the floor before the processor takes action in response to an event determined to be occurring within area 100. This may occur based on the processor accessing a map of area 100 before the processor takes action in response to an event determined to be occurring within area 100. The user can instruct the processor to create a map of area 100 (for example, on a personal computing device communicating with the communication unit of device 102) before the processor takes action in response to an event determined to be occurring within area 100.

[0095] As described herein, when processing the set of data received from the radar, the processor may ignore portions of area 100. This can occur at each beam position (also known as a stair) where the radar scans the field of view in a "mowing" pattern. Each scan position is called a stair. Each scan position returns a numerical list, where each element of the list represents a reflector at a certain distance from the radar. This scanning process results in a voxelized 3D map of the space formed by the processor. Voxels in this map can be zeroed if they are outside the geofenced area. If all voxels within a stair should be zeroed, the processor can adjust the scan pattern to avoid a particular stair in order to increase the detection speed of other scan areas. In background subtraction, once the 3D voxel map arrives at the processor, it can be compared to a voxel map representing areas that the radar should not record. Voxels that should not be recorded can be zeroed. After position detection, the set of data from the radar is processed by the processor, and after the position of the person / object is calculated, if the position of the person / object is outside area 100 (e.g., polygonal area), no further processing is performed on that person / object, and its position is not reported / stored. After machine learning, in order to calculate virtual skeleton keypoints of an object 104 that lives in or is located within area 100, the set of data from the radar is processed by the processor, and depending on a threshold of how many virtual skeleton keypoints are outside the permitted sensing area, as shown in Figure 14, virtual skeleton keypoints may not be reported. In this way, the processor can form a three-dimensional map of area 100 based on the set of data, such that the three-dimensional map has areas outside area 100 that are zeroed out based on radar and scans from inside area 100, before the processor takes action in response to an event that it has determined is occurring within area 100.The defined area may include a first volume of space and a second volume of space, and the processor may access a threshold associated with the first volume and be programmed to request the radar to adjust based on a zeroing region that satisfies the threshold so that the radar does not track the first volume in order to facilitate the radar tracking the second volume before the processor takes action in response to an event that it has determined is occurring in area 100. The processor may access a scan map of area 100 formed on a set of data and having a set of voxels, access a non-scan map of area 100 for the radar, compare the scan map with the non-scan map, identify a subset of voxels from the set of voxels, and zero out the subset of voxels before the processor takes action in response to an event that it has determined is occurring in area 100. A radar operating within area 100 may track an object 104 outside area 100, generate another set of data based on the tracking of the object 104 outside area 104, and send that other set of data to the processor so that the processor can determine whether the object 104 is outside area 100 and take another action (e.g., discard, remove, delete, or ignore that other set of data) in response to the determination that the object 104 is outside area 100. For example, as shown in Figure 14, the processor may form a three-dimensional skeletal model that simulates the object 104 based on the set of data, determine whether the object 104 is experiencing an event within area 100 based on whether the three-dimensional skeletal model meets or does not meet a threshold (e.g., matches a virtual skeletal signature for a given event), and take an action in response to an event determined to be occurring within area 100 based on whether the three-dimensional skeletal model meets or does not meet a threshold.For example, the processor forms a 3D region model that simulates region 100 based on a set of data, and a 3D skeleton model that simulates object 104 within the 3D region model based on a set of data, determines whether object 104 is experiencing an event within region 100 based on whether the 3D skeleton model within the 3D region model meets or does not meet a threshold (for example, by matching a virtual skeleton signature within the virtual model region to a given event), and takes action in response to the event determined to be occurring within region 100 based on whether the 3D skeleton model within the 3D region model meets or does not meet a threshold. The processor determines whether the three-dimensional skeletal model (with or without virtual model regions) meets a threshold based on a set of virtual movements of the three-dimensional skeletal model (e.g., joint movements, torso movements, head movements, arm movements, leg movements, neck movements, end effector movements), identifies a set of atomic movements of the three-dimensional skeletal model corresponding to the set of virtual movements (e.g., joint bending and straightening, elbow bending and straightening, leg bending and straightening, torso movement), and correlates the set of atomic movements to events.

[0096] As described herein, the processor can reset the geofence based on radar movement. The radar, in conjunction with accelerometers, gyroscopes, inertial measurement units, and geolocation units (e.g., GPS, GLONASS), detects its own movement and can prompt the user (e.g., via a mobile app communicating with a speaker unit, user interface unit, and communication unit) to rerun the geofencing routine. Alternatively, device 102 uses its sensors to determine how the radar has moved and updates the internal representation of the geofence. The processor can store a master voxel map representing the space (without human / object movement) during a preceding geofence measurement. This can be done by averaging voxel maps recorded during the previous geofence, or by explicitly detecting an object 104 and subtracting its influence from the voxel map. The processor then calculates the same voxel map at the new location and calculates a coordinate transformer to realign the master voxel map to the new voxel map. The learned transformer can be used to convert the original geofence information to the processor's new coordinate frame. Thus, the processor may access a movement threshold before taking action in response to an event determined to be occurring within area 100, access a geofence created by the user (for example, via a user interface unit or mobile app communicating with the communication unit of device 102) before taking action in response to an event determined to be occurring within area 100, and take another action, including a geofence, based on whether the movement threshold is met before taking action in response to an event determined to be occurring within area 100. That other action may include modifying the geofence, resetting the geofence to its default state, or initiating a user guide (for example, via a user interface unit or speaker unit, or via a mobile app communicating with the communication unit of device 102) to re-geofence.The other action could include determining how the radar moved within area 100 before the processor takes action in response to an event determined to be occurring within area 100, and modifying the geofence based on how the radar moved within area 100 before the processor takes action in response to an event determined to be occurring within area 100.

[0097] As described herein, the processor can adjust geofencing in response to changes in the environment (e.g., the purchase or construction of new space, or permission from an adjacent owner to view into the adjacent owner's space, or a reconsideration of a previous space owned by the user but excluded from sensing due to other concerns such as personal privacy or visitors, as shown in Figures 1-2). This can occur in various forms. The user can be presented with a map of their space formed based on a set of data from radar (e.g., via a user interface unit or mobile app communicating with the communication unit of device 102). The user can then delete or add spaces (e.g., via a user interface unit or mobile app communicating with the communication unit of device 102). The user can repeat the geofencing procedure described above. The user can notify the processor (e.g., via a user interface unit or mobile app communicating with the communication unit of device 102) that the user is adding or deleting spaces, and can repeat the geofencing procedure outlined above only for the new spaces that the user wishes to add or delete. Thus, the processor can access the geofence before taking action in response to an event determined to be occurring within area 100, and can modify the geofence before taking action in response to an event determined to be occurring within area 100, and in response to accessing user input (e.g., a user interface unit or microphone unit) indicating that the processor will modify the geofence.

[0098] As described herein, the processor functions to recognize activities of daily living such as excretion, dressing, eating, or other daily living activities. Recognizing these activities is straightforward if the radar knows how the space is designed. For example, handwashing is most likely to occur in the bathroom, and cooking is likely to occur in the kitchen. Or, eating may occur in the kitchen or dining room, but less likely in the bedroom. Thus, the processor may be programmed to identify functional spaces within area 100. Room identification can proceed as well as setting the geometry of the rooms or defining area 100 (e.g., geofencing). The user then explicitly labels each identified space (e.g., via a user interface unit or mobile app communicating with a communication unit of device 102). Labeling may be done by an app via a mobile phone or tablet operated by the user, by the user uttering the name of the room and a microphone unit recording the utterance, or by the user looking at a map generated by the processor (e.g., via a user interface unit or mobile app communicating with a communication unit of device 102) and drawing and defining the space. A processor can infer the use of a given space from several probabilistic pliers (statistics learned from a general population). For example, a person who remains relatively still in a horizontal position for several hours in the evening is likely sleeping, as inferred by the processor. The processor also infers that the surrounding area is likely a bedroom. In a more complex example, a certain type of radar return corresponds to a seated person, as inferred by the processor. The space where the person is usually seated, as inferred by the processor, could be a toilet, a favorite chair, or a dining table. If, as determined by the processor, the person is only sitting in that place for a short time, then, as inferred by the processor, the person is likely using the toilet, and the area immediately surrounding that activity is likely a toilet. Note that space does not need to be defined by walls, steps, or straight lines. Rather, space can be a more general concept of probability density.In other words, certain activities are more likely to occur in specific areas of the house, as inferred by the processor. Just as activities can be used to identify or infer the type of space inferred by the processor, knowing the common name of a space can be used as prior information for the processor to identify an activity. If a space is known to the processor to be a kitchen, then, as inferred by the processor, it is unlikely that someone is sleeping or using the toilet in that space, and it is more likely that someone is preparing a meal, opening the refrigerator, or eating. This type of inference can be solved with a Bayesian network. Thus, the user may be instructed or the processor may be programmed to assign identifiers (e.g., kitchen, bathroom) to sub-areas within area 100 so that the processor determines, based on a set of data and identifiers, whether an object 104 is experiencing an event in a sub-area, and takes action in response to an event determined to be occurring in a sub-area. Identifiers can be assigned to sub-areas via a user operating a communication unit (e.g., via a communication unit, Bluetooth unit, or Wi-Fi unit) so that the processor determines, based on the data set and identifier, whether object 104 is experiencing an event within the sub-area, and takes action in response to events determined to be occurring within the sub-area.If device 102 includes a microphone and a processor is coupled to the microphone (e.g., mechanically, electrically, or logically) to control the microphone (e.g., to receive data), the processor may assign identifiers to sub-regions by instructing the user (e.g., speaking) to output a sound corresponding to an identifier, so that the microphone takes in sound as an acoustic input and sends the acoustic input to the processor, which determines, based on the set of data and the identifier, whether an object is experiencing an event in the sub-region and takes action in response to the event determined to be occurring in the sub-region. The processor identifies sub-areas within area 100 based on the data set, infers the area type of the sub-area (e.g., kitchen, bathroom), classifies the sub-area based on the area type, and assigns identifiers (e.g., kitchen, bathroom). Based on the data set and area type, the processor determines whether object 104 is experiencing an event within the sub-area and assigns identifiers (e.g., kitchen, bathroom) to the sub-area based on the area type so that it takes action in response to the event determined to be occurring within the sub-area based on the identifier. The user can assign identifiers to area 100 (e.g., via a user interface unit, microphone unit, or mobile app communicating with the communication unit of device 102) so that the processor determines whether object 104 is experiencing an event within area 100 based on the data set and identifier and takes action in response to the event determined to be occurring within area 100.The user can assign an identifier to area 100 by operating a communication unit (e.g., mobile phone, tablet computer, wearable computer) that communicates with device 102 (e.g., wireless, waveguide, wired) so that the processor can determine whether object 104 is experiencing an event in area 100 based on a set of data and identifiers, and take action in response to events determined to be occurring in area 100. If the device includes a microphone and the processor is coupled to the microphone (e.g., mechanically, electrically, logically), the processor can transmit an acoustic input to the processor (e.g., via a user interface unit, speaker unit) that the user outputs (e.g., via a user interface unit, speaker unit) so that the microphone takes sound as an acoustic input, and the processor can determine whether object 104 is experiencing an event in area 100 based on a set of data and identifiers, and take action in response to events determined to be occurring in area 100.

[0099] As described herein, device 102 is positioned within area 100 for the radar to track objects 104 that reside within or are located within area 100. The radar can be installed anywhere within area 100, but preferred locations are corners or a predetermined distance from corners. This offers several technical advantages. For example, radar antennas can be made inexpensively from flat circuit boards, but some antennas positioned on flat circuit boards cannot physically achieve a 180-degree field of view (although some can). In practice, this field of view 130 will not significantly exceed approximately 120 degrees horizontally without a 3D antenna structure. Therefore, a radar positioned in a corner maximizes or ensures that the most accurate portion of the radar's field of view 130 is directed directly towards the area of ​​interest. Furthermore, radar operates by transmitting light in radio waves and measuring how the signal is reflected from objects in space. If radar is shone directly onto a wall, the wall acts as a prominent reflector, bouncing most of the radar's light back onto the radar and blinding it. In many houses, some, many, most, or all of the walls are built perpendicular to each other, so by placing the radar in a corner, the center of the radar's field of view 130 will be at an angle of approximately 45 degrees to most walls, significantly reducing the amount of radar energy reflected by the walls. Furthermore, the processor can be programmed to recommend where to position the radar. For example, after the radar has operated in area 100 for a predetermined time, the processor can know where object 104 spends most of its time in area 100. Based on the geofence area where the radar should operate and where people spend the most time, the processor can recommend an alternative position that reduces the distance to object 104 or reduces the number of obstacles between the radar and object 104.Thus, if area 100 has a corner, the processor can instruct the user (for example, via a mobile app communicating with the communication unit of device 102 via a user interface unit or speaker unit) to position device 102 at the corner or within a preset distance from a corner in area 100 containing object 104 (for example, within approximately 3, 2, or 1 foot or meter, or collinear with the corner, or such that window 214 or its central partition is at approximately 45 degrees to the corner). The processor may be programmed to generate recommendations on where to position or reposition device 102 or radar based on the data. The recommendations can optimize the minimum average distance between object 104 and radar in area 100, or the minimum number of obstructions or obstacles between object 104 and radar in area 100. The recommendations can be output to the user (for example, via a mobile app communicating with the communication unit of device 102 via a user interface unit or speaker unit).

[0100] As described herein, the processor is programmed to handle occlusion within the field of view 130. These occlusions induce shadows (e.g., coverage gaps), which are areas where radar tracking accuracy is reduced or absent. However, even in such situations, the processor can recover as much information as possible. Regarding the measurement and representation of shadows, while setting up geofencing, the reflectance of object 104 is recorded as object 104 moves within area 100, generating a 3D mean reflectance voxel map, where each voxel records the mean reflectance of object 104 at that point or that nothing was detected. One approach to this is to directly use the 3D mean reflectance model as a representation of the shadows. Voxels for which there are no direct measurements can take the reflectance value of voxels close to the radar. Another approach is to plot polar coordinates that approximate the shadows cast by occlusions. Such a frustration is defined by five planes. Four planes can be plotted, and each plane can include a point representing the center of the radar. One plane is orthogonal to the vector radiating from the center of the radar. Each plane can be calculated to align with the edges of a 3D mean reflectivity model, where the edges correspond to voxel transitions from high reflectivity to low reflectivity. Edge detection can be performed by a 3D Sobel edge detector (or other suitable technique). Furthermore, the processor can track occlusion. The processor can determine the likelihood that an object 104 has fallen behind an occlusion, as shown in Figure 14, by using guidance from observational information regarding the movement of a virtual skeleton (e.g., running, walking, crawling), the velocity of the center of mass of the blob or skeleton, the acceleration of the center of mass of the blob or skeleton, and the geometry of the room. The movement of the virtual skeleton can be recorded by recording the xyz positions of the skeleton keypoints over time, and the virtual blob is a primitive form of radar orientation that detects large reflectivity not attributable to a static object in a 3D voxel map of radar data, as shown in Figure 14. For example, as shown in Figure 14, the radar observes the virtual skeletal keypoint, velocity, and acceleration of a person entering and exiting the shadow within X time (e.g., approximately 15 seconds, approximately 30 seconds).While this may not necessarily require raising an alert, the processor can still calculate the position of the person in the shadow. However, if the person does not move out of the shadow within Y time (e.g., approximately 20 seconds, approximately 40 seconds), the processor can choose to suppress the warning if it has previously observed this behavior. Otherwise, the processor can confirm the person's safety with an elevator or escalation alert. For example, the processor can request a speaker unit to play a sound asking if the person is safe, the processor can request a communication unit to contact the person via telephone, or call / notify / message emergency contacts, or contact emergency services. The processor can handle occlusion based on manifestations of human motion models to infer activity patterns for prediction and classification. Three-axis spatial information, along with time-series timestamps, velocity, acceleration, rotation angle, and relative distance, represents the temporal and spatial displacement of body parts in motion and is used to effectively handle occlusion or shadows and to warn of events. Furthermore, as described herein, by using multiple devices 102, these shadows can be minimized by having non-overlapping fields of view 130, although overlap is possible, because their transmissions do not interfere with each other.

[0101] As described herein, device 102 may have radar housed in a single housing, or device 102 may have radar distributed across multiple housings, or multiple devices 102 may operate in interoperability or cooperation. For example, device 102 may include a transmitter or transmitter antenna in a first housing and a receiver or receiver antenna in a second housing, these housings may be spaced apart from the first housing. These configurations of distributing radar operation allow such antennas to be positioned at different locations within area 100, for example, by providing one or more transmitting antennas and one or more receiving antennas in separate housings. This diversity of locations broadens the field of view covered, improves the accuracy of object detection within area 100 because each transmitter / receiver pair has a different viewpoint, increases spatial information (e.g., more accurate measurement of various aspects of a person / object), and helps cover the shadows of the sensors. These antennas can be installed in a variety of ways. For example, there may be a set of transmitting antennas in one housing and another set of receiving antennas in another housing. For example, there may be a first transmitting antenna set and a first receiving antenna set in a first housing, and a second transmitting antenna set or a second receiving antenna set in a second housing. For example, device 102 may include any combination of receiving and transmitting antennas grouped in a housing, as long as the transmission of at least a portion of at least one transmitting antenna is received by at least one receiving antenna. In some cases, various considerations may be necessary. For example, operating multiple physically separated transmitting and receiving antennas within area 100 can present technical challenges. For instance, if these antennas are not in a common housing and sensing is performed at the speed of light, coordinating how the antennas and their associated processors must communicate becomes considerably more complex than when the transmitting and receiving antennas are wired to a common processor. Furthermore, interference may sometimes need to be considered.For example, each radar's transmitting antenna may need to operate in a way that does not interfere with other radars. There are several ways to do this. For example, each transmitting antenna can operate at a different frequency or bandwidth, or each transmitting antenna can transmit different digital patterns, such as transmitting different Goray complementary pairs. For example, each transmitting antenna can transmit at different times, so there are various ways to perform time synchronization. For example, the first and second processors of the first and second devices 102 can synchronize their clocks (for example, by a network time protocol via these communication units, designating one of the devices 102 to act as the master clock and communicating the time content to each slave clock using a second communication channel such as Z-Wave). In this way, each antenna can transmit within a preset window of time, each window of time is divided into short periods corresponding to a known synchronization error. For example, if each transmitting antenna is paired with a receiving antenna and shares a processor, the processor can preset the digital patterns for each radar so that each radar transmits in sequence. Each radar transmits a pattern upon receiving it from the preceding radar in sequence. Transmission begins with the first radar in the sequence transmitting the pattern, without waiting for the pattern to be received. Once all antennas have finished transmitting, the pattern loops. For example, if each transmitting antenna is paired with a receiving antenna and a processor is shared, a randomized backoff scheme can be used (e.g., like code division multiple access (CDMA) or Wi-Fi transmission).

[0102] As described herein, the radar can be installed in various locations within area 100 for various use cases. The radar may be positioned by electric field strength, and to comply with regulatory electric field strength limits, some transmitting antennas may be able to measure the power of the surrounding electric field and notify the user (e.g., via a user interface unit, speaker unit, or a mobile app running on a mobile phone or tablet communicating with the communication unit of device 102) when the electric field strength allows for the positioning of device 102. In one possible positioning scheme, a housing with a transmitting antenna is placed first and transmission begins. Subsequently, additional housings with receiving or transmitting antennas are placed. In these additional units, if the receiving antenna is paired with at least one transmitting antenna in the same housing, the receiving antenna can search for an electric field strength where the measured electric field strength and the expected electric field strength of the paired transmitting antenna do not exceed regulatory limits. For additional units, if the receiving antenna is not paired with at least one transmitting antenna in the same housing, the receiving antenna can measure the electric field strength and notify the user if the electric field is too weak to detect. The radar can be positioned by a blueprint, and a blueprint of area 100 can be provided to device 102, and the radar can be positioned to maximize coverage so as to be guided by a user interface unit or speaker unit, or a mobile app running on a mobile phone or tablet communicating with the communication unit of device 102. The radar can be positioned by guidelines (e.g., specific or general) which allow the antenna to be positioned using guidelines such as approximate separation distance, height from the ground, distance from the ceiling, and forward-facing direction facing inward into the house. The radar may also be positioned by a calculated blueprint / 3D map, and after or during this positioning process, each device 102 can calculate a blueprint based on a set of data from the radar: approximating the positions of floors, ceilings, and walls corresponding to large radar returns.The user can be presented with suggested placements or suggestions for improving placement based on a calculated diagram (guided, for example, by a user interface unit or speaker unit, or by a mobile app running on a mobile phone or tablet communicating with the communication unit of device 102). The radar can be positioned based on the observed user locations within area 100 after the radar has been initially positioned and observed several common locations of users within area 100. The radar can suggest new positions that better cover frequently used locations. Thus, the radar may include a transmitter and a receiver, and a processor is coupled to the transmitter and receiver (e.g., mechanically, electrically, or logically) to control them. Device 102 may include a first housing and a second housing, the first housing hosting the transmitter and the second housing hosting the receiver, and the first and second housings being spaced apart from each other (e.g., with an air gap between them, within approximately 5, 4, 3, 2, or 1 foot or meter). The first and second housings can be separated from each other before the processor takes action in response to an event determined to be occurring in area 100 based on the field strength of a transmitter in area 100, a received floor plan of area 100, general or specific guidelines for area 100, a general or specific wizard for area 100, a general or specific calculated floor plan for area 100, a general or specific map for area 100, or the observed location of an object 104 in area 100. The first and second housings may face each other or not, and may include facing corners. Furthermore, there may be a first device 102 (a first housing hosting a first radar having a first processor, a first transmitter, and a first receiver) and a second device 102 (a second housing hosting a second radar having a second processor, a second transmitter, and a second receiver), wherein the first radar does not interfere with the second radar.The first housing and the second housing may be spaced apart from each other, or they may be at opposite corners, the first transmitter is configured to transmit a first signal receivable by the first and second receivers, and the second transmitter is configured to transmit a second signal receivable by the first and second receivers. The first housing and the second housing can be spaced apart from each other based on the field strength of the first or second transmitter in the defined area, a received diagram of the defined area, general or specific guidelines for the defined area, general or specific wizards for the defined area, a general or specific calculated diagram of the defined area, a general or specific map for the defined area, or the observed position of an object in the defined area. The first processor and the second processor may or may not communicate with each other. The first radar and the second radar may or may not overlap in the field of view 130.

[0103] As described herein, there may be situations where device 102 requires synchronization of scan patterns. For multiple physically separated antennas to operate effectively as a single radar system, each transmit-receive antenna pair needs to know, to some extent, how the other antennas are steering their transmit or receive beams. Coordination may be necessary to build a consistent model of area 100. For example, if the purpose of the processor is to build a 3D voxel map of area 100 as disclosed herein, the receiving antenna can systematically scan area 100, but if the transmit antenna is not transmitting in the same direction as the receiving antenna is scanning, the receiving antenna may record nothing or fail to detect the reflectivity of the object because there was no transmit antenna transmitting a digital pattern scattered from the object. Thus, in some use cases, there may be one transmit antenna in one housing and one receive antenna in another housing. For each steer (each beam direction) of the transmit antenna, the receiving antenna performs a complete scan of area 100 (all possible beam steers). The coordination of when the transmitting antenna should switch beams and when the receiving antenna should scan area 100 can be communicated via a communication unit (e.g., Wi-Fi unit, Bluetooth unit, Zigbee unit, Z-wave unit, cellular unit). In another use case, a transmit / receive antenna pair in one housing faces another transmit / receive antenna pair in another housing. The transmit / receive antennas used by radar can also be used as a communication channel between two radars. For example, one transmitting antenna directs its beam in one direction and transmits a pattern. After receiving a certain number of patterns, the receiving antenna repeats the transmission of a different pattern. When the original antenna receives this alternative pattern, the original antenna stops transmitting the first pattern and begins receiving the alternative pattern. Physically separated radars need to know each other's beam steering patterns.This information can be communicated using a pre-configured radar antenna or by using a different communication strategy via a communication unit (e.g., Wi-Fi unit, Bluetooth unit, Zigbee unit, Z-wave unit, cellular unit). If one radar fails to receive a sufficient number of transmissions from the other radar within a pre-configured time, the radar can either resume the pattern using a different communication strategy, skip sections of the pattern where the other radar could not receive transmissions, or abandon cooperation and operate as an independent radar unit. Furthermore, it is possible to unify the data by further aggregating and processing the data collected from separate radars. For example, if a processor can detect a virtual skeleton using two radars, additional calculations can be used to determine whether that virtual skeleton represents the same physical person, and furthermore, as shown in Figure 14, the measured coordinates of the virtual skeleton can be averaged between the measurements of the two separate radars. Data unification can occur in a single housing where (a) all radars transmit measurement data to the same housing (via transmitting antennas or via other communication methods via communication units such as Wi-Fi units or Bluetooth units), (b) a cloud computing service where all devices 102 transmit measurement data to a cloud computing service, or (c) a distributed system where each device 102 transmits its data to a subset of other devices 102 for additional processing.

[0104] As described herein, device 102 can be initially calibrated or configured, or recalibrated or reconfigured. One way this can happen is based on device 102 receiving various information about the attributes of object 104 (e.g., skeletal length, heart rate, respiratory rate). As part of participant onboarding, object 104 is marked at a certain distance from the radar in predictable movements so that the processor can measure and mark skeletal length and center, normal gait pattern, normal gait speed, or other attributes in order to create a persona or identifiable profile (person attributes) of object 104. For example, object 104 may be guided to sit still in front of the radar unit for a few seconds to measure resting heart rate and respiratory rate as part of the object 104 profile (e.g., guided by a user interface unit or speaker unit, or a mobile app running on a mobile phone or tablet communicating with the communication unit of device 102). Heart rate is extracted by processing the radar signal in the time domain and using principal component analysis and other machine learning techniques to show phase variations attributable to the heart rate in the time-series data. Respiratory rate is measured by higher harmonic peak selection and other machine learning techniques. The recorded respiratory rate or heart rate can be used as a baseline signature of object 104 for subsequent learning. These vital signs are captured for continuous tracking based on the identification of object 104. Furthermore, the processor can enable voice recognition setup for object 104, and the processor can acquire a voice profile of object 104 and program it to voice-identify, distinguish, supplement, or reinforce object 104 from other objects 104 for radar tracking. The processor may also be programmed for the identification and tracking of residents within area 100. For example, the processor can identify participants from objects 104 tracked within area 100 and enable identification profiling of objects 104 by utilizing the object 104's profile when tracking vital signs (e.g., resting heart rate, gait pattern, resting respiratory rate, depression index).For example, the processor can identify a new object 104 (e.g., a new resident residing in or located in area 100) by retrieving some details of this new resident for future reference or to track one or more participants with the assistance of participants. In this way, the processor can access a set of attributes of object 104 before taking action in response to an event determined to be occurring within area 100, and can be programmed to create a profile (or persona) of object 104 based on the set of attributes so that the processor can determine, based on the set of data and profile, whether object 104 is experiencing an event within area 100. The profile can serve as a baseline for the processor to determine, based on the set of data, whether object 104 is experiencing an event within area 100. If device 102 includes a microphone and a processor is coupled to the microphone (e.g., mechanically, electrically, or logically), the processor outputs to object 104 (e.g., guided by a user interface unit or speaker unit, or a mobile app running on a mobile phone or tablet that communicates with the communication unit of device 102) (e.g., the microphone captures an acoustic input based on voice sound, the processor forms an audio profile of object 104, and determines, based on the set of data and the audio profile, whether object 104 is experiencing an event within area 100).

[0105] As described herein, the processor may request the radar to switch frequencies within the Ku-band, K-band, or Ka-band (for example, to increase the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation), or the processor may request the radar to switch between at least two of the Ku-band, K-band, or Ka-band (for example, the processor may also request the radar to switch modality between Doppler mode and time-of-flight mode (for example, to increase the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation). These requests can arise in a variety of ways.

[0106] For example, there may be multiple radars operating at different frequencies in a single housing. One radar is Ku-band for position and attitude detection of objects 104 within area 100, and another radar is Ka-band for vital detection of objects within area 100. Based on a set of data from the radar operating in the Ku-band, the processor can decide when to turn on or activate the Ka-band radar based on the position of a target (human or animal) within a known range where heart rate can be reliably detected. Alternatively, the processor can request the radar to shift carrier frequencies between the Ku-band, K-band, or Ka-band depending on the desired operating mode (e.g., position tracking in Ku-band, vitals in Ka-band, both = K-band), thereby saving power, controlling heat dissipation, and controlling other operating or tracking parameters (e.g., accuracy, precision, resolution). By using a set of high-frequency components (e.g., beamformers, up / down converters, oscillators), costs can be reduced at the expense of operating outside or near the frequency capabilities of these components. For radars that can shift frequencies within the Ku-band, K-band, or Ka-band, the high-frequency components have a desired operating range, and operating outside this frequency range may require more power to transmit at the same field strength. Which frequencies to use within the Ku-band, K-band, or Ka-band, or how to switch frequencies within the Ku-band, K-band, or Ka-band, can also be based on achieving the desired power usage profile. Furthermore, switching frequencies within the Ku-band, K-band, or Ka-band may be necessary if the high-frequency components are overheating due to being pushed to operate at frequencies close to or outside their design range.Thus, the radar may be configured to operate in the Ku-band or Ka-band, and the processor may be programmed to activate the radar so that the radar operating in the Ku-band or Ka-band within area 100 tracks an object 104 that is housed or located within area 100, generates a set of data based on tracking an object 104 that is housed or located within area 100 in the Ku-band or Ka-band, and sends the set of data to the processor so that the processor determines based on the set of data whether the object 104 is experiencing an event within area 100 and takes action corresponding to the event determined to be occurring within area 100. The processor may switch the radar between the K-band and the Ku-band or Ka-band based on parameters that meet or do not meet thresholds (e.g., increasing the resolution of the object 104 or area 100 or its contents, saving power, controlling heat dissipation). Note that the radars are configured to operate in parallel in the K-band and the Ku-band or Ka-band without interfering with each other. If there are at least two devices 102 operating in interaction or coordination within area 100, one of these devices 102 switches between the Ku-band, K-band, or Ka-band based on parameters that meet or do not meet a threshold (e.g., increasing the resolution of object 104 or area 100 or its contents, saving power, controlling heat dissipation), or parameters that meet or do not meet a threshold (e.g., increasing the resolution of object 104 or area 100 or its contents, saving power, controlling heat dissipation). These two radars may or may not be spaced apart within area 100 and may be located in the corners of area 100. As a result, these two devices 102 may or may not identify the same or different events, and may or may not take the same or different actions. Note that when tracking in the Ku-band, K-band, or Ka-band, the two radars may or may not operate in parallel, or may or may not interfere with each other.

[0107] For example, the radar may switch modality between Doppler mode (or another radar modality) and time-of-flight mode (or another radar modality) when requested by the processor, based on the processor determining whether various criteria, signatures, or thresholds are met or not (e.g., to increase the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation), as disclosed herein. Note that such switching may or may not operate in series or in parallel, may or may not interfere with each other, and may or may not be accompanied by frequency switching or band switching, regardless of whether the radar is operating in the Ku-band, K-band, Ka-band, or other bands, as disclosed herein. For example, the radar may have a first radar unit operating in Doppler mode and a second radar unit operating in time-of-flight mode, and the processor may request that the first radar unit operate in Doppler mode and then switch to the second radar unit to operate in time-of-flight mode, or vice versa, based on the processor determining whether or not various criteria, signatures, or thresholds are met (for example, parallel or serial radar mode operation is also possible, as disclosed herein, to increase the resolution of the object 104 or area 100 or its contents, or to manage power or heat dissipation). Note that the first and second radar units may be hosted by a common housing (e.g., internally and externally) as disclosed herein, or each may have its own housing that is spaced apart from one another (e.g., within about 5, 4, 3, 2, or 1 foot or meter).Similarly, for example, the radar can operate in parallel or serial radar mode, but can also operate in Doppler mode or time-of-flight mode, as disclosed herein, based on the processor's determination of whether various criteria, signatures, or thresholds are met or not (e.g., to increase the resolution of object 104 or area 100 or its contents, or to manage power or heat dissipation), the processor may require the radar to operate in Doppler mode and then switch to time-of-flight mode, or vice versa.

[0108] Various embodiments of this disclosure can be implemented in a data processing system suitable for storing and / or executing program code, including at least one processor directly or indirectly coupled to a memory element via a system bus. The memory element may include, for example, local memory used during the actual execution of the program code, bulk storage, and cache memory that provides temporary storage for at least some of the program code to reduce the number of times the code is retrieved from bulk storage during execution.

[0109] I / O devices (including, but not limited to, keyboards, displays, pointing devices, DASDs, tapes, CDs, DVDs, thumb drives, and other memory media) can be coupled to the system directly or via an intermediary I / O controller. Network adapters can also be coupled to the system to enable data processing systems to be coupled to other data processing systems or remote printers or storage devices via an intermediary private or public network. Modems, cable modems, and Ethernet cards are just a few of the types of network adapters available.

[0110] This disclosure can be embodied in a system, method, and / or computer program product. The computer program product may include a computer-readable storage medium (or medium) having computer-readable program instructions thereon for causing a processor to execute aspects of this disclosure. The computer-readable storage medium may be a tangible device capable of holding and storing instructions used by an instruction execution device. The computer-readable storage medium may be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage mediums includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, punch cards, and other mechanically encoded devices or grooved raised structures having instructions recorded thereon, and any suitable combination thereof.

[0111] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium or via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network, to an external computer or external storage device, to the respective computing / processing device. The network can consist of copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives computer-readable program instructions from the network and transfers the computer-readable program instructions for storage on a computer-readable storage medium within each computing / processing device.

[0112] Computer-readable program instructions for performing the operations of this disclosure may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microphone code, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk and C++, and conventional procedural programming languages ​​such as the C programming language or similar programming languages. A code segment or machine-executable instruction may represent a procedure, function, subprogram, program, routine, subroutine, module, software package, class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or hardware circuit by passing information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., may be passed, transferred, or transmitted via any suitable means, including memory sharing, message passing, token passing, network transmission, etc. Computer-readable program instructions may run entirely on the user's computer, or as a standalone software package, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be made to an external computer (e.g., via the Internet using an Internet service provider).In various embodiments, for example, an electronic circuit including a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) can be personalized by executing computer-readable program instructions using state information of computer-readable program instructions in order to perform an aspect of the present disclosure.

[0113] Aspects of this disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block in a flowchart and / or block diagram, and combinations of blocks in a flowchart and / or block diagram, can be implemented by computer-readable program instructions. Various exemplary logic blocks, modules, circuits, and algorithmic steps described in connection with embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clarify this compatibility between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in terms of their functionality. Whether such functionality is implemented as hardware or as software depends on the design constraints imposed on the particular application and the overall system. A person skilled in the art can implement the described functionality in various ways for each specific application, but such implementation decisions should not be construed as causing a departure from the scope of this disclosure.

[0114] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of the systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, which consist of one or more executable instructions for performing a specified logical function. In some alternative embodiments, the functions described in a block may occur in a different order than that shown in the figure. For example, two blocks shown consecutively may actually be executed substantially simultaneously, and depending on the functions involved, the blocks may be executed in reverse order. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented by a special-purpose hardware-based system that performs a specified function or action, or a combination of special-purpose hardware and computer instructions.

[0115] Words such as "then" and "next" are not intended to restrict the order of steps; these words are simply used to guide the reader through the explanation of the method. While a process flow diagram can describe operations as sequential processes, many operations can be executed in parallel or concurrently. Furthermore, the order of operations can also be rearranged. A process corresponds to a method, function, procedure, subroutine, subprogram, etc. If a process corresponds to a function, its termination corresponds to the function's return to the calling function or main function.

[0116] While various embodiments have been described and explained in detail herein, those skilled in the art will know that various modifications, additions, substitutions, etc., can be made without departing from the disclosure. Therefore, these modifications, additions, substitutions, etc., are considered to be within the scope of the disclosure. [Item 1] It is a method, Providing a device to a user, wherein the device includes a processor and a time-of-flight radar, the processor is coupled to the time-of-flight radar, the time-of-flight radar is configured to operate in the K-band, and the processor is programmed to allow the user to set geofences over or define areas of interest. Positioning the device within a defined area having an object located therein, activating the time-of-flight radar to operate in the K-band within the defined area, the time-of-flight radar operating in the K-band within the defined area tracking the object located within the defined area, generating a set of data based on tracking the object located within the defined area, transmitting the set of data to the processor, instructing the user to determine, based on the set of data, whether the object is experiencing an event within the defined area, to ignore the set of data outside the area of ​​interest, and to take action in response to the event determined to be occurring within the defined area, Includes, (i) The processor is programmed to enable the user to geofence the area of ​​interest or define the area of ​​interest based on the fact that the defined area has an inner circumference, the user is instructed to walk along the inner circumference, and the time-of-flight radar is notified of when the time-of-flight radar should measure the user's position, or (ii) The processor is programmed to enable the user to geofence the area of ​​interest or define the area of ​​interest based on the fact that the processor provides or draws a map of the defined area to a computing device, and the user is instructed to position the time-of-flight radar within the map so that the processor of the device can access it. [Item 20] The processor is programmed to reset the region of interest based on the fact that the time-of-flight radar has been moved after being positioned within the defined region where the object is located. The method described in item 1. [Item 22] The processor is programmed to adjust the region of interest in response to changes in the environment. The method described in item 1. [Item 28] It is a system, Equipped with a processor and a time-of-flight radar, A system in which the processor is coupled to the time-of-flight radar, the time-of-flight radar is configured to operate in the K-band, the processor is programmed to allow a user to set a geofence in or define a region of interest, the device is configured to be positioned within a defined region having an object therein, the time-of-flight radar operating in the K-band within the defined region tracks the object located within the defined region, generates a set of data based on tracking the object located within the defined region, transmits the set of data to the processor so that the processor determines, based on the set of data, whether the object is experiencing an event within the defined region, takes action in response to the event determined to be occurring within the defined region, and the processor is programmed to reset the region of interest based on the time-of-flight radar being moved after being positioned within the defined region where the object is located. [Explanation of symbols]

[0117] 102 devices 106 Stand 108 Sofa 110 Tables 112 chairs 114 Oven 116 Refrigerator 120 Toilets 122 TUB 124 sink 126 outlets 128 Wall 132 Wall 136 windows Doors 134 and 138

Claims

1. It is a device, A device comprising a housing that hosts a processor, an artificial intelligence (AI) accelerator, a time-of-flight (TOF) radar, and a communication interface, which can be placed within a defined area having an object inside, the housing being capable of (a) the TOF radar detecting the object based on its respiration rate, tracking the object within the defined area, generating a set of data based on tracking the object within the defined area, and transmitting the set of data to the processor, (b) the processor forming a three-dimensional skeletal model based on the set of data, (i) enabling the AI ​​accelerator to determine, based on the set of data and the three-dimensional skeletal model, whether the object is experiencing an event within the defined area, and (ii) enabling the communication interface to take action based on the AI ​​accelerator's determination that the object is experiencing the event within the defined area based on the set of data and the three-dimensional skeletal model.

2. The device according to claim 1, wherein the AI ​​accelerator capacity-determines, based on the set of data, whether the object has experienced the event within the defined area.

3. The TOF radar is the device according to claim 1, which operates in the K-band.

4. The device according to claim 1, wherein the object is a first object, the set of data is a first set of data, the TOF radar tracks a second object within the defined area, generates a second set of data based on the tracking of the second object within the defined area, and the processor can transmit the second set of data to the processor so that the AI ​​accelerator can distinguish the first object from the second object and determine, based on the set of data, whether the object is experiencing the event within the defined area.

5. The device according to claim 4, wherein the second object is a pet.

6. The device according to claim 1, wherein the housing can host a set of microphones and receive a set of acoustic inputs generated from the objects in the defined area in order to enable the processor to confirm or verify the set of data.

7. The device according to claim 1, wherein the set of data is a first set of data, (a) the TOF radar can track the object outside the defined area, generate a second set of data based on tracking the object outside the defined area, and transmit the second set of data to the processor, and (b) the processor is programmed to discard, remove, delete or ignore the second set of data.

8. The device according to claim 1, wherein the event relates to the object that remains stationary within the defined area for a predetermined period of time.

9. The device according to claim 1, wherein the event is related to the object being absent from the defined area for a predetermined period of time.

10. The device according to claim 1, wherein the event relates to the object not being tracked within the defined area for a predetermined period of time while the object is within the defined area.

11. The device according to claim 1, wherein the defined area has corners, and the processor is programmed to generate an output relating to where to position or reposition the housing before the action.

12. The device according to claim 1, wherein the event is a diagnostic estimation or diagnostic prediction.

13. The device according to claim 1, wherein the TOF radar has a field of view of approximately 120 degrees horizontally and approximately 90 degrees vertically.

14. The device according to claim 1, wherein the TOF radar includes a set of phased arrays, each having a set of patch antennas.

15. The device according to claim 1, wherein the processor is programmed to access a set of attributes of the object before the action and to create a profile of the object based on the set of attributes before the action, and the AI ​​accelerator determines, based on the set of data and the profile, whether the object is experiencing the event within the defined area.

16. The device according to claim 1, wherein the TOF radar is actively cooled.

17. The device according to claim 1, wherein the TOF radar is passively cooled.

18. The defined area is located inside a building, and the device is as described in claim 1.

19. The device according to claim 1, wherein the housing hosts at least one of an accelerometer, a gyroscope, a compass, a light sensor, a temperature sensor, a humidity sensor, or a particulate sensor.

20. It is a method, Users The method includes arranging a housing that hosts a processor, an artificial intelligence (AI) accelerator, a time-of-flight (TOF) radar, and a communication interface within a defined area having an object inside, and (a) the TOF radar can detect the object based on its respiration rate, track the object within the defined area, generate a set of data based on tracking the object within the defined area, and transmit the set of data to the processor. (b) The processor forms a three-dimensional skeletal model based on the set of data; (i) the AI ​​accelerator determines, based on the set of data and the three-dimensional skeletal model, whether the object is experiencing an event within the defined area; and (ii) the communication interface takes action based on the AI ​​accelerator's determination that the object is experiencing the event within the defined area.

Images