Detecting Abnormal Behavior in Intelligent Buildings

Abstract

A method and system for detecting abnormal behavior in an intelligent building are disclosed. A method includes: detecting the presence of a user at the intelligent building; retrieving the user's profile; monitoring the actions of the user in the intelligent building for each of a plurality of aspects; comparing the actions with the historical actions of the user stored in the profile; and determining that there is abnormal behavior for the user.

Description

Technical Field

[0001] Exemplary embodiments relate to the field of electronics. In particular, the present disclosure relates to methods and systems for detecting abnormal behavior in a smart building. Background Art

[0002] Today's technology has enabled the integration of new technologies into buildings that provide various benefits. For example, power consumption can be reduced by using smart technologies, as discussed in more detail in the US patent application entitled "Predicting the Impact of Flexible Energy Demand on Thermal Comfort" (serial number 62 / 644836). Smart building technology can provide optimization of energy use and improved usability for the tenants, owners, employees, and other users of the building. One way to provide improved usability is by detecting abnormal behavior. Brief Description of the Drawings

[0003] According to one embodiment, a method and system for detecting abnormal behavior in a smart building are disclosed. A method includes: detecting the presence of a user at a smart building; retrieving a user profile; for each of a plurality of aspects, monitoring the actions of the user in the smart building; comparing the actions with the historical actions of the user stored in the profile; and determining that there is abnormal behavior for the user.

[0004] In addition to one or more of the above features, or as an alternative, further embodiments may include where the profile includes a historical pattern of movement of the user within the smart building.

[0005] In addition to the above features, or as an alternative, further embodiments may include where determining the presence of abnormal behavior includes determining that the current movement pattern of the user is inconsistent with the historical movement pattern of the user.

[0006] In addition to the above features, or as an alternative, further embodiments may include where the profile includes access-granted privileges; and abnormal behavior includes an attempt to misuse the access-granted privileges.

[0007] In addition to the above features, or as an alternative, further embodiments may include where determining that there is abnormal behavior for the user includes: accessing the user's calendar; and comparing the calendar with the movement pattern of the user within the building.

[0008] In addition to the above features, or alternatively, further embodiments may include where the profile includes the user's preferences for one or more aspects; and the abnormal behavior includes the user implementing settings inconsistent with the profile for one or more aspects.

[0009] According to one embodiment, a method and system for detecting free-standing session user groups in a smart building are disclosed. A method includes: detecting the presence of more than one user at the smart building; determining the orientation and location of each user; determining one or more free-standing session user groups based on the orientation and location of each user; monitoring the interactions between and within the one or more free-standing session user groups; and tracking each free-standing session user group in real time.

[0010] In addition to the above features, or alternatively, further embodiments may include determining that there is abnormal behavior for one of the one or more free-standing session user groups.

[0011] In addition to the above features, or alternatively, further embodiments may include optimizing an emergency response plan based on the behavior of the one or more free-standing session user groups.

[0012] In addition to the above features, or alternatively, further embodiments may include where determining the free-standing session group includes looking for an O-space that includes a blank space surrounded by multiple users, where the multiple users orient toward the O-space.

[0013] In addition to the above features, or alternatively, further embodiments may include where monitoring the interactions between and within the one or more free-standing session user groups includes forming a graph representing each of the users within one of the free-standing session user groups; and determining an entropy or other global complexity measure for estimating the groupness associated with the graph.

[0014] In addition to the above features, or alternatively, further embodiments may include shaping it by transforming the graph into a topological object as a simplicial complex.

[0015] In addition to the above features, or alternatively, further embodiments may include using a persistent homology algorithm to analyze the simplicial complex.

[0016] In addition to the above features, or alternatively, further embodiments may include where determining the connectivity between simplicial complexes includes detecting transient groups and persistent groups in a circular pattern.

[0017] In addition to the above features, or as an alternative, additional embodiments may include monitoring the time evolution of a simplicial complex by using topological entropy (persistent entropy). BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The following description should not be regarded as restrictive in any way. Referring to the accompanying drawings, like elements are numbered alike:

[0020] Figure 1 is a flowchart illustrating the operation of one or more embodiments;

[0021] Figure 2 is a flowchart illustrating the operation of one or more embodiments;

[0022] Figure 3 is a block diagram of a computer system capable of performing one or more embodiments;

[0023] Figure 4 is a block diagram of an exemplary computer program product;

[0024] Figure 5 is a flowchart illustrating the operation of one or more embodiments;

[0025] Figure 6 is a block diagram illustrating the operation of one or more embodiments;

[0026] Figure 7 is a flowchart illustrating the operation of one or more embodiments;

[0027] Figure 8 is a flowchart illustrating the operation of one or more embodiments; and

[0028] Figure 9 is a schematic diagram of group formation. DETAILED DESCRIPTION

[0029] A detailed description of one or more embodiments of the disclosed devices and methods is presented herein by way of illustration and not limitation with reference to the accompanying drawings.

[0030] The term "about" is intended to include the degree of error associated with a particular quantity of measurement based on the available equipment at the time of filing the application.

[0031] The terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the disclosure. As used herein, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms as well. It will be further understood that the terms "comprises" and / or "comprising", when used in this specification, specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0032] Thermal comfort in an indoor location is achieved by using heating, ventilation, and air conditioning (HVAC) units placed throughout the indoor location. HVAC can be very expensive and accounts for up to 65% of a building's energy consumption.

[0033] In the past, there have been many different ways to control thermal comfort and thus the energy consumed to achieve a certain level of thermal comfort. A very approximate way of doing this was to manually control the air conditioning and heating units - turning them on and off as needed depending on whether the building's occupants were comfortable. Later, thermometers were added - if the temperature was sensed to be too high, the air conditioning system could be turned on to cool the room. If the temperature was sensed to be too low, the heating system could be turned on to warm the room. As technology became more sophisticated, additional methods were added.

[0034] Advances in technology have enabled the use of machine learning methods and systems to monitor and learn the thermal comfort levels of occupants. Using voting techniques, one or more embodiments can determine the comfort level of each occupant in a group of occupants. Thereafter, the thermal profile can be updated based on the feedback received.

[0035] In addition to the thermal profile, additional parameters may be stored in a complete profile. In one or more embodiments, machine learning methods and systems can be used to monitor and learn various categories of information (also referred to as "aspects"). The complete profile includes various information mined from historical data, which can be used to improve building efficiency and / or increase user convenience or experience. In one or more embodiments, five or more aspects can be mined in order to gain insights about the user in various different categories. These aspects are stored as tuples in a knowledge base.

[0036] Typically, a tuple can be formed by: the type of input data; the data preprocessing chain; and which computational model is used to process the input data. A smart building can access the tuple to customize the experience of each user in the smart building based on the mined data. The data is preprocessed and machine learning algorithms are used on the data to perform predictions on the data or to perform clustering on the data (to determine the similarity between users). A global score can be defined for each aspect and each user. Thereafter, the global scores can be compared and clustered among multiple users based on the similarity.

[0037] These aspects will now be described. Thermal comfort is described in more detail in the co-pending U.S. patent application entitled “Machine-Learning Method for Conditioning Individual or Shared Areas” (serial number 62 / 644813), the content of which is incorporated herein by reference.

[0038] To briefly explain the thermal profile, in a smart building, the concept of thermal comfort can be used to control the heating, ventilation, and air conditioning (HVAC) system. Thermal comfort uses various measurements of a room to estimate the thermal comfort level of the room. The measurements can include temperature, humidity, wind speed, etc. A field that estimates the thermal comfort level of the conditioned room can be generated. Using one of various different methods, a user can indicate whether the user is comfortable in the current thermal situation. If the user is too cold or too hot, the deviation of the user from the estimate can be stored in the thermal profile. The thermal profile will indicate, for example, that a particular user generally likes his room warmer or cooler than most people. The thermal profile can allow the smart building to sense or predict that a user has entered a particular room and adjust the thermal comfort level of the room when the user enters the room. This can even be done in advance if, for example, the user's calendar or reservation indicates that the user will be at a particular location at a particular time.

[0039] The use of such a thermal profile also has advantages in terms of building efficiency. If a user does not like a cold room, unnecessary use of the air conditioning system can be avoided. If a room is not being used, heating or conditioning of the room is not required if it is known that the room will be at a comfortable temperature when the room is occupied.

[0040] Another aspect that can be monitored and stored in the user profile is visual (or lighting) comfort. Visual comfort can include various different aspects of a room related to the user's vision. This can include lighting, blackout curtains, blinds, etc.

[0041] Regarding lighting comfort, which depends on the quantity and quality of lighting, each user may have a different comfort level. The quantity of lighting may include the amount of lighting measured, for example, in lumens using a light meter. Generally, some users may prefer a brighter environment than others. Some users may have poor night vision and thus prefer lighting that is brighter than that of other users. Other users may be sensitive to bright lighting and prefer less bright areas. The quantity of lighting may also include lighting from windows. Shades, blinds, and other window coverings can be controlled through a smart building (e.g., by using motorized window coverings) to provide the desired amount of lighting. Sensors can measure the amount of natural light in a room and adjust the brightness in the room based on the natural light. The quality of lighting may include various aspects of the type of lighting. Aspects such as the color temperature of the lighting can also be monitored and adjusted. For example, it may be found that a certain user prefers a natural color temperature (e.g., approximately 5000K) during the day and a "warmer" color temperature (e.g., approximately 2700K) in the evening. Then, when the user is in the room, the color temperature of the light can be adjusted to meet his preference.

[0042] Service interaction refers to the ways in which a user interacts with the various services provided by a building. For example, one user may use the elevator four times a day, and another user may use the elevator eight times a day. One user may prefer a particular cafeteria, while different users may use different cafeterias more frequently.

[0043] Service interaction data can be used in combination with data on movement patterns. Movement patterns refer to the areas of the building that a user utilizes. These movements can be tracked in one or more of a variety of different ways. For example, some buildings have access cards or key cards that utilize various technologies such as RFID or magnetic stripes, which enable the cardholder to access certain areas of the building. Additionally, some buildings are now adding access technologies to mobile electronic devices such as smart phones, tablets, MP3 players, e-readers, smart watches, health trackers, and any other type of device with computing capabilities. Then, those mobile electronic devices can be used to gain access to individual rooms. Other access authorization devices can use biometric information, such as fingerprint readers, facial recognition, retinal scans, and other biometric devices that rely on human characteristics to authorize access to rooms or areas of a building. Information regarding access to a room or area can be stored as movement patterns.

[0044] In addition, various sensors placed throughout the intelligent building can allow tracking of users as they move through the building. The sensors can be of any type. For example, face recognition algorithms can be used in combination with camera devices to determine when a user enters certain areas of the intelligent building. The user's mobile electronic device can be used in combination with wireless transmitters (such as Bluetooth, WiFi, Near Field Communication (NFC), ANT, and other wireless protocols). Signals can be sent through the wireless transmitter. When the mobile electronic device receives a signal, the mobile electronic device can transmit a response signal. The response signal can be associated with a specific mobile electronic device. Then, each mobile electronic device can be associated with a user. In this way, the movement of the user can be tracked to determine what areas of the building the user frequently enters and exits.

[0045] Another aspect is the health status. The health status can include any type of health information that is typically tracked using a mobile electronic device. For example, heart rate and body temperature can be tracked to determine if a user is ill. If the user is ill, the room where the user is located can be adjusted to improve the user's comfort.

[0046] These aspects can be combined with context information. The context information can be broadly classified into context information unrelated to people and context information related to people.

[0047] The context information unrelated to people includes information that is the same for each user within the building. Examples of context information unrelated to people include information about the building (such as the layout of the building, the materials of the building, the dimensions of the building, the orientation, etc.) and weather information (such as temperature, cloud cover, sunrise / sunset time, etc.).

[0048] Context information related to a person is information specific to a particular user. For example, the range of a user's visits can be part of this context information. Although the above usage anticipates a single user, where heat and lighting comfort are prioritized for the single user, there are often cases where there are multiple users in a room. In such cases, the heat and lighting comfort are set such that a greater number of users are within a particular comfort level. When determining the comfort level for a group of users, certain users can be prioritized so that their preferences are given greater weight. For example, a hotel may choose to prioritize the comfort of guests over that of employees. Thus, the status of a user as a guest or an employee can be considered as part of the context information related to a person. For a particular user, the status as a guest or an employee can change based on the context. For example, a user may be an employee at a hotel, but may be a guest at the same chain of hotels at a different location (e.g., on vacation). This status can also carry over to different enterprises. For example, the profile of a user at an office building can be shared with a hotel. Thus, when a user goes on vacation (or on a business trip) to a hotel, the user's preferences regarding heat comfort and lighting can be retrieved and used to make the user's stay at the hotel more enjoyable.

[0049] The various aspects described above can be combined with context information and used together to provide a better experience for the user and to improve the efficiency of the building. For example, based on the tracked movement patterns, a building can predict that a particular user will wake up at 6:00 am. The heat and lighting conditions of the room can be optimized for that user. Subsequently, based on the predicted user location, the area where the user has breakfast can be prepared in advance for the user. This can also be done in an office environment, where the meeting room can be prepared for the user even before the user enters the meeting room. In this way, the comfort conditions and preferences of the user can be discovered and anticipated. Additionally, the same profile can be shared among multiple buildings and used as a digital signature of the user across buildings. For example, a chain of hotels may have the profile of a user. When the user enters another hotel in the chain, the user's preferences regarding heat comfort, lighting comfort, etc. can be set for him, even if the user has never been to that particular hotel before, thus providing him with a consistent user experience. The sharing among buildings will be discussed in more detail below.

[0050] In some embodiments, for privacy reasons, any of the features listed above can be turned off. Although some users may appreciate the features that customize the user experience in a smart building, other users may value their privacy. In some embodiments, the user can turn off one or more of the tracking features at any time. For embodiments where the profile is shared among multiple smart buildings, the user can have the following options: turn on tracking in some buildings (e.g., the user's own home) and turn off tracking in public buildings (e.g., hotels).

[0051] In response to Figure 1 , a method 100 for operating one or more illustrated embodiments is presented. Method 100 is merely exemplary and is not limited to the embodiments presented herein. Method 100 may be employed in many different embodiments or examples not specifically depicted or described herein. In some embodiments, the procedures, processes, and / or activities of method 100 may be performed in the order presented. In other embodiments, one or more of the procedures, processes, and / or activities of method 100 may be combined, skipped, or performed in a different order. In some embodiments, method 100 may be executed by system 300.

[0052] Method 100 illustrates a process for creating a profile for an intelligent building. Sense a user (block 102). Sensing may be performed using one of a variety of different methods. For example, various different sensors may detect the presence of a user at a location. The sensors may include camera devices, audio sensors, card readers, wireless transducers that detect the presence of a mobile electronic device, and the like.

[0053] Thereafter, monitor the actions of the user (block 104). As described above, monitoring may include the movement of the user and the use of the building's facilities (e.g., rooms, elevators, restaurants, vending machines, etc.) by the user. In some embodiments, monitoring may include integration with the user's electronic calendar. For example, a user may use an electronic calendar system to maintain their calendar. One or more embodiments may be linked to the user's calendar (with the user's permission) to determine where the user will be, e.g., meeting or having a meeting in a specific conference room or office within a building complex.

[0054] In addition to monitoring, there may be a set of access control rules associated with the profile. The access control rules may be configured to permit or prohibit a user from accessing a certain set of resources. The resources may include actuators, controls, sensors, or commands. The resources may also include floors of the building or rooms within the building. Thus, the profile may indicate that some people (e.g., those engaged in maintenance work) have the right to access areas that are otherwise restricted to the general public. Similarly, the access control rules may indicate that a user whose apartment is on the twelfth floor of the building only has the right to access the twelfth floor of the building (and any common areas of the building).

[0055] For employees, the access control rules may indicate that the user is able to have the ability to change certain parameters that are not accessible to typical tenants. For example, security personnel may have access to elevator controls that are not available to general users of the building.

[0056] The access control rules may also include context information related to the user (such as the user's role or scope of visit) and context information unrelated to the user, such as information related to the physical layout of the building.

[0057] In some embodiments, access control rules can be applied in a given environment even when there are no environment-related instructions located within the user's profile. In such cases, alternative applicable rules can be identified by using the available context information. For example, if the user's context is a guest, the user might be configured to always be able to control the lighting and HVAC parameters in the guest room.

[0058] Monitoring can also include user feedback (block 106). User feedback can take the form of a human-machine interface (HMI) by which the user interacts with the intelligent building. Exemplary human-machine interfaces can include mobile electronic devices or wall-mounted terminals. An exemplary use of the HMI is that the user can use his mobile electronic device to indicate that he is too hot. The intelligent building will record the adjustments made with respect to the current thermal comfort level in the area where the user is located. Similarly, the user can make similar notifications regarding lighting preferences.

[0059] A profile is constructed using the collected data and user feedback (block 108). The profile can include information about each of the aspects described above and any other aspects that might be useful for the intelligent building.

[0060] The profile can be shared with other buildings (block 110). This can include other buildings within the same location (e.g., other buildings in an office park or university campus), related buildings (e.g., buildings operated by the same entity), or subscribers to the profile service.

[0061] After that, whenever the user enters a location with his profile, the profile can be retrieved (block 112), and the user's environment can be adjusted based on the information in the profile.

[0062] In a group setting (i.e., the user is located in a room or area with multiple users), the profiles for each user can be retrieved and analyzed as described above. The adjustments can be based on scores assigned to each user using machine learning techniques that take into account the similarities between the users (block 114).

[0063] For Figure 2 , a method 200 of operating illustrating one or more embodiments is presented. Method 200 is merely exemplary and is not limited to the embodiments presented herein. Method 200 can be employed in many different embodiments or examples not specifically depicted or described herein. In some embodiments, the procedures, processes, and / or activities of method 200 can be performed in the order presented. In other embodiments, one or more of the procedures, processes, and / or activities of method 200 can be combined, skipped, or performed in a different order. In some embodiments, method 200 can be executed by system 300.

[0064] In addition to the above, user habits can be mined from historical data and then used to improve the capabilities of the physical access control system. Mine the user's historical usage data from the historical data generated during user / building interactions (as detailed below for Figure 6 ). Usage data can be mined to determine context information for use with the access control system. In this case, the focus of profiling is to identify patterns in user habits. These patterns can include visited rooms, used elevators, used facilities, and used services.

[0065] The method can include an integration platform. The integration platform can be used to obtain the history of building-tenant interactions from heterogeneous building systems. Exemplary heterogeneous building systems include, but are not limited to, access control systems, camera devices, occupancy sensors, indoor positioning beacons, agenda information, and the structural floor plans and models of the building. The learned patterns are used to analyze access events (e.g., the use of a card reader or other access granting device).

[0066] Method 200 details an algorithm that can be used to determine potential security threats or other abnormal behaviors when analyzing access events. Method 200 assumes that a profile already exists. A combination of sensors, access granting devices, etc. is used to continuously monitor the user's location (block 202). As described above, this can include monitoring the user's mobile electronic device and the user's key card or other access granting device. The user's activities can be compared with the activities previously stored in the knowledge base (block 204). The user's activities can include the user's habits, including visited rooms, used elevators, utilized facilities and services, and other tracking aspects described above. If an anomaly is detected, a potential security threat is indicated (block 206). Thereafter, further investigation can be performed on the user's movements and actions (block 208).

[0067] If a user's credentials are used in an area where the user does not normally go or where the user has no access rights, this may indicate that a dangerous person has the credentials. For example, in the case of an office building, if a user only goes to the fifth and eighth floors, those trends can be stored in the user's profile in the knowledge base. If the user is accessing the tenth floor, this can be marked as unusual. This may not be an immediate alert because the user may have a good reason to be on the tenth floor when he normally does not go there. For example, an annual meeting may be held on the tenth floor. Or, the user may be delivering a package that he received by mistake.

[0068] Similarly, some configuration files include access permission privileges. A user may be permitted to visit certain rooms restricted by a key card or a mobile electronic device reader. However, the user may not be permitted to visit other rooms restricted by a key card or a mobile electronic device reader. If a user attempts to visit a room or area that he is not permitted to visit, an exception may be triggered.

[0069] However, using the user's credentials in an atypical manner may indicate that the user has lost his key card or mobile electronic device. Investigation can be carried out in one of various different ways. For example, the user's calendar (if the user has previously granted access) can be compared with the user's movement. If an annual meeting on the tenth floor is found in the calendar, the exception is explained and no further investigation is required.

[0070] In some instances, if the exception cannot be explained, the user's credentials can be further reviewed (block 210). In this way, each action of the user can be monitored more closely to ensure that the user is not a dangerous person.

[0071] For Figure 5 , a method 500 for illustrating the operation of one or more embodiments is presented. Method 500 is merely exemplary and is not limited to the embodiments presented herein. Method 500 can be employed in many different embodiments or examples not specifically depicted or described herein. In some embodiments, the procedures, processes, and / or activities of method 500 can be executed in the order presented. In other embodiments, one or more of the procedures, processes, and / or activities of method 500 can be combined, skipped, or executed in a different order. In some embodiments, method 500 can be executed by system 300.

[0072] Method 500 is a method for automatically measuring a user's satisfaction with a smart building. The user can indicate comfort or discomfort for each aspect measured. The comfort level of the user can be measured to discover uncomfortable situations and rank the uncomfortable situations. This can be useful for building management to repair or improve their building. Method 500 determines the user's satisfaction during a visit and can be configured to determine the satisfaction for each visit. Method 500 is used after a user profile has been established. Observe the actions of the user over a specific time period (block 502). The length of the time period can be set to any convenient length. In some embodiments, the length of time is approximately one week. Based on the user's profile, use machine learning techniques to generate a set of user expectations (block 504). Obtain the current environmental state of the building (block 506). The environmental state includes lighting comfort and thermal comfort. The environmental state can also include data about the building, such as the presence of faults and the current population of the building (how many people are currently occupying the building).

[0073] Measure the satisfaction of the user (block 508). This can be done using a human-machine interface (HMI). An exemplary HMI is a mobile electronic device. A software application (also known as an "app") can be executed on the mobile electronic device such as a smart phone, a tablet, or a smart watch. On the application, the user can record his satisfaction level. In some embodiments, the user can record details of why he feels his current satisfaction level.

[0074] The satisfaction level, the deviation from the user's expectations, and the current environmental state are recorded and stored in a knowledge base (block 510). Once such data has been obtained for multiple users, the data is analyzed and the generation of expectations is optimized (block 512). The analysis can include assigning weights to each aspect and other tracked information. For example, although crowding may not be an aspect, crowding may affect thermal comfort. Therefore, a weight can be assigned to crowding to predict how crowding affects the user's satisfaction level. Feature ranking methods can be used to rank each aspect. In this way, using data from each user, the aspects with the most information can be determined, such as the aspects that need the most improvement. Exemplary algorithms that can be used include the F-test and the mutual information algorithm.

[0075] Machine learning techniques can be used to optimize the solution for assigning weights to each condition. In some embodiments, a support vector machine can be used as a classifier. Once the system has performed the classification, corrective actions can be used to address the problems found. In some embodiments, the Learning Modulo Theory (LMT) method can be used as an optimization solver for determining weights.

[0076] Once the weights have been determined, the method 500 can be re-iterated to further refine the weights. The weights can be used to determine how the intelligent building should best respond to certain conditions. For example, certain levels of crowding may affect the user in an unforeseen way, meaning that the intelligent building should respond to certain environmental conditions in a different way than when the building is less crowded.

[0077] Cluster analysis can be used to find users with similar feelings. These clusters can be used to discover statistically significantly uncomfortable conditions. This can involve statistical tests such as the Kolmogorov-Smirnov test to compare the clusters and highlight statistically relevant differences.

[0078] In this way, an automated system for detecting and weighting uncomfortable conditions during user occupancy of a building is disclosed. The intelligent building can implement corrective actions to improve the user experience based on satisfaction-based measurements.

[0079] Figure 6The block diagram of system 600 for the purpose of mining and deploying user profiles for a smart building is depicted. First, a data collection phase is conducted. User 602 interacts with the smart building through various interfaces 610, 612, and 614. Although only three interfaces are shown in Figure 6 , it should be understood that a greater or fewer number of interfaces may be used. Interfaces 610, 612, and 614 represent any way in which user 602 can interact with the smart building. These may include the user's own mobile electronic device, a key card or other access card, an elevator call button, an access control device, a light switch, other traditional control systems (e.g., a thermostat), etc. Each of interfaces 610, 612, 614 may interact with a building service integration platform 616. The building service integration platform 616 serves as a link between each of interfaces 610, 612, and 614 and the actual services provided by the smart building. The services may include access control 620 (such as door locks and other entry control devices), elevators 622, HVAC 624, and lighting 626. It should be understood that lighting 626 may include not only control of light fixtures, but also control of window coverings (e.g., blackout curtains, blinds, etc.). Each interaction of user 602 with the smart building through the building service integration platform 616 is processed by an aspect manager 630. The aspect manager 630 may receive additional information (such as context information) from a knowledge base 640. As more and more data is collected for each user, the information processed by the aspect manager 630 is stored in a distributed profile repository 632.

[0080] The aspect manager 630 may collect event information and mine data through each aspect. Thereafter, an ideal machine learning algorithm may be determined for processing the data. This may be done using an iterative process, where a new machine learning algorithm is used for each iteration to determine which algorithm produces the best model. Thereafter, the selected aspect model is stored in the profile repository 632.

[0081] The second phase that may be conducted is the "adaptive formulation" phase. This phase applies the profile to each aspect. This phase uses system 600, which will be discussed in conjunction with Figure 7 the flowchart 700 of

[0082] A sensing event occurs (block 702). The occurrence can be an input by user 602 through one of interfaces 610, 612, or 614. Alternatively, it can be a sensor that acts as one of the interfaces detecting user 602. The event is filtered to determine if there are any relevant events (block 704). There may be instances where multiple interfaces detect the same event or related events. For example, going to a certain floor to enter a certain room can be considered a related event. Then, information is retrieved from the profile repository 632 (block 706) to determine which model to use based on the aspect and the user, and information is retrieved from the knowledge base 640 (block 708) to gather context information.

[0083] Then, the aspect manager 630 selects a model and executes the selected model (block 710) to recommend a course of action (block 712). Subsequently, the recommended course of action is executed through the building services integration platform 616. For example, the course of action can be to change the lighting, call the elevator, change the HVAC settings, etc.

[0084] Another feature of the above system is the ability to share profiles across multiple sites. The user profiles described above can be portable across different sites. This can include not only buildings owned or operated by the same entity (such as a university campus or an office park or a hotel chain), but also buildings owned or operated by different entities. In other words, a hotel chain can share configuration information with an office building, or a shopping mall, or an apartment building owned or operated by a different entity.

[0085] As described above, user profiles are created by fusing and coordinating data from one or more building system interfaces (e.g., interfaces 610, 612, and 614). The profile for each user is stored in the distributed profile repository 632. Each system and the associated building are described in the knowledge base 640 according to a shared conceptual structure. Each profile in the profile repository 632 is updated periodically based on the user's interaction with the systems operating in each visited building and is linked to the shared description (e.g., context information) stored in the knowledge base 640. In this way, each user's profile can be seamlessly applicable across numerous environments.

[0086] Afterwards, when a user enters an unvisited environment, the user's profile can be retrieved. The profile can include credentials, a historical log of the user's activities, user-related attributes, a collection of action / resource request templates, etc. The unvisited environment can be compared with the environments previously visited by the user and the context of the environments. Afterwards, based on the user's profile and the context of the places he has previously visited, the environment of the unvisited location can be adjusted to an estimated comfort level. For example, if a user has previously visited a hotel as a guest, different hotels can use that information to determine the appropriate environment for a hotel that the user has not visited before as a guest. Information about the user's preferences as an employee can be given less weight because those locations do not share the same context. The user's profile can be regarded as a "digital signature" that can enable advanced interfaces and provide an overall, cohesive, and personalized experience across different buildings and the systems therein.

[0087] In addition, access control policies can be issued by different authorities. The access control policy is used to represent in a declarative manner whether access is permitted or prohibited. The access control policy can also include context information related to the user and context information unrelated to the user.

[0088] The profile can include user authorization information that details access control rules that are portable among multiple buildings. Context information related to the user (e.g., whether the user is an employee or a guest) can be used to determine the user's access rights to the system. In addition, context information unrelated to the user (e.g., the size and shape of the building, the uses of various rooms, etc.) can be used to determine thermal comfort and lighting comfort.

[0089] There can be alternative applicable access control rules identified by the available context information. For example, it may always be allowed for a user to access the lighting and HVAC parameters of a hotel room where the user is a registered guest.

[0090] In some embodiments, it may be desirable to detect the user's intent for various advanced building applications. Exemplary applications can include exits, occupancy-based building control, and destination management systems. It may be desirable to model the behavior of groups in order to learn the user's intent while protecting the privacy of the user and detecting abnormal behavior.

[0091] In Figure 8 FIG. 800 is a flowchart showing such a use case. The environment is monitored using one or more sensors (block 802). The monitored environment can be a room inside a building or a public area near the building, etc. The environment can include several rooms. The sensors used can be any one sensor or a combination of sensors. Exemplary sensors can include camera devices, presence detection sensors, wireless transceivers, etc.

[0092] Detect people (block 804) within the monitored environment. A group of people can be sensed in one of various different ways. A group can be broadly understood as a social unit that includes a number of members with identities and relationships among each other. The types of groups analyzed can include a Focused Conversation Group (FCG). An FCG is the totality of co-present people who are participating in an ad-hoc encounter. They can be considered a focused encounter. Exemplary FCGs can include a party, a renovation session, or an office meeting.

[0093] An FCG can be detected by calculating a facing formation (also known as an F-formation) from the spatial location and orientation of the occupants. An F-formation is the proper organization of three social spaces, as Figure 9 illustrated. The O-Space 910 is a convex empty space surrounded by people (902) participating in the social interaction, where each participant faces inward towards the O-Space 910. There are no external people within the O-Space 910. The P-Space 920 is a ring surrounding the O-Space 910. The people within the FCG are located within the P-Space 920. The R-Space 930 is the space surrounding the P-Space 920 and is also monitored by the FCG participants.

[0094] Refer back to Figure 8 , and use the orientation and location of the users to determine the presence of one or more FCGs within the user group. Track the orientation and location of each user to find the O-Space (block 806).

[0095] Groups can change over time. Two groups of four people can become a group of five and a group of three, a single group of eight people, or any group among various different sizes. Additional people can join or leave the group when entering or leaving the monitored area. Groups can expand into subgroups, can merge, can disappear, and so on. Graph and topological methods can be used to determine these behaviors. Exemplary methods of calculating entropy include using the Von Neumann entropy equation to measure groupness over time and using persistent entropy to detect group fusion in circular patterns.

[0096] Each interaction within and between groups can be represented as a time - related network (block 808). This can be achieved using an undirected graph. A graph is a mathematical structure for modeling pairwise relationships between objects. A graph is an ordered pair G=(V, E) that includes a set V of vertices (or nodes or points) together with a set E of edges associated with two vertices. Using the Von Neumann entropy equation, entropy can be found as it relates to the graph. In addition to entropy, other global complexity measures may be determined, which can be used to estimate the groupness associated with the graph. Using persistent homology techniques, shapes can be clustered and higher - dimensional correlations that cannot otherwise be ascertained by classical statistical methods can be discovered. Thus, the graph can be transformed into a topological object as a simplicial complex (block 810). A simplicial complex is a set consisting of points, line segments, triangles, and their n - dimensional counterparts. Using a persistent homology algorithm, the simplicial complex can be analyzed to track the group over time (block 812). This can be achieved by determining the connectivity between simplicial complexes by detecting transient and persistent groups in circular patterns. In one or more embodiments, the temporal evolution of the simplicial complex can be analyzed using topological entropy (persistent entropy). Once the model has been deployed, it can be used for real - time analysis (block 814). In this way, abnormal behavior can be detected for groups and individuals (including but not limited to those abnormal behaviors discussed for Figure 2 those discussed). In other words, the intentions within individual groups and between groups can be mined. The mined information can be used to discover abnormal behavior within a group. Additionally, the mined information can be utilized to develop emergency response plans, such as ideal escape routes and potential problems on existing routes (block 816).

[0097] Figure 3 A high - level block diagram depicting a computer system 300 that can be used to implement one or more embodiments is shown. More particularly, computer system 300 can be used to implement the hardware components of a system capable of performing the methods described herein. Although one exemplary computer system 300 is shown, computer system 300 includes: a communication path 326 that connects computer system 300 to additional systems (not depicted); and may include one or more wide - area networks (WANs) and / or local - area networks (LANs), such as the Internet, an intranet, and / or a wireless communication network. Computer system 300 and the additional systems communicate via communication path 326, for example to transfer data between them.

[0098] The computer system 300 includes one or more processors, such as processor 302. The processor 302 is connected to a communication infrastructure 304 (e.g., a communication bus, a cross-over bar, or a network). The computer system 300 may include a display interface 306 that forwards graphical, text, and other data from the communication infrastructure 304 (or from a frame buffer not shown) for display on a display unit 308. The computer system 300 also includes a main memory 310 (preferably a random access memory (RAM)), and may also include an auxiliary memory 312. The auxiliary memory 312 may include, for example, a hard disk drive 314 and / or a removable storage drive 316, which represent, for example, a floppy disk drive, a tape drive, or an optical disk drive. The hard disk drive 314 may take the form of a solid state drive (SSD), a conventional disk drive, or a hybrid of the two. There may also be more than one hard disk drive 314 included within the auxiliary memory 312. The removable storage drive 316 reads from and / or writes to a removable storage unit 318 in a manner well known to those skilled in the art. The removable storage unit 318 represents, for example, a floppy disk, a compact disk, a magnetic tape, or an optical disk that is read from and written to by the removable storage drive 316. As will be appreciated, the removable storage unit 318 includes a computer-readable medium having computer software and / or data stored therein.

[0099] In an alternative embodiment, the auxiliary memory 312 may include other similar components for allowing a computer program or other instructions to be loaded into the computer system. Such components may include, for example, a removable storage unit 320 and an interface 322. Examples of such components may include: a program cartridge and cartridge interface (such as those found in video game devices); a removable memory chip (such as an EPROM, a Secure Digital card (SD card), a Compact Flash card (CF card), a Universal Serial Bus (USB) memory, or a PROM) and an associated socket; and other removable storage units 320 and interfaces 322 that allow software and data to be transferred from the removable storage unit 320 to the computer system 300.

[0100] The computer system 300 may also include a communication interface 324. The communication interface 324 allows software and data to be transferred between the computer system and external devices. Examples of the communication interface 324 may include a modem, a network interface (such as an Ethernet card), a communication port, or a PC card slot and card, a universal serial bus port (USB), etc. The software and data transferred via the communication interface 324 take the form of signals, which may be, for example, electrical signals, electromagnetic signals, optical signals, or other signals that can be received by the communication interface 324. These signals are provided to the communication interface 324 via a communication path (i.e., a channel) 326. The communication path 326 carries the signals and may be implemented using wires or cables, optical fibers, telephone lines, mobile phone links, RF links, and / or other communication channels.

[0101] In this description, the terms "computer program medium", "computer usable medium", and "computer readable medium" are used to refer to media such as main memory 310 and auxiliary memory 312, removable storage drive 316, and the hard disk installed in hard disk drive 314. The computer program (also known as computer control logic) is stored in main memory 310 and / or auxiliary memory 312. The computer program may also be received via the communication interface 324. Such computer programs, when executed, enable the computer system to perform the features discussed herein. In particular, the computer program, when executed, enables the processor 302 to perform the features of the computer system. Therefore, such computer programs represent the controller of the computer system. Thus, from the above detailed description, it can be seen that one or more embodiments provide technical benefits and advantages.

[0102] Now referring Figure 4 to, generally shown is a computer program product 400 including a computer-readable storage medium 402 and program instructions 404 according to an embodiment.

[0103] Embodiments may be systems, methods, and / or computer program products. The computer program product may include one or more computer-readable storage media having computer-readable program instructions thereon for causing a processor to perform aspects of the embodiments of the present invention.

[0104] A computer-readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer-readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing devices. A non-exhaustive list of more specific examples of the computer-readable storage medium includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device (such as a punched card or a raised structure in a groove having instructions recorded thereon), and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be understood as being a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse passing through an optical fiber cable), or an electrical signal transmitted through a wire.

[0105] The computer-readable program instructions described herein can be downloaded from the computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network such as, for example, the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the corresponding computing / processing device.

[0106] The computer-readable program instructions for carrying out the embodiments may include assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source code or object code written in any combination of one or more programming languages including object-oriented programming languages such as Smalltalk, C++, etc. and conventional procedural programming languages such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, an electronic circuit including, for example, a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) may execute the computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuit so as to carry out the embodiments of the present invention.

[0107] The embodiments may be implemented using one or more techniques. In some embodiments, a device or system may include one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the device or system to carry out one or more method acts as described herein. In some embodiments, various mechanical components known to those skilled in the art may be used.

[0108] The embodiments may be implemented as one or more devices, systems, and / or methods. In some embodiments, the instructions may be stored on one or more computer program products or computer-readable media such as a transient and / or non-transient computer-readable medium. The instructions, when executed, may cause an entity (e.g., a processor, a device, or a system) to carry out one or more method acts as described herein.

[0109] Although the present disclosure has been described with reference to one or more exemplary embodiments, those skilled in the art will understand that various changes may be made and equivalents may be substituted for its elements without departing from the scope of the present disclosure. Additionally, many modifications may be made to adapt a particular situation or material to the teachings of the present disclosure without departing from its basic scope. Therefore, it is intended that the present disclosure not be limited to the particular embodiments disclosed as the best mode contemplated for carrying out the present disclosure, but that the present disclosure will include all embodiments falling within the scope of the claims.

Claims

1. A computer-implemented method for detecting groups of independent session users in a smart building, comprising: Detecting the presence of more than one user in the smart building; Determining the orientation and spatial location of each user; Determining one or more groups of independent session users based on the orientation and spatial location of each user by calculating face formation from the orientation and spatial location of the users and by finding an O-space that includes a blank space surrounded by multiple users, wherein the multiple users are oriented towards the inside of the O-space; Monitoring the interactions between and within the one or more groups of independent session users, wherein the interactions are represented as a time-dependent graph; Using topological objects to track each group of independent session users in real time; Performing real-time analysis on the groups of independent session users in order to detect abnormal behavior for the groups of independent session users and individual users, thereby mining the intentions of individual groups of independent session users and between groups of independent session users; And Using the mined information to find abnormal behavior within a group of independent session users and to formulate an escape route.

2. The computer-implemented method according to claim 1, wherein: Monitoring the interactions between and within the one or more groups of independent session users includes determining an entropy or other global complexity measure for estimating groupness associated with the graph.

3. The computer-implemented method according to claim 1, further comprising: Using a persistent homology algorithm to analyze the topological objects, wherein each topological object is a simplicial complex.

4. The computer-implemented method according to claim 3, wherein: Analyzing the simplicial complex includes detecting transient and persistent groups in circular patterns.

Images