Systems and methods for mitigating false alarms in building management systems

Abstract

And giving an alarm. One of a plurality of generative artificial intelligence (AI) large language model-based autonomous master agents is activated based at least in part on an alarm type of an alarm, and performs an initial analysis of the alarm and creates a reasonable cause for the alarm. The corresponding autonomous main agent based on the generative AI large language model autonomously distributes each reasonable reason in the reasonable reasons to one or more autonomous sub-agents based on the generative AI large language model, and the one or more autonomous sub-agents based on the generative AI large language model execute analysis on the distributed reasonable reasons; and returning the result to the autonomous main agent based on the generative AI large language model, wherein the autonomous main agent based on the generative AI large language model classifies the alarm as a false alarm or a true alarm.

Description

Technical Field

[0001] This disclosure relates generally to building management systems, and more specifically to mitigating false alarms in building management systems. Background Technology

[0002] A building management system is a system for controlling and / or monitoring a building or other facility. A building management system may include, for example, HVAC systems, security systems, video management systems, access control systems, fire protection systems, and / or any other suitable building control system. In many cases, a building management system issues an alarm when an anomaly is detected in the building and / or during the operation of the building management system. Alarms typically must be acknowledged and / or otherwise resolved by the building's operators or other personnel. In some cases, a building management system may issue an alarm indicating a potential problem or difficulty is occurring, even if no such problem or difficulty is actually occurring in the building. These alarms may be referred to as false alarms. When a false alarm occurs, the operator typically needs to respond, which can waste a significant amount of the operator's time and divert their attention from actual, genuine alarms. What is desired is a method and system for automatically determining whether an alarm is a false alarm or a genuine alarm. Summary of the Invention

[0003] This disclosure relates generally to building management systems, and more specifically to mitigating false alarms in building management systems. An example may exist in a method for handling alarms in a building management system (BMS) of a facility, wherein the BMS includes multiple BMS components placed at known locations around the facility, and the multiple BMS components include multiple sensors. An exemplary method includes: receiving multiple alarms from the BMS, and normalizing each of the multiple alarms to a normalized alarm format, wherein the normalized alarm format includes at least an alarm type and an alarm timestamp. For at least some of the multiple alarms, one corresponding autonomous master agent based on a generative artificial intelligence (AI) large language model is activated, at least partially based on the alarm type of the corresponding alarm, wherein the corresponding autonomous master agent based on the generative AI large language model is trained using domain knowledge corresponding to the alarm type of the corresponding alarm. The corresponding autonomous master agent based on the generative AI large language model performs an initial analysis of the corresponding alarm and creates one or more initial scenarios for determining whether the corresponding alarm is a false alarm or a true alarm, wherein each of the one or more initial scenarios is associated with one or more scenario domains. The corresponding autonomous master agent based on a generative AI large language model autonomously assigns each initial scene from one or more initial scenes to one or more autonomous sub-agents based on a generative AI large language model, at least in part, based on one or more scene domains of the corresponding initial scene. Each of these autonomous sub-agents is trained using domain knowledge corresponding to the corresponding scene domain. Each autonomous sub-agent performs analysis of the assigned initial scene and returns the result to the autonomous master agent based on the generative AI large language model that assigned the initial scene to the corresponding autonomous sub-agent. The autonomous master agent based on the generative AI large language model receives results from each autonomous sub-agent based on the generative AI large language model, which is assigned a corresponding initial scenario from the autonomous master agent based on the generative AI large language model, and classifies alarms as false alarms or true alarms based at least in part on the received results.

[0004] Another example could exist in an alarm processing system for alarms in a building management system (BMS) for a facility, wherein the BMS includes multiple BMS components placed at known locations around the facility, and the multiple BMS components include multiple sensors. The system includes: an input / output terminal; and a controller operatively coupled to the input / output terminal. The controller is configured to receive multiple alarms from the BMS via the input / output terminal, wherein each of the multiple alarms has an alarm type. For at least some of the multiple alarms, the controller is configured to activate, at least partially based on the alarm type of the corresponding alarm, one of a plurality of generative artificial intelligence (AI) large language model-based autonomous master agents, wherein the corresponding generative AI large language model-based autonomous master agent is trained using domain knowledge corresponding to the alarm type of the corresponding alarm. The corresponding generative AI large language model-based autonomous master agent performs an initial analysis of the corresponding alarm and creates one or more plausible causes for the corresponding alarm, wherein each of the one or more plausible causes is associated with one or more corresponding domains among a plurality of domains. The corresponding autonomous master agent based on a generative AI large language model autonomously assigns each of the one or more rational reasons among a plurality of autonomous sub-agents based on a generative AI large language model, at least in part, based on one or more domains of the corresponding rational reasons. Each of the corresponding autonomous sub-agents based on a generative AI large language model is trained using domain knowledge corresponding to the corresponding domain of the rational reason. Each autonomous sub-agent based on a generative AI large language model performs analysis of the assigned rational reasons and returns the results to the autonomous master agent based on a generative AI large language model that assigned the rational reasons to the corresponding autonomous sub-agent. The autonomous master agent based on the generative AI large language model receives results from each autonomous sub-agent based on the generative AI large language model, which is assigned a corresponding reasonable cause from the autonomous master agent based on the generative AI large language model, and classifies alarms as false alarms or true alarms based at least in part on the received results.

[0005] Another example may exist in a non-transitory computer-readable medium storing instructions. When the instructions are executed by one or more processors, the one or more processors are caused to: receive multiple alerts from the BMS, wherein each of the multiple alerts has an alert type. For at least some of the multiple alerts, the one or more processors are caused to: activate, at least in part, one of a plurality of autonomous master agents based on a generative artificial intelligence (AI) large language model, a corresponding autonomous master agent based on the alert type of the corresponding alert, wherein the corresponding autonomous master agent based on the generative AI large language model is trained using domain knowledge corresponding to the alert type of the corresponding alert. The corresponding autonomous master agent based on the generative AI large language model performs an initial analysis of the corresponding alert and creates one or more plausible causes for the corresponding alert, wherein each of the one or more plausible causes is associated with one or more corresponding domains among a plurality of domains. The corresponding autonomous master agent based on a generative AI large language model autonomously assigns each of the one or more rational reasons among a plurality of autonomous sub-agents based on a generative AI large language model, at least in part, based on one or more domains of the corresponding rational reasons. Each of the corresponding autonomous sub-agents based on a generative AI large language model is trained using domain knowledge corresponding to the corresponding domain of the rational reason. Each autonomous sub-agent based on a generative AI large language model performs analysis of the assigned rational reasons and returns the results to the autonomous master agent based on a generative AI large language model that assigned the rational reasons to the corresponding autonomous sub-agent. The autonomous master agent based on the generative AI large language model receives results from each autonomous sub-agent based on the generative AI large language model, which is assigned a corresponding reasonable cause from the autonomous master agent based on the generative AI large language model, and classifies alarms as false alarms or true alarms based at least in part on the received results.

[0006] The foregoing summary is provided to facilitate understanding of the innovative features unique to this disclosure and is not intended to be a complete description. A full understanding of this disclosure can be obtained by considering the entire specification, claims, drawings, and abstract as a whole. Attached Figure Description

[0007] This disclosure will be more fully understood by taking into account the following description of various examples in conjunction with the accompanying drawings, in which:

[0008] Figure 1This is a schematic block diagram illustrating an exemplary system for handling alarms from a building management system (BMS);

[0009] Figure 2A and Figure 2B These are flowcharts that collectively illustrate exemplary methods for handling alarms;

[0010] Figure 3A and Figure 3B These are flowcharts that collectively illustrate a series of exemplary steps that can be executed by one or more processors that execute instructions stored on a non-transitory computer-readable medium; and

[0011] Figure 4A , Figure 4B and Figure 4C This is a flowchart that collectively illustrates the exemplary method.

[0012] While this disclosure is subject to various modifications and alternatives, its details have been shown by way of example in the accompanying drawings and will be described in detail. However, it should be understood that this disclosure is not intended to limit it to the specific examples described. Rather, it is intended to cover all modifications, equivalents, and alternatives that fall within the substance and scope of this disclosure. Detailed Implementation

[0013] The following description should be read with reference to the accompanying drawings, in which similar elements in the different drawings are numbered in the same manner. The drawings are not necessarily drawn to scale and depict examples that are not intended to limit the scope of this disclosure. While examples of various elements are illustrated, those skilled in the art will recognize that many of the examples provided have suitable alternatives that can be utilized.

[0014] This document assumes that all numbers are modified by the term “about” unless otherwise explicitly stated. Expressions of numerical ranges using endpoints include all numbers contained within that range (e.g., 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.80, 4, and 5).

[0015] As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” include plural references, unless otherwise expressly stated. As used in this specification and the appended claims, the term “or” is generally used in its meaning to include “and / or,” unless otherwise expressly stated.

[0016] It should be noted that references to "one embodiment," "some embodiments," or "other embodiments" in the specification indicate that the described embodiments may include specific features, structures, or characteristics, but each embodiment need not necessarily include that specific feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Additionally, when a specific feature, structure, or characteristic is described in conjunction with an embodiment, it is conceivable that, whether explicitly described or not, that feature, structure, or characteristic may be applied to other embodiments, unless otherwise expressly stated otherwise.

[0017] Figure 1 This is a schematic block diagram illustrating an exemplary system 10 for processing alarms from a building management system (BMS) 12 of a facility. The BMS system 12 may include, for example, an HVAC system, a security system, a video management system, an access control system, a fire protection system, and / or any other suitable building control system. The BMS 12 includes multiple BMS components 14 placed at known locations around the facility. In some cases, the BMS components 14 include multiple sensors, such as, but not limited to, window opening sensors, door opening sensors, glass break detectors, motion detectors, fire sensors, smoke sensors, gas sensors, temperature sensors, humidity sensors, etc. Sensors may include cameras and associated video analytics algorithms. In some cases, some or all sensors may be battery powered. The BMS components 14 can detect conditions from which one or more alarms 16 may be generated. Alarms 16 may be provided to the system 10. As should be understood, some of the alarms 16 may be false alarms, meaning that no difficulty has actually occurred and it may simply be a sensor malfunction or other condition, and some of the alarms 16 may be true alarms, meaning that a difficulty has occurred. System 10 processes alarm 16 to help determine which alarms in alarm 16 are false alarms and which are true alarms.

[0018] The exemplary system 10 includes an input / output port 18 through which the system 10 receives alarms 16 from the BMS 12. In some cases, the input / output port 18 can be used to transmit communications, including commands, to the BMS 12. A controller 20 is operatively coupled to the input / output port 18. In some cases, the controller 20 may be considered to include multiple generative artificial intelligence (AI) large language model master agents 22 and multiple generative artificial intelligence (AI) large language model (LLM) sub-agents 24. In some cases, one or more of the generative AI large language model master agents 22 and / or one or more of the generative AI large language model sub-agents 24 may be stored locally. In some cases, one or more of the generative AI large language model master agents 22 and / or one or more of the generative AI large language model sub-agents 24 may be stored in a remote or cloud-based server (not shown) communicating with the controller 20.

[0019] Controller 20 is configured to receive multiple alarms from BMS 12 via input / output 18, each of the multiple alarms having an alarm type. Sometimes, the alarm type corresponds to and / or originates from the type of sensor that detected the corresponding alarm condition. For at least some of the multiple alarms, controller 20 is configured to activate, at least in part, one of the multiple generative artificial intelligence (AI) large language model-based autonomous master agents 22 corresponding to the alarm type of the corresponding alarm 16, where the corresponding generative AI large language model-based autonomous master agent 22 is trained using domain knowledge corresponding to the alarm type of the corresponding alarm. For example, if the alarm type is a security system alarm type, the corresponding generative AI large language model-based autonomous master agent 22 can be trained using domain knowledge corresponding to security systems. In some cases, the corresponding generative AI large language model-based autonomous master agent 22 performs an initial analysis of the corresponding alarm 16 and creates one or more plausible causes for the corresponding alarm 16, where each of the one or more plausible causes is related to one or more corresponding domains among multiple domains. In some cases, the corresponding autonomous master agent 22 based on a generative AI large language model autonomously assigns each of the one or more rational reasons among the rational reasons to one or more autonomous sub-agents 24 of a plurality of autonomous sub-agents based on a generative AI large language model, at least in part, based on one or more domains of the corresponding rational reasons. Each of the corresponding autonomous sub-agents 24 is trained using domain knowledge corresponding to the corresponding domain of the rational reason. In some cases, each of the autonomous sub-agents 24 performs analysis of the assigned rational reasons and returns the results to the autonomous master agent 22 that assigned the rational reasons to the corresponding autonomous sub-agent 24. In some cases, the autonomous master agent 22 based on the generative AI large language model receives results from each of the autonomous sub-agents based on the generative AI large language model 24, which is assigned a corresponding reasonable cause from the autonomous master agent 22 based on the generative AI large language model, and classifies the alarm 16 as a false alarm or a true alarm based at least in part on the received results.

[0020] In some cases, one or more autonomous sub-agents based on generative AI large language models 24, when performing analysis of assigned reasonable causes, autonomously assign one or more sub-tasks to one or more other autonomous sub-agents based on generative AI large language models 24, at least in part, based on the domain of the sub-task and the corresponding domain knowledge on which the autonomous sub-agents based on generative AI large language models 24 are trained. In some cases, the autonomous master agent 22 based on generative AI large language models can report confidence scores when classifying an alert as a false alarm or a true alarm. In some cases, the autonomous master agent 22 based on generative AI large language models can report the reasoning behind classifying an alert as a false alarm or a true alarm (e.g., a text-based description of reasoning). In some cases, one or more of the autonomous master agents 22 based on generative artificial intelligence (AI) large language models and / or one or more of the autonomous sub-agents 24 based on generative AI large language models can collect data from one or more of the multiple BMS components 14 of BMS 12, and the autonomous master agent 22 based on generative AI large language models can report the collected data that supports classifying alarm 16 as a false alarm or a true alarm.

[0021] In some cases, one or more BMS components among the multiple BMS components 14 may include one or more video management system (VMS) components of the BMS12 video management system (VMS), and one of the autonomous sub-agents 24 based on a generative AI large language model may be a video management system (VMS) analyzer sub-agent trained using domain knowledge related to the BMS12 video management system (VMS). In some cases, one or more BMS components among the multiple BMS components 14 may include one or more fire detection components of the BMS12 fire detection system, and one of the autonomous sub-agents 24 based on a generative AI large language model may be a fire detection analyzer sub-agent trained using domain knowledge related to the BMS12 fire detection system. In some cases, one or more BMS components among the multiple BMS components 14 may include one or more security system components of the BMS12 security system, and one of the autonomous sub-agents 24 based on a generative AI large language model may be a security system analyzer sub-agent trained using domain knowledge related to the BMS12 security system. In some cases, one or more BMS components among the multiple BMS components 14 may include one or more HVAC components of the heating, ventilation, and / or air conditioning (HVAC) system of BMS 12, and one of the autonomous sub-agents 24 based on a generative AI large language model is an HVAC analyzer sub-agent trained using domain knowledge relevant to the HVAC system of BMS 12. These are just examples. In some cases, an autonomous master agent and / or an autonomous sub-agent based on a generative AI large language model trained in one domain (e.g., a fire detection system) may collect information and / or assign tasks to one or more autonomous master agents and / or autonomous sub-agents based on a generative AI large language model trained in one or more different domains (video management system, HVAC system).

[0022] Figure 2A and Figure 2BThis is a flowchart illustrating an exemplary method 26 for alarm processing (such as alarm 16) of a building management system (BMS) (such as BMS 12) for a facility, wherein the BMS includes multiple BMS components (such as BMS component 14) placed at known locations around the facility, and wherein the multiple BMS components include multiple sensors. In some cases, one or more of the multiple BMS components may include one or more video management system (VMS) components of the BMS, and one of the autonomous sub-agents based on a generative AI large language model may be a video management system (VMS) analyzer sub-agent trained using domain knowledge related to the BMS's video management system (VMS). In some cases, one or more of the multiple BMS components may include one or more fire detection components of the BMS's fire detection system, and one of the autonomous sub-agents based on a generative AI large language model may be a fire detection analyzer sub-agent trained using domain knowledge related to the BMS's fire detection system. In some cases, one or more BMS components among multiple BMS components may include one or more security system components of the BMS's security system, and one of the autonomous sub-agents based on a generative AI large language model may be a security system analyzer sub-agent trained using domain knowledge related to the BMS's security system. In some cases, one or more BMS components among multiple BMS components may include one or more HVAC components of the BMS's heating, ventilation, and / or air conditioning (HVAC) system, and one of the autonomous sub-agents based on a generative AI large language model may be an HVAC analyzer sub-agent trained using domain knowledge related to the BMS's HVAC system. These are just examples.

[0023] The exemplary method 26 includes: receiving multiple alerts from the BMS, as indicated in box 28; normalizing each of the multiple alerts into a normalized alert format, wherein the normalized alert format includes at least a normalized alert type and a normalized alert timestamp, as indicated in box 30; and for at least some of the multiple alerts, activating, at least in part, one of the multiple autonomous master agents based on a generative artificial intelligence (AI) large language model (such as autonomous master agent 22 based on a generative AI large language model) among the multiple autonomous master agents based on the corresponding alert, wherein the corresponding autonomous master agent based on a generative AI large language model is trained using domain knowledge corresponding to the alert type of the corresponding alert, as indicated in box 32.

[0024] The corresponding autonomous master agent based on a generative AI large language model performs initial analysis on the corresponding alarm and creates one or more initial scenarios to determine whether the corresponding alarm is a false alarm or a true alarm, wherein each of the one or more initial scenarios is associated with one or more scenario domains, as indicated in box 34. The corresponding autonomous master agent based on a generative AI large language model autonomously assigns each of the one or more initial scenarios to one or more autonomous sub-agents based on a generative AI large language model (such as autonomous sub-agent 24 based on a generative AI large language model) at least in part based on the one or more scenario domains of the corresponding initial scenarios. Each of the corresponding autonomous sub-agents based on a generative AI large language model is trained using domain knowledge corresponding to the corresponding scenario domain, as indicated in box 36. Each autonomous sub-agent based on a generative AI large language model performs analysis of the assigned initial scene and returns the results to the autonomous master agent based on a generative AI large language model that assigned the initial scene to the corresponding autonomous sub-agent based on a generative AI large language model, as indicated in box 38.

[0025] continue Figure 2B An autonomous master agent based on a generative AI large language model receives results from each autonomous sub-agent based on a generative AI large language model, which is assigned a corresponding initial scene from the autonomous master agent based on a generative AI large language model. The master agent classifies alerts as false or true alerts, at least in part based on the received results, as indicated in box 40. In some cases, when performing analysis of the assigned initial scene, one or more autonomous sub-agents based on a generative AI large language model may autonomously assign one or more sub-tasks to one or more autonomous sub-agents based on a plurality of generative AI large language model-based autonomous sub-agents, at least in part based on the domain of the sub-task and the domain knowledge trained on the corresponding autonomous sub-agent.

[0026] In some cases, method 26 may further include: the autonomous master agent based on the generative AI large language model reporting a confidence score when classifying an alert as a false alarm or a true alarm, as indicated in box 42. In some cases, method 26 may further include: the autonomous master agent based on the generative AI large language model reporting the reasoning behind classifying an alert as a false alarm or a true alarm (e.g., a text-based description of the reasoning), as indicated in box 44. In some cases, one or more of a plurality of autonomous master agents based on the generative AI large language model and / or one or more of a plurality of autonomous sub-agents based on the generative AI large language model may be configured to collect data from one or more of a plurality of BMS components of the BMS. In some cases, the autonomous master agent based on the generative AI large language model may report the collected data that supports classifying an alert as a false alarm or a true alarm. In some cases, one or more autonomous master agents and / or one or more autonomous sub-agents based on generative artificial intelligence (AI) large language models among multiple autonomous master agents may change the operation of one or more BMS components among multiple BMS components (e.g., turn on, turn off, change mode, change speed, change setpoint, change schedule, etc.) in response to classifying an alarm as a true alarm, and may not change the operation of one or more BMS components among multiple BMS components in response to classifying an alarm as a false alarm, as indicated in box 46.

[0027] Figure 3A and Figure 3BThis is a flowchart illustrating a series of exemplary steps 48 that can be executed by one or more processors executing instructions stored on a non-transitory computer-readable medium. For example, one or more processors may be part of controller 20. One or more processors may be made to receive multiple alarms from the BMS, each of the multiple alarms having an alarm type, as indicated in box 50. For at least some of the multiple alarms, one or more processors may be made to: activate, at least in part, a corresponding autonomous master agent based on a generative artificial intelligence (AI) large language model among multiple autonomous master agents based on the alarm type of the corresponding alarm, wherein the corresponding autonomous master agent based on the generative AI large language model is trained using domain knowledge corresponding to the alarm type of the corresponding alarm, as indicated in box 52. The corresponding autonomous master agent based on the generative AI large language model performs an initial analysis of the corresponding alarm and creates one or more plausible causes and / or one or more plausible contributors to the corresponding alarm, wherein each of the one or more plausible causes / contributors is associated with one or more corresponding domains among multiple domains, as indicated in box 54. The corresponding autonomous master agent based on a generative AI large language model autonomously assigns each of the one or more rational causes / contributors to one or more autonomous sub-agents based on a generative AI large language model, at least in part, based on one or more domains of the corresponding rational cause / contributor. Each of the corresponding autonomous sub-agents is trained using domain knowledge corresponding to the corresponding domain of the rational cause, as indicated in box 56. Each autonomous sub-agent performs analysis on the assigned rational cause / contributor and returns the results to the autonomous master agent that assigned the rational cause / contributor to the corresponding autonomous sub-agent, as indicated in box 58.

[0028] continue Figure 3BAn autonomous master agent based on a generative AI large language model receives results from each autonomous sub-agent based on a generative AI large language model, which is assigned a corresponding reasonable cause / contributor from the autonomous master agent based on the generative AI large language model. The master agent classifies alerts as false or true alerts, at least in part based on the received results, as indicated in box 60. In some cases, when performing analysis of the assigned reasonable cause / contributor, one or more autonomous sub-agents based on the generative AI large language model may autonomously assign one or more sub-tasks to one or more autonomous sub-agents based on the domain of the sub-task and the domain knowledge trained on the corresponding autonomous sub-agent, as indicated in box 62. In some cases, the autonomous master agent based on a generative AI large language model reports the confidence score when classifying an alert as a false alarm or a true alarm, as indicated in box 64. In some cases, the autonomous master agent based on a generative AI large language model can report the reasoning behind classifying an alert as a false alarm or a true alarm, as indicated in box 66.

[0029] Figure 4A , Figure 4B and Figure 4C This is a flowchart illustrating exemplary method 68. Method 68 begins with alarm triggering, as indicated in box 70. In some cases, alarm triggering may include indication of alarms from one or more of an access system, a VMS (Video Management System), an intrusion detection system, and / or various third-party systems, as indicated in box 72. The alarm import module normalizes the incoming alarms to a common format and extracts certain alarm features, including the alarm type and alarm timestamp, as indicated in box 74. Next, the incoming alarms are received and processed, as indicated in box 76. In some cases, importing and processing may include normalizing alarm data (e.g., converting all temperature values ​​to Fahrenheit, converting all time entries to 24-hour clock values, etc.), as indicated in box 78.

[0030] Method 68 Figure 4B The process continues, where analysis of the alarm occurs, as indicated in box 80. This may include alarm routing mechanisms, as indicated in box 82. This may include, for example, identifying sensor type and / or alarm type. A master agent is accessible, such as one of the generative AI large language model-based autonomous master agents 22, as indicated in box 84. In some cases, control proceeds to box 86, where initial analysis is performed. In some cases, control may optionally (or additionally) proceed to... Figure 4C Box 100 is above, where information integration occurs. Control moves from box 86 to box 88, where it is determined whether further analysis is appropriate. In some cases, a sub-agent may be activated, such as the autonomous sub-agent based on a generative AI large language model in box 24, as indicated in box 90. In some cases, control may move to vision-based analysis, as indicated in box 92, and / or to context-based analysis (other sensor data), as indicated in box 94. Vision-based analysis at box 92 and / or context-based analysis at box 94 can access data and / or domain knowledge from data knowledge sources, box 96. From vision-based analysis at box 92, control may move to... Figure 4C Box 100 above. From the context-based analysis at box 94, controls can be switched to... Figure 4C The box above is 100.

[0031] Figure 4C It begins with collaborative reasoning and decision-making, as shown in Box 98. This may include information integration as indicated in Box 100, generative AI reasoning as indicated in Box 102, and decision-making and reporting as indicated in Box 104. Control then moves to action and confirmation, as shown in Box 106. If the alarm is determined to be a false alarm, it can be automatically logged and confirmed without operator input, as indicated in Box 108. If the alarm is determined to be genuine and not a false alarm, the operator is alerted, as indicated in Box 110.

[0032] To illustrate this with a concrete example, in the first scenario, an alarm indicating "Unauthorized personnel have intruded into a specific area" is triggered on the facility. In this example, the alarm is issued by an intrusion video analysis algorithm of the Video Management System (VMS). The following steps can be used to process this alarm scenario.

[0033] 1. Alarm triggered

[0034] Event: VMS detected an unauthorized person in a restricted area.

[0035] Source: An alert was issued by intrusion analysis from the VMS system.

[0036] 2. Alarm introduction and initial processing

[0037] Alarm import module:

[0038] Normalization: The raw alert data from VMS is normalized, including:

[0039] Timestamp: The precise time of the event.

[0040] Location: The specific area where the intrusion was detected.

[0041] Sensor type: Video analytics.

[0042] Severity level: High, due to unauthorized intrusion.

[0043] Initial description: A textual description from VMS analysis indicating an unauthorized intrusion.

[0044] 3. Agent activation and initial scene generation

[0045] Main Agent

[0046] Role: A generative AI model that acts as the central processing unit for alerts.

[0047] Initial scene generation:

[0048] The main agent analyzes the type and source of events (VMS).

[0049] It creates one or more initial scenes, such as:

[0050] Unauthorized intrusion by an individual is known.

[0051] False alarms caused by sensor malfunction.

[0052] Environmental factors that affect sensor accuracy.

[0053] Alert routing: Based on the initial scenario, the main agent distributes alerts to one or more relevant sub-agents.

[0054] 4. Sub-agent activation and analysis

[0055] VMS Proxy (Video Management System Proxy)

[0056] activation :

[0057] The main agent activates the VMS sub-agent, which is trained to handle video analytics alerts.

[0058] Domain knowledge :

[0059] VMS sub-agents possess detailed domain knowledge of video analytics, camera placement, typical intrusion patterns, and historical data.

[0060] Data collection :

[0061] VMS sub-agents interact with the live VMS to collect additional information, such as:

[0062] Video feed: Real-time and recorded feeds from surrounding cameras.

[0063] Movement patterns: Analyze the intruder's movements.

[0064] Facial recognition: Try using facial recognition to identify people.

[0065] Scene update :

[0066] The initial scene is updated based on the new video evidence, and a confidence level is assigned to each scene.

[0067] Activation of additional sub-agents

[0068] Based on the findings of the primary agent and / or sub-agents, update the confidence level of existing scenarios and / or create new scenarios if necessary, and assign the new scenarios to the appropriate sub-agents.

[0069] Environmental monitoring agency :

[0070] Trigger: Based on the updated scenario, the main agent or sub-agent determines the need for environmental data.

[0071] Data collection:

[0072] Data is collected from environmental sensors (e.g., temperature, humidity, motion sensors) to detect abnormal conditions.

[0073] Scene Update:

[0074] It provides its findings, leading to further updates on the scenario and its confidence level.

[0075] 5. Dynamic collaboration and decision-making

[0076] Collaborative Reasoning :

[0077] The main agent integrates information from all activity sub-agents.

[0078] The scene is dynamically updated as new evidence is collected.

[0079] If any scenario requires additional information, the main agent or sub-agent will trigger further sub-agent activation.

[0080] Generative AI Inference :

[0081] Scenario refinement: Continuously refine scenarios based on new data.

[0082] Probability allocation: Adjusting the probability of each scenario based on the overall evidence and the knowledge base of generative AI.

[0083] Decision-making and reporting :

[0084] Classification: Determine whether the alarm is a false alarm or an actual threat.

[0085] Detailed report: Use the following to compile the report:

[0086] Key evidence from video feeds, access logs, and environmental data.

[0087] Reasoning for this decision.

[0088] Confidence level in decision-making.

[0089] Recommended actions (e.g., scheduling safety, ignoring alerts).

[0090] 6. Actions and Confirmations

[0091] False alarm :

[0092] Log recording: Record events for future analysis and model training.

[0093] Self-calibration: Updates the model's understanding of the normal baseline to reduce future false alarms.

[0094] Potential threats :

[0095] Warning Operator: Utilizes compiled reports to send detailed alerts to human operators.

[0096] Immediate Response: Provides operators with actionable insights to respond quickly to threats.

[0097] In another example, an alarm indicating "a fire has been detected in the storage area" is triggered. This alarm is issued by a combination of smoke detectors, thermal sensors, and video analytics. The following steps can be used to handle this alarm scenario.

[0098] 1. Alarm triggered

[0099] Event: A fire was detected in the storage area.

[0100] Source: The alarm was issued from multiple sources:

[0101] Smoke detector: Detects smoke in the area.

[0102] Thermal sensor: detects rapid increases in temperature.

[0103] Video analytics: Detecting visual signs of fire or smoke.

[0104] 2. Alarm introduction and initial processing

[0105] Alarm import module:

[0106] Normalization: Raw alarm data from different sensors is normalized, including:

[0107] Timestamp: The precise time of the event.

[0108] Location: The specific storage area where the fire was detected.

[0109] Sensor types: smoke detectors, thermal sensors, video analytics.

[0110] Severity level: Severe, due to potential fire hazard.

[0111] Initial description: A textual description of the fire detection indication from the sensor.

[0112] 3. Agent activation and initial scene generation

[0113] Main Agent

[0114] Role: A generative AI model that acts as the central processing unit for alerts.

[0115] Initial scene generation:

[0116] The main agent analyzes the type and source of alarms (smoke detectors, thermal sensors, video analytics).

[0117] It creates an initial scene, such as:

[0118] An actual fire occurred in the storage area.

[0119] False alarms caused by sensor malfunction.

[0120] Environmental factors that trigger correct or incorrect identification (e.g., dust or steam).

[0121] Alert routing: Based on the initial scenario, the main agent directs alerts to the relevant sub-agents.

[0122] 4. Sub-agent activation and analysis

[0123] Fire detection agency

[0124] activation:

[0125] The primary agent activates the fire detection agent, which is trained to handle fire-related alarms.

[0126] Domain knowledge:

[0127] Fire detection agents possess detailed domain knowledge of fire detection sensors, typical fire behaviors, and historical data.

[0128] Data collection:

[0129] The fire detection agent interacts with the real-time system to collect additional information, such as:

[0130] Sensor readings: Current and historical readings from the smoke detector and thermal sensor.

[0131] Video feed: Real-time and recorded feeds from cameras in the storage area.

[0132] Scene Update:

[0133] The initial scene is updated based on new sensor and video evidence, and a confidence level is assigned to each scene.

[0134] Activate additional sub-agents when necessary.

[0135] Environmental monitoring agency:

[0136] Trigger: The fire detection agent or master agent determines the need for environmental data.

[0137] Data collection:

[0138] Data is collected from environmental sensors (e.g., humidity, dust levels, air quality) to detect anomalies.

[0139] Scene Update:

[0140] The findings are provided to the master agent, which updates the confidence level of existing scenarios and / or creates new scenarios as necessary.

[0141] Maintenance Agent:

[0142] Trigger: Based on update scenarios, the main agent or sub-agent determines the need for maintenance data. Data Collection:

[0143] Query the maintenance system for the most recent maintenance logs and report any recent work performed on the fire detection system or in the storage area.

[0144] Scene Update:

[0145] It provides its findings, leading to further updates on the scenario and its confidence level.

[0146] 5. Dynamic collaboration and decision-making

[0147] Collaborative Reasoning:

[0148] The main agent integrates information from all activity sub-agents.

[0149] The scene is dynamically updated as new evidence is collected.

[0150] If any scenario requires additional information, the main agent or sub-agent will trigger further sub-agent activation.

[0151] Generative AI Inference:

[0152] Scenario refinement: Continuously refine scenarios based on new data.

[0153] Probability allocation: Adjusting the probability of each scenario based on the overall evidence and the knowledge base of generative AI.

[0154] Decision-making and reporting:

[0155] Classification: Determine whether the alarm is a false alarm or an actual fire threat.

[0156] Detailed report: Use the following to compile the report:

[0157] Key evidence from sensor readings, video feeds, and environmental data.

[0158] Reasoning for this decision.

[0159] Confidence level in decision-making.

[0160] Recommended actions (e.g., dispatching fire services, ignoring alarms).

[0161] 6. Actions and Confirmations

[0162] False alarm:

[0163] Log recording: Record events for future analysis and model training.

[0164] Self-calibration: Updates the model's understanding of the normal baseline to reduce future false alarms.

[0165] Potential threats:

[0166] Warning Operator: Utilizes compiled reports to send detailed alerts to human operators.

[0167] Immediate Response: Provides operators with actionable insights to respond quickly to fire threats.

[0168] Instead of a single LLM, this paper employs a multi-agent system, as described, to simulate multiple human domain experts sitting together and analyzing alerts. Each agent is specialized in its domain, questioning each other's decisions, cross-validating information, and collaborating to gather insights. This provides enhanced reasoning and robust support for decision-making. Additionally, this approach helps combat the illusion of generative AI systems by providing mechanisms for cross-validation and continuous refinement based on diverse inputs from multiple experts.

[0169] Although several exemplary embodiments of this disclosure have been described thus, those skilled in the art will readily understand that other embodiments can be made and used within the scope of the appended claims. However, it should be understood that this disclosure is merely illustrative in many respects. Changes may be made to details, particularly those relating to shape, size, arrangement of parts, and exclusion and order of steps, without departing from the scope of this disclosure. The scope of this disclosure is, of course, defined by the language expressed in the appended claims.

Claims

1. A method for alarm processing of alarms in a building management system (BMS) (12) for a facility, wherein the BMS includes a plurality of BMS components (14) placed at known locations around the facility, wherein the plurality of BMS components include a plurality of sensors, the method comprising: Receive multiple alarms from the BMS; Each of the plurality of alarms is normalized into a normalized alarm format, wherein the normalized alarm format includes at least an alarm type and an alarm timestamp; For at least some of the plurality of alerts, at least in part based on the alert type of the corresponding alert, one of the autonomous master agents based on the generative artificial intelligence (AI) large language model is activated, wherein the corresponding autonomous master agent based on the generative AI large language model is trained using domain knowledge corresponding to the alert type of the corresponding alert. The corresponding autonomous master agent based on a generative AI large language model performs an initial analysis of the corresponding alarm and creates one or more initial scenarios to determine whether the corresponding alarm is a false alarm or a real alarm, wherein each of the one or more initial scenarios is associated with one or more scenario domains; The corresponding autonomous master agent based on the generative AI large language model autonomously assigns each initial scene in one or more initial scenes to one or more autonomous sub-agents based on the generative AI large language model, at least in part, based on the one or more scene domains of the corresponding initial scene. Each autonomous sub-agent based on the generative AI large language model is trained using domain knowledge corresponding to the corresponding scene domain. Each of the generative AI large language model-based autonomous sub-agents performs analysis of the assigned initial scene and returns the result to the generative AI large language model-based autonomous master agent that assigned the initial scene to the corresponding generative AI large language model-based autonomous sub-agent; and The autonomous master agent based on the generative AI large language model receives the results from each of the autonomous sub-agents based on the generative AI large language model, the autonomous sub-agents based on the generative AI large language model are assigned a corresponding initial scenario from the autonomous master agent based on the generative AI large language model, and classify the alarm as a false alarm or a true alarm based at least in part on the received results.

2. The method according to claim 1, wherein when one or more autonomous sub-agents based on generative AI large language models perform the analysis on the assigned initial scene, they autonomously assign one or more sub-tasks to one or more autonomous sub-agents based on generative AI large language models, at least in part based on the domain of the sub-task and the domain knowledge trained on the corresponding autonomous sub-agent based on generative AI large language models.

3. The method according to claim 1, wherein: The autonomous master agent report based on the generative AI large language model reports the confidence score when classifying the alarm as a false alarm or a true alarm; and / or The autonomous master agent report based on the generative AI large language model categorizes the alerts as either false or true based on reasoning.

4. The method of claim 1, wherein one or more of the plurality of autonomous master agents based on generative artificial intelligence (AI) large language models and / or one or more of the autonomous sub-agents based on generative AI large language models are configured to collect data from one or more of the plurality of BMS components of the BMS, optionally wherein the autonomous master agent based on the generative AI large language model reports the collected data supporting the classification of the alarms as false alarms or true alarms.

5. The method according to any one of claims 1 to 4, wherein one or more of the plurality of autonomous master agents based on generative artificial intelligence (AI) large language models and / or one or more of the autonomous sub-agents based on generative AI large language models change the operation of one or more of the plurality of BMS components in response to classifying the alarm as a true alarm, and does not change the operation of one or more of the plurality of BMS components in response to classifying the alarm as a false alarm.

6. The method according to any one of claims 1 to 4, wherein one or more of the plurality of BMS components include one or more video management system (VMS) components of the video management system (VMS) of the BMS, and one of the autonomous sub-agents based on the generative AI large language model is a video management system (VMS) analyzer sub-agent trained using domain knowledge related to the video management system (VMS) of the BMS.

7. The method according to any one of claims 1 to 4, wherein one or more of the plurality of BMS components include one or more fire detection components of the fire detection system of the BMS, and one of the autonomous sub-agents based on the generative AI large language model is a fire detection analyzer sub-agent trained using domain knowledge related to the fire detection system of the BMS.

8. The method according to any one of claims 1 to 4, wherein one or more of the plurality of BMS components include one or more security system components of the security system of the BMS, and one of the autonomous sub-agents based on the generative AI large language model is a security system analyzer sub-agent trained using domain knowledge related to the security system of the BMS.

9. The method according to any one of claims 1 to 4, wherein one or more of the plurality of BMS components include one or more HVAC components of the heating, ventilation and / or air conditioning (HVAC) system of the BMS, and one of the autonomous sub-agents based on the generative AI large language model is an HVAC analyzer sub-agent trained using domain knowledge related to the HVAC system of the BMS.

10. A system for alarm processing of alarms in a building management system (BMS) (12) for a facility, wherein the BMS includes a plurality of BMS components (14) placed at known locations around the facility, wherein the plurality of BMS components include a plurality of sensors, the system comprising: Input / output (18); Controller (20), operatively coupled to the input / output, configured to: Multiple alarms (16) are received from the BMS via the input / output terminal, each of the multiple alarms having an alarm type; For at least some of the multiple alerts, at least in part based on the alert type of the corresponding alert, activate one of the multiple autonomous master agents (22) based on the generative artificial intelligence (AI) large language model, wherein the corresponding autonomous master agent based on the generative AI large language model is trained using domain knowledge corresponding to the alert type of the corresponding alert; The corresponding autonomous master agent based on the generative AI large language model performs an initial analysis of the corresponding alarm and creates one or more reasonable causes for the corresponding alarm, wherein each of the one or more reasonable causes is related to one or more corresponding domains in a plurality of domains; The corresponding autonomous master agent based on the generative AI large language model autonomously assigns each of the one or more rational reasons among the rational reasons to one or more autonomous sub-agents (24) of the plurality of autonomous sub-agents based on the generative AI large language model, at least in part, based on the one or more domains of the corresponding rational reasons, wherein each autonomous sub-agent based on the generative AI large language model is trained using domain knowledge corresponding to the corresponding domain of the rational reason; Each of the generative AI large language model-based autonomous sub-agents performs an analysis of the assigned rationale and returns the result to the generative AI large language model-based autonomous master agent that assigned the rationale to the corresponding generative AI large language model-based autonomous sub-agent; and The autonomous master agent based on the generative AI large language model receives the results from each of the autonomous sub-agents based on the generative AI large language model, the autonomous sub-agents based on the generative AI large language model are assigned a corresponding reasonable cause from the autonomous master agent based on the generative AI large language model, and classify the alarm as a false alarm or a true alarm based at least in part on the received results.

Images