Context-aware elm security analysis and response

Abstract

An improved context-aware security system may include an artificial intelligence (Al) reasoning security system. The context-aware security system may be configured to receive security input data from a plurality of input sources, generate a contextual security analysis, identify a security response, and send instructions to a security system output device to execute the security response. The context-aware security system may include a plurality of security input sources configured to provide security inputs and contextual data, including credential readers, physical access controllers, security sensors, environmental sensors, or external information sources. The context-aware security system may include a directive input subsystem configured to receive security directives from an authorized user interface. A plurality of security system output devices may be configured to execute the security response, including credential reading systems, biometric systems, alerting systems, logging systems, or notification systems.

Description

CONTEXT-AWARE LLM SECURITY ANALYSIS AND RESPONSETECHNICAL FIELD

[0001] Embodiments described herein generally relate to intelligent security systems using Large Language Models (LLMs) for context-aware security management.BACKGROUND

[0002] Conventional security systems use predefined rules to identify security threats and responses. An example home security system may identify an unauthorized entry into the home and respond by contacting the homeowner and the relevant authorities. However, these conventional security systems focus security threat detection on predefined inputs and rules, and may not identify security threats outside of those predefined inputs and rules. Similarly, these conventional security systems focus on predefined security threat responses. What is needed is an improved security system to identify security threats and generate improved security responses.BRIEF SUMMARY

[0003] The following presents a simplified summary of one or more embodiments of the present disclosure to provide a basic understanding of such embodiments. This summary is not an extensive overview of all contemplated embodiments, and is intended to neither identify key or critical elements of all embodiments, nor delineate the scope of any or all embodiments.

[0004] The context-aware security system described herein may be configured to receive security input data from a plurality of input sources, generate a contextual security analysis based on the contextual data, identify a security response based on the contextual security analysis, and send instructions to a security system output device to execute the security response. The context-aware security system may include an artificial intelligence (Al) reasoning system, such as a Large Language Model (LLM) system. The context-aware security system may include a plurality of security input sources configured to provide security inputs and contextual data, including credential readers, physical access controllers, security sensors, environmental sensors, or external information sources. The context-aware security system may include a directive input subsystem configured to receive security directives from an authorized user interface. A plurality of security system output devicesmay be configured to execute the security response, including credential reading systems, biometric systems, alerting systems, logging systems, or notification systems.

[0005] While multiple embodiments are disclosed, still other embodiments of the present disclosure will become apparent to those skilled in the art from the following detailed description, which shows and describes illustrative embodiments of the invention. As will be realized, the various embodiments of the present disclosure are capable of modifications in various obvious aspects, all without departing from the scope of the present disclosure. Accordingly, the drawings and detailed description are to be regarded as illustrative in nature and not restrictive.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] In the drawings, which are not necessarily drawn to scale, like numerals may describe similar components in different views. Like numerals that have different letter suffixes may represent different instances of similar components. Some embodiments are illustrated by way of example, and not limitation, in the figures of the accompanying drawings:

[0007] FIG. 1 is a diagram of a context-aware security system.

[0008] FIG. 2 is a diagram depicting a context-aware security system with directive input.

[0009] FIG. 3 is a diagram of a context-aware reasoning action system.

[0010] FIG. 4 is a flowchart depicting a context-aware security method.

[0011] FIG. 5 is a block diagram of neural network training.

[0012] FIG. 6 illustrates a block diagram schematic of various components of an example reader, such as may be used with the antenna systems and methods discussed herein.DETAILED DESCRIPTION

[0013] The context-aware security systems and methods described herein provide improved security responses to security inputs. A context-aware security system includes an Al reasoning system configured to receive security input data from a plurality of input sources, generate a contextual security analysis based on the contextual data, identify a security response based on the contextual security analysis, and send instructions to a security system output device to execute the security response. The Al reasoning system may include an LLM, such as a trained text-to-text LLM model. The Al reasoning system may analyze the security inputs based on one or more input directives. Each input directive may includean authorized (e.g., received from an authorized user or received through an authorized user interface) command or instruction that guides the operation or response of the context-aware security system. Based on the analysis of the inputs, the context-aware security system may generate one or more output actions. Some of the output actions may result in feedback inputs to the context-aware security system, which may incorporate these feedback inputs to decide on the next set of actions.

[0014] The context-aware security system may be used to provide improved analysis of the context of various security events. The security needs for a given secure asset (e.g., a secure room, a secure office) may vary according to context, including the value of the assets being protected, the location of the context-aware security system, the time of day, recent occurrence of suspicious events, the presence or absence of people in the protected area, and other context factors. In an example, an office may be protected by a controlled entry, and may have variable authentication requirements. In an example of the variable authentication requirements, during normal business hours when many office workers are present, the context-aware security system may use reduced authentication requirements (e.g., minimal authentication, no authentication) to allow employees to enter and exit the office. After business hours on weekdays, the context-aware security system may require single-factor authentication to ensure that only authorized personnel (e.g., late-working executives, cleaning people, guards) are entering or exiting the office. For a predetermined late night (e.g., between midnight and 4:00 a.m.) or during a holiday weekend (e.g., Memorial Day weekend), a valid credential might be sufficient to enable a guard to enter the suite, but any other authorized holder may be required to use multifactor authentication, such as presenting both a valid credential and a corresponding matching biometric identification (e.g., fingerprint, face, iris).

[0015] The context-aware security system may manage access based on an employee status. In an example, a retail storeroom may contain expensive merchandise with controlled access capable of multifactor authentication. A store long-term employee who regularly goes in and out of the storeroom multiple times in a day may require one credential, whereas a new employee or an employee that typically does not access the storeroom may require the use of two or more types of multifactor authentication. The context-aware security system may also manage access based on store status. For example, if there has been a recent increase in store thefts (e.g., shrinkage), the context-aware security system may increase a required level of authentication for everyone until the store security status is changed.

[0016] The context-aware security system may be used to provide improve residential security. For residential protection, the context-aware security system may generate security responses based on time of day, local crime statistics, house occupancy (e.g., whether residents are home or away), or other residential-specific conditions. The context-aware security system may use conditional analyses to improve the security response. In an example, if there are auditory sensors that are tuned to glass breakage, then the context-aware security system may use the residential-specific conditions to determine whether a glass breakage event is a critical event (e.g., a break-in attempt through a window) or a normal event (e.g., a bowl is dropped on the kitchen floor). In the critical event case, the glass breakage event may be determined to be critical based on higher alert residential-specific conditions (e.g., the house is vacant, the event occurs in the middle of the night, the house is located in a relatively high-crime area), and the context-aware security system may generate a security response that includes sounding an audible alarm, calling emergency response (e.g., 911), and alerting the residents (e.g., phone call, text message). In the normal event case, if the glass breakage event occurs in the middle of the day, the house is currently occupied, and there are no other anomalous conditions, the context-aware security system may generate a security response that includes alerting the residents or taking no action.

[0017] Various features of the Al reasoning system are described herein with respect to an LLM system, however other types of Al reasoning systems may be used without departing from the scope of this disclosure. These Al reasoning systems may include Al models that have been trained to understand causes, effects, correlations, and statistical properties across a wide range of security scenarios and circumstances. The Al reasoning systems may be trained on text (e.g., text-to-text LLMs), may be trained on images and videos (e.g., as used with multimodal Al systems), may be trained to be aware of fundamental properties of physics (e.g., object permanence, inability for solid objects to travel through other solid objects), or may be trained in various academic disciplines (e.g., criminology, psychology, logic). In contrast with security systems that may take an action based on a set of security rules, the Al reasoning systems provide context-aware reasoning that is due substantially to one or more of these types of generalized trainings.

[0018] FIG. 1 is a diagram of a context-aware security system 100. The context- aware security system 100 includes one or more security input devices 105, which may include a physical access controller 110. The physical access controller 110 may include a credential reader, a biometric system, a keypad, an electronic lock, or other physical access controller. The security input devices 105 may include security system sensors 115, such assensors to detect smoke, fire, glass breakage, carbon monoxide, radon gas, water leakage, temperature, humidity presence, ambient light, ambient noise, acoustic sensors tuned to specific activities such as speaking, and other sensors. The security input devices 105 may include other security inputs 120, such as visitor management systems, a clock (e.g., access based on time of day), calendar (e.g., access based on work days), information technology (IT) systems (e.g., employee active directory), physical access systems, logical access systems, text entry systems, speech-to-text systems, imaging and video systems (e.g., near infrared, thermal, visible), outputs from other security systems, outputs from other Al reasoning systems, news and weather sources, emergency response information (e.g., police, fire), or security inputs.

[0019] The security input devices 105 can each provide one or more security inputs to the context-aware Al reasoning system 125. The context-aware Al reasoning system 125 provides an automated reasoning system that is capable of ingesting security inputs. The automated reasoning system may use the security inputs to generate a contextual security analysis, such as identifying security threats and determining appropriate security responses. The automated reasoning system may be stateless (e.g., no memory of prior events), and may receive an input context log of recent reasoning events along with the current inputs. The context input log may include a complete record of all inputs, intermediate steps, and actions taken, or the context input log may include a summary of any or all of these elements.

[0020] After generating the contextual security analysis, the security instructions are sent to a security output device 130. The security output device 130 may include a security device or security system capable of executing security responses based on instructions received from the context-aware Al reasoning system 125. The security output device 130 may include one or more authentication systems, and the security instructions may include requesting authentication from an authentication system. The one or more authentication systems may include credential reading systems (e.g., physical credential reading systems, digital credential reading systems), biometric recognition systems (e.g., facial, fingerprint, iris, gait, voice), continuous authentication systems, human authentication contacts (e.g., security personnel, management personnel, parents, guardians), personal identification number (PIN) pads, keypads, touchscreens, presentation attack detection (PAD) systems, or other authentication systems. The security output device 130 may include a notification system, and the security instructions may include instructing the notification system to generate a notification for one or more users. The notification system may include an alert system (e.g., emergency personnel, designated contacts), logging systems, notificationsystems, posting systems (e.g., blogging, vlogging, social media posting), physical access controllers, web search systems, terminal command line functionality, or text to speech systems. The security output device 130 may also include specialized systems, such as cryptographic systems, computational systems, statistical systems, anomaly detection operating on behavioral data or other data sources, pre-trained neural networks, or other specialized systems.

[0021] FIG. 2 is a diagram depicting a context-aware security system 200 with directive input. The context-aware security system 200 includes a directive input device 210 and one or more security input devices 215. The directive input device 210 may include one or more various secure devices, such as one or more secured computer user interfaces, one or more secured mobile devices, one or more specialized security control panels, or one or more other secure devices. The directive input device 210 may include a computing device configured to receive input security directives from an authorized user interface, where the input security directives may include instructions that guide the response of the context-aware Al reasoning system 220. An authorized user may interact with a directive input device 210 and provide a directive input prompt using words, sentences, paragraphs, or security example descriptions. An example directive input prompt provided by a user to a directive input device 210 may be as follows:You control a physical access security system that can use multiple factors for authentication (digital credentials, biometrics, passcodes). You want to allow someone into a controlled space as easily as possible while ensuring their identity and maintaining security. You have access to a behavioral monitoring system, a calendar, a clock, and can request a biometric scan and / or a passcode after the credential if you feel that the conditions warrant it.

[0022] The directive input device 210 may provide various directive inputs, such as instructions, guidance, or automated reasoning examples, to an automated reasoning subsystem within the context-aware Al reasoning system 220. The directive inputs may be in the form of text, which may originate from typed commands or a speech-to-text system. In an example, the context-aware Al reasoning system 220 may receive directive inputs from the directive input device 210 and receive security inputs from the security input device 215 as part of a single transmission. The directive input device 210 may have associated restrictions to reduce or prevent unauthorized generation of directive inputs, such asrestrictions based on predetermined directive input devices, directive input device locations, or other restrictions.

[0023] The context-aware Al reasoning system 220 provides an automated reasoning system that receives input directives from the directive input device 210 and receives security inputs from the one or more security input devices 215. The context-aware Al reasoning system 220 uses these input directives and security inputs to generate a contextual security analysis, such as identifying security threats and determining appropriate security responses. The context-aware Al reasoning system 220 may send instructions reflecting the appropriate security responses to a security output device 230, which may implement the appropriate security responses.

[0024] The security output device 230 may also provide feedback to the context- aware Al reasoning system 220, enabling the context-aware Al reasoning system 220 to update and refine its security analyses and responses based on the outcomes of executed security actions. The security output device 230 may execute the security responses as instructed by the context-aware Al reasoning system 220, and the security output device 230 may collect data on the results and effectiveness of these actions. In an example, the collected data may include whether an intruder was successfully deterred, whether access was correctly granted or denied, or if any security breaches occurred despite the security measures taken. The context-aware Al reasoning system 220 may use this collected data to update the security context and refine future security responses based on the effectiveness of previous similar security responses. This analysis may include determining the accuracy of threat assessments, assessing the effectiveness of the chosen security response, and identifying patterns or anomalies that may indicate a need for adjustments in the security strategy. The context-aware Al reasoning system 220 may also update internal Al reasoning models (e.g., retraining, fine-tuning) to further improve security threat predictions and security responses to similar situations in the future. For instance, if the feedback from the security output device 230 indicates that certain types of alarms are consistently ignored or ineffective, the context-aware Al reasoning system 220 might adjust its response strategy to either escalate the actions or choose an alternative response more likely to secure the premises.

[0025] The context-aware Al reasoning system 220 may include an abstraction framework to improve interaction with one or more of the directive input device 210 and the one or more security input devices 215. The abstraction framework may provide a standardized interface that allows seamless interaction with different Al reasoning systems, regardless of underlying differences in Al reasoning model size, parameters, or interfacerequirements. The abstraction framework may enable the context-aware Al reasoning system 220 be model-agnostic, enabling the context-aware Al reasoning system 220 to operate with different Al reasoning models without needing customization for each model’s specific requirements (e.g., application programming interface (API) requirements, data handling requirements). The abstraction framework may also enable the context-aware Al reasoning system 220 to support various Al reasoning agents, without needing customization for each agent. The Al reasoning agents can perform specialized tasks or access external resources. In an example, an Al reasoning agent in a context-aware security system may be designed to autonomously analyze video surveillance feeds, identify potential security threats based on behavioral patterns, and alert security personnel. The use of an abstraction framework with the context-aware security system 200 may improve the flexibility and efficiency of the context-aware security system 200, and may facilitate implementation of other advanced Al reasoning model features and Al reasoning agent capabilities.

[0026] FIG. 3 is a diagram of a context-aware reasoning action system 300. The context-aware reasoning action system 300 extends the functionality of the context-aware Al reasoning system 330 by using a reasoning action component 315 to generate a reasoning action (ReAct) plan or additional context for the context-aware Al reasoning system 330. The context-aware reasoning action system 300 includes a reasoning action user input 310, which is configured to receive security directives from an authorized user interface. The reasoning action user input 310 may include a touchscreen panel, a keyboard, a voice command system, or other authorized user interface.

[0027] The reasoning action component 315 may receive security input from one or more security input devices 305, receive one or more reasoning action (ReAct) user inputs from the reasoning action user input 310, and receive feedback from the security output device 340. The reasoning action component 315 improves the performance of the context- aware Al reasoning system 330 by combining reasoning steps and action steps. A reasoning component of the reasoning action component 315 may include analyzing and processing the one or more security inputs, one or more react inputs, and security device feedback. An action component of the reasoning action component 315 may include instructions to perform various tasks based on conclusions drawn from the reasoning process, such as retrieving additional security system information, interacting with external systems (e.g., security input device 305, security output device 340), or instructing the security output device 340 to execute commands.

[0028] The reasoning action component 315 may include plan generation 320, which may generate a ReAct plan for the context-aware Al reasoning system 330. The generated ReAct plan may include a security protocol to follow in case of a detected threat. The security protocol may include a detailed set of steps for responding to various combinations of security inputs and react inputs, including identifying instructions for the security output device 340 based on context and security inputs provided by one or more security input devices 305. The ReAct plan may include a set of steps for gathering feedback from the security output device 340 at the reasoning action component 315 and updating the ReAct plan and context to be provided to the context-aware Al reasoning system 330.

[0029] The reasoning action component 315 may include context generation 325, which may generate additional context for the context-aware Al reasoning system 330. The additional context may be based on the reasoning action user input(s) 310, such as indicating a heightened security status based on a security status input provided by an authorized user at the reasoning action user input 310. Additional examples of additional context generation include identifying potential security threats based on security input provided by one or more security input devices 305 or determining an updated status of the context-aware security system based on feedback provided by the security output device 340.

[0030] An example of a user interaction with the reasoning action component 315 via the reasoning action user input 310 to generate a ReAct plan or additional context may be as follows:Human: I will tell you the conditions and you will tell me what action you will take.Al: Sure, I am ready to help with the scenario you provided. Please tell me the conditions.Human: It is the middle of a workday, with many employees nearby and the person who presented their credential is a long term employee who has passed through this controlled entryway multiple times today.Al: Under these conditions, it appears that this is a routine scenario for the individual in question. To allow access:1. Authenticate the digital credential: The system should first authenticate the digital credential presented by the employee. This should be a quick and automated process.2. Behavioral monitoring: The behavioral monitoring system can verify the behavior of the individual, compare it to the normal behavior patterns of the employee, and raise a flag if something is unusual.Since this is a long-term employee who has already passed through this controlled entryway multiple times today, and considering the environment (middle of a workday with many employees nearby), it is reasonable to assume that the risk of identity fraud is relatively low. Therefore, the system may not require additional authentication steps such as a biometric scan or a passcode in this instance, unless the behavioral monitoring system detects something unusual.

[0031] The user may interact further via the reasoning action user input 310 to revise the react plan or additional context is as follows:Human: This is a different set of conditions. It is the middle of a long weekend holiday and late at night. The person presenting the credential is not a guard and is not normally at work at this time. No one else is in proximity of the controlled entryway.Al: Under these conditions, the scenario is less routine and the risk is higher.To allow access:1. Authenticate the digital credential: The first step remains the same, authenticate the digital credential presented by the individual.2. Behavioral monitoring: The system should also check the individual's behavior against their normal pattern. If any anomalies are detected, this should increase the level of suspicion.3. Biometric scan: Given the unusual time and absence of other employees, it is crucial to verify the individual's identity beyond their digital credential. A biometric scan (like fingerprint, facial recognition, or iris scan) would provide a higher level of certainty.4. Passcode: In addition to the biometric scan, it might be prudent to require a passcode. This provides multi-factorauthentication which is considered highly secure. Since the person is not normally at work at this time, this extra step will ensure that the person is indeed authorized and isn't under duress.This approach maximizes the level of security and ensures that the person accessing the space is indeed authorized to do so.Based on this interaction, the reasoning action component 315 may generate the response plan to reflect these steps for allowing access, and may generate additional context to indicate the security scenario is less routine and the security risk is higher.

[0032] The context-aware Al reasoning system 330 is configured to receive the reasoning action plan or additional context from the reasoning action component 315, receive security input from one or more security input devices 305, and generate instructions for the security output device 340. The security output device 340 is configured to execute the security response based on the instructions received from the context-aware Al reasoning system 330, such as alerting systems, logging systems, or notification systems. The context- aware Al reasoning system 330 may receive feedback from the security output device 340, enabling the context-aware Al reasoning system 330 to improve and adapt subsequent security instructions based on the outcomes of previous actions.

[0033] FIG. 4 is a flowchart depicting a context-aware security method 400. The method 400 includes sequence of operations performed by a context-aware Al reasoning system to manage security responses based on contextual data received from various input sources. Method 400 includes receiving 410 a security input data set from a security system device. The security input data set may include data relevant to a security context, such as security data from any credential readers, physical access controllers, security sensors, environmental sensors, external information sources, security input devices, or other security data sources.

[0034] Method 400 includes generating 420 a contextual security analysis at an Al reasoning system based on the security input data set. The generation of the contextual security analysis may include evaluating the received security input data set and any contextual data to determine the security context and identifying any potential security threats. The generated contextual security analysis may include a security context and at least one identified security threat.

[0035] Method 400 includes identifying 430 a security response based on the contextual analysis. The identified security response may include an action or set of actionsto address the identified security threat. The security response may include alerting security personnel, activating physical security measures, adjusting surveillance system settings, or other security response.

[0036] Method 400 includes sending 440 instructions to a security system output device to execute the security response. The security system output device may include one or more security devices capable of executing the identified security response, such as credential reading systems, biometric systems, alerting systems, logging systems, notification systems, or other security devices. The instructions cause the security system output device to execute the identified security response to address the identified security threat.

[0037] Method 400 may further include receiving, at a directive input subsystem, a security directive from an authorized user interface subsequent to generating the contextual security analysis, then updating the Al reasoning system based on the security directive.

[0038] Method 400 may further include generating a directive security analysis at the Al reasoning system based on the security directive and the security input data set, identifying a directed response based on the directive security analysis, and sending instructions to the security system output device to execute the directed response. Method 400 may further include authenticating a source of the security directive to ensure the security directive originates from an authorized user.

[0039] Method 400 may further include generating a reasoning action plan at a multi- step reasoning framework based on the security input data set, identifying a plurality of security response actions and an associated action sequence based on the reasoning action plan, and sending instructions to one or more of a plurality of security system devices to execute the plurality of security response actions. The multi-step reasoning framework may include a ReAct framework. The plurality of security response actions may include at least one of alerting security personnel, activating physical security measures, or adjusting surveillance system settings. The plurality of security response actions may be prioritized based on an urgency and severity associated with the at least one identified security threat. Method 400 may further include receiving a plurality of security response action outcomes and revising the Al reasoning system based on plurality of security response action outcomes.

[0040] Method 400 may further include receiving a security response outcome, generating a revised security analysis at the Al reasoning system, identifying a revised security response based on the revised security analysis, and sending instructions to the security system output device to execute the revised security response. Identifying therevised security response may include generating an Al reasoning prompt at the Al reasoning system based on the security response outcome.

[0041] Method 400 may further include identifying a security severity associated with the security response. Sending instructions to the security system output device may include sending an indication of the security severity. The contextual security analysis may include at least one of a security system status or an identified potential security threat.

[0042] The security response may include initiating an authentication request. The security system output device includes at least one of a credential reading security system, a biometric security system, a continuous authentication security system, a personal identification number security system, an alerting security system, a logging security system, or a notification security system. The security system device may include at least one of a credential reader, a physical access controller, a security sensor, or an input from an external security system.

[0043] Method 400 may further include generating a text data set based on the security input data set for use by the Al reasoning system. The security input data set may include a non-text data set. The non-text data set may include at least one of a video input data set or an audio input data set.

[0044] The Al reasoning system may include an abstraction framework to facilitate interaction between the Al reasoning system and a plurality of external security devices. An abstraction security input may be received at the abstraction framework. The instructions to the security system output device may be generated based on the abstraction security input. This abstraction framework provides a standardized interface that allows seamless interaction with various Al reasoning s, enabling the context-aware Al reasoning to operate with different Al reasoning s without the need for customization to meet each model’s specific API or data handling requirements. Additionally, the abstraction framework supports various Al reasoning agents, enabling the Al reasoning agents to perform specialized tasks without needing individual customization. In an example, an Al reasoning agent may autonomously analyze video surveillance feeds, identify potential security threats based on behavioral patterns, and alert security personnel. The abstraction framework may improve the flexibility and efficiency of the context-aware security system, facilitating the integration and implementation of advanced Al reasoning features and agent capabilities.

[0045] The Al reasoning system may include a multimodal Al reasoning model; and the security input data set includes a plurality of different input types. The plurality ofdifferent input types may include at least one of security imaging data, text input, and security system status data.

[0046] This multi-agent structure allows for specialized handling of various security scenarios by assigning specific tasks to agents best equipped to handle them. In an example, a first agent may be dedicated to analyzing video feeds for unauthorized access, a second agent may be dedicated to monitoring network traffic for potential cyber threats, and a third agent may be dedicated to handling biometric authentication processes. This targeted agent allocation may improve response times, and may improve the accuracy and efficiency of the security measures. By distributing tasks among specialized agents, the multi-agent system may manage complex security operations more effectively by allocating separate aspects of security to be handled by an agent with the most relevant capabilities and resources. This multi-agent structure also allows for scalability, as additional agents can be added to handle new types of security inputs or to handle additional security needs.

[0047] FIG. 5 is a block diagram of neural network training 500. The context-aware Al reasoning security system may include a specialized type of artificial neural network (ANN). LLMs and other Al reasoning models may function by analyzing vast amounts of text data or other data types (e.g., multi-modal LLMs) to interpret inputs and generate security instructions and other outputs. The Al reasoning systems fall within the field of artificial intelligence (Al) analysis, which is concerned with developing decision-making systems to perform cognitive tasks that have traditionally required a living actor, such as a person. The Al analysis of security system inputs may be performed by a context-aware Al reasoning system as described herein. An Al reasoning system (and ANNs generally) includes a computational structure that may be loosely modeled on biological neurons. Generally, Al reasoning systems encode information (e.g., data or decision making) via weighted connections (e.g., synapses) between nodes (e.g., neurons).

[0048] As used herein, the term “LLM” may refer to a language model that was built using the transformer architecture, which relies on self-attention mechanisms to capture relationships between words in a text. The transformer architecture includes components like multi-head attention, feed-forward networks, and positional encodings, organized into stacks of encoder and / or decoder layers. These models are pre-trained on vast datasets and finetuned for specific tasks, allowing them to generate and understand text effectively. While LLM refers to large language models, models of various sizes may be used without departing from the scope of this disclosure. Small, medium, and large models may vary by their number of parameters and capabilities. Small models, such as TinyBERT with around 4.4 to14.5 million parameters, are designed for resource-constrained environments and basic language tasks. Medium models include BERT Base with 110 million parameters, offering a balance of performance and efficiency suitable for many NLP applications. Large models like GPT-3, with 175 billion parameters, demonstrate advanced capabilities in understanding and generating human-like text, performing well on a wide range of complex tasks. Reference to LLMs in this disclosure may refer to small, medium, and large language models, and may refer to language models that are larger than large language models.

[0049] Many ANNs, including LLMs, are represented as matrices of weights that correspond to the modeled connections. ANNs operate by accepting data into a set of input neurons that often have many outgoing connections to other neurons. At each traversal between neurons, the corresponding weight modifies the input and is tested against a threshold at the destination neuron. If the weighted value exceeds the threshold, the value is again weighted, or transformed through a nonlinear function, and transmitted to another neuron further down the ANN graph. If the threshold is not exceeded, then the value is usually not transmitted to a down-graph neuron and the synaptic connection remains inactive. The process of weighting and testing continues until an output neuron is reached, where the pattern and values of the output neurons constitute the result of the ANN processing.

[0050] The correct operation of most ANNs relies on correct weights. However, ANN designers may not know which weights will work for a given application. ANN designers typically choose a number of neuron layers or specific connections between layers including circular connection, but the ANN designer does may not know which weights will work for a given application. Instead, a training process is used to arrive at appropriate weights. However, determining correct synapse weights is common to most ANNs. The training process proceeds by selecting initial weights, which may be randomly selected. Training data is fed into the ANN and results are compared to an objective function that provides an indication of error. The error indication is a measure of how wrong the ANN’S result was compared to an expected result. This error is then used to correct the weights. Over many iterations, the weights will collectively converge to encode the operational data into the ANN. This process may be called an optimization of the objective function (e.g., a cost or loss function), whereby the cost or loss is minimized.

[0051] Backpropagation is a technique whereby training data is fed forward through the ANN, where “forward” means that the data starts at the input neurons and follows the directed graph of neuron connections until the output neurons are reached, and the objective function is applied backwards through the ANN to correct the synapse weights. At each stepin the backpropagation process, the result of the previous step is used to correct a weight. Thus, the result of the output neuron correction is applied to a neuron that connects to the output neuron, and so forth until the input neurons are reached. Backpropagation has become a popular technique to train a variety of ANNs.

[0052] Neural network training 500 may include an Al reasoning system 510 that is trained using a processing node 520. The processing node 520 may be a CPU, GPU, field programmable gate array (FPGA), digital signal processor (DSP), application specific integrated circuit (ASIC), or other processing circuitry. In an example, multiple processing nodes may be employed to train different layers of the Al reasoning system 510, or even different nodes 560 within layers. Thus, a set of processing nodes 520 is arranged to perform the training of the Al reasoning system 510.

[0053] The set of processing nodes 520 is arranged to receive a training set 530 for the Al reasoning system 510. The training set 530 may include previously stored data from one or more security sensor devices. The Al reasoning system 510 comprises a set of nodes 560 arranged in layers (illustrated as rows of nodes 560) and a set of inter-node weights 570 (e.g., parameters) between nodes in the set of nodes. In various embodiments, an Al reasoning system 510 may use as few as two layers of nodes, or the Al reasoning system 510 may use as many as ten or more layers of nodes. The number of nodes 560 or number of node layers may be selected based on the type and complexity of the context-aware Al reasoning system. In various examples, the Al reasoning system 510 includes a node layer corresponding to multiple sensor data types or a node layer corresponding to multiple security contexts. In an example, the training set 530 is a subset of a complete training set of data from one or more input security devices. Here, the subset may enable processing nodes with limited storage resources to participate in training the Al reasoning system 510.

[0054] The training data may include multiple numerical values that are representative of a context-aware security system classification 540, such determining whether a security sensor input in the current context constitute a critical event or a normal event. During training, each value of the training is provided to a corresponding node 560 in the first layer or input layer of Al reasoning system 510. Once Al reasoning system 510 is trained, each value of the input 550 to be classified is similarly provided to a corresponding node 560 in the first layer or input layer of Al reasoning system 510. The values propagate through the layers and are changed by the objective function.

[0055] As noted above, the set of processing nodes is arranged to train the neural network to create a trained neural network. Once trained, the input security system data 550will be assigned into categories such that data input into the Al reasoning system 510 will produce valid context-aware security system classifications 540. Training may include supervised learning, where portions of the training data set are labeled using context-aware security system classifications 540. After an initial supervised learning is completed, the Al reasoning system 510 may undergo unsupervised learning, where the training data set is not labeled using context-aware security system classifications 540. For example, the Al reasoning system 510 may be trained initially by supervised learning using previously classified security system data, and subsequently trained by unsupervised learning using newly collected security system data. This unsupervised learning using newly collected security system data enables the system to adapt to various context-aware security system input types. This unsupervised learning also enables the system to adapt to changes in the security system context.

[0056] The training performed by the set of processing nodes 560 is iterative. In an example, each iteration of the training the neural network is performed independently between layers of the Al reasoning system 510. Thus, two distinct layers may be processed in parallel by different members of the set of processing nodes. In an example, different layers of the Al reasoning system 510 are trained on different hardware. The members of different members of the set of processing nodes may be located in different packages, housings, computers, cloud-based resources, etc. In an example, each iteration of the training is performed independently between nodes in the set of nodes. This example is an additional parallelization whereby individual nodes 560 (e.g., neurons) are trained independently. In an example, the nodes are trained on different hardware.

[0057] The number and types of context-aware security system classifications 540 may be modified to add, remove, or modify context-aware security system classifications 540. This may enable the Al reasoning system 510 to be updated via software, which may enable modification of the context-aware Al reasoning system without replacing the entire model. A software update of the context-aware security system classifications 540 may include initiating additional supervised learning based on a newly provided set of input data with associated context-aware security system classifications 540. A software update of the context-aware security system classifications 540 may include replacing the currently trained Al reasoning system 510 with a separate Al reasoning system 510 trained using a distinct set of input data or context-aware security system classifications 540.

[0058] FIG. 6 illustrates a block diagram schematic of various components of an example context-aware security system 600, such as may be used to implement the context-aware security systems and methods discussed herein. In general, system 600 can include one or more of a memory 602, a processor 604, one or more antennas 606, a communication module 608, a network interface device 610, a user interface 612, and a power source 614 (e.g., power supply). System 600 may include a device affixed to a surface (e.g., wall, door), though system 600 may also be a free-standing device or a portable device (e.g., mobile electronic device).

[0059] Memory 602 can be used in connection with the execution of application programming or instructions by processor 604, and for the temporary or long-term storage of executable instructions 616 (e.g., program instructions, program instruction sets) or security data 618, such as security device inputs, context data, or directive inputs. For example, memory 602 can contain executable instructions 616 that are used by the processor 604 to run other components of system 600 and generate security device instructions based on security data 618.

[0060] Memory 602 can comprise a computer readable medium that can be any medium that can contain, store, communicate, or transport data, program code, or instructions for use by or in connection with system 600. The computer readable medium can be, for example but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples of suitable computer readable medium include, but are not limited to, an electrical connection having one or more wires or a tangible storage medium such as a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or EEPROM), Dynamic RAM (DRAM), any solid-state storage device, in general, a compact disc read-only memory (CD-ROM), or other optical or magnetic storage device. Computer readable media includes but is not to be confused with, computer readable storage medium, which is intended to cover all physical, non-transitory, or similar embodiments of computer readable media.

[0061] Processor 604 can correspond to one or more computer processing devices or resources. For instance, processor 604 can be provided as silicon, as a Field Programmable Gate Array (FPGA), an Application-Specific Integrated Circuit (ASIC), any other type of Integrated Circuit (IC) chip, a collection of IC chips, or the like. As a more specific example, processor 604 can be provided as a microprocessor, Central Processing Unit (CPU), or plurality of microprocessors or CPUs that are configured to execute instructions sets stored in an internal memory 620 or memory 602.

[0062] Antenna 606 may include one or more antennas, and may be configured to provide for wireless communications between, for example, system 600 and one or more security system input devices, user input devices, or security output devices. Antenna 606 can be arranged to operate using one or more wireless communication protocols and operating frequencies such as the IEEE 802.15.1, Bluetooth, Bluetooth Low Energy (BLE), near field communications (NFC), ZigBee, GSM, CDMA, Wi-Fi, RF, UWB, and the like. By way of example, antenna 606 may include RF antenna(s), and as such, may transmit or receive RF signals through free space to be received / transferred by another electronic device having an RF transceiver.

[0063] Communication module 608 may be configured to communicate according to any suitable communications protocol with one or more different systems or devices either remote or local to system 600, such as a control panel or external control device.

[0064] Network interface device 610 includes hardware to facilitate communications with other devices, such as a control panel or host server over a communication network, using any one of a number of transfer protocols (e.g., frame relay, internet protocol (IP), transmission control protocol (TCP), user datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.). Example communication networks can include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), mobile telephone networks (e.g., cellular networks), Plain Old Telephone (POTS) networks, wireless data networks (e.g., IEEE 802.11 family of standards known as Wi-Fi or IEEE 802.16 family of standards known as WiMax), networks based on the IEEE 802.15.4 family of standards, and peer-to-peer (P2P) networks, among others. In some examples, network interface device 610 can include an Ethernet port or other physical jack, a Wi-Fi card, a Network Interface Card (NIC), a cellular interface (e.g., antenna, filters, and associated circuitry), or the like. In some examples, network interface device 610 can include one or more antennas to wirelessly communicate using, for example, at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) techniques.

[0065] User interface 612 can include one or more input devices or display devices. Examples of suitable user input devices that may be included in user interface 612 include, without limitation, one or more buttons, a keyboard, a mouse, a touch-sensitive surface, a stylus, a camera, a microphone, a PIN pad, touch screen, fingerprint reader, magnetic stripe reader, chip reader, etc. Examples of suitable user output devices that may be included in user interface 612 include, without limitation, one or more LEDs, an LCD panel, a display screen, a touchscreen, one or more lights, a speaker, etc. It should be appreciated that userinterface 612 can also include a combined user input and user output device, such as a touch- sensitive display or the like.

[0066] Power source 614 may be any suitable internal power source, such as a battery, capacitive power source or similar type of charge-storage device, etc., or can include one or more power conversion circuits suitable to convert external power into suitable power (e.g., conversion of externally supplied AC power into DC power) for components of the system 600. Power source 614 can also include some implementation of surge protection circuitry to protect the components of system 600 from power surges.

[0067] System 600 can also include one or more busses or interlinks 622 operable to transmit communications between the various hardware components of the reader. A system bus or interlink 622 may be any of several types of commercially available bus structures or bus architectures. A computing device or credential reader manager may reconfigure the system 600 by connecting a device to the system 600 via bus or interlink 622, such as by changing device parameters (e.g., configurable interpolling delays), by overwriting a device management policy, by updating software, by reflashing firmware, or other reconfigurations.Additional Examples

[0068] Example l is a method for security management, the method comprising: receiving a security input data set of input data from one or more security system devices; generating a contextual security analysis at an artificial intelligence (Al) reasoning system based on the security input data set of input data, the contextual security analysis including a security context and at least one identified security threat; identifying a security response based on the contextual security analysis; and sending instructions to a security system output device to execute the security response.

[0069] In Example 2, the subject matter of Example 1 includes receiving, at a directive input subsystem, a security directive from an authorized user interface subsequent to generating the contextual security analysis; and updating the Al reasoning system based on the security directive.

[0070] In Example 3, the subject matter of Example 2 includes wherein identifying the security response is further based on the security directive.

[0071] In Example 4, the subject matter of Examples 2-3 includes authenticating a source of the security directive to ensure the security directive originates from an authorized user.

[0072] In Example 5, the subject matter of Examples 1-4 includes generating a reasoning action plan at a multi-step reasoning framework based on the security input data set of input data; and identifying a plurality of security response actions and an associated action sequence based on the reasoning action plan; wherein generating the security response is further based on the plurality of security response actions and the associated action sequence.

[0073] In Example 6, the subject matter of Example 5 includes wherein: the plurality of security response actions includes at least one of alerting security personnel, activating physical security measures, or adjusting surveillance system settings; and plurality of security response actions is prioritized based on at least one of an urgency or a severity associated with the at least one identified security threat.

[0074] In Example 7, the subject matter of Examples 1-6 includes receiving a plurality of security response action outcomes; and revising the Al reasoning system based on plurality of security response action outcomes.

[0075] In Example 8, the subject matter of Examples 1-7 includes receiving a security response outcome; generating a revised security analysis at the Al reasoning system; identifying a revised security response based on the revised security analysis; and sending instructions to the security system output device to execute the revised security response.

[0076] In Example 9, the subject matter of Example 8 includes wherein identifying the revised security response includes generating an LLM prompt at the Al reasoning system based on the security response outcome.

[0077] In Example 10, the subject matter of Examples 1-9 includes identifying a security severity associated with the security response, wherein sending instructions to the security system output device includes sending an indication of the security severity.

[0078] In Example 11, the subject matter of Examples 1-10 includes wherein the contextual security analysis includes at least one of a security system status or an identified potential security threat.

[0079] In Example 12, the subject matter of Examples 1-11 includes wherein the security system output device includes at least one of a credential reader, a physical access controller, a security sensor, or an input from an external security system.

[0080] In Example 13, the subject matter of Examples 1-12 includes wherein: the security response includes initiating an authentication request; and the security system output device includes at least one of a credential reading security system, a biometric security system, a continuous authentication security system, a personal identification number securitysystem, an alerting security system, a logging security system, or a notification security system.

[0081] In Example 14, the subject matter of Examples 1-13 includes wherein the security input data set of input data includes a non-text data set, the method further including generating a text data set based on the security input data set of input data for use by the Al reasoning system.

[0082] In Example 15, the subject matter of Example 14 includes wherein the nontext data set includes at least one of a video input data set or an audio input data set.

[0083] In Example 16, the subject matter of Examples 1-15 includes wherein: the Al reasoning system includes a multimodal LLM model; and the security input data set of input data includes a plurality of different input types.

[0084] In Example 17, the subject matter of Examples 1-16 includes wherein: the Al reasoning system includes a plurality of agents within a multi-agent system, each of the plurality of agents within the multi-agent system configured to receive predetermined security input types; and the multi-agent system is configured to allocate a plurality of security tasks to a subset of the plurality of agents within the multi-agent system.

[0085] Example 18 is a system for security management, the system comprising: one or more security system devices to generate a security input data set of input data; an artificial intelligence (Al) reasoning system including processing circuitry and a memory storing instructions, the instructions causing the Al reasoning system to: generate a contextual security analysis based on the security input data set of input data, the contextual security analysis including a security context and at least one identified security threat; and identifying a security response based on the contextual security analysis; and a security system output device to execute the security response.

[0086] In Example 19, the subject matter of Example 18 includes a directive input subsystem to receive a security directive from an authorized user interface subsequent to generating the contextual security analysis, wherein the Al reasoning system is updated based on the security directive.

[0087] In Example 20, the subject matter of Example 19 includes wherein identifying the security response is further based on the security directive.

[0088] In Example 21, the subject matter of Examples 19-20 includes the instructions further causing the Al reasoning system to authenticate a source of the security directive to ensure the security directive originates from an authorized user.

[0089] In Example 22, the subject matter of Examples 18-21 includes the instructions further causing the Al reasoning system to: generate a reasoning action plan at a multi-step reasoning framework based on the security input data set of input data; and identify a plurality of security response actions and an associated action sequence based on the reasoning action plan; wherein generating the security response is further based on the plurality of security response actions and the associated action sequence.

[0090] In Example 23, the subject matter of Example 22 includes wherein: the plurality of security response actions includes at least one of alerting security personnel, activating physical security measures, or adjusting surveillance system settings; and plurality of security response actions is prioritized based on at least one of an urgency or a severity associated with the at least one identified security threat.

[0091] In Example 24, the subject matter of Examples 18-23 includes the instructions further causing the Al reasoning system to: receive a plurality of security response action outcomes; and revise the Al reasoning system based on plurality of security response action outcomes.

[0092] In Example 25, the subject matter of Examples 18-24 includes the instructions further causing the Al reasoning system to: receive a security response outcome; generate a revised security analysis at the Al reasoning system; identify a revised security response based on the revised security analysis; and send instructions to the security system output device to execute the revised security response.

[0093] In Example 26, the subject matter of Example 25 includes wherein identifying the revised security response includes generating an LLM prompt at the Al reasoning system based on the security response outcome.

[0094] In Example 27, the subject matter of Examples 18-26 includes the instructions further causing the Al reasoning system to identify a security severity associated with the security response, wherein sending instructions to the security system output device includes sending an indication of the security severity.

[0095] In Example 28, the subject matter of Examples 18-27 includes wherein the contextual security analysis includes at least one of a security system status or an identified potential security threat.

[0096] In Example 29, the subject matter of Examples 18-28 includes wherein the one or more security system devices include at least one of a credential reader, a physical access controller, a security sensor, or an input from an external security system.

[0097] In Example 30, the subject matter of Examples 18-29 includes wherein: the security response includes initiating an authentication request; and the security system output device includes at least one of a credential reading security system, a biometric security system, a continuous authentication security system, a personal identification number security system, an alerting security system, a logging security system, or a notification security system.

[0098] In Example 31, the subject matter of Examples 18-30 includes wherein the security input data set of input data includes a non-text data set, the instructions further causing the Al reasoning system to generate a text data set based on the security input data set of input data for use by the Al reasoning system.

[0099] In Example 32, the subject matter of Example 31 includes wherein the nontext data set includes at least one of a video input data set or an audio input data set.

[0100] In Example 33, the subject matter of Examples 18-32 includes wherein: the Al reasoning system includes a multimodal LLM model; and the security input data set of input data includes a plurality of different input types.

[0101] In Example 34, the subject matter of Examples 18-33 includes wherein: the Al reasoning system includes a plurality of agents within a multi-agent system, each of the plurality of agents within the multi-agent system configured to receive predetermined security input types; and the multi-agent system is configured to allocate a plurality of security tasks to a subset of the plurality of agents within the multi-agent system.

[0102] Example 35 is a non-transitory computer-readable medium storing instructions that, when executed by a processor of a device, cause the device to: receive a security input data set of input data from one or more security system devices; generate a contextual security analysis at an artificial intelligence (Al) reasoning system based on the security input data set of input data, the contextual security analysis including a security context and at least one identified security threat; identify a security response based on the contextual security analysis; and send instructions to a security system output device to execute the security response.

[0103] In Example 36, the subject matter of Example 35 includes the instructions further causing the device to: receive, at a directive input subsystem, a security directive from an authorized user interface subsequent to generating the contextual security analysis; and update the Al reasoning system based on the security directive.

[0104] In Example 37, the subject matter of Example 36 includes wherein identifying the security response is further based on the security directive.

[0105] In Example 38, the subject matter of Examples 36-37 includes the instructions further causing the device to authenticate a source of the security directive to ensure the security directive originates from an authorized user.

[0106] In Example 39, the subject matter of Examples 35-38 includes the instructions further causing the device to: generate a reasoning action plan at a multi-step reasoning framework based on the security input data set of input data; and identify a plurality of security response actions and an associated action sequence based on the reasoning action plan; wherein generating the security response is further based on the plurality of security response actions and the associated action sequence.

[0107] In Example 40, the subject matter of Example 39 includes wherein: the plurality of security response actions includes at least one of alerting security personnel, activating physical security measures, or adjusting surveillance system settings; and plurality of security response actions is prioritized based on at least one of an urgency or a severity associated with the at least one identified security threat.

[0108] In Example 41, the subject matter of Examples 35-40 includes the instructions further causing the device to: receive a plurality of security response action outcomes; and revise the Al reasoning system based on plurality of security response action outcomes.

[0109] In Example 42, the subject matter of Examples 35-41 includes the instructions further causing the device to: receive a security response outcome; generate a revised security analysis at the Al reasoning system; identify a revised security response based on the revised security analysis; and send instructions to the security system output device to execute the revised security response.

[0110] In Example 43, the subject matter of Example 42 includes wherein identifying the revised security response includes generating an LLM prompt at the Al reasoning system based on the security response outcome.

[0111] In Example 44, the subject matter of Examples 35-43 includes the instructions further causing the device to identify a security severity associated with the security response, wherein sending instructions to the security system output device includes sending an indication of the security severity.

[0112] In Example 45, the subject matter of Examples 35-44 includes wherein the contextual security analysis includes at least one of a security system status or an identified potential security threat.

[0113] In Example 46, the subject matter of Examples 35-45 includes wherein the security system output device includes at least one of a credential reader, a physical access controller, a security sensor, or an input from an external security system.

[0114] In Example 47, the subject matter of Examples 35-46 includes wherein: the security response includes initiating an authentication request; and the security system output device includes at least one of a credential reading security system, a biometric security system, a continuous authentication security system, a personal identification number security system, an alerting security system, a logging security system, or a notification security system.

[0115] In Example 48, the subject matter of Examples 35-47 includes wherein the security input data set of input data includes a non-text data set, the instructions further causing the device to generate a text data set based on the security input data set of input data for use by the Al reasoning system.

[0116] In Example 49, the subject matter of Example 48 includes wherein the nontext data set includes at least one of a video input data set or an audio input data set.

[0117] In Example 50, the subject matter of Examples 35-49 includes wherein: the Al reasoning system includes a multimodal LLM model; and the security input data set of input data includes a plurality of different input types.

[0118] In Example 51, the subject matter of Examples 35-50 includes wherein: the Al reasoning system includes a plurality of agents within a multi-agent system, each of the plurality of agents within the multi-agent system configured to receive predetermined security input types; and the multi-agent system is configured to allocate a plurality of security tasks to a subset of the plurality of agents within the multi-agent system.

[0119] Example 52 is at least one machine-readable medium including instructions that, when executed by processing circuitry, cause the processing circuitry to perform operations to implement of any of Examples 1-51.

[0120] Example 53 is an apparatus comprising means to implement of any of Examples 1-51.Example 54 is a system to implement of any of Examples 1-51.Example 55 is a method to implement of any of Examples 1-51.Additional Notes

[0121] The above detailed description includes references to the accompanying drawings, which form a part of the detailed description. The drawings show, by way ofillustration, specific embodiments that may be practiced. These embodiments may also be referred to herein as “examples.” Such embodiments or examples can include elements in addition to those shown or described. However, the present inventors also contemplate examples in which only those elements shown or described are provided. Moreover, the present inventors also contemplate examples using any combination or permutation of those elements shown or described (or one or more aspects thereof), either with respect to a particular example (or one or more aspects thereof), or with respect to other examples (or one or more aspects thereof) shown or described herein. That is, the above-described embodiments or examples or one or more aspects, features, or elements thereof may be used in combination with each other.

[0122] As will be appreciated by one of skill in the art, the various embodiments of the present disclosure may be embodied as a method (including, for example, a computer- implemented process, a business process, or any other process), apparatus (including, for example, a system, machine, device, computer program product, or the like), or a combination of the foregoing. Accordingly, embodiments of the present disclosure or portions thereof may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, middleware, microcode, hardware description languages, etc.), or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present disclosure may take the form of a computer program product on a computer-readable medium or computer-readable storage medium, having computerexecutable program code embodied in the medium, that define processes or methods described herein. A processor or processors may perform the necessary tasks defined by the computer-executable program code. In the context of this disclosure, a computer readable medium may be any medium that can contain, store, communicate, or transport the program for use by or in connection with the systems disclosed herein. As indicated above, the computer readable medium may be, for example but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples of suitable computer readable medium include, but are not limited to, an electrical connection having one or more wires or a tangible storage medium such as a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a compact disc read-only memory (CD-ROM), or other optical, magnetic, or solid state storage device. As noted above, computer-readable media includes but is not to be confusedwith, computer-readable storage medium, which is intended to cover all physical, non- transitory, or similar embodiments of computer-readable media.

[0123] In the foregoing description various embodiments of the present disclosure have been presented for the purpose of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise form disclosed. Obvious modifications or variations are possible considering the above teachings. The various embodiments were chosen and described to provide the best illustration of the principals of the disclosure and their practical application, and to enable one of ordinary skill in the art to use the various embodiments with various modifications as are suited to the particular use contemplated. All such modifications and variations are within the scope of the present disclosure as determined by the appended claims when interpreted in accordance with the breadth they are fairly, legally, and equitably entitled.

Claims

CLAIMSWhat is claimed is:

1. A method for security management, the method comprising: receiving a security input data set of input data from one or more security system devices; generating a contextual security analysis at an artificial intelligence (Al) reasoning system based on the security input data set of input data, the contextual security analysis including a security context and at least one identified security threat; identifying a security response based on the contextual security analysis; and sending instructions to a security system output device to execute the security response.

2. The method of claim 1, further including: receiving, at a directive input subsystem, a security directive from an authorized user interface subsequent to generating the contextual security analysis; and updating the Al reasoning system based on the security directive.

3. The method of claim 2, wherein identifying the security response is further based on the security directive.

4. The method of claim 2, further including authenticating a source of the security directive to ensure the security directive originates from an authorized user.

5. The method of claim 1, further including: generating a reasoning action plan at a multi-step reasoning framework based on the security input data set of input data; and identifying a plurality of security response actions and an associated action sequence based on the reasoning action plan; wherein generating the security response is further based on the plurality of security response actions and the associated action sequence.

6. The method of claim 5, wherein:the plurality of security response actions includes at least one of alerting security personnel, activating physical security measures, or adjusting surveillance system settings; and plurality of security response actions is prioritized based on at least one of an urgency or a severity associated with the at least one identified security threat.

7. The method of claim 1, further including: receiving a plurality of security response action outcomes; and revising the Al reasoning system based on plurality of security response action outcomes.

8. The method of claim 1, further including: receiving a security response outcome; generating a revised security analysis at the Al reasoning system; identifying a revised security response based on the revised security analysis; and sending instructions to the security system output device to execute the revised security response.

9. The method of claim 8, wherein identifying the revised security response includes generating an LLM prompt at the Al reasoning system based on the security response outcome.

10. The method of claim 1, further including identifying a security severity associated with the security response, wherein sending instructions to the security system output device includes sending an indication of the security severity.

11. The method of claim 1, wherein the contextual security analysis includes at least one of a security system status or an identified potential security threat.

12. The method of claim 1, wherein the security system output device includes at least one of a credential reader, a physical access controller, a security sensor, or an input from an external security system.

13. The method of claim 1, wherein:the security response includes initiating an authentication request; and the security system output device includes at least one of a credential reading security system, a biometric security system, a continuous authentication security system, a personal identification number security system, an alerting security system, a logging security system, or a notification security system.

14. The method of claim 1, wherein the security input data set of input data includes a non-text data set, the method further including generating a text data set based on the security input data set of input data for use by the Al reasoning system.

15. The method of claim 14, wherein the non-text data set includes at least one of a video input data set or an audio input data set.

16. The method of claim 1, wherein: the Al reasoning system includes a multimodal LLM model; and the security input data set of input data includes a plurality of different input types.

17. The method of claim 1, wherein: the Al reasoning system includes a plurality of agents within a multi-agent system, each of the plurality of agents within the multi-agent system configured to receive predetermined security input types; and the multi-agent system is configured to allocate a plurality of security tasks to a subset of the plurality of agents within the multi-agent system.

18. A system for security management, the system comprising: one or more security system devices to generate a security input data set of input data; an artificial intelligence (Al) reasoning system including processing circuitry and a memory storing instructions, the instructions causing the Al reasoning system to: generate a contextual security analysis based on the security input data set of input data, the contextual security analysis including a security context and at least one identified security threat; and identifying a security response based on the contextual security analysis; and a security system output device to execute the security response.

19. The system of claim 18, further including a directive input subsystem to receive a security directive from an authorized user interface subsequent to generating the contextual security analysis, wherein the Al reasoning system is updated based on the security directive.

20. The system of claim 19, wherein identifying the security response is further based on the security directive.

21. The system of claim 19, the instructions further causing the Al reasoning system to authenticate a source of the security directive to ensure the security directive originates from an authorized user.

22. The system of claim 18, the instructions further causing the Al reasoning system to: generate a reasoning action plan at a multi-step reasoning framework based on the security input data set of input data; and identify a plurality of security response actions and an associated action sequence based on the reasoning action plan; wherein generating the security response is further based on the plurality of security response actions and the associated action sequence.

23. The system of claim 22, wherein: the plurality of security response actions includes at least one of alerting security personnel, activating physical security measures, or adjusting surveillance system settings; and plurality of security response actions is prioritized based on at least one of an urgency or a severity associated with the at least one identified security threat.

24. The system of claim 18, the instructions further causing the Al reasoning system to: receive a plurality of security response action outcomes; and revise the Al reasoning system based on plurality of security response action outcomes.

25. The system of claim 18, the instructions further causing the Al reasoning system to: receive a security response outcome; generate a revised security analysis at the Al reasoning system;identify a revised security response based on the revised security analysis; and send instructions to the security system output device to execute the revised security response.

26. The system of claim 25, wherein identifying the revised security response includes generating an LLM prompt at the Al reasoning system based on the security response outcome.

27. The system of claim 18, the instructions further causing the Al reasoning system to identify a security severity associated with the security response, wherein sending instructions to the security system output device includes sending an indication of the security severity.

28. The system of claim 18, wherein the contextual security analysis includes at least one of a security system status or an identified potential security threat.

29. The system of claim 18, wherein the one or more security system devices include at least one of a credential reader, a physical access controller, a security sensor, or an input from an external security system.

30. The system of claim 18, wherein: the security response includes initiating an authentication request; and the security system output device includes at least one of a credential reading security system, a biometric security system, a continuous authentication security system, a personal identification number security system, an alerting security system, a logging security system, or a notification security system.

31. The system of claim 18, wherein the security input data set of input data includes a non-text data set, the instructions further causing the Al reasoning system to generate a text data set based on the security input data set of input data for use by the Al reasoning system.

32. The system of claim 31, wherein the non-text data set includes at least one of a video input data set or an audio input data set.

33. The system of claim 18, wherein: the Al reasoning system includes a multimodal LLM model; and the security input data set of input data includes a plurality of different input types.

34. The system of claim 18, wherein: the Al reasoning system includes a plurality of agents within a multi-agent system, each of the plurality of agents within the multi-agent system configured to receive predetermined security input types; and the multi-agent system is configured to allocate a plurality of security tasks to a subset of the plurality of agents within the multi-agent system.

35. A non-transitory computer-readable medium storing instructions that, when executed by a processor of a device, cause the device to: receive a security input data set of input data from one or more security system devices; generate a contextual security analysis at an artificial intelligence (Al) reasoning system based on the security input data set of input data, the contextual security analysis including a security context and at least one identified security threat; identify a security response based on the contextual security analysis; and send instructions to a security system output device to execute the security response.

36. The non-transitory computer-readable medium of claim 35, the instructions further causing the device to: receive, at a directive input subsystem, a security directive from an authorized user interface subsequent to generating the contextual security analysis; and update the Al reasoning system based on the security directive.

37. The non-transitory computer-readable medium of claim 36, wherein identifying the security response is further based on the security directive.

38. The non-transitory computer-readable medium of claim 36, the instructions further causing the device to authenticate a source of the security directive to ensure the security directive originates from an authorized user.

39. The non-transitory computer-readable medium of claim 35, the instructions further causing the device to: generate a reasoning action plan at a multi-step reasoning framework based on the security input data set of input data; and identify a plurality of security response actions and an associated action sequence based on the reasoning action plan; wherein generating the security response is further based on the plurality of security response actions and the associated action sequence.

40. The non-transitory computer-readable medium of claim 39, wherein: the plurality of security response actions includes at least one of alerting security personnel, activating physical security measures, or adjusting surveillance system settings; and plurality of security response actions is prioritized based on at least one of an urgency or a severity associated with the at least one identified security threat.

41. The non-transitory computer-readable medium of claim 35, the instructions further causing the device to: receive a plurality of security response action outcomes; and revise the Al reasoning system based on plurality of security response action outcomes.

42. The non-transitory computer-readable medium of claim 35, the instructions further causing the device to: receive a security response outcome; generate a revised security analysis at the Al reasoning system; identify a revised security response based on the revised security analysis; and send instructions to the security system output device to execute the revised security response.

43. The non-transitory computer-readable medium of claim 42, wherein identifying the revised security response includes generating an LLM prompt at the Al reasoning system based on the security response outcome.

44. The non-transitory computer-readable medium of claim 35, the instructions further causing the device to identify a security severity associated with the security response, wherein sending instructions to the security system output device includes sending an indication of the security severity.

45. The non-transitory computer-readable medium of claim 35, wherein the contextual security analysis includes at least one of a security system status or an identified potential security threat.

46. The non-transitory computer-readable medium of claim 35, wherein the security system output device includes at least one of a credential reader, a physical access controller, a security sensor, or an input from an external security system.

47. The non-transitory computer-readable medium of claim 35, wherein: the security response includes initiating an authentication request; and the security system output device includes at least one of a credential reading security system, a biometric security system, a continuous authentication security system, a personal identification number security system, an alerting security system, a logging security system, or a notification security system.

48. The non-transitory computer-readable medium of claim 35, wherein the security input data set of input data includes a non-text data set, the instructions further causing the device to generate a text data set based on the security input data set of input data for use by the Al reasoning system.

49. The non-transitory computer-readable medium of claim 48, wherein the non-text data set includes at least one of a video input data set or an audio input data set.

50. The non-transitory computer-readable medium of claim 35, wherein: the Al reasoning system includes a multimodal LLM model; and the security input data set of input data includes a plurality of different input types.

51. The non-transitory computer-readable medium of claim 35, wherein:the Al reasoning system includes a plurality of agents within a multi-agent system, each of the plurality of agents within the multi-agent system configured to receive predetermined security input types; and the multi-agent system is configured to allocate a plurality of security tasks to a subset of the plurality of agents within the multi-agent system.

Images