Detection and threat assessment system and method

Abstract

A detection and threat assessment system includes a memory for computer executable components and a processor to execute the computer executable components. In embodiments, the computer executable components may include a pose estimation component, a region of interest component, and a cloud-based recognition component. A pose estimation component may generate a pose estimation representation of a patron including one or more coordinates representing the location of one or more body parts. A region of interest component may determine if the pose estimation representation is within a region of interest. A cloud-based recognition component may compare detected facial information of the patron with a database of user information to identify the person. Further, the detection and threat assessment system may comprise a weapons detection component including at least one of an ultra-wideband (UWB) radar, a metal detector, and a magnetometer to determine if the patron possesses a weapon.

Description

DETECTION AND THREAT ASSESSMENT SYSTEM AND METHODRELATED APPLICATIONS

[0001] This application claims the benefit of priority to U.S. Provisional Application No. 63 / 697,241 , filed September 20, 2024, and U.S. Provisional Application No. 63 / 697,230, filed September 20, 2024, the disclosures of which arc incorporated herein by reference in their entireties.TECHNICAL FIELD

[0002] The present disclosure generally relates to a detection and threat assessment system and method, including a detection and threat assessment system that can provide, inter alia, real-time advanced face recognition with queue management to improve security and accuracy.BACKGROUND

[0003] Facial recognition systems have significantly advanced over the past few decades, transitioning from rudimentary pattern recognition techniques to sophisticated deep learning models. These systems are designed to identify or verify individuals by analyzing and comparing facial features. Modem face recognition technologies typically involve several key processes: face detection, feature extraction, and face matching. Convolutional Neural Networks (CNNs) and other advanced machine learning algorithms have been instrumental in improving the accuracy and efficiency of these systems. They are now widely deployed in various applications, including security and surveillance, user authentication on mobile devices, and social media tagging.

[0004] Despite these advancements, current face recognition systems may still encounter substantial challenges. Issues related to accuracy, latency, and computational demands can significantly impact their usefulness and / or performance, particularly in dynamic environments where individuals may be partially occluded and / or located at varying distances from the camera.

[0005] Accuracy may be significant for the effectiveness of face recognition systems. However, several factors can adversely affect their performance. For example, variations in lighting, facialexpressions, and occlusions (such as glasses, masks, or facial hair) can change or degrade the accuracy of such systems.

[0006] Latency, the delay between the input of facial data and the output of recognition results, may be a significant challenge in face recognition systems. High latency can hinder the effectiveness of real-time applications such as live surveillance and instant user authentication. Factors contributing to high latency include the complexity of the algorithms, the size of the dataset being processed, and the computational resources available. Reducing latency without materially compromising accuracy can require optimizing both the software and hardware components of the face recognition system.

[0007] Additionally, facial recognition systems can be computationally intensive, requiring substantial processing power to analyze and match facial features accurately. The need for high computational resources can be a significant barrier, particularly in environments with limited hardware capabilities. This challenge is intensified by the increasing demand for higher resolution images and more sophisticated algorithms, which can further strain computational resources. Efficient resource management and the development of more efficient algorithms can help to mitigate the high computational demands of face recognition systems.

[0008] In addition to facial recognition systems, weapons detection systems can play an important role in enhancing public safety, such as in environments vulnerable to violence or attacks, such as airports, schools, government buildings, and large public gatherings. Conventional weapon detection methods, such as metal detectors and manual searches, can be instrumental in mitigating risks but possess inherent limitations, including a narrow detection range, limited material recognition capabilities, and a high likelihood of false alarms. These traditional systems primarily rely on the detection of metal objects, making it difficult to identify non-metallic threats or those concealed in innovative ways. Additionally, manual search methods are labor-intensive and can be prone to human error, which may reduce overall system reliability.

[0009] Thus, there is a need for improved detection and threat assessment systems that are accurate and useful in light of challenges related to capturing identifying information associated with an individual passing through a security zone.

[0010] According to the present disclosure, there is provided a detection and threat assessment system, as set forth in the appended claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0011] While the claims are not limited to a specific illustration, an appreciation of various aspects may be gained through a discussion of various examples. The drawings are not necessarily to scale, and certain features may be exaggerated or hidden to better illustrate and explain an innovative aspect of an example. Further, the exemplary illustrations described herein are not exhaustive or otherwise limiting, and are not restricted to the precise form and configuration shown in the drawings or disclosed in the following detailed description. Exemplary illustrations are described in detail by referring to the drawings as follows:

[0012] FIG. 1 is a block diagram system view of an embodiment of a detection and threat assessment system according to teachings of the present disclosure.

[0013] FIG. 2 is a block diagram system view of an embodiment of a detection and threat assessment system according to teachings of the present disclosure.

[0014] FIG. 3 is a block diagram system view of an embodiment of a detection and threat assessment system according to teachings of the present disclosure.

[0015] FIG. 4 is a diagram view of a pose estimation representation and associated pose keypoints for a variety of body parts of a patron according to teachings of the present disclosure.

[0016] FIG. 5 is a block diagram system view of an embodiment of a detection and threat assessment system according to teachings of the present disclosure.

[0017] FIG. 6 is a system flow diagram of an embodiment of a detection and threat assessment system according to teachings of the present disclosure.

[0018] FIGS. 7A, 7B, and 7C are perspective views illustrating a housing and / or ROI of an embodiment of a detection and threat assessment system according to teachings of the present disclosure.

[0019] FIG. 8A and 8B arc perspective views illustrating the identification of one or more patrons within the ROI via the pose estimation system according to teachings of the present disclosure.

[0020] FIG. 9A is a block diagram view illustrating a negative match of a detection and threat assessment system according to teachings of the present disclosure.

[0021] FIG. 9B is a block diagram view illustrating a positive match of a detection and threat assessment system according to teachings of the present disclosure.

[0022] FIG. 10 is a block diagram system view of an embodiment of a detection and threat assessment system according to teachings of the present disclosure.

[0023] FIG. 11 is a block diagram system view of an embodiment of a detection and threat assessment system according to teachings of the present disclosure.

[0024] FIG. 12 is a block diagram system view of an embodiment of a detection and threat assessment system according to teachings of the present disclosure.

[0025] FIG. 13 is a block diagram system view of an embodiment of a detection and threat assessment system according to teachings of the present disclosure, and with multiple image captures from at least two camera sources.

[0026] FIG. 14 is a system flow diagram of an embodiment of a detection and threat assessment system according to teachings of the present disclosure, and with a second-look approach.

[0027] FIGS. 15A, 15B, 15C, and 15D are image captures illustrating a patron passing through an embodiment of a detection and threat assessment system according to teachings of the present disclosure.

[0028] FIGS. 16A and 16B are image captures illustrating patrons identified as possessing weapons via embodiments of a detection and threat assessment system according to teachings of the present disclosure.

[0029] FIG. 17 is a system flow diagram view of an embodiment of a detection and threat assessment system analyzing visual weapons, concealed weapons, and pose estimation data according to teachings of the present disclosure.

[0030] FIG. 18 is a system flow diagram view of an embodiment of a detection and threat assessment system operable for face detection and recognition according to teachings of the present disclosure.

[0031] FIG. 19 is a system flow diagram view of an embodiment of a detection and threat assessment system operable for queue management and timestamp synchronization according to teachings of the present disclosure.

[0032] FIG. 20 is a system flow diagram view of an embodiment of a detection and threat assessment system operable for determining pose estimation representations of patrons within the region of interest according to teachings of the present disclosure.

[0033] FIG. 21 is a system flow diagram view of an embodiment of a detection and threat assessment system operable for determining pose estimation representations of patrons within the region of interest according to teachings of the present disclosure.DETAILED DESCRIPTION

[0034] In the drawings, where like numerals and characters indicate like or corresponding pails throughout the several views, exemplary illustrations are shown in detail. The various features of the exemplary approaches illustrated and described with reference to any one of the figures may be combined with features illustrated in one or more other figures, as it will be understood that alternative illustrations that may not be explicitly illustrated or described may be able to be produced. The combinations of features illustrated provide representative approaches for typical applications. However, various combinations and modifications of the features consistent with the teachings of the present disclosure may be desired for particular applications or implementations.

[0035] This application incorporates by reference in its entirety PCT Application, PCT / US2019 / 023347, filed March 21, 2019, entitled “System and Method for detecting object patterns using ultra-wideband (UWB) radar.” The entire contents of which are hereby incorporated by reference as if set forth fully herein, including any figures, tables, and claims.

[0036] In embodiments, such as generally illustrated in FIG. 1, a detection and threat assessment system 100 may be operable to determine the presence of a concealed / visual weapon (e.g., object of interest, contraband, etc.) and / or may identify a patron (e.g., individual / person / by slander) walking within a region of interest (or RO I). The detection and threat assessment system 100 may comprise a camera module 102, which may be operatively or communicatively connected with one or more other modules / sy stems. Further, the detection and threat assessment system 100 may include a pose estimation module 104, a face recognition module 106 (e.g., a cloud-based face recognition module), and / or a visual weapons detection module 108. Additionally, the detection and threat assessment system 100 may include a central Al server 110 configured to manage processing and / or transmission of data from the connected modules / systems. As generally illustrated, the pose estimation module 104, the face recognition module 106, and / or the visual weapon detection module 108 may be communicatively connected with a server - e.g., central Al server 110.

[0037] In embodiments, an Al concealed weapon detection module 112 may be operable to determine the presence of a concealed object (e.g., weapon, contraband, etc.). Further, the Al concealed weapon detection module 112 may identify a walk-type of a patron disposed within a region of interest. In situations where a patron is walking in an abnormal manner (e.g., such as favoring the left or right side), the patron may be concealing a weapon (e.g., an object of interest); and in situations where a patron is walking in a normal manner, the patron may not be concealing a weapon. A display 114 may be communicatively connected with the central Al server 110 such that a variety of notifications (e.g., audio and / or visual) may be detectable. The display 114 may include any input or output device to facilitate receipt or presentation of information in audio, visual, or tactile form, or a combination thereof. Examples of a display 1 14 may include, without limitation, a touchscreen, cathode ray tube display, light-emitting diode display, electroluminescent display, electronic paper, plasma display panel, liquid crystal display, high- performance addressing display, thin-film transistor display, organic light-emitting diode display, surface-conduction electron-emitter display, laser TV, carbon nanotubes, quantum dot display, interferometric modulator display, projector device, and the like. Notifications transmitted to the display may relate to patron identities, ticket validations, concealed weapons, and visible weapons (objects of interest), among other things. Display 114 may include any number and / or size of displays 114 indicating any variety of information associated with identification and / or object detection.

[0038] With further reference to FIG 1, a central Al server 110 may manage the communication / activation of one or more connected applications and / or modules. For example and without limitation, the central Al server 110 may integrate one or more network(s) 116, server(s) 118, and / or database(s) 120 with connected applications and / or modules.

[0039] In embodiments, such as generally illustrated in FIG. 2, a detection and threat assessment system 100 may integrate face detection, pose estimation, and / or queue management to improve security and accuracy of weapon and pose detection / identification, among other things. The detection and threat assessment system 100 may capture video frames, detect faces, evaluate pose using multiple keypoints (which may also be referred to as landmarks, pose estimation representation coordinates), and / or use a queue to manage asynchronous processing. The detection and threat assessment system 100 may be configured for the aforementioned operations and / ormay include modules and / or systems to execute one or more various functions of the detection and threat assessment system 100 (e.g., such as illustrated by FIGS. 1, 2, and 3).

[0040] Further, the detection and threat assessment system 100 may validate the presence and position of individuals within a region of interest (ROI) and may communicate with a machine learning module (e.g., central Al server 110 and / or another external module / system) for weapon detection and / or for assigning one or more persons with unique identifying information (e.g., correlation with facial features, clothing, and other identifying personal information).

[0041] Generally, as shown in FIG. 2, the detection and threat assessment system 100 may include one or more of a variety of components and / or modules. For example, the detection and threat assessment system 100 may include shared components, such as a processor 200 and / or a memory 202. The detection and threat assessment system 100 (e.g., the facial recognition pipeline) may include a variety of modules / components such as: a camera module 102, a pose estimation module 104, a ROI module 210, a face detection module 212, a face filter module 214, a cloudbased face recognition module 106, and / or a queue management module 216. The one or more various modules / components may be communicatively connected to one another to perform functions of the detection and threat assessment system 100. Additionally or alternatively, one or more modules / components may include associated processors and memory.

[0042] In embodiments, the camera module 102 may be configured to capture video frames with at least 5 frames per second (fps). The camera module 102 may be connected with one or more cameras and / or camera systems (e.g., which may include connecting with existing camera systems) to receive video input. Video input received via the camera module 102 may be further analyzed by other various modules of the detection and threat assessment system 100 (e.g., the ROI module 210, the face filter module 214, and / or the face recognition module 106).

[0043] With examples, the face detection module 212 may be configured to parse video information received via the camera module 102 to detect faces (of patrons / persons) within the captured frames. The camera module 102 may communicate with the face detection module 212 for processing the video information. Further, the camera module 102 may include and / or be connected with one or more advanced face detection algorithms via the face detection module 212 (e.g., updated / external detection algorithms may be applied to the detection and threat assessment system 100). The one or more advanced face detection algorithms may detect faces within the received video information. For example, the face detection module 212 may, in real-time,determine the sizes of one or more detected faces. The detection and threat assessment system 100 may be used in connection with large venues where multiple patrons may be present in a captured frame; therefore, the face detection module 212 may sort / filter the detected faces by proximity. For faces that are disposed closer to the camera and / or camera system (e.g., of the camera module 102), the face detection module 212 may detect a larger face size. Similarly, for faces that are disposed farther from the camera and / or camera system (e.g., of the camera module 102), the face detection module 212 may detect a smaller face size.

[0044] Additionally, the face detection module 212 may filter out one or more faces that are not large enough (e.g., faces that are not substantially close to the camera module 102) based on a threshold face size. With embodiments, the threshold face size may be about 5,000 pixels or more or less depending on the venue and proximate area. Where larger quantities of patrons pass through a surveyed area, the threshold face size may be larger, about 7,000 pixels or more or less to accommodate for the increased traffic. In further examples, the threshold face size may be as low as 500 pixels to accommodate a low-power operation mode (e.g., where lower resolution cameras can be included in the camera module 102). The threshold face size may be greater than 7,000 pixels in to accommodate a high-power operation mode (e.g., where higher resolution cameras may be included in the camera module 106). The threshold face size may include any number of pixels that can accomplish facial recognition. In alternative embodiments, functions of the face detection module 212 may be conducted via the pose estimation module 104.

[0045] In embodiments, the pose estimation module 104 may be configured to conduct pose estimation on one or more patrons with detected faces that meet the threshold face size. For example, once the face detection module 212 identifies a patron’s face that meets the threshold face size, the pose estimation module 104 may be operable to determine a pose (e.g., a walk / position) of the identified patron.

[0046] With embodiments, the pose estimation module 104 may conduct pose estimation for a selection of patrons. For example and without limitation, the selection may include patrons attempting entry onto the physical premises of a venue (e.g., through a gate or checkpoint) that meet the threshold face size. Such patrons may walk by and / or be disposed substantially proximate the camera or camera systems (e.g., the camera module 102) connected with the detection and threat assessment system 100. Upon identifying the patrons to analyze (e.g., via the face detection module 212 and / or the pose estimation module 104), a plurality of pose keypoints may begenerated to assist in determining the position of various body parts. For example and without limitation, body parts (e.g., pose keypoints) may include a nose, neck, shoulder, elbow, wrist, hip, knee, ankle, and / or ear. Further, the pose estimation module 104 may associate one or more body parts (e.g., pose keypoints) with a left side and / or right side of the body. As generally illustrated in FIG. 4, the pose estimation module 104 may generate a pose estimation representation 400 comprising the various pose keypoints for one or more patrons as they walk proximate to the camera or connected camera systems (e.g., the camera module 102).

[0047] With embodiments, the one or more various pose keypoints may connect to form the pose estimation representation 400. Further, the pose estimation module 104 may utilize the pose estimation representation 400 to determine a pose of an associated patron. The pose estimation representation 400 may indicate whether a patron is walking or stationary, facing a certain direction, as well as other physiological information (e.g., health related information). For example, once identifying one or more extremities (e.g., legs, arms, shoulder, and / or feet), it can be determined whether a patron is in a walking position, stationary, or other position. The pose estimation module 104 may utilize a coordinate comparison algorithm in analyzing / processing one or more pose keypoints. Additionally, the coordinate comparison algorithm may be operable to determine whether pose keypoints are within the ROI, and / or may analyze movement patterns for one or more pose keypoints.

[0048] In further embodiments, the pose estimation module 104 may be configured to carry out the functions of the face detection module 212, and / or may be configured to remove the detected face from the remainder of the pose estimation representation 400 for one or more patrons (e.g., the pose estimation module 104 may crop out a face region (e.g., upper region) from the pose estimation representation 400). For example, the pose estimation module 104 may remove pose keypoints that reside above the shoulder keypoints (keypoints 5 and 6) for further facial recognition processing. As generally shown in FIG. 4, the pose estimation representation 400 may include pose keypoints associated with a nose (0), left ear (3), and right car (4). The pose estimation module 102 may determine an orientation of a patron’s face via the detectable pose keypoints. For example and without limitation, when the left ear (3) may be detected and the right ear (4) is not, the patron’s face may be determined to be facing to the right. Similarly, when the right ear (4) may be detected and the left ear (3) is not, the patron’s face may be determined to be facing to the left.

[0049] Further, upon detecting pose keypoints associated with the face of a patron, the pose estimation module 104 may crop said detected pose keypoints from the remainder of the pose estimation representation 400 and may utilize additional and / or alternative means of facial identification.

[0050] Additionally, the detection and threat assessment system 100 may utilize a Region of Interest (RO I) module 210 that may communicate with the one or more various modules of the detection and threat assessment system 100. The RO1 module 210 may be configured to determine whether a patron is within the ROI if one or more body parts fall within a threshold region. For example, the threshold region may indicate a distance for a patron to be within the camera module 102. Further, the ROI module 210 may define a region and / or a frame / cell that may be further used to filter out undesired individuals (e.g., persons that are not attempting entry into a venue, and are merely proximate the camera module 102). The ROI moule 210 may remove patrons standing nearby and / or outside the defined frame / cell if key body parts are not disposed within the ROI. For example and without limitation, the ROI module 210 may determine if the shoulders (5,6), hips (11,12), and / or nose (0) are within the defined ROI. If one or more of a variety of body parts of an individual are within the defined frame / cell, but do not include the key body parts then it can be determined that the individual is not substantially proximate the camera module 102 for recognition. In examples, an ankle (14, 15), knee (13, 14), and / or ear (3,4) may be within the defined region of interest without additional detected pose keypoints; therefore, the ROI module 210 may determine that the associated patron / individual is not substantially within the ROI.

[0051] With examples, as multiple patrons / individuals pass by an entrance proximate the camera module 102, the ROI module 210 may filter the patrons to be analyzed for entry from the patrons that are passing by. Such a filtering process may be conducted via the face filter module 214 and / or the face recognition module 106. Further, the face filter module 214 and / or the face recognition module 106 may determine if the patron is walking within the ROI via information from the ROI module 210 and may detect a facial image of the patron via the face filter module 214 and / or the face recognition module 106 to be used for further analysis. With alternative embodiments, filtering and / or detection may be carried out via the pose estimation module 104, or any of the aforementioned modules. The face filter module 214 and / or the face recognition module 106 may capture / save a facial image of the patron and may use the facial image for further processing. Further processing may include actions such as identifying persons in the facial image,applying filters, and other actions associated with processing an image for analysis (e.g., which may involve communicating with other modules of the detection and threat assessment system 100).

[0052] In examples, such as generally illustrated in FIG. 3, the central Al server 110 and / or the queue management module 216 may maintain a queue of facial information (e.g., of patrons detected via the camera module 102). For example and without limitation, facial information may include the facial images as identified via the face filter module 214 and / or the face recognition module 106. The queue management module 216 and / or the central Al server 110 may manage the asynchronous processing of facial data, as well as the results from a connected cloud-based recognition module 500 (e.g., a cloud-based API as shown in FIG. 5). Further, the queue management module 216 may assist multiple components of the pipeline to run in parallel (e.g., as generally illustrated in FIGS. 5 and 6).

[0053] With embodiments, the queue management module 216 may include a variety of one or more threads. For example and without limitation, the queue management module 216 may include a capture thread and / or a processing thread. The capture thread may be responsible for capturing data and receiving responses from the cloud-based API (e.g., the cloud-based recognition module 500). Additionally, the capture thread may facilitate storing the captured data in the queue, for example, by prompting the data for processing via the processing thread. In examples, the processing thread may be responsible for monitoring the queue. Generally, the processing thread may decide on a course of action for each piece of data, and further, may communicate with external systems communicatively connected with the detection and threat assessment system 100, which may include one or more (machine-learning) weapons detection systems / modules (e.g., see FIG. 3). A function of the queue management module 216 may also involve monitoring timing of information received by the detection and threat assessment system 100 (e.g., and the connected modules). Further, the queue management module 216 may monitor whether the cloud-based API (e.g., the cloud-based recognition module 500) response time exceeds a threshold value. Such a threshold value may be set via a user interface or input, and / or may be automatically set based on a standard of performance for the detection and threat assessment system 100. If the time exceeds the threshold value, the queue management module 216 may discard the result and process the subsequent data waiting in the queue. For example, if facial data recognition is not completed within the threshold window, the detection and threat assessment system 100 may discard thedetected facial data and may preprocess the subsequently detected patron (e.g., the subsequent facial data awaiting in the queue).

[0054] In embodiments, such as generally illustrated in FIG. 3, the detection and threat assessment system 100 may include a radar module 300 (e.g., which may include at least one of an ultra-wideband (UWB) radar and / or a metal detector or magnetometer) to detect the presence of weapons (e.g., or objects of interest). The queue management module 216 and / or the central Al server 110 may generate one or more timestamps for data of the face recognition module 106 and / or the pose estimation module 104. Further, the camera module 102 may capture frames with associated timestamps from the pose estimation module 104 and / or the Al concealed weapon detection module 112. As generally illustrated, the pose estimation module 104 may communicate with the face recognition module 106 and / or the visual weapon detection module 108 to generate a facial result 302 (buffered with timestamps) and / or a visual weapon result 304 (buffered with timestamps). Additionally, the radar module 300 may communicate with the Al concealed weapon detection system 112 to generate a concealed weapon result 306 (buffered with timestamps). The central Al server 110 may be operable to sync all buffered results via the corresponding timestamps (e.g., via a timestamp comparison algorithm), which may be transmitted to the display 114. In this manner, the queue management module 216 and / or the central Al server 110 may manage high traffic areas without performance degradation.

[0055] In further examples, the cloud-based recognition module 500 may be configured to handle / manage / execute facial recognition and user identification processes. Generally, the cloudbased recognition module 500 may receive facial information (e.g., via the facial recognition module 106) and may compare the received facial information with information on a connected database 120 (e.g., a facial user-identification based database connected via one or more networks as illustrated in FIG. 3). The connected database 120 may comprise a directory of user profiles including associated patron identifying information, such as facial images for comparison with detected facial information. The cloud-based recognition module 500 may be connected to the database 120, and / or the database 120 may generally be connected with the detection and threat assessment system 100 to facilitate analyzing facial information of patrons entering a venue. The cloud-based recognition module 500 may search for a match for the detected facial information, and upon confirming a match, may produce other information associated with the patron (e.g., ticket information, prior entry information, and / or other biometric and / or personal information).Identified patrons may have an associated user profile that contains a variety of other information associated with the patron. In cases where no user profile can be matched with the detected facial information, the cloud-based recognition module 500 (e.g., or the facial recognition module 106) may generate a user profile for the patron.

[0056] The block diagram of FIG. 5 generally illustrates the architecture of the detection and threat assessment system 100, which may include one or more of the aforementioned modules. The detection and threat assessment system 100 may include a first stage 5A, a second stage 5B, and / or a third stage 5C of operation.

[0057] For example and without limitation, in the first stage 5A of operation, detection and threat assessment system 100 may determine if a patron is walking through the region of interest (e.g., a ROI). A ROI may be a door, an opening, a gate, a walkway (e.g., a fixed walkable area) and / or a checkpoint for a venue (e.g., see FIGS. 7A-7C for an example of such a gate and / or cell). The gate and / or cell may include a housing to store components associated with the camera module 102, the face detection module 212, the pose estimation module 104, the ROI module 210, the face detection module 212, the face filter module 214, the face recognition module 106, the queue management module 216, and / or the cloud-based recognition module 500 (e.g., a cloud-based API for face matching). Additionally, the gate and / or cell may include a display 114 with visual status indicators to alert security personnel as patrons pass. The detection and threat assessment system 100 may determine whether the patron is intending to enter the venue based on proximity to the region of interest (e.g., the entry gate / walkway). Patrons that have a set number of pose keypoints (or specific pose keypoint) within the region of interest may be considered to be entering the venue. The set number of pose keypoints may correspond with a proximity to the camera module 102 and / or walkway. The set number or specific keypoints may be adjusted for venue and / or accuracy of security.

[0058] With examples, the first stage 5A of operation may include the face detection module 212, the face filter module 214, and / or the pose estimation module 104 identifying facial and / or pose information of one or more patrons within the ROI. Once a patron and an associated position are identified (e.g., via the pose estimation representation 400), the facial recognition module 106 may assign facial recognition information to the patron.

[0059] In embodiments, such as during the second stage 5B of operation, the detection and threat assessment system 100 may facilitate identifying facial information and / or weapons information.For example and without limitation, the face detection module 212, the face filter module 214, and / or the pose estimation module 104 may capture facial information for further processing and analysis after identifying the patron is entering the venue via pose estimation (e.g., via the first stage 5A of operation). The detection and threat assessment system 100 may include a machine learning visual weapons detection system 108, which may be configured to analyze the presence of a weapon in response to identifying a pose estimation representation 400 and / or walk of a patron. In the second stage 5B of operation, the face detection module 212 and / or the face recognition module 106 may identify a user profile associated with the detected facial information. In further examples, such as in the third stage 5C of operation, the detection and threat assessment system 100 may facilitate detecting weapons via connected external systems and modules. Various connected external systems and modules may include cloud serverless functions, cloud object storage, a dashboard, and a cloud database.

[0060] In further examples, such as generally illustrated in FIGS. 1, 2, 3, and 5, the detection and threat assessment system 100 may be connected with one or more of a variety of external systems, modules, networks, and / or devices. Aspects of systems (e.g., the pose estimation system 100A) apparatuses, and / or processes described in this disclosure can constitute machineexecutable component(s) embodied within the machine(s), e.g., embodied in one or more computer readable mediums (or media) associated with one or more machines. Such component(s), when executed by the one or more machines, e.g., computer(s), computing device(s), virtual machine(s), etc. can cause the machine(s) to perform the operations described. Repetitive description of like elements employed in one or more embodiments described herein is omitted for sake of brevity.

[0061] With examples, the detection and threat assessment system 100 can optionally include a server device (or “server”), one or more networks, and one or more devices (not shown). The pose estimation system 100 can also include or otherwise be associated with at least one processor 200 that executes computer executable components stored in memory 202. The detection and threat assessment system 100 may further include a system bus that can couple various components including, but not limited to, a camera module 102 (or camera), a face detection module 212 (or face detector), a pose estimation module 104 (or pose estimator); a ROI module 210, a face detection module 212 (or face detector); a face filter module 214 (or face filter); a queue management module 216 (or queue manager); a cloud face recognition module 106; a visual weapons detection system / module 108 (or weapons detector); a machine learning (ML) module(or ML engine), and an input source module (or input source), e.g., which may be configured to receive any variety of input information, as well as associated input / output (I / O) interfaces. The detection and threat assessment system 100 can be any suitable computing device (also referred to as “computer”) or set of computing devices that can be communicatively coupled to devices, nonlimiting examples of which can include, but are not limited to, a server computer, a computer, a mobile computer, a mainframe computer, an automated testing system, a network storage device, a communication device, a web server device, a network switching device, a network routing device, a gateway device, a network hub device, a network bridge device, a control system, or any other suitable computing device. A device can be any device that can communicate information with the detection and threat assessment system 100 and / or any other suitable device that can employ information provided by the detection and threat assessment system 100. It is to be appreciated that the detection and threat assessment system 100, components, models or devices can be configured to communicate with other networks and other computing devices.

[0062] With examples, FIG. 6 generally illustrates a block flow diagram of the detection and threat assessment system 100 during the first stage 5A, the second stage 5B, and / or the third stage 5C of operation. Further, the detection and threat assessment system 100 may capture video frames at 5 fps to capture pose and / or facial data of one or more patrons. The detection and threat assessment system 100 may evaluate a pose (e.g., pose estimation representation) of a patron that falls within the ROI. The ROI module 210 may determine whether patrons are walking within the ROI by analyzing the coordinates of the various pose keypoints (e.g., via the pose estimation representation 400). For example and without limitation, the pose estimation module 108 may confirm whether the location of a patron’s centroid and hip keypoints are within the ROI. If the specified keypoints are determined to be within the ROI, the facial information of the patron may be used for further processing and identification (e.g., via the cloud-based recognition module 500 communicating with a database 120 to confinn an identity / profile). For patrons that are walking within the ROI and include facial information of at least the threshold pixel value, the facial information of the associated patron may be used for further processing. The pose estimation module 104 may check for multiple patrons within a frame and / or may validate the orientation of the multiple patrons. If a patron is within the ROI and the orientation of the patron is facing the gate / walkway, the detection and threat assessment system 100 may engage in facial identification.

[0063] Faces that are at least about 5,000 pixels (e.g., threshold pixel value) may be processed, and other faces that do not meet the threshold pixel value may be removed. For example and without limitation, the pose estimation module 104 (e.g., or any of the connected modules) may apply a Yunet face detector algorithm. Generally, faces that do not meet the threshold pixel value may be associated with patrons that are too far from the camera module / gate and are not attempting to enter the venue.

[0064] As one or more faces are confirmed within the RO1, the detection and threat assessment system 100 may use the detected facial information (e.g., exceeding 5,000 pixels) to conduct facial recognition. Conducting facial recognition, via the face recognition module 106 may include matching the detected facial information with an existing user profile including associated information. In cases where a user profile cannot be matched with the identified patron, a new user profile may be generated for future use.

[0065] In embodiments, a queue of facial information may be maintained via the queue management module 216 (e.g., the central Al server 110) to manage asynchronous processing. Further, the queue management module 216 may communicate with one or more weapon detection systems, face detection modules, filter and detection modules, pose estimation modules, and / or databases to facilitate weapons detection, facial identification, and / or cloud API responses. The cloud-based API (e.g., the cloud-based recognition module 500) may facilitate comparing the detected face image against a database 120 so that the detection and threat assessment system 100 may identify the user profile (e.g., the user information) if a match is found. Generally, the detection and threat assessment system 100 may ensure accurate face detection by integrating realtime body part identification with machine learning weapons detection models, validating walks via pose estimation, and / or efficiently managing data processing via the queue management module 216. The system 100 may be operable to employ visual and / or concealed weapon detection after or during facial recognition processes.

[0066] A method of pose estimation may include prompting the camera module (e.g., the camera and / or camera systems) to capture data. The method may include determining if one or more patrons are walking within the ROI. For example and without limitation, the camera module may capture video at least about 5 fps or more or less (e.g., depending on the desired accuracy of the system 100). The method of pose estimation may include determining whether detected faces (e.g., via the camera module and / or face detection module) include at least the threshold pixelamount (e.g., in some cases about 5,000 pixels or more). In this manner, the face detection module may filter out patrons that do not have an associated facial image satisfying the threshold pixel amount (e.g., patrons may be facing away from a camera or may be too far for the camera to register a sufficient image). Further, the method may include detecting and analyzing pose keypoints for one or more patrons to determine if the one or more patrons are within the designated ROI. If one or more identified pose keypoints

[0067] The method of pose estimation may include transmitting the facial information of the patrons within the ROI to a cloud-based recognition service, which may be via the cloud-based recognition module.

[0068] In embodiments, the method may include maintaining a queue of facial information such that the pose estimation method may be executed / conducted in an asynchronous manner. The method may include comparing the queue of facial information with a weapons detection machine learning (ML) model for tagging facial information. For example and without limitation, the weapons detection ML model may assign information generated via the weapons detection ML model with the facial information. Additionally, in an asynchronous manner, the method may include comparing the facial information with a database of facial information in addition to detecting weapons. The method may include matching the facial information of detected patrons within the ROI with existing facial information on a connected database (e.g., a collection of facial data). In this manner, the method of pose estimation may include generating a notification upon determining a match and / or a notification that no match exists.

[0069] In embodiments, such as generally illustrated in FIGS. 7A, 7B, and 7C, the detection and threat assessment system 100 may include a walkway 700 disposed between a first partition 702 (stanchion / barrier), and / or a second partition 704 (stanchion / barrier). The partitions 702, 704 may guide one or more patrons to the walkway 700 (e.g., the ROI) where the detection and threat assessment system 100 may process entry and scan for weapons. The display 114 (e.g., an LED panel) of the detection and threat assessment system 100 may indicate the status of a security check. The display 114 may indicate whether a weapon (e.g., or other prohibited items) is disposed on a patron passing through. In examples, the display 114 may indicate the location of an identified weapon on the patron. The camera module 102 may generate one or more images from one or more angles with associated timestamps to increase the accuracy and processing of weapon detection. The camera module 102 may capture images of patrons within 2 meters of the partitions702, 704 (e.g., which may correspond with the defined ROI). Using an image taken in closeproximity of the patron may provide for sufficient resolution, allowing for lower-resolution and lower-cost cameras to be included in the camera module 102. As is considered, with iterations of weapon detection, the central Al server 110 may be trained to detect objects of interest. Further, one or more UWB radars, metal detectors, and / or magnetometers may be disposed on and / or in any of the partitions 702, 704.

[0070] With embodiments, the detection and threat assessment system 100 may be operable to evaluate if a weapon is present within the scanning area. The detection and threat assessment system 100 may include at least one ultra-wideband (UWB) radar and at least one metal detector or magnetometer, each configured to detect contraband moving through a region-of-interest (ROT) or the scanning area (e.g., which may comprise the radar module 300). The detection and threat assessment system 100 employ the radar module 300 (e.g., a UWB radar) to detect an object of interest (e.g., a weapon, contraband, etc.) that may be present on or near a patron as the patron moves through the ROI (e.g., the scanning area / walkway), while the at least one metal detector or magnetometer is employed to detect any contraband that may be present below the individual’s torso as the individual moves through the ROI.

[0071] In examples, the radar module 300 may be operable to determine whether a weapon or object or interest is present on an individual. Generally, the radar module 300 may be trained, via a learning environment, having ground truth data patterns consistent with two conditions: (1) when a weapon or object of interest (OOI) present on the patron; and (2) when no weapon or object of interest is present on the patron. The ground truth data patterns may be obtained in conditions that will be used to detect the presence of a known object, weapon, or object of interest.

[0072] For instance, in the example above, ground truth calibration data patterns are obtained under two conditions - one where an OOI is present on an individual, and a second where the individual is present but the OOI is not. According to the disclosure, the OOI may be a weapon or handgun, but is not limited to such a device and may instead include any material that may be held on an individual that can be identified with a unique signature. Thus, an OOI may include contraband (such as a flask for carrying liquor), a rifle, or any device that may be deemed dangerous or illegal in a particular venue. For the two conditions described, UWB scanning data is obtained by projecting electromagnetic waves from the UWB to the ROI, and data reflected from objects within the ROI is gathered and analyzed to generate the ‘ground truth’ data patterns.In one example, the calibration patterns are obtained on a test stand and without a human present. Additionally, it is contemplated that the environment or ROI where calibration occurs may itself change while screening individuals against a calibrated background. When in use and screening is occurring the ROI may change due to movement of an item within the ROI, such as movement of a background item such as a garbage can or other item or items that form a background signature.

[0073] The resultant training set of calibration data thereby includes the two primary data sets obtained during the calibration step, one that (in this example) includes weapon or OO1 pattern data, and the second having no weapon or pattern OOI data. Both datasets are labeled or otherwise identified as the ground truth datasets that are run through a training or calibration heuristic based on the CNN architecture. The heuristic is a multi-layered heuristic that performs a convolution process on the datasets, ultimately yielding the prediction function. Once the prediction function is set and the calibration data is determined for the given environment where deployed, screening commences in a live environment.

[0074] Systems and techniques discussed herein often refer to employing one or more metal detectors. These systems and techniques, however, may employ either metal detectors or magnetometers or a combination of both. As such, when the term “metal detector” is used herein, it is understood that one or more magnetometers, or some combination of magnetometer(s) and metal detector(s), could instead be employed.

[0075] In examples, such as generally illustrated in FIG. 8A, the detection and threat assessment system 100 may capture video information of one or more patrons. As generally shown, multiple patrons (e.g., Patron A, Patron B, Patron C, Patron D, Patron E) are present within the framecaptures. In a first instance (e.g., FIG. 8A), the first frame-capture, may include a first patron A, a second patron B, and a third patron C. The detection and threat assessment system 100 may detect facial information for the first patron A, the second patron B, and / or the third patron C. As generally shown, the detection and threat assessment system 100 may generate pose estimation representations for the first patron A and / or the second patron B; and may further determine if the coordinates of the one or more pose keypoints are within the defined ROI. Additionally or alternatively, the face detection module 212 may compare detected faces with a threshold of about 5,000 pixels to filter out one or more patrons. The detection and threat assessment system 100 may determine that the third patron C is not within the ROI and therefore may be filtered out from the pose estimation and facial recognition process. The detection and threat assessment system100 continue to match patrons with user profiles of a connected database and may perform other functions associated with security, such as weapons detection.

[0076] Turning to FIG. 8B, illustrating a second frame-capture, the first patron A and the second patron B have passed the gate / cell and one or more new patrons are visible (e.g., a fourth patron D and / or a fifth patron E). The detection and threat assessment system 100 may generate pose estimation representations 400 for the fourth patron D and may regenerate a pose estimation representation 400 for the second patron B (after determining the patrons are within the RO1). Further, the detection and threat assessment system 100 may determine that the fifth patron E is not within the ROI (e.g., via a facial pixel data threshold and / or via confirming coordinates of one or more pose key points).

[0077] With example, FIGS. 9A and 9B generally illustrate the cloud-based recognition module 500 communicating with the queue management module 216 and / or the face recognition module 106 to compare detected facial information with a database 120 of registered facial information (e.g., faces). The detection and threat assessment system 100 may search the database 120 for a match, and if no match is found within a threshold time, the detection and threat assessment system 100 may return a “no match” indication (e.g., FIG. 9A). Conversely, the detection and threat assessment system 100 may return a “positive match” indication if a match is found (e.g., FIG. 9B).

[0078] In embodiments, the detection and threat assessment system 100 may be further configured for detecting concealed weapons passing through the ROI (e.g., via the Al concealed weapons detection system 112), as well as apparent / visually-present weapons (e.g., via the visual weapon detection module 108). The detection and threat assessment system 100 may facilitate identifying the presence of a weapon (e.g., a concealed weapon) and / or a walk of an patron / individual passing through the ROI. Generally, in examples, the detection and threat assessment system 100 may generate a first signal corresponding with pose estimation and visual weapon detection and / or a second signal corresponding with concealed weapons. A connected neural network (e.g., a central Al server 110 as shown in FIG. 3) may determine the presence of a weapon (e.g., visually present or concealed) via the first signal and / or the second signal. The detection and threat assessment system 100 may provide single shot detection, where the system 100 operates in one go managed via the central Al server 110 thus improving efficiency and / or processing time. Output / results of the detection and threat assessment system 100 may betransmitted to the display 114 (e.g., which may comprise a visual and / or audio alert, among other communication) .

[0079] As generally illustrated in FIG. 10, once a patron is determined to be within the ROI, the pose estimation module 104 may determine an associated pose estimation representation 400. Concurrently, earlier, or later, the detection and threat assessment system 100 may be configured to determine the presence of a concealed weapon.

[0080] The radar module 300 may include a variety of sensors, such as a radar sensor 1000 and / or a metal detector sensor 1002. Further, the radar sensor 1000 and / or the metal detector sensor 1002 may be communicatively connected with the Al concealed weapons detection module 112. The Al concealed weapons detection module 112 may detect concealed / hidden weapons that may not be identified via the visual weapon detection module 108. In examples, the radar sensor 1000 of radar module 300 may comprise an ultra-wideband (UWB) radar and / or the metal detector sensor 1002 may comprise a magnetometer configured to detect contraband moving through the ROI.

[0081] In embodiments, the Al concealed weapon detection module 112 may employ the UWB radar of the radar sensor 1000 to detect contraband that may be present on, near, and / or above a patron’ s torso (e.g., or lower body) as the patron moves through the ROI, and / or the metal detector sensor 1002 may detect any contraband that may be present below the patron’s torso (e.g., or torso region) as the patron moves through the ROI. Generally, the Al concealed weapon detection module 112 may integrate results from the radar sensor 1000 and the metal detector sensor 1002 (e.g., via a signal analysis algorithm or weapons detection algorithm) to determine the presence of a concealed weapon, via single shot detection (e.g., which may also be managed via the queue management module 216).

[0082] In examples, single-shot detection (SSD) may be implemented via the detection and threat assessment system 100 for object detection, including applications such as weapon detection, face recognition, and traffic monitoring. SSD refers to a type of deep learning model that can detect objects in images or video frames in a single forward pass through the network, making it both efficient and fast. SSD may be designed to be computationally efficient, allowing for real-time object detection, which is crucial for time-sensitive applications such as surveillance or autonomous systems. The architecture of the detection and threat assessment system 100 may be optimized to process images quickly without sacrificing accuracy (e.g., as evidenced by using a single image to determine the presence of a weapon). The SSD model may use a convolutionalneural network (CNN) as a backbone to extract features from the image. The SSD model may then apply multiple convolutional filters at different scales and aspect ratios, allowing the system to detect objects of various sizes. The CNN may be configured to predict both bounding box coordinates and a confidence score for the presence of an object in each predefined region.

[0083] Systems and techniques discussed herein often refer to employing one or more metal detectors. These systems and techniques, however, may employ either metal detectors or magnetometers or a combination of both. As such, when the term “metal detector” is used herein, it is understood that one or more magnetometers, or some combination of magnetometers ) and metal detector(s), could instead be employed.

[0084] As generally shown in FIG. 10, the detection and threat assessment system 100 may be connected with a logic model 1008 (e.g., which may be the central Al server 110) to analyze results and / or to further transmit one or more signals to the display 114 (e.g., and / or generate an audio signal). The logic model 1008 may be a trained model that may improve the accuracy of weapon and / or walk identification as individuals pass through the ROI and are analyzed via the detection and threat assessment system 100.

[0085] Turning next to FIG. 11, a workflow diagram of the detection and threat assessment system 100 is illustrated. One or more patrons (e.g., individuals / bystanders) may be standing and / or disposed proximate the ROI of the detection and threat assessment system 100. The detection and threat assessment system 100 may determine whether the one or more patrons are disposed within the ROI, and / or the system 100 may process weapon and / or pose estimation for those patrons within the ROI. As generally shown, “Bystander 1”, “Bystander 2”, and “Bystander 3” may not be analyzed via the detection and threat assessment system 100, while the patron present within the ROI (e.g., “patron”) may be analyzed. In examples, the bystanders may be security guards possessing weapons and the detection and threat assessment system 100 may be configured to filter out false positives corresponding with said security guards.

[0086] The camera module 102 may capture an image of the patron as they pass through the ROI (walkway), and / or the image may be analyzed via a walk identification model 1200 to determine whether the patron is walking in a normal manner (e.g., walking in a first state correspond with no concealed objects) or an abnormal manner (e.g., walking in a second state correspond with a concealed object). The walk identification module 1200 may communicatively connect with the pose estimation module 104 to classify the pose estimation representation 400, and / or the motionof the pose estimation representation 400 (e.g., which may include analyzing patron movement patterns via a pattern recognition algorithm). For example and without limitation, a patron walking (e.g., captured via the pose estimation representation 400) in an abnormal manner may correspond to a walking tendency of a patron having a concealed item / weapon. The concealed item / weapon may be indicated via a patron’s walk (e.g., via one or more pose estimation representations 400, captured / generated at one or more instances) that favors a side.

[0087] In examples, the walk identification module 1200 may be configured to calculate statistical assessments of walks / positions for patrons by training on population data (e.g., which may involve Al components and / or modules). For example, the walk identification module 1200 may be trained on population data of patrons with and without concealed objects such that a baseline may be determined for comparison in evaluating a walking state of a patron. For items concealed on the right side of the patron (e.g., proximate the right leg, right arm, etc.), the patron may walk with increased weight distribution on the right side, further, the pose estimation representation 400 may reflect a similar weight distribution and / or walk pattern. For example, one or more pose keypoints on the right side of the patron may be disposed lower than a corresponding baseline location for such pose keypoints (e.g., which may be supplemented by personal data of the patron in the database 120). For items concealed on the left side of the patron (e.g., proximate the left leg, left arm, etc.), the individual may walk with increased weight distribution on the left side, further, the pose estimation representation 400 may reflect a similar weight distribution and / or walk pattern. For example, one or more pose keypoints on the left side of the patron may be disposed lower than a corresponding baseline location for such pose keypoints (e.g., which may be supplemented by personal data of the patron in the database 120). Generally, for items / weapons having increased weight and / or size, the individual may be leaning toward the side concealing the weapon, or toward the opposite side for larger / heavier objects. Alternatively, weapons may be concealed underneath an arm of the individual, and the walk identification model may be configured to recognize such a walk as abnormal (e.g., analyzing movement of the arm pose keypoint of via the pose estimation representation 400, and further, limited movement of an arm may correspond with concealment).

[0088] With examples, the walk identification module 1200 may analyze movement (e.g., a range of motion) for one or more pose keypoints, and further, may determine an unusual or abnormal range of motion for such pose keypoints (e.g., body parts). An unusual / abnormal rangeof motion may correspond with concealing a weapon and / or object of interest. For example and without limitation, if a pose keypoint representing a shoulder moves with an abnormal range of motion (e.g., a drooping shoulder and / or limited movement of the shoulder may correspond to an object concealed under an arm), the walk identification model 1200 may tag / label the patron as having an abnormal / unusual walk pattern. Such a determination may trigger one or more notifications, for example, that may be broadcast on the display 114.

[0089] Further, the walk identification module 1200 may communicate with a logic model (e.g., the score interpretation logic model 1008) to analyze results and / or to further transmit the results to the display 114 (e.g., and / or generate an audio signal).

[0090] In embodiments, as generally illustrated in FIG. 12, the weapon system 100B may use the captured image from the camera module 102 to determine if a weapon is visually present on an individual, and / or to identify the walk of the individual. For example and without limitation, the image capture system may take an image of the individual on the walkway (e.g., within the ROI), and further, the image may be analyzed via the Al concealed weapon detection module 112 (e.g., which may be integrated / connected with Al capabilities). Additionally, the Al concealed weapon detection module 112 and / or the walk identification module 1200 may generate a score for detected objects of interest (e.g., items / weapons), which may further be interpreted via the score interpretation logic model 1008 to analyze results and / or to further transmit the results to the display 114 (e.g., and / or generate an audio signal). With examples, the display 114 may indicate the location of the weapon on the patron.

[0091] With embodiments, such as generally illustrated in FIG. 13, the camera module 102 may include a first camera 1300 disposed proximate a first end (e.g., an entry) of the ROI (e.g., walkway), and / or a second camera 1302 disposed proximate a second end (e.g., an exit) of the ROI (e.g., walkway). In this manner, the first camera 1300 may capture a first image 1304 (e.g., a frontfacing image) of the patron, and / or the second camera 1302 may capture a second image 1306 (e.g., a rear-facing image) of the patron. A single (e.g., combined) image may be generated from the first image and the second image which may be passed to the Al concealed weapon detection module 112 for analysis. Generally, the accuracy of the detection and threat assessment system 100 may be improved with additional images from a variety of angles. For example, some objects of interest (e.g., weapons) may only be visible from a specific view; thus, using more than one image may improve weapon detectability. In other embodiments, the first image 1304 and thesecond image 1306 may be captured from any variety of angles to assist in weapon detection (e.g., which may include a variety of elevations).

[0092] Further, the Al concealed weapon detection module 112 may generate a score for detected objects of interest (e.g., items / weapons), which may further be interpreted via the score interpretation logic model 1008. The score may be analyzed via the logic model 1008, and a signal may be transmitted to the display 114 (e.g., and / or generate an audio signal).

[0093] In examples, such as generally illustrated in FIG. 14, the detection and threat assessment system 100 may include one or more analyses (e.g., including initial and / or second-look analyses) conducted via the Al concealed weapon detection module 112. For example and without limitation, the camera module 102 may capture an image 1400 of a patron, and subsequently, the image 1400 may be analyzed via the Al concealed weapon detection module 112. The detection and threat assessment system 100 may determine if the individual is within the ROI via the image 1400. If an object of interest (e.g., a weapon) is found on the hand of a patron, as detected via the Al concealed weapon detection module 112 (e.g., or other detection module), the hand region of the image 1400 may be passed through the Al concealed weapon detection module 112 for a second-look analysis. Passing a portion of the image 1400 through the Al concealed weapon detection module 112 may improve accuracy of detection and / or reduce false positive results.

[0094] With examples, the Al concealed weapon detection module 112 may generate a score (e.g., as shown in FIGS. 16A and 16B) for detected objects of interest (e.g., items / weapons), which may further be interpreted via the score interpretation logic model 1008 to analyze results and / or to further transmit the results to the display 114 (e.g., to generate an audio / visual signal). For example

[0095] In embodiments, the detection and threat assessment system 100 may connect with the database 120 including registered weapon owners and / or security personnel authorized to carry a weapon. The detection and threat assessment system 100 may compare the identity of the individual possessing a weapon with patron information on the database 120, and may not generate / transmit an alert signal if the identity is matched with an authorized individual (e.g., authorized to cany a weapon into said venue).

[0096] In further examples, the detection and threat assessment system 100 may be configured for optimized / improved throughput, and in other examples, the system 100 may be configured forthroughput-plus, an increased data transfer, low-latency, minimal packet loss, and possibly error correction (e.g., via the neural network).

[0097] With examples, FIGS. 15A, 15B, 15C, and 15D illustrate examples of a patron passing through the ROI of the detection and threat assessment system 100 in various poses / positions via the various captured images. As generally shown in FIG. 15A, the patron is walking within the ROI with both hands at the side (e.g., both hands are clearly visible). FIG. 15B illustrates a patron walking in the ROI with both hands above shoulders (e.g., both hands are visible). FIG. 15C illustrates a patron walking within the ROI with hands at the side and close to pockets (e.g., both hands are visible). FIG. 15D illustrates a patron walking within the ROI with both hands in pockets (e.g., no hands are visible).

[0098] In further examples, FIGS. 16A and 16B illustrate examples of the detection and threat assessment system 100 detecting weapons on one or more patrons passing through the ROI (e.g., traversing along the walkway). As generally shown, as a patron passes through the ROI, the detection and threat assessment system 100 (e.g., the Al concealed weapon detection module 112) may identify a weapon and / or may assign a correlating weapon score. For example, high danger items (e.g., automatic weapons) may be identifiable and / or assigned a weapon score (e.g., 0.9); medium danger items (e.g., semi-automatic weapon) may be identifiable and / or assigned a weapon score (e.g., 0.8); and low danger items (e.g., handguns) may be identifiable and / or assigned a weapon score (e.g., 0.7). In some cases, no weapon score may be assigned when no weapon is detected. In examples, knives and drugs (contraband) may be assigned a lower weapon score than automatic weapons, semi-automatic weapons, and / or handguns. In further examples, the detection and threat assessment system 100 may identify weapon scores on a scale from 0-100 in assessing danger and / or facilitating a response to such detection and threat assessment. High, medium, and low scores may correspond to one or more security responses. For high and / or medium scores, a security team may be deployed to manage the threat. For low scores, a security team may be notified such that additional surveillance may be conducted, among other security responses.

[0099] Turning to the examples of FIGS. 17, 18, 19, 20, and 21, the threat and detection system 100 may include a variety of system / process flow diagrams for the various connected modules, components, models, databases, algorithms, among others. The detailed description of FIGS. 17- 21 set forth certain exemplary embodiments of a threat and detection system 100 and associated methods of operation. It will be understood, however, that the embodiments described herein arepresented solely for purposes of illustration and explanation and are not intended to be limiting. Numerous variations, modifications, substitutions, and equivalents may be made to the structures, components, processes, parameters, and configurations disclosed, and such variations are deemed to fall within the scope of the present disclosure as defined by the appended claims. Individual features, elements, and steps of the described embodiments may be employed independently of one another or in any suitable combination or sub-combination, whether or not such combinations are explicitly illustrated or described, without departing from the spirit and scope of the invention.

[0100] As shown in FIG. 17, an embodiment of the threat and detection system 100 may include a method of sensing input data from the radar module 300, the camera module 102, and / or the pose estimation module 104. The data correlation module (e.g., the queue management module 216) may analyze information from the various modules and may generate a result. For example, the result may correspond to a threat score (e.g., which may be 0-1 as shown in FIGS. 16A and 16B, and in other examples may be between 0-100 depending on the desired sensitivity of the system 100). Further, the threat score may be a weighting of the various connected modules. As an example, visual weapon detection may include a first weight (e.g., of 40%), the concealed weapons detection may include a second weight (e.g., of about 35%), and / or pose estimation may include a third weight (e.g., about 25%). In other examples, the aforementioned weights may be adjusted for the desired sensitivity. For confidence scores above a threshold amount (e.g., such as 70%) the system 100 may continue to process security information of such patrons. In examples such as where a high threat is detected, security personnel may be prompted to respond. In other examples, such as where a medium threat is detected, security personnel may be notified. In yet further examples, such as where a low threat is detected, surveillance may increase of the identified patron.

[0101] As illustrated in FIG. 18, an embodiment of a threat and detection system 100 may include a method of face detection and / or recognition. Additionally, the method may include determining a pixel count of an image (e.g., the facial region of a patron), determining a processing mode in relation to the determined pixel count, normalizing and extracting data associated with the image, generating feature vectors for comparison, and determining a match based on a similarity score threshold.

[0102] As illustrated in FIG. 19, an embodiment of a threat and detection system 100 may include a method of managing the asynchronous processing of data from the facial recognition andweapons detection functions of the various connected components / modules, which may include generating timestamps for matching patron data (e.g., pose estimation data, facial recognition data, and / or weapon data).

[0103] As illustrated in FIG. 20, an embodiment of a threat and detection system 100 may include a method of determining pose estimation representations of patrons within the region of interest. Additionally, the method may include preprocessing an image, generating heatmaps for the one or more sensed patrons, determining one or more pose estimation representations, and determining if the one or more pose estimation representations are disposed within the ROI.

[0104] As illustrated in FIG. 21, an embodiment of a threat and detection system 100 may include a method of facial recognition and patron detection. Further, the method may include determining if a pose estimation representation is disposed within the ROI, selecting primary subjects for analysis, detecting a face of the primary subject, and / or matching the detected face with a database of patrons for identification.

[0105] With embodiments, the systems and / or methods described herein describe possible implementations, and that the structures, features, operations and / or steps may be rearranged or otherwise modified and that other implementations are possible. Furthermore, aspects from two or more of the methods may be combined.

[0106] Aspect 1 : A computerized detection and threat assessment system, the system comprising: processing circuitry, including at least one processor; memory coupled to the processing circuitry and storing executable instructions that, when executed by the processing circuitry, cause the system to implement: a pose estimation component that analyzes visual data by applying at least one trained convolutional neural network model to generate a pose estimation representation of a patron including one or more pose estimation representation coordinates representing the location of one or more body pails in pixel coordinates within captured image frames; a region of interest component that determines if one or more of the pose estimation representation coordinates is within a region of interest defined by boundary coordinate data stored in the memory and processed using coordinate comparison algorithms; and a cloud-based recognition component that compares detected facial image data or information of the patron with a database of user information for identity verification of the patron byextracting facial feature vectors using deep learning algorithms and computing similarity scores via distance metrics.

[0107] Aspect 2: The system of aspect 1, wherein the executable instructions further cause the system to implement: a weapons detection component including at least one of an ultra- wideband (UWB) radar, a metal detector, and a magnetometer operatively connected to the processing circuitry via sensor interface circuits to determine if the patron has or possesses a weapon by processing sensor signals using signal analysis algorithms that compare detected signatures to predetermined weapon profile data stored in the memory; and a queue management component comprising data structures stored in the memory and queue processing algorithms executed by the processing circuitry to manage asynchronous processing of the cloud-based recognition component and the pose estimation component by assigning and tracking timestamps for processing tasks.

[0108] Aspect 3: The system of any of aspects 1 or 2, wherein the queue management component includes a queue of facial information to identify the patron stored as indexed data records in the memory and accessed by the processing circuitry through database query algorithms.

[0109] Aspect 4: The system of any of aspects 1 through 3, wherein the queue management component generates timestamps for one or more pose estimation representations to pair or associate with information received via the cloud-based recognition component in matching identities with pose estimation representations wherein the pairing is performed by the processing circuitry executing timestamp correlation algorithms that match records having timestamps within a predetermined synchronization window.

[0110] Aspect 5 : The system of any of aspects 1 through 4, wherein the executable instructions further cause the system to implement: a face detection component that applies a face detection algorithm comprising at least one of a Haar cascade classifier or a deep neural network model to determine if a detected face of the patron is greater than a pixel threshold by computing bounding box dimensions and calculating total pixel area using geometric computation algorithms.

[0111] Aspect 6 : The system of any of aspects 1 through 5, wherein the pixel threshold is at least 5,000 pixels as determined by multiplying width and height dimensions of a computed facial bounding box.

[0112] Aspect 7: The system of any of aspects 1 through 6, wherein the pixel threshold is at least 7,000 pixels as determined by multiplying width and height dimensions of a computed facial bounding box.

[0113] Aspect 8: The system of any of aspects 1 through 7, wherein the pose estimation component is configured to determine whether key body parts are disposed within a region of interest via the pose estimation representation using a coordinate comparison algorithm that computes Boolean intersection results by comparing pose coordinate values to region boundary coordinate values stored in the memory.

[0114] Aspect 9 : The system of any of aspects 1 through 8, wherein the queue management component receives information from the cloud-based recognition component, the weapons detection component, and an Al concealed weapon detection component; and the queue management component synchronizes and / or correlates the information by matching records based on one or more timestamps and using a timestamp comparison algorithm that determines temporal correlation within configurable time windows.

[0115] Aspect 10: A multi-modal threat detection system, the system comprising: processing circuitry; memory coupled to the processing circuitry and storing executable instructions that, when executed by the processing circuitry, cause the system to implement: a visual weapon detection component that analyzes video frame data using trained neural network algorithms comprising convolutional layers and classification models to identify weapon-shaped objects and generates a first signal in response to detecting visual characteristics matching predetermined weapon signatures stored in a weapon signature database in the memory; and a concealed weapon detection component comprising at least one of an ultra-wideband (UWB) radar sensor, a metal detection sensor, and a magnetometer sensor operatively coupled to the processing circuitry via analog-to-digital conversion circuits, wherein the concealed weapon detection component generates a second digital signal upon detecting electromagnetic signatures or metallic properties characteristic of a concealed weapon by comparing sensed readings to threshold values stored in the memory using signal processing algorithms that analyze frequency domain characteristics.

[0116] Aspect 11: The system of aspect 10, wherein the concealed weapon detection component communicates with a walk-identification module comprising executable instructions stored in the memory that, when executed by the processing circuitry, analyze patron movement patterns by applying gait analysis algorithms that extract kinematic parameters from sequential image frames in determining a walking state of a patron disposed within a region of interest.

[0117] Aspect 12: The system of aspect 10 or 11, wherein the walk-identification module determines the patron to possess a concealed weapon if the patron is walking in an abnormal or defined state using a pattern recognition algorithm that compares extracted gait parameters to baseline gait models stored in the memory and computes deviation scores using statistical comparison methods.

[0118] Aspect 13: The system of any of aspects 10 through 12, further comprising: a camera module and associated image processing circuits configured to receive one or more images from one or more cameras connected with the processing circuitry via video interface circuits; and the walk- identification module configured to analyze the one or more images to determine the walking state of the patron by executing motion analysis algorithms that compute optical flow vectors and derive kinematic measurements from temporal image sequences.

[0119] Aspect 14: The system of any of aspects 10 through 13, further comprising a radar module comprising at least one of an ultrawideband (UWB) radar, a metal detector sensor, and a magnetometer operatively coupled to the processing circuitry via sensor interface circuits to detect the presence of a weapon by processing sensor data using a signal analysis algorithm that performs frequency domain analysis and pattern matching against stored weapon signatures.

[0120] Aspect 15: The system of any of aspects 10 through 14, wherein the camera module is configured to receive one or more images from a first camera and a second camera connected with the memory and the processing circuitry via video interface circuits; wherein the first camera captures a first image and the second camera captures a second image; a first perspective of the first image is different than a second perspective of the second image; and the concealed weapon detection component is configured to generate the second signal based on the first image and the second image by executing multi- view fusion algorithms that perform stereo analysis and depth computation to enhance detection accuracy.

[0121] Aspect 16: The system of any of aspects 10 through 15, wherein at least one of the first signal and the second signal is transmitted to a display module via a communication interface circuit.

[0122] Aspect 17: A computer-implemented method for automated threat detection and assessment comprising: generating, via a pose estimation component executing on processing circuitry, a pose estimation representation of a patron including one or more coordinates representing the location of one or more body parts by applying trained neural network algorithms to analyze visual data and extract skeletal joint positions; determining, via a region of interest component executing on the processing circuitry, if all or part of the pose estimation representation is within a region of interest by executing coordinate comparison algorithms that evaluate pose coordinates against stored boundary data; comparing, via a cloud-based recognition component executing on the processing circuitry, detected facial information of the patron with a database of user information to identify the patron by computing facial feature vectors using deep learning algorithms and calculating similarity scores; determining, via a visual weapons detection component including at least one of an ultra- wideband (UWB) radar, a metal detector, and a magnetometer operatively connected to the processing circuitry, if the patron possesses a weapon by analyzing sensor data using a weapon detection algorithm that processes electromagnetic signatures and compares them to stored threat profiles; determining, via a concealed weapon detection component executing on the processing circuitry if the patron has or possesses a concealed weapon by analyzing sensor data using pattern recognition algorithms; and managing, via a queue management component executing on the processing circuitry, asynchronous processing of the cloud-based recognition component, the visual weapons detection component, and the concealed weapon detection component by executing task scheduling algorithms that coordinate processing using timestamp-based synchronization.

[0123] Aspect 18: The method of aspect 17, further comprising generating a signal, via the concealed weapon detection component, indicating the presence of a weapon in connection withthe patron and transmitting the signal to an alert processing system using communication protocols.

[0124] Aspect 19: The method of aspect 17 or 18, further comprising communicating with a database of information connected to the cloud-based recognition component in determining an identity of the patron by executing database query algorithms and secure communication protocols.

[0125] Aspect 20: The method of any of aspects 17 through 19, further comprising generating one or more timestamps via the queue management component to match detected facial information of the patron with an associated pose estimation representation and storing the correlated data in indexed database records using data management algorithms.

[0126] Aspect 21: A computerized detection and threat assessment system, the system comprising: a housing disposed proximate a walkway, the housing comprising: a camera system having at least one camera operable to detect one or more patrons traversing the walkway; a radar system operable to detect if the one or more patrons possess a weapon; processing circuitry, including at least one processor; and memory coupled to the processing circuitry and storing executable instructions that, when executed by the processing circuitry, cause the system to implement: a pose estimation component that analyzes visual data from the camera system to generate a pose estimation representation of the one or more patrons including one or more pose estimation representation coordinates representing the location of one or more body parts; a region of interest component that determines if one or more of the pose estimation representation coordinates is within a region of interest; and a cloud-based recognition component that compares detected facial image data, via the camera system, with a database of user information for identity verification of the one or more patrons.

[0127] Aspect 22: The system of Aspect 21, wherein the radar system further comprises at least one of an ultra-wideband (UWB) radar, a metal detector, and a magnetometer.

[0128] Aspect 23: The system of Aspect 22, wherein the housing further comprises a display operable to broadcast an alert corresponding to identifying a weapon with an associated patron.

[0129] Aspect 24: The system of Aspect 21, wherein the housing further comprises a first partition and a second partition directing the one or more patrons through the walkway.

[0130] In embodiments, with reference to UWB radars and the like, such UWB radars may be configured in a variety of manners. According to one example, UWB transmitter array may use a7.3 GHz center frequency with a 1.5 GHz bandwidth. Differential RF terminals may be used for low noise and distortion, yielding high sensitivity in both static and dynamic applications. In general, the disclosed device may utilize very low power levels significantly below Federal Communications Commission (FCC) Class B limits for electronic devices designated for residential space, enabling its use in most worldwide markets. In one example, bi-phase, or binary phase, coding may be used for transmitting pulses for spectrum spreading. Also according to the disclosure, a master / slave Serial Peripheral Interface (SP1) may be employed, where a synchronous serial communication interface may be used for short-distance communication, with Quad SPI mode employed for higher data rates. Digital down-conversion converts digitized, band limited signal to a lower frequency signal and at a lower sampling rate, and further filtering may be applied. A small footprint Chip Scale Packaging may be used for high density integration. In one example, a 3"xl.5"x0.375" board may be used having low power requirements to facilitate battery operation of UWB transmitter array. An impulse Radar Transceiver System on a Chip (SoC) may be used with a commercially available UWB chip. The radar chips may be mounted on a development board along with pre-processor, as well as transmit and receive devices.

[0131] In some embodiments, the detection and threat assessment system 100 may comprise at least one processor / CPU (e.g., Intel® Core™ i7 or ARM Cortex-A series) coupled via a highspeed system bus to memory (e.g., DDR4 RAM), persistent storage (e.g., NVMe SSD), a display controller, and one or more communication interfaces (e.g., Ethernet, Wi-Fi, 5G). The processor may execute instructions stored in a non-transitory computer-readable medium to implement the modules described herein, including but not limited to pose estimation, face recognition, region- of-interest filtering, queue management, and Al-based concealed weapon detection. Machine learning models such as convolutional neural networks (e.g., ResNet-50) may be trained using a dataset of images and deployed using frameworks such as TensorFlow or PyTorch with GPU acceleration (e.g., NVIDIA CUDA).

[0132] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer,special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable instruction execution apparatus, create a mechanism for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0133] The non-transitory computer-readable media includes all types of computer-readable media, including magnetic storage media, optical storage media, and solid-state storage media and specifically excludes signals. It should be understood that the software can be installed in and sold with the device. Alternatively, the software can be obtained and loaded into the device, including obtaining the software via a disc medium or from any manner of network or distribution system, including, for example, from a server owned by the software creator or from a server not owned but used by the software creator. The software can be stored on a server for distribution over the Internet, for example.

[0134] Computer-readable storage media (medium) exclude (excludes) propagated signals per se, can be accessed by a computer and / or processor(s), and include volatile and non-volatile internal and / or external media that is removable and / or non-removable. For the computer, the various types of storage media accommodate the storage of data in any suitable digital format. It should be appreciated by those skilled in the ail that other types of computer readable medium can be employed such as zip drives, solid state drives, magnetic tape, flash memory cards, flash drives, cartridges, and the like, for storing computer executable instructions for performing the novel methods (acts) of the disclosed architecture.

[0135] It should be understood that a computer, a system, and / or a processor as described herein may include a conventional processing apparatus known in the art, which may be capable of executing preprogrammed instructions stored in an associated memory, all performing in accordance with the functionality described herein. To the extent that the methods described herein are embodied in software, the resulting software can be stored in an associated memory and can also constitute means for performing such methods. Such a system or processor may further be of the type having ROM, RAM, RAM and ROM, and / or a combination of non-volatile and volatile memory so that any software may be stored and yet allow storage and processing of dynamically produced data and / or signals.

[0136] It will be appreciated that references to machine learning means a model that uses algorithms to learn from data and make predictions, which may be implemented as a specificprocessing unit (e.g., a dedicated processing unit) configured to perform one or more specialized operations for the computer system or configured to perform any of the disclosed method acts or other functionalities.

[0137] It should be further understood that an article of manufacture in accordance with this disclosure may include a non-transitory computer-readable storage medium having a computer program encoded thereon for implementing logic and other functionality described herein. The computer program may include code to perform one or more of the methods disclosed herein. Such embodiments may be configured to execute via one or more processors, such as multiple processors that are integrated into a single system or arc distributed over and connected together through a communications network, and the communications network may be wired and / or wireless. Code for implementing one or more of the features described in connection with one or more embodiments may, when executed by a processor, cause a plurality of transistors to change from a first state to a second state. A specific pattern of change (e.g., which transistors change state and which transistors do not), may be dictated, at least partially, by the logic and / or code.

[0138] For purposes of this document, each process associated with the disclosed technology may be performed continuously and by one or more computing devices. Each step in a process may be performed by the same or different computing devices as those used in other steps, and each step need not necessarily be performed by a single computing device.

[0139] Various examples / embodiments are described herein for various articles and / or methods. Numerous specific details are set forth to provide a thorough understanding of the overall structure, function, manufacture, and use of the examples / embodiments as described in the specification and illustrated in the accompanying drawings. It will be understood by those skilled in the art, however, that the examples / embodiments may be practiced without such specific details. In other instances, well-known operations, components, and elements have not been described in detail so as not to obscure the examples / embodiments described in the specification. Those of ordinary skill in the art will understand that the examples / embodiments described and illustrated herein are nonlimiting examples, and thus it can be appreciated that the specific structural and functional details disclosed herein may be representative and do not necessarily limit the scope of the embodiments.

[0140] Accordingly, even though the present disclosure has been described in detail with reference to specific examples, it will be appreciated that the various modifications and changes can be made to these examples without departing from the scope of the present disclosure as setforth in the claims. It is anticipated and intended that future developments will occur in the technologies discussed herein, and that the disclosed article, device and / or method will be incorporated into such future developments. Thus, the specification and the drawings are to be regarded as an illustrative thought instead of merely restrictive thought.

[0141] Reference throughout the specification to “examples, “in examples,” “with examples,” “various embodiments,” “with embodiments,” “in embodiments,” or “an embodiment,” or the like, means that a particular feature, structure, or characteristic described in connection with the example / embodiment is included in at least one embodiment. Thus, appearances of the phrases “examples, “in examples,” “with examples,” “in various embodiments,” “with embodiments,” “in embodiments,” or “an embodiment,” or the like, in places throughout the specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more examples / embodiments. Thus, the particular features, structures, or characteristics illustrated or described in connection with one embodiment / example may be combined, in whole or in part, with the features, structures, functions, and / or characteristics of one or more other embodiments / examples without limitation given that such combination is not illogical or non-functional. Moreover, many modifications may be made to adapt a particular situation or material to the teachings of the present disclosure without departing from the scope thereof.

[0142] It should be understood that references to a single element are not necessarily so limited and may include one or more of such elements. Further, all numbers expressing dimensions, ratios and the like, used in the specification and claims, are to be understood to encompass tolerances and other deviations as represented by the term “about” or “approximately.”

[0143] Joinder references (e.g., attached, coupled, connected, and the like) are to be construed broadly and may include intermediate members between a connection of elements, relative movement between elements, direct connections, indirect connections, fixed connections, movable connections, operative connections, indirect contact, and / or direct contact. As such, joinder references do not necessarily imply that two elements are directly connected / coupled and in fixed relation to each other. Connections of electrical components, if any, may include mechanical connections, electrical connections, wired connections, and / or wireless connections, among others. The use of “e.g.” in the specification is to be construed broadly and is used to provide non-limiting examples of embodiments of the disclosure, and the disclosure is not limited to such examples.Uses of “and” and “or” are to be construed broadly (e.g., to be treated as “and / or”). For example, and without limitation, uses of “and” do not necessarily require all elements or features listed, and uses of “or” are inclusive unless such a construction would be illogical.

[0144] All matter contained in the above description or shown in the accompanying drawings shall be interpreted as illustrative only and not limiting. Changes in detail or structure may be made without departing from the present disclosure.

[0145] All terms used in the claims are intended to be given their broadest reasonable constructions and their ordinary meanings as understood by those knowledgeable in the technologies described herein unless an explicit indication to the contrary in made herein. In particular, use of the singular articles such as “a,” “the,” “said,” etc. should be read to recite one or more of the indicated elements unless a claim recites an explicit limitation to the contrary. Further, the use of “at least one of’ is intended to be inclusive, analogous to the term and / or. As an example, the phrase “at least one of A, B and C” includes A only, B only, C only, or any combination thereof (e.g. AB, AC, BC or ABC). Additionally, use of adjectives such as first, second, etc. should be read to be interchangeable unless a claim recites an explicit limitation to the contrary.

Claims

CLAIMSWhat is claimed is:

1. A computerized detection and threat assessment system, the system comprising: processing circuitry, including at least one processor; and memory coupled to the processing circuitry and storing executable instructions that, when executed by the processing circuitry, cause the system to implement: a pose estimation component that analyzes visual data to generate a pose estimation representation of a patron including one or more pose estimation representation coordinates representing the location of one or more body parts; a region of interest component that determines if one or more of the pose estimation representation coordinates is within a region of interest; and a cloud-based recognition component that compares detected facial image data or information of the patron with a database of user information for identity verification of the patron.

2. The system of claim 1, wherein the executable instructions further cause the system to implement: a weapons detection component including at least one of an ultra-wideband (UWB) radar, a metal detector, and a magnetometer operatively connected to the processing circuitry to determine if the patron has or possesses a weapon; and a queue management component to manage asynchronous processing of the cloud-based recognition component and the pose estimation component.

3. The system of claim 2, wherein the queue management component includes a queue of facial information to identify the patron.

4. The system of claim 3, wherein the queue management component generates timestamps for one or more pose estimation representations to pair or associate with information received via the cloud-based recognition component in matching identities with pose estimation representations.

5. The system of claim 1, wherein the executable instructions further configure the system to implement: a face detection component that applies a face detection algorithm to determine if a detected face of the patron is greater than a pixel threshold.

6. The system of claim 5, wherein the pixel threshold is at least 5,000 pixels.

7. The system of claim 5, wherein the pixel threshold is at least 7,000 pixels.

8. The system of claim 1, wherein the pose estimation component is configured to determine whether key body parts are disposed within a region of interest via the pose estimation representation using a coordinate comparison algorithm.

9. The system of claim 2, wherein the queue management component receives information from the cloud-based recognition component, the weapons detection component, and an Al concealed weapon detection component; and the queue management component synchronizes and / or correlates the information by the use of one or more timestamps and using a timestamp comparison algorithm.

10. A multi-modal threat detection system, the system comprising: processing circuitry; and memory coupled to the processing circuitry and storing executable instructions that, when executed by the processing circuitry, cause the system to implement: a visual weapon detection component that analyzes video frame data using trained neural network algorithms to identify weapon-shaped objects and generates a first digital signal in response to detecting visual characteristics matching predetermined weapon signatures; and a concealed weapon detection component comprising at least one of an ultra- wideband (UWB) radar sensor, a metal detection sensor, and a magnetometer sensor operatively coupled to the processing circuitry, wherein the concealed weapon detection componentgenerates a second digital signal upon detecting electromagnetic signatures or metallic properties characteristic of a concealed weapon by comparing sensed readings to values in the memory.

11. The system of claim 10, wherein the concealed weapon detection component communicates with a walk-identification module that, when executed by the processing circuitry, analyzes patron movement patterns in determining a walking state of a patron disposed within a region of interest.

12. The system of claim 11, wherein the walk- identification module determines the patron to possess a concealed weapon if the patron is walking in an abnormal or defined state using a pattern recognition algorithm.

13. The system of claim 11, further comprising: a camera module configured to receive one or more images from one or more cameras connected with the processing circuitry; and the walk-identification module configured to analyze the one or more images to determine the walking state of the patron.

14. The system of claim 10, further comprising a radar module comprising at least one of an ultrawideband (UWB) radar, a metal detector sensor, and a magnetometer operatively coupled to the processing circuitry to detect the presence of a weapon by processing data using a signal analysis algorithm.

15. The system of claim 13, wherein the camera module is configured to receive one or more images from a first camera and a second camera connected with the memory and the processing circuitry; wherein the first camera captures a first image and the second camera captures a second image; a first perspective of the first image is different than a second perspective of the second image; and the concealed weapon detection component is configured to generate the second digital signal based on the first image and the second image.

16. The system of claim 10, wherein at least one of the first digital signal and the second digital signal is transmitted to a display.

17. A computer-implemented method for automated threat detection and assessment comprising: generating, via a pose estimation component executing on processing circuitry, a pose estimation representation of a patron including one or more coordinates representing the location of one or more body parts; determining, via a region of interest component executing on the processing circuitry, if all or part of the pose estimation representation is within a region of interest; comparing, via a cloud-based recognition component executing on the processing circuitry, detected facial information of the patron with a database of user information to identify the patron; determining, via a visual weapons detection component including at least one of an ultra-wideband (UWB) radar, a metal detector, and a magnetometer operatively connected to the processing circuitry, if the patron possesses a weapon by analyzing data using a weapon detection algorithm; determining, via a concealed weapon detection component executing on the processing circuitry if the patron has or possesses a concealed weapon; and managing, via a queue management component executing on the processing circuitry, asynchronous processing of the cloud-based recognition component, the visual weapons detection component, and the concealed weapon detection component.

18. The method of claim 17, further comprising generating a digital signal, via the concealed weapon detection component, indicating the presence of a weapon in connection with the patron.

19. The method of claim 17, further comprising communicating with a database of information connected to the cloud-based recognition component in determining an identity of the patron.

20. The method of claim 19, further comprising generating one or more timestamps via the queue management component to match detected facial information of the patron with an associated pose estimation representation.

21. A computerized detection and threat assessment system, the system comprising: a housing disposed proximate a walkway, the housing comprising: a camera system having at least one camera operable to detect one or more patrons traversing the walkway; a radar system operable to detect if the one or more patrons possess a weapon; processing circuitry, including at least one processor; and memory coupled to the processing circuitry and storing executable instructions that, when executed by the processing circuitry, cause the system to implement: a pose estimation component that analyzes visual data from the camera system to generate a pose estimation representation of the one or more patrons including one or more pose estimation representation coordinates representing the location of one or more body parts; a region of interest component that determines if one or more of the pose estimation representation coordinates is within a region of interest; and a cloud-based recognition component that compares detected facial image data, via the camera system, with a database of user information for identity verification of the one or more patrons.

22. The system of claim 21, wherein the radar system further comprises at least one of an ultra- wideband (UWB) radar, a metal detector, and a magnetometer.

23. The system of claim 22, wherein the housing further comprises a display operable to broadcast an alert corresponding to identifying a weapon with an associated patron.

24. The system of claim 21, wherein the housing further comprises a first partition and a second partition directing the one or more patrons through the walkway.

Images