SYSTEM AND METHOD FOR A HIGH PERFORMANCE, VENDOR-INDEPENDENT INFERENCE APPLIANCE - Patent application

Abstract

Urgent screening in high throughput, secure environments such as emergency rooms or security typically requires multiple devices. The devices used for screening are diverse and typically sourced from multiple vendors. Currently, the devices are typically standalone. Furthermore, large devices are expensive and have high capital costs with long commercial lifespans that sometimes approach 10 years or more. These characteristics result in duplication of computing resources, obsolete computing infrastructure, and lack of interconnection between elements. Aspects of the invention include devices, systems, and methods that provide vendor-independent interconnection between multiple elements of a defined security or medical environment. The disclosed approach decouples AI algorithms from data generating systems, improving flexibility in the deployment of new technologies and algorithms. It also allows for agile adaptation to changes in threats, regulatory requirements, or market developments.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit under 35 USC §119(a) to U.S. patent application Ser. No. 63 / 293,018, filed Dec. 22, 2021, the contents of which are incorporated herein in their entirety for all purposes. [Background technology]

[0003] background

[0004] 1. Field

[0005] Aspects of the embodiments are directed to systems and methods for image data processing, and more particularly, to electrical digital data processing in conjunction with a high performance, vendor independent inference appliance that cannot be accessed by users and / or associated devices in the field during operation.

[0006] 2. Conventional Technology

[0007] Emergency screening in high throughput, secure environments such as emergency rooms or security requires multiple related art devices. Examples of related art devices used for screening are diverse and sourced from multiple vendors. In airport screening environments, for example, screening of personal items such as carry-on bags or air cargo (in airport screening environments) occurs simultaneously with in-person screening using different technologies and devices. In some related art situations, multiple vendors with multiple screening software and device parameters may be arranged side-by-side in parallel screening lines. Summary of the Invention [Problem to be solved by the invention]

[0008] Related art screening equipment is stand-alone. Furthermore, large equipment such as cargo and checkpoint X-ray and CT scanners or medical CT and MRI scanners are expensive and have high capital costs with long commercial lifespans that sometimes approach 10 years or more. As a result, related art problems and shortcomings exist, including but not limited to duplication of computing resources, obsolete computing infrastructure and lack of interconnection between elements. Other operational shortcomings are slow screening, reduced detection efficiency, increased labor requirements, vulnerability (e.g., failure of a single element causes service interruption), and lack of flexibility to reorganize in the context of new diseases, security threats, or new technologies (e.g., artificial intelligence). [Means for solving the problem]

[0009] Abstract

[0010] Aspects of the embodiments disclosed herein are directed to devices, systems, processes and methods that provide vendor-neutral interconnection between multiple elements of a defined security or medical environment, including, but not limited to, airport screening lines, hospital imaging diagnostic centers, passive and active screening technologies for perimeter security or loss prevention, without allowing a user or operator access to the appliance.

[0011] This aspect decouples artificial intelligence (AI) algorithms from data generation (i.e., making them inaccessible to field users and / or field operators during operation), allowing for greater flexibility in deploying additional (e.g., new) technologies and algorithms without requiring recertification of the OEM equipment itself, which is a slow and expensive process. Described herein are several example implementations that consolidate and mediate the output of multiple different vendors' software packages to present a unified data set and image format that can be used for human and / or algorithmic analysis.

[0012] Implementations can be updated quickly with new software to address new priorities or threats, accommodate new equipment, and integrate new data and algorithms as requirements evolve. Also, exemplary devices can be updated or replaced with new hardware as computing power evolves in short or sporadic quality improvement cycles. For highly regulated environments, such as security and medical applications, appliances and their algorithms can be recertified more quickly than if the entire certified system were to go through a recertification process. Appliances and their algorithms can respond quickly to changing threats, regulatory requirements, market concerns, and market conditions.

[0013] In accordance with these aspects, a method and apparatus are provided for accessing data from multiple devices in a vendor-independent manner and presenting the data in a unified format for at least security, loss prevention, or medical screening purposes.

[0014] Additionally, it provides embedded software functionality, including artificial intelligence software, to network-enabled devices capable of managing data flows at speeds that are practical and appropriate for the environment in question.

[0015] Additionally, it provides a GPU-based high-performance, high-throughput appliance for AI inference with failover. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 shows a high performance device according to an example implementation that can scan and transfer reconstructed images over a low latency high speed network such as InfiniBand® (IB) with Remote Direct Memory Access (RDMA) while interacting with one or more OEM scanner computers to perform multi-algorithm Automated Threat Recognition (ATR) inference via multiple GPUs.

[0017] [Diagram 2] Figure 2 shows a distributed system according to an example implementation with a command and control system (CCS) deployed as a dedicated server. A cloud-based CCS can also be used (not shown in Figure 2).

[0018] [Diagram 3] Figure 3 shows an example of the implementation of a secure system in a complex security environment.

[0019] [Figure 4] FIG. 4 illustrates a simplified example of logic flow through the disclosed systems and devices according to an example implementation.

[0020] [Diagram 5] FIG. 5 illustrates an example of the logic flow through the disclosed system with multiple input devices and load balancing / failover capabilities according to an example implementation.

[0021] [Figure 6] FIG. 6 illustrates exemplary data and analysis management in an algorithmic process flow diagram according to an example implementation.

[0022] [Figure 7] FIG. 7 illustrates several representative illustrative embodiments by way of example implementations in which the disclosed systems, processes, methods and apparatus can improve system and human performance and functionality. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Detailed Description

[0024] The following detailed description provides further details of the figures and examples of the present application. Reference numbers in the figures and redundant element descriptions are omitted for clarity. Terms used throughout this specification are provided by way of example and are not intended to be limiting. For example, the use of the term "automatic" may include fully automatic or semi-automatic implementations with user or operator control over certain aspects of the implementation depending on the desired implementation of the skilled artisan practicing the embodiments of the present invention. Furthermore, sequential terms such as "first", "second", "third", etc. may be used in this specification and claims merely for labeling purposes and should not be limited to referring to actions or items described occurring in the order described. Actions or items may be arranged in different orders or performed in parallel or dynamically without departing from the scope of this application.

[0025] Introduction

[0026] Aspects of example implementations include devices, systems, and methods that provide vendor-agnostic interconnection between multiple elements of a defined security or medical environment, including but not limited to airport inspection checkpoint lines, hospital imaging diagnostic centers, perimeter security, or passive and active screening technologies, or technologies for loss prevention. Example implementations decouple AI algorithms from data generating systems, improving flexibility in the deployment of new technologies and algorithms, and reducing regulatory burden or commercial overhead and expenses that may be required as computing technologies, customer priorities, and algorithm sophistication evolve. Example implementations are directed to a scalable centralized platform for multi-input / multi-output artificial intelligence-based algorithms hosted on either physical hardware or cloud environments. Many novel features are expected to emerge through these example implementations, including but not limited to the following attributes:

[0027] Decoupling algorithms from the equipment of the original equipment manufacturer (OEM) scanner (e.g., baggage, on-person, cargo). For example, but not limited to, algorithms may include automated threat recognition (ATR) algorithms in the case of transportation screening security or medical diagnostic algorithms in the case of medical diagnostics.

[0028] The devices and processes are supported along with supporting software libraries that are supported by the clients.

[0029] Apparatus and processes capable of housing multiple threat recognition, diagnostic or loss prevention algorithms, such as prohibited object algorithms, stroke detection algorithms, explosive detection algorithms or other similar AI-based algorithms, from different vendors and developers on a single platform.

[0030] Load balancing and failover capabilities are built into a single system, providing both security and robustness.

[0031] Algorithm changes and hardware changes (including but not limited to new OEM equipment or internal computing infrastructure such as Graphical Processing Unit (GPU) or Central Processing Unit (CPU) components) that can be replaced without affecting the certified equipment provided by the Original Equipment Manufacturer (OEM).

[0032] Optional surveillance and remote viewing support.

[0033] The proximity of a dedicated computing platform with controlled (e.g., completely restricted) access to the cloud and the Internet is expected to provide resilient services that are resistant to Internet service attacks such as Denial of Service (DoS) or other challenges that arise in computing services provided by the cloud or wireless Internet. Other benefits of the access-controlled platform include, but are not limited to, reduced exposure of data, including private or protected data, increased security of software systems and processes, and reduced access of the network to external hacking or cyber attacks.

[0034] On-site localization of a dedicated computing platform improves the platform's resistance to power outages and other challenges in a dedicated environment where attached power is available.

[0035] There is provision for using high speed communication connections with hardware elements, significantly reducing latency and increasing efficiency.

[0036] The reduced latency allows certain calculations to be done in near real time, enabling "look-back" and "look-forward" capabilities.

[0037] It should be noted that for the purposes of this disclosure, "locating a person, object or item in near real time" is defined as determining the current or previous location of a person, object or item within milliseconds, seconds or minutes of the actual time that the person, object or item was physically present at that location.

[0038] Further, "look-back" is defined as the ability to track a person, object or item in time and space for a period and to a location prior to the alarm occurrence time. In some cases, the "look-back" capability can register a person, object or item in one camera view and identify the person, object or item in a second camera view at a different time.

[0039] Further, "look forward" is defined as the ability to track a person, object or item in time and space for a period and to a location after the alarm occurrence time. In some cases, the "look forward" capability can register a person, object or item in one camera view and identify the person, object or item in a second camera view at a different time.

[0040] The above-mentioned novel features and other novel features disclosed herein provide inventive features that are ideally suited for low risk tolerance settings, such as "always on" security settings where the system needs to be resilient against hacking attempts, service interruptions, or power fluctuations.

[0041] Additional features that may or may not be present in various iterations are described below as example implementations.

[0042] Example implementation

[0043] The hardware-based or cloud-based platform includes a computer server or servers with two or more GPUs and a high-speed network connection to one or more screening devices, including but not limited to magnetic resonance imaging (MRI) devices, x-ray devices, computed tomography (CT) screeners, millimeter wave scanners, terahertz screeners, infrared imagers, metal detectors, or ultrasound technology. Other methods of acquiring images can be envisioned by those skilled in the art. Additionally, although the implementations refer to imaging, other forms of data may be processed, including but not limited to digital audio files that can be converted into images.

[0044] Various implementations host multiple algorithms from multiple vendors.

[0045]

[46] Compliance with multiple imaging standards includes, but is not limited to, the Digital Imaging and Communications in Security (DICOS) standard and the Digital Imaging and Communications in Medicine (DICOM) standard.

[0046] Network connections are disclosed that use the fastest path available. Examples include high speed Ethernet, so-called "InfiniBand" technology, or fiber optic technology. Interconnect paths may evolve in the future. Thus, continued technological advances beyond the exemplary technologies listed above are anticipated and incorporated as extensions of the general concept of "fastest path available."

[0047] If available, the throughput of connections and algorithms are managed using available open software platforms. Exemplary platforms that may be used in some implementation reductions include the U.S. government sponsored Open Platform Software Library (OPSL), which is an example of an Open Architecture Software Library (OASL). Multiple similar government sponsored or commercial software platforms and libraries can be supported by the present invention and can be envisioned by one of ordinary skill in the art.

[0048] The computation supports the recognition of desirable targets or conditions to improve decision triage. Examples of targets or conditions include medical conditions or diagnoses, identification of security threats, or identification of unauthorized objects or conditions. Examples of medical conditions or diagnoses may include the presence of a stroke, tumor, cyst, abscess, fracture, disease, or other condition that may be suspected or diagnosed based on imaging criteria, where such conditions may require human medical decision making to be performed acutely or chronically to prevent progression or undesirable further development related to the health of the individual undergoing the imaging procedure.

[0049] An example of an unauthorized object or state may include an object that should have been removed (such as an airport security environment). Other examples of unauthorized objects or states may include objects that should not be present (such as merchandise in loss prevention applications, or hidden data storage devices in commercial environments such as data centers where personal or proprietary information is stored).

[0050] Other examples of unauthorized objects or conditions may include unauthorized individuals or animals in a secured area (e.g., animals near a civilian or military aircraft runway, or unauthorized individuals in a secured government or commercial facility). Other examples of unauthorized objects or conditions may include the accumulation of foreign objects on an aircraft runway (Runway Foreign Object Debris, or FOD). Other examples of unauthorized objects or conditions may include individuals in the vicinity of a sensitive area that represents a security threat (such as individuals with weapons or suspected explosive devices detected outside a school, office, or other location).

[0051] Those skilled in the art can envision multiple other examples. The goal of the present invention is not simply to store the information from the detection, but to integrate this information into a larger system that alerts humans to potential situations, threats or circumstances, and allows human intervention to modify, mitigate or control the situation.

[0052] Facilitating human or AI interaction to alter, mitigate or control an identified situation is referred to below as “triage.”

[0053] In some example implementations, additional automated steps may occur to facilitate human triage. In this situation, an operator or human agent may only interact or make triage decisions after the automated AI triage as directed by the algorithms supported by the present invention. In this case, the embedded AI algorithms support human decision-making regarding the nature of medical care, additional security steps, or other actions required to address the alert provided by the example implementation.

[0054] Figure 1 shows an exemplary hardware embodiment in the form of a server-like device. The device in Figure 1 is a high performance device that can communicate with one or more OEM scanner computers 130 and transfer scanned and reconstructed images over a low latency high speed network using the fastest available path. In this embodiment, InfiniBand (IB) with Remote Direct Memory Access (RDMA) for running ATR inference with multiple algorithms on multiple GPUs is shown.

[0055] Turning to Figure 1, it shows a system having the following example elements: An exemplary OEM scanning device 105 is coupled to an exemplary OEM scanner computer 130 having memory / storage 110 for storing information from the scanning device, a GPU 115 for image reconstruction, and a "highest speed path available" 120 (represented here as InfiniBand or optical power capability).

[0056] In this exemplary embodiment, the scanning device generates images from a computed tomography (CT) scanner in a format, including but not limited to a format compliant with an open architecture software library (eg, OPSL125).

[0057] More specifically, with respect to 125, OPSL is one representative embodiment of OASL, however, the disclosed systems, methods and apparatus support software suites from any source, including open source platforms, proprietary software platforms, privileged software platforms, or proprietary software platforms.

[0058] Additionally, the disclosed hardware representation includes an algorithm appliance 185 having input capabilities 175 matching the OEM scanner computer output capabilities (represented here as InfiniBand or optical input), memory 150 configured to receive information, and at least one inference GPU (in this embodiment, four inference GPUs 155, 160, 165, 170 are shown).

[0059] In relation to the memory 150 of the algorithm appliance, immediate access memory capable of managing normal processes can be combined with archival memory that allows data capture for the continuous improvement of algorithms in a development environment.

[0060] In many applications, high speed network switches in conjunction with RoCE InfiniBand RDMA 140, link elements 120 and 175 (e.g., InfiniBand or optical input elements) aid in scaling the system for large local networks of imaging devices or multiplexing of multiple such networks. RoCE InfiniBand RDMA 140 is a high speed system. However, as technology advances, different interconnect methods are foreseen depending on the use case, available technology and state of the art.

[0061] In this exemplary embodiment, the automated threat recognition software (e.g., a non-transitory computer readable medium configured to execute machine readable instructions) associated with the ATR may also be OASL 180 compliant. A series of software packages are loaded and embedded, including but not limited to software dedicated to the detection of prohibited items (PI), explosives (EXP), items of commerce such as laptops, shoes or other items (SoC), or other software packages appropriate for a particular use case 190.

[0062] With respect to the algorithms in an algorithm appliance, in many applications secure, cryptographic software validation is built into the process to ensure that the only algorithms hosted are those approved to run on the appliance; as an alternative to approval, screening or authentication may occur depending on the application.

[0063] The platform depicted in Figure 1 has many advantages. Some advantages of this architecture include:

[0064] Using this platform, item recognition and triage decisions can be computed, and automated triage can be followed by operator / human agent interaction and human triage.

[0065] While certain representative embodiments focus on specific use cases, some embodiments may provide broader use cases, such as the use of this platform for classification of objects, items, or situations to enhance and improve the efficiency of human triage. For example, without limitation, one example of classification to improve triage includes an airport runway use case to identify names and locations of animals (e.g., "turkey, bear") that represent a threat to departing and landing aircraft.

[0066] Another example of such classification in a loss prevention application may include recognition of unauthorized items (e.g., "suspected data storage device detected" or "suspected weapon (knife) detected"). Another example of such classification in a transit hub may include recognition of unauthorized situations (e.g., "abandoned baggage detected, please investigate").

[0067] Another example of such a classification relevant to a medical application may include the name and location of a medical condition (eg, "suspected pneumothorax in the left lower abdomen" or "suspected fracture in the right femur").

[0068] In such cases, and in similar situations as disclosed above, a quick location of the area of ​​interest can be displayed to a human operator to enable quick triage. Faster triage can enable more efficient and safe management of the condition, threat or situation. Various iterations and extensions of the above will be apparent to those skilled in the art.

[0069] Because example implementations obtain information from multiple devices, in some iterations a comprehensive report may detail multiple areas of interest. Some examples of report formats may have the following elements:

[0070] "In patient [NAME], an abdominal x-ray revealed a left pelvic fracture at time [TIME], a chest x-ray revealed a rib fracture at time [TIME], and a head CT scan revealed an intracerebral hemorrhage at time [TIME]."

[0071] "A suspected turkey was detected in quadrant [a1], GPS coordinate [x1,y1] at [TIME]. A suspected FOD was detected in quadrant [a2], GPS coordinate [x1,y1] near runway [z] at time [TIME]."

[0072] In various examples, localization may use multiple imaging devices and multiple detection algorithms from multiple hardware and algorithm developers. Various additional examples will be apparent to those skilled in the art.

[0073] In various iterations, private or public software libraries may be accessed. For example, some security applications may utilize open architecture libraries such as OPSL to aggregate security algorithms and data streams or for training AI. Other similar open or proprietary software platforms exist and may be useful in other applications. Thus, various arrangements for the exchange of shared platforms, algorithms, and OEM devices will occur to those skilled in the art.

[0074] Several attributes of the exemplary system shown in FIG. 1 are provided.

[0075] First, the screening assessments in the example system can utilize a layered (or hierarchical) approach, where low-level classifications are made quickly and provided to actors quickly, while higher-level assessments (such as those that require further assessment) are made afterwards.

[0076] Second, in the centralization and integration of computing hardware for automated screening, a single system can perform automated artificial intelligence processes such as, but not limited to, recognition, localization and / or classification for multiplexed scanner systems.

[0077] One related anticipated result of aspects of the process described above may include reduced software load and unload times.

[0078] Another anticipated outcome associated with this process may include being able to alert at a group level of important or concerning situations. Examples of group notifications may include:

[0079] "An explosive has been detected in security lane 5."

[0080] Another anticipated outcome associated with this process may include the ability to achieve a high level of detection synthesis. Some examples of notifications may include:

[0081] "The assembly of disassembled parts in lanes 1, 3 and 4 can be used to assemble a firearm."

[0082] "The inert components detected in lanes 1 and 2 have the potential to form explosives when combined."

[0083] Another related benefit that arises from hardware centralization and integration is that it enables a common output platform, such as a common threat awareness platform that displays results from multiple types of OEM equipment.

[0084] Another related benefit resulting from hardware centralization and consolidation is that it enables efficient and rapid remote reage.

[0085] Related advantages beyond the illustrative examples set forth above will occur to those skilled in the art.

[0086] Third, the presence of a high speed network connection allows the computer server to access the scanner data at speeds approaching the memory access speeds of a fully integrated, stand-alone screening system.

[0087] Related advantages beyond the representative examples set forth will occur to those skilled in the art.

[0088] In a scaled-up system, multiple servers can be collocated, as shown in Figure 2. Turning to Figure 2, note that in some implementations multiple algorithms can be hosted across multiple servers. Figure 2 shows an example of collocation of multiple sensors.

[0089] Multiple scanner devices can be linked in a network. Although three scanner devices are depicted as 230, 231, and 232 in FIG. 2, other numbers of additional scanner devices and computers can be linked. Each of the scanner computing devices 230 / 231 / 232 associated with the scanner imaging device 205 / 206 / 207 has memory 210 / 211 / 212, a reconfigurable GPU 215 / 216 / 217, and optical inputs / outputs (e.g., I / O switches) 220 / 221 / 222. In various iterations, OASLs such as Open Platform Software Libraries (OPSL) 225 / 226 / 227 may be utilized to aggregate security algorithms and data streams or for training AI.

[0090] The Command Control System (CCS) can direct the data to the appropriate algorithm regardless of location using an InfiniBand® (IB) or optical switch 235 or other interconnect method suitable for the particular use case. Additionally, the optical switch 235 can be coupled to remote viewing at 296.

[0091] The linking of multiple scanners to a single algorithm appliance via switch 235 is represented as 285, and a second load balancing algorithm appliance 286 can be utilized in situations of peak data flow or due to malfunction or problems with the first algorithm appliance 285. Each of the appliances 285 / 286 has memory 250 / 251, one or more inference GPUs 255-270 / 256-271, and optical inputs / outputs (e.g., I / O switches) 275 / 276. In various iterations, an OASL such as the Open Platform Software Library (OPSL) 280 / 281 may be utilized to aggregate security algorithms and data streams or for training the AI.

[0092] In this case, one purpose of load balancing is to improve the resilience and reliability of the system in the face of potential system failures. Specifically, in this expression, the process integrated with the CCS maintains proper load balancing among the interconnected systems. For example, in this situation, if one GPU, server or platform is unavailable or slow due to the number of processes already running on it, the distributed system reallocates the load data to idle server processes available in the distributed network.

[0093] Another purpose of load balancing is to allow routine maintenance, system updates or other activities to occur seamlessly without changing system capacity.

[0094] One aspect of the implementation includes treating the capabilities of AI algorithms as needed "critical infrastructure." "Critical infrastructure" is often over-engineered. Buildings and bridges are designed not just to manage loads but to manage unexpected challenges and to withstand deterioration. In a similar manner, we provide systems, methods, and processes to create safe and resilient systems to serve AI in critical security, safety, loss prevention, healthcare, and other applications.

[0095] Returning to Figures 1 and 2, in some iterations, multiple algorithms are hosted on a single machine (see, e.g., 190, 290, and 291). The ability to host and manage multiple algorithms provides the advantage that algorithms can be managed in a variety of secure systems. As technology and algorithm management advances, additional systems with this type of capability will continue to be developed and will be provided to those skilled in the art in the future.

[0096] Figures 1 and 2 show example implementations of the device. An example implementation from a user's perspective is shown in Figure 3. Turning to Figure 3, an exemplary embodiment related to airport security is shown, an example of this type being familiar to most of the general public.

[0097] This implementation shows an appliance labeled inference appliance 340. The appliance utilizes one or more computation units 341, 342, 343 (three shown in this example) for computation and load balancing. The appliance 340 interacts through high-speed connections (grey lines with different weights) to three security zones (Zone 1, Zone 2, Zone 3). An organizational diagram shows the relationship between computation algorithms and devices.

[0098] To summarize Figure 3, appliances supporting security tasks are shown in specific security zones 310, 320, 330, 340. Specific sensors currently used for security are shown as examples, but it can be assumed that many types of sensors used in secure and trusted environments have similar characteristics. Other arrangements of sensors and different types of sensor networks will occur to those skilled in the art. An exemplary embodiment is shown in Figure 3.

[0099] Security Team 1 310, including one or more agents (two of whom are shown as 315, 316), is associated with one or more checkpoints in Area 1, Zone 1 utilizing passive terahertz wave imaging (two sensor devices are shown as 311, 312). Sensors 311 and 312 have AI solutions computed by AI Suite 3 343, which includes a set of integrated machine learning, artificial intelligence and image processing software systems optimized to generate solutions from passive terahertz images (see Figures 1, 2, 4, 5 and 6 for more details).

[0100] Security team 2 (320) is associated with a checkpoint utilizing one or more active millimeter wave scanners (two shown labeled 321 and 322) and one or more computed tomography (CT) scanners (two shown labeled 323 and 324).

[0101] The associated millimeter wave scanner is used for screening to ensure that prohibited items are not brought in. Several individuals from Security Team 2 (two individuals labeled 325 and 326) are focused on evaluating the results of the millimeter wave screening. The AI ​​solutions from the millimeter wave screening have been calculated by AI Suite 2 341, which is comprised of a set of software programs with similar characteristics to AI Suite 3 343, optimized for millimeter wave detection parameters.

[0102] The associated CT scanner is used for baggage screening. Several individuals from Security Team 2 (two individuals labeled 327 and 328) are focused on evaluating the results of the CT baggage screening. The AI ​​solution from the CT screening has an AI solution calculated by AI Suite 1 labeled 342. However, in this case, CT scanner 323 is a device manufactured by manufacturer B and CT scanner 324 is a device manufactured by manufacturer C. In this case, separate image processing software and logic (IP1 and IP2 labeled 346 and 347) may be required to enable proper analysis by AI Suite 1. In other cases, the two manufacturers may require entirely different AI suites optimized for their respective specific devices.

[0103] Security Team 3 (330) engages in soft target surveillance in pre-check (and post-check, if necessary) areas by monitoring multiple video feeds. A "soft target" in this context includes an individual or target object in an area where there are unscreened or unscreened individuals, processes, equipment, or systems.

[0104] Turning to Security Team 3, a series of sensors feed data to an Inference Appliance 340 via a high-speed connection. AI solutions are computed by AI Suite 4 (344). At least one unique feature characterizes the requirements of Security Team 3, including the following aspects:

[0105] Unlike security agents 315, 316, 325, 326, 327, 328, a single human agent (shown as 335) of security team 3 can monitor multiple screens at once, a situation that is common in building security, perimeter security, and other similar settings where large geographic areas are monitored.

[0106] In this case, a rapid scene analysis AI protocol may be used. Such algorithms may be utilized, and one recent example of the type of algorithm used for this purpose is an algorithmic approach called "You Only Look Once" (YOLO).

[0107] Rather than paying attention to a single screen, Security Team 3 agents issue an alert on a screen and location when a potential problem situation is detected (in this case represented by the black arrow pointing to the screen labeled 351).

[0108] Area 2, labeled 350, denotes a second security zone. In the context of this demonstration, Area 2 represents a checked baggage area. In Area 2, another inference appliance, labeled 355, has a different algorithm specialized for different sensor characteristics and requirements. This representation shows that communication between the two inference appliances may be used to improve or optimize the performance and efficiency of the overall system.

[0109] To summarize Figure 3, specific sensor types and relationships are shown to provide a relevant example of the complexity of sensor types and relationships in security, medical technology, and loss prevention. This example is intended to touch upon some of the complexity of these types of networks and supports the implementation examples set forth in this patent and the associated claims. Some representative points illuminated by this use case include, but are not limited to:

[0110] The design, specification and objectives of the algorithms are task specific. The present disclosure supports multiple tasks on one platform.

[0111] By configuring algorithms specialized for different tasks on a single platform, it is possible to efficiently develop interactions between algorithms and between humans involved in collaborative tasks.

[0112] Putting these two points together, Figure 3 shows that AI support can be optimized as a human-centric activity: Decision support is designed to improve human decision-making among team members with different roles.

[0113] The disclosed implementations (interchangeably referred to as inference appliances or algorithm appliances) enable more robust and informed interactions between various team members.

[0114] To clarify this point, consider a situation in which a weapon is discovered by agent 327 monitoring the CT data. In this embodiment, an individual being screened or who has recently been screened can immediately notify agent 325 and agent 335 that a weapon has been detected in their carry-on baggage.

[0115] In summary, Figure 3 shows a secure platform that hosts a suite of vendor-agnostic AI algorithms optimized to work holistically in complex environments with multiple different security protocols and objectives. Some advantages of this system are detailed below.

[0116] One or more secure appliance systems monitor multiple sensors, reducing the burden on human agents.

[0117] Vendor-agnostic algorithms enable consistent and reliable performance at security checkpoints.

[0118] Threat information can be sent to agents in a particular area or can be sent quickly to agents in multiple areas.

[0119] Communication latency is optimized to allow optimal decision times and, where appropriate, robust "look forward" and "look back" capabilities to identify the past and current locations of the person or object that raised the alarm.

[0120] The system exhibits resilient and robust performance during Internet service interruptions or power outages.

[0121] The system limits external access to zero. Consequences of such external access restrictions include:

[0122] Reduce exposure of data, including private and protected data.

[0123] Improve the security of software systems and processes.

[0124] Reducing access to the network to external hacking or cyber attacks.

[0125] Hardware and software updates are not tied to the lifespan of the device, and rapid updates are supported for both hardware and software.

[0126] In Figure 3, human oriented aspects of the disclosed devices, systems and methods are described. To provide novel human-centric features of the devices, the aspects of Figure 4 and Figure 5 are provided. In particular, a relevant feature(s) of the disclosed devices relate to managing data in an environment where multiple vendors use vendor-specific software packages to convert sensor information into images.

[0127] A representative simplified example of a typical logic flow from an original equipment manufacturer (OEM) to a disclosed appliance device is shown in Figure 4. To summarize Figure 4, raw information from a sensor or series of sensors is collected at 406 by a scanner device, transitioned to an OEM computing platform 435, and connected to an algorithmic appliance device 485 via a high speed connection 445. The elements of the device described above are identical to those described above in connection with Figure 1.

[0128] A typical logic flow through the OEM device starts with raw image data 405 and passes to a data acquisition and transfer routine 430 involving hardware and software residing in an OEM scanner computer 435. A typical logic flow through the OEM scanner computer includes the following:

[0129] Transfer of RAW image data from the sensor 405 to the internal memory 410.

[0130] For example, the GPU 415 may be used to generate raw image data from the raw data in memory, resulting in the reconstructed image data 420 .

[0131] Through switch 425 , the images are transferred to the disclosed device, here labeled algorithm appliance 485 , over a high speed connection using high speed network transfer protocol 445 .

[0132] An interrelated logic flow then occurs in the algorithm appliance using command, control and inference routine 480. The command, control and inference routine labeled as 480 in Figure 4 is representative of the AI ​​suites shown in Figure 3 (see Figure 3, AI Suite 1, AI Suite 2, AI Suite 3 and AI Suite 4). A more detailed explanation of the command, control and inference routine 480 allows a detailed description of the sequence of steps.

[0133] First, the reconstructed image data must be present in memory 470 where it is accessible to the AI ​​algorithm.

[0134] The AI ​​algorithms running on the various graphics processing units (GPUs) then access the data from memory (see 450, 455, 460, 465). The algorithms represented as 450, 455, 460, 465 may evaluate the data independently or may pass data between the algorithms.

[0135] Data from the algorithmic analysis is returned to memory (470) and passed to networking (475).

[0136] The resulting assessment is then either transferred directly to a secondary station such as a viewing station 440 for merging with the image data or transferred back to the OEM device via a high speed network transfer protocol 435 to the OEM scanner computer for merging. In Figure 4, the second process is depicted.

[0137] Figure 4 shows the data flow as follows:

[0138] To generate reconstructed image data 420 , the raw data 405 is sent to an OEM scanner computer 435 using a data acquisition and transfer routine 430 .

[0139] The reconstruction data 420 is sent via a high speed network transport protocol 445 to a command, control and reasoning routine 480 of the algorithm appliance device

[0480] .

[0140] Inference is performed on one or more GPUs using AI algorithms 450, 455, 460, 465, working alone or in collaboration to generate a threat representation.

[0141] Inference is returned to the OEM scanner computer 435 using a high speed network transport protocol 445 .

[0142] The corrected or merged image is transferred from the OEM computer to another device, such as the viewing station 440. The corrected or merged information includes both data from the OEM device and additional AI-generated inference information from the algorithmic appliance.

[0143] While FIG. 4 represents one particular methodology, modifications of this logic flow may be made depending on the nature of the OEM's equipment, the particular data characteristics, and the efficiency desired for high speed data flow.

[0144] Comparing Figure 4 with Figure 3, a logical flow is provided in Figure 4 that is depicted in Figure 3 as a logical flow from various OEM devices (located in Zone 1, Zone 2 and Zone 3 in Figure 3) to the OEM appliance (the inference appliance in Figure 3), with the merged data finally reaching various users or security teams (Security Team 1, Security Team 2 and Security Team 3) in the form of useful images with AI-assisted localization of situations or objects of interest (or absence of such).

[0145] Figure 5 shows a distributed logic flow involving a series of OEM devices. The elements of the devices mentioned above are the same as those described above in relation to Figure 2. In Figure 5:

[0146] RAW images (505, 507, 509) generated from a variety of OEM scanners (506, 508, 510).

[0147] The reconstructed images (525, 526, 527) are generated via various vendor-specific data acquisition and transfer routines (535, 537, 539), each of which is located in a vendor-specific OEM scanner computer (536, 538, 540).

[0148] The data is transmitted via a high speed network transfer routine 545 to one or more algorithmic appliance computing platforms (597, 598).

[0149] AI inference occurs through a variety of GPU-based AI algorithms (555, 560, 565, 570, 556, 561, 566, 571).

[0150] Return of the inferred information via a high speed network transfer routine 545 returns the information to the OEM devices (536, 538, 540) and one or more remote viewing routines 599.

[0151] The remote viewing routine then distributes the relevant information to the viewing station. With reference to the example of Figure 3, the remote viewing routine transmits this information to various security teams.

[0152] 4 and 5 show the logic flow through the hardware elements.

[0153] Turning now to Figure 6, a schematic diagram is provided illustrating one representative example of the software organization and associated data flows and processing across the elements disclosed above. Turning to Figure 6, the software, systems and methods can be separated into the Scanner and Front Facing System (601), the Algorithm Appliance / Load Balancing and Failover System (602), and the Interconnects (620, 625, 630, 640).

[0154] The scanner and front-facing system 601 represents an external element that links to the appliance systems, processes and methods.

[0155] The associated appliance / load balancing and failover system 602 serves as a schematic diagram incorporating the systems, methods, processes and apparatus disclosed in this patent. Interconnections (not shown) are provided between the outward facing system and the disclosed systems, methods, processes and apparatus (620, 625, 630, 640).

[0156] Turning to the representation of the outward facing system 601, note that the raw data is generated from the imaging scanner 605 as well as the reconstructed image. In some cases, the reconstructed image is generated by proprietary OEM reconstruction software (610), as shown in Figure 6. In other cases, a third party may be responsible for the image reconstruction and reconstruction software.

[0157] The reconstructed image 610 in this representation is then passed 615 to the usual OEM processing and algorithms.

[0158] In the context of the disclosed systems, methods, processes and apparatus, the reconstructed image in this representation is also forwarded to a third party algorithm sub-process 620. The third party algorithm sub-process represents a connection point between the disclosed systems, processes, methods and apparatus and the OEM equipment.

[0159] The third party algorithm in this representation is then forwarded to the high speed network process 630 which forwards the data to the appliance command control process 640.

[0160] Once the data is transferred to the appliance command and control process 640, the systems and methods disclosed above allow the various OEM images to be appropriately distributed to various algorithm suites, such as one or more of those illustrated in FIG.

[0161] Algorithm Pre-Process 1 (645), Algorithm Inference 1 (650), and Algorithm Post-Process 1 (655) are similar to AI Suite 1 in FIG.

[0162] Algorithm Pre-Process 2 (646), Algorithm Inference 2 (651), and Algorithm Post-Process 2 (656) are similar to AI Suite 2 in FIG.

[0163] Algorithm Pre-Processing 3 (647), Algorithm Inference 3 (652), and Algorithm Post-Processing 3 (657) are similar to AI Suite 3 in FIG.

[0164] Alternatively, multiple algorithms may operate as part of a single analysis or series of analyses on a particular OEM image (i.e., 641, 642, 643), all contained within a single software suite, and integrated in various ways with a single OEM image.

[0165] Following post-processing, in a representative example, the algorithm results are integrated with appropriate OEM reconstructed images 610 by third party algorithms 620 via high speed network process 630 and forwarded to an outward facing results display process 625 that supports one or more functions including providing AI augmented information to teams involved in high trust environments dedicated to the safety, security or health of humans or systems.

[0166] Turning specifically to the algorithm appliance 602, a more detailed description of the process begins with a discussion of the purpose of the command and control process. When image data is received by the appliance, the command and control process 640 determines the appropriate algorithm and GPU to which the data needs to be routed for inference and subsequent post-processing. The following process follows.

[0167] Processes 641 / 642 / 643 / 660 represent macro processes inside the appliance. Depending on the number of computing devices (GPUs) available in the appliance, these processes can run in parallel on the same image or on different images depending on the use case.

[0168] Processes 641 / 642 / 643 are schematic representations of computational processes (three representations shown) subordinate to the primary function of the algorithmic appliance to provide tailored AI output to a user. Although three processes are shown, it is anticipated that any number of processes may be processed by a given algorithmic appliance depending on task requirements and hardware capabilities. Association of Processes 641 / 642 / 643.

[0169] The reconstructed images are sent to appropriate pre-processing steps 645 / 646 / 647 to obtain processed images in a format compatible with the model to work with.

[0170] This pre-processing step can have multiple sub-processes. Examples of pre-processing methods include scaling the image to a shape / size suitable for the AI ​​model to infer, adding (salt and pepper) noise, changing the color of the image, rendering the image in grayscale, ray tracing, etc.

[0171] All pre-processing steps may have functions that are understandable to those skilled in the art.

[0172] In some cases, the pre-processing step is a "data augmentation" step, where "data augmentation" is a step dedicated to increasing the robustness or stability of an AI model.

[0173] For example, the provided data can be augmented by adding salt and pepper noise, changing image scale, modifying color parameters, thresholding the image, distorting, reflecting, rotating the image and similar steps that would occur to one skilled in the art, to generate a solution that is robust to real-world environments.

[0174] In other cases, in addition to or to supplement the purpose of data augmentation, some pre-processing steps can aid in object localization or definition of object boundaries. Ray tracing and thresholding are examples of pre-processing steps that can aid in data augmentation by allowing multiple views of potential targets to be presented, as well as allowing more precise localization within the image of the desired target.

[0175] In summary, pre-processing data within the algorithm appliance serves an important purpose: improving the robustness of third-party algorithmic AI solutions in real-world environments.

[0176] The pre-processed data is then sent to the AI ​​inference process 650 / 651 / 652. AI inference is realized by an "inference engine" that evaluates and analyzes the pre-processed data and applies logical rules to a knowledge base to provide actionable results. These processes can be standard AI inference algorithms (e.g., resnet, yolo) or custom algorithms built for specific problems (CT scanning of bags or detecting human threats in airports, detecting brain strokes in CT images, etc.).

[0177] The output of the AI ​​inference process provides results that in some cases require post-processing steps 655 / 656 / 657. Post-processing of third-party AI algorithms may be applied. For example:

[0178] Post-processing may be required so that the results can be reintegrated with the original OEM image so that it can be presented for external use.

[0179] Post-processing may be required to enable results obtained from one manufacturer's equipment to be displayed on another manufacturer's platform.

[0180] For example, a global alert across multiple systems requires steps to allow the data to be displayed on multiple displays.

[0181] Furthermore, by developing validated and comparable data sets at different sites, new external facing processes can be developed.

[0182] For example, consider the example of Figure 3 above. In this example, a particular site, such as a particular airport, may have more false positives than expected.

[0183] In this representative example, the existence of authentication results across multiple sites allows for the development of external record-keeping applications that report detections compared to non-detections.

[0184] Such applications may enable third party algorithms and regulators to track algorithm performance on a site-by-site basis. Parties that may benefit from such validated results include:

[0185] Third Party Algorithm Producers

[0186] airport

[0187] Government Regulatory Bodies

[0188] OEM manufacturers are also considered users of the system. The algorithmic appliances can store data from routine activities to provide robust data from the field. This robust data can facilitate important quality control functions. Example:

[0189] Data from a single manufacturer can be stored over time. Analyzing a single manufacturer's process over time offers the following advantages:

[0190] Quality control over time relating to the output of the machine can be stored and analyzed, allowing measurements to be made that can detect machine degradation as well as quickly detect machine failure.

[0191] Data can be stored that can be used by the manufacturer or a third party to obtain more data for algorithm or device refinement.

[0192] Data from many manufacturers can be compared. Being able to compare data from multiple device manufacturers allows regulatory agencies to compare and contrast output quality between detector domains.

[0193] In related technology fields, the proprietary nature and siloed availability of OEM data often hinders such comparisons.

[0194] Thus, the present disclosure enables regulatory agencies to use the features and functionality described herein to significantly improve quality across entire fields or areas of operation.

[0195] Regulatory agencies and host entities (such as airports, other transportation hubs, hospitals, schools, or any entity using an appliance for loss prevention or perimeter security) are also considered applications of algorithmic appliances. When permitted and applicable, algorithmic appliance data can be provided to these entities for quality metrics and team development.

[0196] Related to the above, and more broadly, potential perimeter security breaches at a nuclear power plant can be rapidly transmitted to multiple locations, both local and remote, via a secure transmission protocol.

[0197] Further related to the above, the existence of a medical emergency, such as a stroke, can be securely transmitted to multiple users in a medical application.

[0198] In some cases, user notification may serve an immediate function. Returning again to the example discussed in Figure 3, in a security environment, individuals monitoring security cameras can quickly notify other agents of an impending concern in a secure manner (through a results display process). Similarly, individuals at a particular screening location can quickly notify individuals monitoring security cameras that an individual or group of individuals needs to be monitored and tracked.

[0199] Those skilled in the art will envision other examples.

[0200] Enabling multiple AI algorithms to work in concert with user-facing capabilities in a reliable, robust and resilient manner across diverse environments, including multiple human teams, multiple manufacturers and multiple settings, is a key novel feature of the exemplary systems, processes, methods and apparatus. The diversity of user-facing capabilities that the algorithmic appliances provide for human agents, commercial and government users, device manufacturers and regulatory agencies is another key novel feature of the exemplary systems, processes, methods and apparatus.

[0201] The disclosed systems, processes, methods and apparatus can be applied to many environments. While Figure 3 illustrates a specific security application (airport security environment), similar complex environments exist in a variety of environments that include conditions or situations that require regional coordination to provide optimal service or care. Such applications include, but are not limited to:

[0202] An integrated customs and border protection device that provides intelligent, integrated information across multiple border crossings.

[0203] A port entry AI network for detecting anomalies in cargo containers.

[0204] An integrated loss prevention network for your business or data center.

[0205] Equipment located in locations that participate in regional care networks, including (but not limited to) stroke networks, cardiovascular disease networks, or trauma networks, where feeder hospitals or lower acute care hospitals partner with tertiary care centers for interventional care.

[0206] The ER of a large hospital is a place where rapid diagnostic capabilities using multiple devices are effective for triage care.

[0207] Transport environments such as commercial railroad environments or metropolitan transport environments (buses, subways, trains).

[0208] A large commercial warehouse and "wish realization" center.

[0209] Critical power infrastructure settings from power grid substations to power plants (including nuclear plants).

[0210] A military base or civilian security station such as a police station.

[0211] Multiplexed security systems including multiple sensor types and locations such as secure perimeters where optical, terahertz or infrared, radar or other data streams are combined to provide complex information via multiple device types and algorithms.

[0212] Figure 7 shows some examples of various AI software implementations. These images are intended to illustrate, in a limited way, that these applications represent important emergent technology applications that are not speculative but require the novel solutions described in this disclosure.

[0213] Specifically, items 701, 702, 703, and 704 in FIG. 7 show successive frames from a low-resolution camera. The items show output from a suite of AI algorithms applied to pair packages with associated individuals as well as to identify abandoned packages. The particular approach at issue uses an AI approach called "You Only Look Once" (YOLO), although multiple similar algorithms have been developed that can track items in near real-time in sequential images. By using the processes, methods, and apparatus disclosed in this patent, many unique and novel enhancements are available, such as the following enhancements:

[0214] Input from multiple sensors (in this case optical cameras) is combined, allowing individuals to be seamlessly tracked from one camera to another in near real-time.

[0215] The identification of potential concerns or security threats may be integrated with other information, such as information collected from other sensor systems.

[0216] Integration with other network systems can enhance near real-time look-forward and look-back capabilities via distributed camera systems and sensor systems of other systems.

[0217] System-wide alerts can be distributed across a secure local area network.

[0218] Protected or confidential information, including sensitive security systems, information and methods, is secured in a local area network.

[0219] Load balancing and failover capabilities are provided to remote sensors, resulting in a robust system that is resilient to point attacks, communication outages, and power outages.

[0220] Returning to items 701, 702, 703, and 704 in Figure 7, note that while the AI ​​algorithm suite demonstrated in this example focuses specifically on identifying and tracking luggage and bags, a similar approach can be used to identify other types of objects, such as drones or individuals with weapons. These features lead to applications in multiple secure systems in "always-on" security environments in public environments such as transportation stations, perimeter security surveillance, schools, stadiums, or supermarkets.

[0221] In Figure 7, 720 represents a secure AI-based medical platform. In an increasingly insecure cyber environment, applications in the medical environment may focus on securely managing access and AI evaluation of Protected Health Information (PHI).

[0222] Similar to the system depicted in Figure 3, an imaging device 721 provides information to an algorithmic appliance 722 which can then distribute the information for remote viewing including hardwired secure systems (723) and wireless devices (tablet devices represented as 724 and cell phones represented as 725). In this case, the repository of protected health information and hospital internal systems are secured within a secure local area network and wireless output of limited information is sent only to identified specific users.

[0223] Item 730 shows an example of a medical application, where a subarachnoid hemorrhage (731) is detected by a suite of AI algorithms in a CT scan, and an intraventricular hemorrhage (732) is further detected.

[0224] Item 740 identifies turkeys in the border security system. Far from being benevolent, territorial wildlife can be a threat to both military and civilian aircraft.

[0225] At 750, an example AI algorithm for runway foreign object debris (FOD) detection is shown, including detection (755) and processing by an AI algorithm (760) with object identification (770). In relation to 750, an AI algorithm time is disclosed.

[0226] Rather, the inventive systems, methods, devices, and processes in this disclosure recognize that in many settings, optimal use of AI will require the integration of one or more AI algorithms or suites of algorithms to support complex goals, such as those required to provide services specialized in protecting the health, safety, or security of humans or systems.

[0227] When humans collaborate in a high-trust environment, only trusted and vetted members join the team. AI increasingly supports human teams in critical tasks, becoming a force multiplier and augmenting agent for the human teams. The disclosed systems, processes, apparatuses and methods provide secure hardware and software systems that support and mediate optimal use of trusted, verified and secure collaborative AI algorithm support in a high-trust environment.

[0228] A limited number of implementations have been shown and described. These implementations are provided to convey the subject matter described herein to those skilled in the art. It should be understood that the subject matter described herein can be embodied in various forms without being limited to the implementations described. The subject matter described herein can be practiced without the features specifically defined or described, or with different elements or features not described. Those skilled in the art will understand that changes can be made in these implementations without departing from the subject matter described herein, as defined in the appended claims and their equivalents.

Claims

Claim 1: A computer-implemented method for processing data from multiple devices to generate actionable output for screening in a secure environment, comprising: receiving data relating to an image from at least one of the plurality of devices; providing the received data to a processor and performing acquisition of the data at the processor; transferring said data to an appliance over a high speed network; In the appliance, command, control, and inference are performed by one or more graphical processing units (GPUs) to generate the actionable output, the inference including inference associated with automated threat recognition (ATR) for security screening without being associated with a scanner that scans the image to generate data associated with the image, and the appliance is not accessed by any device during operation; using an ATR algorithm that performs at least one of a look-back analysis and a look-forward analysis to track conditions in time or space; providing the actionable output over the high speed network for performing an action; and wherein the data is accessed by multiple software and hardware vendor devices in a software and hardware vendor independent manner, and further wherein multiple ATR algorithms are executable on the appliance.

2. The computer-implemented method of claim 1 , wherein the commands, controls, and inferences include security detection threats that identify security threats.

3. 10. The computer-implemented method of claim 1, wherein said commanding, controlling, and inferencing comprises applying artificial intelligence to automatically detect potential threats without user input.

4. The computer-implemented method of claim 1 , wherein the commanding, controlling, and inference comprises identifying a serious medical condition or diagnosis.

5. 10. The computer-implemented method of claim 1, wherein said commanding, controlling, and inferring comprises applying artificial intelligence to automatically detect a disease or condition without user input.

6. The computer-implemented method of claim 1 , wherein the data from the multiple devices is presented and managed in a common format.

7. 7. The computer-implemented method of claim 6, wherein the common format includes at least one of the United States Digital Imaging and Communications in Security (DICOS) image format or the Digital Imaging and Communications in Medicine (DICOM) image format.

8. The computer-implemented method of claim 1, wherein the lookback analysis comprises tracking the history of an identified object or individual prior to an event.

9. 1. A system for collecting and processing data from multiple devices to generate actionable output, comprising: a scanner device configured to scan image-related data from at least one of the plurality of devices; a processor configured to receive the scanned data and perform the acquisition of the data; a high-speed network configured to transfer data from the processor to the appliance; an appliance configured to execute command, control, and inference by one or more graphical processing units (GPUs) to generate the actionable output, the inference including inference associated with automated threat recognition (ATR) for security screening without being associated with a scanner that scans the image to generate data associated with the image, the appliance not being accessed by any device during operation; wherein the actionable output is provided over the high speed network for performing an action; the appliance uses an ATR algorithm that performs at least one of a look-back analysis or a look-forward analysis to track conditions over time or space; The system wherein the data is accessed by multiple software and hardware vendor devices in a software and hardware vendor independent manner, and further wherein multiple ATR algorithms are executable on the appliances.

10. 10. The system of claim 9, wherein the appliance is configured to process the data at a rate configured for the security screening and to present and manage the data in a common format.

11. The system of claim 9 , wherein the high speed network has built-in connectivity.

12. 10. The system of claim 9, wherein the graphical user interface (GUI) is located only at a remote location and allows only remote access and monitoring of security activities.

13. 10. The system of claim 9, wherein one of the plurality of software and hardware vendor devices can connect to one or more other devices of the plurality of devices to access a network of devices.

14. 10. The system of claim 9, further comprising geographically separated access points to enable geographically based screening and networking capabilities.

15. The system of claim 9 , wherein one of the appliances is capable of supporting multiple lanes of a security checkpoint.

16. 10. The system of claim 9, wherein the appliance is configured to reduce the computation, power, and cooling required by the processor while providing failover and redundancy, and to perform load balancing so that if one of the GPUs fails, another of the GPUs provides the computation necessary for proper operation of the security facility.

17. 10. The system of claim 9, wherein the scanner device comprises an X-ray or computed tomography (CT) screener for baggage, a millimeter wave screen or a terahertz screener, and the high-speed network is a high-speed Ethernet (registered trademark), InfiniBand (registered trademark) or an optical network.

18. 10. The system of claim 9, wherein connections are managed through an open architecture software library.

19. The computer-implemented method of claim 1, wherein the look-forward analysis comprises predicting a future location or state of an identified object or individual.

Images