SYSTEM AND METHOD FOR IMPLEMENTING AN ARTIFICIAL INTELLIGENCE SECURITY PLATFORM - Patent application

Abstract

The present invention relates to an AI platform to simplify artificial intelligence (AI) security for enterprise and other applications. One embodiment of the present invention implements an AI security platform that secures AI / ML models while keeping configuration and implementation simple and streamlined. One embodiment of the present invention provides visibility into AI models across an entire organization as well as other enterprise structures.

Description

[Technical field]

[0001] The present invention relates to an artificial intelligence (AI) platform for streamlining AI security for enterprise and other applications. [Background technology]

[0002] Artificial intelligence is one of the largest and fastest growing segments in the software world. As these technologies advance, companies will increase their use of AI and data to create competitive advantage.

[0003] More than half of global companies are using AI in at least one business function. The global AI software industry is expected to double in just a few years. Industrial manufacturing, financial services, technology, retail and life sciences are adopting AI at the fastest pace.

[0004] Forecasts indicate that 75% of organizations will shift from piloting AI to operationalizing it. In fact, AI is predicted to increase productivity levels by around 40% across front-end and back-end processes. This change in direction will be driven by better accessibility to data and the development of highly flexible models to adapt to specific business needs. Regulators have taken note of the increasing use of AI and are developing regulatory compliance frameworks to protect stakeholders. Thus, the industry is expected to see increased use of AI, the creation of large amounts of data, the proliferation of cyber attacks, and the enactment of regulations.

[0005] The implementation of AI and ML models into business processes is accelerating rapidly. However, industry experts are ill-prepared to protect, detect, and respond to attacks against AI and ML models. Companies leveraging AI and ML models are expected to adapt to a rapidly changing regulatory landscape.

[0006] There is an assumption in the industry today that AI / ML implementations are already secure. However, standards and protocols do not adequately address security for AI platforms. In addition, there are many different types of models and AI implementations running on various different ecosystems. Current AI implementations are not standardized and are very complex.

[0007] It would therefore be desirable to have a system and method that can overcome the aforementioned shortcomings of known systems. Summary of the Invention

[0008] According to one embodiment, the present invention relates to a computer-implemented system and computer-implemented method for implementing an artificial intelligence (AI) security platform. The system comprises a security center platform that serves as a central software-as-a-service (SaaS) portal and collects and aggregates telemetry data across one or more AI implementations, an interactive user interface configured to track the one or more AI implementations and provide an analytical dashboard for identifying one or more adversarial security issues, and a computer server comprising a computer processor coupled to the interactive user interface and the security center platform, the computer server further comprising a data integrity module configured to provide data security and integrity in the one or more AI implementations and further provide adversarial protection, a platform configuration monitoring interface configured to provide configuration monitoring related to the one or more AI implementations, and a model security module configured to provide insight into how one or more models are performing from a security perspective and further provide model recognition and anomalous model behavior analysis.

[0009] According to another embodiment, the present invention relates to a computer-implemented system for implementing an AI security platform, the system comprising: a core security platform comprising a computer processor in communication with an API gateway, a container orchestration platform, and at least one memory component; an interface in communication with one or more collector agents receiving data from one or more AI services; an asset discovery engine configured to collect metadata about one or more assets used for development and delivery of at least one AI pipeline, the metadata relating to asset type, activity, data source, and model; and an analytics engine configured to identify potential threats from adversarial attacks by monitoring the one or more AI services, the one or more assets, and the at least one AI pipeline to generate at least one security score that evaluates the security posture of the entity, the at least one security score being based on one or more of security incidents, alerts, and security findings.

[0010] According to another embodiment, the present invention relates to a computer-implemented method for implementing an AI security platform, the method including: communicating with one or more AI services by one or more collector agents; collecting metadata by an asset discovery engine about one or more assets used in the development and delivery of at least one AI pipeline, the metadata relating to asset type, activity, data source, and model; monitoring by an analytics engine the one or more AI services, the one or more assets, and the at least one AI pipeline to identify potential threats from adversarial attacks; generating by the analytics engine at least one security score assessing a security posture of an entity, the at least one security score being based on one or more of a security incident, an alert, and a security investigation result; and providing by an interface security health data representative of the security posture of the entity, the security health data including the at least one security score.

[0011] As enterprises invest in AI systems, security and model performance become key concerns. Embodiments of the present invention understand various aspects of AI including security, performance, optimization, bias, etc. Various embodiments of the present invention enable entities to reach a point of satisfaction with AI in core and critical business processes. The innovative system seeks to break down AI silos to gain a deeper understanding of the AI ​​portfolio.

[0012] These and other advantages are described in greater detail in the detailed description that follows. [Brief description of the drawings]

[0013] In order to facilitate a more complete understanding of the present invention, reference is made to the accompanying drawings, which should not be construed as limiting the invention, but are intended only to illustrate different aspects and embodiments of the invention. [Figure 1] FIG. 1 is an exemplary architecture diagram according to one embodiment of the present invention. [Diagram 2] FIG. 1 is a diagram of an exemplary architecture according to one embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram of an exemplary application architecture according to one embodiment of the present invention. [Figure 4] FIG. 2 is a diagram of a collector architecture according to one embodiment of the present invention. [Diagram 5] FIG. 2 illustrates an asset discovery process according to one embodiment of the present invention. [Figure 6] FIG. 2 is a diagram of an exemplary collector data architecture according to one embodiment of the present invention. [Figure 7] 1 is an exemplary flow chart for adversarial detection according to an embodiment of the present invention. [Figure 8A] 1 is an exemplary diagram of a Security Center interface according to one embodiment of the present invention. [Figure 8B] 1 is an exemplary diagram of a Security Center interface according to one embodiment of the present invention. [Figure 9] FIG. 2 is an exemplary diagram of a pipeline manager interface according to one embodiment of the present invention. [Figure 10] FIG. 13 is an exemplary diagram of an entity details interface according to one embodiment of the present invention. [Figure 11] FIG. 13 is an exemplary diagram of a new pipeline creation interface according to one embodiment of the present invention. [Figure 12] FIG. 2 is an exemplary diagram of a notable events interface according to one embodiment of the present invention. [Figure 13] 1 is an exemplary diagram of a suspicious activity interface according to one embodiment of the present invention. [Figure 14] FIG. 2 is an exemplary diagram of a business details interface according to one embodiment of the present invention. [Figure 15] FIG. 2 is an exemplary diagram of a cloud assets interface according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] To illustrate various features of the present invention, exemplary embodiments of the present invention will be described. The embodiments described herein are not intended to limit the scope of the invention, but rather to provide examples of the components, uses, and operation of the invention.

[0015] An embodiment of the present invention secures artificial intelligence (AI) / machine learning (ML) models while simplifying and streamlining configuration and implementation. An embodiment of the present invention provides visibility into AI models across an entire organization as well as other enterprise structures.

[0016] The implementation of AI and ML models into every business process is accelerating. These systems integrate data from multiple sources and utilize learning algorithms that are usually of unknown provenance. This makes it difficult for entities, such as chief information security officers (CISOs) and other users, to have the visibility, cybersecurity, and risk management to ensure adequate protection against AI-specific risks.

[0017] According to an exemplary implementation, the AI ​​security platform may include SaaS applications supporting the AI / ML platform to simplify and streamline security, risk management, and monitoring of critical AI security controls. The AI ​​security platform combines the ability to understand and reduce security risks from the use of AI / ML technologies in a streamlined manner.

[0018] In one embodiment of the present invention, the AI ​​security platform enables visibility into major security threats and monitors threats from major AI / ML engines offered by various providers (e.g., Microsoft®, Google®, IBM, KPMG, Amazon®, etc.). Other platform support may be based on client demand, market opportunity, and / or other considerations. Additional security, risk, and / or ethics controls may be added.

[0019] An embodiment of the present invention may support a variety of users, industries, and applications. Exemplary personas may include the CISO (primary), the AI ​​security team (primary), the AI ​​engineering team (secondary), and the application security team (secondary).

[0020] According to an exemplary illustration, a persona may include a CIO / CISO / Director user within an organization. Regulations are rapidly changing and this user may be tasked with securing data and systems and protecting the organization. In this example, the organization may seek to apply industry leading best practices to end-to-end AI processes to simplify security of AI systems and better understand the impact of breaches or attacks on AI systems. Expectations may include protecting against major threats and breaches, simplifying the process of securing AI, improving visibility and transparency, and addressing upcoming regulatory compliance.

[0021] An embodiment of the present invention provides visibility into key security threats to AI / ML systems. This provides AI security awareness, meaning that clients can be aware of security risks posed by AI components, including breaches to data, models, etc. An embodiment of the present invention facilitates integration and is easy to plug into existing ecosystems, e.g., working with various AI / ML platforms. The innovative system promotes ethical AI / ML decision-making, so clients can ensure that models comply with ethical and / or other standards. Thus, an embodiment of the present invention provides a standard way to verify deployed models and their functionality.

[0022] 1 is a diagram of an exemplary architecture according to an embodiment of the present invention. The AI ​​security center platform 110 may support various functions, including a data integrity module 120, a platform configuration monitoring (API) 130, and a model security module 140.

[0023] One embodiment of the present invention gains insight into AI implementations and helps clients safely and securely embed AI into their core business functions. The innovative system allows users to monitor their AI implementations and understand how their data is being attacked to further protect against various threat scenarios.

[0024] The AI ​​Security Center Platform 110 may create security standards and associated frameworks for adversarial AI. The AI ​​Security Center Platform 110 provides a single place to track platforms, models, and / or data sources. This may include AI portfolio and threat / vulnerability tracking, as well as manual and automatic mapping. The AI ​​Security Center Platform 110 further provides an analytical dashboard, identification and prioritization of key risks, collection and aggregation of telemetry data (related to core risks), analysis of telemetry across internal systems and against peers, and user management.

[0025] According to an embodiment of the present invention, the AI ​​security center platform 110 functions as a central management (SaaS) portal. The AI ​​security center platform 110 aggregates the individual functions of the solution, manages the client journey, provides dashboards and reports, and provides management for users. The AI ​​security center platform 110 may further provide an interactive user interface for various users, such as the CISO and the AI ​​security team. The interactive user interface may generate a single, unified view into adversarial security protection. The interactive user interface may further manage the client journey and aggregate the functions of the platform's components to ensure data integrity, secure configuration, and model behavior.

[0026] The data integrity module 120 ensures that data is secure. The data integrity module 120 provides functionality focused on the structured data model. This may be accomplished by applying core framework, extract, transform and load (ETL) pipeline testing, standardized dataset integrity, and identifying when a training set has been infected. The data integrity module 120 may provide various types of protection including adversarial / dataset poisoning protection 122 and integration / man-in-the-middle attack protection 124.

[0027] Adversarial / Dataset Poisoning Protection 122 may leverage blueprints from IEEE, OpenAI, MITRE, etc. Additionally, Adversarial / Dataset Poisoning Protection 122 may identify adversarial attack signals from peers, as well as other functionality such as data tagging, tokenization, and file security (e.g., PKL file security, etc.).

[0028] Additionally, the AI ​​Security Center platform 110 supports data integrity via a data integrity module 120 that allows various users, such as CISOs and AI security professionals, to monitor the security of the training (and operational) datasets used in various key AI / ML learning systems. One embodiment of the present invention develops the ability to understand and protect against adversarial dataset poisoning attacks (and man-in-the-middle attacks) to ensure proper data security and integrity in AI / ML architectures. This is represented by Adversarial / Dataset Poisoning Protection 122 and Integration / Man-in-the-Middle Attack Protection 124, as shown in FIG. 1. Currently, CISOs and AI security teams struggle to ensure the integrity and security of both AI training sets and ongoing datasets that AI / ML learning systems rely on to provide secure, reliable, and error-free results. One embodiment of the present invention provides protection and visibility against data poisoning and other data-centric attacks against AI / ML architectures that can cause significant errors and compromise risks.

[0029] Platform configuration monitoring (API) 130 provides platform configuration monitoring, which may include analyzing security configurations against vendor, industry, and best practices. Other features may include mechanisms for setting security baselines, configuration impact analysis, pre-production reviews, continuous monitoring, and configuring platform and quarantine functionality. Platform configuration monitoring (API) 130 may further support secure configuration, represented by 132, which may be platform specific.

[0030] Platform configuration monitoring (API) 130 may analyze various models and generate baselines to ensure that identified and tracked systems match the baselines. For example, platform configuration monitoring (API) 130 may analyze how a system is configured and compare the current configuration to best practices or standards.

[0031] The AI ​​security center platform 110 supports secure configuration, as indicated by 132, which provides the ability to map and monitor AI / ML platform and model configurations against secure configuration standards to determine whether models are adequately protected from adversarial risks. An embodiment of the present invention provides visibility and monitoring of configuration settings to ensure that AI / ML platforms are securely configured. In current systems, many CISOs and AI security professionals lack visibility into the configuration of platforms that support AI / ML integration. This creates significant security holes that, if exploited, could expose systems to serious adversarial attacks, data leakage, and privacy risks. An embodiment of the present invention allows users to understand and monitor their security posture against manufacturer best practices and guidance to reduce security risks.

[0032] Secure configuration 132 may determine whether the system is securely configured by ensuring that the AI ​​platform has a privacy configuration that complies with various standards (e.g., vendor standards, industry standards, best practices, etc.). Secure configuration 132 may also perform security baseline setting. For example, one embodiment of the present invention may determine whether the system complies with security standards and, if not, how to modify the system or bring it into compliance with the appropriate standard. This may be performed across multiple platforms.

[0033] Model security module 140 may include model recognition 142 and anomalous model behavior analysis 144. Additionally, model security module 140 may provide the ability to use MITRE (or other frameworks, curated knowledge bases and models) as a guide to continuously test models (e.g., 24 / 7) and support inference risk (e.g., data and results are as intended).

[0034] For example, the model security module 140 may allow a user to understand how a model is performing from a security perspective. One embodiment of the present invention determines whether a hacker has modified the model.

[0035] The AI ​​security center platform 110 supports behavior monitoring and analysis through model security module 140, which applies techniques to identify, understand, and monitor adversarial threats to ML / systems and models through model recognition 142 and anomaly model behavior analysis 144. Anomaly model behavior analysis 144 supports AI models for security and may use SQL / chaos models to further test the system.

[0036] An embodiment of the present invention provides visibility and monitoring to reduce the cyber risk of AI / ML learning models integrated into core business processes that may be subject to attack. Compromised models pose fundamental risks to patient health, financial decisions, and can cause significant disruptions in the long term. Because these systems are often "black boxes," errors may or may not be properly identified or managed over time. These systems typically attempt to remove people from the process who should be the "check" on the process results. An embodiment of the present invention reduces the cyber risk of anomalous AI model behavior based on adversarial AI threats.

[0037] One embodiment of the present invention provides visibility into the AI ​​pipeline from a security perspective. The innovative system provides automated mapping of AI platforms, models, and data sources. Additionally, telemetry data can be collected and further analyzed. One embodiment of the present invention supports the development of standards and associated frameworks.

[0038] According to one embodiment of the present invention, an automated mapping model may identify a particular signature and map and identify an AI system based on the signature, which may include identifying calls to external data sources. For example, one embodiment of the present invention may infer how a system is processing training data and determine the type of model being applied.

[0039] An embodiment of the present invention collects telemetry data to provide visibility across various AI pipelines within an organization. This may also be done across multiple clients, platforms, entities, industries, etc. An embodiment of the present invention may be extended to determine various metrics across one or more entities. For example, an embodiment of the present invention may determine the highest attack rate in a particular domain or application (e.g., clinical trials, pharmaceuticals, etc.). For any determination, an embodiment of the present invention may provide countermeasures to modify and / or better secure the AI / ML implementation.

[0040] An embodiment of the present invention provides real-time visibility into the AI ​​pipeline, pinpointing attacks, potential attacks, and providing better protection against current and future attacks. An embodiment of the present invention may, for example, identify potential indicators of exposure to data poisoning, providing insight and increasing an entity's trust in the AI ​​platform.

[0041] An embodiment of the present invention may support a variety of user experiences including mobile devices, virtual reality or augmented reality, and the like. Figure 2 is a diagram of an exemplary architecture according to an embodiment of the present invention. Figure 2 illustrates one example, and other variations in implementation and design may be realized. As shown in Figure 2, additional functionality may be provided by an AI Trust Center Platform 210, AI Bias 220, AI Model Insights 230, and AI Infrastructure Security 240.

[0042] Figure 2 illustrates an exemplary system that integrates components to build confidence in implementing models in core business processes. The system of Figure 2 provides insight into the models, for example, whether the models have been tested against other standards in different countries.

[0043] The AI ​​Trust Center Platform 210 brings together elements such as cyber and model management, model insights, model behavior, and model bias. Embodiments of the present invention develop a trusted, centralized, and integrated AI security dashboard SaaS platform that provides visibility into the overall health of an enterprise's AI pipeline.

[0044] One embodiment of the present invention provides simplified AI security management by mapping the AI ​​pipeline, monitoring AI systems for key threats, and helping organizations respond to the impact of AI systems such as data loss, threats to AI. Through a holistic view of an enterprise's AI / ML health, an embodiment of the present invention protects systems both on-premise and in the cloud, providing a solution to the gaps in the cybersecurity market. Through dashboard alerts and trend analysis, an embodiment of the present invention provides triage support for suspicious events and pursues their remediation.

[0045] Various benefits may include transparency into AI / ML threats, visibility into threats and team engagement, a simplified view into the ecosystem of AI and ML models that allows those without a data science background to understand the security risks posed by the models, a SaaS model that allows for rapid deployment, and increased efficiency and time / cost savings.

[0046] One embodiment of the present invention relates to a self-service, easy-to-integrate platform that monitors AI systems at various points in the AI ​​building lifecycle to provide alerts for various suspicious activities, threats, etc. This may include static analysis, monitoring of training data and features, drift in model accuracy, versioning and drift analysis of model files, and runtime analysis of outputs generated using these systems.

[0047] One embodiment of the present invention relates to Cross-Platform AI Pipeline Discovery, which utilizes a collector-based architecture to integrate with APIs and webhooks across multiple external AI development and delivery platforms to collect metadata about assets that can be used for the development and delivery of AI pipelines.

[0048] The collected metadata may relate to asset types, activities, data sources, models, experiments, jobs and notebooks, endpoints and deployments. Metadata for asset types may include datasets, models, experiments, jobs, notebooks, endpoints and deployments. Metadata for activities may include versioning, activity type, activity date and user / system context. Metadata for data sources may include format, storage, size, sourcing, artifact location and characteristics. Metadata for models may include performance metrics, supporting libraries and frameworks, sourcing and artifact location. Metadata for experiments, jobs and notebooks may include computation context, execution history, execution time, input / output and artifact location. Metadata for endpoints and deployments may include computation context, supporting libraries and frameworks, authentication context and artifact location.

[0049] Metadata may be used to create and / or maintain asset records within the core platform, which may be made available as part of the pipeline management process. Pipelines may be built by linking assets and users through manual and automated relationship mapping to create a unified context for linked assets across the various platforms where the assets reside.

[0050] An embodiment of the present invention relates to Pipeline Lineage Discovery. An embodiment of the present invention may utilize asset metadata from external AI development and delivery platforms to build asset lineage by analyzing available information to discover relationships between assets. The final built lineage may form a graph from which the ancestors and descendants of a given asset may be obtained. Inference of relationships between assets may utilize relational data available from external AI development and delivery platforms as well as analytical data collected from other sources including code references, tokenization, and / or identification attributes to build asset signatures that may be utilized for lineage discovery. The combination of lineages allows for centralization and relevant context for pipeline-specific lineage discovery.

[0051] An embodiment of the present invention relates to an Adversarial Detection Engine. In one embodiment of the present invention, the process of detecting security findings, incidents, and alerts may be driven by a workflow that coordinates the execution of analysis and evaluation of both events and related assets between an external AI development and distribution platform, collector agents, and a core platform.

[0052] One embodiment of the present invention develops and applies a behavioral model for Inside Threat Detection. According to one embodiment of the present invention, activity logged for individual users within an external AI development and delivery platform may be analyzed against the pipeline context of the pipeline users and associated responsibilities provided during the creation of the pipeline. The validity of the activity from an authorization perspective may be evaluated against the set of assigned responsibilities in combination with regression and classification based modeling, which collectively form a pattern matching system built on historical time series data on user interactions to assess the likelihood of an insider threat.

[0053] One embodiment of the present invention relates to analysis for the detection of adversarial AI threats. According to one embodiment of the present invention, decision trees and parallel executions may be designed to use multiple detection methods in identifying adversarial AI threats. Each individual analysis execution may provide intermediate results that, in combination, may be used to determine the presence of one or more adversarial AI threats with greater accuracy than the individual results.

[0054] One embodiment of the present invention relates to an AI pipeline collector agent. According to one embodiment of the present invention, a collector agent built to operate within multiple cloud and on-prem environments may be used to provide integration with external AI development and deployment platforms, asset discovery, event monitoring, analytics execution, and adversarial threat detection, with the purpose of providing outputs related to AI security. The collector agent may be designed with deployment flexibility, security of both agent components and data, and scalability to accommodate big data processing requirements. The collector agent may be capable of self-updating a threat library to dynamically manage detection capabilities as new threats are discovered and new detections are created. The integration system is pluggable, so that integrations may be enabled, disabled, created, and / or destroyed by remote commands without requiring direct intervention in the infrastructure.

[0055] One embodiment of the present invention relates to framework-agnostic AI pipeline analysis. According to one embodiment of the present invention, a context-aware framework detection function may be used to unify analysis execution. Detection may be performed using pattern matching on asset artifacts and metadata to infer relevant frameworks and capabilities. The detected frameworks and capabilities may be verified or overridden within the core platform as part of pipeline asset management. The final set of frameworks and capabilities may then be provided to the analysis execution process to recreate the assets in a representation that abstracts framework and data specific attributes to provide a unified processing of the analysis.

[0056] One embodiment of the present invention is directed to an AI card that enables evidence collection to create a compliance, security, and brand-related representation of an AI system. An AI card may contain specific compliance evidence for one or more pipelines, governance information, and / or the entire AI system. Compliance evidence for a given AI card may be specified by an applied card template, which includes the sections and fields necessary to prove compliance based on field definitions. Fields may include free text, dates, numbers, checkboxes, single / multiple selection, and / or optional file upload attachments. Default field values ​​may be automatically populated by formulas built using data from inventory and monitoring data already collected by the core platform. Cards may be shareable within the platform with other users, regulators, file exports such as PDF, third-party platforms, marketplaces, and exchanges, external auditors, and websites via html embed. Versioning, change management, and sharing history of AI cards may be stored in the platform for compliance and governance purposes. AI Card templates can be modified, cloned, and / or created from scratch with a form builder that has specific support for building formula-based auto-fills based on data collected in the core platform.

[0057] An embodiment of the present invention relates to a SaaS solution that continuously monitors an enterprise's AI systems and assets, detects potential threats from adversarial attacks, provides security scoring, and enables teams to gain visibility into the overall health of the enterprise's AI. One embodiment of the present invention provides visibility into the overall health of an enterprise's AI systems by integrating with an enterprise's existing security tools and AI pipelines to continuously scan and monitor AI assets and pipelines for vulnerabilities related to model performance and health. One embodiment of the present invention further provides security recommendations based on the confidence of exposed assets. For example, the collector utility can be installed and deployed in the environment where AI models are developed and deployed. The collector utility can be connected to the system where development is running. This allows data to be easily pulled and aggregated from various systems and a predefined set of tests can be applied against those pipelines.

[0058] Various user capabilities may include loading asset information relevant to the ML system into the environment; broadly categorizing assets according to their lifecycle stage(s); viewing datasets (e.g., raw data, training data used for model training, etc.); viewing experiments; viewing models in development and models deployed in production (e.g., containers, dependencies, input and output distributions, etc.); and viewing isolated assets (e.g., status, changes over time, etc.).

[0059] One embodiment of the present invention implements a zero-day AI threat engine. Anonymized data and telemetry in the monitoring data collected in the core platform can be analyzed using unsupervised learning models to provide information on trends, potential threats, etc. The collected trend and threat information can be used to create zero-day findings in the management platform to be distributed to customer segments based on the relevance and presence of threat agents. Zero-day findings can represent unique and potentially novel attacks against AI systems or against industry segments that cannot otherwise be detected without the aggregated data collected and analyzed by the unsupervised learning system that supports trend and threat discovery.

[0060] One embodiment of the present invention relates to automated detection of Adversarial Synthetic Data Augmentation. Data augmentation, generative AI, and use of public data can be tracked as part of provenance and lineage metadata collected by the collector to initiate a detection workflow to identify adversarial synthetic data. This detection technique can combine data provenance-based defenses with batch clustering to identify inconsistencies in the augmented data that may indicate synthetic or public data poisoning, where each batch is analyzed for factors including statistical properties, cross-entropy, and clustering in an unsupervised learning model, which are then aggregated and fed into a supervised model built to detect anomalies across batches.

[0061] 3 is an exemplary application architecture according to one embodiment of the present invention. Core platform 310 may include edge services 312 that interact with browser 302. Browser 302 may run on a variety of devices including computer servers, client systems, mobile devices, smart devices, etc. Browser 302 may include an identity user interface (UI) 304, an administration UI 306, and a customer UI 308. Edge services 312 may include an identity provider (IDP) 314, a content delivery network (CDN) 316, and an application gateway 318.

[0062] Core platform 310 may include an API gateway 320, a container orchestration platform 330, a service bus 340, a cache 350, and a database such as SQL DB 360.

[0063] The API gateway 320 may include an authentication API 321 , a public API 322 , an administration API 323 , an application API 324 , and a collector API 325 . The container orchestration platform 330 may include operations management 331, reporting 332, inventory management 333, and inventory monitoring 334.

[0064] Operational management 331 may include a configuration interface that provides effective security management by providing control over user management, integration, scoring, and / or platform customization. User management ensures proper access and permissions, integration may be important for connecting to an organization's existing environment, and score configuration allows for the assessment of individual security scores and supports platform customization to meet unique security needs. User and role management may control access to sensitive information, enforce role-based access control, and track user activity.

[0065] Scoring configuration allows for the customization of incident, alert and security finding scoring, severity levels and scoring ranges to match an organization's evolving security objectives, enabling incident prioritization and response tailored to an organization's needs, improving security posture and ensuring effective management.

[0066] Reporting 332 provides high level insight into the health of programs, assets, and / or pipelines, as well as monitoring through charts / graphics, metrics, and key details. Reports can be designed to provide quick, comprehensive information for managers, directors, and executives who do not require detailed information. Various platform users access these reports.

[0067] Health may be represented by various scores and metrics, including security scores (e.g., assessed security posture), model / dataset security scores (e.g., security scores across various pipeline models and datasets), asset breakdown (e.g., total number and breakdown of assets), pipeline, model, dataset breakdown (e.g., total number of pipelines / models / datasets and corresponding breakdown of health states), organizational pipeline (e.g., comprehensive overview of security state of different components across organizational pipelines), etc. Lifecycle may be represented by pipeline and model state (e.g., percentage of pipelines in training, pre-production, and production stages), dataset breakdown (e.g., percentage of internal and external datasets), orphaned assets (e.g., insecure assets not currently monitored or managed by the pipeline), and resilience recommendations (e.g., actionable insights to strengthen the organization's security posture). Security may be represented by security trends (e.g., charts of number of incidents and alerts over time), incident breakdown (e.g., high severity, medium severity, and low severity), alert breakdown (e.g., high priority, medium priority, and low priority), and major incidents (e.g., incidents sorted by severity).

[0068] Inventory management 333 may relate to asset and pipeline management. Inventory management 333 may manage and track multiple types of assets in an AI development pipeline. These assets may include datasets, models, experiments, and deployments. These are considered critical elements in the development and deployment of AI models and may be further monitored and managed throughout their lifecycle to ensure security and performance.

[0069] Asset types may include datasets, models, experiments, and deployments. Other asset types may be supported. A dataset may represent a collection of data used to train and evaluate a model for its accuracy and performance. According to one embodiment of the present invention, supported dataset types may include tables, images, text, etc.

[0070] A model may represent data organization and standardization techniques to recognize patterns in the data. A model may be trained on a dataset and retrained with feedback and / or new data to improve the model's performance. A variety of model frameworks may be supported, including automated machine learning platforms. A variety of model objectives may be supported, including regression, classification, etc.

[0071] Regression may refer to a statistical analysis technique used to model and analyze the relationship between a dependent variable and one or more independent variables. Target metrics may include normalized RMS error, Spearman correlation, R2 score, and normalized mean absolute error. Classification may refer to a machine learning task that involves assigning data points to predefined categories or classes based on some features or attributes. For example, the goal may be to train a model to accurately predict class labels for new, unseen data points. Target metrics may include accuracy, precision, recall, F-score (F1), ROC / AUC, etc.

[0072] An experiment may refer to the process of building, training, testing, and analyzing the results to evaluate a machine learning model. An experiment may represent a single machine learning task, which may include multiple steps, including data preparation, model selection, training, and evaluation.

[0073] Deployment may refer to the process of taking a trained machine learning model, putting it into production, and making it available to users in real applications, which may include taking the trained model, packaging the model with any necessary dependencies, and deploying the model to a target environment.

[0074] According to an embodiment of the present invention, a pipeline may encompass various assets and stages in the journey of an AI model from training to deployment. When creating a pipeline, an organization may provide identifiable contextual information, such as business criticality and business situation, to enable understanding of higher risk areas. A pipeline may also include related assets (e.g., datasets, models, experiments, and deployments) and a list of team members who have been assigned roles in the pipeline.

[0075] In one embodiment of the present invention, a security score may be calculated by individual assets, individual pipelines, and multiple pipelines. From a pipeline perspective, a security score may be calculated based on the assets contained in that pipeline. Any alerts or incidents generated may be linked directly to individual pipelines depending on the criticality of each pipeline. For example, a pipeline containing the same assets as another pipeline may have different alerts or incidents if they have a high criticality as opposed to a low criticality.

[0076] An embodiment of the present invention determines a security score to assess the security posture of an organization or a single pipeline. The security score algorithm may be based on multiple factors including vulnerabilities, incidents, surface area, etc. of the AI ​​pipeline. Scores may be applied on various scales, e.g., 0-1 scale, 0-100 scale, etc., where a score of 0 represents all assets having high severity and high confidence incidents, and a score of 1 or 100 represents no incidents, alerts, and security findings across all assets. The security score may be calculated by combining the impact from incidents, alerts, and / or security findings. Pipelines may have defined criticality, with the higher the criticality, the greater the impact from incidents, alerts, and security findings.

[0077] The security score may be applied in a variety of ways, including applying it to the entire organization by combining the scores of all pipelines, applying it to each individual pipeline by combining the scores of all assets contained in the pipelines, applying it to the organization's model by combining the scores of all models contained in the pipelines, applying it to the organization's dataset by combining the scores of all datasets contained in all pipelines, applying it to each individual pipeline's model by combining the scores of all models contained in all pipelines, and applying it to each individual pipeline's dataset by combining the scores of all datasets contained in all pipelines.

[0078] One embodiment of the present invention creates a new pipeline to address a specific business need, a process that may include providing contextual details, defining the desired outcome, and determining the required inputs, tools, and resources.

[0079] Embodiments of the invention may configure and / or update pipeline details to allow a user to modify existing pipeline information, including reassigning a pipeline to a new project, adjusting business criticality after a security review, and / or managing team members associated with a pipeline.

[0080] One embodiment of the present invention is managing the assets of a pipeline, which is key to keeping the assets updated and valid, allowing users to modify the assets, for example, selecting new models, incorporating new datasets for model retraining, etc.

[0081] Inventory monitoring 334 may relate to monitoring events, incidents, alerts, and security findings. Inventory monitoring 334 may represent a toolkit for continuous security information collection and analysis that provides real-time visibility into security events, detects threats and anomalies, and provides users with relevant information for rapid decision making. An embodiment of the present invention may act as an early warning system that allows organizations to detect and respond to security incidents before they escalate, thereby proactively managing security posture and reducing the risk of data loss or data theft.

[0082] An event may refer to any change that occurs in a source system and acts as a trigger for the monitoring process. Examples of events may include the creation of a dataset, the failure of an experiment run, the addition of a new updated version of a model, the removal of a deployment, etc.

[0083] Events may capture important changes in an organization's security posture. Events may form the basis of threat detection, response, and analysis. An embodiment of the present invention may use continuous event monitoring to detect security threats and provide important information for fast, informed decisions. Events may be important because they provide accurate and actionable security information in real time. An embodiment of the present invention may aggregate and analyze events to provide a comprehensive security view, allowing users to rapidly respond to threats. In essence, events play a key role in the monitoring and response process, ensuring protection of critical assets by providing the information necessary to detect, evaluate, and respond to security incidents.

[0084] An incident may refer to a security event that has a significant impact on an organization and requires further response and recovery efforts. In addition, an incident may be generated by the automatic triggering of an alert rule and may be initiated manually after an alert is investigated. For example, an incident may include a severity and confidence rating, a report date, a status of the incident, team members involved, a reason why the incident occurred, associated assets, recommended next steps, an activity log, etc.

[0085] Incident details may provide a user with detailed information about a specific incident. This feature allows a user to access comprehensive information about the incident, including a description of the incident, related activities, and recommended next steps to resolve the issue. This level of detail may be useful to understand the complete context of the incident and enable a user to make informed decisions and take appropriate actions to address the issue.

[0086] An alert may represent a technical notice of a current security risk, such as a vulnerability and its exploitation. An alert may provide an early alert signal indicating a potential change or activity that may require further investigation and attention. An alert may include a priority and confidence rating, a detection date, a status of the alert, associated team members, a reason why the alert was triggered, associated assets, recommended next steps, an activity log, etc.

[0087] Additionally, an alert may represent a notification generated by the core platform in response to a specific security incident or situation. Alerts may be considered important because they provide real-time visibility into potential security risks and allow users to take proactive measures to address them before the risks escalate. Alerts may be triggered by a variety of factors, such as changes in system activity or user behavior. The purpose of an alert may be to keep users informed of the latest security events and provide the information necessary to respond effectively. Alerts further provide timely and relevant information to security teams, thereby minimizing the impact of security incidents and improving the overall security posture of an organization.

[0088] Security findings may represent gaps or deficiencies in information systems, security procedures, controls, and / or implementations that may be exploited or activated by potential threats. These are primarily identified using static analysis techniques. Security findings may include a security assessment, security finding status, description, recommended next steps, a list of affected assets, etc.

[0089] Security finding details provide users with detailed information about a particular security finding. This feature allows users to access comprehensive information about a security finding, including a description of the security finding, associated activities, and recommended next steps to resolve the issue. This level of detail can be useful to understand the complete context of a security finding, allowing users to make informed decisions and take appropriate actions to address the issue. Security finding details can include severity (e.g., potential risk of the security finding if not addressed), status (e.g., current activity status for the incident, such as paused and running), description (e.g., why the incident was created), recommended next steps (e.g., remediation steps), and assets related to the security incident. For example, with a clear view of the affected assets, a user can take appropriate actions to remediate the issue, such as applying a patch or update, reconfiguring assets, or implementing new security measures. This information can also be used to track the progress of resolution efforts and ensure that affected assets are protected in a timely manner.

[0090] An exemplary monitoring workflow may begin with an event trigger due to a change in a source system, initiating a series of monitoring steps to evaluate the situation and determine further updates or actions that need to be taken, which may include the creation of new alerts or incidents and result in a recalculation of security scores for all affected assets and pipelines.

[0091] Service bus 340 may include various queues, such as a report queue 341 , an activity queue 342 , an inventory queue 343 , and a monitor queue 344 . The caches 350 may include an inventory cache 352 , a report cache 354 , and a monitor cache 356 .

[0092] The SQL DB 360 may include a global database 362 and a tenant database(s) 364 . The collector 370 may interface with various AI services 319 via connectors such as an Azure Machine Learning (AzureML) connector 371, a Databricks connector 372, and ML Flow connectors 373, 374, 375. The browser 302 may communicate with a schedule servicer 380, which interfaces with an asset discovery engine 382 and an analytics engine 384 (supporting analytics workers 386), via an administration UI 306. The collector 370 may also include a cache 390 supporting a job queue 392, and a NoSQL database 394 supporting an asset database 396 and an analytics database 398.

[0093] The collector 370 may establish a web socket connection with the security center to get information (e.g., pipeline / asset mapping) since the last time it ran. For example, the collector may launch a log analysis engine to access logs since the last time it ran. In addition, the collector may launch an asset discovery engine to check for assets that have changed since the last time it ran. The collector may add the log and asset information to a local data store. Furthermore, the collector may launch an adversarial analysis engine to check for matches with the newly added data. The collector may then push the information (e.g., log events, new assets, analysis results, etc.) to the core platform 310.

[0094] The asset discovery engine 382 can obtain asset details for assets in the asset hierarchy. The asset details can include an asset identifier, an asset name, an asset type, a sibling relationship, a parent relationship, and the like.

[0095] The analytics engine 384 may run an event analysis process on the new event. The event analysis may be performed on the asset or on a pipeline that includes the related assets. The analysis may be segmented based on the event criteria and the related assets so that only relevant analysis may be performed for each event. The event criteria may include the type of event (e.g., asset event, pipeline event, etc.), related asset information (e.g., type of asset, whether the asset is part of a pipeline, etc.), pipeline information (e.g., dataset, model, experiment, deployment, etc.), and the context of the related asset with respect to the pipeline (e.g., dataset, model, etc.). The analytics engine 384 may run in a prioritized order so that previous analysis results are available for later analysis. Additionally, the analytics engine 384 may receive events as input and load values ​​as needed.

[0096] The analytics engine 384 may perform evaluation and detection functions. For example, an alert rule may be executed once for each pipeline that contains an associated asset. Evaluation may be performed based on criteria specified in each alert rule such that an associated rule may be executed for each event. Alert criteria may include the type of event, asset event, associated pipeline information, etc. Evaluation criteria for each type of asset or event may be executed for each asset / user in each matching pipeline. Collector evaluation may receive an event as input along with an identifier for the specific asset being evaluated and evaluation criteria definitions for the associated evaluation items. Collectors may load values ​​as needed. Detection functions may be performed in the context of the pipeline in which the alert was generated. When a new alert or incident is generated, a rescore may be performed for all assets included in the evaluation, all pipelines that contain those assets, and the overall security score. Alert rules may be evaluated based on events occurring within the collector.

[0097] FIG. 3 may support a variety of implementations, applications, and use cases. For example, use cases may include Data Poisoning, Configuration Vulnerabilities, and Inference Attacks. Data Poisoning may include public or internal datasets being tampered with. Data Poisoning may be more impactful when done on raw datasets as it may have downstream effects. Configuration vulnerabilities may include vulnerabilities that allow unauthorized access to a company's intellectual property (IP), which may be similar to traditional cyber attacks. Inference Attacks may include situations where a malicious actor strategically accesses an app to try to extract or replicate a model, anomalous activity on the production side of an AI pipeline.

[0098] 4 is an architecture of a collector according to an embodiment of the present invention. Security center 410 may receive data from various sources and systems as represented by customer environment(s) 412. Customer environment 412 may support multiple machine learning services, machine learning platforms, etc. In this example, collector 420 may interface with a cloud service such as AzureML 422 for accelerating and managing the lifecycle of machine learning projects, collector 430 may interface with a cloud machine learning platform such as SageMaker 432, collector 440 may interface with a machine learning platform such as Vertex AI 442, and collector 450 may interface with custom ML assets 452 in a custom / on-premise environment. Other external and internal systems may be supported across a variety of different platforms, implementations, and / or architectures.

[0099] FIG. 5 illustrates an asset discovery process according to one embodiment of the present invention. FIG. 5 illustrates interactions between an asset discovery engine 510, an ML platform connector 512, a database 514, and an ML platform 516. The asset discovery engine 510 initiates a job acquisition process and the ML platform connector 512 provides a response. The asset discovery engine 510 may then run an initialization job(s) until no assets / events are discovered. The ML platform connector 512 may then retrieve the asset(s) for the job from the ML platform 516. The asset discovery engine 510 may then store the assets in the database 514. The asset discovery engine 510 may then run an event job through the ML platform connector 512.

[0100] FIG. 6 is an exemplary collector data architecture according to an embodiment of the present invention. FIG. 6 illustrates the interactions between the collector 602, services 604, and machine learning system 606. The collector 602 may include an asset discovery engine instance 620, an ML database 630, a data structure store 640, a Python service 650, and an integration service 660. The services 604 may include an API management (APIM) 670, configuration 672, pipeline 674, and management 676. The APIM 670 allows users to publish, secure, transform, maintain, and monitor multiple APIs. The APIM 670 may enable organizations to create complete API programs by using existing backend services to create a consistent and modern API gateway.

[0101] The asset discovery engine instance 620 may include connectors 610 that may support web hooks (API endpoints) 612, initialization / schedule jobs 614, and event jobs 616. The asset discovery engine instance 620 may also include a persistence layer 618 that interfaces with an ML database 630. The ML database 630 may store and manage data related to ML assets 632 and analysis results 634. A Python service 650 may read / write from the ML database 630. The Python service 650 may include a runtime environment 652 and an executor 654. The data structure store 640 may perform actions such as getting an event queue and sending an event to a queue for the event jobs 616, the Python service 650, and the integration engine 660.

[0102] The integration engine 660 may support synchronization assets 662, a script command receiver 664, and an event subscription receiver 668. The integration engine 660 may interact with services 604 via an APIM 670. The services 604 may include configuration 672, pipeline 674, and management 676.

[0103] As shown in FIG. 6, the asset discovery engine instance 620 may collect resource data from the machine learning system 606 in multiple ways. The asset discovery engine instance 620 may search ML system resources to obtain asset data through initialization and scheduled jobs (represented by 614). API endpoints implemented as webhooks as shown by 612 may be called by the machine learning system 606. The data may be immediately stored in a queue (e.g., a Redis queue) in the data structure store 640 to avoid overload. The data (ML assets 632) may be stored in the ML database 630.

[0104] The analytics engine may explore data of the ML assets, as indicated by 632. The analytics engine may collect data from the ML database 630, analyze the data, and store it, as indicated by 632. The integration engine 660 may send analytics engine data to the service 604 via a RESTful API and then listen to synchronize configurations from the service 604 via web sockets. The Python runtime environment, as indicated by 652, may represent an independent runtime Python environment for dynamically executing Python scripts, as indicated by executor 654.

[0105] FIG. 7 is an exemplary flow chart for adversarial detection according to an embodiment of the present invention. FIG. 7 illustrates a process for detecting security findings, incidents, and / or alerts. In step 710, an event execution may occur in an external system. In step 712, an event discovery may be performed in a collector agent. In step 714, an event analysis may be performed in a collector agent. In step 716, the event and asset metadata may be provided to a core platform. In step 718, an associated pipeline context may be obtained. In step 720, a request for a statistical analysis of the asset and pipeline context may be made to the collector agent. In step 722, the analysis may be performed and provided to the core platform. In step 724, a retrieval of associated evaluation rules may be performed. In step 726, an evaluation against the rules may be performed and provided to the core platform. In step 728, security findings and alerts may be created. In step 730, an associated incident detection rule may be obtained. In step 732, a detection request for the asset and pipeline context may be made to the collector agent. At step 734, detection results may be executed and provided to the core platform. Incident creation and rescoring may be performed at step 736. Although the process of Figure 7 shows several steps being performed in a particular order, it should be understood that embodiments of the invention may be practiced by adding one or more steps to the process, omitting steps in the process, and / or changing the order in which one or more steps are performed.

[0106] 8A and 8B are exemplary diagrams of a security center interface according to an embodiment of the present invention. Figures 8A and 8B show a high-level overview of an enterprise's overall AI pipeline health, visibility into AI assets, and the status of AI notable events and incidents. An embodiment of the present invention determines whether the AI ​​pipeline is operating properly, whether the AI ​​pipeline is compromised, and the downstream and business impacts of the compromise.

[0107] As shown in FIG. 8A, the security center interface may provide details regarding the health of a particular AI system. Health may be assessed based on a variety of metrics including security score, model security score, data security score, total assets, pipelines, AI models, and datasets. Organizational pipeline details may be provided. By providing an overview of key security indicators, FIG. 8A and FIG. 8B help users quickly identify areas of concern and better understand the impact of issues on performance.

[0108] Security Score may represent a measure of the security posture of the pipeline based on an algorithm incorporating incidents, alerts, and security findings. Model Security Score may calculate a security score (e.g., 0-100) across multiple models associated with this pipeline. Data Security Score may calculate a security score (e.g., 0-100) across multiple data sets associated with this pipeline. Total Assets represents the total number and breakdown of assets associated with this pipeline. Pipeline provides a visual representation of all assets in this pipeline and their individual security ratings.

[0109] According to one embodiment of the present invention, the security score may be based on the criticality of the pipeline, the higher the criticality, the greater the impact from incidents, alerts, and security findings.

[0110] According to an embodiment of the invention, a security score may be calculated for an individual asset, a single pipeline, multiple pipelines, etc. For an individual asset, the security score may take into account the impact of an incident, the impact of an alert, and / or the impact of a security investigation result.

[0111] For incident impact, an individual incident impact score may be calculated by taking the values ​​in the scoring matrix that correspond to the incident severity and confidence rating. The overall incident impact score for an asset may be calculated as the maximum of all the individual incident impact scores. For example, if an asset has two incidents, one with medium severity and high confidence and the other with high severity and medium confidence, the overall incident impact score for the asset will be 0.9. An asset with no incidents will have a score of 0.

[0112] For alert impact, an individual alert impact score may be calculated by taking the values ​​of a scoring matrix corresponding to the alert priority and confidence rating. The overall alert impact score for an asset is calculated as the maximum of all individual alert impact scores. For example, if an asset has two alerts, one with medium priority and high confidence and the other with high priority and medium confidence, the overall alert impact score for the asset will be 0.9. An asset with no alerts will have a score of 0.

[0113] For security finding impact, an individual security finding impact score may be calculated by taking the scoring table value that corresponds to the security finding severity rating. The overall security finding impact score for an asset is calculated as the maximum of all the individual security finding impact scores. For example, if an asset has two security findings, one with a medium severity and the other with a high severity, the overall security finding impact score for the asset will be 1.0. An asset with no security findings will have a score of 0.

[0114] For a single pipeline, the security score may be calculated by taking a weighted average of each asset associated with that pipeline. For multiple pipelines, the security score may be calculated by taking a weighted average of each pipeline. If an asset is in multiple pipelines, the asset may have a unique score for each pipeline it belongs to. For example, if an asset A1 is in two pipelines P1 and P2, and P1 has a higher severity level compared to P2, the incidents and alerts related to A1 in pipeline P1 may be different from those in pipeline P2.

[0115] As shown in FIG. 8B, the security center interface may provide data regarding isolated assets and resilience recommendations. Security trends provide a graphic (e.g., line graph) of the number of notable events and incidents over a given time period. A breakdown of notable event risk may be divided into high, medium, and low urgency. A breakdown of incident risk may be divided into high, medium, and low urgency. An incident list represents a graphic (e.g., table) displaying the top number of incidents sorted by urgency and number of assets.

[0116] 9 is an example diagram of a pipeline manager interface according to one embodiment of the present invention. As shown in FIG. 9, a new pipeline can be added. Details of an organization's pipelines can be shown graphically. Other data can be provided including importance, pipeline name / identifier, assets, security score, latest activity, status, etc.

[0117] 10 is an example diagram of an entity details interface according to an embodiment of the present invention. Entity details may include security, assets, and settings. Security details may include security score, model security score, data security score, and AI assets. Pipeline details may be shown graphically.

[0118] 11 is an example diagram of a new pipeline creation interface according to one embodiment of the present invention. Pipeline attributes may include project and business criticality, associated datasets and models, cloud and other integrations, team members with different access rights. Various users may be categorized as personas, which may include data scientists, data science managers, AI engineers, AI development directors, CISOs, etc. As shown in FIG. 11, users may provide pipeline names / identifiers, descriptions, and other business data such as business criticality.

[0119] FIG. 12 is an exemplary diagram of a notable events interface according to an embodiment of the present invention. FIG. 12 provides an organized view to better understand notable events. Notable events may represent any event in the AI ​​pipeline with a non-zero probability of malicious activity. Notable events may be categorized by urgency, type, importance, and confidence of malicious actor involvement. FIG. 12 illustrates notable events, affected assets, high urgency, medium urgency, and low urgency. Other variations in urgency thresholds may be applied. Filters such as all pipelines, last 7 days, and active events may be available. Activity details may include urgency level, pipeline, event type, event subject, importance, confidence, and date / time.

[0120] 13 is an example diagram of a suspicious activity interface according to one embodiment of the present invention. Details may include affected pipeline, urgency, status, severity level, confidence score, team, type of suspicious activity, description, affected assets, and recommended next steps. Activity log details may also be provided.

[0121] 14 is an example diagram of a business details interface according to one embodiment of the present invention. Business details may include importance, industry, business function, and project. 15 is an example diagram of a cloud assets interface according to one embodiment of the present invention. Cloud assets may include details about pipelines, models, and datasets.

[0122] Those skilled in the art will appreciate that the various embodiments described herein are capable of broad utility and application. Thus, while various embodiments are described in detail herein with reference to exemplary embodiments, it should be understood that the disclosure is illustrative and exemplary of the various embodiments, and is made to provide an enabling disclosure. Thus, the disclosure is not intended to be construed as limiting the embodiments and to exclude other such embodiments, adaptations, variations, modifications, and equivalent arrangements.

[0123] The foregoing description provides examples of different configurations and features of embodiments of the present invention. Although specific nomenclature and application / hardware types are described, other names and application / hardware may be used, and the nomenclature is provided as a non-limiting example only. Furthermore, although specific embodiments are described, it should be understood that the features and functions of each embodiment may be combined in any combination within the capabilities of one of ordinary skill in the art. The drawings provide additional exemplary details regarding various embodiments.

[0124] Various exemplary methods are provided herein as examples. The described methods may be performed or otherwise implemented by one or a combination of various systems and modules.

[0125] Use of the term computer system in this disclosure may refer to a single computer or multiple computers. In various embodiments, multiple computers may be networked. The networking may be any type of network, including, but not limited to, wired and wireless networks, local area networks, wide area networks, and the Internet.

[0126] According to an exemplary embodiment, the system software may be implemented as one or more computer program products, e.g., one or more modules of computer program instructions encoded on a computer-readable medium for execution by or to control the operation of a data processing apparatus. The implementation may include single or distributed processing of an algorithm. The computer-readable medium may be a machine-readable store-time device, a machine-readable storage substrate, a memory device, or one or more combinations thereof. The term "processor" encompasses all apparatus, devices, and machines for processing data, including, by way of example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, an apparatus may include software code that creates an execution environment for the computer program, e.g., code constituting a processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof.

[0127] A computer program (also known as a program, software, software application, script, or code) can be written in any type of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program can be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple associated files (e.g., a file that stores one or more modules, subprograms, or portions of code). A computer program can be deployed for execution on one computer, or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0128] A computer may encompass all apparatus, devices, and machines for processing data, including, for example, a programmable processor, computer, or multiple processors or computers. In addition to hardware, a computer may include code that creates an execution environment for the computer program, such as code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these.

[0129] The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and an apparatus may also be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0130] Computer-readable media suitable for storing computer program instructions and data can include all types of non-volatile memory, media and memory devices, including, by way of example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0131] Although the embodiments are specifically shown and described within a framework for performing the analysis, it will be understood that variations and modifications may be made by those skilled in the art without departing from the scope of the various embodiments. Moreover, those skilled in the art will recognize that such processes and systems need not be limited to the specific embodiments described herein. Other embodiments, combinations of the embodiments, as well as uses and advantages, will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. The specification and examples should be considered as illustrative.

Claims

1. 1. A system for implementing an artificial intelligence (AI) security platform, comprising: A security center platform that acts as a central software-as-a-service (SaaS) portal and collects and aggregates telemetry data across one or more AI implementations; an interactive user interface configured to provide an analytical dashboard for tracking the one or more AI implementations to identify one or more adversarial security issues; and a computer server including a computer processor coupled to the interactive user interface and to the security center platform, the computer server comprising: a data integrity module configured to provide data security and integrity in the one or more AI implementations and further provide adversarial protection; a platform configuration monitoring interface configured to provide configuration monitoring associated with the one or more AI implementations; and a model security module configured to provide insight into how one or more models are performing from a security perspective, and further provide model recognition and anomalous model behavior analysis.

2. A system for implementing an artificial intelligence (hereinafter referred to as AI) security platform, a core security platform comprising a computer processor in communication with an API gateway, a container orchestration platform, and at least one memory component; an interface for communicating with one or more collector agents that receive data from one or more AI services; an asset discovery engine configured to collect metadata about one or more assets used for development and delivery of at least one AI pipeline, the metadata relating to asset type, activity, data source, and model; an analytics engine configured to identify potential threats from adversarial attacks by monitoring the one or more AI services, the one or more assets, and the at least one AI pipeline to generate at least one security score that evaluates an entity's security posture, the at least one security score being based on one or more of security incidents, alerts, and security investigation results.

3. The system of claim 2 , wherein the one or more collector agents operate in a customer environment.

4. The system of claim 2 , wherein the at least one memory component comprises at least one service bus, at least one cache, and at least one database.

5. The system of claim 2 , wherein the at least one security score comprises a model security score and a data security score.

6. The system of claim 2 , wherein the interface is configured to display a graphical representation of a tissue pipeline.

7. The system of claim 2 , wherein the interface is configured to display a graphical representation of one or more of notable events and incidents.

8. The system of claim 2 , wherein the interface is configured to display one or more resiliency recommendations.

9. The system of claim 2 , wherein the container orchestration platform includes an operations management module, a reporting module, an inventory management module, and an inventory monitoring module.

10. The system of claim 2 , wherein the at least one security score is calculated by an individual asset, an individual pipeline, or multiple pipelines.

11. 1. A method for implementing an artificial intelligence (AI) security platform, comprising: communicating with one or more AI services by one or more collector agents; collecting, by an asset discovery engine, metadata about one or more assets used for the development and delivery of at least one AI pipeline, the metadata relating to asset type, activity, data source, and model; monitoring, with an analytics engine, the one or more AI services, the one or more assets, and the at least one AI pipeline to identify potential threats from adversarial attacks; generating, by the analytics engine, at least one security score assessing a security posture of an entity, the at least one security score being based on one or more of security incidents, alerts, and security investigation results; and providing, via an interface, security health data representative of the security posture of the entity, the security health data including the at least one security score.

12. The method of claim 11 , wherein the one or more collector agents operate in a customer environment.

13. The method of claim 11 , wherein the at least one memory component comprises at least one service bus, at least one cache, and at least one database.

14. The method of claim 11 , wherein the at least one security score comprises a model security score and a data security score.

15. The method of claim 11 , wherein the interface is configured to display a graphical representation of a tissue pipeline.

16. The method of claim 11 , wherein the interface is configured to display a graphical representation of one or more of notable events and incidents.

17. The method of claim 11 , wherein the interface is configured to display one or more resiliency recommendations.

18. The method of claim 11 , wherein the container orchestration platform includes an operations management module, a reporting module, an inventory management module, and an inventory monitoring module.

19. The method of claim 11 , wherein the at least one security score is calculated by an individual asset, an individual pipeline, or multiple pipelines.

20. 12. The method of claim 11, wherein the one or more collector agents are in communication with the asset discovery engine and the analytics engine, and further in communication with a core platform comprising a computer processor in communication with an API gateway, a container orchestration platform, and at least one memory component.

Images