System and method for implementing an artificial intelligence security platform

Abstract

The present invention relates to an AI platform to simplify artificial intelligence (AI) security for enterprise and other applications. One embodiment of the present invention implements an AI security platform that secures AI / ML models while keeping configuration and implementation simple and streamlined. One embodiment of the present invention provides visibility into AI models across an entire organization as well as other enterprise structures.

Description

Technical Field

[0001] The present invention relates to an AI platform for simplifying artificial intelligence (AI) security for enterprise applications and other applications.

Background Art

[0002] Artificial intelligence is one of the largest and fastest growing areas in the software field. As these technologies progress, companies will increase their use of AI and data to create a competitive advantage.

[0003] More than half of global companies are using AI in at least one business function. The world's AI software industry is expected to double in just a few years. Industrial manufacturing, financial services, technology, retail and life sciences are introducing AI at the fastest pace.

[0004] Predictions show that 75% of organizations will shift from testing and piloting AI to starting operations. In fact, AI is expected to increase productivity levels by about 40% across front-end and back-end processes. Such a directional change is driven by improved access to data and the development of very flexible models to adapt to specific business needs. Regulatory authorities have noticed the increasing use of AI and are formulating a regulatory compliance framework to protect stakeholders. Therefore, in the industry, an increase in the use of AI, the creation of large amounts of data, the spread of cyberattacks, and the enactment of regulations are expected to occur.

[0005] The implementation of AI and ML models into business processes is accelerating rapidly. However, industry experts are not prepared to protect, detect, and respond to attacks against AI and ML models. Companies leveraging AI and ML models are expected to adapt to the rapidly changing regulatory landscape.

[0006] In the current industry, there is an assumption that AI / ML implementations are already secure. However, standards and protocols do not adequately address the security of AI platforms. In addition, there are various types of models and AI implementations running on different ecosystems. Current AI implementations are not standardized and are extremely complex.

[0007] Therefore, it is desirable to have systems and methods that can overcome the aforementioned shortcomings of known systems. [Overview of the Initiative]

[0008] According to one embodiment, the present invention relates to a computer-implemented system and a method for implementing an artificial intelligence (AI) security platform. The system comprises a security center platform that functions as a centrally managed software-as-a-service (SaaS) portal and collects and aggregates telemetry data across one or more AI implementations; an interactive user interface configured to track one or more AI implementations and provide an analytics dashboard for identifying one or more adversarial security issues; and a computer server having a computer processor coupled to the interactive user interface and the security center platform, wherein the computer server further comprises a data integrity module configured to provide data security and integrity in one or more AI implementations and to further provide adversarial protection; a platform configuration monitoring interface configured to provide configuration monitoring related to one or more AI implementations; and a model security module configured to provide insights into how one or more models are performing from a security perspective and to further provide model recognition and anomalous model behavior analysis.

[0009] In another embodiment, the present invention relates to a computer-implemented system for implementing an AI security platform. The system comprises a core security platform comprising a computer processor communicating with an API gateway, a container orchestration platform, and at least one memory component; an interface communicating with one or more collector agents that receive data from one or more AI services; an asset discovery engine configured to collect metadata about one or more assets used for the development and delivery of at least one AI pipeline, the metadata relating to asset type, activity, data source, and model; and an analysis engine configured to identify potential threats from adversarial attacks by monitoring one or more AI services, one or more assets, and at least one AI pipeline and generating at least one security score that evaluates the security posture of an entity, the at least one security score being based on one or more of security incidents, alerts, and security findings.

[0010] In another embodiment, the present invention relates to a computer-implemented method for implementing an AI security platform. The method includes: communicating with one or more AI services by one or more collector agents; collecting metadata by an asset discovery engine relating to one or more assets used for the development and delivery of at least one AI pipeline, wherein the metadata relates to asset type, activity, data source, and model; monitoring one or more AI services, one or more assets, and at least one AI pipeline by an analysis engine to identify potential threats from adversarial attacks; generating at least one security score by an analysis engine that evaluates the security posture of an entity, wherein the at least one security score is based on one or more of security incidents, alerts, and security findings; and providing security health data by an interface that represents the security posture of an entity, wherein the security health data includes at least one security score.

[0011] As companies invest in AI systems, security and model performance become critical concerns. Embodiments of the present invention understand various aspects of AI, including security, performance, optimization, and bias. Through these various embodiments, entities can reach a point of satisfaction with AI in core and critical business processes. The innovative system seeks to break down AI silos in order to gain a deeper understanding of the AI ​​portfolio.

[0012] These and other advantages will be described in more detail in the following detailed explanation. [Brief explanation of the drawing]

[0013] To facilitate a more complete understanding of the present invention, refer to the accompanying drawings. The drawings are not intended to be construed as limiting the present invention, but are intended solely to illustrate different aspects and embodiments of the present invention. [Figure 1] This is an illustrative architecture diagram of one embodiment of the present invention. [Figure 2] This is a diagram of an exemplary architecture according to one embodiment of the present invention. [Figure 3] This is a diagram illustrating an exemplary application architecture according to one embodiment of the present invention. [Figure 4] This is a diagram of the collector architecture according to one embodiment of the present invention. [Figure 5] This figure shows the asset discovery process according to one embodiment of the present invention. [Figure 6] This is a diagram illustrating an exemplary collector data architecture according to one embodiment of the present invention. [Figure 7] This is an exemplary flowchart for adversarial detection using one embodiment of the present invention. [Figure 8A] This is an illustrative diagram of a security center interface according to one embodiment of the present invention. [Figure 8B] This is an illustrative diagram of a security center interface according to one embodiment of the present invention. [Figure 9] This is an illustrative diagram of a pipeline manager interface according to one embodiment of the present invention. [Figure 10] This is an illustrative diagram of an entity detail interface according to one embodiment of the present invention. [Figure 11] This is an illustrative diagram of a new pipeline creation interface using one embodiment of the present invention. [Figure 12] This is an illustrative diagram of a notable event interface according to one embodiment of the present invention. [Figure 13] This is an illustrative diagram of a suspicious activity interface according to one embodiment of the present invention. [Figure 14] An exemplary diagram of a business detail interface according to an embodiment of the present invention. [Figure 15] An exemplary diagram of a cloud asset interface according to an embodiment of the present invention.

Embodiments for Carrying Out the Invention

[0014] To illustrate various features of the present invention, exemplary embodiments of the present invention will be described. The embodiments described herein are not intended to limit the scope of the present invention. Rather, they are intended to provide examples of the components, uses, and operations of the present invention.

[0015] One embodiment of the present invention simplifies and rationalizes configuration and implementation while securing an artificial intelligence (AI) / machine learning (ML) model. One embodiment of the present invention provides visibility regarding AI models across the organization as well as other corporate structures.

[0016] The implementation of AI models and ML models into any business process is accelerating. These systems integrate data from multiple sources and typically utilize learning algorithms of unknown origin. This makes it difficult for entities, such as the Chief Information Security Officer (CISO) and other users, to ensure visibility, cybersecurity, and risk management for appropriate protection against AI-specific risks.

[0017] According to an exemplary implementation form, the AI security platform may include a SaaS application that supports an AI / ML platform for simplifying and rationalizing security, risk management, and monitoring of important AI security controls. The AI security platform has the ability to understand and reduce security risks caused by using AI / ML technologies in a reasonable way.

[0018] In one embodiment of the present invention, the AI security platform enables visibility into major security threats and monitors threats to major AI / ML engines provided by various providers (e.g., Microsoft (registered trademark), Google (registered trademark), IBM, KPMG, Amazon (registered trademark), etc.). Other platform support may be performed based on client requirements, market opportunities, and / or other considerations. Additional security, risk, and / or ethics management may be added.

[0019] One embodiment of the present invention may support various users, industries, and applications. Exemplary personas may include a CISO (primary), an AI security team (primary), an AI engineering team (secondary), and an application security team (secondary).

[0020] According to an exemplary illustration, personas may include CIO / CISO / director users within an organization. Regulations are changing rapidly, and this user may be tasked with protecting data and systems and protecting the organization. In this example, the organization may attempt to apply industry-leading best practices to the end-to-end AI process, simplify the security of the AI system, and better understand the impact of an intrusion or attack on the AI system. What is expected may include protecting against major threats and intrusions, simplifying the process of protecting the security of AI, improving visibility and transparency, and addressing future regulatory compliance.

[0021] One embodiment of the present invention provides visibility into key security threats to AI / ML systems. This provides AI security awareness, meaning that clients can recognize security risks posed by AI components, including breaches of data, models, etc. Embodiments of the present invention facilitate integration and are easy to incorporate into existing ecosystems, for example, working in conjunction with various AI / ML platforms. The innovative system facilitates ethical AI / ML decision-making, allowing clients to verify that models comply with ethical standards and / or other standards. Accordingly, embodiments of the present invention provide a standard method for verifying deployed models and their capabilities.

[0022] Figure 1 shows an exemplary architecture according to one embodiment of the present invention. The AI ​​security center platform 110 may support various functions, including a data integrity module 120, a platform configuration monitoring (API) 130, and a model security module 140.

[0023] One embodiment of the present invention provides insights into AI implementations and helps clients securely and reliably integrate AI into their core business functions. The innovative system allows users to monitor AI implementations, understand data under attack, and gain further protection against various threat scenarios.

[0024] The AI ​​Security Center Platform 110 can create security standards and related frameworks for adversarial AI. The AI ​​Security Center Platform 110 provides a single location for tracking platforms, models, and / or data sources. This may include AI portfolio and threat / vulnerability tracking, as well as manual and automated mapping. The AI ​​Security Center Platform 110 further provides analytics dashboards, identification and prioritization of key risks, collection and aggregation of telemetry data (related to core risks), analysis of telemetry across internal systems and peers, and user management.

[0025] According to one embodiment of the present invention, the AI ​​Security Center Platform 110 functions as a centralized management (SaaS) portal. The AI ​​Security Center Platform 110 aggregates the individual functions of the solution, manages the client journey, provides dashboards and reports, and provides user management. The AI ​​Security Center Platform 110 can further provide an interactive user interface to various users, such as the CISO and the AI ​​security team. The interactive user interface generates a single, integrated view of adversarial security protection. The interactive user interface can further manage the client journey and aggregate the functions of the platform's components to ensure data integrity, secure configuration, and model behavior.

[0026] The data integrity module 120 ensures that data is secure. The data integrity module 120 provides functionality focused on structured data models. This can be achieved by a core framework, extract, transform, and load (ETL) pipeline testing, applying standardized dataset integrity, and identifying cases where the training set is infected. The data integrity module 120 can provide various types of protection, including adversarial / dataset poisoning protection 122 and integration / man-in-the-middle attack protection 124.

[0027] Adversarial / dataset poisoning protection 122 can leverage blueprints from IEEE, OpenAI, MITRE, and others. Furthermore, adversarial / dataset poisoning protection 122 can identify adversarial attack signals from peers, as well as other functionalities such as data tagging, tokenization, and file security (e.g., PKL file security).

[0028] In addition, the AI ​​Security Center Platform 110 supports data integrity through a data integrity module 120, enabling various users, such as CISOs and AI security experts, to monitor the security of training (and operational) datasets used in various leading AI / ML learning systems. One embodiment of the present invention develops the ability to detect and protect against adversarial dataset poisoning attacks (and man-in-the-middle attacks) to ensure proper data security and integrity in AI / ML architectures. This is represented by adversarial / dataset poisoning protection 122 and integration / man-in-the-middle attack protection 124, as shown in Figure 1. Currently, CISOs and AI security teams struggle to ensure the integrity and security of both AI training sets and ongoing datasets on which AI / ML learning systems rely to provide safe, reliable, and error-free results. One embodiment of the present invention provides protection and visibility against data poisoning and other data-centric attacks against AI / ML architectures that could result in significant errors and breach risks.

[0029] Platform Configuration Monitoring (API) 130 provides platform configuration monitoring, which may include analyzing security configurations against vendor, industry, and best practices. Other features may include mechanisms for setting security baselines, configuration impact analysis, pre-production review, continuous monitoring, and configuring platform and isolation capabilities. Platform Configuration Monitoring (API) 130 may further support platform-specific secure configurations, as represented by 132.

[0030] The Platform Configuration Monitoring (API) 130 can analyze various models and generate baselines to ensure that identified and tracked systems conform to the baseline. For example, the Platform Configuration Monitoring (API) 130 can analyze how a system is configured and compare the current configuration to best practices or standards.

[0031] The AI ​​Security Center Platform 110 supports secure configuration, as shown in 132, providing the ability to map and monitor AI / ML platform and model configurations against secure configuration standards to determine whether models are adequately protected from adversarial risks. Embodiments of the present invention provide visibility and monitoring of configuration settings to ensure that AI / ML platforms are securely configured. In current systems, many CISOs and AI security professionals do not have sufficient visibility into the configurations of platforms supporting AI / ML integration. This creates significant security holes that, if exploited, could expose systems to serious adversarial attacks, data breaches, and privacy risks. One embodiment of the present invention enables users to understand and monitor their security posture against manufacturer best practices and guidance to mitigate security risks.

[0032] Secure Configuration 132 can determine whether a system is securely configured by ensuring that the AI ​​platform has a privacy configuration that complies with various standards (e.g., vendor standards, industry standards, best practices, etc.). Secure Configuration 132 can also perform the setting of a security baseline. For example, one embodiment of the present invention can determine whether a system conforms to security standards and, if not, how to modify the system or bring it into conformance with the appropriate standards. This can be performed across multiple platforms.

[0033] The model security module 140 may include model recognition 142 and anomalous model behavior analysis 144. In addition, the model security module 140 may use MITRE (or other frameworks, curated knowledge bases and models) as a guide to continuously test the model (e.g., 24 / 7) and provide support for inference risk (e.g., whether the data and results are as intended).

[0034] For example, the model security module 140 may allow the user to understand how the model functions from a security standpoint. One embodiment of the present invention determines whether a hacker has modified the model.

[0035] The AI ​​Security Center Platform 110 supports behavioral monitoring and analysis through the Model Security Module 140, which applies techniques to identify, understand, and monitor adversarial threats to ML / systems and models through Model Recognition 142 and Anomaly Model Behavior Analysis 144. Anomaly Model Behavior Analysis 144 supports AI models for security and can further test the system using SQL / chaos models.

[0036] One embodiment of the present invention provides visibility and monitoring to mitigate the cyber risks of AI / ML learning models integrated into core business processes that may be vulnerable to attack. Tampered models can pose fundamental risks to patient health and financial decisions, potentially causing significant long-term disruption. Because these systems are often "black boxes," errors may or may not be properly identified or managed over long periods. These systems typically attempt to exclude individuals who should be "checking" the process outcomes from the process itself. One embodiment of the present invention mitigates the cyber risks of anomalous AI model behavior based on adversarial AI threats.

[0037] One embodiment of the present invention provides visibility into the AI ​​pipeline from a security perspective. The innovative system provides automated mapping of AI platforms, models, and data sources. Furthermore, telemetry data can be collected and further analyzed. One embodiment of the present invention supports the development of standards and related frameworks.

[0038] According to one embodiment of the present invention, an automated mapping model may identify specific signatures and, based on those signatures, map and identify an AI system. This may include identifying calls to external data sources. For example, one embodiment of the present invention may infer how a system is processing training data and determine the type of model being applied.

[0039] One embodiment of the present invention collects telemetry data to provide visibility across various AI pipelines within an organization. This can also be done across multiple clients, platforms, entities, industries, etc. Embodiments of the present invention can be extended to determine various metrics across one or more entities. For example, one embodiment of the present invention may determine the highest attack rate in a particular domain or application (e.g., clinical trials, pharmaceuticals, etc.). For any determination, embodiments of the present invention may provide measures to modify and / or better protect AI / ML implementations.

[0040] One embodiment of the present invention provides real-time visibility to AI pipelines, accurately identifies attacks and potential attacks, and provides better protection against current and future attacks. One embodiment of the present invention can, for example, identify potential indicators of exposure to data poisoning. This provides insights and improves the entity's confidence in its AI platform.

[0041] One embodiment of the present invention can support a variety of user experiences, including mobile devices, virtual reality, or augmented reality. Figure 2 is a diagram of an exemplary architecture according to one embodiment of the present invention. Figure 2 is an example, and other variations in implementation and design may be realized. As shown in Figure 2, additional functionality may be provided by the AI ​​Trust Center Platform 210, AI Bias 220, AI Model Insights 230, and AI Infrastructure Security 240.

[0042] Figure 2 illustrates an exemplary system for integrating components to build reliability when implementing a model in a core business process. The system in Figure 2 provides insights into the model, such as whether the model has been tested against other standards in different countries.

[0043] The AI ​​Trust Center Platform 210 integrates elements such as cyber and model management, model insights, model behavior, and model bias. Embodiments of the present invention develop a reliable, centralized, and integrated AI security dashboard SaaS platform that provides visibility into the overall health of an enterprise's AI pipeline.

[0044] One embodiment of the present invention provides simplified AI security management by mapping AI pipelines, monitoring AI systems for key threats, and helping organizations respond to the impacts on AI systems, such as data loss and threats to AI. Through a holistic overview of an enterprise's AI / ML health, embodiments of the present invention protect systems both on-premises and in the cloud and provide solutions to vulnerabilities in the cybersecurity market. Through dashboard alerts and trend analysis, one embodiment of the present invention provides triage support for suspicious events and pursues their improvement.

[0045] The various benefits may include transparency regarding AI / ML threats, visibility into threats and team involvement, a simplified view of the AI ​​and ML model ecosystem that enables even those without a data science background to understand the security risks posed by the models, a SaaS model that enables rapid deployment, and improved efficiency and time / cost savings.

[0046] One embodiment of the present invention relates to a self-service, easily integrated platform for monitoring an AI system at various points in the AI ​​development lifecycle and providing alerts for various suspicious activities, threats, etc. This may include static analysis, monitoring of training data and features, model accuracy drift, model file versioning and drift analysis, and runtime analysis of outputs generated using these systems.

[0047] One embodiment of the present invention relates to Cross-Platform AI Pipeline Discovery. This embodiment utilizes a collector-based architecture to integrate with APIs and webhooks across multiple external AI development and delivery platforms to collect metadata about assets that can be used for the development and delivery of AI pipelines.

[0048] The collected metadata may relate to asset types, activities, data sources, models, experiments, jobs and notebooks, endpoints and deployments. Metadata related to asset types may include datasets, models, experiments, jobs, notebooks, endpoints, and deployments. Metadata related to activities may include version control, activity type, activity date, and user / system context. Metadata related to data sources may include format, storage, size, source information (sourcing), artifact location, and characteristics. Metadata related to models may include performance metrics, supporting libraries and frameworks, source information, and artifact location. Metadata related to experiments, jobs, and notebooks may include computation context, execution history, execution time, input / output, and artifact location. Metadata related to endpoints and deployments may include computation context, supporting libraries and frameworks, authentication context, and artifact location.

[0049] Metadata may be used to create and / or maintain asset records within the core platform, and asset records may be made available as part of the pipeline management process. Pipelines may be built by linking assets and users through manual and automated relationship mapping, forming a unified context about linked assets across the various platforms in which the assets reside.

[0050] One embodiment of the present invention relates to Pipeline Lineage Discovery. Embodiments of the present invention may construct asset lineage by utilizing asset metadata from an external AI development and delivery platform, analyzing available information, and discovering relationships between assets. The ultimately constructed lineage may form a graph from which ancestors and descendants of a given asset can be retrieved. Inference of relationships between assets may utilize analytical data collected from other sources, including code references, tokenizations, and / or identifying attributes, in addition to relational data available from the external AI development and delivery platform, to construct asset signatures which can be used for lineage discovery. The combination of lineages enables centralized and relevance-related context for pipeline-specific lineage discovery.

[0051] Embodiments of the present invention relate to an Adversarial Detection Engine. In one embodiment of the present invention, the process of detecting security findings, incidents, and alerts may be driven by a workflow that coordinates the execution of analysis and evaluation of both events and associated assets between an external AI development and distribution platform, a collector agent, and a core platform.

[0052] One embodiment of the present invention develops and applies a behavioral model for inside threat detection. According to one embodiment of the present invention, recorded activities for individual users within an external AI development and distribution platform can be analyzed in light of the pipeline context regarding pipeline users and associated responsibilities provided during pipeline creation. The effectiveness of the activities from an authorization standpoint can be evaluated against a set of assigned responsibilities in combination with regression and classification-based modeling, which collectively form a pattern matching system built on historical time-series data of user interactions to assess the potential for inside threats.

[0053] One embodiment of the present invention relates to an analysis for detecting adversarial AI threats. According to one embodiment of the present invention, a decision tree and parallel execution may be designed to use multiple detection methods in identifying adversarial AI threats. Each individual analysis run may provide intermediate results, which can be combined to determine the presence of one or more adversarial AI threats with higher accuracy than the individual results.

[0054] One embodiment of the present invention relates to an AI pipeline collector agent. According to one embodiment of the present invention, a collector agent built to operate in multiple cloud and on-prem environments can be used to provide AI security-related outputs, enabling integration with external AI development and deployment platforms, asset discovery, event monitoring, analysis, and adversarial threat detection. The collector agent may be designed to have deployment flexibility, security for both agent components and data, and scalability to meet the requirements of big data processing. The collector agent may be capable of self-updating a threat library to dynamically manage detection capabilities as new threats are discovered and new detections are created. The integration system is pluggable, and as a result, integrations can be enabled, disabled, created, and / or destroyed by remote commands without requiring direct intervention in the infrastructure.

[0055] One embodiment of the present invention relates to framework-agnostic AI pipeline analysis. According to one embodiment of the present invention, analysis execution can be unified using context-aware framework discovery capabilities. Discovery may be performed using pattern matching on asset artifacts and metadata to infer the relevant frameworks and functionalities. The discovered frameworks and functionalities may be validated or overridden within the core platform as part of pipeline asset management. The final set of frameworks and functionalities is then provided to the analysis execution process, and assets may be recreated in abstract representations of frameworks and data-specific attributes to provide unified processing of the analysis.

[0056] One embodiment of the present invention relates to an AI card that enables the collection of evidence for creating statements related to the compliance, security, and branding of an AI system. An AI card may contain specific compliance evidence relating to one or more pipelines, governance information, and / or the entire AI system. The compliance evidence for a given AI card may be specified by an applied card template, which includes the sections and fields necessary to demonstrate compliance based on field definitions. Fields may include free text, dates, numbers, checkboxes, single / multiple selections, and / or optional file upload attachments. Default field values ​​may be automatically populated by formulas constructed using data from inventory and monitoring data already collected by the core platform. Cards may be shareable within the platform with other users, regulatory bodies, file exports such as PDFs, third-party platforms, marketplaces, and exchanges, external auditors, and websites via HTML embedding. Version control, change management, and sharing history of AI cards may be stored on the platform for compliance and governance purposes. AI card templates can be modified, duplicated, and / or created from scratch by a form builder that has specific support for building formula-based automated inputs based on data collected on the core platform.

[0057] Embodiments of the present invention relate to a SaaS solution that continuously monitors an enterprise's AI systems and assets, detects potential threats from adversarial attacks, provides security scoring, and enables teams to gain visibility into the overall health of the enterprise's AI. One embodiment of the present invention provides visibility into the overall health of the enterprise's AI systems by integrating with the enterprise's existing security tools and AI pipelines to continuously scan and monitor AI assets and pipelines for vulnerabilities related to model performance and health. One embodiment of the present invention further provides security recommendations based on the confidence level of the exposed assets. For example, a collector utility may be installed and deployed in the environment where AI models are developed and deployed. The collector utility may be connected to the system in which development is being performed. This makes it possible to easily extract and aggregate data from various systems and apply a predetermined set of tests to those pipelines.

[0058] Various user capabilities may include loading asset information related to the ML system into the environment, broadly categorizing assets according to their lifecycle stage(s), browsing datasets (e.g., raw data, training data used for model training), browsing experiments, browsing models under development and models deployed to production environments (e.g., containers, dependencies, input and output distributions), and browsing isolated assets (e.g., status, changes over time).

[0059] One embodiment of the present invention implements a zero-day AI threat engine. Anonymized data and telemetry in monitoring data collected on the core platform may be analyzed using an unsupervised learning model to provide information on trends, potential threats, etc. The collected trend and threat information may be used on the management platform to create zero-day research findings that should be delivered to customer segments based on the relevance and presence of threat factors. The zero-day research findings may represent unique and potentially novel attacks against AI systems or industry segments that could not be detected by any other means without aggregated data collected and analyzed by an unsupervised learning system that supports trend and threat discovery.

[0060] One embodiment of the present invention relates to the automated detection of adversarial synthetic data augmentation. The use of data augmentation, generative AI, and public data can be tracked as part of the provenance and lineage metadata collected by the collector to initiate a detection workflow for identifying adversarial synthetic data. This detection technique can combine data provenance-based defense with batch clustering to identify inconsistencies in augmented data that may indicate poisoning of synthetic or public data, where each batch is analyzed for elements including statistical properties, cross-entropy, and clustering within an unsupervised learning model, which are then aggregated and fed into a supervised model built to detect anomalies across the entire batch.

[0061] Figure 3 shows an exemplary application architecture according to one embodiment of the present invention. The core platform 310 may include edge services 312 that interact with a browser 302. The browser 302 may run on various devices, including computer servers, client systems, mobile devices, and smart devices. The browser 302 may include an Identity User Interface (UI) 304, a management UI 306, and a customer UI 308. The edge services 312 may include an Identity Provider (IDP) 314, a Content Delivery Network (CDN) 316, and an application gateway 318.

[0062] The core platform 310 may include an API gateway 320, a container orchestration platform 330, a service bus 340, a cache 350, and a database such as SQL DB360.

[0063] The API gateway 320 may include an authentication API 321, a public API 322, a management API 323, an application API 324, and a collector API 325. The container orchestration platform 330 may include operational management 331, reporting 332, inventory management 333, and inventory monitoring 334.

[0064] Operational management 331 may include a configuration interface that provides effective security management by offering controls over user management, integration, scoring, and / or platform customization. User management ensures appropriate access and permissions, integration may be crucial for connecting to the organization's existing environment, and scoring configuration enables the evaluation of individual security scores and supports platform customization to meet unique security needs. User and role management may control access to sensitive information, enforce role-based access control, and track user activity.

[0065] Scoring configurations allow for customization of scoring, severity levels, and scoring scope for incidents, alerts, and security investigation results to align with an organization's evolving security objectives. This enables prioritizing and responding to incidents in a way that suits the organization's needs, improving security posture and enabling effective management.

[0066] Reporting 332 provides high-level insights into program health, assets, and / or pipelines, along with monitoring through charts / graphics, metrics, and key details. The reports can be designed to provide quick and comprehensive information for managers, directors, and executives who do not require detailed information. These reports are accessible to users across various platforms.

[0067] Health can be represented by various scores and metrics, including security scores (e.g., assessed security posture), model / dataset security scores (e.g., security scores across various pipeline models and datasets), asset breakdown (e.g., total number and breakdown of assets), pipeline, model, and dataset breakdown (e.g., total number of pipelines / models / datasets and breakdown of corresponding health states), and organizational pipelines (e.g., a comprehensive overview of the security status of different components across the organization's entire pipeline). The lifecycle can be represented by the state of pipelines and models (e.g., percentages of pipelines in training, pre-deployment, and operational phases), dataset breakdown (e.g., percentages of internal and external datasets), isolated assets (e.g., insecure assets not currently monitored or managed by the pipeline), and resilience recommendations (e.g., actionable insights for strengthening the organization's security posture). Security can be represented by security trends (e.g., charts of the number of incidents and alerts over a period of time), incident breakdowns (e.g., high severity, medium severity, and low severity), alert breakdowns (e.g., high priority, medium priority, and low priority), and critical incidents (e.g., incidents sorted by severity).

[0068] Inventory management 333 may relate to the management of assets and pipelines. Inventory management 333 may manage and track multiple types of assets within the AI ​​development pipeline. These assets may include datasets, models, experiments, and deployments. These are considered critical elements in the development and deployment of AI models and may be further monitored and managed throughout their entire lifecycle to ensure security and performance.

[0069] Asset types may include datasets, models, experiments, and deployments. Other asset types may also be supported. A dataset can represent a collection of data used to train and evaluate a model in terms of its accuracy and performance. According to one embodiment of the present invention, supported dataset types may include tables, images, text, and the like.

[0070] A model can represent data organization and standardization techniques for recognizing patterns within data. Models can be trained on datasets and retrained with feedback and / or new data to improve their performance. Various model frameworks, including automated machine learning platforms, may be supported. A variety of model objectives, including regression and classification, may be supported.

[0071] Regression can represent a statistical analysis technique used to model and analyze the relationship between a dependent variable and one or more independent variables. Target metrics may include normalized RMS error, Spearman correlation, R² score, and normalized mean absolute error. Classification can represent a machine learning task that involves assigning data points to a given category or class based on several features or attributes. For example, the goal may be to train a model to accurately predict the class label for new, unidentified data points. Target metrics may include accuracy, precision, recall, F-score (F1), ROC / AUC, etc.

[0072] An experiment can refer to the process of building, training, testing, and analyzing a machine learning model. An experiment can also represent a single machine learning task that may involve multiple steps, including data preparation, model selection, training, and evaluation.

[0073] Deployment can refer to the process of taking a trained machine learning model, putting it into production, and making it available to users in real applications. This may include obtaining the trained model, packaging the model with its necessary dependencies, and deploying the model to the target environment.

[0074] According to one embodiment of the present invention, a pipeline may encompass various assets and stages in the process from training to deployment of an AI model. When creating a pipeline, organizations may provide identifiable contextual information, such as business importance and business conditions, to help them understand higher-risk areas. The pipeline may also include relevant assets (e.g., datasets, models, experiments, and deployments) and a list of team members to whom pipeline roles are assigned.

[0075] In one embodiment of the present invention, security scores may be calculated based on individual assets, individual pipelines, and multiple pipelines. From a pipeline perspective, security scores may be calculated based on the assets contained within that pipeline. Generated alerts or incidents may be directly linked to individual pipelines, depending on the severity of each pipeline. For example, pipelines containing the same assets as another pipeline may have different alerts or incidents if they have a high severity level as opposed to a low severity level.

[0076] One embodiment of the present invention determines a security score to evaluate the security posture of an organization or a single pipeline. The security score algorithm may be based on multiple factors, including vulnerabilities, incidents, and surface area of ​​the AI ​​pipeline. The score may be applied on various scales, such as a 0-1 scale or a 0-100 scale, where a score of 0 represents that all assets have incidents of high severity and high confidence, and a score of 1 or 100 represents that there are no incidents, alerts, or security findings across all assets. The security score may be calculated by combining the impacts from incidents, alerts, and / or security findings. Pipelines may have a defined importance level, and the higher the importance level, the greater the impact of incidents, alerts, and security findings.

[0077] Security scores can be applied in various ways, including applying them to the entire organization by combining the scores of all pipelines; applying them to each individual pipeline by combining the scores of all assets included in the pipeline; applying them to the organization's models by combining the scores of all models included in the pipeline; applying them to the organization's datasets by combining the scores of all datasets included in all pipelines; applying them to each individual pipeline's model by combining the scores of all models included in all pipelines; and applying them to each individual pipeline's dataset by combining the scores of all datasets included in all pipelines.

[0078] One embodiment of the present invention creates a new pipeline to address specific business needs. This process may include providing contextual details, defining desired outcomes, and determining the required inputs, tools, and resources.

[0079] Embodiments of the present invention enable users to modify existing pipeline information by configuring and / or updating pipeline details, including, but not limited to, reassigning pipelines to new projects, adjusting business importance after security reviews, and / or managing team members associated with pipelines.

[0080] One embodiment of the present invention involves managing pipeline assets, which are key to keeping the assets in an updated and valid state, thereby enabling users to modify assets, for example, by selecting a new model or incorporating a new dataset for model retraining.

[0081] Inventory monitoring 334 may relate to monitoring events, incidents, alerts, and security findings. Inventory monitoring 334 may represent a toolkit for continuous collection and analysis of security information, providing real-time visibility into security events, detecting threats and anomalies, and providing users with relevant information for rapid decision-making. One embodiment of the present invention may function as an early warning system that enables an organization to detect and respond to security incidents before they escalate, thereby proactively managing its security posture and reducing the risk of data loss or data theft.

[0082] An event can refer to any change that occurs within the source system and also serves as a trigger for the monitoring process. Examples of events may include the creation of a dataset, a failed experiment run, the addition of a new, updated version of a model, or the removal of a deployment.

[0083] Events can capture significant changes in an organization's security posture. Events can form the basis for threat detection, response, and analysis. One embodiment of the present invention may use continuous event monitoring to detect security threats and provide critical information for informed and rapid decision-making. Events can be important because they provide real-time, accurate, and actionable security information. One embodiment of the present invention may aggregate and analyze events to provide a comprehensive security view and enable users to respond quickly to threats. Essentially, events play a crucial role in monitoring and response processes, ensuring the protection of critical assets by providing the information needed to detect, assess, and respond to security incidents.

[0084] An incident can refer to a security event that has a significant impact on an organization and requires further response and recovery work. In addition, incidents can be generated by automated triggers of alert rules, or they can be initiated manually after an alert has been investigated. For example, an incident may include a severity and confidence level assessment, the reporting date, the incident status, the team members involved, the reason the incident occurred, the assets involved, recommended next steps, and an activity log.

[0085] Incident details can provide users with detailed information about a specific incident. This feature allows users to access comprehensive information about an incident, including a description of the incident, related activities, and recommended next steps to resolve the issue. This level of detail can be useful in helping users understand the full context of the incident, enabling them to make informed decisions and take appropriate actions to address the problem.

[0086] An alert may represent a technical notification regarding current security risks, such as vulnerabilities and their exploitation. An alert may provide an early warning signal indicating potential changes or activities that may require further investigation and attention. An alert may include a priority and confidence assessment, detection date, alert status, relevant team members, reason the alert was triggered, relevant assets, recommended next steps, and activity logs.

[0087] In addition, alerts can represent notifications generated by the core platform in response to specific security incidents or situations. Alerts are considered important because they provide real-time visibility into potential security risks, enabling users to take proactive measures to address them before risks escalate. Alerts can be triggered by various factors, such as changes in system activity or user behavior. The purpose of alerts may be to keep users informed of the latest security events and provide the information necessary to respond effectively. Furthermore, alerts can minimize the impact of security incidents and improve the organization's overall security posture by providing security teams with timely and relevant information.

[0088] Security findings may represent gaps or vulnerabilities in information systems, security procedures, controls, and / or implementations that could be exploited or activated by potential threats. These are primarily identified using static analysis methods. Security findings may include a security assessment, the status of the findings, a description, recommended next steps, and a list of affected assets.

[0089] Security investigation details provide users with detailed information about specific security investigation findings. This feature allows users to access comprehensive information about security investigation findings, including a description of the findings, related activities, and recommended next steps to resolve the issue. This level of detail can be useful in understanding the full context of the security investigation findings and enabling users to make informed decisions and take appropriate actions to address the issue. Security investigation details may include severity (e.g., the potential risks of the security investigation findings if not addressed), status (e.g., current activity status of the incident, such as paused and running), description (e.g., why the incident was created), recommended next steps (e.g., corrective steps), and assets associated with the security incident. For example, a clear understanding of the affected assets allows users to take appropriate actions to fix the problem, such as applying patches or updates, reconfiguring assets, or implementing new security measures. This information can also be used to track the progress of the resolution work and to ensure that affected assets are protected in a timely manner.

[0090] An exemplary monitoring workflow might begin with an event triggered by a change in the source system, initiating a series of monitoring phases to assess the situation and determine any further updates or actions that need to be taken. This could include creating new alerts or incidents and resulting in a recalculation of security scores for all affected assets and pipelines.

[0091] The service bus 340 may include various queues such as a report queue 341, an activity queue 342, an inventory queue 343, and a monitoring queue 344. Cache 350 may include inventory cache 352, report cache 354, and monitoring cache 356.

[0092] SQL DB360 may include the global database 362 and one or more tenant databases 364. The collector 370 can interface with various AI services 319 via connectors such as the Azure ML connector 371, the Databricks connector 372, and the ML flow connectors 373, 374, and 375. The browser 302 can communicate with the scheduling servicer 380, which interfaces with the asset discovery engine 382 and the analytics engine 384 (supporting the analytics worker 386), via the management UI 306. The collector 370 may also include a cache 390 that supports the job queue 392 and a NoSQL database 394 that supports the asset database 396 and the analytics database 398.

[0093] Collector 370 can establish a WebSocket connection with the security center and retrieve information since the last execution (e.g., pipeline / asset mapping). For example, the collector can start the log analysis engine and access logs since the last execution. In addition, the collector can start the asset discovery engine and check for assets that have changed since the last execution. The collector can add log and asset information to the local data store. Furthermore, the collector can start the adversarial analysis engine and check for matches in the newly added data. The collector can then push the information (e.g., log events, new assets, analysis results, etc.) to the core platform 310.

[0094] The asset discovery engine 382 can retrieve asset details about assets within the asset hierarchy. These asset details may include the asset identifier, asset name, asset type, sibling relationships, parent relationships, etc.

[0095] The analysis engine 384 can perform an event analysis process for new events. Event analysis can be performed on an asset or a pipeline containing related assets. The analysis can be segmented based on event criteria and related assets so that only relevant analyses can be performed for each event. Event criteria may include event type (e.g., asset event, pipeline event), related asset information (e.g., asset type, whether the asset is part of a pipeline), pipeline information (e.g., dataset, model, experiment, deployment), and context of the related asset in relation to the pipeline (e.g., dataset, model). The analysis engine 384 can be executed in order of priority so that previous analysis results are available for later analysis. Furthermore, the analysis engine 384 can receive events as input and load values ​​as needed.

[0096] The analysis engine 384 can perform evaluation and detection functions. For example, an alert rule may be executed once for each pipeline containing the relevant assets. Evaluation may be performed based on criteria specified in each alert rule so that the relevant rule may be executed for each event. The alert criteria may include the type of event, asset events, and relevant pipeline information. Evaluation criteria for each type of asset or event may be performed for each asset / user in each matching pipeline. Collector evaluation may receive events as input, along with the identifier of the specific asset to be evaluated and the definition of evaluation criteria for the relevant evaluation items. The collector may load values ​​as needed. Detection functions may be performed in the context of the pipeline in which the alert was generated. When a new alert or incident is generated, scores may be recalculated for all assets included in the evaluation, all pipelines containing those assets, and the overall security score. Alert rules may be evaluated based on events occurring within the collector.

[0097] Figure 3 can support various implementation forms, applications, and use cases. For example, use cases may include data poisoning, configuration vulnerabilities, and inference attacks. Data poisoning may involve the tampering of public or internal datasets. Data poisoning can be more impactful if performed on raw datasets, as it can affect downstream processes. Configuration vulnerabilities may include vulnerabilities that allow unauthorized access to a company's intellectual property (IP), and these vulnerabilities may resemble traditional cyberattacks. Inference attacks may include situations where a malicious actor strategically accesses an application to extract or replicate models, or anomalous activity on the production side of an AI pipeline.

[0098] Figure 4 shows the collector architecture according to one embodiment of the present invention. The security center 410 can receive data from various sources and systems, such as those represented by customer environments 412. Customer environments 412 may support multiple machine learning services, machine learning platforms, etc. In this example, collector 420 may interface with cloud services such as Azure ML 422 for accelerating and managing the lifecycle of machine learning projects, collector 430 may interface with cloud machine learning platforms such as SageMaker 432, collector 440 may interface with machine learning platforms such as Vertex AI 442, and collector 450 may interface with custom ML assets 452 in a custom / on-premises environment. Other external and internal systems may be supported across various different platforms, implementations, and / or architectures.

[0099] Figure 5 illustrates the asset discovery process according to one embodiment of the present invention. Figure 5 illustrates the interaction between the asset discovery engine 510, the ML platform connector 512, the database 514, and the ML platform 516. The asset discovery engine 510 initiates the job acquisition process, and the ML platform connector 512 provides a response. The asset discovery engine 510 may then execute one or more initialization jobs until no more assets / events are discovered. The ML platform connector 512 may then acquire one or more assets related to the job from the ML platform 516. The asset discovery engine 510 may then store the assets in the database 514. The asset discovery engine 510 may then execute event jobs through the ML platform connector 512.

[0100] Figure 6 shows an exemplary collector data architecture according to one embodiment of the present invention. Figure 6 illustrates the interaction between the collector 602, the service 604, and the machine learning system 606. The collector 602 may include an asset discovery engine instance 620, an ML database 630, a data structure store 640, a Python service 650, and an integration service 660. The service 604 may include an API management (APIM) 670, configuration 672, pipeline 674, and management 676. The APIM 670 enables users to expose, secure, transform, maintain, and monitor multiple APIs. The APIM 670 may enable organizations to create a complete API program by using existing backend services to create a consistent and modern API gateway.

[0101] The asset discovery engine instance 620 may include a connector 610 that can support a webhook (API endpoint) 612, an initialization / scheduling job 614, and an event job 616. The asset discovery engine instance 620 may also include a persistence layer 618 that interfaces with the ML database 630. The ML database 630 may store and manage data related to ML assets 632 and analysis results 634. The Python service 650 may read from and write to the ML database 630. The Python service 650 may include a runtime environment 652 and an executor 654. The data structure store 640 may perform actions on the event job 616, the Python service 650, and the integration engine 660, such as retrieving and sending events to the event queue.

[0102] The integration engine 660 may support a synchronization asset 662, a script command receiver 664, and an event subscription receiver 668. The integration engine 660 may interact with service 604 via APIM 670. Service 604 may include configuration 672, pipeline 674, and management 676.

[0103] As shown in Figure 6, the asset discovery engine instance 620 can collect resource data from the machine learning system 606 in several ways. The asset discovery engine instance 620 can retrieve asset data by searching for ML system resources through initialization jobs and scheduled jobs (represented by 614). API endpoints implemented as webhooks, as shown by 612, can be called by the machine learning system 606. To avoid overload, the data can be immediately stored in a queue (e.g., a Redis queue) within the data structure store 640. The data (ML assets 632) can be stored in the ML database 630.

[0104] The analysis engine may examine data from ML assets, as shown by 632. The analysis engine may collect data from the ML database 630, analyze the data, and store it, as shown by 632. The integration engine 660 may send analysis engine data to service 604 via a RESTful API and then listen to synchronize configurations from service 604 via a WebSocket. A Python runtime environment, as shown by 652, may represent a separate runtime Python environment for dynamically executing Python scripts, as shown by executor 654.

[0105] Figure 7 is an exemplary flowchart for adversarial detection according to one embodiment of the present invention. Figure 7 shows the process for detecting security findings, incidents, and / or alerts. In step 710, the execution of an event may occur in an external system. In step 712, event discovery may be performed on the collector agent. In step 714, event analysis may be performed on the collector agent. In step 716, event and asset metadata may be provided to the core platform. In step 718, the relevant pipeline context may be retrieved. In step 720, a request for statistical analysis of the asset and pipeline context may be made to the collector agent. In step 722, the analysis may be performed and provided to the core platform. In step 724, the retrieval of relevant evaluation rules may be performed. In step 726, the evaluation of the rules may be performed and provided on the core platform. In step 728, security findings and alerts may be generated. In step 730, the relevant incident detection rules may be retrieved. In step 732, a request for detection of the asset and pipeline context may be made to the collector agent. In step 734, the detection results may be executed and provided to the core platform. In step 736, incident creation and score recalculation may be performed. While the process in Figure 7 shows several steps executed in a specific order, it should be understood that embodiments of the present invention can be implemented by adding one or more steps to the process, omitting steps within the process, and / or changing the order in which one or more steps are executed.

[0106] Figures 8A and 8B are illustrative diagrams of a security center interface according to one embodiment of the present invention. Figures 8A and 8B provide a high-level overview of the overall health of an enterprise's AI pipeline, visibility into AI assets, and the status of notable AI events and incidents. One embodiment of the present invention determines whether the AI ​​pipeline is functioning properly, whether the AI ​​pipeline is at risk, and the downstream and business impact of such risk.

[0107] As shown in Figure 8A, the Security Center interface can provide details about the health of a specific AI system. Health can be assessed based on various metrics, including security score, model security score, data security score, all assets, pipeline, AI model, and dataset. Details of the organizational pipeline may also be provided. Figures 8A and 8B help users quickly identify areas of concern and better understand the impact of performance issues by providing an overview of key security metrics.

[0108] The security score can represent a measure of the pipeline's security posture, based on an algorithm that incorporates incidents, alerts, and security investigation results. The model security score can calculate a security score (e.g., 0-100) across multiple models associated with this pipeline. The data security score can calculate a security score (e.g., 0-100) across multiple datasets associated with this pipeline. Total Assets represents the total number and breakdown of assets associated with this pipeline. The pipeline provides a visual representation of all assets within this pipeline and their individual security assessments.

[0109] According to one embodiment of the present invention, the security score may be based on the importance of the pipeline, with higher importance corresponding to greater impact from incidents, alerts, and security findings.

[0110] According to one embodiment of the present invention, security scores may be calculated for individual assets, a single pipeline, multiple pipelines, etc. For individual assets, the security score may take into account the impact of incidents, the impact of alerts, and / or the impact of security findings.

[0111] Regarding the impact of an incident, the impact score for each individual incident can be calculated by taking the values ​​of a scoring matrix corresponding to the severity and confidence levels of the incident. The overall incident impact score for an asset can be calculated as the maximum of the impact scores of all individual incidents. For example, if an asset has two incidents, one with moderate severity and high confidence, and the other with high severity and moderate confidence, the overall incident impact score for the asset would be 0.9. An asset with no incidents would have a score of 0.

[0112] The impact of alerts can be calculated by taking the impact score of each individual alert from a scoring matrix corresponding to the alert's priority and confidence level. The overall alert impact score for an asset is calculated as the maximum of the impact scores of all individual alerts. For example, if an asset has two alerts, one with a medium priority and high confidence level, and the other with a high priority and medium confidence level, the overall alert impact score for the asset would be 0.9. An asset with no alerts would have a score of 0.

[0113] The impact of security findings can be calculated by taking the impact score of each individual security finding from a scoring table corresponding to the severity rating of the security finding. The overall impact score of security findings for an asset is calculated as the maximum of the impact scores of all individual security findings. For example, if an asset has two security findings, one of moderate severity and the other of high severity, the overall impact score of security findings for that asset will be 1.0. Assets with no security findings will have a score of 0.

[0114] For a single pipeline, the security score can be calculated by taking the weighted average of each asset associated with that pipeline. For multiple pipelines, the security score can be calculated by taking a weighted average of each pipeline. If an asset exists in multiple pipelines, the asset may have a unique score for each pipeline to which it belongs. For example, if an asset A1 exists in two pipelines, P1 and P2, and P1 has a higher severity level than P2, then incidents and alerts related to A1 in pipeline P1 may differ from those in pipeline P2.

[0115] As shown in Figure 8B, the Security Center interface can provide data on recommendations regarding isolated assets and resilience. Security trends provide a graphic (e.g., a line graph) of the number of notable events and incidents over a given period. The risk breakdown of notable events can be divided into high, medium, and low urgency. The risk breakdown of incidents can also be divided into high, medium, and low urgency. The incident list represents a graphic (e.g., a table) displaying the top number of incidents sorted by urgency and the number of assets.

[0116] Figure 9 is an illustrative diagram of a pipeline manager interface according to one embodiment of the present invention. As shown in Figure 9, new pipelines can be added. Details of the organization's pipelines can be displayed graphically. Other data may be provided, including importance, pipeline name / identifier, assets, security score, latest activity, status, etc.

[0117] Figure 10 is an illustrative diagram of an entity details interface according to one embodiment of the present invention. Entity details may include security, assets, and configuration. Security details may include security scores, model security scores, data security scores, and AI assets. Pipeline details may be displayed graphically.

[0118] Figure 11 is an illustrative diagram of a new pipeline creation interface according to one embodiment of the present invention. Pipeline attributes may include project and business importance, associated datasets and models, cloud and other integrations, and team members with varying access rights. Various users may be categorized as personas, which may include data scientists, data science managers, AI engineers, AI development directors, CISOs, etc. As shown in Figure 11, users may provide pipeline name / identifier, description, and other business data such as business importance.

[0119] Figure 12 is an exemplary diagram of a noteworthy events interface according to one embodiment of the present invention. Figure 12 provides an organized view for better understanding noteworthy events. A noteworthy event can represent any event in the AI ​​pipeline that has a non-zero probability of being malicious activity. Noteworthy events can be categorized by urgency, type, severity, and confidence level of malicious actor involvement. Figure 12 shows noteworthy events, affected assets, high urgency, medium urgency, and low urgency. Other variations may apply to the urgency thresholds. Filters such as all pipelines, last 7 days, and active events may be available. Activity details may include urgency level, pipeline, event type, target of the event, severity, confidence level, and date / time.

[0120] Figure 13 is an illustrative diagram of a suspicious activity interface according to one embodiment of the present invention. Details may include the affected pipeline, urgency, status, severity level, confidence score, team, type of suspicious activity, description, affected assets, and recommended next steps. Activity log details may also be provided.

[0121] Figure 14 is an illustrative diagram of a business details interface according to one embodiment of the present invention. Business details may include importance, industry, business function, and project. Figure 15 is an illustrative diagram of a cloud asset interface according to one embodiment of the present invention. Cloud assets may include details about pipelines, models, and datasets.

[0122] Those skilled in the art will understand that the various embodiments described herein have broad utility and applications. Therefore, while various embodiments are described in detail herein in relation to exemplary embodiments, it should be understood that this disclosure is illustrative and exemplary of various embodiments and is made to provide a practical disclosure. Accordingly, this disclosure is not intended to be construed as limiting embodiments or excluding other such embodiments, adaptations, variations, modifications, and equivalent configurations.

[0123] The above description provides examples of different configurations and features of embodiments of the present invention. While specific nomenclature and application / hardware types are described, other names and applications / hardware can also be used, and the nomenclature is provided only as a non-limiting example. Furthermore, while specific embodiments are described, it should be understood that the features and functions of each embodiment can be combined in any combination, within the capabilities of those skilled in the art. The drawings provide additional illustrative details relating to various embodiments.

[0124] Various exemplary methods are provided herein by example. The methods described may be carried out by one or a combination of various systems and modules, or by other means.

[0125] The term "computer system" in this disclosure may refer to a single computer or a group of computers. In various embodiments, a group of computers may be networked. The networking may be any type of network, including, but is not limited to, wired networks and wireless networks, local area networks, wide area networks, and the internet.

[0126] According to exemplary embodiments, system software may be implemented as one or more modules of computer program instructions encoded on a computer-readable medium for execution by one or more computer program products, such as data processing devices, or for controlling the operation of data processing devices. The implementation may include single or distributed processing of algorithms. The computer-readable medium may be a machine-readable store-time device, a machine-readable storage board, a memory device, or one or more combinations thereof. The term “processor” encompasses all devices, machines, and equipment for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, equipment may include software code that creates an execution environment for the computer program, such as processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof.

[0127] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed as standalone programs or in any form, including modules, components, subroutines, or other units suitable for use in a computing environment. A program can be stored in part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple collaborative files (e.g., a file containing one or more modules, subprograms, or parts of code). A computer program can be deployed for execution on one computer, located in one site, or distributed across multiple sites and interconnected by a communication network.

[0128] A computer can encompass all devices, machines, and equipment for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, a computer can include code that creates an execution environment for the computer program, such as processor firmware, a protocol stack, a database management system, an operating system, or code that constitutes one or more of these.

[0129] The processes and logic flows described herein may be executed by one or more programmable processors that execute one or more computer programs to perform a function by acting on input data and producing outputs. The processes and logic flows may also be carried out by dedicated logic circuits, such as FPGAs (Field Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the devices may also be implemented as dedicated logic circuits, such as FPGAs (Field Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits).

[0130] Computer-readable media suitable for storing computer program instructions and data may include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be complemented by or incorporated into dedicated logic circuits.

[0131] While the embodiments are specifically shown and described within the framework for conducting the analysis, it will be understood that they can be modified and altered by those skilled in the art without departing from the scope of various embodiments. Furthermore, those skilled in the art will recognize that such processes and systems do not need to be limited to the specific embodiments described herein. Other embodiments, combinations of these embodiments, and their uses and advantages will become apparent to those skilled in the art from the discussion herein and the practice of the embodiments disclosed herein. This specification and the examples should be considered illustrative.

Claims

1. A system for implementing an artificial intelligence (AI) security platform, A security center platform that functions as a centralized software-as-a-service (SaaS) portal and collects and aggregates telemetry data across one or more AI implementations, An interactive user interface configured to track one or more AI implementations and provide an analytical dashboard for identifying one or more adversarial security issues, The computer server includes a computer processor coupled to the interactive user interface and the security center platform, and the computer server is A data integrity module configured to provide data security and integrity in one or more AI implementations, and to further provide adversarial protection, A platform configuration monitoring interface configured to provide configuration monitoring related to one or more AI implementations, A system further comprising a model security module configured to provide insights into how one or more models function from a security perspective, and to further provide model awareness and anomalous model behavior analysis.

2. The security center platform includes an API gateway, a container orchestration platform, and a core security platform comprising a computer processor that communicates with at least one memory component. The interactive user interface includes an interface for communicating with one or more collector agents that receive data from one or more AI services. The aforementioned computer server, An asset discovery engine configured to collect metadata relating to one or more assets used for the development and delivery of at least one AI pipeline, wherein the metadata relates to the asset type, activity, data source, and model, and the asset discovery engine The system according to claim 1, comprising: an analysis engine configured to identify potential threats from adversarial attacks by monitoring one or more AI services, one or more assets, and the at least one AI pipeline and generating at least one security score for evaluating the security posture of an entity, wherein the at least one security score is based on one or more of security incidents, alerts, and security findings.

3. The system according to claim 2, wherein one or more collector agents operate in the customer environment.

4. The system according to claim 2, wherein the at least one memory component includes at least one service bus, at least one cache, and at least one database.

5. The system according to claim 2, wherein the at least one security score includes a model security score and a data security score.

6. The system according to claim 2, wherein the interface is configured to display a graphical representation of the organizational pipeline.

7. The system according to claim 2, wherein the interface is configured to display a graphical representation of one or more notable events and incidents.

8. The system according to claim 2, wherein the interface is configured to display one or more recommendations regarding resilience.

9. The container orchestration platform comprises an operations management module, a reporting module, an inventory management module, and an inventory monitoring module, according to claim 2.

10. The system according to claim 2, wherein the at least one security score is calculated by individual assets, individual pipelines, or multiple pipelines.

11. A method for implementing an artificial intelligence (hereinafter referred to as AI) security platform using the system described in claim 1, The steps include communicating with one or more collector agents that receive data from one or more AI services through an interface included in the aforementioned interactive user interface, A step of collecting metadata relating to one or more assets used for the development and distribution of at least one AI pipeline, using an asset discovery engine included in the computer server, wherein the metadata relates to the asset type, activity, data source, and model, The steps include: using an analysis engine included in the computer server to monitor one or more AI services, one or more assets, and at least one AI pipeline to identify potential threats from adversarial attacks; A step of generating at least one security score that evaluates the security posture of an entity using the analysis engine, wherein the at least one security score is based on one or more of security incidents, alerts, and security investigation results, A method comprising the step of providing security health data representing the security posture of an entity through an interface included in the interactive user interface, wherein the security health data includes at least one security score.

12. The method according to claim 11, wherein one or more collector agents operate in the customer environment.

13. The method according to claim 11, wherein the security center platform includes at least one memory component, the at least one memory component includes at least one service bus, at least one cache, and at least one database.

14. The method according to claim 11, wherein the at least one security score includes a model security score and a data security score.

15. The method according to claim 11, wherein the interface is configured to display a graphical representation of the organizational pipeline.

16. The method according to claim 11, wherein the interface is configured to display a graphical representation of one or more events and incidents of note.

17. The method according to claim 11, wherein the interface is configured to display one or more recommendations regarding resilience.

18. The method according to claim 11, wherein the at least one security score is calculated by individual assets, individual pipelines, or multiple pipelines.

19. The method according to claim 11, wherein the one or more collector agents communicate with the asset discovery engine and the analysis engine, and further communicate with a core platform comprising a computer processor that communicates with an API gateway, a container orchestration platform, and at least one memory component.

Images