Generative artificial intelligence security engine in item list system

Abstract

Methods, systems, and computer storage media for providing generative artificial intelligence (AI) security management using a generative AI security engine in an item listing system. The generative AI security engine supports generative AI security management that is based on security analysis and detection operations for a plurality of generative AI-supporting applications and generative AI models. In operation, a request associated with hint data is transmitted from a generative AI client. Based on transmitting the request, a response generated based on the edited version of the hint data is received at the generative AI client. The hint data is analyzed using a plurality of security engine operations to cause an edited version of the hint data to be generated. The response is generated at the generative AI model using the edited version of the hint data. The response is caused to be generated at an interface associated with the generative AI client.

Description

Technical Field

[0001] This application relates to a generative artificial intelligence safety engine in a project list system. Background Art

[0002] Users can interact with generative AI technologies in different types of applications and services to complete computing tasks. Generative AI refers to a category of AI systems and algorithms that are designed to generate new data or content that is similar to or in some cases completely different from the data on which they were trained. Generative AI systems can create support for text generation, image generation, music and audio generation, video generation, and data synthesis. Specifically, generative AI systems can support project list systems in several ways to improve operational efficiency, customer engagement, and online shopping. For example, project list systems can employ generative AI systems for content generation (e.g., product descriptions), personalized shopping experiences (e.g., recommendation engines), product discovery (e.g., visual search), and virtual assistants (e.g., chatbots). Project list systems can leverage generative AI to enhance project list system functionality through application programming interfaces (APIs), pre-trained models, and custom AI solutions. Summary of the invention

[0003] Aspects of the technology described herein generally relate to systems, methods, and computer storage media for using a generative AI security engine in an item list system to provide generative AI security management, etc. The generative AI security engine supports generative AI security management based on security analysis and detection operations of generative AI-enabled applications ("generative AI applications") associated with generative AI models (e.g., large language models "LLMs") and prompt interfaces. Specifically, the generative AI security engine provides generative AI security engine operations ("security engine operations"), including intent detection, prompt attack detection, restricted data detection and editing, and prompt context and editing, which are used to protect generative AI applications from potential data privacy and data leakage issues.

[0004] In operation, prompt data from a generative AI client is accessed, the prompt data being associated with a request to a generative AI model supporting an artificial intelligence system. The prompt data is analyzed based on a pre-processing security engine operation, the pre-processing security engine operation supports determining how to transmit the prompt data associated with the request to the generative AI model or determining to block the request. Based on analyzing the prompt data, an edited version of the prompt data is generated for the generative AI model, the edited version of the prompt data including an edited data tag associated with an edited portion of the prompt data. The edited version of the prompt data is transmitted to the generative AI model. A response from the generative AI model is accessed and analyzed based on a post-processing security engine operation, the post-processing security engine operation supports determining how to transmit the response to the generative AI client or determining to block the response. Based on analyzing the response, the response is transmitted to the generative AI client or the response to the request is blocked.

[0005] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be an aid in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The technology described herein is described in detail below with reference to the accompanying drawings, in which:

[0007] Figure 1A and Figure 1B is a block diagram of an artificial intelligence system for providing generative AI safety management in an item list system according to aspects of the technology described herein;

[0008] Figure 1C to Figure 1F is a schematic diagram of an artificial intelligence system for providing generative AI safety management in an item list system according to aspects of the technology described herein;

[0009] Figure 2A is a block diagram of an artificial intelligence system for providing generative AI safety management in an item list system according to aspects of the technology described herein;

[0010] Figure 2B is a block diagram of an artificial intelligence system for providing generative AI safety management in an item list system according to aspects of the technology described herein;

[0011] Figure 3 A first exemplary method of providing generative AI safety management in an item list system according to aspects of the technology described herein is provided;

[0012] Figure 4A second exemplary method of providing generative AI safety management in an item list system according to aspects of the technology described herein is provided;

[0013] Figure 5 A third exemplary method of providing generative AI safety management in an item list system according to aspects of the technology described herein is provided;

[0014] Figure 6 A block diagram of an exemplary item list system computing environment suitable for implementing aspects of the techniques described herein is provided;

[0015] Figure 7 A block diagram of an exemplary distributed computing environment suitable for implementing aspects of the techniques described herein is provided; and

[0016] Figure 8 is a block diagram of an exemplary computing environment suitable for implementing aspects of the techniques described herein. DETAILED DESCRIPTION

[0017] Overview

[0018] The project listing system and platform support storing projects (products or assets) in a project database and providing a search system for receiving queries and identifying search result projects based on the queries. Projects (e.g., physical projects or digital projects) refer to products or assets provided to be listed on the project listing platform. The search system supports identifying result projects from the project database for the received query. The project database can be specifically used for a content platform or a project listing platform, such as the EBAY content platform developed by EBAY INC of San Jose, California. The project listing system can also provide applications that support generative AI ("generative AI applications"), which use generative AI models (e.g., large language models "LLM") to create, generate or produce content, data or output. LLM is a specific category of generative AI models that focuses on generating human-like text. Generative AI models (such as GPT (Generative Pre-trained Transformer) and its variants) are designed to generate human-like text or other types of data based on the input they receive (e.g., via a prompt interface). These applications use generative AI to perform various tasks across different fields to provide improvements in automation, efficiency, and human-like interaction.

[0019] Traditionally, project list systems are not configured with comprehensive logic and infrastructure to effectively provide secure management of generative artificial intelligence (AI) for project list systems. The use of generative AI models in project list systems may raise data privacy issues and may cause governance and data leakage issues via insecure prompt interfaces. For example, text prompts or inputs (via prompt interfaces that support interaction with generative AI models) (which can generate responses based on understanding of language and context) may be associated with misinformation and false content, harmful or offensive content, automated attacks, and spam and abuse.

[0020] Additionally, prompt interface security can be difficult to implement. For example, prompt injection can be a security threat from a prompt interface. A prompt injection attack can be performed on an LLM prompt interface, where prompt injection includes manipulating the LLM based on injecting information that deviates from its intended purpose. Adversaries can exploit this type of vulnerability to bypass security measures or filters and gain access to sensitive data. Another concern about generative AI models is the data that is loaded into the LLM for training or fine-tuning. If the content used in training is malicious or contains sensitive information, the LLM may disclose that content through a response, which could cause privacy violations or brand damage. Due to the threat of data leakage through generative AI models, different types of users or customers of generative AI models may be reluctant to share confidential intellectual property if they are at risk of data leakage.

[0021] Implementing only conventional security infrastructure (without a generative AI security engine) results in an inadequate bullet list system. For example, conventional security infrastructure lacks new threat signatures because conventional security measures rely on known threat features, patterns, or known malicious entities. In addition, AI models generate contextual and human-like text responses, which makes it challenging to predefine threat signatures. Other limitations of conventional security infrastructure include the dynamic and contextual dependence of threats, the complexity of natural language, and irresponsible use of AI. Therefore, a more comprehensive bullet list system (with an alternative foundation for performing bullet list system security operations) can improve computational operations and interfaces to provide generative AI security management with prompt interface security.

[0022] Embodiments of the present invention relate to systems, methods, and computer storage media for providing generative AI security management, etc., using a generative AI security engine in an item list system. The generative AI security engine supports generative AI security management, which is based on security analysis and detection of generative AI-enabled applications ("generative AI applications") associated with generative AI models (e.g., large language models "LLM") and prompt interfaces. Specifically, the generative AI security engine provides security engine operations, including intent detection, prompt attack detection, restricted data detection and editing, and prompt context and editing, which are used to protect generative AI applications from potential data privacy and data leakage issues. Generative AI security management is provided using a generative AI security engine that is operationally integrated into an item list system associated with an artificial intelligence security system. The artificial intelligence security system supports a generative AI security engine framework of computing components associated with security engine operations (e.g., pre-processing security engine operations, post-processing security engine operations, and training data set security engine operations) to provide generative AI security management.

[0023] At a high level, a generative AI security engine may be provided as a security framework to support the secure adoption of generative AI models (e.g., LLMs) in artificial intelligence systems. The artificial intelligence system may specifically be an implementation in a project list system. The generative AI security engine may be an LLM security solution that provides the following operations: security engine operations associated with sensitive data leakage detection and prevention; sensitive data redaction for implementing LLM context about redacted data while ensuring the safe use of the LLM; detection of LLM abuse or malicious intent (e.g., prompt attacks); detection of uses that are deemed illegal, unethical, or in violation of corporate policy; and prompt logging and searching (e.g., after the fact). The generative AI security engine may implement functional components that provide operations for, for example, pre-processing security engine operations, post-processing security engine operations, and training data set security engine operations associated with the following operations: intent detection, prompt attack detection, restricted data detection and redaction, and prompt context and redaction.

[0024] Intent detection can include utilizing an intent detection security model (e.g., a neural network machine learning model) and embeddings to classify input text (e.g., prompt data for a request) into security categories, where a training data set already has similar security categories. The input text can be classified into one or more of the following categories: suggesting an attack, malicious intent, violent intent, insulting or defamatory, illegal intent, hateful intent, non-English characters, etc. Based on the classification of the input text, the generative AI security engine can also be configured to take actions, including blocking requests associated with the input text from being transmitted to the generative AI model.

[0025] Hint attack detection can include determining whether the intent of the input text is to manipulate or extract information from the LLM. Hint attack security models (e.g., neural network machine learning models) can be used to support classification of hint data for specific types of hint attacks. Hint data can be classified into different types of hint attacks, including, for example, the following: toxic or offensive content, bias amplification, political manipulation, extremist propaganda, hate speech generation, privacy violation, malware generation, spam or phishing, service abuse, or unexpected results.

[0026] Restricted data detection and redaction may include using a regular expression engine to detect and employing an redaction and replacement engine to replace any hits (i.e., prompt data instances identified as suspicious prompt data instances) with placeholders that clearly explain why the data was redacted. The regular expression engine may be associated with a false positive reduction model with pre-processing and post-processing rules to help reduce false positives for suspicious prompt data instances. A restricted data detection model (e.g., NLP or LLM) and redaction and replacement engine may also be implemented to support detection of sensitive data in prompt data associated with input text, redaction of the sensitive data, and replacement of the redacted sensitive data with a redacted data label.

[0027] Prompt context and editing can include providing placeholders (i.e., edit data tags) for data types that are predefined to be edited. Editing can be performed via the edit and replace engine when the prompt context is determined. For example, for a social security number in a particular prompt context, an edit data tag can be replaced as shown below:<SSN_placeholder_1> The Generative AI model can still process requests that are associated with an edited version of the prompt data that is associated with the Generative AI Client’s input text. If the Generative AI model returns a placeholder (e.g.<SSN_placeholder_1> ), the Generative AI Security Engine can reinsert the original text in the placeholder’s place, thus providing the user with the original data.

[0028] The generative AI security engine can be used to ensure the security of a training data set associated with a machine learning training pipeline used to train a generative AI machine learning model. Specifically, a training data set (e.g., a data instance of the training data set) is subjected to a training data set security engine operation before the training data set is used to train the generative AI model. The training data security engine operation can be performed to ensure that the generative AI model being trained does not include any suspicious data or sensitive data that may lead to data leakage or other generative AI security vulnerabilities.

[0029] It is contemplated that the generative AI security engine may be implemented in different types of artificial intelligence systems associated with different types of operating environments. Implementing the generative AI security engine in an artificial intelligence system associated with an item list system is exemplary and is not meant to limit other variations and combinations of implementing the generative AI security engine in other types of systems.

[0030] Advantageously, embodiments of the present technical solution support the use of a generative AI security engine in a project list system to provide generative AI security management. The generative AI security engine supports generative AI security management based on security analysis and detection operations for multiple generative AI-enabled applications and generative AI models. Generative AI security engine operations provide solutions to problems in generative AI security (e.g., prompting interface security vulnerabilities, data leakage, and data privacy). Generative AI security engine components, infrastructure, and ordered combinations are improvements over conventional security systems that lack support for generative AI security threats and attacks.

[0031] You can refer to the examples and Figures 1A to 1F To describe various aspects of the technical solution. Figure 1A The project list system 100 is shown, which includes an artificial intelligence system 100A, a network 100B, a generative artificial intelligence security engine 110, a generative AI application 120, a generative AI application client 130 and a machine learning engine 140. The project list system 100 corresponds to the following reference Figure 6 An item list system 600 is described.

[0032] The item list system 100 provides a system (e.g., an artificial intelligence "AI" system 100A) including an engine (e.g., a generative AI security engine 110) for performing operations (e.g., security engine operations) discussed herein. The generative AI security engine 110 can be operated with a generative AI application client 130 (e.g., a client device), which can access the item list system 100 to perform tasks using a generative AI application 120 associated with a corresponding generative AI model (e.g., LLM 142). For example, a user (via the generative AI application client 130 (e.g., a prompt interface)) can transmit a request (e.g., a generative AI request with prompt data) to the generative AI application and the LLM to process the request. Based on transmitting the request, the generative AI security engine can use the security components of the generative AI security engine 110 to perform security engine operations (e.g., pre-processing security engine operations, post-processing security engine operations, and training data set security engine operations) to ensure that the request is processed securely.

[0033] refer to Figure 1B , Figure 1B An artificial intelligence system 100A is shown, which includes: a generative AI smart security engine 110, which has security engine operations 112 including intent detection 112A, prompt attack detection 112B, restricted data detection and editing 112C, and prompt context and editing 112D; a security engine interface 114; a generative AI security engine model 116; a generative AI application 120; a generative AI application client 130, having generative AI client interface data 132; a machine learning engine 140, having a generative AI model "LLM 142"); and a prompt data database 150, having prompt data 152 and data types 154.

[0034] Generative AI security engine 110 and generative AI application client provide interfaces (i.e., security engine interface 114 and generative AI application interface) and operations (i.e., security engine operations). Generative AI security engine 110 and generative AI application client 130 may operate in a server-client relationship to provide generative AI security management. For example, a user may transmit a request from generative AI application client 130 to perform a task via generative AI application 120 and LLM 142. Based on the request, generative AI security engine 110 may perform security engine operations 112 to ensure that the request is securely processed in artificial intelligence system 100A.

[0035] The generative AI security engine 110 may perform security engine operations 112 based on the prompt data associated with the request. The security engine operations 112 may include, for example, analyzing the prompt data to determine an intent associated with the prompt data (e.g., intent detection 112); analyzing the prompt data to determine whether the prompt data is associated with a prompt attack (e.g., prompt attack detection 112B); analyzing the prompt data to identify restricted data types and edit restricted data types (e.g., restricted data detection and editing); and determining a prompt context, and editing a data type in the prompt data based on the context (e.g., prompt context and editing).

[0036] The security engine interface 114 may support communication between the generative AI security engine 110 and the generative AI client application 130, the machine learning engine 140, the LLM 142, and the hint data database 150. For example, the generative AI security engine 110 may receive hint data from or transmit hint data to the generative AI application 120. The generative AI security engine 110 may transmit hint data to the LLM 142 and receive responses to requests from the LLM 142. The generative AI security engine 120 may access data types (e.g., data types 154) and historical hint data (e.g., hint data 152) from the hint data database 150.

[0037] The generative AI security engine 110 may transmit a response to a request from the generative AI application 130. The response may be a notification that the request has been blocked, the response may be a response to the request generated based on an edited version of the request; or a response generated via the LLM 142 without editing the request. The generative AI application client 130 may receive any type of response from the generative AI security engine 110 and cause different types of responses to be displayed on a graphical user interface (GUI) associated with the generative AI application. For example, the generative AI application client interface data 132 may include different types of responses and additional GUI interface elements associated with the generative AI application client.

[0038] The generative AI security engine 110 is responsible for providing security engine operations 112 (e.g., pre-processing security engine operations, post-processing security engine operations, and training data set security engine operations) in the artificial intelligence system 100A. The security engine operations 112 protect the infrastructure, data, and assets of the project list system 100 against threats, vulnerabilities, and events. Specifically, the security engine operations guard against threats, vulnerabilities, and threats related to the implementation of generative AI technology via generative AI models and generative AI applications. The security engine operations 112 may be associated with intent detection 112A, prompt attack detection 112B, restricted data detection and editing 112C, and prompt context and editing 112D. It is envisioned that the security engine operations can be classified into three groups: pre-processing security engine operations, post-processing security engine operations, and training data set security engine operations. The pre-processing security engine operations may be operations before at least a portion of the prompt data is transmitted to the generative AI model; the post-processing security engine operations may be operations after the prompt data has been processed at the generative AI model; and the training data set security engine operations may be operations for machine learning training pipelines. The machine learning training pipeline can also optionally be associated with pre-processing safety engine operations and post-processing safety engine operations.

[0039] The security engine operations 112 may be illustrated by reference to different scenarios, including: prompting data retrieval; pipeline threat processing; security resource version checking; prohibited intent determination; regular expression evaluation; security context matching; reducing model-based false positives; evaluating model-based sensitive data; data type edit review; edit and replace execution; generative AI processing; edit data label review; replace edit data labels; prohibited terms in response determination; response communication; request blocking; excluding data instances from training data sets; and approving data instances from training data sets.

[0040] Prompt data retrieval may include retrieving and processing input and additional information from an interface (e.g., a prompt interface) associated with an application (e.g., a generative AI application client 130) and a generative AI model (e.g., LLM 142). Prompt data retrieval may include processing text input from a text box, voice input, or an API request. Prompt data may include a wide range of content and may be formulated as a question, statement, or command. Prompt data may be associated with contextual parameters that may guide the generation of the response (e.g., desired response language, style preferences, content restrictions, etc.). Prompts may also include special tokens (e.g., tags used to indicate a generative AI model in the prompt).

[0041] Pipelining threat processing may include optimizing and simplifying the processing of tip data. Pipelining threat processing may be implemented to ensure efficient and effective tip data evaluation. A database (e.g., tip data database 150) may be maintained to include tip data (e.g., tip data 152) and similar variants that have been previously processed. The tip data and variants may be associated with descriptions, attributes, exploited vulnerabilities, compromise indicators, and other relevant information for threat intelligence and incident response.

[0042] The subsequent prompt data instance is compared with the prompt data instance in the database (i.e., prompt data 152) to determine whether the subsequent prompt data instance is already in the database (i.e., a similar prompt data instance has been successfully processed before without security issues). Different types of matching techniques (e.g., exact matching, fuzzy matching, tokenization, semantic matching) can be used to determine whether the subsequent prompt data has a match in the database. Through pipeline threat processing, a first determination is made as to whether the prompt data instance in the database matches the subsequent prompt data instance. If it is identified as a match, a second determination is made as to whether there are any updates to the generative AI security engine model 116 or the data type 154 since the prompt data instance in the prompt database was processed. If there is already an update, the prompt data instance is processed for additional security operations; however, if there is no update yet, the prompt data instance bypasses the additional security operations. In this way, if the data type and the generative AI security engine model have not been updated, additional processing can be avoided.

[0043] The security resource version check may include determining whether a data type (e.g., data type 154) or a generative AI security engine model (e.g., generative AI security engine model 116) has been updated. The data types may include different sensitive data types that have been identified for security operations when identified in the prompt data. The data types may include personally identifiable information (PII); financial data health information (PHI); and authentication data legal and compliance data, education data, location data, biometric data, sensitive business data; social media data; and criminal records. The data type may also be associated with a prompt context and edit, where a data type is edited for a specific prompt context but may not be edited in other prompt contexts. The data type may be updated periodically to ensure that the prompt data is fully evaluated based on different data types.

[0044] Generative AI security engine models (e.g., generative AI security engine models 116) can be different types of AI security models that employ artificial intelligence and machine learning to enhance security measures and protect against threats. These models are strategically implemented via the generative AI security engine 110 to support security operations. Some example models include models for identifying sensitive data (e.g., natural language processing or deep learning); models for identifying false positives (e.g., anomaly detection models such as statistical models, clustering algorithms); and models for determining intent (e.g., intent classification models: recurrent neural networks (RNNs), convolutional neural networks (CNNs), and transformer models (e.g., Bidirectional Encoder Representations from Transformers (BERT)) and GPT (Generative Pre-Trained Transformers).

[0045] Prohibited intent determination may include evaluating prompt data to determine whether the prompt data includes prohibited intent (e.g., malicious intent or prohibited intent). Prompt data may be processed using a generative AI security engine model to determine whether the prompt data is associated with prohibited intent. A regular expression engine may be employed to perform regular expression evaluation. Regular expression evaluation may include pattern matching evaluation of text in prompt data. Regular expressions may be used to find and extract specific data patterns. Regular expressions may be implemented using various tools (e.g., regular expression libraries that provide advanced regular expression support). Security context matching may be performed to determine whether a hit regular expression matches a security threat context.

[0046] Model-based false positive reduction may include implementing a generative AI security engine model to evaluate prompt data and outputs from one or more previous pre-processing security operations to determine whether the prompt data is a true positive. Data type edit review may include evaluating whether a data type should be edited. For example, the prompt data is analyzed with respect to a list of data types that have been identified for editing. Edit and replace execution may include editing a portion of the data in the prompt data that has been identified as being to be edited, and replacing the data portion with an edit data label. It is contemplated that the prompt context may be evaluated so that a specific data type may be edited from the prompt data based on the prompt context. Generative AI processing may include transmitting the edited prompt data to the generative AI model to process the request.

[0047] Moving to edit data tag review; replacing edit data tags; disabled terms in response determination; transmitting response; and request blocking, these post-processing security engine operations can be performed after receiving a response from a generative AI model. Edit data tag review can include determining whether the response from the generative AI model includes any edit data tags. As discussed, the generative AI model can be configured to process prompt data using placeholders and generate a response with the placeholders, so that the placeholders can be replaced with edited portions of the prompt data before transmitting the response to the generative AI application client. The replace edit data tag operation can include replacing the edit data tag with an edited portion of the prompt data (e.g., original data).

[0048] Determining the prohibited terms in the response may include determining whether there are prohibited terms or responses in the response. The prohibited terms may be based on the data type 154 and additional terms that are identified as bad terms stored in the prompt data database 150. If there are no prohibited terms in the response from the generative AI model, the response may be transmitted (i.e., the response is transmitted), or if there are prohibited terms in the response, the request and response may be blocked from being processed and transmitted. It is contemplated that when a request is blocked, a response may be transmitted to the generative AI application client 130 indicating that the request has been blocked. The response may also include additional descriptive data and parameters associated with the reason why the request was blocked. The response and the additional descriptive data and parameters may be integrated with the generative AI client interface data including a graphical user interface element and displayed. The embodiments described herein contemplate other variations and combinations of transmitting notifications of blocked responses and transmitting additional descriptive data and parameters.

[0049] Turning to the training dataset security engine operations, the training dataset security operations may include selected pre-processing security engine operations associated with excluding and approving training datasets from the training dataset. These training dataset security engine operations may be implemented to process the training dataset before the training dataset is used for model training. The training dataset security engine operations may ensure that the training dataset does not include suspicious data and sensitive data that may cause generative AI security exposure. The machine learning training pipeline may selectively implement pre-processing security engine operations and generative AI security models. Training dataset security operations may also include editing data and adding synthetic data. Synthetic data refers to artificially generated data that mimics the characteristics of real data but does not contain information about actual individuals, objects, or events. It can be created using algorithms, statistical models, or other methods, and is typically used when real data is unavailable, insufficient, or sensitive. At least a portion of the training dataset may be approved for use in the training dataset or excluded from the training dataset.

[0050] refer to Figure 1C , Figure 1CA schematic diagram 100C is shown associated with using a generative AI security engine to provide generative AI security management according to an embodiment described herein. At box 102C, prompt data is received. At box 104C, a determination is made as to whether the prompt data is already stored in a prompt data database. If the prompt data is not already in the prompt data database, then at box 106C, a determination is made as to whether the prompt data has malicious or prohibited intent. If the prompt data is already in the prompt data database, then at box 108C, a determination is made as to whether the data type or model has been updated; and if so, at box 106C, a determination is made as to whether the prompt data has malicious or prohibited intent. If the prompt data is determined to have malicious or prohibited intent, then at box 110C, a request associated with the prompt data is blocked.

[0051] If the hint data does not have malicious or prohibited intent, then at box 112C, a determination is made as to whether there is a regular expression hit, and if so, at box 114C, a determination is made as to whether there is a context match hit; if so, at box 116C, a determination is made as to whether the model recognizes as a true example, and at box 118C, a determination is made as to whether the data type is approved for editing. If the data type is not approved for editing, then at box 110C, the request associated with the hint data is blocked. At box 118C, if the data type is approved for editing, then at box 120C, the hint data is edited and replaced. At box 122C, the request associated with the hint data is transmitted to the LLM. When it is determined at box 108C that the data type or model has not been updated, the request is also transmitted to the LLM.

[0052] At box 124C, a determination is made as to whether one or more redacted data tags are present in the output from the LLM. If a determination is made as to whether one or more redacted data tags are present in the output, then at box 126C, the redacted data tags are replaced with the original content. At box 128C, a determination is made as to whether any prohibited terms are present in the response. If a determination is made as to whether one or more prohibited terms are present in the response, then at box 110C, a request associated with the prompt data is blocked. If a determination is made as to whether prohibited terms are not present in the response, then at box 130C, a response is sent to the user.

[0053] refer to Figure 1D , Figure 1DA schematic diagram 100D associated with using a generative AI security engine to provide generative AI security management according to embodiments described herein is shown. At box 102D, prompt data is received. At box 104D, a determination is made as to whether the prompt data is already stored in a prompt data database. If the prompt data is not already in the prompt data database, at box 106D, a determination is made as to whether the prompt data has malicious or prohibited intent. If the prompt data is already in the prompt data database, at box 108D, a determination is made as to whether the data type or model has been updated; and if so, at box 106D, a determination is made as to whether the prompt data has malicious or prohibited intent. If the prompt data is determined to have malicious or prohibited intent, at box 110D, a request associated with the prompt data is blocked.

[0054] If the hint data does not have malicious or prohibited intent, then at block 112D, a determination is made as to whether the model identifies the sensitive data identified by the model, and at block 114D, a determination is made as to whether the data type is approved for editing. If the data type is not approved for editing, then at block 110D, the request associated with the hint data is blocked. At block 114D, if the data type is approved for editing, then at block 116D, the data is edited and replaced. At block 118D, the request associated with the hint data is transmitted to the LLM. The request is also transmitted to the LLM when it is determined at block 108D that the data type or model has not been updated.

[0055] At block 120D, a determination is made as to whether one or more redacted data tags are present in the output from the LLM. If a determination is made as to whether one or more redacted data tags are present in the output, then at block 122D, the redacted data tags are replaced with the original content. At block 124D, a determination is made as to whether any prohibited terms are present in the response. If a determination is made as to whether one or more prohibited terms are present in the response, then at block 110D, a request associated with the prompt data is blocked. If a determination is made as to whether prohibited terms are not present in the response, then at block 126D, a response is sent to the user.

[0056] refer to Figure 1E , Figure 1E A schematic diagram associated with using a generative AI safety engine in a project list system 100E to provide generative AI safety management is shown. The project list system 100E includes an application 110E, an LLM safety prompt API / library 120E, and an LLM 130E as an exemplary implementation for providing generative AI safety management. Specifically, Figure 1EA generative AI security engine is shown implemented as an LLM security prompt API / library 120E, which is placed between the application 110E and the LLM 130E to prevent data leakage or prompt attacks. The implementation runs as a service API / library that can be called as a standalone component or an integrated component in any application that wants to adopt the corresponding LLM.

[0057] In operation, at step 1, application 110E sends user-controlled input to the LLM security API / library. At step 2, the LLM security API / library sends the edited input to application 110E. At step 3, application 110E queries LLM 130E with the edited user input. At step 4, LLM 130E returns the output to application 110E. Application 110E submits the response from LLM 130E to LLM security API / library 120E. At step 6, LLM security API / library repopulates the edited data to application 110E.

[0058] refer to Figure 1F , Figure 1F A schematic diagram associated with using a generative AI security engine to provide generative AI security management according to embodiments described herein is shown. At block 102F, a training data set is received, the training data set having a plurality of data instances. At block 104F, a determination is made as to whether the data instance has malicious or prohibited intent. If the data instance is determined to have malicious or prohibited intent, then at block 106F, the data instance is excluded from the training data set.

[0059] If the data instance does not have malicious or prohibited intent, then at box 108F, a determination is made as to whether there is a regular expression hit, and if so, at box 110F, a determination is made as to whether there is a context match hit; if so, at box 112F, a determination is made as to whether the model identifies it as a true positive, and if so, at box 114F, a determination is made as to whether the data type is approved for editing. If the data type is not approved for editing, then at box 106F, the data instance is excluded from the training data set. If at box 116F, the data type is approved for editing, then at box 116F, a portion of the data instance is edited, and the synthetic data replaces the edited portion of the data instance. At box 118F, the data instance is approved.

[0060] You can refer to the examples and Figure 2A and Figure 2B to describe aspects of a technical solution. Figure 2A Based on reference Figure 6 , Figure 7 and Figure 8 A block diagram of an exemplary solution environment for implementing an embodiment of the solution. In general, the solution environment includes a solution system suitable for providing an example project list system 100 that can employ the method of the present disclosure. Specifically, Figure 2A 1 shows a high-level architecture of the project list system 100 according to an embodiment of the present disclosure. In addition to other engines, managers, generators, selectors, or components not shown (collectively referred to herein as "components"), Figure 2A The project list platform system 100 corresponds to Figure 1A and Figure 1B .

[0061] refer to Figure 2B , Figure 2B An artificial intelligence system 100A is shown, including: a generative AI smart security engine 110, having security engine operations 112 including pre-processing security engine operations 160, post-processing security engine operations 170 and training data set security engine operations; a security engine interface 114; a generative AI security engine model 116; a generative AI application 120; a generative AI application client 130, having generative AI client interface data 132; a machine learning engine 140, having a generative AI model "LLM 142"); and a prompt data database 150, having prompt data 152 and data types 154.

[0062] The generative AI security engine 110 is responsible for providing generative AI security management for the artificial intelligence system 110A. The generative AI security engine 110 accesses hint data associated with a generative artificial intelligence (AI) client 130 and a generative AI model 142 supporting the artificial intelligence system 110A, the hint data being associated with a request to the generative AI model, and analyzes the hint data (i.e., hint data instances associated with the request) based on a pre-processing security engine operation 160. The pre-processing security engine operation 160 supports determining how to transmit hint data associated with the request to the generative AI model or prevent hint data associated with the request from reaching the generative AI model.

[0063] Based on analyzing the hint data, the generative AI security engine 110 generates an edited version of the hint data for the generative AI model, the edited version of the hint data including an edited data tag associated with the edited portion of the hint data, and the generative AI security engine 110 transmits the edited version of the hint data 142 to the generative AI model. The generative AI security engine 110 accesses a response from the generative AI model and analyzes the response based on post-processing security engine operations. Post-processing security engine operations 170 support determining how to transmit the response to the generative AI hint client or determining to block the response from reaching the generative AI client 130. Based on analyzing the response, the generative AI security engine 110 transmits the response to the generative AI client or blocks the response to the request.

[0064] The generative AI security engine 110 includes pre-processing security engine operations 160, post-processing security engine operations 170, and training data set security engine operations 170 that are selectively used to provide generative AI security management in the artificial intelligence system 110A. The generative AI security engine 110 also includes a plurality of generative AI security engine models 116, which include intent detection models, prompt attack detection models, sensitive data detection models, prompt context detection models, and false positive reduction models that are selectively used to provide generative AI security management in the project list system.

[0065] The pre-processing security engine operation 160 may include pipeline threat processing based on a prompt database (e.g., prompt database 150) that includes a plurality of previously processed prompt data. When it is determined that the prompt data matches an instance of previously processed prompt data in the prompt database and no updates have been performed on the data type and generative AI security engine model associated with the pre-processing security engine operation, one or more other types of pre-processing security engine operations are bypassed. The prompt database 150 also includes a plurality of data types, wherein a first plurality of data types are identified as edited in any context, and wherein a second plurality of data types are identified as edited based on an identified prompt context for a corresponding instance of the prompt data.

[0066] The generative AI security engine 110 may include a regular expression engine and a false positive reduction model, wherein the false positive reduction model evaluates a positive output from the regular expression engine to determine whether the positive output is a false positive. The generative AI security engine 110 may include: an edit and replace engine (which operates to edit a portion of a prompt data instance and replace the edited portion with an edited data label); and an edit data label review and replacement engine, associated with reviewing instances of responses to find edit data labels and replacing edit data labels with previously edited data.

[0067] The training data set security engine operation 180 is responsible for excluding data instances in the training data set from the approved training data set, or approving instances of data in the training data set for the approved training data set. The training data set security engine operation 180 selectively includes one or more pre-processing security engine operations. The machine learning engine 140 can be implemented as part of the generative AI security engine 110, or it can be implemented separately from the generative AI security engine 110. The machine learning engine 140 (e.g., via the generative AI security engine 110) operates to access a training data set associated with an instance of a training generative AI training model; analyze the training data set using the training data set security engine operation; based on analyzing the training data set, generate an edited version of the training data set; or block at least one instance of the training data from the training data set. The edited version of the training data set includes a training data instance that includes synthetic data generated to replace the edited portion of the training data instance.

[0068] Generative AI application client 130 operates to transmit a first request associated with first prompt data; based on transmitting the first request, receive a first response generated based on an edited version of the first prompt data; and cause the first response to be displayed. Generative AI application client 130 may also transmit a second request associated with second prompt data, and based on transmitting the second request, receive a second response including a notification that the second request has been blocked; and cause the second response to be displayed.

[0069] refer to Figure 2B , Figure 2B A generative AI security engine 110, a generative AI application client 130, and a generative AI model 140 are shown for providing generative AI security management in an item list system 100. At box 10, the generative AI security engine 110 accesses a training data set associated with training a generative AI model; at box 12, a security training data set security engine operation is performed on the training data set; at box 14, at least a portion of the training data set is blocked or approved based on the execution of the training data set security engine operation.

[0070] At box 16, the generative AI application client 130 transmits a request including the hint data. At box 18, the generative AI security engine 110 accesses the request; at box 20, performs a plurality of pre-processing security engine operations on the hint data associated with the request; at box 22, based on performing the plurality of pre-processing security engine operations, transmits the request to the generative AI model. At box 24, the generative AI model accesses the request associated with the hint data; at box 26, generates a response to the request; and at box 28, transmits the response to the generative AI security engine.

[0071] At box 30, the generative AI security engine 110 accesses the response; at box 32, performs a plurality of post-processing security engine operations on the response associated with the request; at box 34, based on performing the plurality of post-processing security engine operations on the response, blocks the response; at box 36, based on blocking the response, transmits a notification that the response has been blocked. At box 38, the generative AI client receives a notification that the response to the request has been blocked based on transmitting the request; and at box 40, causes a display of the notification that the response to the request has been blocked.

[0072] Example Method

[0073] refer to Figure 3 , Figure 4 and Figure 5 Flowcharts illustrating methods for providing generative AI safety management using a generative AI safety engine in an item list system. The methods may be performed using the item list system described herein. In an embodiment, one or more computer storage media have computer executable or computer usable instructions embodied thereon that, when executed by one or more processors, may cause the one or more processors to perform a method (e.g., a computer implemented method) in an item list system (e.g., a computerized system or a computer system).

[0074] Steering Figure 3 , a flowchart illustrating a method 300 for providing generative AI security management using a generative AI security engine in an item list system is provided. At box 302, the generative AI security engine accesses prompt data associated with a generative AI client and a generative AI model of an artificial intelligence system. At box 304, the generative AI security engine analyzes the prompt data based on performing a pre-processing security engine operation. The pre-processing security engine operation supports determining how to transmit the prompt data associated with the request to the generative AI model or determining to block the request. At box 306, based on analyzing the prompt data, the generative AI security engine generates an edited version of the prompt data for the generative AI model. At box 308, the generative AI security engine transmits the edited version of the prompt data to the generative AI model. At box 310, the generative AI security engine accesses a response generated from the generative AI model based on the edited version of the prompt data. At box 312, the generative AI security engine analyzes the response based on a post-processing security engine operation. The post-processing security engine operation supports determining how to transmit the response to the generative AI client or determining to block the response. At block 314, based on analyzing the response, the generative AI security engine transmits the response to the generative AI client or blocks the response to the request.

[0075] Steering Figure 4, a flow chart illustrating a method 400 for providing generative AI security management using a generative AI security engine in a project list system is provided. At box 402, the generative AI security engine accesses a training data set associated with training a generative AI model. At box 404, the generative AI security engine analyzes a data instance from the training data set based on a plurality of training data set security operations. At box 406, based on analyzing the data instance, the generative AI security engine generates a redacted version of the data instance, or blocks the data instance of the training data set to an approved training data set for a generative AI machine learning training pipeline.

[0076] Steering Figure 5 , a flow chart illustrating a method 500 for providing generative AI security management using a generative AI security engine in an item list system is provided. At box 502, a generative AI client transmits a first request associated with first prompt data. At box 504, based on transmitting the first request, the generative AI client receives a first response generated based on an edited version of the first prompt data. At box 506, the generative AI client causes the first response to be displayed. At box 508, the generative AI client transmits a second request associated with second prompt data. At box 510, based on transmitting the second request, the generative AI client receives a second response, the second response including a notification that the second request has been blocked. At box 512, the generative AI client causes the second response to be displayed.

[0077] Technical improvements

[0078] Embodiments of the present invention have been described with reference to several inventive features (e.g., operations, systems, engines, and components) associated with an item list system. The described inventive features include: operations, interfaces, data structures, and arrangements of computing resources associated with providing functionality, which is described herein with reference to a generative AI safety engine associated with an artificial intelligence system.

[0079] Embodiments of the present invention relate to the field of computing, and more specifically, to artificial intelligence systems. The exemplary embodiments described below provide a system, method, and program product for performing generative AI security engine operations that provide generative AI security management. Therefore, the present embodiment improves the technical field of artificial intelligence technology and project list platform technology by providing more effective security. For example, the generative AI security engine provides generative AI security engine operations ("security engine operations"), including intent detection, prompt attack detection, restricted data detection and editing, and prompt context and editing, which are used to protect generative AI applications from potential data privacy and data leakage issues. Specifically, specific ways of managing data securely are summarized and presented without using conventional security technologies. Based on improving artificial intelligence technology by improving security features in artificial intelligence systems, the technical solution solves the problem that conventional project list platforms lack the integration of generative AI security engines.

[0080] The functionality of embodiments of the present invention has been further described through implementation and anecdotal examples to demonstrate operations for providing generative AI safety management using a generative AI safety engine in a project inventory system as a solution to a specific problem in artificial intelligence technology to improve computational operations in artificial intelligence systems. Overall, these improvements result in less CPU computation, smaller memory requirements, and increased flexibility in artificial intelligence systems compared to conventional artificial intelligence system operations performed for similar functions.

[0081] Additional support for the detailed description of the invention

[0082] Sample Project List System Environment

[0083] Reference now Figure 6 , Figure 6 An example project list system 600 computing environment is shown in which embodiments of the present disclosure may be employed. Specifically, Figure 6 A high-level architecture of an example project list platform 610 that can host a technical solution environment or a portion thereof is shown. It should be understood that this arrangement and other arrangements described herein are set forth as examples. For example, as described above, many elements described herein can be implemented as discrete or distributed components or combined with other components, and implemented in any suitable combination and position. In addition to or in place of the arrangements and elements shown, other arrangements and elements (e.g., machines, interfaces, functions, commands, and functional groupings) can also be used.

[0084] The project list system 600 can be a cloud computing environment that provides computing resources for functions associated with the project list platform 610. For example, the project list system 600 supports the delivery of computing components and services (including servers, storage, databases, networks, applications, and machine learning associated with the project list platform 610 and client devices 620). Multiple client devices (e.g., client devices 620) include hardware or software to access resources on the project list system 600. The client device 620 may include applications (e.g., client applications 622) and interface data (e.g., client application interface data 624) that support client functions associated with the project list system. The multiple client devices can access the computing components of the project list system 600 via a network (e.g., network 630) to perform computing operations.

[0085] The project list platform 610 is responsible for providing a computing environment or architecture, which includes an infrastructure that supports providing project list platform functions (e.g., e-commerce functions). The project list platform supports storing projects in a project database and provides a search system for receiving queries and identifying search results based on the queries. The project list platform can also provide a computing environment that has features for managing, selling, buying and recommending different types of projects. The project list platform 610 can be specifically used for content platforms, such as the EBAY content platform or e-commerce platform developed by EBAY INC of San Jose, California.

[0086] The project list platform 610 may provide project list operations 630 and project list interfaces 640. The project list operations 630 may include service operations, communication operations, resource management operations, security operations, and fault-tolerant operations that support specific tasks or functions in the project list platform 610. The project list interface 640 may include service interfaces, communication interfaces, resource interfaces, security interfaces, and management and monitoring interfaces that support functions between project list platform components. The project list operations 630 and the project list interfaces 640 may enable communication, coordination, and seamless operation of the project list system 600.

[0087] As an example, functionality associated with the item listing platform 610 may include shopping operations (e.g., product search and browsing, product selection and shopping cart, checkout and payment, and order tracking); user account operations (e.g., user registration and authentication, and user profiles); seller and product management operations (e.g., seller registration and product listings and inventory management); payment and financial operations (e.g., payment processing, refunds, and returns); order fulfillment operations (e.g., order processing and fulfillment and inventory management); customer support and communication interfaces (e.g., customer support chat / email and notifications); security and privacy interfaces (e.g., authentication and authorization, payment security); recommendations and personalization interfaces (e.g., product recommendations and customer reviews and ratings); analytics and reporting interfaces (e.g., sales and inventory reports and user behavior analytics); and APIs and integration interfaces (e.g., APIs for third-party integrations).

[0088] The item list platform 610 may provide an item list platform database (e.g., item list platform database 650) to efficiently manage and store different types of data. The item list platform database 650 may include a relational database, a NoSQL database, a search database, a cache database, a content management system, an analysis database, a payment gateway database, a customer relationship management database, a log and error database, an inventory and supply chain database, and a multi-channel database, which are used in combination to effectively manage data and provide an e-commerce experience for users.

[0089] The project list platform 610 supports applications (e.g., application 660), which are computer programs or software components or services that serve specific functions or sets of functions to meet specific project list platform requirements or user requirements. Applications can be client-side (user-oriented) and server-side (backend). Applications can also include applications without any AI support (e.g., application 662), applications supported by traditional AI models (e.g., application 664), and applications supported by generative AI models (e.g., application 666). As an example, applications can include online storefront applications, mobile shopping applications, implementation and management consoles, payment gateway integration, user accounts and authentication applications, search and recommendation engines, inventory and stock management applications, order processing and fulfillment applications, customer support and communication tools, content management systems, analysis and reporting applications, marketing and promotion applications, multi-channel integration applications, log and error tracking applications, customer relationship management (CRM) applications, security applications, and APIs and Web services, which are used in combination to effectively provide users with e-commerce experiences.

[0090] Project List Platform 610 may include a machine learning engine (e.g., machine learning engine 670). Machine learning engine 670 refers to a machine learning framework or machine learning platform that provides infrastructure and tools for designing, training, evaluating, and deploying machine learning models. Machine learning engine 670 may serve as a backbone for developing and deploying machine learning applications and solutions. Machine learning engine 670 may also provide tools for visualizing data and model results and explaining model decisions to understand how the model makes predictions.

[0091] The machine learning engine 670 can provide the necessary libraries, algorithms, and utilities to perform various tasks within the machine learning workflow. The machine learning workflow can include data processing, model selection, model training, model evaluation, hyperparameter tuning, scalability, model deployment, reasoning, integration, customization, data visualization. The machine learning engine 670 can include pre-trained models for various tasks, thereby simplifying the development process. In this way, the machine learning engine 670 can simplify the entire machine learning process, from data preparation and model training to deployment and reasoning, making it accessible and efficient for different types of users (e.g., customers, data scientists, machine learning engineers, and developers) engaged in a wide range of machine learning applications.

[0092] The machine learning engine 670 can be implemented in the item list system 600 as a component that utilizes machine learning algorithms and techniques (e.g., machine learning algorithm 672) to enhance various aspects of the functionality of the item list system. The machine learning engine 670 can provide a series of machine learning algorithms and techniques for teaching computers to learn from data and make predictions or decisions without being explicitly programmed. These techniques are widely used in various applications in different industries and can include the following examples: supervised learning (e.g., linear regression: classification, support vector machine (SVM)); unsupervised learning (e.g., clustering, principal component analysis (PCA), association rules (e.g., priors); reinforcement learning (e.g., Q learning, deep Q network (DQN)); and deep learning (e.g., neural network, convolutional neural network (CNN) and recurrent neural network (RNN)); and ensemble learning random forest.

[0093] Machine learning training data 120 supports the process of building, training, and fine-tuning machine learning models. Machine learning training data 120 consists of labeled data sets that are used to teach machine learning models to recognize patterns, make predictions, or perform specific tasks. Training data generally includes two main components: input features (X), and labels or target values ​​(Y). Input features may include variables, attributes, or characteristics that are used as input to a machine learning model. Input features (X) may be numeric, categorical, or even textual, depending on the nature of the problem. For example, in a model for predicting housing prices, input features may include the number of bedrooms, square feet, neighborhood, etc. Labels or target values ​​(Y) include the values ​​that the model aims to predict or classify. Labels represent the desired output or ground truth for each corresponding set of input features. For example, in a spam classifier, the label would indicate whether each email is spam or not (i.e., a binary classification). The training process involves presenting training data to the model, and the model learns to make predictions or decisions by identifying patterns and relationships between input features (X) and target values ​​(Y). The machine learning algorithm adjusts its internal parameters during training to minimize the difference between its predictions and the actual labels in the training data. The machine learning engine 670 can use historical and real-time data to train models and make predictions, thereby continuously improving performance and user experience.

[0094] The machine learning engine 670 may include a machine learning model (e.g., machine learning model 676) generated using a machine learning engine workflow. The machine learning model 676 may include a generative AI model and a traditional AI model, both of which may be used in the item list system 600. Generative AI models are designed to generate new data (usually in the form of text, images, or other media) based on patterns and knowledge learned from existing data. Generative AI models can be used in a variety of ways, including: content generation, product image generation, personalized product recommendations, natural language chatbots, and content summarization. Traditional AI models cover a wide range of algorithms and technologies and can be used in a variety of ways, including: recommendation systems, predictive analysis, search algorithms, fraud detection, customer segmentation, image classification, natural language processing (NLP), and A / B testing and optimization. In many cases, a combination of generative AI models and traditional AI models can be used to provide a comprehensive and efficient e-commerce experience, combined with data-driven insights and creativity.

[0095] Machine learning engine 670 may be used to analyze data, make predictions, and automate processes to provide users with a more personalized and efficient shopping experience. For example, product recommendation search and filtering: pricing optimization, inventory and stock management: customer segmentation, customer churn prediction and retention, fraud detection, sentiment analysis, customer support and chatbots, image and video analysis, and advertising targeting and marketing. The specific application of machine learning within item listing platform 610 may vary depending on the specific goals, available data, and resources.

[0096] Example distributed computing system environment

[0097] Reference now Figure 7 , Figure 7 An example distributed computing environment 700 is shown in which embodiments of the present disclosure can be employed. Specifically, Figure 7 A high-level architecture of an example cloud computing platform 710 capable of hosting a technology solution environment or a portion thereof (e.g., a data trustee environment) is shown. It should be understood that this arrangement and other arrangements described herein are set forth only as examples. For example, as described above, many of the elements described herein can be implemented as discrete or distributed components or in combination with other components, and in any suitable combination and location. Other arrangements and elements (e.g., machines, interfaces, functions, commands, and functional groupings) may also be used in addition to or in place of the arrangements and elements shown.

[0098] The data center can support a distributed computing environment 700, which includes a cloud computing platform 710, a rack 720, and a node 730 (e.g., a computing device, a processing unit, or a blade) in the rack 720. A technical solution environment can be implemented using a cloud computing platform 710 that runs cloud services across different data centers and geographic regions. The cloud computing platform 710 can implement a fabric controller 740 component for providing and managing resource allocation, deployment, upgrades, and management of cloud services. Typically, the cloud computing platform 710 is used to store data or run service applications in a distributed manner. The cloud computing infrastructure 710 in the data center can be configured to host and support the operation of endpoints for specific service applications. The cloud computing infrastructure 710 can be a public cloud, a private cloud, or a dedicated cloud.

[0099] The node 730 may be provided with a host 750 (e.g., an operating system or runtime environment) that runs a defined software stack on the node 730. The node 730 may also be configured to perform a specialized function (e.g., a computing node or a storage node) within the cloud computing platform 710. The node 730 is allocated to run one or more parts of a tenant's service application. A tenant may refer to a customer utilizing the resources of the cloud computing platform 710. The service application components of the cloud computing platform 710 that support a specific tenant may be referred to as a multi-tenant infrastructure or lease. In this article, the terms service application, application, or service are used interchangeably and broadly refer to any software or software portion that runs on or accesses storage devices and computing device locations within a data center.

[0100] When the node 730 is supporting multiple individual service applications, the node 730 can be divided into virtual machines (e.g., virtual machine 752 and virtual machine 754). The physical machine can also run separate service applications at the same time. The virtual machine or physical machine can be configured as a personalized computing environment supported by resources 760 (e.g., hardware resources and software resources) in the cloud computing platform 710. It is envisioned that resources can be configured for specific service applications. In addition, each service application can be divided into functional parts so that each functional part can run on a separate virtual machine. In the cloud computing platform 710, multiple servers can be used to run service applications and perform data storage operations in a cluster. Specifically, the server can perform data operations independently, but is disclosed as a single device called a cluster. Each server in the cluster can be implemented as a node.

[0101] The client device 780 can be connected to the service application in the cloud computing platform 710. The client device 780 can be a device that can correspond to the reference Figure 7 Any type of computing device of the computing device 700, for example, the client device 780, can be configured to issue commands to the cloud computing platform 710. In an embodiment, the client device 780 can communicate with the service application through a virtual Internet Protocol (IP) and a load balancer or other means that directs the communication request to a designated endpoint in the cloud computing platform 710. The components of the cloud computing platform 710 can communicate with each other through a network (not shown), which may include but is not limited to one or more local area networks (LANs) and / or wide area networks (WANs).

[0102] Sample computing environment

[0103] Having briefly described an overview of embodiments of the present invention, an example operating environment in which embodiments of the present invention may be implemented is described below in order to provide a general context for various aspects of the present invention. Figure 8, an example operating environment for implementing embodiments of the invention is shown and generally designated as computing device 800. Computing device 800 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should computing device 800 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated.

[0104] The present invention may be described in the general context of computer code or machine-usable instructions, including computer-executable instructions (e.g., program modules) executed by a computer or other machine (e.g., a personal data assistant or other handheld device). Generally, program modules, including routines, programs, objects, components, data structures, etc., refer to code that performs specific tasks or implements specific abstract data types. The present invention may be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more specialized computing devices, etc. The present invention may also be practiced in distributed computing environments, where tasks are performed by remote processing devices linked through a communications network.

[0105] refer to Figure 8 , computing device 800 includes bus 810 that directly or indirectly couples the following devices: memory 812, one or more processors 814, one or more presentation components 816, input / output ports 818, input / output components 820, and an illustrative power supply 822. Bus 810 represents one or more buses (e.g., an address bus, a data bus, or a combination thereof). For conceptual clarity, Figure 8 The various blocks of the present invention are shown with lines, and other arrangements of the described components and / or component functions are also contemplated. For example, a presentation component such as a display device may be considered an I / O component. In addition, a processor has memory. We recognize that this is the nature of the art, and reiterate that Figure 8 The figures are merely illustrative of example computing devices that can be used in conjunction with one or more embodiments of the present invention. No distinction is made between categories such as "workstation," "server," "laptop," "handheld device," etc., as all of these categories are in the same Figure 8 within the scope of and with reference to a “computing device”.

[0106] The computing device 800 typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by the computing device 800, and includes volatile and non-volatile media, removable and non-removable media. By way of example and not limitation, computer-readable media can include computer storage media and communication media.

[0107] Computer storage media includes volatile and nonvolatile, and removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage devices, magnetic cassettes, magnetic tape, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by the computing device 800. Computer storage media themselves do not include signals.

[0108] Communication media typically embodies computer-readable instructions, data structures, program modules, or other data in the form of a modulated data signal (such as a carrier wave or other transmission mechanism), and includes any information transmission medium. The term "modulated data signal" refers to a signal that is set or changed in a manner that encodes the information in the signal, and the signal has one or more characteristics. By way of example and not limitation, communication media include wired media such as a wired network or a direct wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above should also be included within the scope of computer-readable media.

[0109] Memory 812 includes computer storage media in the form of volatile and / or non-volatile memory. Memory may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. Computing device 800 includes one or more processors that read data from various entities such as memory 812 or I / O components 820. Presentation component 816 presents data indications to a user or other device. Exemplary presentation components include display devices, speakers, printing components, vibration components, etc.

[0110] I / O ports 818 allow computing device 800 to be logically coupled to other devices including I / O components 820, some of which may be built-in. Illustrative components include a microphone, joystick, game pad, satellite dish, scanner, printer, wireless devices, and the like.

[0111] Additional structural and functional features of embodiments of the technical solution

[0112] Various components used herein have been identified, and it should be understood that any number of components and arrangements can be used to implement the desired functions within the scope of the present disclosure. For example, for the sake of conceptual clarity, the components in the embodiments depicted in the accompanying drawings are shown with lines. Other arrangements of these and other components can also be implemented. For example, although some components are depicted as single components, many elements described herein can be implemented as discrete or distributed components or combined with other components, and implemented in any suitable combination and position. Some elements can be omitted completely. In addition, as described below, the various functions performed by one or more entities described herein can be performed by hardware, firmware and / or software. For example, various functions can be performed by a processor that executes instructions stored in a memory. Therefore, in addition to or in place of the arrangements and elements shown, other arrangements and elements (e.g., machines, interfaces, functions, commands, and functional groups) can also be used.

[0113] The embodiments described in the following paragraphs may be combined with one or more specifically described alternatives. Specifically, the claimed embodiments may include references to more than one other embodiments in the alternatives. The claimed embodiments may specify additional limitations of the claimed subject matter.

[0114] The subject matter of embodiments of the present invention is specifically described herein to meet statutory requirements. However, this description itself is not intended to limit the scope of this patent. Instead, the inventors have contemplated that the claimed subject matter may also be embodied in other ways in conjunction with other prior art or future technologies to include different steps or combinations of steps similar to the steps described in this document. In addition, although the terms "step" and / or "box" may be used herein to refer to different elements of the method employed, the terms should not be interpreted as implying any particular order between or among the various steps disclosed herein unless and in addition to the order of the various steps being explicitly described.

[0115] For purposes of this disclosure, the word "includes" has the same broad meaning as the word "including," and the word "access" includes "receiving," "referencing," or "retrieving." Furthermore, the word "communicates" has the same broad meaning as the word "receive" or "send," which is facilitated by a software or hardware-based bus, receiver, or transmitter using the communication media described herein. Furthermore, unless otherwise indicated, words such as "a," "an," and "an" include the plural as well as the singular. Thus, for example, where there are one or more features, the constraint of "a feature" is satisfied. Furthermore, the term "or" includes conjunctions, disjunctions, and both (a or b therefore includes a or b, and a and b).

[0116] For the purpose of the above detailed discussion, embodiments of the present invention are described with reference to a distributed computing environment; however, the distributed computing environment described herein is merely exemplary. Components may be configured to perform novel aspects of the embodiments, where the term "configured to" may refer to "programmed to" perform a specific task or implement a specific abstract data type using code. In addition, although embodiments of the present invention may generally refer to the technical solution environment and schematic diagrams described herein, it should be understood that the described techniques may be extended to other implementation contexts.

[0117] The embodiments of the present invention have been described with respect to specific embodiments which are intended in all respects to be illustrative and not restrictive. Alternative embodiments will become apparent to those skilled in the art to which the present invention pertains without departing from the scope of the present invention.

[0118] From the foregoing it will be seen that this invention is well adapted to attain all the ends and objects herein above set forth, together with other advantages which are obvious and inherent to the structure.

[0119] It will be understood that certain features and subcombinations are of utility and may be employed without reference to other features or subcombinations. This is contemplated by the claims and is within the scope of the claims.

Claims

1. A computerized system comprising: one or more computer processors; as well as A computer memory storing computer usable instructions that, when used by the one or more computer processors, cause the one or more computer processors to perform operations comprising: Accessing prompt data associated with a generative artificial intelligence (AI) client and a generative AI model supporting an artificial intelligence system, wherein the prompt data is associated with a request for the generative AI model; analyzing the hint data based on a pre-processing security engine operation, wherein the pre-processing security engine operation supports determining how to transmit the hint data associated with the request to the generative AI model or prevent the hint data associated with the request from reaching the generative AI model; generating, for the generative AI model, an edited version of the prompt data based on analyzing the prompt data, the edited version of the prompt data including an edited data tag associated with an edited portion of the prompt data; transmitting an edited version of the prompt data to the generative AI model; accessing a response from the generative AI model; analyzing the response based on a post-processing security engine operation, wherein the post-processing security engine operation supports determining how to transmit the response to a generative AI prompt client or determining to block the response from reaching the generative AI client; and Based on analyzing the response, transmitting the response to the generative AI client or blocking the response to the request.

2. The system of claim 1 , further comprising a generative AI security engine, the generative AI security engine comprising the pre-processing security engine operations, the post-processing security engine operations, and the training data set security engine operations selectively used to provide generative AI security management in the artificial intelligence system.

3. The system according to claim 1 further includes a generative AI security engine, wherein the generative AI security engine includes multiple generative AI security engine models, and the multiple generative AI security engine models include an intent detection model, a prompt attack detection model, a sensitive data detection model, a prompt context detection model and a false positive reduction model that are selectively used to provide generative AI security management in the project list system.

4. The system of claim 1, further comprising pipeline threat processing based on a prompt database, the prompt database comprising a plurality of previously processed prompt data, wherein: Upon determining that the hint data matches an instance of previously processed hint data in the hint database and no updates have been performed to the data types and generative AI security engine models associated with the pre-processing security engine operations, one or more of the pre-processing security engine operations are circumvented.

5. The system of claim 1, further comprising a plurality of data types, wherein: A first plurality of data types are identified for editing in any context, wherein a second plurality of data types are identified for editing based on an identified prompt context for a corresponding instance of prompt data.

6. The system of claim 1 further comprising a regular expression engine and a false positive reduction model, wherein: The false positive reduction model evaluates positive output from the regular expression engine to determine whether the positive output is a false positive.

7. The system of claim 1, further comprising: an edit and replace engine associated with: editing a portion of an instance of said prompt data and replacing the edited portion with an edit data tag; and an edited data label review and replacement engine associated with reviewing instances of responses to edited data labels and replacing the edited data labels with previously edited data.

8. The system of claim 1, further comprising a training dataset security engine operation associated with: excluding an instance of data in the training dataset from an approved training dataset, or approving an instance of data in the training dataset for use in an approved training dataset, wherein: The training data security engine operations optionally include pre-processing security engine operations.

9. The system of claim 1, wherein the operations further comprise: transmitting a first request associated with first prompt data; based on transmitting the first request, receiving a first response generated based on an edited version of the first prompt data; causing the first response to be displayed; transmitting a second request associated with second prompt data; receiving a second response based on transmitting the second request, the second response including a notification that the second request has been blocked; as well as The second response is caused to be displayed.

10. The system of claim 1, the operations further comprising: Accessing training datasets associated with instances of training generative AI training models; Analyzing the training data set using a training data set security engine operation; generating an edited version of the training data set based on analyzing the training data, the edited version of the training data set comprising synthetic data; or At least one instance of training data is blocked from the training data set.

11. One or more computer storage media having computer executable instructions thereon, which when executed by a computing system having a processor and a memory, cause the processor to perform operations comprising: transmitting a first request associated with first prompt data; Based on transmitting the first request, receiving a first response generated based on an edited version of the first prompt data, wherein the response to the prompt data is generated based on processing the prompt data using a generative artificial intelligence (AI) security engine, the generative AI security engine supporting an artificial intelligence system based on providing pre-processing security engine operations and post-processing security operations for generative AI security management in the artificial intelligence system; causing the first response to be displayed; transmitting a second request associated with second prompt data; based on transmitting the second request, receiving a second response, the second response including a notification that the second request has been blocked; and The second response is caused to be displayed.

12. The medium according to claim 11, wherein The generative AI security engine also includes training dataset security engine operations that support protecting training datasets associated with training a generative AI model in the artificial intelligence system.

13. The medium according to claim 11, wherein The generative AI security engine also includes multiple generative AI security engine models, which include intent detection models, prompt attack detection models, sensitive data detection models, prompt context detection models and false positive reduction models that are selectively used to provide generative AI security management in a project list system.

14. The medium according to claim 11, wherein The first hint data is processed based on the pre-processing security engine operation and the post-processing security engine operation.

15. The medium according to claim 11, wherein The second hint data is processed based solely on the pre-processing security engine operation, wherein the pre-processing security engine operation causes the second hint data to be blocked.

16. A computer-implemented method, the method comprising: Access to training datasets for generative AI models; analyzing the training data set using a training data security engine operation associated with a training data set machine learning pipeline and a generative AI security engine of the artificial intelligence system; Based on analyzing the training data set, generating an edited version of the training data set; as well as Approving an edited version of the training dataset for use in a generative AI machine learning training pipeline.

17. The method of claim 16, further comprising blocking at least one instance of data in the training data set to an approved training data set, wherein: The training data set security engine operations are associated with excluding instances of data from the training data set or approving instances of data for use in the training data set.

18. The method according to claim 16, wherein: The edited version of the training data set includes a training data instance including synthetic data generated to replace an edited portion of the training data instance.

19. The method according to claim 16, wherein: The generative AI security engine includes pre-processing security engine operations, post-processing security engine operations, and training data set security engine operations that are selectively used to provide generative AI security management in the artificial intelligence system.

20. The method according to claim 16, wherein: The training data set security engine operations optionally include pre-processing security engine operations.

Images