Integrated private cloud AI platform

Abstract

The present disclosure provides systems and methods for an integrated private cloud (AI) platform deployable at a customer site. The integrated private cloud AI platform may include components deployed at the customer site that utilize an embedded model that locally accesses / integrates various knowledge bases and vector data repositories at the customer site and permits chat robot accessible queries with previously uploaded embedded LLMs.

Description

[0001] Cross-references to related applications

[0002] This application, together with Indian Patent Application No. 202441065736 (P175628IN(109793-01456)) filed on August 30, 2024 and U.S. Patent Application No. 18 / 822,824(636US) filed on September 3, 2024, the entire contents of which are incorporated herein by reference for all purposes. Background Technology

[0003] Machine learning models are the process by which computers learn from data and make decisions or predictions without being explicitly programmed for a specific task. These models are exposed to more data over time, thereby improving their performance.

[0004] Machine learning models can be categorized into supervised learning models (such as linear regression and classification models), unsupervised learning models (such as clustering and dimensionality reduction), reinforcement learning models, and semi-supervised and self-supervised learning models. One type of machine learning model is the Large Language Model (LLM), which involves both unsupervised and self-supervised learning. In LLMs, models are specifically designed to understand, generate, and manipulate human language to analyze large datasets through advanced neural network architectures and programming execution.

[0005] Because these LLMs are highly complex, customer environments typically run them remotely from the model provider or other remote systems. However, using remote systems can introduce security issues related to transferring data over open networks, restricting access to sensitive data for authorized users, or sharing remote systems with other entities that may be direct competitors of the customer. Attached Figure Description

[0006] The present disclosure is described in detail with reference to the following accompanying drawings, which illustrate one or more various examples. These drawings are for illustrative purposes only and depict only typical, non-limiting aspects of such examples.

[0007] Figure 1 An example of a network configuration that can be implemented for organizations such as businesses, educational institutions, government entities, healthcare facilities, or other organizations is shown.

[0008] Figure 2 These are illustrative AI platforms, external model repositories, and communication structures shown in some of the examples disclosed herein.

[0009] Figure 3 This is an illustrative AI platform among some examples disclosed herein.

[0010] Figure 4These are illustrative communication processes within an AI platform, as shown in some examples of this disclosure.

[0011] Figure 5 It is an example computational component that can be used to implement various features of the model set based on the examples discussed in this article.

[0012] Figure 6 It is a computing component that can be used to implement examples of the disclosed technologies.

[0013] The accompanying drawings are not exhaustive and do not limit this disclosure to the precise form disclosed. Detailed Implementation

[0014] Traditional LLMs are very large (e.g., 50-100GB models), and due to network technology limitations, downloading them to the customer site can take several days. For example, the customer site may be an intranet, a private cloud, or other private network accessible only to members of the authenticated and authorized customer site, while other entities may use the Internet or other public networks to attempt to communicate with the devices at the customer site.

[0015] A compute layer can be added to a customer site to enable communication both inside and outside a private network or private cloud environment. For example, one way to achieve this communication includes deploying a generative AI chatbot module to access data within the customer site and using an AI platform to organize / maintain that data. The AI ​​platform can use embedded and vector data structures to organize existing data in the private cloud, which helps to quickly and easily identify similarities between data and accelerate the search process. The AI ​​platform can also include an LLM (Local Modeling Model) that communicates with the AI ​​chatbot module, which can accelerate data discovery within the private cloud environment. When several models (including embedded models, vector models, LLMs, etc.) are used within the AI ​​platform, it can also facilitate the process of quickly replacing and deploying models to the customer environment. The AI ​​platform also provides better authentication / security to add another layer of protection to the compute environment and restrict access to customer data.

[0016] More specifically, examples of this disclosure describe an integrated private cloud AI platform that can be deployed at a customer site to enable an improved retrieval augmented generation (RAG) integrated AI platform with chatbot-accessible queries on LLMs. For example, the AI ​​platform combines generative LLMs with embeddings with vector-based information retrieval to improve data storage capacity, information retrieval, data security / authentication, and optimize response effectiveness. The AI ​​platform may include components deployed at the customer site that leverage embedded models that access / integrate various knowledge bases (e.g., locally at the customer site) and permit chatbot-accessible queries using previously uploaded embedded LLMs.

[0017] Various examples of AI platforms can be implemented. For instance, an embedded model can be deployed at a customer site as part of an AI platform. The embedded model is configured to encapsulate data associated with the customer site into a multidimensional data structure. Data accessed by the embedded model can be identified by the managing user in a pre-existing location outside the AI ​​platform (e.g., a first knowledge base), or it can be directly uploaded to the AI ​​platform by the managing user (e.g., a second knowledge base). In either case, the AI ​​platform can initiate an authentication / authorization process for the managing user to confirm that the managing user has the appropriate access level to provide the knowledge base and is also permitted to access specific types of data. During authentication / authorization, the AI ​​platform initiates a job to upload the knowledge base (as a transformed embedded format) to the user's namespace (e.g., Kubernetes). TM (Assignment). AI platforms can convert embeddings into vectors and store them in a vector data repository.

[0018] AI platforms can receive user queries via generative AI chatbot modules and initiate RAGs that combine generative LLMs with previously stored embeddings, vectors, and other customer data. For example, queries can be related to customer problems, help solve problems, or provide customized suggestions and solutions. RAGs can combine two different systems (a retrieval system for searching relevant information and an LLM for generating text). Traditional systems may struggle to manage multiple systems and data sources when data comes from multiple data sources and in multiple formats.

[0019] The AI ​​platform can initiate an authentication / authorization process for the client user submitting the query. The policy engine helps determine if the client user has the appropriate access level to request the type of information to be included in the query. During authentication / authorization, the LLM passes the query to the embedding model to access appropriate preprocessed information (including knowledge bases, vectors, and embeddings) and retrieves the data.

[0020] In some examples, AI platforms leverage RAGs that rely on pre-created LLMs. These LLMs can be open-source models or enterprise-created models. AI platforms can provide customers / enterprises with the ability to deploy, manage, and scale these models. Because GPU resources may be limited or otherwise constrained, AI platforms can enable entities to share models across multiple use cases and locations (e.g., finance, marketing, etc.).

[0021] In some examples, AI platforms reuse a single LLM for different use cases. In this example, the knowledge base can be different for each solution accelerator, and the LLM can be the same when depending on / accessing different knowledge bases.

[0022] In some examples, AI platforms can provide the ability to scale automatically based on load, demand, or usage. This can occur during the data preparation phase and during the customer interaction phase.

[0023] In some examples, the AI ​​platform can implement data chunking (e.g., data partitioning or clustering). Depending on the use case, one or more chunking options may be supported, including data chunking based on fixed values ​​or semantic processes. In some examples, the AI ​​platform can provide administrative users with the ability to control access to the AI ​​platform accelerator.

[0024] Throughout this disclosure, technical improvements are described. For example, platform components are interchangeable, allowing customer sites to incorporate their own on-premises knowledge bases or other data without sharing components with the public cloud / platform. Furthermore, the interchangeability of many components of the AI ​​platform allows switching from a first embedding model to a second embedding model without separating the rest of the process. Additionally, in this customer environment, the platform enables multi-layered authentication at several access points, allowing for enhanced security restrictions on data to help maintain data policy-based controls.

[0025] Before describing in detail the various examples of the disclosed systems and methods, it is useful to describe example network installations that can be used to implement these systems and methods in various applications. Figure 1 An example of a network configuration 100 that can be implemented for an organization (such as a business, educational institution, government entity, healthcare facility or other organization) is shown. Figure 1 An example configuration implemented using an organization with multiple users (or at least multiple client devices 110) and possibly multiple physical or geographic sites 102, 132, 142 is shown. Network configuration 100 may include a master site 102 communicating with a network 120 storing the AI ​​platform. Network configuration 100 may also include one or more remote sites 132, 142, each of which may store parts / components of the AI ​​platform. Each of these sites can be accessed by client devices authorized to submit queries to the LLM and previously processed embeddings.

[0026] Main site 102 may include a main network, such as an office network, a home network, or other network installation. The main network may be a private network, which may include security and access controls to restrict access to authorized users of the private network. For example, authorized users may include company employees, residents, and business customers at main site 102.

[0027] exist Figure 1In the example, master site 102 includes controller 104, which communicates with network 120. Controller 104 can provide communication between master site 102 and network 120. In addition to controller 104, master site 102 may have other communication points with network 120. Although a single device associated with controller 104 is shown, master site 102 may include multiple controllers and / or multiple communication points with network 120. In some examples, controller 104 may communicate with network 120 via a router. In other examples, controller 104 provides router functionality to devices in master site 102. In this specification, the term "tunnel" refers to the encapsulation mode for transmitting data between the AP and the controller.

[0028] Controller 104 is operable to configure and manage network devices (such as at the main site 102) and can also manage network devices at remote sites 132, 142. Controller 104 is operable to configure and / or manage switches, routers, access points, and / or client devices connected to the network. Controller 104 itself may be an access point (AP) or provide access point (AP) functionality.

[0029] Controller 104 can communicate with one or more switches 108 and / or wireless access points (APs) 106a-c. Switches 108 and wireless APs 106a-c provide network connectivity for various client devices 110a-j. Using the connection to switch 108 or AP 106a-c, client devices 110a-j can access network resources, including the (main site 102) network and other devices on network 120.

[0030] Examples of client devices may include: desktop computers, laptop computers, servers, web servers, authentication servers, authentication-authorization-accounting (AAA) servers, domain name system (DNS) servers, dynamic host configuration protocol (DHCP) servers, Internet protocol (IP) servers, virtual private network (VPN) servers, network policy servers, mainframes, tablet computers, e-readers, netbook computers, televisions and similar displays (e.g., smart TVs), content receivers, set-top boxes, personal digital assistants (PDAs), mobile phones, smartphones, smart terminals, dumb terminals, virtual terminals, video game consoles, virtual assistants, Internet of Things (IoT) devices, etc.

[0031] Within main site 102, switch 108 is included as an example of an access point for a network established in main site 102 for wired client devices 110i-j. Client devices 110i-j can connect to switch 108 and, through switch 108, can access other devices within network configuration 100. Client devices 110i-j can also access network 120 through switch 108. Client devices 110i-j can communicate with switch 108 via wired or wireless connection 112. In the example shown, switch 108 communicates with controller 104 via wired or wireless connection 112.

[0032] Wireless AP 106a-c is included as another example of an access point for a network established at main site 102 for client devices 110a-h. Each AP in AP 106a-c can be a combination of hardware, software, and / or firmware configured to provide wireless network connectivity to wireless client devices 110a-h. Figure 1 In the example, AP 106a-c can be managed and configured by controller 104. AP 106a-c communicates with controller 104 and the network via connection 112, which can be a wired or wireless interface.

[0033] Network configuration 100 may include one or more remote sites 132. Remote sites 132 may be located in a different physical or geographical location than the main site 102. In some cases, remote sites 132 may be located in the same geographical location as the main site 102, or may be located in the same building, but lack a direct connection to the network located within the main site 102. Instead, remote sites 132 may utilize connections through a different network (e.g., network 120). For example, remote sites 132 (such as...) Figure 1 The remote site 132 shown may be a satellite office, another floor or suite in a building. Remote site 132 may include a gateway device 134 for communicating with network 120. Gateway device 134 may be a router, analog-to-digital modem, cable modem, digital subscriber line (DSL) modem, or some other network device configured to communicate with network 120. Remote site 132 may also include a switch 138 and / or an access point (AP) 136 that communicate with gateway device 134 via wired or wireless connections. Switch 138 and AP 136 provide connectivity to the network for various client devices 140a-d.

[0034] In various examples, remote station 132 can communicate directly with master station 102, enabling client devices 140a-d at remote station 132 to access network resources at master station 102 as if they were located at master station 102. In such an example, remote station 132 is managed by controller 104 at master station 102, and controller 104 provides the necessary connectivity, security, and accessibility to establish the connection between remote station 132 and master station 102. Once connected to master station 102, remote station 132 can be used as part of a private network provided by master station 102.

[0035] In various examples, network configuration 100 may include one or more smaller remote sites 142, including a gateway device 144 and a wireless access point (AP) 146 for communicating with network 120, through which various client devices 150a-b access network 120. Examples of remote sites 142 may represent, for example, a single employee's home or a temporary remote office. Remote sites 142 may also communicate with the main site 102, enabling client devices 150a-b at remote sites 142 to access network resources at the main site 102 as if they were located at the main site 102. Remote sites 142 may be managed by a controller 104 at the main site 102 to enable this transparency. Once connected to the main site 102, remote sites 142 can be used as part of a private network provided by the main site 102.

[0036] Network 120 may be a public or private network (such as the Internet or other communication networks) to allow connectivity between various sites 102, 132, 142 and access to servers 160a-b. Network 120 may include third-party telecommunications lines, such as telephone lines, broadcast coaxial cables, fiber optic cables, satellite communications, cellular communications, etc. Network 120 may include any number of intermediate network devices, such as switches, routers, gateways, servers, and / or controllers, which are not directly part of network configuration 100 but facilitate communication between the various parts of network configuration 100 and between network configuration 100 and other network-connected entities. Network 120 may include various servers 160a-b. In the example, servers 160a-b may include content servers that include providers of various multimedia downloadable and / or streaming content, including audio, video, graphics, and / or text content, or any combination thereof. Examples of content servers 160a-b include web servers, streaming broadcast and video providers, and cable and satellite television providers. Client devices 110a-j, 140a-d, and 150a-b can request and access multimedia content provided by content server 160a-b.

[0037] In another example, servers 160a-b may include flow optimization service servers, which include various information for provisioning services to client devices 110a-j, 140a-d, and 150a-b and optimizing service flows according to the examples disclosed herein. Access points 106a-c, 136, and 146; switch 108; and gateway devices 134 and 144 may request or upload information (such as telemetry data) for optimizing the presentation of services to client devices 110a-j, 140a-d, and 150a-b. This information may include, but is not limited to, measurements or estimates of QoE based on each service flow (e.g., referred to herein as QoE scores); flow characteristics and other QoS measurements, such as, but not limited to, jitter, latency, talk time, latency, etc.; analytics; transport protocols (e.g., OFDMA and MU-MIMO), etc. The information may be stored in a database that may be communicatively coupled to servers 160a and 160b. In the example, servers 160a-b may be cloud-based, which will be understood by those skilled in the art to mean, for example, a system / server remotely hosted on a network (rather than hosted on a local server / computer) and remotely accessible.

[0038] In the cloud-based example of server 160a-b, server 160a-b may store components of both the public and private clouds (shown at 102). In some examples, server 160a-b may store components downloaded locally from the main site 102 or remote site 132 to facilitate the implementation of a private cloud. The private cloud can implement an integrated AI platform at the customer site to organize and maintain a knowledge base of customer data. As described throughout this disclosure, a generative AI chatbot can be used to access the private cloud. In some examples, servers implementing the public cloud may store machine learning models (e.g., embedded models, pre-trained LLMs, etc.) that can be downloaded to and integrated into the private cloud.

[0039] Figure 2 These are illustrative AI platforms, external model repositories, and communication structures shown in some examples of this disclosure. In Example 200, an external machine learning (ML) model repository 210, an AI platform 220, and a communication structure 230 are illustrated, where each device / component communicates via a network. In some examples, the AI ​​platform 220 includes a model downloader engine 222, and the communication structure 230 includes a secondary model repository 232.

[0040] External ML model repository 210 includes pre-trained machine learning models that can be provided to AI platform 220 located at a customer's site. External ML model repository 210 can pre-prepare ML models and deliver them to the customer. In other examples, external ML model repository 210 may include open-source models or models accessible via external systems that can be downloaded to AI platform 220 (via a network through model downloader engine 222).

[0041] AI platform 220 is implemented on a private cloud at the customer's site and includes embedded models that access data stored locally at the customer's site or uploaded to external data sources. For example, AI platform 220 can receive requests to add other knowledge bases or models, which can be achieved using model downloader engine 222. Each request to access data, upload a new model, etc., can precede the user's authentication / authorization process to help confirm that the user has the appropriate access level to perform the action.

[0042] AI platform 220 can also allow access to data / models stored together with the customer's site's private cloud. This data / model can be accessed via a generative AI chatbot module that accepts queries and passes them to the LLM. The LLM can access previously stored data via embedded models, vector data repositories, or other data structures described herein, and retrieve appropriate data to generate query responses.

[0043] Communication architecture 230 may include a coordinator for transmitting communications across various electronic environments. In some examples, the coordinator is implemented as Kubernetes, providing a managed coordinator model. TM Engine. Communication structure 230 can receive / pull models from various sources and store the models in a secondary model repository 232 associated with communication structure 230.

[0044] Figure 3 This is an illustrative AI platform among some examples in this disclosure. Example 300 provides a detailed AI platform that can be similar to... Figure 2 The AI ​​platform 220 is shown in the diagram. Multiple users can interact with the AI ​​platform, including management users 302 and general users 304, each corresponding to a different access level and profile.

[0045] At box 310, the AI ​​platform receives an embedding model. The embedding model may correspond to an algorithm trained to encapsulate information (e.g., a knowledge base) into a multidimensional space. In some examples, this is obtained from an external ML repository (e.g., Figure 2 External ML model library 210 or via Figure 2The communication structure 230 in the document receives the embedded model. In other examples, multiple models are received from an external ML repository (e.g., box 338).

[0046] In some examples, embedded models can be deployed at a customer's site as part of an AI platform. The embedded model can access data and create data segments associated with chunk sizes. Embedding can convert a textual representation of data into a numeric or binary format, which serves as a secondary format of the data. The embedded model can also access a knowledge base at the customer's site, which is then converted to this secondary format.

[0047] In some examples, the knowledge base can be uploaded by users. For example, the knowledge base includes data that can be identified by the managing user 302 in a pre-existing location outside the platform (e.g., a first knowledge base) or uploaded by the managing user 302.

[0048] At box 312, the AI ​​platform stores the embedded model in an embedded model data repository. For example, the embedded model can be stored in the model data repository as a parameter of the model.

[0049] At box 320, the AI ​​platform receives a second knowledge base. For example, administrator user 302 requests the addition of a second knowledge base to the platform. When the second knowledge base is received, the AI ​​platform can transform the knowledge base into a multi-dimensional space associated with the embedded model.

[0050] At box 322, the AI ​​platform can deploy AI applications (e.g., solution accelerators for the AI ​​platform). In some examples, deploying an AI application may include deploying an identified machine learning model on a processor (e.g., a GPU). Deployment can use any metadata that has been identified during the registration process to aid in model deployment. In response to model deployment, other components of the AI ​​platform can interact with the model. In an example where the model is an LLM, the AI ​​platform can deploy the LLM, and a chatbot component can begin interacting with the LLM to generate responses to queries.

[0051] In some examples, administrators (users 302) can view available models registered with the AI ​​platform and select the model they want to implement. Viewing and selecting models can be done through the AI ​​platform's user interface.

[0052] At box 324, the AI ​​platform can register AI applications (e.g., solution accelerators that use the AI ​​platform). In some examples, registering an AI application may include identifying a machine learning model from a remote or cloud location. This process may also identify metadata associated with the model (e.g., the model's name, tags, a description of the training process, and other descriptions of the model).

[0053] In some examples, administrator user 302 can view available models downloaded using the AI ​​platform and select models for registration. The selected models can then be deployed to the AI ​​platform (box 322). Viewing and selecting models can be done through the AI ​​platform's user interface.

[0054] At box 330, the AI ​​platform can initiate an authentication process for administrator user 302. For example, administrator user 302 can provide login credentials (e.g., username and password). The user may have been previously logged in or been prompted for login credentials. In some examples, the user may be associated with an authentication token passed from the client device to the AI ​​platform and included using communication between the user and the platform.

[0055] The policy engine can define appropriate access levels for management users 302. For example, before an embedded model (box 310) or LLM (boxes 372, 374) needs to access a specific dataset, the platform can access the policy engine to confirm whether a user is permitted to access the data that the corresponding model will need to access. In other examples, access to the vector data repository (box 354) can be programmatically restricted and constrained.

[0056] The AI ​​platform can also initiate an authorization process for administrator user 302. Authorization for administrator user 302 can identify the access level that administrator user 302 should be able to access. The authorization process can also confirm that administrator user 302 can upload new knowledge base entries, be granted access to specific types of data, or provide the system with valid authorization / policy information to perform actions.

[0057] Multiple authorization processes can be implemented throughout the system, including uploading new knowledge bases, submitting queries via the AI ​​chatbot module, or other features that allow access to data or input of new data.

[0058] In the illustrative example, users from the marketing department of a client site can be granted access to data different from that of users associated with the human resources (HR) department. Appropriate segmentation strategies or permissions can be provided to the AI ​​platform's authorization process. When a user submits a query (box 360), the authorization level can be provided via the AI ​​chatbot module. Responses can be generated and filtered to include only information from the user's authorization level.

[0059] At box 332, the AI ​​platform executes the EzAIApp embedder. In some examples, the embedder may identify new knowledge bases or other data (e.g., S3 or cloud storage) and copy the data to local and temporary data repositories. In response to data being stored in the temporary data repository, the embedder may initiate an embedding process for the new data, and may also initiate a conversion of the embedding to a vector format to create new embeddings / vectors and store the new embeddings / vectors.

[0060] In some examples, administrator user 302 can interact with the interface to identify the knowledge base and models (e.g., location, metadata, and other information) to be implemented with the AI ​​platform.

[0061] In some examples, the AI ​​platform can identify models that have been registered and deployed as new data. These models are then available for use. The AI ​​platform can initiate the embedding process for registered / deployed models.

[0062] At box 334, the microservice manager provides methods for managing microservices in cloud-native applications. It offers a range of features for managing microservice communication (e.g., traffic management, load balancing, etc.), security (e.g., authentication, authorization, encryption, etc.), and observability (e.g., metrics, tracing, logging, etc.).

[0063] At box 336, the authorization engine can verify that the user requesting an update to the embedded model has the access credentials and policy permissions to make changes to the system.

[0064] At box 340, the AI ​​platform will embed documentation provided to the user's namespace. For example, the user's namespace could include a part of the AI ​​platform where tasks, services, and workloads are deployed in association with the user. The namespace could be Kubernetes. TM Logical partitioning within a cluster helps organize and manage the resources used to perform these processes for users. In some examples, a namespace data structure for each user can help partition cluster resources across multiple users or applications to help avoid scheduling and processing conflicts. Each namespace can have its own set of resources, such as pods, services, and deployments.

[0065] When the authentication and authorization process is completed, the AI ​​platform initiates Kubernetes. TM The task involves uploading the knowledge base to a user's namespace, making it accessible to users associated with that specific namespace. As shown in boxes 342, 344, 346, and 348, the upload can trigger an additional workflow as part of the process of creating a vector data repository at box 352.

[0066] In some examples, the AI ​​platform can also create embeddings of a second knowledge base. The generation of these second knowledge base embeddings allows data associated with the knowledge base to be stored in a format searchable / accessible to the rest of the platform.

[0067] At box 342, the AI ​​platform uses Kubeflow. TM The pipeline could be powered by Kubeflow. TMOther cloud-based jobs / workloads in component orchestration process embedded documents. The role of these jobs can be to execute software-based instructions that initiate the actual interaction with the embedding model to create the embedding. In some examples, Kubeflow... TM Components can also interact with vector repositories to store vectors.

[0068] At box 344, the AI ​​platform can initiate FS services to help integrate and manage data storage systems with Kubernetes. TM And the corresponding namespace. The FS service can run on each user namespace and mount user-specific volumes. The FS service can provide an application programming interface (API) to read and write user-specific volumes.

[0069] Associated with FS services, AI platforms, or Kubernetes TM Components can initiate persistent volumes (PVs), which represent a portion of a data repository that has been provisioned by an administrator or dynamically provisioned using storage classes. In another example, the platform can initiate persistent volume claims (PVCs), which are initiated via a request for a PV with specific storage requirements, such as size and access mode. Then, Kubernetes... TM Bind the PVC to the appropriate PV.

[0070] In another example, the platform can be on Kubernetes. TM Different types of storage classes are defined and dynamically provisioned within the cluster. Storage classes help manage different types of storage systems (e.g., SSDs, HDDs) and their configurations. In another example, the platform can implement various volume types, including EmptyDir (e.g., temporary storage created when a Pod is assigned to a node and exists as long as the Pod is running on that node), HostPath (e.g., enabling Pods to use files or directories on the host node's file system), Network File System (NFS) (e.g., enabling Pods to access NFS shares and access files shared across multiple Pods), or ConfigMap and Secret (e.g., storing configuration data and sensitive information that can be accessed by Pods as files or environment variables).

[0071] At box 346, the AI ​​platform can implement an S3 agent to interact with external services, such as Amazon S3 (Simple Storage Service). The S3 agent can integrate with external services to create persistent storage within a namespace. In some examples, the S3 agent can generate a generic S3 agent interface to connect to external S3 object repositories, thereby reading knowledge bases or other data from remote object repositories. The S3 agent can be implemented to remove the credential / authentication process from user interactions with the AI ​​platform (e.g., using its access token). In some examples, the S3 agent is previously authorized and can interact with external S3 object repositories instead of the user pipeline.

[0072] At box 348, the AI ​​platform can implement a distributed query engine (e.g., Presto) configured to run interactive analytical queries against various data sources, including embedded models (box 310). In some examples, the distributed query engine can also provide operators with graphs to help install and manage query data structures, thereby connecting to external structured data sources (e.g., MySQL databases or PostgreSQL). TM The database can be used to read structured data. This data may correspond to a customer knowledge base stored alongside the structured data source. The AI ​​platform can use this interface to read the knowledge base.

[0073] At box 350, the AI ​​platform can initiate an authorization process for administrator user 302 to upload the vector model (box 352) to the vector data repository (box 354). For example, the authorization for administrator user 302 can identify the access level of locations (e.g., namespaces, etc.) that administrator user 302 should be able to access, and confirm that administrator user 302 has access to the vector model.

[0074] At box 352, the AI ​​platform converts the embedded document into a vector representation. For example, the AI ​​platform stores a second knowledge base in a vector data repository (box 354). Many knowledge bases can be stored in the vector data repository in embedded formats.

[0075] At box 360, the AI ​​platform provides a user-accessible AI chatbot module or virtual assistant (which can be used interchangeably). The chatbot module can provide interfaces to interact with other pre-trained and pre-downloaded models available on the LLM or AI platform. In some examples, the chatbot module is accessible in a public cloud environment.

[0076] The AI ​​platform can receive queries from users via a generative AI chatbot module. Queries can be related to customer problems, offer solutions, or provide customized suggestions / solutions. In some examples, a user submits a query to the AI ​​chatbot module after encountering a 304 error. The AI ​​chatbot module can then forward the query to the authentication associated with the LLM (Local Management System).

[0077] At box 370, the AI ​​platform can initiate an authentication and authorization process for user 304. For example, when a user authenticates using login credentials (e.g., username and password), the AI ​​platform can initiate a first authentication process by comparing the login credentials with those of an authorized user who can access the LLM component. In the example implementing a token, the token can be sent along with the query and received by the LLM. The LLM can determine whether the login credentials or token are associated with a valid user. As a second authentication process, the AI ​​platform can determine whether the query from the user is permitted to ask questions about that particular solution.

[0078] A policy engine can help determine if a customer user has the appropriate access level to request the types of information to be included in a query. For example, a policy engine can implement policy-based access control (PBAC) to evaluate a user's location, role in the customer's environment, level, and other attributes to determine whether they should be granted access privileges. A policy engine can also implement Boolean logic to evaluate the legitimacy of access requests.

[0079] At box 372, the AI ​​platform can provide the query to the LLM converter. In this step, the LLM converter can weigh the importance of different words in the query when determining the response to the query.

[0080] At box 374, the AI ​​platform can use an LLM predictor to access a model data repository (box 376). In some examples, the LLM passes a query to an embedding model to access the appropriate embeddings. The AI ​​platform can access a vector data repository (box 354) to retrieve matching embeddings previously stored in the system. The matching embeddings are returned and used to generate a response to the query.

[0081] Figure 4 This is an illustrative communication process within an AI platform, illustrated in some examples of this disclosure. In Example 400, various components of the AI ​​platform 410 are provided, which may correspond to services, devices, or other components in a private cloud. In this illustration, the components include a knowledge base component 411, an embedding component 412, a vector component 413, a chatbot component 414, and an LLM component 415.

[0082] The knowledge base component 411 includes data that can be identified by the administrator in a pre-existing location outside the platform or uploaded by the administrator.

[0083] Embedding component 412 includes an embedding model that can transform a knowledge base or other data into a multidimensional space. In some examples, the embedding model is an externally available model outside the customer's private cloud environment (e.g., Word2Vec or GloVe). In some examples, the knowledge base or other data can be cleaned to conform to the requirements of the embedding model before being provided to embedding component 412. For text, data can be tokenized, converted to lowercase, and punctuation can be removed. For images, data can be resized, normalized, or otherwise transformed.

[0084] Vector component 413 includes representations of embeddings in a fixed-dimensional space. For example, the output of embedding component 412 could be a vector representing the input data / knowledge base. This vector could correspond to the final hidden state of a text embedding or a fully connected layer of an image embedding.

[0085] Chatbot component 414 includes an interface tool that receives queries from a user and provides output to the user. The interface tool can allow interaction between client devices at an AI platform (e.g., LLM), including input text fields, send buttons, context management (e.g., handling the continuity of the conversation by tracking interaction history), and error handling (e.g., components for managing and displaying errors).

[0086] LLM component 415 includes a large language model (LLM) that has been trained to understand and generate responses to queries. The LLM can analyze unstructured data and structured embeddings / vectors available in the AI ​​platform while generating responses.

[0087] At box 420, the knowledge base component 411 of the AI ​​platform 410 can receive a knowledge base. The knowledge base can be uploaded by an authorized management user or accessed from a knowledge base data repository in the customer's environment.

[0088] At box 422, the embedding component 412 of the AI ​​platform 410 can receive an embedded model. In some examples, the embedded model is received from an external data source.

[0089] At box 425, the embedding component 412 of the AI ​​platform 410 can receive a knowledge base and any other pre-existing customer data and generate their embeddings. The data repository can be accessible within the customer environment. The embedding component 412 can generate embeddings of data accessible within the customer environment.

[0090] At box 430, the vector component 413 of the AI ​​platform 410 can convert pre-existing customer data and knowledge base embeddings into vector representations. For example, in response to identifying a knowledge base, the knowledge base can be uploaded to the user's namespace and converted into vectors.

[0091] At box 432, vector component 413 can store the embedded vector representation in a vector repository.

[0092] At box 440, the chatbot component 414 of the AI ​​platform 410 receives a query from the user. The query can be received via a user interface component that displays the chatbot. In some examples, the chatbot may prompt the user to submit the query.

[0093] At box 442, the chatbot component 414 of the AI ​​platform 410 provides / submits a query to the LLM component 415. In some examples, the LLM may be a single LLM that is reused for multiple purposes and data types, although other implementations are possible without departing from the essence of this disclosure.

[0094] At box 444, the LLM component 415 of the AI ​​platform 410 accesses the data repository via the embedding component 412 (e.g., the embedded model). The data repository can be associated with responses to queries from users. In some examples, even if the LLM can be reused, the embedding of various data sources can be data type-specific.

[0095] At box 446, LLM component 415 accesses the vector data repository via vector component 413 of AI platform 410. LLM component 415 can initiate a similarity comparison between the embedding / vector representation and the features of the query. Relevant vector representations of the data can be retrieved based on the similarity association with the query.

[0096] As an illustrative example, embedding component 412 generates an embedding of the data requested by the query and submits the embedding (as a vector representation) to a vector repository to access previously stored matching embeddings. The LLM can match the query with the results identified by the stored vectors. The embeddings, vectors, and previously stored data can be used as part of a retrieval enhancement generation (RAG) process to return matched results.

[0097] At box 450, the LLM component 415 of the AI ​​platform 410 can generate a response to the query. For example, in response to the LLM processing the query, analyzing context and meaning to understand what was asked, and accessing appropriate data (e.g., embeddings / vectors), the LLM generates a response. In some examples, the LLM can predict and assemble words and phrases to form a coherent and relevant answer. The generated response can be formatted into a clear and readable format.

[0098] At box 455, the LLM component 415 of the AI ​​platform 410 can provide the generated response to the chatbot component 414 of the AI ​​platform 410 (e.g., viewable via the display of the client device).

[0099] It should be noted that the terms “optimization”, “optimal,” etc., used in this document can be used to mean producing or achieving the most efficient or perfect performance possible. However, as one of those skilled in the art who is reading this document will recognize, perfection is not always achievable. Therefore, these terms can also encompass producing or achieving the best possible, most efficient, or most practical performance under given conditions, or producing or achieving performance that is better than that achievable using other settings or parameters.

[0100] Figure 5 Computational components that can be used to implement genealogy-based classification of network events, according to various examples of the disclosed techniques, are shown. Reference is now made to... Figure 5 The computing component 500 can be, for example, a server computer, a controller, or any other similar computing component capable of processing data. Figure 5 In the example implementation, computing component 500 includes hardware processor 502 and machine-readable storage medium 504.

[0101] Hardware processor 502 may be one or more central processing units (CPUs), semiconductor-based microprocessors, and / or other hardware devices suitable for retrieving and executing instructions stored in machine-readable storage medium 504. Hardware processor 502 may retrieve, decode, and execute instructions, such as instructions 506-512, to control a process or operation of genealogy-based classification of network events. Alternatively or supplementary to retrieving and executing instructions, hardware processor 502 may include one or more electronic circuits comprising electronic components for the function of executing one or more instructions, such as field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), or other electronic circuits.

[0102] Machine-readable storage media (such as machine-readable storage media 504) can be any electronic, magnetic, optical, or other physical storage device that contains or stores executable instructions. Therefore, machine-readable storage media 504 can be, for example, random access memory (RAM), non-volatile RAM (NVRAM), electrically erasable programmable read-only memory (EEPROM), storage devices, optical discs, etc. In some examples, machine-readable storage media 504 can be a non-transient storage medium, where the term "non-transient" does not cover transiently propagating signals. As described in detail below, machine-readable storage media 504 can be encoded using executable instructions (e.g., instructions 506-512).

[0103] The hardware processor 502 can execute instructions 506 to receive customer data associated with a customer site and convert the customer data into an embedding using an embedding model.

[0104] The hardware processor 502 can execute instructions 508 to receive search queries from the client device.

[0105] The hardware processor 502 can execute instruction 510 to authenticate the client device.

[0106] The hardware processor 502 can execute instructions 512 to provide a search query to a large language model (LLM). In some examples, the search query can be provided to the LLM in response to an authenticated client device.

[0107] The hardware processor 502 can execute instruction 514 to initiate an authorization process that confirms that the client device is allowed to access search query data.

[0108] Hardware processor 502 can execute instructions 516 to access customer data that is stored as an embed, identified by the LLM. In some examples, access to the customer data can be in response to an authorization process.

[0109] The hardware processor 502 can execute instructions 518 to generate and provide responses to search queries based on customer data.

[0110] Figure 6 A block diagram of an example computer system 600 is depicted, illustrating various examples in which the disclosed technologies described herein can be implemented, including an AI platform with an embedded model, a vector data repository, a chatbot module, an LLM, and other components described herein. The computer system 600 includes a bus 602 or other communication mechanism for transmitting information, and one or more hardware processors 604 coupled to the bus 602 to process the information. For example, the hardware processors 604 may be one or more general-purpose microprocessors.

[0111] Computer system 600 also includes main memory 606, such as random access memory (RAM), cache, and / or other dynamic storage devices, coupled to bus 602 for storing information and instructions to be executed by processor 604. Main memory 606 can also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by processor 604. When stored in storage media accessible to processor 604, these instructions make computer system 600 a dedicated machine customized to perform the operations specified in the instructions.

[0112] The computer system 600 also includes a read-only memory (ROM) 608 or other static storage device coupled to the bus 602 for storing static information and instructions of the processor 604. A storage device 610, such as a disk, optical disk, or USB thumb drive (flash drive), is provided and coupled to the bus 602 for storing information and instructions.

[0113] Computer system 600 can be coupled to display 612, such as a liquid crystal display (LCD) (or touchscreen), via bus 602 for displaying information to a computer user. The display can provide the user with an AI chatbot module to interact with, and can also provide query results generated by LLM and other models. Based on the processes described throughout this disclosure, the displayed data may be limited to data that the user is authorized to access.

[0114] Computer system 600 may include a user interface module to provide a GUI to display 612. The user interface module may be stored as executable software code executed by computing devices(s). For example, this module and other modules may include components such as software components, object-oriented software components, class components and task components, processes, functions, properties, procedures, subroutines, program code segments, drivers, firmware, microcode, circuit systems, data, databases, data structures, tables, arrays, and variables.

[0115] Generally, the terms "component," "engine," "system," "database," and "data repository" used in this article can refer to logic embodied in hardware or firmware, or to a collection of software instructions that may have entry and exit points and are written in programming languages ​​such as Java, C, or C++. Software components can be compiled and linked into an executable program, installed in a dynamic link library, or written in an interpreted programming language such as BASIC, Perl, or Python. It should be understood that software components can be invoked from other components or themselves, and / or can be invoked in response to detected events or interrupts. Software components configured to execute on a computing device can be provided on computer-readable media (such as compressed discs, digital video discs, flash drives, disks, or any other tangible media) or as digital downloads (and may be initially stored in a compressed or installable format that requires installation, decompression, or decryption before execution). This software code can be stored, in part or in whole, on a memory device executing the computing device for execution by the computing device. Software instructions can be embedded in firmware such as EPROM. It should also be understood that hardware components may include connected logic units such as gating and flip-flops, and / or may include programmable units such as programmable gate arrays or processors.

[0116] Computer system 600 may implement the techniques described herein using custom hardwired logic, one or more ASICs or FPGAs, firmware, and / or program logic, which, when combined with the computer system, enable computer system 600 to become a special-purpose machine or to program computer system 600 as a special-purpose machine. According to one example of the disclosed techniques, the techniques herein are executed by computer system 600 in response to processor(s) 604 executing one or more sequences of one or more instructions contained in main memory 606. Such instructions may be read into main memory 606 from another storage medium (such as storage device 610). Execution of the sequence of instructions contained in main memory 606 causes processor(s) 604 to perform the processing steps described herein. In alternative examples, hardwired circuitry may be used in place of or in combination with software instructions.

[0117] As used herein, the term "non-transient medium" and similar terms refer to any medium that stores data and / or instructions that enable a machine to operate in a particular manner. Such non-transient media can include non-volatile media and / or volatile media. Non-volatile media include, for example, optical discs or magnetic disks, such as storage device 610. Volatile media include dynamic memory, such as main memory 606. Common forms of non-transient media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, magnetic tape, or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a perforated pattern, RAM, PROMs, and EPROMs, FLASH-EPROMs, NVRAMs, any other memory chips or cassette tapes, and their networked versions.

[0118] Non-transient media differ from transmission media, but can be used in conjunction with them. Transmission media participate in the transmission of information between non-transient media. For example, transmission media include coaxial cables, copper wires, and optical fibers, including the wires that constitute bus 602. Transmission media can also take the form of sound waves or light waves, such as those generated during radio wave and infrared data communication.

[0119] Computer system 600 also includes an interface 618 coupled to bus 602. Interface 618 provides bidirectional data communication coupled to one or more network links connected to one or more local area networks (LANs). For example, interface 618 may be an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem, or a modem for providing data communication connectivity to a corresponding type of telephone line. As another example, interface 618 may be a LAN card to provide data communication connectivity to a LAN-compatible network (or a WAN component communicating with a WAN). Wireless links may also be implemented. In any such implementation, interface 618 transmits and receives electrical, electromagnetic, or optical signals carrying streams of digital data representing various types of information.

[0120] Network links typically provide data communication to other data devices via one or more networks. For example, a network link can provide a connection to a host computer or a data device operated by an Internet Service Provider (ISP) via a local area network (LAN). The ISP then provides data communication services through the global packet data communication network now commonly referred to as the "Internet." Both LANs and the Internet use electrical, electromagnetic, or optical signals that carry digital data streams. Signals through various networks, signals on network links, and signals through interface 618 are examples of transmission media carrying digital data to and from computer system 600.

[0121] Computer system 600 can send messages and receive data (including program code) through (multiple) networks, network links, and interface 618. In the Internet example, the server can send application request codes through the Internet, ISP, local area network, and interface 618.

[0122] The received code may be executed by processor 604 when it is received, and / or stored in storage device 610 or other non-volatile storage device for later execution.

[0123] Each of the processes, methods, and algorithms described in the preceding sections can be embodied in code components executed by one or more computer systems or computer processors including computer hardware, and can be fully or partially automated by these code components. One or more computer systems or computer processors can also operate to support the execution of related operations in a “cloud computing” environment or as “Software as a Service” (SaaS). Processes and algorithms can be implemented partially or entirely in a dedicated circuit system. The various features and processes described above can be used independently of each other or combined in various ways. Different combinations and sub-combinations are intended to fall within the scope of this disclosure, and certain method or process boxes may be omitted in some implementations. The methods and processes described herein are not limited to any particular order, and the boxes or states associated with them can be executed in other suitable orders, or can be executed in parallel, or in some other way. Boxes or states can be added to or removed from the disclosed examples. The execution of certain operations or processes within an operation or process can be distributed among computer systems or computer processors, residing not only within a single machine but also deployed across multiple machines.

[0124] As used herein, circuits can be implemented using any form of hardware, software, or a combination thereof. For example, one or more processors, controllers, ASICs, PLAs, PALs, CPLDs, FPGAs, logic components, software routines, or other mechanisms can be used to compose the circuit. In implementation, the various circuits described herein can be implemented as discrete circuits, or the described functions and features can be shared partially or wholly among one or more circuits. Although elements of various features or functions may be described separately or claimed as separate circuits, these features and functions can be shared among one or more common circuits, and such description should not require or imply the need for separate circuits to implement these features or functions. In cases where circuits are implemented wholly or partially using software, such software can be implemented to operate in conjunction with a computing or processing system (such as computer system 600) capable of performing the functions described herein.

[0125] As used herein, the term “or” can be interpreted in the sense of inclusion or exclusion. Furthermore, descriptions of resources, operations, or structures in the singular form should not be interpreted as excluding the plural form. Unless explicitly stated otherwise, or otherwise understood in the context in which they are used, conditional language (such as “can,” “could,” “might,” or “may”) is generally intended to convey that some examples include certain features, elements, and / or steps, while other examples do not.

[0126] Unless otherwise expressly stated, the terms and phrases used in this document, and their variations thereof, should be interpreted as open-ended, not restrictive. Adjectives such as “regular,” “traditional,” “normal,” “standard,” “known,” and terms with similar meanings should not be interpreted as limiting the described items to items available at a given time or period, but rather should be understood to cover regular, traditional, normal, or standard techniques that may be available or known at any time now or in the future. In some cases, the presence of broad words and phrases such as “one or more,” “at least,” “but not limited to,” or other similar phrases should not be interpreted as implying an intention or need for a narrower scope in situations where such broad phrases might not be present.

Claims

1. A computer-implemented method comprising: receiving customer data associated with a customer site and converting the customer data to an embedding using an embedding model; receiving a search query from a client device; authenticating the client device; in response to authentication, providing the search query to a large language model (LLM); initiating an authorization process that confirms that the client device is permitted to access the customer data accessible by the search query; in response to the authorization process, accessing the customer data identified by the LLM, the customer data stored as the embedding; and generating and providing a response to the search query based on the customer data.

2. The computer-implemented method of claim 1, wherein the LLM is reused for different search queries and for different use cases.

3. The computer-implemented method of claim 1, wherein the embedding model, the LLM, and the authorization process are implemented in a private cloud stored at the customer site, and the client device is configured to access the private cloud.

4. The computer-implemented method of claim 1, further comprising: storing the embedding in an embedding data store; and storing a vector representation of the embedding in a vector data store.

5. The computer-implemented method of claim 1, wherein the search query is received from a chatbot module accessed by the client device.

6. The computer-implemented method of claim 1, further comprising: receiving the embedding model at the customer site; and automatically accessing the customer data at the customer site once the embedding model is activated.

7. The computer-implemented method of claim 1, further comprising: receiving a knowledge base from an administrative user prior to the search query being received; converting the knowledge base to a second embedding using the embedding model; and storing the second embedding in a vector data store at the customer site.

8. The computer-implemented method of claim 7, wherein an automated processing job uploads the second embedding to a namespace associated with the client device.

9. The computer-implemented method of claim 7, wherein the embedding accessed by the LLM is stored in a vector data store.

10. A private cloud platform comprising: a memory storing instructions; and a processor communicatively coupled to the memory and configured to execute the instructions to: receive customer data associated with a customer site and convert the customer data to an embedding using an embedding model; receive a search query from a client device; authenticate the client device; in response to authentication, provide the search query to a large language model (LLM); initiate an authorization process that confirms that the client device is permitted to access the customer data accessible by the search query; in response to the authorization process, access the customer data identified by the LLM, the customer data stored as the embedding; and generate and provide a response to the search query based on the customer data. ​ ​ ​ ​ ​ generating and providing a response to the search query based on the customer data.

11. The private cloud platform of claim 10, wherein the LLM is reused for different search queries and for different use cases.

12. The private cloud platform of claim 10, wherein the embedding model, the LLM, and the authorization process are implemented in a private cloud stored at the customer site, and the client device is configured to access the private cloud.

13. The private cloud platform of claim 10, wherein the processor is further configured to: store the embedding in an embedding data store; and store the vector representation of the embedding in a vector data store.

14. The private cloud platform of claim 10, wherein the search query is received from a chatbot module accessed by the client device.

15. The private cloud platform of claim 10, wherein the processor is further configured to: receive the embedding model at the customer site; and automatically access the customer data at the customer site once the embedding model is activated.

16. The private cloud platform of claim 10, wherein the processor is further configured to: receive a knowledge base from an administrative user prior to the search query being received; convert the knowledge base to a second embedding using the embedding model; and store the second embedding in a vector data store at the customer site.

17. The private cloud platform of claim 16, wherein an automated processing job uploads the second embedding to a namespace associated with the client device.

18. A non-transitory computer-readable storage medium storing a plurality of instructions executable by a processor, the plurality of instructions when executed by the processor cause the processor to: receive customer data associated with a customer site and convert the customer data to an embedding using an embedding model; receive a search query from a client device; authenticate the client device; in response to authentication, provide the search query to a large language model (LLM); initiate an authorization process that confirms that the client device is allowed to access the customer data accessible by the search query; in response to the authorization process, access the customer data identified by the LLM, the customer data stored as the embedding; and generate and provide a response to the search query based on the customer data.

19. The non-transitory computer-readable storage medium of claim 18, wherein the LLM is reused for different search queries and for different use cases.

20. The non-transitory computer-readable storage medium of claim 18, wherein the embedding model, the LLM, and the authorization process are implemented in a private cloud stored at the customer site, and the client device is configured to access the private cloud.

Images