Method and system for efficient encryption of fine tuned large language models

Abstract

A method and system to provide secure fine-tune training and inferences for large language models using a Fully Homomorphic Encryption (FHE) process is disclosed. A trained public large language model with a matrix of public weights is stored in plaintext. A private training set of proprietary data is stored as a set of encrypted weights. An encrypted query is received. The encrypted query is processed via an encrypted data path with the set of encrypted weights for trained large language model. The encrypted query is processed via a plaintext data path with the set of public weights in plaintext for the trained large language model. A low rank adaptation (LoRA) approach may be used in fine-tuning the model to reduce computational complexity and memory access. Encrypted results are obtained from the processing via the encrypted data path and the plaintext weight path.

Description

METHOD AND SYSTEM FOR EFFICIENT ENCRYPTION OF FINE TUNED LARGE LANGUAGE MODELSPRIORITY CLAIM

[0001] The present disclosure claims priority to and benefit of U.S. Provisional Patent Application No. 63 / 591,713, filed October 19, 2023. The contents of that application are hereby incorporated by reference in their entirety.TECHNICAL FIELD

[0002] The present disclosure relates generally to security applications. More particularly, aspects of this disclosure relate to techniques to protect proprietary large language models and data with efficient encryption.BACKGROUND

[0003] Large Language Models (LLMs), also known as generative Artificial Intelligence (Al), transformers, or Natural Language Processing (NLP), have now become, in a very short space of time, a foundational artificial intelligence technique. They provide human level intelligence across the knowledge base they are trained upon. The key metric, which is proportional to the amount of knowledge they hold, is the number of parameters or weights that they hold. Very large models start in the low billions of parameters moving up to the low trillions of parameters. The training sets on the largest models can comprise the entirety of the accessible internet.

[0004] Using existing pretrained LLMs as a starting point it is possible to perform additional training using very specific knowledge to produce LLMs that are experts in the narrow knowledge domain. This process of taking an existing LLM and continuing its training is known as fine-tuning. Fine tuning involves taking a pre-existing model that has been trained on a large dataset, such as a language model like GPT-3, and refining it for a specific task or domain. During fine-tuning, the model is further trained on a smaller, domain-specific dataset. This process adapts the model parameters to the nuances of the target task, thus improving performance and making the model more capable in handling specific tasks. Fine-tuning is a cost-effective and efficient way to leverage the knowledge learned by a pre-trained model while tailoring the model to specific applications, reducing the need for extensive training from scratch. Since the most compute intensive operations have already been performed in the development of the initial model, the fine-tuning process only requires a small percentage of the weights to be modified to incorporate the additional, expert level information. Typically,only 1 / 100 to 1 / 10,000 of the exiting parameters need to be modified. This has proven to be a game-changer in many information processing tasks, allowing for rapid development of domain specific Al solutions with high accuracy and applicability.

[0005] Training and inference are two crucial phases in the lifecycle of Large Language Models (LLMs), such as GPT-3. These models are pre-trained on vast corpora of text data and then fine-tuned for specific tasks before they can be effectively deployed for real-world applications.

[0006] The training of LLMs is a computational resource-intensive process that typically involves two main steps: pre-training, which requires the majority of the computational resources and fine-tuning. During pre-training, the model is exposed to a massive amount of text from the internet, learning to predict the next word in a sentence. This helps the model acquire a vast amount of world knowledge and linguistic patterns. The training process involves updating billions of model parameters using powerful computational resources, typically state of the art NVIDIA GPUs, which can take days, weeks or even months to complete. Following pre-training, fine-tuning is performed on a narrower dataset with labeled examples for a specific task, such as a company’s intellectual property. Fine-tuning adapts the model parameters to the target task, making it more effective and contextually relevant. Fine- tuning is a critical step that tailors the LLM for practical applications taking from human level to expert level intelligence.

[0007] Once an LLM has been trained and fine-tuned, the trained and fine-tuned LLM can be used for inference, which involves making predictions or generating text for specific tasks. During inference, input data, typically in the form of text, is fed into the model. The model processes the input, generates output, and provides predictions or text generation. Inference can be performed both in real-time applications and non-real-time applications. Real-time applications are sensitive to latency, such as chatbots, language translation services, content generation, and more. Non-real-time applications, such as batch processing of document, information, software, hardware designs and the like are not sensitive to latency. The LLMs are deployed on high computational powered cloud servers in data centers, or in a private data center of a company, which enables deployment to all the employees of the company, software as a service to paying customers, or even access on the general Internet. Inference with LLMs is revolutionizing industries by automating tasks that previously required a human expert.

[0008] The issue with producing expert-level fine-tuned LLM is that they, by definition, must include valuable proprietary information. The entirety of expertise, intellectual property, knowledge base, trade secrets, and confidential information of a company from inception tothe present time can be incorporated into a fine-tuned LLM. Thus, there are four existing problems with fine-tuned LLMs. Two of these problems are on the inference portion of the fine-tuned LLM. First, from the perspective of the owner of the fine-tuned LLM model, the theft of one of these fine-tuned LLM is catastrophic. The LLM can enable competitors to produce products or offer services that compete with the owners’ business. Alternatively, if the LLM is simply released on the Internet, the revenue of the original company may be driven to zero. Thus, there is a need to prevent the use by unauthorized third parties of a fine-tuned LLM.

[0009] Second, a user of these LLMs is providing queries to the fine-tuned LLM and receiving results from these queries. The queries may contain intellectual property of the user and the answers to these queries may contain new and novel intellectual property that the owner of the fine-tuned LLM now has access to. There is a need to prevent the leakage of intellectual property of a user into the LLM from the queries as well as protect any new and novel intellectual property in the responses form the LLM.

[0010] The training portion of the fine-tuned LLM also presents two problems. First, the training of the fine-tuned parameters is typically performed using plain-text or human readable data. The fine-tuned parameters must be protected from theft or disclosure by either external parties or even internal employees during this entire process. There is a need to protect against the theft of the plaintext LLM data as it is undergoing fine-tuning. Second, the information that is used for the fine-tuning process may be highly sensitive, proprietary, classified or protected under privacy laws such as the European GDPR rules or the HIPP A rules in the US.

[0011] Thus, current solutions to these problems require encryption of the weights of the LLM as well as the inputs to the LLMs. The encryption protects the valuable weights as well as input inquires and responses.

[0012] Currently, encryption techniques relate to public / private key mechanisms that require an intensive level of computing power to brute force solve the encryption. Such systems are currently secure because of the corresponding intensive level of computing power necessary to solve such encryption. However, with the advent of potential quantum computers, standard encryption techniques may be vulnerable to being solved by a quantum computer. Thus, new types of quantum secure encryption have been proposed, such as fully homomorphic encryption (FHE). FHE allows computations on ciphertext without having to perform decryption. This allows delegation of sensitive data analysis computations on encrypted data. The FHE allows computations such a Boolean operation, Integer arithmetic operation, Floating-point arithmetic operation on ciphertext without decryption. Thus, sensitive dataanalysis (computations) may be performed on encrypted data without ever decrypting the data. There are several open-source frameworks of fully homomorphic encryption one such framework is the Concrete library that implements the Fully Homomorphic Encryption over the Torus (TFHE) procedure. A second such framework is the OpenFHE framework which supports multiple schemes including BGV, BFV, CKKS, TFHE, and FHEW.

[0013] The Concrete library is an open-source library developed in Rust that builds on the state-of-art TFHE cryptosystem. The Concrete library provides a user friendly interface making FHE easy to integrate. The Concrete library deals with inputs of arbitrary format and comes with an extensive set of operations for manipulating ciphertexts, including a programmable bootstrapping process. Learning With Errors (LWE) is a quantum robust method of cryptography applicable to FHE that is conjectured to be hard to solve, and thus is useful in cryptography.

[0014] There is a need to be able to perform training of a LLM without exposing the training information during the fine-tuning process. There is a further need to incorporate quantum proof encryption in the training of an LLM without burdensome computational requirements. There is also a need to protect the weights of a proprietary fine-tuned LLM via encryption.SUMMARY

[0015] The term embodiment and like terms, e.g., implementation, configuration, aspect, example, and option, are intended to refer broadly to all of the subject matter of this disclosure and the claims below. Statements containing these terms should be understood not to limit the subject matter described herein or to limit the meaning or scope of the claims below. Embodiments of the present disclosure covered herein are defined by the claims below, not this summary. This summary is a high-level overview of various aspects of the disclosure and introduces some of the concepts that are further described in the Detailed Description section below. This summary is not intended to identify key or essential features of the claimed subject matter. This summary is also not intended to be used in isolation to determine the scope of the claimed subject matter. The subject matter should be understood by reference to appropriate portions of the entire specification of this disclosure, any or all drawings, and each claim.

[0016] According to certain aspects of the present disclosure, an example method to provide secure training of a fine-tuned large language model is disclosed. A trained public large language model is stored as a matrix of public weights in plaintext in a memory. A private training set of proprietary data is stored in memory. A matrix of encrypted private weights isloaded. The public large language model is fine-tuned using the private training set. The matrix of encrypted private weights is updated from the fine-tuning.

[0017] A further implementation of the example method is where the encrypted private weights are reduced by low rank adaptation (LoRA). Another implementation is where the matrix of private weights is smaller than the matrix of public weights. Another implementation is where the encrypted private weights are stored as a fully connected encrypted matrix. Another implementation is where the encryption is performed via a Fully Homomorphic Encryption (FHE) process.

[0018] Another example training system to provide secure training of a fine-tuned large language model using a Fully Homomorphic Encryption (FHE) process is disclosed. A memory stores a trained public large language model as a matrix of public weights in plaintext and a private training set of proprietary data as a matrix of encrypted private weights. A processor is coupled to the memory. The processor is operable fine-tune the public large language model by the private training set. The processor updates the matrix of encrypted private weights in the memory from the fine tuning.

[0019] According to certain aspects of the present disclosure, an example method to provide secure inference of a fine-tuned large language model using encryption is disclosed. A trained public large language model with a matrix of public weights in plaintext is stored in a memory. A set of encrypted weights obtained from the fine-tuned large language model with a training set of proprietary data is stored in the memory. An encrypted query is received via a data link. The encrypted query is processed via an encrypted data path with the set of encrypted weights for the trained large language model. The encrypted query is processed via a plaintext data path with the set of public weights in plaintext for the trained large language model. An encrypted result is produced from the processing via adding the encrypted data path and the plaintext data path.

[0020] A further implementation of the example method is where the encrypted private weights are reduced by low rank adaptation (LoRA). Another implementation is where the encrypted private weights are stored as a fully connected encrypted matrix. Another implementation is where the encryption is performed via a Fully Homomorphic Encryption (FHE) process.

[0021] According to certain aspects of the present disclosure, an example method to provide secure inference of a fine-tuned large language model using encryption is disclosed. The query is encrypted and the encrypted query is provided to a fine-tuned foundation layer of a large language model. The fine-tuned foundation layer includes an encrypted matrix of proprietary weights. The encrypted query is provided to a public foundation layer of a large languagemodel, where the public foundation layer includes a plaintext matrix of public weights. The outputs of the fine-tuned foundation layer and the outputs of the public foundation layer are added to provide an encrypted inference output.

[0022] Another implementation of the example method is where the encryption is performed via a Fully Homomorphic Encryption (FHE) process.

[0023] According to certain aspects of the present disclosure, an example system to output a response to query is disclosed. The system includes a first processing core configured to receive the query. A second processing core is configured as a fine-tuned foundation layer of a large language model that accepts the encrypted query, wherein the fine-tuned foundation layer includes an encrypted matrix of proprietary weights. A third processing core configured as a public foundation layer of a large language model that accepts the encrypted query. The public foundation layer includes a plaintext matrix of public weights. A fourth processing core is configured to add the outputs of the fine-tuned foundation layer and the outputs of the public foundation layer to provide an encrypted inference output. A network is coupled to each of the processing cores and allows communication among each of the processing cores.

[0024] A further implementation of the example system is where the first, second, third and fourth processing cores include at least one of a Reduced Instruction Set Computing (RISC) processor core, a special purpose processing core; or a RISC processor core with a more special purpose processing cores embedded within. Another implementation is where the first, second, third and fourth processing cores are in an array of RISC processor cores interconnected with an array of special purpose processing cores. Another implementation is where the first, second, third and fourth processing cores are part of one or more arrays of processing cores. Another implementation is where the encryption is performed via a Fully Homomorphic Encryption (FHE) process.

[0025] According to certain aspects of the present disclosure, an example non-transitory computer readable medium having stored thereon instructions is disclosed. The instructions, when executed by a processor unit, cause the processor unit to encrypt a query and provide the encrypted query to a fine-tuned foundation layer of a large language model. The fine-tuned foundation layer includes an encrypted matrix of proprietary weights. The instructions cause the processor unit to provide the encrypted query to a public foundation layer of a large language model. The public foundation layer includes a plaintext matrix of public weights. The instructions cause the processor to add the outputs of the fine-tuned foundation layer and the outputs of the public foundation layer to provide an encrypted inference output.

[0026] A further implementation of the example computer readable medium is where the processor unit includes a plurality of configurable processing cores. Each of the configurable processing cores is configured to perform at least one of the instructions. Another implementation is where the encryption is performed via Fully Homomorphic Encryption (FHE).

[0027] The above summary is not intended to represent each embodiment or every aspect of the present disclosure. Rather, the foregoing summary merely provides an example of some of the novel aspects and features set forth herein. The above features and advantages, and other features and advantages of the present disclosure, will be readily apparent from the following detailed description of representative embodiments and modes for carrying out the present invention, when taken in connection with the accompanying drawings and the appended claims. Additional aspects of the disclosure will be apparent to those of ordinary skill in the art in view of the detailed description of various embodiments, which is made with reference to the drawings, a brief description of which is provided below.BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The disclosure will be better understood from the following description of exemplary embodiments together with reference to the accompanying drawings, in which:

[0029] FIG. 1 A is a diagram of a chip having four dies each having multiple processing cores;

[0030] FIG. IB is a simplified diagram of one of the dies on the chip shown in FIG. 1 A;

[0031] FIG. 2A is a block diagram of the array of cores in the die in FIG. IB;

[0032] FIG. 2B is a three-dimensional view of the array of cores in the die in FIG. IB;

[0033] FIG. 3 A is a configuration for one of the cores in the example core array in FIG. 2B;

[0034] FIG. 3B is a set of configurations for multiple cores in the example core array in FIG. 2B;

[0035] FIG. 4 is an example process for using Low-Rank Adaptation of Large Language Models (LoRA);

[0036] FIG. 5A is a high level diagram of a single layer plaintext fine-tuned model according to known methods;

[0037] FIG. 5B is a high level diagram of a single layer, encrypted fine-tuned model showing the example method to efficiently protect fine-tuned LLMs;

[0038] FIG. 6A shows a mathematical representation of a prior art LLM mapping system;

[0039] FIG. 6B shows a mathematical representation of a private prior art LLM mapping system;

[0040] FIG. 7 is a table showing the computations necessary for the prior art systems in FIGs. 6A-6B;

[0041] FIG. 8 is a block diagram showing reduction of a matrix of fine-tuning weights using a low-rank adaptation algorithm by using two lower ranked matrices;

[0042] FIG. 9 is a flow diagram of the example method of efficient encrypted training for a fine-tuned large language model; and

[0043] FIG. 10 is a flow diagram of an example encrypted inference routine performed by the example fine-tuned large language model.

[0044] The present disclosure is susceptible to various modifications and alternative forms. Some representative embodiments have been shown by way of example in the drawings and will be described in detail herein. It should be understood, however, that the invention is not intended to be limited to the particular forms disclosed. Rather, the disclosure is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims.DETAILED DESCRIPTION

[0045] The present inventions can be embodied in many different forms. Representative embodiments are shown in the drawings, and will herein be described in detail. The present disclosure is an example or illustration of the principles of the present disclosure, and is not intended to limit the broad aspects of the disclosure to the embodiments illustrated. To that extent, elements, and limitations that are disclosed, for example, in the Abstract, Summary, and Detailed Description sections, but not explicitly set forth in the claims, should not be incorporated into the claims, singly, or collectively, by implication, inference, or otherwise. For purposes of the present detailed description, unless specifically disclaimed, the singular includes the plural and vice versa; and the word “including” means “including without limitation.” Moreover, words of approximation, such as “about,” “almost,” “substantially,” “approximately,” and the like, can be used herein to mean “at,” “near,” or “nearly at,” or “within 3-5% of,” or “within acceptable manufacturing tolerances,” or any logical combination thereof, for example.

[0046] The present disclosure is directed toward an effective security solution for implementing private large-language-models (LLM) systems. The example method is directed toward protecting the LLM through encryption such as FHE, using plaintext for the publicly known weights, but protecting all other LLM components, including any proprietary weights for fine tuning through an encryption scheme such as FHE. The use of plaintext reduces thecomputation requirements for protecting the LLM through computationally expensive encryption techniques such as FHE.

[0047] The example method allows a client to send their protected data in ciphertext format to the service provider and then to receive the desired results processed by the LLM service provider in ciphertext format. The ciphertext format allows protection from the service provider being able to read the protected private data of the client. At the same time, the example method and system also allows the service provider to keep all private modeling parameters and information of LLM fully protected through the inference service stage. This fully prevents the client from obtaining the private model parameters and information provided by the service provider. By using the low-rank adaption based fine-turning algorithm on ciphertext data, the example method can reduce the computational complexity more than 100 times for performing private LLM inference and training. The great reduction of computational complexity provided by the example method makes real-time implementation of private LLM inference practical to protect the LLM and the input data.

[0048] The example method and system thus solves the four existing problems with fine-tuned LLMs, namely: 1) protecting the proprietary information of an owner of a fine-tuned LLM; 2) protecting the proprietary information of a user of the fine-tuned LLM; 3) protecting proprietary fine-tuning parameters in a training data set for an LLM; and 4) protecting training information when training a fine-tuned LLM.

[0049] FIG. 1 A shows an example chip 100 that may implement the example fine-tuned LLM, which is subdivided into four identical dies 102, 104, 106, and 108. Each of the dies 102, 104, 106, and 108 include multiple processor cores, support circuits, serial interconnections and serial data control subsystems. For example, the dies 102, 104, 106, and 108 may each have 4,096 processing cores as well as serializer / deserializer (SERDES) interconnection lanes to support different communication protocols. There are die-to-die parallel connections between the dies 102, 104, 106 and 108. Thus, each of the dies 102, 104, 106, and 108 in this example are interconnected by Interlaken connections. The chip 100 is designed to allow one, two or all four of the dies 102, 104, 106, and 108 to be used. The pins on a package related to un-used dies are left unconnected in the package or the board. The dies are scalable as additional chips identical to the chip 100 may be implemented in a device or a circuit board. In this example, a single communication port such as an Ethernet port is provided for the chip 100. Of course, other ports may be provided, such as one or more ports for each die.

[0050] FIG. IB is a block diagram of one example of the die 102. The die 102 includes a fractal array 130 of processing cores. The processing cores in the fractal array 130 areinterconnected with each other via a system interconnect 132. The entire array of cores 130 serves as the major processing engine of the die 102 and the chip 100. In this example, there are 4096 cores in the fractal array 130 that are organized in a grid.

[0051] The system interconnection 132 is coupled to a series of memory input / output processors (MIOP) 134. The system interconnection 132 is coupled to a control status register (CSR) 136, a direct memory access (DMA) 138, an interrupt controller (IRQC) 140, an I2C bus controller 142, and two die to die interconnections 144. The two die to die interconnections 144 allow communication between the array of processing cores 130 of the die 102 and the two neighboring dies 104 and 108 in FIG. 1A.

[0052] The chip includes a high bandwidth memory controller 146 coupled to a high bandwidth memory 148 that constitute an external memory sub-system. The chip also includes an Ethernet controller system 150, an Interlaken controller system 152, and a PCIe controller system 154 for external communications. In this example each of the controller systems 150, 152, and 154 have a media access controller, a physical coding sublayer (PCS) and an input for data to and from the cores. Each controller of the respective communication protocol systems 150, 152, and 154 interfaces with the cores to provide data in the respective communication protocol. In this example, the Interlaken controller system 152 has two Interlaken controllers and respective channels. A SERDES allocator 156 allows allocation of SERDES lines through quad M-PHY units 158 to the communication systems 150, 152 and 154. Each of the controllers of the communication systems 150, 152, and 154 may access the high bandwidth memory 148.

[0053] In this example, the array 130 of directly interconnected cores are organized in tiles with 16 cores in each tile. The array 130 functions as a memory network on chip by having a high-bandwidth interconnect for routing data streams between the cores and the external DRAM through memory input / output processors (MIOP) 134 and the high bandwidth memory controller 146. The array 130 functions as a link network on chip interconnection for supporting communication between distant cores including chip-to-chip communication through an “Array of Chips” Bridge module. The array 130 has an error reporter function that captures and filters fatal error messages from all components of array 130.

[0054] FIG. 2A is a detailed diagram of the array of cores 130 in FIG. IB. FIG. 2B is a three- dimensional image of the array of cores 130 in FIG. 2B. The array of cores 130 is organized into four core clusters such as the clusters 200, 210, 220, and 230 shown in FIG. 3 A. For example, the cluster 200 includes cores 202a, 202b, 202c, and 202d. Each of the four cores in each cluster 200 such as cores 202a, 202b, 202c, and 202d are coupled together by a router204. FIG. 3B shows other clusters 210, 220, and 230 with corresponding cores 212a-212d, 222a-222d and 232a-232d and corresponding routers 214, 224, and 234.

[0055] As may be seen specifically in FIG. 2B, in this example, each of the cores 202a, 202b, 202c, and 202d has up to four sets of three interconnections [L, A, R], For example, a core in the center of the array such as the core 202d includes four sets of interconnections 240, 242, 244, and 246 each connected to one of four neighboring cores. Thus, core 202b is connected to the core 202d via the interconnections 240, core 202c is connected to the core 202d via the interconnections 242, core 212b is connected to the core 202d via the interconnections 244, and core 202c is connected to the core 202d via the interconnections 246. A separate connector 248 is coupled to the wire router 204 of the cluster 200. Thus, each core in the middle of the array, has four sets of interconnections, while border cores such as the core 202c only have three sets of interconnections 250, 252, and 246 that are connected to respective cores 202a, 202d, and 212a. In order to configure the cores of the example array of processing cores 130 in FIG. 2A, the inputs of certain blocks may be changed to configure blocks for one of the three different function blocks. The functions may be configured by simply changing the inputs of the processing cores.

[0056] FIG. 3A shows a block diagram of an example processing core 300 that includes a reconfigurable arithmetic engine (RAE) 310. The RAE 310 may be configured and reconfigured to perform relevant mathematical routines such as matrix multiplications, Fast Fourier Transforms (FFT), Inverse FFTs (IFFT), point wise multiplication, Softmax and other related nonlinear functions in the LLM. The RAE 310 includes input reorder queues, a multiplier shifter-combiner network, an accumulator and logic circuits. The RAE 310 operates in several modes, such as operating as an ALU, and include a number of floating point and integer arithmetic modes, logical manipulation modes (Boolean logic and shift / rotate), conditional operations, and format conversion. In this example, the RAE 310 may process streaming data with floating point or integer arithmetic and Boolean logic. The RAE 310 includes comparison logic that can set exception flags, be used to gate data flow, or substitute data based on compare results. The RAE 310 supports mixed precision at the individual core level.

[0057] The RAE 310 includes three inputs 312, 314, and 316 and three outputs 322, 324, and 326. The RAE 310 receives the output data from a program executed by another RAE 330 and output data from another program executed by another RAE 332. An aggregator (AGG) 334 provides an output of aggregated data from different sources to the RAE 310. A memory read output 336 and a memory write output 338 also provide data to the RAE 310. The memoryoutputs 336 and 338 provide access to a memory such as an SRAM that stores operand data, and optionally may also store configurations or other instructions for the RAE 310.

[0058] Each of the output data of the RAE 330, RAE 332, aggregator 334, memory read output 336 and the memory write output 338 are provided as inputs to three multiplexers 342, 344, and 346. The outputs of the respective multiplexers 342, 344, and 346 are coupled to the respective inputs 312, 314, and 316 of the RAE 310.

[0059] There are two versions of configuration of computational cores which can dynamically switch from one type to the other. A set of cores may be configured as a full RISC-V processor with associated SRAM able to execute traditional control flow programs as a function representing the computation within a dataflow node. RISC-V for Legacy code is supported by configuring multiple cores under software control. This may be used to produce software GPUs or other types of cores from the multiple cores. The processing cores such as the FracTLcores offered by Comami are an efficient set of transistors for streaming data driven workloads, with a dynamic programming scheduler such as the TruStream programming scheduler offered by Cornami and memory, created from a set of RAE Cores. In this example, the FracTLcores can scale up to 64,000,000 cores across chips and systems at near linear scale. The use of the architecture of processing cores results in reduction in processing cost. The cores may employ a Data-Flow Programming Model resulting in a 5x reduction in processing cost. A Data-Defining-Function Computation for the cores may result in a 6x reduction in processing cost. A data Read / Write with a Tensor pattern applied to the cores may result in a 6x reduction in processing cost.

[0060] FIG. 3B is a diagram of four configurations 350, 360, 370, and 380 of the array of cores in FIG. 2B as either a RISC-V processor or a specialized ALU internal module. The configurations 350, 360, 370, and 380 can dynamically switch from one type to the other by reconfiguring some or all of the computational cores in the configurations. The first configuration 350 is a set of cores configured as a full RISC processor 352 with associated SRAM able to execute traditional Control Flow programs as a function representing the computation within a dataflow node. In this example, the RISC processor 352 includes sixteen separate cores. Another configuration 360 is sixteen independently reconfigurable and programmable ALUs, that are each cores 362 (termed FracTLcores® available from Comami in this example). Each of the cores 362 have associated SRAM supporting multiple simultaneous integer and floating point computations of up to 128-bits. The configuration 360 thus is a set of cores that are configured as individual FracTLcores. The configuration 370 includes one or more RISC cores 372 that are a set of sixteen cores in this example. The RISCcore 372 can have additional individual or multiple FracTLcores 374 incorporated within them to accelerate specific RISC functions. Alternatively, the additional cores 374 may be designated for data path / arithmetic acceleration, enhancing ALU performance. Thus, to implement a standard 64 bit RISC processor such as the RISC-V processor 352 in this example, sixteen cores are configured to become the RISC-V. Optional additional cores may be added to the configuration to provide hardware acceleration to math operations performed by the RISC. For example, a normal RISC processor does not have hardware to perform a cosine function. Thus, an additional core may be added and configured to perform a hardware cosine operation. This enhances the ISA instruction set of the RISC processor by adding the hardware accelerated cosine function that may be accessed by the RISC processor. The configuration 380 has a set of cores that is configured into two individual groupings of cores configured as RISC processors 382 and cores that are configured as ALUs (e.g., FracTLcores) 384.

[0061] The combination of the fractal core architecture with components shown in FIGs. 1 A- 3B may be deployed for the example secure training of LLMs. At the top level, a computational fabric is composed of software defined cores in FIG. 3B that include one or more of the fractal core arrays in FIGs. 1A-2B. Each of the individual cores in the fractal core arrays may be configured into a reconfigurable arithmetic engine shown in FIG. 3A. The software defined cores may thus be scaled up to different chip architectures.

[0062] “Low-Rank Adaptation of Large Language Models” (LoRA) is a specific technique in the field of LLMs. LoRA focuses on adapting large language models, like GPT-3, to reduce their computational requirements during fine-tuning and inference. LoRA leverages low-rank factorization techniques to approximate the weight matrices in these models, reducing their memory and computational footprint without significant loss in performance.

[0063] LoRA is a technique that uses linear algebra (matrix decomposition) for providing a faster fine-tuning approach of machine learning models. LoRA enables fine-tuning by language models by reducing parameters about 10,000 times and the GPU requirement by three times.

[0064] Given a model, essentially, a few new parameters are added to the model, namely Matrices: A, B, a scaling factor (alpha), and a rank (r). Scaling factor is usually given (alpha = 1). There are also options for rank, including r = 1, 4, or 16, etc. The main parameters, matrices A and B, need to be tuned using training algorithms.

[0065] FIG. 4 shows a process 400 to perform the LoRA technique between a client 410 and an LLM provider 412 that may be Cloud based. In an input stage, the client 410 encrypts their query data 420 and receives an encrypted output 430 in an output stage. The query data 420 isencrypted via a vector 422 and results in FHE ciphertext 424. The provider 412 receives the encrypted data in ciphertext 424 via the Cloud.

[0066] The provider 412 runs a large language model 440. In this example, LoRA (Low-Rank Adaptation of Large Language Models) is implemented by starting with a Large Language Model (LLM) such as the LLM 440. The LLM 440 includes a set of pre-trained weights 442. A ciphertext input 444 is provided to the set of pre-trained weights 442 and two low-rank matrices: matrix A (450) and matrix B (452). The rank value of matrix A (450) and matrix B (452) is r. The initial value of matrix A 450 could be chosen any random values in the normal (Gaussian) distribution. The initial value of matrix B 452 could be also either a random value or simply zero. A ciphertext output 446 is generated by the result of the pre-trained weights 442, and the matrix A 450 and the matrix B 452. The two matrices 450 and 452 (A and B) are tuned by the corresponding training algorithms.

[0067] Thus, a large pre-trained language model such as GPT-3 may be selected to be used as a starting point. These models have a massive number of parameters and required substantial computational resources for their training. Target layers are then selected. In LoRA, specific layers within the model are selected for low-rank adaptation. These layers are typically chosen based on their computational intensity and importance to the target task.

[0068] Model complexity is then reduced by applying a technique termed “low-rank factorization” to these layers, which simplifies the calculations of the model by using fewer numbers. The simplified model is then fine-tuned. This involves training the model on a specific task using these reduced parameters. This fine-tuning process tailors the model to the task. The result is a tailored model that is more efficient and memory-friendly, making the tailored model suitable for real-world use in applications like chatbots, translation, or text generation.

[0069] A computer system such as that based on the chip 100 in FIG. 1A may thus be configured as a training system to provide secure training of a fine-tuned large language model using a Fully Homomorphic Encryption (FHE) process. The system includes a memory, such as a high bandwidth memory, that stores a trained public large language model as a matrix of public weights in plaintext and a private training set of proprietary data as a matrix of encrypted private weights. A processor unit is coupled to the memory. The processor unit may be any suitable processing device such as the array of cores 130 in FIG. IB. The processor unit may be configured to fine-tune the public large language model by the private training set. The processor unit updates the matrix of encrypted private weights in the memory from the fine tuning.

[0070] The tailored model with the updated matrix of encrypted private weights is then checked to determine suitability for performance. This ensures that the simplified tailored model still performs well for the target task. The tailored model can achieve a good balance between efficiency and accuracy.

[0071] FIG. 5A is a high level diagram of a single layer plaintext fine-tuned model 500 according to a prior art known method. The model 500 includes a series of layers 510, 512, and 514. Details for the layer 512 are shown as an example. The layer 512 includes a multiplexer 520 and an adder 540. The layer 512 includes foundation layer that includes pretrained weights 530 and fine-tuned weights 532.

[0072] The result of the previous layer 510 is sent to both the foundation layers pretrained weights 530 and the fine-tuned layers Weights / Parameters 532. The foundation layer is a standard fully connected layer neural network or a transformer layer neural network. The finetuned layer is either a standard fully connected layer neural network or a transformer layer neural network or a LORA (low rank adaption layer neural network). The output of the foundation layer 530 is added to the output of the fine tuned layer 532 via the adder 540 to be sent to the next layer of the model such as the layer 514. In this example, the weights in the foundation layer 530 and the fine-tuned layer 532 are in plaintext to speed computation of the layer. However, this risks the disclosure of the model 500 as the plaintext weights are observable.

[0073] FIG. 5B is a high level diagram of a single layer, encrypted fine-tuned LLM model system 550 showing the example method that allows efficient protection of fine-tuned LLMs. The model 550 includes a series of layers 560, 562, and 564. Details for the layer 562 are shown as an example. The layer 562 includes a multiplexer 570 and an adder 590. The layer 562 includes foundation layer that includes pretrained weights 580 in plaintext and fine-tuned weights 582 that are encrypted.

[0074] Previous FHE encrypted results (tokens) of a layer are sent to both the foundation the foundation layer 580 with pretrained plaintext weights and the fine-tuned layer 582 with FHE encrypted weights / parameters. The foundation layer is a standard fully connected layer neural network or a transformer layer neural network with its param eters / weights in Plain-Text (no encryption). The input tokens to the multiplexer 570 are FHE encrypted * ciphertexts. The fine-tuned layer 582 is either a standard fully connected layer neural network or a transformer layer neural network or a LORA (low rank adaption layer neural network) where the parameters / weights are FHE encrypted. The output of the foundation layer 582, which is anencrypted ciphertext is FHE Homomorphically added via the adder 590 to the output of the fine-tuned layer 582 which is also encrypted and then sent to the next layer of the model.

[0075] As explained above, the input query to the private-LLM is encrypted. The output response is also encrypted. The fine-tuned parameters, which contain the proprietary data used fortraining, are also encrypted. However, the original foundational model remains in plaintext. A conventional private-LLM which attempts to perform encrypted operations on both the foundational model and the fine-tuning model requires enormous computational horsepower. This example method only requires FHE operations on the fine-tuned parameters thus reducing the total computation horsepower requirement by 10,000x.

[0076] FIG. 6A shows an equation that is the basis of a prior art LLM mapping system. As shown in the equation in FIG. 6A, LLM inference may be treated as a nonlinear mapping, F(X), from input X to output Y. This mapping relationship are uniquely determined by the LLM weights W, which mainly consists of four different types of matrices, namely, a Key Matrix, a Value Matrix, a Query Matrix and a Forward Neural Network Matrix. Through offline training stages, these matrices can be determined by some supervised learning algorithms. Once these weights are determined, the desired output can be obtained by performing many layers matrix multiplication as well as some basic nonlinear operations such as Gaussian error linear unit, SoftMax, and Layer-Norm.

[0077] In FIG. 6A, the processing steps in prior art methods include: 1) The Client sends their data X (Plaintext) to the server which owns LLM parameters; 2) The Server performs the LLM operation with X as inputs of LLM; and 3) The Server sends the generated output Y to the Client. The output Y is equal to F(X). With this application and implementation scenario, both client and server are currently fully transparent and there is neither privacy or protection or security among their input data, output data, as well as those customized fine-tuning parameters owned by the server.

[0078] FIG. 6B shows an equation that is the basis of a prior art private LLM mapping system. In order to protect the privacy and offer the desired security, the private LLM system in FIG. 6B is increasingly becoming the necessary implementation scenario.

[0079] In FIG. 6B, the processing steps include: 1) The Client first encrypts their data X (Plaintext) to the ciphertexts Xcby using some algorithms to support fully Homomorphic Encryption (FHE) such as CKKS or tFHE; 2) The Client sends their encrypted data Xcto the server which owns the LLM parameters; 3) The Server performs the LLM operation with Xcas inputs of LLM; and 4) The Server sends the generated output Ycto the Client, where Ycequals F(XC) and also should equal (F(X)C), that is, the ciphertexts corresponding the plaintextsF(X); and 5) The Client decrypts the received ciphertext Ycto finally get the desired result, which is equal to F(X).

[0080] The interconnection between the client and the LLM servers are all performed on encrypted data in FIG. 6B, which could achieve the desire goal in terms of privacy and security. However, the computational complexity of the process in FIG. 6B is too huge to implement with current implementation platforms such as those based on GPUs or FPGAs. Even on the fractal core based architecture in FIG. 3B, computational complexity is a challenge for running the secure private LLM in FIG. 6B.

[0081] FIG. 7 is a table 700 that compares the computations (TM: trillion multiplicationequivalent) required in FIG. 6A (prior art without encryption of the LLM) and FIG. 6B (prior art with encryption of LLM parameters) for GPT-3 parameters as an example model, which has about 96 layers, 13,000 forward neurons and 50,600 hidden neurons. It can be seen that the computational complexity will increase at least 100 times to perform private-LLM inference, which prevents LLMs from being available for wider use to protect the LLMs.

[0082] In order to attack this problem, the example method is based on a low rank adaptation based private-LLM system. FIG. 8 is a diagram of an example LoRA based Private LLM 800 and an equivalent equation 810. All LLM parameter matrices are decomposed into two parts: namely a pre-trained LLM weight matrix W and a Fine-Tuning weights matrix, AW. Unlike other existing fine-tuning algorithms where both W and AW have both the same dimensions and rank, AW (d x d) in the example solution 816 can be generated by two much lower-rank matrices 812 and 814 (A and B) as shown in FIG. 6B where the rank r could be as small as 10 which is 1000 times less than d.

[0083] Thus, the system 800 takes an input 820 from either a client query token or the output of the previous layer as shown in Fig.5B. The system 800 creates a set of pre-trained LLM weights 822 and a set of LoRA based fine tuning weights 824. The weights are then applied to the LLM output 826.

[0084] With this decomposition, the fine-tuning weights may be obtained in ciphertext domain and may be used to perform the desired LLM processing on ciphertexts. Since the dimensions of the matrices 812 and 814 (A and B) are much smaller, the corresponding computational complexity for ciphertext would be as low as the one for plaintexts, which means the reduction in 100 times can be achieved by the example LoRA based solution.

[0085] During the LoRA Training Phase (with LWE as an example):Input of LLM :Targeted Output:Learning: A(i + 1) = A(i) + 8A(i)eB(i + 1) = B(i) + 8B(i)e

[0086] FIG. 9 is a flow diagram of the example method of efficient encrypted training for a fine-tuned large language model. A trained Public Large Language Model is selected with unencrypted parameters (910). The selected parameters are loaded into a plaintext matrix of weights / parameters (912). Any previously fine-tuned encrypted weights are loaded into a matrix of encrypted private weights (914). Alternatively, the matrix of encrypted private weights is initialized if no fine-tuned encrypted weights are available. A private training set of proprietary data is then encrypted by CKKS to tFHE for example (916). The encrypted private training set is input into the Large Language Model as encrypted tokens (918). Fine-tuning of the matrix of encrypted private weights is performed using standard learning algorithms (e.g., backpropagation / gradient descent) using the encrypted tokens (920). Only the encrypted private weights A(i) and B(i) are updated in the fine-tuning process (922) as shown in the above Equation.

[0087] The routine then determines whether the desired accuracy is achieved by the LLM in relation to the proprietary training set (924). If the desired accuracy is not achieved, the routine loops back to encrypting the private data set (918). If the desired accuracy is achieved, the routine saves the matrix of encrypted private weights (926). The saved private weights are encrypted and thus allow a fine-tuned LLM to be used with encryption protection for the private weights after completion of the training process.

[0088] In this fine-tune training stage, one implementation may be using the chip 100 in FIG. 1A for the server. Thus, the cores of the chip 100 in the server may be configured to first receive the ciphertext Xcand a targeted output Ycfrom the client or the data owner via a data link interface. The chip 100 in the server may then be configured to perform all the matrix multiplications and updating LoRA weights (A(i+1) and B(i+ 1 )) shown in the above Equation and all other operations from 916 through 926 of Figure 9.

[0089] During the LoRA Inference Phase, the following are determined:Input of NN: St+ AX + E]Private-LLM Inference to get Output:which will be a very close approximation of the targeted ciphertexts.

[0090] FIG. 10 is a flow diagram of an example encrypted inference routine performed by the example fine-tuned large language model. The routine first loads a trained public largelanguage model with a matrix of public weights in plaintext (1010). A private training set of proprietary data is loaded as a set of encrypted weights (1012). The routine receives an encrypted query from a client (1014). The encrypted queries are processed via an encrypted data path with the set of encrypted weights for the trained large language model (1016). The encrypted queries are processed via a plaintext data path with the set of public weights in plaintext for the trained large language model (1018). The encrypted results are processed from the processing via the encrypted data path and the plaintext data path (1020). Thus, the queries as well as the private weights remain encrypted and thus are secure. However, since the public weights remain in plaintext, significant computational savings are realized to allow secure use of a private LLM.

[0091] One example hardware implementation includes configuring the chip 100 in FIG. 1A for in the server. The chip 100 is thus configured to receive the ciphertext Xcfrom Client via the data link interface; read the plaintext weights W of the general LLM (step 1010 of FIG. 10); and read the ciphertext weights A and B of the fine-tuned LLM from memory or from the owner of the encrypted weights (step 1012 of FIG. 10). Then the chip 100 in the server may be configured to perform all the matrix multiplications and nonlinear functions as shown in the above equation. The chip 100 may be configured to generate ciphertexts, which is the encrypted version (1020) of the desired output.

[0092] The flow diagrams in FIGs. 9-10 are representative of example machine readable instructions for training of an encrypted LLM and inference by an encrypted LLM. In this example, the machine readable instructions comprise an algorithm for execution by: (a) a processor; (b) a controller; and / or (c) one or more other suitable processing device(s). The algorithm may be embodied in software stored on tangible media such as flash memory, CD- ROM, floppy disk, hard drive, digital video (versatile) disk (DVD), or other memory devices. However, persons of ordinary skill in the art will readily appreciate that the entire algorithm and / or parts thereof can alternatively be executed by a device other than a processor and / or embodied in firmware or dedicated hardware in a well-known manner (e.g., it may be implemented by an application specific integrated circuit [ASIC], a programmable logic device [PLD], a field programmable logic device [FPLD], a field programmable gate array [FPGA], discrete logic, etc.). For example, any or all of the components of the interfaces can be implemented by software, hardware, and / or firmware. Also, some or all of the machine readable instructions represented by the flowcharts may be implemented manually. Further, although the example algorithm is described with reference to the flowcharts illustrated in FIGs. 9-10, persons of ordinary skill in the art will readily appreciate that many other methodsof implementing the example machine readable instructions may alternatively be used. For example, the order of execution of the blocks may be changed, and / or some of the blocks described may be changed, eliminated, or combined.

[0093] The terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting of the invention. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. Furthermore, to the extent that the terms “including,” “includes,” “having,” “has,” “with,” or variants thereof, are used in either the detailed description and / or the claims, such terms are intended to be inclusive in a manner similar to the term “comprising.”

[0094] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. Furthermore, terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0095] While various embodiments of the present invention have been described above, it should be understood that they have been presented by way of example only, and not limitation. Numerous changes to the disclosed embodiments can be made in accordance with the disclosure herein, without departing from the spirit or scope of the invention. Thus, the breadth and scope of the present invention should not be limited by any of the above described embodiments. Rather, the scope of the invention should be defined in accordance with the following claims and their equivalents.

[0096] Although the invention has been illustrated and described with respect to one or more implementations, equivalent alterations, and modifications will occur or be known to others skilled in the art upon the reading and understanding of this specification and the annexed drawings. In addition, while a particular feature of the invention may have been disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of the other implementations as may be desired and advantageous for any given or particular application.

[0097] An example implementation 1 is a method to provide secure training of a fine-tuned large language model, the method comprising: storing a trained public large language model as a matrix of public weights in plaintext in a memory; storing a private training set of proprietary data in the memory; loading a matrix of encrypted private weights;fine-tuning the public large language model using the private training set; and updating the matrix of encrypted private weights from the fine-tuning.

[0098] Implementation 2 may be the method of implementation 1, wherein the encrypted private weights are reduced by low rank adaptation (LoRA).

[0099] Implementation 3 may be the method of any one of implementations 1-2, wherein the matrix of private weights is smaller than the matrix of public weights.

[0100] Implementation 4 may be the method of any one of implementations 1-3, wherein the encrypted private weights are stored as a fully connected encrypted matrix.

[0101] Implementation 5 may be the method of any one of implementations 1-4, wherein the encryption is performed via a Fully Homomorphic Encryption (FHE) process.

[0102] Implementation 6 may be a training system to provide secure training of a fine-tuned large language model using a Fully Homomorphic Encryption (FHE) process, the system comprising: a memory storing a trained public large language model as a matrix of public weights in plaintext and a private training set of proprietary data as a matrix of encrypted private weights; and a processor coupled to the memory, the processor operable to: fine-tune the public large language model by the private training set; and update the matrix of encrypted private weights in the memory from the fine tuning.

[0103] Implementation 7 may be the system of implementation 6, wherein the encrypted private weights are reduced by low rank adaptation (LoRA).

[0104] Implementation 8 may be the system of any one of implementations 6-7, wherein the matrix of private weights is smaller than the matrix of public weights.

[0105] Implementation 9 may be the system of any one of implementations 6-8, wherein the encrypted private weights are stored as a fully connected encrypted matrix.

[0106] Implementation 10 may be the system of any one of implementations 6-9, wherein the encryption is performed via a Fully Homomorphic Encryption (FHE) process.

[0107] Implementation 11 may be a method to provide secure inference of a fine-tuned large language model using encryption, the method comprising: storing a trained public large language model with a matrix of public weights in plaintext in a memory; storing a set of encrypted weights obtained from the fine-tuned large language model with a training set of proprietary data in the memory; receiving an encrypted query via a data link;processing the encrypted query via an encrypted data path with the set of encrypted weights for the trained large language model; processing the encrypted query via a plaintext data path with the set of public weights in plaintext for the trained large language model; and producing an encrypted result from the processing via adding the encrypted data path and the plaintext data path.

[0108] Implementation 12 may be the method of implementation 11, wherein the encrypted private weights are reduced by low rank adaptation (LoRA).

[0109] Implementation 13 may be the method of any one of implementations 11-12, wherein the encrypted private weights are stored as a fully connected encrypted matrix.

[0110] Implementation 14 may be the method of any one of implementations 11-13, wherein the encryption is performed via a Fully Homomorphic Encryption (FHE) process.

[0111] Implementation 15 may be a method to provide an inference in response to a query, comprising: encrypting the query; providing the encrypted query to a fine-tuned foundation layer of a large language model, wherein the fine-tuned foundation layer includes an encrypted matrix of proprietary weights; providing the encrypted query to a public foundation layer of a large language model, wherein the public foundation layer includes a plaintext matrix of public weights; and adding outputs of the fine-tuned foundation layer and outputs of the public foundation layer to provide an encrypted inference output.

[0112] Implementation 16 may be the method of implementation 15, wherein the encryption is performed via a Fully Homomorphic Encryption (FHE) process.

[0113] Implementation 17 may be a system to output a response to a query, the system comprising: a first processing core configured to receive the query and encrypt the query; a second processing core configured as a fine-tuned foundation layer of a large language model that accepts the encrypted query, wherein the fine-tuned foundation layer includes an encrypted matrix of proprietary weights; a third processing core configured as a public foundation layer of a large language model that accepts the encrypted query, wherein the public foundation layer includes a plaintext matrix of public weights;a fourth processing core configured to add the outputs of the fine-tuned foundation layer and the outputs of the public foundation layer to provide an encrypted inference output; and a network is coupled to each of the processing cores and allows communication among each of the processing cores.

[0114] Implementation 18 may be the system of implementation 17, wherein the first, second, third and fourth processing cores include at least one of a Reduced Instruction Set Computing (RISC) processor core, a special purpose processing core; or a RISC processor core with a more special purpose processing cores embedded within.

[0115] Implementation 19 may be the system of any one of implementations 17-18, wherein the first, second, third and fourth processing cores are in an array of RISC processor cores interconnected with an array of special purpose processing cores.

[0116] Implementation 20 may be the system of any one of implementations 17-19, wherein the first, second, third and fourth processing cores are part of one or more arrays of processing cores.

[0117] Implementation 21 may be the system of any one of implementations 17-20, wherein the encryption is performed via a Fully Homomorphic Encryption (FHE) process.

[0118] Implementation 22 may be a non-transitory computer readable medium having stored thereon instructions that, when executed by a processor unit, cause the processor unit to: encrypt a query; provide the encrypted query to a fine-tuned foundation layer of a large language model, wherein the fine-tuned foundation layer includes an encrypted matrix of proprietary weights; provide the encrypted query to a public foundation layer of a large language model, wherein the public foundation layer includes a plaintext matrix of public weights; and add the outputs of the fine-tuned foundation layer and the outputs of the public foundation layer to provide an encrypted inference output.

[0119] Implementation 23 may be the non-transitory computer readable medium of implementation 22, wherein the processor unit includes a plurality of configurable processing cores, wherein each of the configurable processing cores configured to perform at least one of the instructions.

[0120] Implementation 24 may be the non-transitory computer readable medium of any one of implementations 22-23, wherein the encryption is performed via Fully Homomorphic Encryption (FHE).

Claims

CLAIMSWHAT IS CLAIMED IS:

1. A method to provide secure training of a fine-tuned large language model, the method comprising: storing a trained public large language model as a matrix of public weights in plaintext in a memory; storing a private training set of proprietary data in the memory; loading a matrix of encrypted private weights; fine-tuning the public large language model using the private training set; and updating the matrix of encrypted private weights from the fine-tuning.

2. The method of claim 1, wherein the encrypted private weights are reduced by low rank adaptation (LoRA).

3. The method of claim 1 , wherein the matrix of private weights is smaller than the matrix of public weights.

4. The method of claim 1, wherein the encrypted private weights are stored as a fully connected encrypted matrix.

5. The method of claim 1, wherein the encryption is performed via a Fully Homomorphic Encryption (FHE) process.

6. A training system to provide secure training of a fine-tuned large language model using a Fully Homomorphic Encryption (FHE) process, the system comprising: a memory storing a trained public large language model as a matrix of public weights in plaintext and a private training set of proprietary data as a matrix of encrypted private weights; and a processor coupled to the memory, the processor operable to: fine-tune the public large language model by the private training set; and update the matrix of encrypted private weights in the memory from the fine tuning.

7. The system of claim 6, wherein the encrypted private weights are reduced by low rank adaptation (LoRA).

8. The system of claim 6, wherein the matrix of private weights is smaller than the matrix of public weights.

9. The system of claim 6, wherein the encrypted private weights are stored as a fully connected encrypted matrix.

10. The system of claim 6, wherein the encryption is performed via a Fully Homomorphic Encryption (FHE) process.

11. A method to provide secure inference of a fine-tuned large language model using encryption, the method comprising: storing a trained public large language model with a matrix of public weights in plaintext in a memory; storing a set of encrypted weights obtained from the fine-tuned large language model with a training set of proprietary data in the memory; receiving an encrypted query via a data link; processing the encrypted query via an encrypted data path with the set of encrypted weights for the trained large language model; processing the encrypted query via a plaintext data path with the set of public weights in plaintext for the trained large language model; and producing an encrypted result from the processing via adding the encrypted data path and the plaintext data path.

12. The method of claim 11, wherein the encrypted private weights are reduced by low rank adaptation (LoRA).

13. The method of claim 11, wherein the encrypted private weights are stored as a fully connected encrypted matrix.

14. The method of claim 11, wherein the encryption is performed via a Fully Homomorphic Encryption (FHE) process.

15. A method to provide an inference in response to a query, comprising: encrypting the query; providing the encrypted query to a fine-tuned foundation layer of a large language model, wherein the fine-tuned foundation layer includes an encrypted matrix of proprietary weights; providing the encrypted query to a public foundation layer of a large language model, wherein the public foundation layer includes a plaintext matrix of public weights; and adding outputs of the fine-tuned foundation layer and outputs of the public foundation layer to provide an encrypted inference output.

16. The method of claim 15, wherein the encryption is performed via a Fully Homomorphic Encryption (FHE) process.

17. A system to output a response to a query, the system comprising: a first processing core configured to receive the query and encrypt the query; a second processing core configured as a fine-tuned foundation layer of a large language model that accepts the encrypted query, wherein the fine-tuned foundation layer includes an encrypted matrix of proprietary weights; a third processing core configured as a public foundation layer of a large language model that accepts the encrypted query, wherein the public foundation layer includes a plaintext matrix of public weights; a fourth processing core configured to add the outputs of the fine-tuned foundation layer and the outputs of the public foundation layer to provide an encrypted inference output; and a network is coupled to each of the processing cores and allows communication among each of the processing cores.

18. The system of claim 17, wherein the first, second, third and fourth processing cores include at least one of a Reduced Instruction Set Computing (RISC) processor core, a special purpose processing core; or a RISC processor core with a more special purpose processing cores embedded within.

19. The system of claim 17, wherein the first, second, third and fourth processing cores are in an array of RISC processor cores interconnected with an array of special purpose processing cores.

20. The system of claim 17, wherein the first, second, third and fourth processing cores are part of one or more arrays of processing cores.

21. The system of claim 17, wherein the encryption is performed via a Fully Homomorphic Encryption (FHE) process.

22. A non-transitory computer readable medium having stored thereon instructions that, when executed by a processor unit, cause the processor unit to: encrypt a query; provide the encrypted query to a fine-tuned foundation layer of a large language model, wherein the fine-tuned foundation layer includes an encrypted matrix of proprietary weights; provide the encrypted query to a public foundation layer of a large language model, wherein the public foundation layer includes a plaintext matrix of public weights; and add the outputs of the fine-tuned foundation layer and the outputs of the public foundation layer to provide an encrypted inference output.

23. The non-transitory computer readable medium of claim 22, wherein the processor unit includes a plurality of configurable processing cores, wherein each of the configurable processing cores configured to perform at least one of the instructions.

24. The non-transitory computer readable medium of claim 22, wherein the encryption is performed via Fully Homomorphic Encryption (FHE).

Images