Methods for calculating the output values ​​of functions in a transformer-based AI module, methods for enhancing transformer-based AI modules, and computer systems (transformer-based AI modules with activation functions).

Abstract

The higher the degree of the polynomial used to approximate a nonlinear function, the more accurate the approximation can become. However, this can come with higher computational costs when evaluating the polynomial. [Solution] A technique is described for a method for calculating a fully homomorphically encrypted output value of a function of a transformer-based AI module, depending on the fully homomorphically encrypted input value of the function, the function representing a nonlinear function to be approximated by the function. The method comprises a step of providing a fully homomorphically encrypted pre-calculated term independent of the input value. The method further comprises a step of using the transformer-based AI module to calculate the output value of the function depending on the pre-calculated term and a polynomial, each polynomial depending on the difference between the input value and the central value associated with each polynomial.

Description

Technical Field

[0001] This disclosure relates to the field of digital computer systems, and more particularly, to fully homomorphic encryption (FHE).

Background Art

[0002] Fully homomorphic encryption (FHE) is a powerful cryptographic technique that enables computations on encrypted data without the need to decrypt it. This feature is particularly relevant to artificial intelligence (AI) models, as it provides a framework for ensuring data privacy and security while enabling efficient computations. FHE helps organizations comply with strict data privacy regulations. However, FHE-type mathematical operations are computationally expensive compared to processing plaintext data. This can slow down AI model training and inference. The computational resources required to perform FHE-type mathematical operations for training AI models can be so high that some applications may not be feasible.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The higher the degree of the polynomial used for approximating a non-linear function, the more accurate the approximation can be. However, this can involve a higher computational cost when evaluating the polynomial. Particularly with respect to the convolution of the coefficients of the cryptographic polynomial, the number of homomorphic operations can increase exponentially with the degree of the polynomial, so the higher computational cost can pose difficulties in implementing FHE-transformer-based AI modules.

Means for Solving the Problems

[0004] In some embodiments, the techniques described herein relate to a method for calculating a fully homomorphically encrypted output value of a function of a transformer-based AI module, depending on a fully homomorphically encrypted input value of the function, wherein the function represents a nonlinear function to be approximated by the function. The method comprises a step of providing a fully homomorphically encrypted pre-calculated term independent of the input value. The method further comprises a step of using the transformer-based AI module to calculate the output value of the function depending on the pre-calculated term and a polynomial. Each polynomial depends on the difference between the input value and a central value associated with each of the polynomials. The associated central value indicates the center point of each of the polynomials, and the associated central value is different for each polynomial.

[0005] In some embodiments, the techniques described herein relate to a method for enhancing a transformer-based AI module, the transformer-based AI module including a nonlinear function for calculating an output value of the nonlinear function depending on an input value. The method comprises a step of substituting the nonlinear function with a function, the function depending on a pre-calculated term independent of the input value and a polynomial. Each polynomial depends on the difference between the input value and a central value associated with the respective polynomial, where the associated central value indicates the center point of the respective polynomial, and the associated central value is different for each polynomial. Furthermore, the function is designed to define an approximate output value of the nonlinear function in response to the input value being presented as an argument to the nonlinear function.

[0006] Additional aspects of this disclosure relate to systems and computer program products configured to perform the methods described above. This summary is not intended to illustrate any aspect, implementation, and / or all embodiments of the disclosure. [Brief explanation of the drawing]

[0007] In the following examples, the diagrams will be used for illustrative purposes and will be explained in more detail.

[0008] [Figure 1] This is a flowchart of a method for calculating the fully homomorphically encrypted output value of a function of an artificial intelligence module (AI module), as an example of this subject.

[0009] [Figure 2] This diagram shows the data flow of the AI ​​module and the function module of the AI ​​module.

[0010] [Figure 3] This is a diagram showing a set of polynomials.

[0011] [Figure 4] This figure shows the associative polynomial based on the set of polynomials shown in Figure 3.

[0012] [Figure 5] This figure shows a shortened and modified version of the associative polynomial shown in Figure 4.

[0013] [Figure 6] This diagram shows the data flow of the AI ​​module for repeated use.

[0014] [Figure 7] This figure shows the data elements calculated within the AI ​​module's attention module during the first iteration of using the AI ​​module.

[0015] [Figure 8] This figure shows the data elements calculated within the AI ​​module's attention module during the second iteration of using the AI ​​module.

[0016] [Figure 9]FIG. is a diagram showing a variant of a method in which calculated data elements of an attention module of an AI module can be cached in a cache device of a computer system.

[0017] [Figure 10] FIG. shows a computing environment according to an example of the present subject matter. DETAILED DESCRIPTION

[0018] The present disclosure relates to the field of digital computer systems, and more particularly, to a method for calculating a homomorphically encrypted output value of a function of an artificial intelligence module (AI module).

[0019] The descriptions of the various examples of the present disclosure are presented for purposes of illustration and are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the examples described. The technical terms used herein are selected to best explain the principles of the examples, practical applications, or technical improvements over the technologies found in the market, or to enable other skilled artisans to understand the embodiments disclosed herein.

[0020] Calculating the encrypted output value of a function that depends on a polynomial can enable calculating the output value with higher accuracy compared to determining the output value depending on only one polynomial. The output value can approximate the output value of a non-linear function in the input value. The non-linear function can be a sigmoid or Gaussian Error Linear Unit (GELU) function of a neural network of an AI module, particularly an activation function, or an exponential function of a softmax function of an AI module, particularly an attention module of an AI module.

[0021] In general, using polynomials to approximate nonlinear functions can enable the application of the approximated nonlinear function to fully homomorphically encrypted data, such as input values. This allows mathematical operations associated with the nonlinear function to be approximated to be performed in the ciphertext domain without the requirement of decrypting the encrypted input values. Homomorphic encryption is a cryptographic technique that can enable computations to be performed directly on encrypted data, such as encrypted input values, without first requiring decryption. The results of these computations remain encrypted, and in particular, remain homomorphically encrypted. The results can only be decrypted using a decryption key. Homomorphic encryption (HE) can enable certain types of operations on encrypted data, such as addition or multiplication.

[0022] Conversely, fully homomorphic encryption (FHE) can support arbitrary calculations on encrypted data. In one example, FHE can even support unlimited addition and multiplication on encrypted data. This can be achieved, for example, by using techniques such as bootstrapping to control noise in the encrypted data. Therefore, by fully homomorphic encryption of the pre-computed terms and input values, it may be possible in the ciphertext domain to perform unlimited addition and multiplication on intermediate data resulting from mathematical operations based on the pre-computed terms and / or input values. In one example, the output value may be calculated depending on the intermediate data.

[0023] The phrase "encrypted," as used herein, may be an abbreviation for "fully homomorphically encrypted." Exemplary terms determined as a result of calculations based on encrypted input values ​​and / or pre-calculated terms can be fully homomorphically encrypted, and this is a fundamental principle of FHE. For example, a calculated output value can be fully homomorphically encrypted because it depends on fully homomorphically encrypted input values ​​and pre-calculated terms.

[0024] Calculating polynomial-dependent output values ​​to approximate the nonlinear functions of AI modules, particularly transformer-based AI modules, may enable the provision of AI modules in the form of FHE AI modules, and especially transformer-based AI modules in the form of FHE transformer-based AI modules. This may enable the training of AI modules, particularly transformer-based AI modules, without decrypting sensitive training data for training. Therefore, in one example, the AI ​​module is an FHE AI module. In particular, the transformer-based AI module can be an FHE transformer-based AI module.

[0025] In one example, an input value can be fully homomorphically encrypted by encoding the input value into an input polynomial and then fully homomorphically encrypting the coefficients of the input polynomial using an exemplary fully homomorphic encryption scheme. The encrypted coefficients of the input polynomial can represent the input value in the ciphertext domain. The encryption scheme may be, for example, the BGV (Brakerski-Gentry-Vaikuntanathan), BFV (Brakerski-Fan-Vercauteren), or CKKS (Cheon-Kim-Kim-Song) encryption scheme. The encryption scheme may be based on lattice cryptography.

[0026] Similarly, a polynomial of a function may be provided in the ciphertext domain to compute the encrypted output value of the function depending on the encrypted input value. In one example, each coefficient of a polynomial may be encoded into its own cipher polynomial. The term “cipher polynomial” is used herein to distinguish a polynomial from a polynomial in the ciphertext domain, i.e., a polynomial representing the coefficients of a cipher polynomial. A cipher polynomial may also include coefficients. The coefficients of a cipher polynomial may be fully homomorphically encrypted by applying an exemplary fully homomorphic encryption scheme to the coefficients of the cipher polynomial, thereby yielding the encrypted coefficients of the cipher polynomial.

[0027] Similarly, providing homomorphically encrypted pre-computed terms may be accompanied by providing homomorphically encrypted coefficients of further polynomials that can represent pre-computed terms in the ciphertext domain.

[0028] In one example, calculating the output value may involve performing mathematical operations, such as summation and / or convolution, on the encrypted coefficients of the input polynomial, the encrypted coefficients of the cryptographic polynomial, and / or the encrypted coefficients of further polynomials. The proposed method may be applicable independently of the FHE scheme used to encrypt the input values. The AI ​​module may be configured to apply homomorphic operations, such as summation or convolution, to the encrypted coefficients of the input polynomial, the encrypted coefficients of further polynomials, and / or the encrypted coefficients of the cryptographic polynomial. For example, the AI ​​module may be designed to add the encrypted coefficients of the input polynomial with the encrypted coefficients of the cryptographic polynomial and / or the encrypted coefficients of further polynomials. In another example, the AI ​​module may be designed to perform convolution of the encrypted coefficients of the input polynomial with the encrypted coefficients of the cryptographic polynomial and / or the encrypted coefficients of further polynomials. In addition, the AI ​​module may be configured to perform modular reduction in order to keep the degree of the intermediate polynomial resulting from homomorphic operations between the encrypted coefficients of the input polynomial, the encrypted coefficients of the cryptographic polynomial, and / or the encrypted coefficients of further polynomials within a defined range.

[0029] In one example, the output value of a function may be expressed in the form of an output polynomial. The coefficients of the output polynomial may be obtained by applying homomorphisms to the encrypted coefficients of the input polynomial, the encrypted coefficients of further polynomials, and / or the encrypted coefficients of the cryptographic polynomial. The coefficients of the output polynomial may be automatically obtained in encrypted form when applying homomorphisms.

[0030] Generally, the higher the degree of the polynomial used to approximate a nonlinear function, the more accurate the approximation can be. However, this can come with higher computational costs when evaluating the polynomial. In particular, with respect to the convolution of coefficients of cryptographic polynomials, the number of homomorphic operations can increase exponentially with the degree of the polynomial, so higher computational costs can make it difficult to implement FHE transformer-based AI modules. On the other hand, in the context of FHE, the higher the accuracy of the approximation of a nonlinear function, the less noise can be generated per computation. If the noise reaches a certain level, the results of homomorphic mathematical operations performed in the ciphertext domain may no longer be precisely decipherable. Therefore, the accuracy of the approximation of the nonlinear function may be specified to ensure that the results of computations performed by an FHE AI module can be decipherable.

[0031] On the other hand, since the calculation of the function's output value is performed in relation to polynomials, and these polynomials are centered at different central points, the specified accuracy of the approximation of the nonlinear function can be achieved using lower-order polynomials compared to calculating the output value by relying on only one polynomial centered at a single central point.

[0032] In other words, by using different central values ​​for each polynomial and calculating the output value dependent on the polynomial, it may be possible to reduce the degree of the polynomial while maintaining the same or similar accuracy in approximating the nonlinear function. This can reduce the computational cost for training and inference of FHE AI modules and especially FHE transformer-based AI modules. By having different central values ​​that each polynomial can represent, and by determining the encrypted output value of the function dependent on the polynomial, the output value can be generated as if different polynomials, in the sense that they are unfolded around different central points, were used. In this way, a defined accuracy of the function can be achieved with respect to the input range of the function where the central points are located. It is understood that the input value of the function can be located within the input range. In one example, the central points may be evenly distributed within the input range. In one example, the determination of the output value dependent on the polynomial may involve calculating a linear combination of the polynomials.

[0033] Furthermore, when providing pre-calculated terms, these terms do not need to be computed in response to receiving input values. Therefore, computation time can be saved by loading them instead of newly computing pre-calculated terms to determine the output value of the function. This can reduce the computational cost for training or inference of FHE AI modules, particularly FHE transformer-based AI modules. In one example, pre-calculated terms may be provided by storing them in the computer system's cache device.

[0034] In one example, the input values ​​may be internal values ​​of the AI ​​module, determined depending on the input dataset of the AI ​​module. The proposed method may involve determining several output values ​​of a set of functions of the AI ​​module, where the set of functions may include the functions mentioned above and further functions, each of which may approximate each of the further nonlinear functions of the AI ​​module. Each of the further functions in the set of functions may be constructed in a similar manner to the functions described above and / or in similar manner to variations of the functions mentioned below. In one example, all of the nonlinear functions of the AI ​​module may be approximated by the set of functions.

[0035] In one example, each polynomial may depend on a set of powers for each polynomial. Each power in each set may depend on the difference between the input value and the central value associated with each polynomial, as the base of each power, and on a natural number as the exponent of each power. The natural number may increase with each power of the polynomial.

[0036] In most applications, the difference between the input value and each of its central values ​​can lie between -1 and 1, where the numbers -1 and 1 can refer to the plaintext domain. Therefore, raising the base of each power to a natural number of that power can yield a value of each power that is equal to or smaller than the difference, since the exponent of each power is a natural number, e.g., 1, 2, 3, etc. The natural number may start from 1. As the natural number may increase with each power of the polynomial, the value of each power may decrease with each power of the polynomial. The higher the degree of each power, the smaller the value of each power can become. Therefore, powers can be used to adapt the accuracy of the approximation of nonlinear functions.

[0037] Therefore, by having each polynomial depend on a set of powers, the accuracy of the approximation of a nonlinear function may be adjustable so that it can satisfy a specified accuracy. Thus, powers can provide flexibility to satisfy the specified accuracy. This can make it possible to reduce the noise generated in each computation when applying the function in the FHE AI module. The accuracy of the approximation of a nonlinear function may be determined by the power of the set of powers for each polynomial whose exponent is largest. All polynomials can be expanded to the same degree.

[0038] In one example, each polynomial may depend on a set of products over each polynomial. Each product of each set may contain, as a factor, one of the pre-calculated terms and one of the powers of each set of powers. By multiplying together one of the pre-calculated terms and one of the powers, each polynomial may be analogous to a Taylor polynomial. Just as Taylor polynomials are known for their ability to approximate any function with arbitrary accuracy, this may allow for approximating nonlinear functions with greater accuracy. This may further reduce the noise generated in each computation when applying functions in the FHE AI module.

[0039] In one example, the pre-calculated term may depend on the derivatives of a nonlinear function evaluated at the center point. This may allow the output value to be calculated in a manner similar to the Taylor approximation of the nonlinear function. Thus, this may allow the nonlinear function to be approximated with greater accuracy. In one example, each polynomial may depend on a set of derivatives of a nonlinear function evaluated at the respective center point associated with each polynomial. Each derivative in the set of derivatives may be evaluated at the respective center point associated with each polynomial. The order of the derivatives in each set of derivatives may increase with each derivative. The derivatives may be pre-calculated and may be part of the pre-calculated term. This may speed up computation time during training and inference of the AI ​​module.

[0040] In one example, each polynomial could represent a Taylor polynomial for a nonlinear function. Each Taylor polynomial could be centered around the central value associated with that polynomial.

[0041] Each Taylor polynomial can approximate a nonlinear function near its respective center point. Each Taylor polynomial can be expanded around its respective center point. Since polynomials do not contain an infinite number of terms, each Taylor polynomial can only form a portion of its respective Taylor series that approximates the nonlinear function near its respective center point. This example can be considered a variation of the polynomial that is most similar to its respective Taylor series expanded around its respective center point. Therefore, this may allow for further improvement in the accuracy of the approximation of the nonlinear function.

[0042] In one example, calculating the output value of a function may involve evaluating a linear combination of polynomials. The output value of the function may then be obtained in the form of an average of different polynomials. Since each polynomial is centered on a different of the central points, the average can provide an efficient method for approximating a nonlinear function that can match a specified accuracy in cases where the input values ​​lie within the range where the central values ​​are located.

[0043] In one example, the nonlinear function may be a Gaussian Error Linear Unit (GELU) function. The GELU function may be the activation function of neurons in a neural network of an AI module. The GELU function smoothly combines the behavior of ReLU and sigmoid. Instead of performing a hard cutoff like ReLU, which sets negative values ​​to zero, the GELU function may allow negative values ​​to pass through but scale them down. This may allow the derivative to be computed in order to expand the polynomial in the form of a Taylor polynomial for the GELU function. In one example, the GELU function may be of the following form: GELU(x)≒0.5x(1+tanh[√(2 / π)×(x+0.044715x 3 )])

[0044] Here, "x" may represent the input value.

[0045] In one example, the nonlinear function may be the exponential function of the softmax function. The softmax function is a mathematical function that transforms a vector of real numbers into a probability distribution. The following is the formulation for the softmax function: vector z = [z1, z2, ..., z n Regarding ], the AI ​​module may be configured to apply the softmax function as follows: For i=1,2,...,n and j=1,2,...,n, softmax(z i )=e^(z i ) / Σ j [e^(z j )]

[0046] Here, "e" represents the exponential function. The property of the softmax function is that for all i, the softmax function is (z i )∈[0,1] and all softmax(z i The sum over ) may be equal to 1. In one example, the input value may be one of the entries in the vector z. Therefore, in the case where the nonlinear function is an exponential function of the softmax function, the AI ​​module may be configured to apply the function multiple times to a set of input values ​​for the function in order to realize the softmax function. The input value may be one of the input values ​​of the set of input values. The input values ​​of the set of input values ​​may be presented sequentially as inputs to the function.

[0047] In one example, determining the output value of a function that depends on a polynomial and a pre-calculated term may involve providing an associative polynomial. An associative polynomial may be obtained by combining terms that have the same natural number as their exponent. The terms may be obtained by expanding each set of powers of each polynomial and arranging the expanded sets of powers together in the form of a single formula. The combination of terms may involve combining like terms among the terms. The single formula may be in the form of a linear combination of polynomials. An associative polynomial may be considered as a single aggregated polynomial. In this example, the polynomials may be considered aggregated in the form of a single aggregated polynomial, i.e., an associative polynomial.

[0048] The associative polynomial may include pre-calculated terms and further terms. The pre-calculated terms may include the derivatives mentioned above. Further terms may depend on the input values ​​and may include natural numbers as exponents. In one example, further terms may include the input values ​​or their respective central values ​​as bases.

[0049] The calculation of the output value of a function that depends on polynomials and pre-calculated terms may further include evaluating additional terms depending on the input values.

[0050] Furthermore, the calculation of the output value of a function that depends on polynomials and pre-calculated terms may include calculating the output value as a result of evaluating an associative polynomial based on the pre-calculated terms and further evaluated terms. In this case, the calculation of the polynomial-dependent output value may be performed by evaluating an associative polynomial based on the pre-calculated terms and further evaluated terms. Performing the calculation of the output value by evaluating an associative polynomial instead of evaluating each polynomial one by one and adding up the results of each evaluation of the polynomial can speed up the determination of the function's output value.

[0051] In one example, an associative polynomial may be provided in the ciphertext domain to compute an encrypted output value of a function depending on an encrypted input value. In one example, each coefficient of the associative polynomial may be encoded into a further ciphertext polynomial. The term “further ciphertext polynomial” is used herein to distinguish such further ciphertext polynomials from the ciphertext polynomial. Further ciphertext polynomials may also include coefficients. The coefficients of a further ciphertext polynomial may be fully homomorphically encrypted by applying an exemplary fully homomorphic encryption scheme to the coefficients of the further ciphertext polynomial, thereby yielding the encrypted coefficients of the further ciphertext polynomial. The encrypted coefficients of a further ciphertext polynomial may represent an associative polynomial in the ciphertext domain. In one example, the encrypted coefficients of a further ciphertext polynomial may be part of a pre-calculated term.

[0052] In one example, the AI ​​module may be configured to perform homomorphic operations on the encrypted coefficients of an input polynomial and the encrypted coefficients of a further cryptographic polynomial, such as summation and / or convolution. For example, the AI ​​module may be designed to add the encrypted coefficients of the input polynomial with the encrypted coefficients of a further cryptographic polynomial. In another example, the AI ​​module may be designed to perform a convolution of the encrypted coefficients of the input polynomial with the encrypted coefficients of a further cryptographic polynomial. In addition, the AI ​​module may be configured to perform modular reduction to keep the degree of any further intermediate polynomials resulting from homomorphic operations between the encrypted coefficients of the input polynomial and the encrypted coefficients of the further cryptographic polynomials within a defined range.

[0053] In one example, the AI ​​module may be a transformer-based large language model (LLM). Since transformer-based LLMs are widely used, an AI module that is a transformer-based LLM with a function represented by an associative polynomial can provide a solution for applying the transformer-based LLM to encrypted data, such as encrypted input values, while keeping the computation time for calculating the output of the transformer-based LLM within a reasonable limit.

[0054] In one example, the function may be a function of the attention module in a transformer-based LLM. The attention module may be a self-attention module or a cross-attention module. Attention modules are widely used in transformer-based LLMs. An attention module may allow the projection of internal embedding vectors computed based on the input dataset of the AI ​​module mentioned above onto a subspace and the comparison of the projected internal vectors with each other. This may allow for a deeper understanding of the internals of the problem presented by the input dataset. Since such functionality is crucial for modern transformer-based LLMs, the attention module may represent one of the most important components of a modern transformer-based LLM. The fact that the function is a function of the attention module in a transformer-based LLM may allow for faster computation of the output of the transformer-based LLM in the ciphertext domain.

[0055] In one example, the method may further include receiving a first encrypted input. In one example, the first input may be provided as input to an AI module. The first input may be in the form of a first phrase, e.g., a first question phrase, particularly in cases where the AI ​​module is designed in the form of an LLM, e.g., a transformer-based LLM. In one example, the first encrypted input may be provided in the form of encrypted coefficients of a first input polynomial.

[0056] The method may further include using the AI ​​module to determine the fully homomorphically encrypted entries of the first vector of the AI ​​module's attention module, in particular the first key vector, the first value vector and the first query vector, in the form of fully homomorphically encrypted coefficients of a first further polynomial that depends on the first input. Each one encrypted coefficient of the first further polynomial may represent the ciphertext of each one entry of the first vector in plaintext.

[0057] In one example, the AI ​​module may be configured to split a first input into multiple first input tokens and generate a first input embedding vector for each first input token, in particular by using the AI ​​module's embedding module. The AI ​​module may be configured to generate a set of first input embedding polynomials depending on the encrypted coefficients of the first input polynomial. The set of first input embedding polynomials may be represented as ciphertext in the form of the encrypted coefficients of the first input embedding polynomial.

[0058] Furthermore, the AI ​​module may be designed to project the first input embedding vector into its respective subspace. The AI ​​module may be designed to perform the projection by vector / matrix multiplication using projection matrices, e.g., a query projection matrix, a key projection matrix, and a value projection matrix. The first vector may be derived from these projections. In the case where the first vector includes a first key vector, a first value vector, and a first query vector, there may be three subspaces. In the plaintext domain, the first key vector may be represented as the first key matrix, the first value vector as the first value matrix, and the first query vector as the first query matrix.

[0059] In the ciphertext domain, a first vector may be represented in the form of a first tensor. For example, a first key vector may be represented as a first key tensor in the ciphertext domain, a first value vector may be represented as a first value tensor in the ciphertext domain, and a first query vector may be represented as a first query tensor in the ciphertext domain. Each entry in a first tensor, such as a first key tensor, a first value tensor, and a first query tensor, may be designed as one form of the encrypted coefficients of a further first polynomial. Since each entry in a first vector may be represented as a set of coefficients of a further first polynomial, a first tensor may have at least one more dimension than a first vector. In the case where a first key vector is also represented in the form of a first key tensor in the ciphertext domain, a first key tensor may have two more dimensions than a single first key vector.

[0060] The AI ​​module may be designed to compute the encrypted coefficients of a first further polynomial by performing a convolution of the encrypted coefficients of a first input embedding polynomial with encrypted projection matrix coefficients representing the entries of the projection matrix. The entries of the projection matrix may be expressed in the form of a projection polynomial. Therefore, the projection matrix coefficients representing the entries of the projection matrix may be the coefficients of the projection polynomial.

[0061] Generally, the encrypted coefficients of the first input embedding polynomial, the representation of the entries through the first further polynomial and projection polynomial can be considered as representations of the entries in the first input embedding vector, the first vector and projection matrix in the ciphertext. The encrypted coefficients of the first input embedding polynomial and projection polynomial may be obtained by encrypting the coefficients of the first input embedding polynomial and projection polynomial using the exemplary FHE scheme mentioned above.

[0062] The method may further include storing a portion of a first vector, in particular a first key vector and a first value vector, in a portion of a first tensor. For example, an AI module may be configured to store a first key tensor and a first value tensor. The entries in the portion of the first tensor may include coefficients of a portion of a first further polynomial associated with a portion of the first vector.

[0063] The method may further include determining the input values ​​of the function depending on the entries of a first vector. This may include variations in which the input values ​​are part of a set of input values ​​for the function, and the function approximates a softmax function, in particular the exponential softmax function of the Attention Module of the AI ​​Module.

[0064] For example, for simplicity, we consider a plaintext domain here, but the AI ​​module may be configured to compute the input values ​​as a result of multiplying one of several columns of a first key matrix with one of several rows of a first query matrix. In one example, for simplicity, we consider a plaintext domain here, but the AI ​​module may be designed to compute each input value of a set of input values ​​by performing the multiplication of each column of the first key matrix with each row of the first query matrix.

[0065] The AI ​​module may be configured to perform these multiplications in the ciphertext domain by performing a convolution of the entries of a first key tensor with the entries of a first query tensor. As described above, the input values ​​may be expressed in the form of an input polynomial, where the coefficients of the input polynomial are encrypted. Similarly, a set of input values ​​may be expressed in the form of an input polynomial, where the coefficients of the input polynomial are encrypted. Multiple input polynomials may comprise a single input polynomial.

[0066] The AI ​​module may be configured to determine the output value of a function depending on the input value. To achieve this, the AI ​​module may determine the output value of a function as a result of evaluating the associative polynomial, according to one of the modifications described above. In the case where the function approximates the exponential function of the softmax function, the evaluation of the associative polynomial may be applied multiple times, depending on each input value in the set of input values, in order to apply the softmax function.

[0067] In one example, for simplicity, we consider a plaintext domain, but the set of input values ​​may be represented in the form of an input matrix whose elements can be the input values ​​of the set of input values. Following this example, the AI ​​module may apply a softmax function to the elements of the input matrix column by column, or row by row, to obtain an output matrix. The function may be applied multiple times to obtain each element of the output matrix as a result of the softmax function. In this way, the function may be applied once for each input value of the set of input values, resulting in a set of output values ​​of the function. Each output value of the function may approximate the function value of an exponential function, depending on each input value of the set of input values. The AI ​​module may be configured to apply the softmax function given above, depending on the output values ​​of the set of output values ​​of the function. Hereafter, the output matrix may be referred to as the softmax matrix.

[0068] The AI ​​module may be configured to compute entries in a softmax matrix in the ciphertext domain by applying homomorphic mathematical operations, such as summation or convolution, to the encrypted coefficients of the input polynomial. Each entry in the softmax matrix may be expressed in the form of the encrypted coefficients of the softmax polynomial. Therefore, in the ciphertext domain, the softmax matrix may be expressed in the form of a softmax tensor. The entries in the softmax tensor may be the encrypted coefficients of the softmax polynomial.

[0069] The method may further include determining a first output of the AI ​​module depending on the output value of a function. The first output of the AI ​​module may be determined depending on a softmax matrix. Since the output matrix depends on the output value of the function, in this case the first output of the AI ​​module may also depend on the output value of the function. In one example, for simplicity, we consider the plaintext domain here, the AI ​​module may be configured to produce a first contextualized embedding vector by generating a linear combination of first value vectors using entries of a softmax matrix as linear factors. Each first contextualized embedding vector may be obtained from each linear combination of first value vectors using entries of each column of the softmax matrix as linear factors. The AI ​​module may be configured to generate a first contextualized embedding vector in the ciphertext domain by performing a convolution of entries of a first value tensor with entries of a softmax tensor.

[0070] In an exemplary application, the first output of the AI ​​module may be in the form of a first response phrase, particularly in the case where the AI ​​module is an LLM, such as a transformer-based LLM. The AI ​​module may be configured to use an embedding module to generate the first response phrase, depending on a first contextualized embedding vector. In one example, the embedding module may be designed to associate tokens with their respective embedding vectors and vice versa.

[0071] The method may further include receiving a second encrypted input. The second input may be provided as an updated input to an AI module. The second input may be a second question phrase. In one example, the second encrypted input may be provided in the form of encrypted coefficients of a second input polynomial.

[0072] The method may further include using an AI module to determine a second vector, in particular a second key vector, a second value vector, and a second query vector, depending on a second input. The AI ​​module may be configured to generate the second vector, similar to the first vector, but using the second input instead of the first input. For example, the AI ​​module may be designed to generate fully homomorphically encrypted entries of the second vector, in particular a second key vector, a second value vector, and a second query vector, in the form of fully homomorphically encrypted coefficients of a second further polynomial, depending on the second input. The encrypted coefficients of the second further polynomial may represent entries of the second vector in the ciphertext.

[0073] In one example, the AI ​​module may be configured to split a second input into multiple second input tokens and generate a second input embedding vector for each second input token, in particular by using the AI ​​module's embedding module. The AI ​​module may be configured to generate a set of second input embedding polynomials depending on the encrypted coefficients of the second input polynomials. The set of second input embedding polynomials may be represented as ciphertext in the form of the encrypted coefficients of the second input embedding polynomials.

[0074] Furthermore, the AI ​​module may be designed to project the second input embedding vectors into their respective subspaces. The AI ​​module may be designed to perform the projection of the second input embedding vectors by vector / matrix multiplication using projection matrices, e.g., a query projection matrix, a key projection matrix, and a value projection matrix. The second vectors may be derived from these projections. In the plaintext domain, the second key vector may be represented as the second key matrix, the second value vector as the second value matrix, and the second query vector as the second query matrix.

[0075] In the ciphertext domain, a second vector may be represented in the form of a second tensor. For example, a second key vector may be represented as a second key tensor, a second value vector as a second value tensor, and a second query vector as a second query tensor. Each entry in a second tensor, such as a second key tensor, a second value tensor, and a second query tensor, may be designed as one form of the encrypted coefficients of a second further polynomial.

[0076] The AI ​​module may be designed to compute the encrypted coefficients of a second further polynomial by performing a convolution of the encrypted coefficients of a second input embedding polynomial with the encrypted projection matrix coefficients representing the entries of the projection matrix.

[0077] The method may further include determining a second input value of the function depending on a stored first tensor and a second vector. Following the example given above, where the function can approximate the exponential function of the softmax function of the attention module, the second input value may be part of a second set of input values ​​for the function, and the function may be applied to the input values. The input values ​​of the second set of input values ​​may be presented sequentially as inputs to the function.

[0078] In one example, the AI ​​module may be configured to assemble a stored first tensor with a portion of a second tensor in order to compute a second input value depending on the stored first tensor and second vector. Since the second tensor may represent a second vector in the ciphertext, computed a second input value depending on the stored first tensor and second vector can be considered equivalent to computed a second input value depending on the stored first tensor and second tensor.

[0079] In one example, an AI module may assemble a first key tensor with a second key tensor by, for example, adding a second key tensor to the first key tensor, thereby yielding a combined key tensor. Similarly, an AI module may assemble a first value tensor with a second value tensor by, for example, adding a second value tensor to the first value tensor, thereby yielding a combined value tensor. Such assemblies may be represented in the plaintext domain as the assembly of the first key matrix with the second key matrix yielding a combined key matrix, and the assembly of the first value matrix with the second value matrix yielding a combined value matrix.

[0080] In one example, for simplicity, we consider a plaintext domain here, but the AI ​​module may be configured to compute a second input value as a result of multiplying one of several columns of an assembled key matrix with one of several rows of a second query matrix. In one example, for simplicity, we consider a plaintext domain here, but the AI ​​module may be designed to compute each input value of a second set of input values ​​by performing the multiplication of each column of an assembled key matrix with each row of a second query matrix.

[0081] The AI ​​module may be configured to perform these multiplications in the ciphertext domain by performing a convolution of the entries of the assembled key tensor with the entries of the second query tensor. As described above, the second input value may be expressed in the form of a second input polynomial, where the coefficients of the second input polynomial are encrypted. Similarly, the second set of input values ​​may be expressed in the form of a second input polynomial, where the coefficients of the second input polynomial are encrypted. The second input polynomial may contain the second input polynomial.

[0082] The AI ​​module may be configured to determine a second output value of the function depending on a second input value. To achieve this, the AI ​​module may determine a second output value of the function as a result of evaluating an associative polynomial using the second input value as input to the function, according to one of the modifications described above. The evaluation of the associative polynomial may be applied multiple times, depending on each input value of the second set of input values, in order to apply the softmax function.

[0083] In one example, for simplicity, a plaintext domain is considered here, but the second set of input values ​​may be represented in the form of a second input matrix whose elements can be the input values ​​of the second set of input values. Following this example, the AI ​​module may apply a softmax function to the elements of the second input matrix column by column, or row by row, to obtain a second output matrix. The function may be applied multiple times to obtain each element of the second output matrix as a result of the softmax function. In this way, the function may be applied once for each input value of the second set of input values, yielding a second set of output values ​​of the function. Each second output value of the function may approximate the function value of an exponential function depending on each input value of the second set of input values. The AI ​​module may be configured to apply the softmax function given above depending on the output values ​​of the second set of output values ​​of the function. Hereafter, the second output matrix may be referred to as the second softmax matrix.

[0084] The AI ​​module may be configured to compute entries for a second softmax matrix in the ciphertext domain by applying homomorphic mathematical operations, such as summation or convolution, to the encrypted coefficients of a second input polynomial. Each entry for the second softmax matrix may be expressed in the form of the encrypted coefficients of the second softmax polynomial. Therefore, in the ciphertext domain, the second softmax matrix may be expressed in the form of a second softmax tensor. The entries for the second softmax tensor may be the encrypted coefficients of the second softmax polynomial.

[0085] The method may further include determining the second output of the AI ​​module depending on the second output value of the function. The second output of the AI ​​module may be determined depending on a second softmax matrix. Since the second output matrix depends on the second output value of the function, in this case the second output of the AI ​​module may also depend on the second output value of the function.

[0086] In one example, for simplicity, we consider the plaintext domain here, but the AI ​​module may be configured to generate a linear combination of second value vectors using entries of a second softmax matrix as linear factors, yielding a second contextualized embedding vector. Each second contextualized embedding vector may be obtained from each linear combination of second value vectors using entries of each one column of the second softmax matrix as linear factors. The AI ​​module may be configured to generate these second contextualized embedding vectors in the ciphertext domain by performing a convolution of entries of a second value tensor with entries of a second softmax tensor.

[0087] In an exemplary application, the second output of the AI ​​module may be in the form of a second response phrase, particularly in the case where the AI ​​module is an LLM, such as a transformer-based LLM. The AI ​​module may be configured to use an embedding module to generate the second response phrase in reliance on a second contextualized embedding vector.

[0088] The advantage of storing the first tensor, particularly the first key tensor and the first value tensor, is that the information obtained by computing the first output of the AI ​​module can be used to compute the second output of the AI ​​module. In the case where the first key tensor and the first value tensor are not stored, all entries in the assembled key tensor and assembled value tensor must be newly computed to generate the second output of the AI ​​module. In this case, the first input must also be provided as input to the AI ​​module to compute the second output of the AI ​​module. The storage of the first key tensor and the first value tensor can also be considered a KV cache of polynomial coefficients in an FHE environment. Since applying the AI ​​module to encrypted data may require more computational steps compared to applying it to decrypted data, a KV cache of the first key tensor and the first value tensor can make it possible to reduce the number of computational steps required. Therefore, the overall time efficiency of the AI ​​module can be improved.

[0089] In one example, the method may further include providing a cache device for storing the first tensor. The cache device may be a cache device of a computer system, e.g., an L3 or L4 cache. In one example, the cache device may be a shared cache device shared by two or more processors of a set of processors. The set of processors may be located on a single chip. In one example, the cache device may be cache 821 as mentioned below, and the set of processors may be processor set 810 as mentioned below. The method may further include dividing the first tensor, in particular the first key tensor and the first value tensor, into multiple chunks of the first tensor. The method may further include storing the chunks in memory blocks of the cache device. The blocks may be independently accessible to read the chunks of the first tensor independently of each other. In one example, the AI ​​module may be configured to allocate blocks independently in a cache device, so that the blocks are independently accessible to write chunks to and read chunks from each other separately.

[0090] The AI ​​module may include a block table containing mappings between chunks of a first tensor and the blocks in which those chunks are stored. Blocks may be independently accessible using the block table. In one example, the block table may be designed in the form of a lookup table.

[0091] In one example, chunks of different tensors of the first tensor may be stored in a common block of blocks. A block may also be called a page.

[0092] In one example, the first page may store the first part of the first key tensor and the first part of the first value tensor. The second page may store the second part of the first key tensor and the second part of the first value tensor. This may allow for flexible computation of the second softmax matrix and / or the second contextualized embedding vector mentioned above. In one example, the block table may contain information about the number of filled positions in each block. This may allow for filling gaps in blocks, thereby reducing the risk of fragmentation of the cache device.

[0093] In one example, the method may further include allocating memory blocks discontinuously in the device. This could mean that blocks can be allocated anywhere in the cache device without the constraint that the logical internal order of the first tensor may be preserved. This could further reduce the risk of fragmentation in the cache device. In addition, allocating memory blocks discontinuously in the device may allow for gaps to be left between blocks that store the first tensor.

[0094] Instead of allocating a large contiguous block of the cache device to cache the first key tensor and the first value tensor, the cache device's memory is divided into smaller, in particular, fixed-size memory blocks, i.e., multiple pages. Each page may be allocated, stored, and accessible independently. This can allow for more efficient use of the cache device, as it does not need to find a large contiguous space within the cache device to store the entire first key tensor and / or the entire first value tensor. This may be an advantage for the dynamic use of the AI ​​module, which may include receiving more inputs than just the second input, for example, several further inputs in sequence. In addition, this can alleviate the parallel processing of the second input matrix mentioned above across several different CPUs. To achieve this, the first part of the second query tensor and the first part of the first key tensor may be sent to a first CPU of the computer system, and the second part of the second query tensor and the second part of the first key tensor may be sent to a second CPU of the computer system. In addition, as explained above, the distribution of the first tensor between blocks can make it possible to load chunks separately into the first or second CPU.

[0095] In the case where the first tensor is cached as contiguous data in a single block in the large contiguous space of the cache device, loading only a portion of the first tensor, such as the first part of the first key tensor, may not be possible because the first tensor can only be loaded as a whole, i.e., as a single data package, from a single block of the cache device. This can limit the AI ​​module in applications where the size of the first tensor is limited to what can be loaded into a single CPU. Since the use of the FHE AI module involves a larger first tensor compared to its use for unencrypted data, this can limit the area of ​​application for the AI ​​module. Such limitations can be eliminated by dividing the first tensor into multiple chunks and storing the chunks in blocks, where the blocks may be independently accessible to read the chunks of the first tensor independently of each other, as described above.

[0096] Figure 1 is a flowchart illustrating a method for calculating a fully homomorphically encrypted output value of an artificial intelligence module (AI module) function, as shown in Figure 2, based on a fully homomorphically encrypted input value of the function, as described in this subject. For illustrative purposes, the method described in Figure 1 may be implemented in the computing environment 800 shown in Figure 10, but is not limited to this implementation.

[0097] The method may include the following steps. The processor set 810 of the computer 801 shown in Figure 10 may be configured to execute the steps in operational steps. In step 10001, a fully homomorphically encrypted pre-calculated term may be provided. The pre-calculated term may be independent of the input values. The function may represent a nonlinear function to be approximated by the function. In one example, the nonlinear function may be a GELU function or a softmax function.

[0098] In step 10002, the output value of the function may be calculated depending on a pre-calculated term and a polynomial. Each polynomial may depend on the difference between the input value and the central value associated with each polynomial. The associated central value may indicate the center point of each polynomial. The associated central value may differ for each polynomial.

[0099] Figure 2 shows AI module 1, which can be considered an example of the AI ​​module mentioned above. AI module 1 may include a function module 10 for realizing a function in order to approximate a nonlinear function. Function module 10 may be configured to compute an encrypted output value 110 depending on an encrypted input value 101. Input value 101 can be considered an example of the input value mentioned above. Similarly, output value 110 can be considered an example of the output value mentioned above. Function module 10 may be configured to compute output value 110 depending on a fully homomorphically encrypted pre-computation term 120 and the input value 101. Pre-computation term 120 may function as an example of the pre-computation term mentioned above. Furthermore, function module 10 may be configured to compute the output value 110 of the function depending on the pre-computation term 120 and on a polynomial. Therefore, function module 10 may be considered an example of the implementation of a function in AI module 1.

[0100] Each polynomial may depend on a set of powers for that polynomial. Each power in each set may depend on the difference between the input value and the central value associated with each polynomial, as the base of the power, and on a natural number as the exponent of the power. The natural number may increase with each power of the polynomial.

[0101] Figure 3 shows an example of a polynomial. In this example, the polynomial may contain "n" polynomials. Figure 3 may, as an example, only show the first polynomial 130, the second polynomial 230, and the nth polynomial 330. For simplicity, the descriptions of mathematical operations shown in Figures 3, 4, and 5 may refer to the plaintext domain. It is understood that AI module 1 may be configured to perform these mathematical operations in the ciphertext domain. To achieve this, AI module 1, in particular function module 10, may be configured to apply homomorphic mathematical operations such as summation and convolution to the encrypted coefficients of the input polynomial described above, the encrypted coefficients of the cryptographic polynomial, the encrypted coefficients of further cryptographic polynomials, and / or the encrypted coefficients of further polynomials.

[0102] The first polynomial 130 may depend on a first difference 150 between the input value 101, shown as "x" in Figure 3, and the first central value 140, shown as x1 in Figure 3. The first central value 140 may be associated with the first polynomial 130. The first central value 140 may indicate a first central point centered on the first polynomial 130.

[0103] The first polynomial 130 may depend on a first set of powers. The first difference 150 may be the base of each power in the first set of powers. Each natural number in the first set of natural numbers may be equal to the exponent of each power in the first set of powers. The natural numbers in the first set of numbers may increase with each power in the first set of powers. For example, the first set of powers may include a first power 161, a second power 162, and a third power 163. The exponent of the first power 161 may be equal to 1, the exponent of the second power 162 may be equal to 2, and the exponent of the third power 163 may be equal to 3.

[0104] Figure 3 may also show an example in which the first polynomial 130 may depend on a first set of products. According to the example shown in Figure 3, the first set of products may include a first product 181, a second product 182, and a third product 183. The first product 181 may include the first coefficient 171 of the first polynomial 130 as a first factor, and the first power 161 as a second factor. The second product 182 may include the second coefficient 172 of the first polynomial 130 as a first factor, and the second power 162 as a second factor. The third product 183 may include the third coefficient 173 of the first polynomial 130 as a first factor, and the third power 163 as a second factor.

[0105] In addition to the first set of products, the first polynomial 130 may include a first principal 190 which may be equal to the function value of the nonlinear function at the first central value 140. The nonlinear function approximated by the function of AI module 1 may be shown as "f" in Figure 3. The first polynomial 130 may be constructed as the sum of the first principal 190, the first product 181, the second product 182, and the third product 183.

[0106] Figure 3 also shows an example of how the first polynomial 130 may take the form of the first Taylor polynomial when expanded at the first central value 140. In this example, the first coefficient 171 of the first polynomial 130 may be equal to the first derivative of the nonlinear function at the first central value 140. Furthermore, according to this example, the second coefficient 172 of the first polynomial 130 may be equal to the quotient where the dividend is the second derivative of the nonlinear function at the first central value 140 and the divisor is 2 factorial. The third coefficient 173 of the first polynomial 130 may be equal to the quotient where the dividend is the third derivative of the nonlinear function at the first central value 140 and the divisor is 3 factorial.

[0107] The second polynomial 230 may depend on a second difference 250 between the input value 101 and the second central value 240, which is shown as x2 in Figure 3. The second central value 240 may be associated with the second polynomial 230. The second central value 240 may indicate a second central point centered on the second polynomial 230.

[0108] The second polynomial, 230, may depend on a second set of powers. The second difference, 250, may be the base of each power in the second set of powers. Each natural number in the second set of natural numbers may be equal to the exponent of each power in the second set of powers. The natural numbers in the second set of numbers may increase with each power in the second set of powers. For example, the second set of powers may include a first power 261, a second power 262, and a third power 263. The exponent of the first power 261 may be equal to 1, the exponent of the second power 262 may be equal to 2, and the exponent of the third power 263 may be equal to 3.

[0109] Figure 3 may also show an example in which the second polynomial 230 may depend on a second set of products. According to the example shown in Figure 3, the second set of products may include a first product 281, a second product 282, and a third product 283. The first product 281 may include the first coefficient 271 of the second polynomial 230 as a first factor, and the first power 261 as a second factor. The second product 282 may include the second coefficient 272 of the second polynomial 230 as a first factor, and the second power 262 as a second factor. The third product 283 may include the third coefficient 273 of the second polynomial 230 as a first factor, and the third power 263 as a second factor.

[0110] In addition to the second set of products, the second polynomial 230 may include a second principal 290 which may be equal to the function value of the nonlinear function at the second central value 240. The second polynomial 230 may be constructed as the sum of the second principal 290, the first product 281, the second product 282, and the third product 283.

[0111] Figure 3 also shows an example of how the second polynomial 230 may take the form of the second Taylor polynomial expanded at the second central value 240. In this example, the first coefficient 271 of the second polynomial 230 may be equal to the first derivative of the nonlinear function at the second central value 240. Furthermore, according to this example, the second coefficient 272 of the second polynomial 230 may be equal to the quotient where the dividend is the second derivative of the nonlinear function at the second central value 240 and the divisor is 2 factorial. The third coefficient 273 of the second polynomial 230 may be equal to the quotient where the dividend is the third derivative of the nonlinear function at the second central value 240 and the divisor is 3 factorial.

[0112] The nth polynomial 330 may depend on the nth difference 350 between the input value 101 and the nth center value 340, which is shown as xn in Figure 3. The nth center value 340 may be associated with the nth polynomial 330. The nth center value 340 may indicate the nth center point centered on the nth polynomial 330.

[0113] The nth polynomial 330 may depend on the nth set of powers. The nth difference 350 may be the base of each power in the nth set of powers. Each natural number in the nth set of natural numbers may be equal to the exponent of each power in the nth set of powers. The natural numbers in the nth set of numbers may increase with each power in the nth set of powers. For example, the nth set of powers may include the first power 361, the second power 362, and the third power 363. The exponent of the first power 361 may be equal to 1, the exponent of the second power 362 may be equal to 2, and the exponent of the third power 363 may be equal to 3.

[0114] Figure 3 may also show an example in which the nth polynomial 330 may depend on the nth set of products. According to the example shown in Figure 3, the nth set of products may include a first product 381, a second product 382, ​​and a third product 383. The first product 381 may include the first coefficient 371 of the nth polynomial 330 as a first factor, and the first power 361 as a second factor. The second product 382 may include the second coefficient 372 of the nth polynomial 330 as a first factor, and the second power 362 as a second factor. The third product 383 may include the third coefficient 373 of the nth polynomial 330 as a first factor, and the third power 363 as a second factor.

[0115] In addition to the nth set of products, the nth polynomial 330 may include the nth principal 390, which may be equal to the function value of the nonlinear function at the nth central value 340. The nth polynomial 330 may be constructed as the sum of the nth principal 390, the first product 381, the second product 382, ​​and the third product 383.

[0116] Figure 3 also shows an example in which the nth polynomial 330 may take the form of the nth Taylor polynomial expanded at the nth central value 340. In this example, the first coefficient 371 of the nth polynomial 330 may be equal to the first derivative of the nonlinear function at the nth central value 340. Furthermore, according to this example, the second coefficient 372 of the nth polynomial 330 may be equal to the quotient where the dividend is the second derivative of the nonlinear function at the nth central value 340 and the divisor is 2 factorial. The third coefficient 373 of the nth polynomial 330 may be equal to the quotient where the dividend is the third derivative of the nonlinear function at the nth central value 340 and the divisor is 3 factorial.

[0117] It is understood that the polynomials in the first polynomial 130, the second polynomial 230, and the nth polynomial 330 can be expanded to degrees higher than 3, as shown in Figure 3. In one example, a polynomial may be expanded up to a maximum of degree 1024.

[0118] In the following example, for the sake of simplification, it may be assumed that the polynomial is expanded up to a maximum of degree 2. According to one variation of this example, the function may be constructed in the form of the associative polynomial 500 shown in Figure 5. The associative polynomial 500 can be obtained by combining the terms of a polynomial having the same natural number as its exponent. The terms of the polynomial can be obtained by expanding each set of powers of each polynomial and arranging them together in the form of a single formula 400, as shown in Figure 4. For example, the second power 162 and the third power 163 may be expanded by applying the binomial theorem. The expanded sets of powers, e.g., the first set of powers, the second set of powers, and so on up to the nth set of powers, and each unit, e.g., the first unit 190, the second unit 290 up to the nth unit 390, may be arranged together in the form of a single formula 400. A single formula 400 may include a first further product having a first term 121 as a first factor and an input value 101 as a second factor, a second further product including a second term 122, a third term 123 as a first factor and the square of the input value 101 as a second factor, a fourth term 124, and a third further product having an input value 101 as a first factor and a fifth term 125 as a second factor. The associative polynomial 500 may be obtained by combining like terms of formula 400. For example, the second term 122 may be added to the fourth term 124 to obtain the first associative term 501. Similarly, the first term 121 may be added to the fifth term 125 to obtain the second associative term 502. The first associative term 501 may be the first coefficient of the associative polynomial 500. The second associative term 502 may exhibit the second coefficient of the associative polynomial 500. The third term 123 may be considered the third coefficient of the associative polynomial 500.

[0119] For example, function module 10 may be configured to compute an output value 110 by evaluating an associative polynomial 500. The evaluation of the associative polynomial 500, which depends on the input value 101, may include adding the first associative term 501 to the product of the second associative term 502 and the input value 101, and the product of the third term 123 and the square of the input value 101. By computing the output value 110 by evaluating the associative polynomial 500, which depends on the input value 101, it can be noted that the output value 110 is still computed depending on polynomials, for example, the first polynomial 130, the second polynomial 230, and up to the nth polynomial 330, because the associative polynomial 500 is obtained by combining these polynomials. Furthermore, the calculation of the output value 110 by evaluating the associative polynomial 500 depending on the input value 101 may also depend on the difference mentioned above, where each difference may be the difference between the input value 101 and the respective central values ​​associated with each polynomial, because the associative polynomial 500 is obtained depending on the difference.

[0120] In one example, the first and second combined terms 501 and 502 may be part of the pre-calculated terms 120 shown in Figure 2. In one example, the pre-calculated terms 120 may be stored in the cache 821. The processor set 810 may execute the AI ​​module 1. Execution of the AI ​​module 1 may include executing the function module 10 at least once. The function module 10 may be configured to load the pre-calculated terms 120, in particular the first and second combined terms 501 and 502, from storage 824 into the cache 821. In one example, the function module 10 may include a command to load the pre-calculated terms 120 into the cache 821. The function module 10 may be configured to add the first combined term 501 to the product of the second combined term 502 and the input value 101, and the product of the third term 123 and the square of the input value 101, in order to obtain the output value 110.

[0121] Figure 7 shows an example in which AI module 1 may include an attention module 70 and an artificial neural network 80 (ANN80). The example given in Figure 6 can also be seen as an example in which AI module 1 is designed in the form of a transformer-based large-scale language model.

[0122] AI module 1 may be configured to generate output 1010 of AI module 1 depending on input 1001 of AI module 1, using attention module 70 and, in particular, ANN 80. Below, variations of function module 10 are described in which function module 10 is used to realize a function such that the function approximates a nonlinear function, in particular an exponential function, of attention module 70.

[0123] Figure 7 shows one example of how the processor set 810 may execute the attention module 70. For simplicity, the execution of the attention module 70 is explained using vectors and matrices to illustrate the basic principles of the attention module 70 in the plaintext domain.

[0124] AI module 1 may include an embedding module 90 for converting the first input mentioned above into a first input token 610. The first input token 610 may be an example of the first input token mentioned above. The first input can be considered an example of input 1001 of AI module 1 shown in Figures 2 and 6. The embedding module 90 may be configured to generate a first input embedding vector 620 depending on the first input token 610. The first input embedding vector 620 may be an example of the first input embedding vector mentioned above.

[0125] The attention module 70 may be configured to project the first input embedding vector 620 onto each of the subspaces mentioned above. For example, the attention module 70 may multiply the first input embedding vector 620 by the query projection matrix mentioned above to obtain the first query vector 630. Furthermore, the attention module 70 may perform multiplication of the first input embedding vector 620 with the key projection matrix mentioned above to obtain the first key vector 640. In addition, the attention module 70 may perform multiplication of the first input embedding vector 620 with the value projection matrix to obtain the first value vector 650. In response to generating the first key vector 640 and the first value vector 650, the AI ​​module may be configured to initialize a command to store the first key vector 640 and the first value vector 650 in a cache device of the computer 801, for example, cache 821. The first key vector 640 and the first value vector 650 may be stored in the cache 821 in the ciphertext in the form of the first key tensor 641 and the first value tensor 651. The first key tensor 641 and the first value tensor 651 may represent examples of the first key tensor and the first value tensor mentioned above, respectively.

[0126] For clarity, the elements of vectors and matrices are shown in the form of rectangles in Figure 7. In one example, the attention module 70 may be configured to multiply each of the first key vectors 640 by each of the first query vectors 630. The result of each of these vector / vector multiplications may be shown in the form of a small rectangle representing an element of the input matrix 660. The input matrix 660 may be an example of the input matrix mentioned above.

[0127] The attention module 70 may be configured to apply a softmax function column by column to the input matrix 660, yielding a softmax matrix 670. The softmax matrix 670 may be an example of the softmax matrix mentioned above. In Figure 7, the elements of the softmax matrix 670 are shown in the form of small rectangles. The sum of all elements in each column of the softmax matrix 670 may be equal to 1. In one example, the attention module 70 may send a request to the function module 10 to compute the output value of the function depending on the input value of the function. The attention module 70 may send a request whenever the computation of an exponential function may be required to execute the softmax function. Therefore, the attention module 70 may send several requests. Each request may contain each of the input values ​​of the input matrix 660. The attention module 70 may receive the output value of the function from the function module 10. The function may approximate the exponential function of the softmax function mentioned above. Therefore, in this case, the approximated function "f" shown in Figures 3 and 4 may be an exponential function.

[0128] In one example, the attention module 70 may be configured to generate a linear combination of first value vectors 650 using entries of a softmax matrix 670 as linear factors, resulting in a first contextualized embedding vector 680. Each first contextualized embedding vector of the first contextualized embedding vector 680 may be obtained from each linear combination of the first value vectors 650, using entries from each single column of the softmax matrix 670 as linear factors.

[0129] The first contextualized embedding vector 680 may be an example of the first contextualized embedding vector mentioned above. In one example, AI module 1 may be configured to determine output 1010 depending on the first contextualized embedding vector 680. Output 1010 may be an example of the first output of the AI ​​module mentioned above. In one example, AI module 1 may input the first contextualized embedding vector 680, or further data that can be generated depending on the first contextualized embedding vector 680, into ANN80. In this case, ANN80 may be configured to determine output 1010 depending on the first contextualized embedding vector 680 and / or further data. In one example, the activation function of ANN80 may be approximated using a set of further function modules of AI module 1, which are not shown in the figure. The functions of the further set of functions may be designed similarly to function module 10. The function approximated by the functions of the further set of functions may be a GELU function. Therefore, in this case, the approximated function "f" shown in Figures 3 and 4 may be the GELU function described above.

[0130] In the case where AI Module 1 is designed in the form of a transformer-based LLM, AI Module 1 may include several further attention modules and several further ANNs. The attention modules may include self-attention modules and cross-attention modules. It is assumed that the transformer-based LLM is known, and therefore the structure of the transformer-based LLM is not explicitly described.

[0131] Figure 8 shows how the AI ​​module 1 may execute the attention module 70 in response to receiving a second encrypted input 1002 of the AI ​​module 1, as shown by the dotted line in Figure 6. The embedding module 90 may convert the second input 1002 into a second input token 1810, which may be an example of the second input token mentioned above. The embedding module 90 may be configured to generate a second input embedding vector 1820 depending on the second input token 1810, which may be an example of the second input embedding vector mentioned above.

[0132] The attention module 70 may be configured to project the second input embedding vector 1820 onto each of the subspaces mentioned above. For example, the attention module 70 may multiply the second input embedding vector 1820 by the query projection matrix mentioned above to obtain the second query vector 1830. Furthermore, the attention module 70 may multiply the second input embedding vector 1820 by the key projection matrix mentioned above to obtain the second key vector 1840. In addition, the attention module 70 may multiply the second input embedding vector 1820 by the value projection matrix to obtain the second value vector 1850.

[0133] For clarity, the elements of vectors and matrices are shown in the form of rectangles in Figure 8. In one example, attention module 70 may be configured to multiply each of the second query vectors 1830 by each of the second key vectors 1840 and each of the first key vectors 640. To perform these multiplications in the ciphertext domain, AI module 1 may use the first key tensor 641 and the second query tensor and second key tensor mentioned above, which are stored in cache 821. The second query tensor may represent the second query vector 1830 in the ciphertext domain, as described above. Similarly, the second key tensor may represent the second key vector 1840 in the ciphertext domain. The result of each of these vector / vector multiplications may be shown in the form of a small rectangle representing each of the elements of the second input matrix 1860. The AI ​​module, in particular the attention module 70, may be configured to perform these vector / vector multiplications in the ciphertext domain by performing the aforementioned convolution of the entries of the second query tensor with the first key tensor 641 and the entries of the second key tensor.

[0134] The attention module 70 may be configured to apply a softmax function column by column to a second input matrix 1860 to obtain a second softmax matrix 1870. The second softmax matrix 1870 may be an example of the second softmax matrix mentioned above. In Figure 8, the elements of the second softmax matrix 1870 are shown in the form of small rectangles. The sum of all elements in each column of the second softmax matrix 1870 may be equal to 1. In one example, the attention module 70 may send a request to the function module 10 to calculate the output value of the function depending on the input value of the function. The attention module 70 may send a request whenever it is necessary to calculate an exponential function to perform the softmax function in order to determine the second softmax matrix 1870. Thus, the attention module 70 may send several requests. Each request may contain each of the input values ​​of the second input matrix 1860. The attention module 70 may receive the output value of a function from the function module 10. The function may approximate the exponential function of the softmax function mentioned above.

[0135] In one example, the attention module 70 may be configured to generate a combined set of value vectors 2850, which includes a first value vector 650 and a second value vector 1850. For example, the attention module 70 may concatenate the first value vector 650 with the second value vector 1850 to obtain the combined set of value vectors 2850.

[0136] In one example, the attention module 70 may be configured to generate a linear combination of a set of value vectors 2850 using entries from a second softmax matrix 1870 as linear factors, resulting in a second contextualized embedding vector 1880. Each second contextualized embedding vector of the second contextualized embedding vector 1880 may be obtained from each linear combination of the set of value vectors 2850, using entries from each of the columns of the second softmax matrix 1870 as linear factors.

[0137] The second contextualization embedding vector 1880 may be an example of the second contextualization embedding vector mentioned above. In one example, AI module 1 may be configured to depend on the second contextualization embedding vector 1880 to determine a second output 1020, shown by the dotted line in Figure 6. The second output 1020 may be an example of the second output of the AI ​​module mentioned above.

[0138] Similar to the reception of the second input 1002, further inputs to AI module 1 may be received, and for each further input, further outputs to AI module may be generated in each iteration. In each iteration, a new set of key vectors, such as the second key vector 1840, and a new set of value vectors, such as the second value vector 1850, may be generated, and the storage of these vectors in cache 821 in the ciphertext domain in the form of new key tensors and new value tensors may be performed in the same way as the storage of the first key tensor 641 and the first value tensor 651. In this way, the information obtained by processing the previous input to AI module 1 may be used to process the current input. In this case, the first input to AI module 1 may be the previous input, and the second input to AI module 1 may be the current input. Since the attention mechanism may have a quadratic complexity with respect to the length of the input, the computational requirements for the computer system may be reduced by caching new key tensors and new value tensors in each iteration.

[0139] Figure 9 shows an example of how entries in the second input matrix 1860 may be computed in parallel using multiple processors of a set of processors 810. In one example, the set of processors 810 may include a first processor 810.1, a second processor 810.2, a third processor 810.3, and a fourth processor 810.4. The first processor 810.1 may include a first cache 1821, the second processor 810.2 may include a second cache 2821, the third processor 810.3 may include a third cache 3821, and the fourth processor 810.4 may include a fourth cache 4821. Caches 1821, 2821, 3821, and 4821 may be individual caches for processors 810.1, 810.2, 810.3, and 810.4, for example, in the form of L1 or L2 caches, while cache 821 may be a shared cache for processors 810.1, 810.2, 810.3, and 810.4. Cache 821 may be individually accessible by each of processors 810.1, 810.2, 810.3, and 810.4, which are also referred to below as thread processors.

[0140] In one example, AI module 1, in particular attention module 70, may be configured to allocate different memory blocks of cache 821 to cache different parts of the first key tensor and different parts of the first value tensor.

[0141] The first part of the first key tensor labeled as 1640.1 in Figure 9 and hereafter referred to as the first key tensor part 1640.1 may represent, in the ciphertext domain, the first part of the first key vector 640 labeled as 640.1 in Figure 8 and hereafter referred to as the first key vector part 640.1. The first key vector part 640.1 may contain one or more vectors from the first key vector 640. For each entry of the first key vector part 640.1, the first key tensor part 1640.1 may contain each set of encrypted coefficients of the first further polynomial mentioned above.

[0142] Similarly, the second part of the first key tensor labeled as 1640.2 in Figure 9 and hereafter referred to as the second key tensor part 1640.2 may represent, in the ciphertext domain, the second part of the first key vector 640 labeled as 640.2 in Figure 8 and hereafter referred to as the second key vector part 640.2. The second key vector part 640.2 may contain one or more vectors from the first key vector 640. For each entry of the second key vector part 640.2, the second key tensor part 1640.2 may contain each set of encrypted coefficients of the first further polynomial mentioned above.

[0143] In Figure 9, the first part of the first value tensor labeled as 1650.1 and hereafter referred to as the first value tensor part 1650.1 may represent, in the ciphertext domain, the first part of the first value vector 650, labeled as 650.1 in Figure 8 and hereafter referred to as the first value vector part 650.1. The first value vector part 650.1 may contain one or more vectors from the first value vector 650. For each entry of the first value vector part 650.1, the first value tensor part 1650.1 may contain each set of encrypted coefficients of the first further polynomial mentioned above.

[0144] Similarly, the second part of the first value tensor labeled as 1650.2 in Figure 9 and hereafter referred to as the second value tensor part 1650.2 may represent, in the ciphertext domain, the second part of the first value vector 650 labeled as 650.2 in Figure 8 and hereafter referred to as the second value vector part 650.2. The second value vector part 650.2 may contain one or more vectors from the first value vector 650. For each entry of the second value vector part 650.2, the second value tensor part 1650.2 may contain each set of encrypted coefficients of the first further polynomial mentioned above.

[0145] The first part of the second query tensor labeled as 2830.1 in Figure 9 and hereafter referred to as the first query tensor part 2830.1 may represent, in the ciphertext domain, the first part of the second query vector 1830 labeled as 1830.1 in Figure 8 and hereafter referred to as the first query vector part 1830.1. The first query vector part 1830.1 may contain one or more vectors from the second query vector 1830. For each entry of the first query vector part 1830.1, the first query tensor part 2830.1 may contain each set of encrypted coefficients of the second further polynomial mentioned above.

[0146] Similarly, the second part of the second query tensor labeled as 2830.2 in Figure 9 and hereafter referred to as the second query tensor part 2830.2 may represent, in the ciphertext domain, the second part of the second query vector 1830 labeled as 1830.2 in Figure 8 and hereafter referred to as the second query vector part 1830.2. The second query vector part 1830.2 may contain one or more vectors from the second query vector 1830. For each entry of the second query vector part 1830.2, the second query tensor part 2830.2 may contain each set of encrypted coefficients of the second further polynomial mentioned above.

[0147] The first key tensor portion 1640.1, the second key tensor portion 1640.2, the first value tensor portion 1650.1, and the second value tensor portion 1650.2 can be considered examples of the aforementioned chunks of the first tensor.

[0148] AI module 1, in particular attention module 70, may include commands for dividing cache 821 into memory blocks, for example, a first block 821.1, a second block 821.2, a third block 821.3, and a fourth block 821.4, depending on the predicted memory requirements of AI module 1. AI module 1 may be configured to specify the predicted memory requirements depending on input 1001, for example, depending on the length of input 1001.

[0149] AI module 1, in particular attention module 70, may include a command or set of commands for initializing the cache of the first key tensor portion 1640.1 in the first block 821.1 in response to generating the first key tensor portion 1640.1; a command or set of commands for initializing the cache of the second key tensor portion 1640.2 in the second block 821.2 in response to generating the second key tensor portion 1640.2; a command or set of commands for initializing the cache of the first value tensor portion 1650.1 in the third block 821.3 in response to generating the first value tensor portion 1650.1; and a command or set of commands for initializing the cache of the second value tensor portion 1650.2 in the fourth block 821.4 in response to generating the second value tensor portion 1650.2.

[0150] Figure 9 may show an example in which the first key tensor portion 1640.1, the second key tensor portion 1640.2, the first value tensor portion 1650.1, and the second value tensor portion 1650.2 can be cached in a memory block of cache 821, where the blocks are allocated discontinuously in cache 821. An empty block 821.1 of the memory block of cache 821 may have a second block 821.2 and a third block 821.3 as its neighbors. The phrase "blocks are allocated discontinuously" may also include an example not shown in the figure in which an empty block 821.5 is located between two blocks of a cache device that store only two distinct portions of the first key tensor portion 1640.1, the second key tensor portion 1640.2, the first value tensor portion 1650.1, or the second value tensor portion 1650.2.

[0151] The main processor 810.0 of the processor set 810 may be configured to access blocks 821.1, 821.2, 821.3, and 821.4 individually and to load the first key tensor portion 1640.1, the second key tensor portion 1640.2, the first value tensor portion 1650.1, and the second value tensor portion 1650.2 separately into the thread processor's cache. The cache of the main processor 810.0 may store the value of the first pointer 911 pointing to the first block 821.1, the value of the second pointer 912 pointing to the second block 821.2, the value of the third pointer 913 pointing to the third block 821.3, and the value of the fourth pointer 914 pointing to the fourth block 821.4, in order to load the first key tensor portion 1640.1, the second key tensor portion 1640.2, the first value tensor portion 1650.1, and the second value tensor portion 1650.2 separately into the cache of the main processor 810.0.

[0152] In one example, the main processor 810.0 may be configured to compute the first query tensor portion 2830.1 and the second query tensor portion 2830.2. Figure 9 may show one example of how the main processor 810.0 may distribute chunks of the first tensor in order to compute entries for the second input matrix 1860 in parallel.

[0153] The main processor 810.0 may send the first pointer 911 and the first query tensor portion 2830.1 to the first processor 810.1. The first processor 810.1 may load the first key tensor portion 1640.1 into the first cache 1821. In one example, the first processor 810.1 may compute a first encrypted set of polynomials that can represent the first entry 1860.1 of the second input matrix 1860 in the ciphertext domain.

[0154] The main processor 810.0 may send the first pointer 911 and the second query tensor portion 2830.2 to the second processor 810.2. The second processor 810.2 may load the first key tensor portion 1640.1 into the second cache 2821. In one example, the second processor 810.2 may compute a second encrypted set of polynomials that can represent the second entry 1860.2 of the second input matrix 1860 in the ciphertext domain.

[0155] Similarly, the main processor 810.0 may send the second pointer 912 and the first query tensor portion 2830.1 to the third processor 810.3. The third processor 810.3 may load the second key tensor portion 1640.2 into the third cache 3821. In one example, the third processor 810.3 may compute an encrypted third set of polynomials that can represent the third entry 1860.3 of the second input matrix 1860 in the ciphertext domain.

[0156] Similarly, the main processor 810.0 may send the second pointer 912 and the second query tensor portion 2830.2 to the fourth processor 810.4. The fourth processor 810.4 may load the second key tensor portion 1640.2 into the fourth cache 4821. In one example, the fourth processor 810.4 may compute an encrypted fourth set of polynomials that can represent the fourth entry 1860.4 of the second input matrix 1860 in the ciphertext domain.

[0157] Dividing the cache 821 into blocks and making them independently accessible could enable thread processors to access the first tensor, for example, the first key tensor portion 1640.1, the second key tensor portion 1640.2, the first value tensor portion 1650.1, and the second value tensor portion 1650.2. This could enable faster computation of the second input matrix 1860, and thereby faster computation of the output of AI module 1.

[0158] The term “module,” as used herein, refers to any known or future-developed hardware, software, e.g., executable programs, artificial intelligence, fuzzy logic, or any combination thereof, for executing functions associated with a “module,” or as a result of executing functions associated with a “module.” For example, the main processor 810.0 may include an AI module program containing instructions for processor 810.0 or a thread processor. The main processor 810.0 or a thread processor may execute functions associated with AI module 1 when processing instructions in the AI ​​module program. Each function associated with AI module 1 may be described using the introductory phrase, “...AI module 1 is configured to….” Similarly, the main processor 810.0 may include a function module program containing instructions for processor 810.0 or a thread processor. The main processor 810.0 or a thread processor may execute functions associated with function module 10 when processing instructions in the function module program. Each function associated with function module 10 may be described using the introductory phrase, “...function module 10 is configured to….” The AI ​​module program and the function module program may be part of code 900 as described below.

[0159] The computing environment 800 includes an example of an environment for executing at least some of the computer code involved in performing the method of the present invention, such as code 900 for calculating fully homomorphically encrypted output values ​​of functions of an artificial intelligence module relating to one of the modifications described above. The computing environment 800 may be one example of the computer systems mentioned above. In addition to block 900, the computing environment 800 includes, for example, a computer 801, a wide area network (WAN) 802, an end user device (EUD) 803, a remote server 804, a public cloud 805, and a private cloud 806. In this embodiment, the computer 801 includes a processor set 810 (including processing circuits 820 and a cache 821), a communication fabric 811, volatile memory 812, persistent storage 813 (including an operating system 822 and blocks 900 as identified above), a peripheral device set 814 (including a user interface (UI) device set 823, storage 824, and an Internet of Things (IoT) sensor set 825), and a network module 815. The remote server 804 includes a remote database 830. The public cloud 805 includes a gateway 840, a cloud orchestration module 841, a host physical machine set 842, a virtual machine set 843, and a container set 844.

[0160] Computer 801 may take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device currently known or to be developed in the future that is capable of executing programs, accessing networks, or querying databases such as remote database 830. As is well understood in the field of computer technology, and depending on the technology, the execution of the computer implementation method may be distributed among multiple computers and / or across multiple locations. On the other hand, in this presentation of the computing environment 800, in order to keep the presentation as concise as possible, the detailed discussion focuses on a single computer, specifically computer 801. Computer 801 may be located in the cloud, although it is not shown in the cloud in Figure 10. On the other hand, computer 801 is not required to be located in the cloud, except to any extent that can be definitively shown.

[0161] The processor set 810 includes one or more computer processors of any type currently known or to be developed in the future. The processing circuitry 820 may be distributed across multiple packages, e.g., multiple coordinated integrated circuit chips. The processing circuitry 820 may implement multiple processor threads and / or multiple processor cores. The cache 821 is memory located within the processor chip package and is typically used for data or code that should be available for rapid access by threads or cores running on the processor set 810. The cache memory is typically organized into multiple levels, depending on its relative proximity to the processing circuitry. Alternatively, some or all of the cache for the processor set may be located "off-chip". In some computing environments, the processor set 810 may operate using qubits and be designed to perform quantum computing.

[0162] Computer-readable program instructions are typically loaded onto computer 801 to implement a computer implementation method by having the processor set 810 of computer 801 perform a series of operational steps, thereby instantiating the instructions thus executed in the computer implementation method contained herein (collectively referred to as the "Methods of the Invention"), for example, the flowchart and / or method specified in the description of a method for calculating a fully homomorphically encrypted output value of a function of an artificial intelligence module relating to one of the modifications mentioned above. These computer-readable program instructions are stored in various types of computer-readable storage media, such as the cache 821 and other storage media discussed below. The program instructions and associated data are accessed by the processor set 810 to control and direct the execution of the Methods of the Invention. In the computing environment 800, at least some of the instructions for executing the Methods of the Invention may be stored in block 900 in persistent storage 813.

[0163] The communication fabric 811 is a signal conduction path that enables various components of the computer 801 to communicate with one another. Typically, this fabric is made up of switches and conductive paths, such as buses, bridges, physical input / output ports, and similar switches and conductive paths. Other types of signal communication paths, such as fiber optic communication paths and / or wireless communication paths, may be used.

[0164] Volatile memory 812 is any type of volatile memory currently known or to be developed in the future. Examples include dynamic type random access memory (RAM) or static type RAM. Typically, volatile memory 812 is characterized by random access, but this is not required unless explicitly stated. In computer 801, volatile memory 812 is located in a single package and resides inside computer 801, but alternatively or in addition, volatile memory may be distributed across multiple packages and / or located externally to computer 801.

[0165] Persistent storage 813 is any form of non-volatile storage for a computer, currently known or to be developed in the future. The non-volatility of this storage means that the stored data is maintained whether or not power is directly supplied to computer 801 and / or persistent storage 813. Persistent storage 813 may be read-only memory (ROM), but typically at least a portion of the persistent storage allows for writing, deleting, and rewriting of data. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. Operating system 822 may take several forms, such as various known proprietary operating systems or open-source portable operating system interface (CSI) type operating systems employing a kernel. The code contained in block 900 typically includes at least some computer code involved in performing the method of the present invention.

[0166] The peripheral device set 814 includes a set of peripheral devices for the computer 801. Data communication connections between the peripheral devices and other components of the computer 801 may be implemented in various ways, such as Bluetooth® connection, Near-Field Communication (NFC) connection, cable connection (such as a universal serial bus (USB) type cable), insertable connection (e.g., a secure digital (SD) card), connection via a local area communication network, and even connection via a wide area network such as the Internet. In various embodiments, the UI device set 823 may include components such as a display screen, speaker, microphone, wearable device (such as goggles and smartwatches), keyboard, mouse, printer, touchpad, game controller, and haptic device. Storage 824 is external storage such as an external hard drive, or insertable storage such as an SD card. Storage 824 may be persistent and / or volatile. In some embodiments, storage 824 may take the form of a quantum computing memory device for storing data in the form of qubits. In embodiments where computer 801 is required to have a large amount of storage (for example, computer 801 locally stores and manages a large database), this storage may be provided by peripheral storage devices designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. The IoT sensor set 825 consists of sensors that can be used in Internet of Things applications. For example, one sensor may be a thermometer and another may be a motion detector.

[0167] The network module 815 is a collection of computer software, hardware, and firmware that enables computer 801 to communicate with other computers via the WAN 802. The network module 815 may include hardware such as a modem or Wi-Fi® signal transceiver, software for packetizing and / or depacketizing data for transmission over a communication network, and / or web browser software for communicating data over the Internet. In some embodiments, the network control and network forwarding functions of the network module 815 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software-defined networking (SDN)), the control and forwarding functions of the network module 815 are performed on physically separate devices, such that the control function manages several different network hardware devices. Computer-readable program instructions for performing the method of the present invention can typically be downloaded from an external computer or external storage device to computer 801 via a network adapter card or network interface included in the network module 815.

[0168] WAN802 is any wide area network (e.g., the Internet) capable of transmitting computer data over non-local distances by any currently known or future-developed technology for transmitting computer data. In some embodiments, WAN802 may be replaced and / or supplemented by a local area network (LAN), such as a Wi-Fi network, designed to transmit data between devices located in a local area. WANs and / or LANs typically include computer hardware such as copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0169] The end-user device (EUD) 803 is any computer system used and controlled by an end-user (e.g., a customer of the company operating computer 801) and can take any of the forms discussed above in relation to computer 801. Typically, EUD 803 receives useful and valuable data from the operation of computer 801. For example, in a hypothetical case where computer 801 is designed to provide recommendations to the end-user, these recommendations would typically be communicated from the network module 815 of computer 801 to EUD 803 via WAN 802. In this way, EUD 803 can display or otherwise present the recommendations to the end-user. In some embodiments, EUD 803 may be a client device such as a thin client, heavy client, mainframe computer, or desktop computer.

[0170] The remote server 804 is any computer system that provides at least some data and / or functionality to computer 801. The remote server 804 may be controlled and used by the same entity that operates computer 801. The remote server 804 represents a machine that collects and stores useful and valuable data for use by other computers, such as computer 801. For example, in the hypothetical case where computer 801 is designed and programmed to provide recommendations based on historical data, this historical data may be provided to computer 801 from the remote database 830 of the remote server 804.

[0171] Public Cloud 805 is any computer system available for use by multiple entities, providing on-demand availability of computer system resources and / or other computer functions, particularly data storage (cloud storage) and computing power, without direct active management by the user. Cloud computing typically leverages resource sharing to achieve coherence and economies of scale. Direct active management of the computing resources of Public Cloud 805 is performed by the computer hardware and / or software of the Cloud Orchestration Module 841. The computing resources provided by Public Cloud 805 are typically implemented by virtual computing environments running on various computers that make up the computers of the host physical machine set 842, which is the area of ​​physical computers available in and / or to Public Cloud 805. Virtual computing environments (VCEs) typically take the form of virtual machines from the virtual machine set 843 and / or containers from the container set 844. These VCEs may be stored as images and are understood to be transferable either as images or after instantiation of VCEs, among and between various physical machine hosts. The cloud orchestration module 841 manages image transfer and storage, deploys new VCE instances, and manages active instanceizations of VCE deployments. The gateway 840 is a collection of computer software, hardware, and firmware that enables the public cloud 805 to communicate over the WAN 802.

[0172] Here, some further explanation of virtualized computing environments (VCEs) is provided. A VCE can be stored as an "image." A new active instance of a VCE can be instantiated from an image. Two well-known types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature in the kernel that allows for the existence of multiple isolated user-space instances called containers. These isolated user-space instances typically behave as actual computers from the perspective of the programs running in them. Computer programs running on a normal operating system can utilize all of that computer's resources, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, programs running inside a container can only use the contents of the container and the devices allocated to the container; this feature is known as containerization.

[0173] Private Cloud 806 is similar to Public Cloud 805, except that its computing resources are available only for use by a single enterprise. While Private Cloud 806 is shown communicating with WAN 802, in other embodiments, the private cloud may be completely isolated from the internet and accessible only via a local / private network. A hybrid cloud is a combination of multiple clouds of different types (e.g., private, community, or public cloud types), often implemented by different vendors. Each of the multiple clouds remains a distinct and isolated entity, but the larger hybrid cloud architecture is bound together by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability between the multiple configured clouds. In this embodiment, both Public Cloud 805 and Private Cloud 806 are part of a larger hybrid cloud.

[0174] Cloud computing services and / or microservices (not shown separately in Figure 10): Private and public clouds are programmed and configured to deliver cloud computing services and / or microservices (unless otherwise indicated, the term “microservices” is interpreted to encompass larger “services” of any size). Cloud services are typically infrastructure, platforms, or software hosted by a third-party provider and made available to users over the internet. Cloud services facilitate the flow of user data from front-end clients (e.g., user-side servers, tablets, desktops, laptops) over the internet to the provider’s systems, and vice versa. In some embodiments, cloud services may be configured and organized according to an “as a service” technology paradigm, where something is presented to internal or external customers in the form of cloud computing services. An as a service offering typically provides endpoints that various customers interface with. These endpoints are typically based on a set of APIs. One category of as-a-service offerings is Platform as a Service (PaaS), where a service provider provisions, instantiates, runs, and manages modular bundles of code that customers can use to instantiate a computing platform and one or more applications, typically without the complexity of building and maintaining the associated infrastructure. Another category is Software as a Service (SaaS), where software is centrally hosted and allocated on a subscription basis. SaaS is also known as on-demand software, web-based software, or web-hosted software.The four technical subfields involved in cloud services are: deployment, integration, on-demand, and virtual private networks.

[0175] Various aspects of this disclosure are described by explanatory text, flowcharts, block diagrams of computer systems and / or block diagrams of machine logic included in computer program product (CPP) embodiments. With respect to any flowchart, operations may be performed in a different order than those shown in a given flowchart, depending on the technology involved. For example, again depending on the technology involved, two operations shown in consecutive flowchart blocks may be performed in reverse order, as a single integrated stage, simultaneously, or with at least partial time overlap.

[0176] Computer program product embodiment ("CPP embodiment" or "CPP") is a term used in this disclosure to describe any set of one or more storage media ("mediums") that collectively comprise a set of one or more storage devices that collectively contain machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. A "storage device" is any tangible device capable of holding and storing instructions for use by a computer processor. Computer-readable storage media may be, but are not limited to, electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, mechanical storage media, or any suitable combination of those described above. Some known types of storage devices, including these media, include: diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices (such as pits / lands formed on the main surface of a punch card or disk), or any suitable combination of the foregoing. When the term "computer-readable storage medium" is used in this disclosure, it shall not be construed as storage in the form of transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides, optical pulses passing through optical fiber cables, electrical signals communicated through wires, and / or other transmitting media.As those skilled in the art will understand, data typically moves at several intermittent points during the normal operation of a storage device, such as during access, defragmentation, or garbage collection; however, this does not make the storage device transient, because data is not transient while it is stored.

Claims

1. A method for calculating a fully homomorphically encrypted output value of a function of a transformer-based AI module, depending on the fully homomorphically encrypted input value of the function, wherein the function represents a nonlinear function to be approximated by the function, and the method is: A step of providing a fully homomorphically encrypted pre-calculation term independent of the aforementioned input value; and The step of using the transformer-based AI module to calculate the output value of the function, depending on the pre-calculated term and the polynomial. A method comprising the following, wherein each polynomial depends on the difference between the input value and the central value associated with each polynomial, the associated central value indicates the center point of each polynomial, and the associated central value is different for each polynomial.

2. The method according to claim 1, wherein each of the aforementioned polynomials depends on a set of powers for each of the aforementioned polynomials, and each of the aforementioned powers of the set depends on the difference between the input value and the central value associated with each of the aforementioned polynomials as the base of each aforementioned power, and on a natural number as the exponent of each aforementioned power, the natural number increases with each power of the aforementioned polynomial.

3. The method according to claim 2, wherein each of the polynomials depends on a set of products of each of the polynomials, and each of the products of each of the sets includes as factors one of the pre-calculated terms and one of the powers of each of the sets.

4. The step of calculating the output value of the function depending on the polynomial and the pre-calculated term is: A step of providing an associative polynomial resulting from combining terms having the same natural number as their exponent, wherein the terms are obtained by expanding each set of powers of each of the respective polynomials and arranging the expanded sets of powers together in the form of a single mathematical expression, the associative polynomial comprising the pre-calculated terms and further terms that depend on the input values ​​and include the natural number as their exponent; A step of evaluating the further terms depending on the input value; and A step in which the output value is calculated as a result of evaluating the combined polynomial based on the pre-calculated term and the further evaluated term. The method according to claim 2, further comprising:

5. The method according to claim 4, wherein the step of providing the associative polynomial further includes the step of providing fully homomorphically encrypted coefficients of the associative polynomial, the encrypted coefficients of the associative polynomial represent the associative polynomial in the ciphertext domain.

6. The method according to claim 5, wherein the fully homomorphically encrypted coefficients of the associative polynomial are part of the pre-calculated term.

7. The method according to any one of claims 1 to 6, wherein the pre-calculation term depends on the derivative of the nonlinear function evaluated at the center point.

8. The method according to any one of claims 1 to 6, wherein each of the aforementioned polynomials represents the respective Taylor polynomial for the aforementioned nonlinear function, and each of the aforementioned Taylor polynomials is centered on the respective central value associated with the respective polynomial.

9. The method according to any one of claims 1 to 6, wherein the step of calculating the output value of the function includes the step of evaluating a linear combination of the polynomials.

10. The method according to any one of claims 1 to 6, wherein the nonlinear function is a Gaussian error linear unit function.

11. The method according to any one of claims 1 to 6, wherein the nonlinear function is an exponential function of the softmax function.

12. The method according to any one of claims 1 to 6, wherein the transformer-based AI module is a transformer-based large-scale language model.

13. The method according to claim 12, wherein the function is a function of the attention module of the transformer-based large-scale language model.

14. The aforementioned method is: The first step is to receive the encrypted input; A step of determining fully homomorphically encrypted entries for a first key vector, a first value vector, and a first query vector of the attention module of the transformer-based AI module in the form of fully homomorphically encrypted coefficients of a first further polynomial that depends on the first encrypted input, using the transformer-based AI module; A step of storing a portion of the first key vector and the first value vector in the form of a first tensor, wherein the entries of the first tensor include the coefficients of a portion of the first further polynomial associated with the first key vector, the first value vector and the portion of the first query vector; A step of determining the input values ​​of the function depending on the entries in the first key vector, the first value vector, and the first query vector; A step of determining the first output of the transformer-based AI module depending on the output value of the function; The second step is to receive the encrypted input; A step of using the transformer-based AI module to determine a second key vector, a second value vector, and a second query vector of the attention module, depending on the second encrypted input; A step of determining a second input value for the function, depending on the stored first tensor and the second key vector, second value vector, and second query vector; A step of determining a second output value of the function depending on the second input value; and A step in which the second output of the transformer-based AI module is determined depending on the second output value of the function. The method according to any one of claims 1 to 6, further comprising:

15. The aforementioned method, A step of providing a cache device for storing the first tensor; The step of dividing the first tensor into a plurality of chunks of the first tensor; and Steps to store the plurality of chunks in the memory blocks of the cache device. The method according to claim 14, further comprising the memory block being independently accessible to read the plurality of chunks of the first tensor independently of each other.

16. The aforementioned method is: The step of allocating the memory blocks discontinuously in the cache device. The method according to claim 15, further comprising:

17. A method for enhancing a transformer-based AI module, wherein the transformer-based AI module includes a nonlinear function for calculating an output value of the nonlinear function depending on an input value, and the method is: Steps to replace the aforementioned nonlinear function with a function. A method comprising, wherein the function depends on a pre-calculated term independent of the input value and on a polynomial, each polynomial depending on the difference between the input value and the central value associated with each polynomial, the associated central value indicating the center point of each polynomial, the associated central value being different for each polynomial, and the function being designed to define an approximate output value of the nonlinear function in response to the input value being presented as an argument to the nonlinear function.

18. A computer system for calculating a fully homomorphically encrypted output value of a function, depending on a fully homomorphically encrypted input value of a function, in an artificial intelligence module (AI module), particularly a transformer-based AI module, wherein the function represents a nonlinear function to be approximated by the function, and the computer system: To provide a fully homomorphically encrypted pre-computation term independent of the aforementioned input value, Using the AI ​​module, the output value of the function is calculated depending on the pre-calculated term and the polynomial. A computer system configured to perform the following, wherein each of the polynomials depends on the difference between the input value and the central value associated with each of the polynomials, the associated central value indicates the center point of each of the polynomials, and the associated central value is different for each polynomial.

19. The computer system according to claim 18, wherein each of the aforementioned polynomials depends on a set of powers for each of the aforementioned polynomials, and each of the aforementioned powers of the set depends on the difference between the input value and the central value associated with each of the aforementioned polynomials as the base of each aforementioned power, and on a natural number as the exponent of each aforementioned power, the natural number increases with each power of the aforementioned polynomial.

20. Each of the aforementioned polynomials depends on a set of products of each of the aforementioned polynomials, and each of the aforementioned products of each set includes as a factor one of the aforementioned pre-calculated terms and one of the aforementioned powers of each of the aforementioned sets, the aforementioned pre-calculated term depends on the derivative of the nonlinear function evaluated at the center point; Each of the aforementioned polynomials represents the respective Taylor polynomial for the aforementioned nonlinear function, and each of the aforementioned Taylor polynomials is centered on the respective central value associated with the respective polynomial; The computer system according to claim 19, wherein the computer system is configured to calculate the output value of the function by evaluating a linear combination of the polynomials.

Images