Apparatus and method for enumerating lattice points

Abstract

A lattice prediction device (200) predicts the number of lattice points contained within a bounded region in a vector space. The bounded region is defined by a radius value, and the lattice points represent digital signals in a lattice constructed on the vector space. The lattice is defined by a lattice generator matrix including entries. The lattice prediction device (200) includes a calculation unit (201) configured to determine the predicted number of lattice points by applying a machine learning algorithm to input data derived from the radius value and the entries of the lattice generator matrix.

Description

[Technical field]

[0001] The present invention relates generally to computer science, and more particularly to a method and apparatus for solving the problem of enumeration of lattice points in a finite lattice. [Background technology]

[0002] Lattices are efficient tools that have many applications in several fields, such as computer science, coding theory, digital communication and storage, and cryptography.

[0003] In computer science, lattices are used, for example, to construct integer linear programming algorithms that are used to factor polynomials over the rational numbers and to solve systems of polynomial equations.

[0004] In coding theory, lattices are used, for example, to construct efficient error-correcting codes and efficient algebraic space-time codes for data transmission over noisy channels or data storage (e.g., in cloud computing systems). Signal constellations with lattice structures are used for signal transmission in both Gaussian and single-antenna Rayleigh fading channels.

[0005] In digital communications, trellises are used, for example, for the detection of coded or uncoded signals transmitted over wireless multiple-input multiple-output channels.

[0006] In cryptography, lattices are used, for example, for the construction of secure cryptographic primitives that are robust against attacks, especially in post-quantum cryptography, and for the security proofs of key cryptosystems. Exemplary lattice-based cryptosystems include encryption schemes (e.g., the GGH encryption scheme and the NTRUEEncrypt), signatures (e.g., the GGH signature scheme), and hash functions (e.g., SWIFFT and LASH for lattice-based hash functions).

[0007] Lattice problems are a class of optimization problems on lattices. They have been addressed for decades and include the shortest vector problem (SVP), the closest vector problem (CVP), and the lattice point enumeration problem. In practical applications, such lattice problems arise, for example, in data detection in wireless communication systems, in resolving integer ambiguities in carrier phase GNSS positioning systems, and for the construction or security proofs of cryptographic algorithms.

[0008] A lattice of dimension n ≥ 1 is a regular finite arrangement of points in an n-dimensional vector space V, given by a basis denoted B and a norm denoted N. In geometry and group theory,

number

number

number

number

[0009] Solving the shortest vector problem in an n-dimensional lattice L over a vector space V of basis B and norm N consists of finding the shortest non-zero vector in the lattice L, which is measured by norm N. Exemplary approaches to solving the shortest vector problem based on the Euclidean norm include: For example, the lattice enumeration disclosed in "R. Kannan, Improved Algorithms for Integer Programming and related Lattice Problems, In Proceedings of the Fifteenth Annual ACM Symposium on Theory of Computing, pages 193-206" For example, the random sampling reduction disclosed in “C.P. Schnorr, Lattice Reduction by Random Sampling and Birthday Methods, In Proceedings of Annual Symposium on Theoretical Aspects of Computer Science, Pages 145-156, Springer, 2003” For example, the lattice sieve disclosed in "M. Ajtai, R. Kumar, and D. Sivakumar, A Sieve Algorithm for the Shortest Lattice Vector Problem, In Proceedings of the Thirty-third Annual ACM Symposium on Theory of Computing, pages 601-610, 2001" - Calculation of Voronoi cells of a lattice, as disclosed, for example, in "D. Micciancio and P. Voulgaris, A deterministic Single Exponential Time Algorithm for Most Lattice Problems based on Voronoi Cell Computations, SIAM Journal on Computing, vol. 42, pages 1364-1391"; and -For example, the discrete Gaussian sampling method disclosed in "D. Aggrawal, D. Dadush, O. Regev, and N. Stephens-Davidwowitz, Solving the Shortest Vector Problem in 2n time Using Discrete Gaussian Sampling, In Proceedings of the Forty-seventh Annual ACM Symposium on Theory of Computing, pages 733-742, 2013" Includes:

[0010] Lattice enumeration and random sampling reduction require superexponential time and memory. Lattice sieves, computing the Voronoi cells of a lattice, and discrete Gaussian sampling require extremely complex computations that scale polynomially in the lattice dimension.

[0011] Solving the nearest vector problem in an n-dimensional lattice L on a vector space V with basis B and metric M consists of finding a vector in the lattice L that is closest to some vector v in the vector space V (not necessarily in the lattice L), as measured by the metric M. Exemplary techniques used to solve the nearest vector problem include the Fincke-Pohst variants disclosed in "U. Fincke and M. Pohst, Improved Methods for Calculating Vectors of Short Length in a Lattice, Including a Complexity Analysis".

[0012] Lattice point enumeration in an n-dimensional lattice L on a vector space V of basis B and metric M consists of counting (i.e., determining the number of) lattice points inside some n-dimensional bounded region (ball or sphere), denoted S, in the vector space V. The number of lattice points inside a sphere of dimension n is proportional to the heaping of the sphere.

[0013] Figure 1 shows a vector space

number

number

[0014] The lattice enumeration problem is closely related to the recent vector problem and the shortest vector problem, and is known to be NP-hard to solve exactly. Existing techniques require highly complex computations that grow with the dimension of the lattice, making its implementation in practical systems difficult. [Prior art documents] [Non-patent literature]

[0015] [Non-Patent Document 1] R. Kannan,Improved Algorithms for Integer Programming and related Lattice Problems,In Proceedings of the Fifteenth Annual ACM Symposium on Theory of Computing,pages 193-206 [Non-Patent Document 2] CPSchnorr,Lattice Reduction by Random Sampling and Birthday Methods,In Proceedings of Annual Symposium on Theoretical Aspects of Computer Science,Pages 145-156,Springer,2003 [Non-Patent Document 3] M. Ajtai, R. Kumar, and D. Sivakumar, A Sieve Algorithm for the Shortest Lattice Vector Problem, In Proceedings of the Thirty-third Annual ACM Symposium on Theory of Computing, pages 601-610, 2001 [Non-Patent Document 4] D. Micciancio and P. Voulgaris, A deterministic Single Exponential Time Algorithm for Most Lattice Problems based on Voronoi Cell Computations, SIAM Journal on Computing, vol. 42, pages 1364-1391 [Non-Patent Document 5] D. Aggrawal, D. Dadush, O. Regev, and N. Stephens-Davidwowitz, Solving the Shortest Vector Problem in 2n time Using Discrete Gaussian Sampling, In Proceedings of the Forty-seventh Annual ACM Symposium on Theory of Computing, pages 733-742, 2013 [Non-Patent Document 6] U.Fincke and M.Pohst,Improved Methods for Calculating Vectors of Short Length in a Lattice,Including a Complextiy Analysis Summary of the Invention [Problem to be solved by the invention]

[0016] Therefore, there is a need to devise computationally less intensive and more efficient techniques for solving lattice-related problems, including lattice point enumeration problems and nearest vector problems. [Means for solving the problem]

[0017] To address these and other problems, a lattice predictor is provided for predicting a number of lattice points inside a bounded region in a vector space, the bounded region being defined by a radius value, and the lattice points representing digital signals in a lattice constructed on the vector space, the lattice being defined by a lattice generator matrix including entries, the lattice predictor includes a computation unit configured to determine a predicted number of lattice points by applying a machine learning algorithm to input data derived from the radius value and the entries of the lattice generator matrix.

[0018] According to some embodiments, the computation unit may be configured to perform a QR decomposition on the lattice generator matrix, thereby providing an upper triangular matrix, and the computation unit is configured to determine the input data by performing a multiplication between each element of the upper triangular matrix and the inverse of the radius value.

[0019] According to some embodiments, the machine learning algorithm may be a supervised machine learning algorithm selected from the group including support vector machines, linear regression, logistic regression, naive Bayes, linear discriminant analysis, decision trees, k-nearest neighbor algorithms, neural networks, and similarity learning.

[0020] According to some embodiments, the supervised machine learning algorithm may be a multi-layer deep neural network, which includes an input layer, one or more hidden layers, and an output layer, each layer including a plurality of computational nodes, the multi-layer deep neural network being associated with model parameters and an activation function, the activation function being implemented on at least one computational node among the plurality of computational nodes of the one or more hidden layers.

[0021] According to some embodiments, the activation function may be selected from the group including a linear activation function, a sigmoid function, a ReLU function, a TanH function, a softmax function, and a CUBE function.

[0022] According to some embodiments, the computation unit may be configured to identify model parameters from training data received during a training phase, the computation unit configured to identify a plurality of training datasets from the training data and an expected lattice point count, each expected lattice point count being associated with a training dataset among the plurality of training datasets, the training phase comprising two or more processing iterations, and in each processing iteration, the computation unit: - processing a deep neural network using a training data set among a plurality of training data as an input to provide an intermediate lattice point number associated with the training data set; - determining a loss function from the expected lattice points and the number of intermediate lattice points associated with the training data set; -Identify updated model parameters by applying an optimization algorithm according to the minimization of a loss function It is configured as follows.

[0023] According to some embodiments, the optimization algorithm may be selected from the group including the AdaDelta optimization algorithm, the AdaGrad optimization algorithm, the Adaptive Moment Estimation algorithm, the Nesterov Accelerated Gradient Method algorithm, the Nesterov Accelerated Adaptive Moment Estimation algorithm, the RMSprop algorithm, the Stochastic Gradient Optimization algorithm, and the Adaptive Learning Rate optimization algorithm.

[0024] According to some embodiments, the loss function may be selected from a group including a mean squared error function and an exponential log-likelihood function.

[0025] According to some embodiments, the computation unit may be configured to identify the initial model parameters from a randomly generated set of values ​​for the first processing iteration.

[0026] According to some embodiments, the computation unit may be configured to pre-determine the expected number of lattice points from the radius value and the lattice generator matrix by applying a list sphere decoding algorithm or a list spherical-bound stack decoding algorithm.

[0027] A lattice prediction method is also provided for predicting a number of lattice points inside a bounded region in a vector space, the bounded region being defined by a radius value, and the lattice points representing digital signals in a lattice constructed on the vector space, the lattice being defined by a lattice generator matrix including entries, the lattice prediction method comprising applying a machine learning algorithm to input data derived from the radius value and the entries of the lattice generator matrix to determine a predicted number of lattice points.

[0028] A computer program product is also provided for predicting a number of lattice points inside a bounded region in a vector space, the bounded region being defined by a radius value, and the lattice points representing digital signals in a lattice constructed on the vector space. The lattice is defined by a lattice generating matrix including entries. The computer program product includes a non-transitory computer-readable storage medium and instructions stored on the non-transitory computer-readable storage medium that, when executed by a processor, cause the processor to apply a machine learning algorithm to input data that provides a predicted number of lattice points derived from the radius value and the entries of the lattice generating matrix.

[0029] Advantageously, embodiments of the present invention allow for a less computationally intensive solution to the lattice enumeration problem.

[0030] Advantageously, embodiments of the present invention provide a grid point enumeration technique that provides reliable results compared to existing coverage in the literature.

[0031] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various embodiments of the present invention. [Brief description of the drawings]

[0032] [Figure 1] Vector Space

number

[0033] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Embodiments of the present invention provide an apparatus, method, and computer program for predicting the number of lattice points inside a bounded region in a space vector with fewer computational steps using machine learning methods.

[0034] To facilitate understanding of the embodiments of the present invention, some definitions and notations that will be used hereinafter are set forth below.

[0035] K refers to the field, i.e., the algebraic structure over which addition, subtraction, multiplication, and division are defined.

[0036] Let V refer to an n-dimensional (finite dimensional) K-vector space over a field K.

[0037] B = {v1,...,v n} specifies the K basis for the vector space V.

[0038] N(.) specifies the norm of the vector space V.

[0039] m(.) specifies the metric of the vector space V.

[0040] An n-dimensional lattice K-lattice constructed on a vector space V is a non-unique lattice basis B={v1,...,v n}. A lattice Λ is a set of n linearly independent vectors v1,...,v p The scope is defined by

number

[0041] Vectors v1,...,v p denotes a non-unique lattice basis for the lattice Λ.

[0042] M∈V nxn A lattice generator matrix, denoted by , refers to a matrix whose column vectors represent a non-unique lattice basis for the lattice Λ.

[0043] The lattice point u belonging to the lattice Λ is u = Ms,s∈K (2) refers to an n-dimensional vector, u∈V, that can be written as a function of the lattice generator matrix M according to

[0044] u min The shortest vector, denoted by,λ,is the vector in the lattice Λ,measured by the norm N. min and has a minimum length denoted by λ min =min U∈Λ{0} N(u) (3) refers to a nonzero vector such that

[0045] The shortest vector problem is to find the shortest non-zero vector u in a vector space V that belongs to a lattice Λ and has a shortest length measured by a norm N. min The shortest vector problem is an optimization problem that aims to find u min =argmin u∈Λ{0} N(u) (4) We want to solve the optimization problem given by:

[0046] The nearest vector problem refers to an optimization problem that aims to find, for a vector v in a vector space V, a vector u in a lattice Λ that is closest to vector v, where the distance between vectors v and u is measured by a metric m. u cvp =argmin u∈Λ{0} m (v-u) (5) We want to solve the optimization problem given by:

[0047] A lattice enumeration problem refers to an optimization problem that aims to count (i.e., determine the number of) lattice points inside a bounded region in a vector space V. Since a lattice point corresponds to a vector u=Ms, solving a lattice enumeration problem in a bounded region in a vector space V defined by a radius value r and centered at the origin is to enumerate vectors u∈Λ that belong to the lattice Λ and have a metric m(u) less than or equal to the radius value r, where m(u)≦r.

[0048] The lattice enumeration problem is closely related to the shortest vector problem and the closest vector problem. For example, by definition of the corresponding optimization problem, solving the lattice enumeration problem where the radius value is equal to the shortest vector length may provide the number of lattice points with the shortest length. In addition, solving the lattice enumeration problem where the metric m(u) corresponds to the distance between a vector in the vector space and other vectors belonging to the lattice may provide the number of closest vectors inside a bounded region of a vector belonging to the vector space.

[0049] Vector space in Euclidean space

number

number

number

number

number

[0050] lattice generator matrix

number

number

number

number

[0051] Exemplary

number

number

[0052] Euclidean Vector Space

number

number

[0053] Constructed in Euclidean space

number

[0054] D(K,θ k=1,...K N refers to a multi-layer deep neural network that consists of an input layer, K≧2 layers including one or more hidden layers, and an output layer, and interconnected artificial neurons (hereinafter also called "nodes" or "computational nodes"). The number of layers K represents the depth of the deep neural network, and the number of nodes in each layer represents the width of the deep neural network. (k) specifies the width of the kth layer and corresponds to the number of computational nodes in the kth layer.

[0055] A multi-layer deep neural network is a k=1,...K The model parameters θ are associated with the model parameters denoted as θ and the activation function denoted as σ. The activation function σ refers to a computational nonlinear function that determines the output of a certain neuron in the hidden layer of a multi-layer deep neural network. k=1,...K is the parameter θ for k=1,...,K. k contains the set of

number

number

number

[0056] L is the loss function, which refers to a mathematical function used to estimate the loss (also called the "error" or "cost") between the estimate (also called the "mean") and the expectation during the training process of a deep neural network.

[0057] Optimizer (also called "optimization algorithm" or "gradient descent optimization algorithm") refers to the optimization algorithm used to update the parameters of a deep neural network during the training phase.

[0058] An epoch refers to the number of times a deep neural network learns training data during a training phase.

[0059] A mini-batch is a subset of the training data that is extracted from the training data and used in an iteration of the training phase. The mini-batch size refers to the number of training data samples in each mini-batch.

[0060] The learning rate (also called the "step size") in a gradient descent algorithm is a scalar value that is multiplied with the magnitude of the gradient.

[0061] Embodiments of the present invention provide apparatus, methods, and computer program products that enable the solving of lattice enumeration problems and that can be used in conjunction with solving the latest vector problem and the shortest vector problem. Such lattice problems arise in several fields and applications, including, but not limited to, computer science, coding, digital communication and storage, and cryptography. Thus, embodiments of the present invention may be implemented in a variety of digital systems designed to store, process, or communicate information in digital form. Exemplary applications include, but are not limited to: -Digital electronics, -Communications (e.g. digital data encoding and decoding using lattice signal constellations), - Data processing (e.g. in computing networks / systems, data centers), - data storage (e.g. cloud computing), - cryptography (e.g. for data protection and for controlling and authenticating access to data, devices and systems, e.g. in the automotive industry to ensure anti-theft, in mobile phones to control and authenticate access to batteries and accessories, in the banking industry to protect the security of bank accounts, financial transactions and data, in the pharmaceutical industry to protect the security of medical data and medical devices such as implantable medical devices, in FPGA confidentiality to protect hardware security for electronic components), -others Includes:

[0062] Exemplary digital systems include, but are not limited to: - communication systems (e.g. radio, wireless, single antenna communication systems, multiple antenna communication systems, fiber optic based communication systems); -communication equipment (e.g. transceivers of single or multi-antenna equipment, base stations, relay stations for coding and / or decoding of digital uncoded or coded signals represented by signal constellations, mobile phones, computers, laptops, tablets, drones, IoT devices); -storage systems and devices (e.g., cloud computing applications and cloud servers, mobile storage devices, etc.); - digital electronic devices such as RFID tags and electronic keys, smart cards, tokens used to store keys, smart card readers such as automated teller machines, and memory cards and / or hard disks with logon access monitored by cryptographic mechanisms for communication, data processing or storage (e.g., smart cards, smart card readers such as automated teller machines, and memory cards and / or hard disks with logon access monitored by cryptographic mechanisms), and cryptographic systems and equipment used to implement lattice cryptography (e.g., GGH cryptography and NTRUEEncrypt), lattice signatures (e.g., GGH signatures), and lattice hash functions (e.g., SWIFFT and LASH); -integer programming systems / devices (e.g. computers, quantum computers), - a positioning system (e.g. in GNSS for carrier phase GNSS integer ambiguity resolver), -others Includes:

[0063] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention provides an apparatus, method, and computer program product for solving the lattice enumeration problem by predicting the number of lattice points inside a bounded region in a vector space.

number

number

number

number

number

[0064] Referring to FIG. 2, the number N of lattice points u∈Λ in a finite dimensional lattice Λ inside a bounded domain, denoted by S, in some vector space V on which the lattice Λ is constructed pred A lattice predictor 200 is provided for predicting M. The bounded region is defined by a radius value denoted r. The lattice Λ is ij Lattice generator matrix containing the elements shown in

number

[0065] The lattice prediction apparatus 200 may be implemented in a digital data processing, communication, or storage device or system, including, but not limited to, those digital systems and applications mentioned above that apply to digital data transmission, processing, or storage.

[0066] Embodiments of the present invention rely on the use of artificial intelligence models and algorithms to solve the lattice enumeration problem. Thus, the lattice prediction device 200 receives a radius value r and a lattice generator matrix M, and processes a machine learning algorithm to predict the number of predicted lattice points N predThe lattice prediction device 200 may include a computation unit 201 configured to determine, where the machine learning algorithm operates using input data derived from the radius value r and the elements of the lattice generator matrix M. The lattice prediction device 200 may include a storage unit 203 configured to store the radius value r and the lattice generator matrix M and to load these values ​​into the computation unit 201.

[0067] According to some embodiments, the computation unit 201 may be configured to perform a QR decomposition M=QR on the lattice generator matrix, thereby obtaining an upper triangular matrix

number

number

number

[0068] A machine learning algorithm takes as input an input vector

number

[0069] According to some embodiments, the machine learning algorithm may be a supervised machine learning algorithm that maps input data to predicted data using a function identified based on labeled training data consisting of labeled input-output pairs. Exemplary supervised machine learning algorithms include, but are not limited to, support vector machines (SVMs), linear regression, logistic regression, naive Bayes, linear discriminant analysis, decision trees, k-nearest neighbor algorithms, neural networks, and similarity learning.

[0070] In a preferred embodiment, the supervised machine learning algorithm may be a multi-layer perceptron, which is a multi-layer feed-forward artificial neural network made up of at least three layers.

[0071] Referring to FIG. 3, a multi-layer deep neural network D(K,θ) is constructed with an input layer 301, at least two layers (K≧2) including one or more hidden layers 303 and an output layer 305. k=1,...,K , σ) 300 is shown. Each layer among the input layer 301, one or more hidden layers 303, and the output layer 305 includes a number of artificial neurons or computational nodes 3011.

[0072] The multi-layer deep neural network 300 is fully connected. Thus, each computational node of one layer is connected to each computational node of the next layer with a specific weighting, i.e., it combines the input from the connected node of the previous layer with a set of weights that amplify or attenuate the input value. The output of each layer is simultaneously the input of the next layer, starting with the input layer 301, which is configured to receive input data.

[0073] With the exception of the input computational nodes, i.e., the computational nodes 3011 in the input layer, each computational node 3011 in one or more hidden layers executes a nonlinear activation function σ that maps the computational node's weighted inputs to the computational node's outputs.

[0074] According to the multi-layer structure, deep neural networks map

number

number

number

number

number

number

number

[0075] The input-weight product performed at the kth layer computation node is the weight matrix W (k)and the input vector x that is processed as input by the kth layer. k-1 The product function W between (k) x k-1 These input-weight products are then summed and the sum is included in the activation function σ.

[0076] According to some embodiments, the activation function may be executed on at least one computational node 3011 of a plurality of computational nodes of one or more hidden layers 303.

[0077] According to some embodiments, an activation function may be performed at each node of the hidden layer.

[0078] According to some embodiments, the activation function may be selected from the group including a linear activation function, a sigmoid function, a TanH function, a softmax function, a rectified linear unit (ReLU) function, and a CUBE function.

[0079] A linear activation function is an identity function that does not change the signal.

[0080] The sigmoid function converts the nearly infinite range of an independent variable into a simple probability between 0 and 1. It is a non-linear function that takes some value as input and outputs another value between '0' and '1'.

[0081] The TanH function represents the relationship between hyperbolic sine and hyperbolic cosine: tanh(x)=sinh(x) / cosh(x).

[0082] Softmax activation generates a logistic regression and returns a probability distribution over mutually exclusive output classes. A softmax activation function may be implemented in the output layer of a deep neural network.

[0083] The ReLU activation function activates a neuron when its input is higher than a certain threshold. In particular, the certain threshold may be equal to zero ('0'), in which case the ReLU activation function outputs a value of zero if the input variable is negative, and outputs the input variable according to the identity function if the input variable is positive. Mathematically, the ReLU function may be expressed as σ(x)=max(0,x).

[0084] According to some embodiments, the computing device 201 may be configured to pre-identify and update model parameters of the multi-layer deep neural network from training data during a training phase. The training phase (also called the “learning phase”) includes the model parameters θ k=1,...,K is a global optimization problem performed to adjust the multi-layer deep neural network to minimize a prediction error that quantifies how close the network is to the ideal model parameters that provide the best prediction. The model parameters may initially be set to initial parameters, which may be generated, for example, randomly. The initial parameters are then updated during a training phase and adjusted to allow the neural network to converge to the best prediction.

[0085] According to some embodiments, a multi-layer deep neural network may be trained using a backpropagation supervised learning scheme, where training data is used to predict unknown data.

[0086] Backpropagation is an iterative process of forward and backward propagating information through different layers of a multi-layer deep neural network.

[0087] In the forward propagation phase, the neural network receives training data including training input values ​​and expected values ​​(also called "labels") associated with the training input values, the expected values ​​corresponding to the expected output of the neural network when the training input values ​​are used as input. The expected values ​​are known by the lattice predictor 200 in the application of supervised machine learning techniques. The neural network propagates the training data throughout the multi-layer neural network and identifies expected values ​​(also called "intermediate values") corresponding to the predictions obtained for the training input values. The training data is propagated in such a way that all of the computational nodes included in the different layers of the multi-layer deep neural network apply their respective transformations or calculations to the input values ​​received from the computational nodes of the previous layer and send their output values ​​to the computational nodes of the next layer. Once the data has been propagated to all layers and all the computational nodes have performed their respective calculations, the output layer generates estimates corresponding to the training data.

[0088] The final step of the forward propagation phase consists of comparing the expectation associated with the training data with the estimate obtained when the training data was passed as input through the neural network. This comparison allows to measure how good / bad the estimate is with respect to the expectation and to update the model parameters with the goal of bringing the estimate closer to the expectation so that the prediction error (also called "estimation error" or "cost") is close to zero. The prediction error may be estimated using a loss function based on a gradient scheme that updates the model parameters in the direction of the gradient of the objective function.

[0089] The forward propagation phase is followed by a backpropagation phase, in which the model parameters, e.g. the interconnection weights of the computational nodes 3011, are gradually adjusted in reverse order by application of an optimization algorithm until a good prediction is obtained and the loss function is minimized.

[0090] First, the calculated prediction error is propagated backwards, starting from the output layer, to all computational nodes 3011 of one or more hidden layers 303 that directly contribute to the calculation of the estimate. Each computational node receives a portion of the total prediction error based on its relative contribution to the output of the deep neural network. This process is repeated for each layer, until finally all computational nodes of the deep neural network receive a prediction error corresponding to their relative contribution to the total prediction error. Once the prediction error has been propagated backwards, the layer parameters, e.g. the first layer parameters (i.e. weights) and the second layer parameters (i.e. biases), may be updated by applying an optimization algorithm according to the minimization of a loss function.

[0091] According to some embodiments, during the training phase, the computation unit 201 may be configured to compute a loss function and update the model parameters according to a “batch gradient descent” method by updating the model parameters for the entire training data.

[0092] According to some embodiments, the computing unit 201 may be configured to update the model parameters during the training phase according to online learning by adjusting the model parameters for each sample of the training data. Using online learning, a loss function is evaluated for each sample of the training data. Online learning is also referred to as "online training" and "stochastic gradient descent."

[0093] According to some embodiments, during the training phase, the computation unit 201 may be configured to update the model parameters from the training data according to mini-batch learning (also called “mini-batch gradient descent”) using mini-batches of data of size s b A mini-batch of data is s b Therefore, the computation unit 201 divides the training data into a set of size s b s of the input data into two or more data batches, each batch being bThe input data is then fed to the network in batches. For each mini-batch of data fed through the neural network, the loss function is evaluated and the model parameters are updated for each mini-batch of data. The forward and backpropagation phases are thus performed for each mini-batch of data until the last batch.

[0094] According to some embodiments, the computing unit 201 may be configured to pass all training data through the deep neural network 300 multiple times (called epochs) during the training process. The number of epochs may be increased until an accuracy metric assessing the accuracy of the training data begins to decrease or continues to increase (e.g., if possible overfitting is detected).

[0095]

number

number

[0096] Based on supervised learning, the training samples may be labeled, i.e., associated with known expected output values ​​(also called "targets" or "labels"), which correspond to the output of the deep neural network when the training samples are used as inputs to the deep neural network. More specifically, m=1,...,Nb s For each sample x *,m is the expected number of lattice points inside a bounded region of radius r

number

[0097] According to some embodiments in which mini-batch learning is used, the computation unit 201 may be configured to identify (update or adjust) model parameters in mini-batches extracted from received training data during the training phase. In such an embodiment, the computation unit 201 may be configured to (*,1) ,x (*,2) ,...,x (*,NB) into a set of multiple NB of training data, denoted by b , and s b Each mini-batch x (*,1) s b Sample of x *,m m is 1 to Nb s Varies between mini-batch x (*,1) Also, S l The training samples are also specified by Nb s are drawn from the training sample of S l ⊂S.

[0098] For each mini-batch x for l=1,...,NB (*,1) is a mini-batch of data x (*,1) The expected number of lattice points that can be obtained by a deep neural network when is used as an input to the deep neural network.

number

number

[0099] On this training data and expected output values, the computational unit 201 may be configured to perform the forward and backpropagation phases of the training process.

[0100] Based on mini-batch training, the training phase may include two or more processing iterations. In each processing iteration, the computation unit 201 may be configured as follows: - mini-batch x of multiple training data (*,l) as input and process the deep neural network to generate a mini-batch x (*,l) Associated with

number

number

number

number

number

number

number

number

[0101] For the first processing iteration, the computation unit 201 may be configured to identify initial model parameters, which are used during the turn propagation phase of the first processing iteration of the training process. More specifically, the computation unit 201 may be configured to generate a multi-layer deep neural network D(K,θ k=1,...,k ,σ) the initial first layer parameters associated with each of the K layers

number

number

[0102] According to some embodiments, the computation unit 201 may be configured to identify initial first layer parameters and initial second layer parameters associated with different layers of the deep neural network randomly from a set of random values, e.g. according to a standard normal distribution.

[0103] According to some embodiments, the optimization algorithm used to adjust the model parameters and identify updated model parameters may be selected from the group including the AdaDelta optimization algorithm, the AdaGrad optimization algorithm, an Adaptive Moment Estimation algorithm (ADAM) that computes an adaptive learning rate for each model parameter, the Nesterov Accelerated Gradient Method (NAG) algorithm, the Nesterov Accelerated Adaptive Moment Estimation (Nadam) algorithm, the RMSprop algorithm, a stochastic gradient optimization algorithm, and an adaptive learning rate optimization algorithm.

[0104] According to some embodiments, the loss function considered to evaluate the prediction error or loss may be selected among a group including the Mean Squared Error function (MSE) used for linear regression and the Exponential Log-Likelihood (EXPLL) function used for Poisson regression.

[0105] According to some embodiments in which a mean squared error function is used, the loss function calculated for the lth mini-batch of data is

number

[0106] According to some embodiments, the computation unit 201 computes each mini-batch S for l=1,...,NB. l The expected number of lattices associated with

number

[0107] Referring to FIG. 4, in some vector space V on which the lattice Λ is constructed, the number N of lattice points u∈Λ in the finite dimensional lattice Λ inside the bounded domain denoted by S is pred A lattice prediction method is also provided for predicting . The bounded region is defined by a radius value r. Λ is a Euclidean space

number

number

number

[0108] In step 401, the lattice generator matrix

number

[0109] In step 403, a QR decomposition of the lattice generator matrix M=QR may be performed, whereby an upper triangular matrix

number

number

[0110] In step 405, from the received radius value r and the elements of the grid generating matrix M, N (0) =n 2 Input data vector containing true inputs

number

[0111] In step 407, the input vector

number

[0112] According to some embodiments, the machine learning algorithm may be a supervised machine learning algorithm selected from the group including support vector machines, linear regression, logistic regression, naive Bayes, linear discriminant analysis, decision trees, k-nearest neighbor algorithms, neural networks, and similarity learning.

[0113] According to a preferred embodiment, the supervised machine learning algorithm is a multi-layer feedforward artificial neural network D(K,θ k=1,...,K , σ), which is composed of an input layer and at least two layers (K≧2) including one or more hidden layers and an output layer, and has model parameters θ k=1,...,K and the activation function σ, and the model parameters θ k=1,...,K is multiple sets of layer parameters

number

[0114] According to some embodiments, the activation function may be selected from the group including a linear activation function, a sigmoid function, a TanH function, a softmax function, a rectified linear unit (ReLU) function, and a CUBE function.

[0115] According to some embodiments in which the machine learning algorithm is a multi-layer deep neural network, step 407 may include substeps performed to identify updated model parameters according to a backpropagation supervised training or learning process that uses the training data to train the multi-layer deep neural network.

[0116] According to some embodiments, the model parameters may be updated during the training process according to "batch gradient descent" by computing a loss function and updating the model parameters over the entire training data.

[0117] According to some embodiments, the model parameters may be updated during the training process according to online learning by adjusting the model parameters for each sample of the training data and computing the loss for each sample of the training data.

[0118] According to another embodiment, the model parameters may be updated from the training data during the training process according to mini-batch learning using mini-batches of data, each of which may be of size s b A mini-batch of data is s b Therefore, the training data is a subset of the training samples of size s b Each batch may be divided into two or more mini-batches of data, where each batch is s of input data. b The input data is then fed to the network in mini-batches. The loss function is evaluated for each mini-batch of data, and the model parameters are updated for each mini-batch of data.

[0119] FIG. 5 illustrates the model parameters θ that provide the best prediction in terms of minimizing the prediction error, according to some embodiments using mini-batch learning. k=1...K To identify k=1,...,K 1 is a flowchart showing a method for training σ.

[0120] In step 501, nb s Training sample of

number

number

number

number

[0121] In step 503, the training data is a training data set x (*,1) ,x (*,2) ,...,x (*.NB) The training data set may be divided into S b A set of training examples of size s b For each mini-batch x (*,1) is a mini-batch of data x (*.1) The expected number of lattice points that a deep neural network is expected to obtain when used as input to the network.

number

number

[0122] The training process may include two or more processing iterations, which are repeated until a stopping condition is reached, which may relate to the number of processed mini-batches of training data and / or to the suitability of the updated model parameters in terms of minimizing the prediction error obtained from the updated model parameters.

[0123] In step 505, a first processing iteration may be performed during which initial model parameters may be identified for use in processing a first mini-batch of data. More specifically, in step 505, a multi-layer deep neural network D(K,θ k=1 , ...K ,σ) the initial first layer parameters associated with each of the K layers

number

number

[0124] According to some embodiments, the initial first layer parameters and initial second layer parameters associated with different layers of the deep neural network may be identified randomly from a random set of values, for example according to a standard normal distribution.

[0125] Steps 507-513 may be repeated to process mini-batches of data until a stopping condition is reached. A processing iteration of the training process consists of steps 509-513, which process multiple training sets x, for l=1,...,NB. (*,1) Mini batch in x (*,1) This relates to the processing of

[0126] In step 509, the multi-layer deep neural network is configured to generate a mini-batch x (*,1) as input, which results in a mini-batch x (*,1) Associated with

number

number

[0127] In step 511, the processed mini-batch x (*,1) About minibatch x (*,1) The number of known expected lattice points associated with

number

number

number

[0128] In step 513, a loss function is calculated by applying an optimization algorithm.

number

number

number

[0129] According to some embodiments, the optimization algorithm may be selected from the group including the AdaDelta optimization algorithm, the AdaGrad optimization algorithm, the Adaptive Moment Estimation algorithm, the Nesterov Accelerated Gradient Method algorithm, the Nesterov Accelerated Adaptive Moment Estimation algorithm, the RMSprop algorithm, the Stochastic Gradient Optimization algorithm, and the Adaptive Learning Rate optimization algorithm.

[0130] According to some embodiments, the loss function may be selected from a group including a mean squared error function and an exponential log-likelihood function.

[0131] According to some embodiments, step 501 selects each mini-batch S for l=1,...,NB. l The expected number of lattice points associated with

number

[0132] The number N of lattice points u∈Λ in a finite dimensional lattice Λ inside a bounded domain S of some vector space V on which the lattice Λ is constructed pred A computer program product is also provided for predicting the bounded region, r, where Λ is a Euclidean space

number

number

number

[0133] The performance of the provided lattice prediction device and method has been evaluated through several simulation experiments. Figures 6 to 10 show the results obtained considering different lattice dimensions n from 2 to 10. ij are modeled as iid zero-mean Gaussian random variables with unit variance. The training data used for each lattice dimension contains 50000 training samples. Mini-batch learning is considered in these simulation experiments, for which the training samples are of size s b The neural network is divided into batches of NB=2500 with NB=20. The Adaptive Moment Estimation (Adam) optimization algorithm is used with an adaptive learning rate equal to 0.001. The multi-layer deep neural network takes a vector of dimension n 2 It consists of an input layer that takes a vector of , a hidden layer with up to 10, and an input layer that generates the predicted number of grid points inside a bounded region of a certain radius as a prediction. The number of computational nodes in the hidden layer depends on the grid dimension and is chosen to be greater than or equal to the number of input variables.

[0134] Figures 6 and 7 show error histograms for evaluating the prediction error during the training phase between the expected number of lattice points and the estimated value during the training phase for lattices of dimensions n = 5 and n = 10, respectively. The figures in Figures 6 and 7 show that the percentage of points for which the proposed prediction method provides accurate predictions is high.

[0135] Figure 8 illustrates the variation of normalized root mean square deviation (NRMSD) values ​​as a function of the number of hidden layers for two lattice dimensions n=4 and n=6. Normalized root mean square deviation evaluates the ratio between the mean square deviation (used as a metric to evaluate the prediction error) and the average value. Figure 8 shows that the NRMSD decreases with increasing number of hidden layers, and at the same time, a sufficient number of hidden layers equal to 3 is sufficient to achieve high prediction accuracy.

[0136] Figures 9 and 10 show the performance of a multi-layer deep neural network for a lattice dimension n=5, considering the training and test sets, respectively. The predicted output of the multi-layer deep neural network is plotted against the target output, i.e. the number of predicted lattice points is plotted against the expected number of lattice points. The figures in Figures 9 and 10 show that the number of predicted lattice points is concentrated around the y=x axis in a bounded region (sphere) for small radius values. This indicates that the predictive model according to an embodiment of the present invention fits the concentration of lattice points and provides accurate predictions. Some accuracy may also be obtained for higher radius values.

[0137] The apparatus, methods, and computer program products described herein may be implemented in various ways. For example, these techniques may be implemented in hardware, software, or a combination thereof. In the case of a hardware implementation, the processing elements of the lattice prediction device 200 may be implemented, for example, according to a hardware-only configuration (e.g., one or more FPGAs, ASICs, or VLSI integrated circuits with corresponding memory), or according to a configuration having both VLSI and digital signal processors (DSPs).

[0138] Additionally, the methods described herein may be implemented by computer program instructions provided to a processor of any type of computer to produce a machine with the processor executing the instructions to perform the functions / operations specified herein. These computer program instructions may also be stored on a computer readable medium capable of directing a computer to function in a particular manner. To that end, the computer program instructions may be loaded into a computer and cause the execution of a series of operational steps, thereby creating a computer implemented process such that the executed instructions provide a process for performing the functions specified herein.

Claims

1. A lattice prediction device (200) for predicting a number of lattice points inside a bounded region in a vector space, the bounded region being defined by a radius value, the lattice points representing digital signals in a lattice constructed on the vector space, the lattice being defined by a lattice generator matrix including entries, the lattice prediction device (200) comprising: a computation unit (201) configured to determine a predicted number of lattice points by applying a machine learning algorithm to input data derived from the radius value and the entries of the lattice generator matrix.

2. 2. The lattice prediction device of claim 1, wherein the computation unit is configured to perform a QR decomposition on the lattice generator matrix, thereby providing an upper triangular matrix, and the computation unit is configured to determine the input data by performing a multiplication between each element of the upper triangular matrix and the inverse of the radius value.

3. 2. The lattice prediction apparatus (200) of claim 1, wherein the machine learning algorithm is a supervised machine learning algorithm selected from the group including: support vector machine, linear regression, logistic regression, naive Bayes, linear discriminant analysis, decision tree, k-nearest neighbor algorithm, neural network, and similarity learning.

4. 4. The lattice prediction apparatus (200) of claim 3, wherein the supervised machine learning algorithm is a multi-layer deep neural network (300) including an input layer (301), one or more hidden layers (303), and an output layer (305), each layer including a plurality of computational nodes (3011), the multi-layer deep neural network (300) being associated with model parameters and an activation function, the activation function being implemented on at least one computational node (3011) among the plurality of computational nodes of the one or more hidden layers.

5. The lattice predictor (200) of claim 4, wherein the activation function is selected from the group comprising a linear activation function, a sigmoid function, a ReLU function, a TanH function, a softmax function, and a CUBE function.

6. The computation unit (201) is configured to identify the model parameters from training data received during a training phase, the computation unit (201) is configured to identify a plurality of training data sets from the training data and an expected lattice point count, each expected lattice point count being associated with a training data set among the plurality of training data sets, the training phase comprising two or more processing iterations, and in each processing iteration, the computation unit (201) performs: - processing the deep neural network (300) using a training data set from among the plurality of training data as input, thereby providing an intermediate lattice point number associated with the training data set; - determining a loss function from the expected number of lattice points and the intermediate number of lattice points associated with the training data set; - Identifying updated model parameters by applying an optimization algorithm according to the minimization of said loss function The lattice prediction device (200) of claim 4, configured to:

7. 7. The lattice predictor (200) of claim 6, wherein the optimization algorithm is selected from the group comprising: AdaDelta optimization algorithm, AdaGrad optimization algorithm, Adaptive Moment Estimation algorithm, Nesterov's Accelerated Gradient Method algorithm, Nesterov Accelerated Adaptive Moment Estimation algorithm, RMSprop algorithm, Stochastic Gradient Optimization algorithm, and Adaptive Learning Rate optimization algorithm.

8. The lattice predictor (200) of claim 6, wherein the loss function is selected from the group comprising a mean squared error function and an exponential log-likelihood function.

9. 7. The lattice prediction device (200) of claim 6, wherein the calculation unit (201) is configured to identify initial model parameters from a set of randomly generated values ​​for a first processing iteration.

10. 7. The lattice prediction device (200) of claim 6, wherein the calculation unit (201) is configured to pre-determine the expected number of lattice points from the radius value and a lattice generator matrix by applying a List-Sphere Decoding Algorithm or a List-Spherical-Bound Stack Decoding Algorithm.

11. A lattice prediction method executed by a computer for predicting a number of lattice points inside a bounded region in a vector space, the bounded region being defined by a radius value, lattice points representing digital signals in a lattice constructed on the vector space, the lattice being defined by a lattice generating matrix including components, the lattice prediction method comprising the step of determining a predicted number of lattice points by applying a machine learning algorithm to input data derived from the radius value and components of the lattice generating matrix.

12. 1. A computer program for predicting a number of lattice points inside a bounded region in a vector space, the bounded region being defined by a radius value, the lattice points representing digital signals in a lattice constructed on the vector space, the lattice being defined by a lattice generator matrix including entries, the computer program comprising instructions that, when executed by a processor, cause the processor to apply a machine learning algorithm to input data derived from the radius value and entries of the lattice generator matrix to provide a predicted number of lattice points.

Images