Method for optimizing a linear code

Abstract

The invention relates to a method for optimizing a function (F), the function (F) being a linear code with which a control matrix is associated, the function (F) comprising parameters that are the elements of the control matrix, the parameters taking their values from a set (E) of discrete values, the method using another function (L), referred to as a loss function, the method, which is implemented by a data processing device (100), comprising: executing a gradient descent step on the loss function (L) in order to obtain a parameter value (V1, V2, V3), the obtained values optimizing the function (F).

Description

Method for optimizing a function

[0001] The technical field is that of function optimization and in particular the search for efficient codes, codes used in telecommunications.

[0002] More specifically, the invention relates to a method for optimizing a function. The function in question is defined by parameters that take their values ​​from a set of discrete values. The optimization of the function is achieved by obtaining parameter values ​​that optimize the function.

[0003] In particular, the function in question can be a linear code. A linear code can be expressed as a matrix whose size depends on the coding scheme applied. Encoding a word of a given length is then achieved by performing a matrix multiplication of the word in question by the matrix corresponding to the code. When the word to be encoded is composed of bits, the matrix forming the code is also composed of bits, and the matrix multiplication corresponding to the encoding is performed according to the addition and multiplication operations of bits 0 and 1. These principles can be generalized to words to be encoded composed of discrete values ​​other than bits 0 and 1, and the matrix forming a code is then composed of these same discrete values. These discrete values ​​can be, for example, symbols corresponding to phase modulations used to represent signals in wireless communications.The set of values ​​in question can typically be formed of 4, 8 or 16 distinct symbols and a matrix forming a linear code to directly encode words formed of these symbols will also be formed of these 4, 8 or 16 distinct symbols.

[0004] The typical size of a matrix forming a linear code will be, for example, 16 or 24 by 32. The linear code implemented by this matrix is ​​therefore a function with 24 * 32 = 768 parameters. In other classic examples, the size of a matrix forming a linear code is 200 by 256, or 51,200 parameters. These parameters take binary values ​​(0 or 1) when the linear code encodes binary words, or values ​​from a set of 4, 8, or 16 distinct symbols when the linear code encodes words from these sets. In all cases, the elements of the matrix forming a linear code are the parameters of a function (the linear code in question), and these parameters take values ​​from a set of discrete values, which will most often be binary values ​​but may belong to a larger set of discrete values.

[0005] The matrix of a linear code is a rectangular matrix. This means that when a word is encoded by a matrix multiplication, the resulting word is longer than the original word. For example, when the linear code is a 200 by 256 matrix, it will encode 200-bit words, and the resulting word, obtained by the matrix multiplication, is a 256-bit word. This resulting word is often called the code. It's worth noting that, for a given linear code, in the sense of a function, only a subset of codes, in the sense of possible result words, can be obtained, and not all possible 256-bit words.

[0006] The extra bits in the resulting word contain redundant information that allows the original word to be recovered when it is transmitted and transmission errors alter the values ​​of elements within the transmitted word. This redundant information consists of relationships between different bits that enable error detection. For example, a very simple code is as follows: it involves adding a single bit to a binary word of length 2, which encodes the parity of the original word. The word 00 will then be encoded as 000; the word 11 will be encoded as 110 because 11 has the same parity as 00; and the words 01 and 10 will be encoded as 011 and 101, respectively. When a word is transmitted, for example 000, and there is a transmission error, resulting in, for example, 010 at the receiver, the error is detected immediately.By adding more redundant information, a correction more likely than others can be obtained, which this very simple code does not allow since, once an error has been detected, it is not possible to say which one is the most likely.

[0007] The relationships between redundant information can be defined in another matrix, called the parity check matrix. A linear code can be equivalently defined by the matrix that encodes a word by matrix multiplication or by the parity check matrix. When the linear code matrix is ​​of size K by N, where N is the length of the resulting word obtained after encoding, the parity check matrix is ​​of size N – K by N, that is, the size of a rectangular matrix describing the links between redundant information. In the case of 200 by 256, the parity check matrix therefore has a size of 56 * 256 = 14,336. Even though the parity check matrix is ​​smaller than the complete matrix defining the linear code, the order of magnitude of the number of parameters that must be fixed to define the code is the same, namely several thousand, or even tens of thousands.

[0008] These different concepts on linear codes are well known to those skilled in the art and will not be described further. State of the art

[0009] Such linear codes, which allow for error detection and correction, are widely used to encode and decode information transmitted over unreliable channels. This is known as channel coding. These codes are very frequently used in telecommunications. To define a linear code such as those used in wireless digital communication systems (4G or 5G, for example), it is therefore necessary to define a matrix that includes several thousand or even tens of thousands of parameters. These parameters are the different elements of the matrix and take their values ​​from a set of discrete values, generally bits 0 or 1.

[0010] Several traditional techniques exist for constructing linear codes. These often suffer from the drawback of focusing on codes as mathematical objects and neglecting important practical aspects for their use in telecommunications. One such important practical aspect is the simplicity of the decoder associated with the code. A practical decoder, for example, uses belief propagation. This is known as a belief propagation decoder. Another potentially important practical aspect is the code's composition: a code with more zeros is considered more advantageous than one with more ones.

[0011] Furthermore, we have seen that finding a new code involves finding several thousand, or even tens of thousands, of different parameters. Given the size of the problem to be solved, it is natural to seek to use machine learning techniques to search for new linear codes.

[0012] A well-known machine learning technique for defining functions with a very large number of parameters is gradient descent. This technique is particularly useful for learning the weights of an artificial neural network, which represent the function's parameters. In this scenario, a neural network implements a function (for example, a pattern recognition function), and this function includes a very large number of parameters (the weights of the thousands of neurons in the network). Gradient descent allows the values ​​of these parameters to be learned automatically. In other words, while gradient descent is initially used to find points that are local or global minima of a function, it can also be used to optimize function parameters, as we will see later.

[0013] The gradient descent technique works as follows. First, a function called the loss function is defined. This function must be minimized to optimize the function for which we are seeking parameter values. The learning objective is then to find parameter values ​​that minimize this loss function. The loss function is composed of a set of functions corresponding to the effect of each parameter on the overall loss function. To apply the gradient descent technique, one must be able to calculate partial derivatives for each of the functions corresponding to each parameter. Values ​​are given for all parameters, for example, random values, and the loss function is defined along with the set of partial derivatives for each parameter.

[0014] Defining the loss function allows for the integration of important criteria into the learning process. For example, when learning a linear code, the loss function can incorporate a preferred decoder, such as an information propagation decoder. In this way, the code is learned with a suitable target decoder in mind.

[0015] In a traditional approach, gradient descent seeks to find a point where the loss function is minimal. To do this, a random point is given, the loss function is calculated, and the next point is obtained by moving in the opposite direction of the function's derivative, with each step proportional to the magnitude of the derivative at that point. Visually, one descends a slope, moving faster as the slope increases, until reaching a minimum, which may be the global minimum of the function or, more often, a local minimum.

[0016] Here, the goal is to learn parameter values ​​to minimize a function as a whole, rather than finding a specific point where the function is minimal. To achieve this, the learning process will proceed in batches. A batch of possible points for the loss function is defined, and the values ​​and partial derivatives of the loss function at these points are obtained. The parameter values ​​are then modified in the inverse direction of the partial derivative for that parameter, by a quantity proportional to that partial derivative. In other words, the steeper the slope of the loss function for the partial derivatives of a given parameter at a batch of calculation points, the more that parameter will be modified, in the inverse direction of the slope.The evolution of the parameter value following the evaluation of a batch can, for example, be obtained by averaging the different partial derivatives for all points in the batch. The idea is to minimize the loss function over all the points where it is defined by adjusting the parameters. A stopping criterion is given, for example, when the evolution of the loss function falls below a given threshold or even when a certain number of batches have been provided for training.

[0017] This process is very computationally intensive. To be performed efficiently, the gradient descent algorithm runs on dedicated processors that were originally used in computers for graphics processing. These are GPUs (Graphics Processing Units) and not the CPUs (Central Processing Units) more commonly known in computing. Using GPUs to perform gradient descent calculations imposes an additional constraint. When using GPUs, the format of the calculations must be defined in advance so that the processing unit can implement optimized chains of a given size. In practical terms, this means that, when executing a gradient descent algorithm, the batch size must be fixed beforehand and cannot be changed during the algorithm's execution if a GPU is used instead of a CPU.Using a CPU instead of a GPU would provide more flexibility but at the cost of a factor of ten or even one hundred in terms of computing performance.

[0018] This gradient descent technique is well-established and widely used for training neural networks. It therefore allows for the automatic learning of values ​​for tens of thousands of parameters for a given function, in a reasonable computation time thanks to the use of GPUs.

[0019] However, gradient descent cannot be used to learn the parameter values ​​of a linear code. This is because this technique only applies to differentiable functions and to parameters that take real values. Linear code parameters, however, take binary or, more generally, discrete values. Therefore, this technique, originating from the mathematics of real functions, cannot be applied here, even though it can be used to learn the weights of a neural network, which are real parameters.

[0020] Furthermore, an additional difficulty arises if one seeks to learn the discrete parameters of a function that is a linear code.

[0021] Indeed, linear coding techniques are well-established, and currently available codes are very efficient. This means that learning linear codes will be applicable to datasets for which excellent results are already available. In other words, events where improvements are needed in the machine learning coding function occur very rarely. These events are cases where a word is encoded, transmitted with one or more errors, and the decoding process fails to recover the original word because the number of transmission errors has rendered the code ineffective. These are therefore uncorrected errors, which the code aims to minimize. The training datasets must therefore be very large to ensure that there are representative events of uncorrected errors that can guide the learning process.

[0022] However, the number of typical parameters sought by machine learning is in the thousands. Since these are binary parameters, the set of possible combinations is 2 to the power of the number of parameters. Therefore, by necessity, the learning process takes place in a much smaller space than that of all possible combinations. Furthermore, representative events that can guide learning (uncorrected errors) are rare and become increasingly rare as learning progresses and the learned code improves and becomes more efficient. When such representative events do occur, they will exert a potentially disproportionate influence on the actual learning results, since they are so infrequent. It is therefore difficult to achieve convergence in learning, especially with the parameters sought being discrete values.

[0023] The invention improves the situation.

[0024] According to a first functional aspect, the invention relates to a method for optimizing a function, called the function to be optimized, the function to be optimized being a linear code to which a control matrix is ​​associated, the function to be optimized comprising parameters which are the elements of said control matrix, said parameters taking their values ​​from a set of discrete values, the method using another function, called the loss function, the method, implemented by a data processing device, comprising: – executing a gradient descent step of the loss function allowing to obtain a parameter value, the values ​​obtained optimizing the function to be optimized.

[0025] Thanks to this invention, the gradient descent method, which has proven effective for learning large quantities of real-world function parameters, such as neural network weights, is modified to allow its use in contexts where the parameters to be learned take discrete values. To this end, a loss function, which is notably differentiable with respect to the parameters considered, is defined as in the classical gradient descent technique. This loss function is such that optimizing the function to be optimized is equivalent to minimizing the loss function. The various partial derivatives of the loss function with respect to the parameters of the function to be optimized can be obtained for points in the set for which the loss function is defined.All these elements correspond to a classical gradient descent technique, but the process according to the invention differs in that the evolutions of parameter values ​​obtained from the partial derivatives, as well as the actual application of these evolutions to obtain values, are constrained so that the parameter values ​​do not leave the set of discrete values ​​that they can take.

[0026] Indeed, in the invention, performing a gradient descent step of the loss function yields a parameter value, given that parameter values ​​belong to a set of discrete values. The parameter value provided by this gradient descent step of the loss function is therefore constrained, after the execution of a gradient descent step, to belong to the set of discrete values. Performing a gradient descent step corresponds, for example, to performing an elementary step of a gradient descent process. In particular embodiments, performing a gradient descent step corresponds to performing a single gradient descent step or a single step of a gradient descent process.In other particular embodiments, executing a gradient descent step is equivalent to executing a part of the steps of a gradient descent process, said part being distinct from the totality of the steps of the gradient descent process.

[0027] In a classical gradient descent technique, the value changes inversely to the slope of the parameter, and proportionally to the slope's magnitude. Such a change, proportional to a continuous quantity, would almost certainly result in a value outside the set of discrete values, when applied to a given parameter. The constraint introduced by the invention, however, ensures that the gradient descent technique, while effective for learning large quantities of parameter values, will not cause the values ​​to fall outside the set of discrete values ​​to which they are limited.Such a constraint of maintaining in the set of discrete values ​​is brought to one or more values ​​resulting from the execution of a gradient descent step and is applied after the execution of one step or part of the total number of steps of the gradient descent process and not after the execution of all the gradient descent steps.

[0028] In one embodiment, the method comprises: obtaining a parameter value from the execution of a gradient descent step of the loss function, the parameter value obtained belonging to the set of discrete values.

[0029] According to one embodiment, the gradient descent step includes obtaining an evolution of a value of at least one parameter as a function of a partial derivative at at least one point of the loss function.

[0030] Gradient descent involves obtaining the evolution of the values ​​of the parameters to be optimized according to the partial derivatives of the loss function. A conventional gradient descent would involve proportionally following the partial derivatives of the loss function. Such an evolution of parameter values ​​proportional to their partial derivatives would imply that the parameter would fall outside the set of discrete values. However, in the invention, the evolutions of the parameter values ​​in the method are obtained and then applied in such a way that the values ​​remain within the set of discrete values ​​after the obtained evolutions are applied.We will see later that, in some implementation examples, the changes obtained are such that the values ​​do not leave the set of discrete values ​​and, in other implementation examples, the changes in values ​​obtained do not guarantee that the parameters do not leave the set of discrete values, but these changes in values ​​are applied in a way that guarantees that the parameters do not leave the set of discrete values.

[0031] According to one embodiment, which can be carried out alternatively or cumulatively with the previous embodiment, an evolution obtained of a value of at least one parameter considered includes an increment by one or a decrement by one in inverse function of the sign of the partial derivative at at least one point of the loss function obtained for the parameter considered as well as a zero evolution of a value of a parameter considered if this parameter already has the value of the minimum or the maximum of the set of discrete values.

[0032] Thanks to this implementation, the parameter values ​​evolve within the predetermined set of discrete values. If the partial derivative of the loss function for a parameter is positive, the value of that parameter will decrease by one (and conversely, increase by one if the partial derivative is negative). However, if the minimum or maximum of possible values ​​is reached, there will be no further evolution beyond that point, even if the sign of the partial derivative dictates otherwise. Thus, the set of parameter values ​​is never exceeded. The values ​​evolve in the opposite direction to the signs of the partial derivatives because the optimization sought for the parameters of the function to be optimized corresponds to the minimization of the loss function.In contrast to the classical gradient descent technique, the evolutions of values ​​are not proportional to the amplitude of the partial derivatives but depend only on their signs and proceed by discrete jumps of one (or are zero) to ensure that the value of the parameter concerned remains among the set of discrete values.

[0033] According to one embodiment, which can be carried out alternatively or cumulatively with the preceding embodiments, an evolution obtained of a value of at least one parameter considered includes a variation by a relative integer, the sign of the variation being inversely related to the sign of the partial derivative at at least one point of the loss function obtained for the parameter considered, and the absolute value of the variation being a function of the magnitude of the partial derivative, said absolute value being reduced if necessary to constrain the parameter not to exceed the minimum or maximum of the set of discrete values.

[0034] Thanks to this embodiment, the method can be adapted to cases where the set of discrete values ​​that the parameters can take is larger than the binary set of 0 and 1. In this case, it can be advantageous to consider large partial derivatives, which is not possible when the changes occur in steps of only one. The sign of the partial derivative will determine the direction of the change, as before, but the step size can be larger than one when dealing with large partial derivatives. However, to constrain the resulting parameter values ​​to remain within the set of discrete values, the changes must be constrained. The changes are integers, and their absolute value is constrained so as not to exceed the minimum or maximum, if necessary.

[0035] According to one embodiment, which can be carried out alternatively or cumulatively with the preceding embodiments, the process is iterative and includes, for one iteration, the selection of a set of points from the domain of definition of the loss function, said set of points being called a batch, and the step of obtaining an evolution of the value of a parameter considered includes the calculation of an average of the loss function for the points of the batch and the evolution of the value of the parameter considered is an inverse function of the partial derivative of the average of the loss function for that parameter.

[0036] This implementation allows for the use of a large number of points in the loss function to train the parameters of the function to be optimized. To achieve this, sets of points are created, generally through random sampling. The evolution of a parameter value then becomes a function of the partial derivative of the loss function averaged over the points in the set. In this way, it is possible to consider a greater number of points during the training process, which aims to minimize the loss function, rather than working point by point across the domain.

[0037] In general, the calculations required for the process will be performed by a GPU unit rather than a CPU unit to achieve greater computational efficiency. Consequently, the point set size will remain the same across different iterations of the process to take advantage of GPU optimizations.

[0038] According to one embodiment, which can be carried out alternatively or cumulatively with the preceding embodiments, the process is iterative and includes, for one iteration, the selection of a set of points from the domain of definition of the loss function, said set of points being called a batch, and the step of obtaining an evolution of the value of a parameter considered includes the calculation of partial derivatives of the loss function for the parameter considered at the points of the batch, followed by the reduction of these partial derivatives to increments or decrements in inverse function of the sign of the partial derivatives, and the combination of these increments and decrements into a single increment or decrement, the evolution of the value being obtained by combining these increments and decrements.

[0039] This embodiment offers an alternative calculation method to avoid calculating the average loss function across all points in a batch. Rather than considering the partial derivatives of the loss function averaged across the batch points, the individual partial derivatives are obtained at each point in the batch and quantified as increments or decrements inversely proportional to their sign. These increments and decrements are then reduced to a single increment or decrement that can be applied to obtain the value of the parameter under consideration. In this embodiment, each point in the batch has the same potential influence on the possible evolution of a parameter's value, whereas in the previous embodiment, the evolution depends on an average of the partial derivatives, and points with higher partial derivatives therefore have a greater influence.This alternative embodiment may be more suitable for equalizing the influences brought about by the different points of the lot.

[0040] According to one embodiment, which can be carried out alternatively or cumulatively with the preceding embodiments, the partial derivative as a function of which a parameter value evolution is obtained is reduced to zero if its absolute value is less than a given threshold.

[0041] Thanks to this embodiment, learning is improved. Indeed, small values ​​of partial derivatives can have a disproportionate effect in the process according to the invention, since a very small partial derivative value could lead to a discrete, and therefore large, change compared to minimal continuous changes. It is therefore preferable, below a certain threshold, to set the partial derivatives to zero rather than risk them triggering discrete changes.

[0042] According to one embodiment, which can be carried out alternatively or cumulatively with the previous embodiments, the step of obtaining a value of a parameter includes a step of combining the changes obtained with the changes obtained during the previous steps and a step of updating the value of a parameter according to the combined changes.

[0043] Thanks to this implementation, the performance of the learning algorithm is improved. We have seen that one difficulty with learning algorithms is their ability to function in the presence of the curse of dimensionality, meaning that the elements of interest for learning are very rare among the set of possible points. For linear code learning, this can mean that the points corresponding to uncorrected errors, which are the ones of interest for learning better coding, will appear very rarely in a given set of points in the loss function used in an iteration of the process.To compensate for this scarcity, and given that the size of a point set is fixed once and for all due to the use of a single GPU for calculations, it is useful to combine and aggregate the results of partial derivatives obtained from multiple sets before updating the parameter value based on this combination. In this way, a larger number of points in the domain are considered before adjusting the parameter value for a given set size.

[0044] According to one embodiment, which can be carried out alternatively or cumulatively with the preceding embodiments, the step of obtaining a value of a parameter according to the evolution obtained includes updating a counter associated with the parameter as a function of the partial derivatives of the loss function with respect to the respective parameters and the value obtained only changes when the counter associated with the parameter has exceeded a given threshold.

[0045] According to an embodiment which can be carried out cumulatively with the previous mode, the counters associated with the parameters add together the inverse values ​​of the partial derivatives of the loss function with respect to the respective parameters obtained during the previous iterations.

[0046] According to an embodiment which can be carried out alternatively with the previous mode, the counters associated with the parameters add together the inverse signs of the partial derivatives of the loss function with respect to the respective parameters obtained during the previous iterations.

[0047] Thanks to these implementations, the performance of the learning algorithm is further improved. As seen previously, it is useful to combine the results from several batches before modifying a parameter value. Using a counter associated with the parameter allows this combination to be performed simply and efficiently. These counters must take partial derivatives into account in a manner inverse to the signs of the partial derivatives to properly perform gradient descent, which aims to minimize the loss function.

[0048] Two different modes of counter operation are possible according to the invention. In the first mode, the values ​​of the partial derivatives for a given parameter are added together and can be averaged to give a maximum value. When the counter exceeds a certain threshold, the value of the parameter will change while remaining constrained by the set of discrete values ​​that the parameter can take. In the second mode, only the signs of the partial derivatives are added in the counter. The counters can then increase or decrease by one depending on the sign of the partial derivative in question. Here again, when the counter exceeds a certain threshold, the value of the parameter will change while remaining constrained by the set of discrete values ​​that the parameter can take.The advantage of the counter that uses averaged partial derivatives is that it takes their magnitude into account, whereas the second method offers the benefit of simpler calculation. In both cases, the changes must be in the opposite direction to the sign of the partial derivatives.

[0049] In our experiments, the scarcity of points of interest for learning—a scarcity that increases as learning yields a better solution—means that their effect tends to be disproportionate in the absence of a compensatory mechanism achieved by combining results from multiple sets. As soon as a point provides useful learning information, it will be used, even if excessively. This can trigger oscillations in parameter values ​​around target values ​​as soon as relevant information becomes available for learning. This effect is amplified by the fact that the changes in values ​​are discrete. Indeed, in the traditional gradient descent technique, changes in values ​​are proportional to the magnitude of the partial derivatives.This allows us to approach the minimums sought by the method more or less quickly and to avoid exceeding them and oscillating around them as soon as informational points are used for learning.

[0050] To avoid this pitfall, the method according to the invention introduces sets of counters associated with the parameters. A parameter value will only change (in one direction or the other), for example, after its associated counter has recorded a number of changes to be made according to the signs of partial derivatives that exceed a given threshold, or after the sum of the partial derivatives has reached a certain threshold, depending on the embodiment. This evolution of the method according to the invention avoids the phenomenon observed in some versions of the method, namely oscillations in values ​​and an excessive impact on the learning of isolated points. Changes are only implemented if they apply to several batches, which will cause the associated counters to exceed their threshold.

[0051] Furthermore, combining the results from multiple batches before triggering a parameter value change allows for calculations with a large amount of training data while controlling computational complexity. For example, let's say the batch of calculation points is 32 points in size. This size is fixed, as we've seen, because the calculations are performed on GPU units. If we set the threshold for triggering a parameter value change at 20, this means that a parameter value change can only occur after considering 640 points (32 times 20), which corresponds to a very large batch, likely unmanageable in a reasonable timeframe.

[0052] According to one embodiment, which can be implemented alternatively or cumulatively with the preceding embodiments, when a value of a parameter changes after the counter has exceeded a given threshold, the counters associated with parameters that are below another threshold, less than or equal to the given threshold, are reset to zero.

[0053] According to an embodiment that can be implemented cumulatively with the previous embodiment, said other threshold is equal to the given threshold.

[0054] This embodiment refines the use of counters to guide parameter value changes. In our experiments, it is preferable, when at least one parameter value change is determined, to reset the counters associated with the parameters, since subsequent calculation points of the loss function will be obtained with changed parameter values. It can be decided to reset all counters below the given threshold, or only those below another, smaller threshold, thus retaining the counters located between the two thresholds, which have already accumulated a number of indicators regarding the direction in which they should change.

[0055] According to one embodiment, which can be realized alternatively or cumulatively with the preceding embodiments, when a value of a parameter changes after the counter has exceeded a given threshold, the values ​​of the parameters whose associated counters are close to the given threshold at a given distance are also changed.

[0056] This embodiment further refines the use of counters. We have seen that the value of a parameter is changed when its associated counter reaches a certain threshold, and the other counters are then reset to zero. This embodiment also allows for the adjustment of parameter values ​​whose associated counters have not yet reached the threshold but are close to it, for example, within a range of 1 to 5, so as not to reset information accumulated during training. All counters can then be reset after these parameter value changes have been applied.

[0057] According to one embodiment, which can be implemented alternatively or cumulatively with the preceding embodiments, the given threshold is set at a value between 10 and 30.

[0058] In our experiments, the threshold for a counter beyond which a parameter will change should not be too low. Furthermore, counters that are too high tend to delay certain changes in parameter values. The range of values ​​between 10 and 30 for this threshold yields good results, with a preference for a threshold set at 20.

[0059] According to one embodiment, which can be implemented alternatively or cumulatively with the previous embodiments, the given threshold increases as a direct function of the number of parameter values ​​to be learned.

[0060] In our experiments, the larger the code we're trying to learn, the more often it corrects rare events. This means that the events useful for learning—the uncorrected errors—are very infrequent. The learning process will therefore need high thresholds to judge the relevance of these rare events.

[0061] According to one embodiment, which can be carried out cumulatively or alternatively with the previous embodiments, the combination of the evolutions obtained includes the estimation of a confidence interval for the values ​​of the partial derivatives associated with the respective parameters using a binomial law, and the update of the parameter value is done when this estimation exceeds a certain threshold.

[0062] This embodiment offers another way to aggregate calculation results across multiple batches. Here, instead of using counters, the partial derivative values ​​taken at different points are used to estimate a confidence interval for the value of that partial derivative across all points in the domain. A method such as Agresti-Coull can be used for this. Once the confidence interval indicates that the partial derivative value for a given parameter has very likely exceeded a given threshold (positive or negative), the parameter in question can be increased or decreased depending on the sign of the threshold.

[0063] According to one embodiment, which can be carried out alternatively or cumulatively with the previous embodiments, the process includes a step of random initialization of the values ​​of the parameters of the function to be optimized.

[0064] This embodiment provides a starting point for the parameter values ​​of the function to be optimized. In our experiments, a random selection is effective and does not prevent the efficient implementation of the process according to the invention.

[0065] According to one embodiment, which can be carried out alternatively or cumulatively with the previous embodiments, the process includes a step of random initialization of the values ​​of the parameters of the function to be optimized according to a differentiated probability for the different values ​​of the set of discrete values ​​that the parameters can take.

[0066] In our experiments, it can be useful to differentiate the probabilities of obtaining a given discrete value for a parameter according to the values ​​belonging to the set of possible discrete values. For example, in a linear code for the binary case, randomly selecting the code parameters with a probability of 70% for the value 0 and 30% for the value 1 gives good results because it initiates the process with a parameter density of 1 close to that of a high-performing code.

[0067] According to one embodiment, which can be implemented alternatively or cumulatively with the preceding embodiments, the function to be optimized is a linear code and the parameters of the function to be optimized are the elements of the control matrix associated with said linear code.

[0068] Thanks to this embodiment, the invention is applied to the specific case of linear codes and allows for efficient learning of a linear code. The parameters whose values ​​are learned are those of the control matrix, thus slightly reducing the size of the learning problem compared to direct learning of the linear code matrix. The parameters of the function to be optimized are then the elements of the code's parity matrix and will generally be bits. The parameters therefore take their values ​​from a set of discrete values, namely, in this case, the set of values ​​0 and 1. In other examples, the linear code is applied to words formed from symbols that are not binary but belong to larger sets of values. These can be, for example, codes directly encoding phase modulations used in wireless communications.Depending on the wireless communication methods deployed, the phase modulations used can be, for example, 2, 4, 8 or 16. For example, in the cases 4, 8 or 16, the codes used will be matrices whose elements will belong to finite fields of size 4, 8 or 16 (respectively Z / 4Z, Z / 8Z and Z / 16Z) for which a minimum and a maximum are defined (respectively, 0 and 3, 0 and 7, 0 and 15) and the changes of parameter values ​​in these finite fields will be done by additions or subtractions of integers which will be limited where appropriate so as not to exceed the minimum or the maximum.

[0069] According to one embodiment, which can be realized alternatively or cumulatively with the preceding embodiments, the function to be optimized is a linear code and the loss function is measured according to the cross entropy between a result word of encoding by the function to be optimized and the decoding of this result word by a decoder using the control matrix associated with the linear code of the function to be optimized and a point of a batch is provided to the loss function by applying random errors to the word formed only of minimal symbols among the set of discrete values.

[0070] This embodiment allows us to define a loss function suitable for learning a linear code. The loss function is defined as the cross-entropy measured between a known word, the result of encoding, and its decoding after random errors have been applied to the known word. If the encoding is efficient, the decoding will recover the known word even in the presence of errors, and the cross-entropy will be zero. Optimizing the encoding therefore amounts to minimizing this loss function. The known word must be a possible encoding result; for all possible linear codes, the word formed solely of minimal symbols, for example, a word consisting only of 0s for a binary code, is such a word.

[0071] According to a first material aspect, the invention relates to a data processing device implementing a method for optimizing a function, called the function to be optimized, the function to be optimized being a linear code to which a control matrix is ​​associated, the function to be optimized comprising parameters which are the elements of said control matrix, said parameters taking their values ​​from a set of discrete values, the method using another function, called the loss function, said device comprising the following module: An execution module of a gradient descent step of the loss function allowing to obtain a parameter value, the values ​​obtained optimizing the function to be optimized.

[0072] In one embodiment of the data processing device, the execution module of a gradient descent step includes an element enabling the evolution of a value of at least one parameter as a function of a partial derivative at at least one point of the loss function.

[0073] In one embodiment, the data processing device further includes a module for obtaining a parameter value from the execution of a gradient descent step of the loss function, the parameter value obtained belonging to the set of discrete values.

[0074] According to another material aspect, the invention relates to a computer program capable of being implemented by a data processing device, the program comprising code instructions which, when executed by a processor, carries out the steps of the obtaining process defined above.

[0075] According to another material aspect, the invention relates to a data carrier on which is recorded a computer program comprising a sequence of instructions for the implementation of the obtaining process defined above.

[0076] According to another material aspect, the invention relates to a linear code obtained by a process according to the invention implemented by a data processing device according to the invention.

[0077] The data carrier can be any entity or device capable of storing programs. For example, the carrier can include a storage medium, such as a ROM (e.g., a CD-ROM or a microelectronic circuit ROM), or a magnetic recording medium such as a hard drive. Alternatively, the carrier can be a transmissible medium such as an electrical or optical signal, which can be transmitted via an electrical or optical cable, by radio, or by other means. The program according to the invention can, in particular, be uploaded to a network such as the Internet. Alternatively, the information carrier can be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the method in question.The program according to the invention can use any type of computer technology in terms of compiled programming languages, interpreted languages, or a combination of both, as well as in terms of operating systems. Brief description of the figures

[0078] The invention will be better understood upon reading the following description, given by way of example, and made with reference to the accompanying drawings in which:

[0079] represents a data processing device implementing a method for obtaining the parameter values ​​of a function from a set of discrete values.

[0080] illustrates an example of a sequence of steps implemented in a process according to the invention.

[0081] illustrates an example of implementing the process to obtain the parameter values ​​of a linear code.

[0082] illustrates the performance of a linear code whose parameter values ​​were obtained according to a process according to the invention. Detailed description

[0083] Lare represents an example of a data processing device 100 implementing a method for obtaining the values ​​V1, V2, V3 of parameters P1, P2, P3 of a function F from a set E of discrete values.

[0084] The process includes the prior selection of another function L, called the loss function.

[0085] In this example embodiment, the device 100 includes a module 101 for executing a gradient descent step of the loss function L. This module 101 includes, in this example embodiment, an element for obtaining, for the parameters P1, P2, P3, evolutions Δ1, Δ2, Δ3 of the values ​​V1, V2, V3 of the parameters P1, P2, P3 as a function of the partial derivatives D1, D2, D3 of the loss function L for the parameters P1, P2, P3 at a point of the domain of definition of the loss function L.

[0086] Furthermore, in this example of embodiment, the device 100 includes a module 102 for obtaining new values ​​V1, V2, V3 of the parameters P1, P2, P3 according to the evolutions obtained Δ1, Δ2, Δ3, the new values ​​obtained being constrained to belong to the set E of discrete values.

[0087] The data processing device 100 has the hardware architecture of a conventional computer. It includes at least one processor, RAM and read-only memory such as Flash or ROM (not shown in the figure), as well as input / output devices such as, in some cases, keyboards and / or screens (not shown in the figure), and network ports allowing communication with other entities and servers via a communication network such as the Internet (not shown in the figure).

[0088] The processing units present in the processing device 100 and used to carry out the process according to the invention will be, in some embodiments of the graphics processing units, referred to as GPUs (not shown in the figure). In other examples, the processing units are CPUs (not shown in the figure). The use of a GPU processing unit to carry out the process according to the invention provides greater performance to the calculations required to perform the process according to the invention, and therefore allows for more advanced learning within a given computation time, at the expense of the flexibility in defining the calculations that can be performed by a CPU. Several embodiments of the invention are specifically adapted to the use of GPU processing units.

[0089] Device 100 will, for example, be a computer server that performs the process of obtaining a set of values ​​V1, V2, V3 of parameters P1, P2, P3 of a function F, said values ​​V1, V2, V3 belonging to a set E of discrete values. The function F is, for example, a linear code, and the parameters P1, P2, P3 of the code are the elements of the code's check matrix, i.e., bits that take the values ​​0 or 1. In some cases, the code can encode several symbols before transmission, and the values ​​V1, V2, V3 of the parameters P1, P2, P3 are no longer bits but still belong to a set E of discrete values.

[0090] If device 100 is a computer server, it will perform a learning process that will then provide an optimized function F, for example, a new linear code. Device 100 can also belong to a cloud architecture, and the calculations required to perform the learning process are then carried out on computing, memory, and network resources provided on demand by the cloud system.

[0091] In other embodiments, the data processing device 100 belongs to equipment that implements the function F, whose values ​​V1, V2, V3 of parameters P1, P2, P3 are obtained by the process implemented by the device 100. For example, the device 100 can be included in telecommunications equipment, such as a router, a mobile network base station, or any other telecommunications equipment. The device 100 can then learn a linear code and distribute it directly to the equipment in which it is included. In some embodiments, the device 100 can use actual traffic data passing through the telecommunications equipment in which it is included to improve the implementation of the data acquisition process.

[0092] Module 101 of the data processing device 100 will allow us to obtain evolutions Δ1, Δ2, Δ3 of the values ​​V1, V2, V3 of the parameters P1, P2, P3 as a function of the partial derivatives D1, D2, D3 of the loss function L for the parameters P1, P2, P3 at a point of the domain of definition of the loss function L.

[0093] In an example implementation of the process, Module 101 will calculate all the partial derivatives of the loss function L for a set, called a batch, of points in the domain of definition of the loss function L with respect to the parameters of the function to be optimized F. From these partial derivatives, Module 101 can, for example, calculate their average values ​​over all the points in the batch to obtain a single value. The average values ​​thus obtained can be directly considered as the possible evolutions Δ1, Δ2, Δ3 of the values ​​V1, V2, V3 of the parameters P1, P2, P3. Such possible evolutions Δ1, Δ2, Δ3, obtained as averages over the different points in the batch of the partial derivatives D1, D2, D3, will obviously not, in general, be discrete evolutions.

[0094] Another example of implementing the process involves module 101 of device 100 simply obtaining the signs of the partial derivatives of the loss function L for a set, called a batch, of points in the domain of definition of the loss function L with respect to the parameters of the function to be optimized F. From these signs of partial derivatives, module 101 can, for example, obtain possible changes Δ1, Δ2, Δ3 in the values ​​V1, V2, V3 of the parameters P1, P2, P3 by adding the number of positive signs of partial derivatives for a given parameter and subtracting the number of negative signs. This example of implementation will yield changes in parameter values ​​that will be discrete but can be greater than +1 or -1.

[0095] In some implementation examples, partial derivatives D1, D2, and D3 whose absolute values ​​are below a given threshold are considered to be zero. This is to avoid including in the learning process values ​​that are too small, as they would be insignificant and are best ignored. This cancellation can occur before averaging the partial derivatives D1, D2, and D3 across all points in the dataset, or after averaging the partial derivatives across all points in the dataset.

[0096] Other examples of implementation are possible which allow module 101 to obtain possible evolutions Δ1, Δ2, Δ3 of the values ​​V1, V2, V3 of the parameters P1, P2, P3 of the function to be optimized F as a function of the partial derivatives D1, D2, D3 of the loss function L at one or more points of the domain of definition of the loss function L.

[0097] Module 102 of the data processing device 100, meanwhile, will allow us to obtain new values ​​V1, V2, V3 of the parameters P1, P2, P3 according to the changes obtained Δ1, Δ2, Δ3 by module 101, the new values ​​obtained being constrained to belong to the set E of discrete values.

[0098] We have seen that module 101 can obtain possible evolutions Δ1, Δ2, Δ3 of the values ​​V1, V2, V3 which are not discrete quantities or, even if they are discrete quantities in some implementation examples, which can be greater than +1 and -1. Such evolutions applied to discrete values, which for example in the case of learning binary parameters are the values ​​0 or 1, would have the consequence that the new values ​​V1, V2, V3 obtained after applying the evolutions Δ1, Δ2, Δ3 would no longer generally belong to the set E to which the values ​​V1, V2, V3 of the parameters P1, P2, P3 must belong.

[0099] Module 102 must therefore apply the obtained evolutions Δ1, Δ2, Δ3 while ensuring a constraint that the values ​​V1, V2, V3 remain within the set E of discrete values. Several implementations are possible to achieve this.

[0100] For example, in some embodiments, the changes Δ1, Δ2, Δ3 are real quantities obtained by averaging the partial derivatives D1, D2, D3 over the different points of a batch. In these examples, module 102 can use a threshold system to transform the real changes Δ1, Δ2, Δ3 obtained by module 101 into discrete changes that ensure that the new values ​​V1, V2, V3 will always belong to the set E of discrete values.

[0101] Furthermore, if a discrete evolution is applied to a value that is already at or near the minimum or maximum of the set E of discrete values, the 102 module will reduce the evolutions Δ1, Δ2, Δ3 to ensure that the minimum or maximum is not exceeded.

[0102] In examples where the changes Δ1, Δ2, Δ3 obtained by module 101 are already discrete quantities, module 102 can also use a threshold system. These thresholds can serve, on the one hand, to prevent excessively large changes that would push the values ​​V1, V2, V3 of the parameters P1, P2, P3 outside the set E of discrete values. On the other hand, these thresholds can ensure that a change only occurs when a large number of training points point towards a given change, thus avoiding numerous oscillations caused by a small number of insignificant data points. For example, in a set of 32 points, the evolutions Δ1, Δ2, Δ3 can be obtained by summing the signs of the partial derivatives D1, D2, D3 and limited to evolutions of +1 or -1 if the parameters P1, P2, P3 have values ​​V1, V2, V3 which are binary.Rather than implementing these changes of +1 or -1 as soon as the changes Δ1, Δ2, Δ3 obtained by module 101 are respectively positive or negative, it is possible to use a threshold, for example +30 or -30, to implement an actual change of +1 or -1 in the values ​​V1, V2, V3. In this example, a change will only be effective if at least 30 points in the set of 32 have a partial derivative D1, D2, D3 of the same sign, which indicates that almost all the points in the set give the same indication in their partial derivative.

[0103] In implementation examples, the process is iterative and module 102 performs the combination of the evolutions Δ1, Δ2, Δ3 obtained by module 101 during several successive steps before proceeding to a change of the values ​​V1, V2, V3 of the parameters P1, P2, P3.

[0104] This combination can include, for example, the use of counters. The changes Δ1, Δ2, Δ3 obtained by module 101 can, for example, be reduced to the sign of the partial derivatives D1, D2, D3 for the parameters P1, P2, P3, and therefore to increments or decrements of 1 in the opposite direction to the sign of the partial derivatives D1, D2, D3 averaged over all the points in the set. These changes Δ1, Δ2, Δ3 are then combined by module 102 into a counter associated with the parameters P1, P2, P3. The changes in the values ​​V1, V2, V3 will only be effective when one of the counters exceeds a given threshold, and will apply to the values ​​of the parameters for which the associated counter has reached the given threshold. The other counters will then be reset to zero in the implementation examples. It is also possible not to reset all the counters to zero, but to keep those that are close to the given threshold.It is also possible to choose to change the values ​​of the parameters associated with counters that have not reached the given threshold but are close to it, and then reset all the counters to zero.

[0105] In another example of a combination, the values ​​of the changes Δ1, Δ2, Δ3 obtained by module 101 are equal to the partial derivatives D1, D2, D3 of the loss function L, possibly averaged over all the points of a batch of calculations. These different changes Δ1, Δ2, Δ3 are then summed in counters associated with the parameters, and, as in the previous example, module 102 performs an actual change in the values ​​V1, V2, V3 of the parameters P1, P2, P3 only when these counters exceed a given threshold. Unlike the previous example, this combination takes into account the magnitude of the partial derivatives D1, D2, D3 and not just their sign.As in the previous mode, the effective evolutions of the values ​​V1, V2, V3 must be in the opposite direction of the signs of the partial derivatives D1, D2, D3 since the optimization of the function F that we seek to achieve by modifying the values ​​V1, V2, V3 of the parameters P1, P2, P3 corresponds to a minimization of the loss function L.

[0106] In another example of a combination, the process does not use counters but rather an estimation of the confidence interval for the values ​​of the partial derivatives D1, D2, and D3. As before, the changes Δ1, Δ2, and Δ3 obtained are directly related to the partial derivatives D1, D2, and D3. Module 102 will then, as it receives the changes Δ1, Δ2, and Δ3, estimate the confidence interval within which the partial derivatives for the parameters P1, P2, and P3 lie for the entire domain. The Agresti-Coull method can be used to calculate this confidence interval. When sufficient certainty is reached, Module 102 can perform an actual change in the values ​​V1, V2, and V3 of the parameters P1, P2, and P3 for which sufficient confidence is present, and reset all estimates for subsequent measurements.

[0107] Figure 1, for its part, presents an example of a sequence of steps implemented in a process according to the invention.

[0108] In this example implementation, the process is iterative. An initialization step for the values ​​V1, V2, V3 of the parameters P1, P2, P3 has taken place, but is not shown in the figure. This initialization can be random or expert-led. When it is random, different probabilities can be assigned to the various values ​​present in the set E of discrete values ​​to initialize each parameter, in order to provide a relevant starting point for the process.

[0109] Step S1 corresponds to the execution of a gradient descent step of the loss function L, which allows obtaining the possible evolutions Δ1, Δ2, Δ3 of the values ​​V1, V2, V3 by the modulus 101 as a function of the partial derivatives D1, D2, D3 of the loss function L with respect to the parameters P1, P2, P3. These possible evolutions can be, for example, increments or decrements of "one" inversely proportional to the sign of the partial derivatives D1, D2, D3 averaged over all the points of the batch.

[0110] Step S2 corresponds to a combination, over successive iterations, of the changes Δ1, Δ2, Δ3. In the example presented here, this combination is performed using counters. The changes of +1 and -1 are added at each iteration to counters C1, C2, C3 associated with the parameters P1, P2, P3.

[0111] When the counters C1, C2, C3 have been updated with the changes Δ1, Δ2, Δ3 obtained in step S1, it is tested whether one or more counters Ci among the counters C1, C2, C3 has reached a given threshold S. This is represented by a test in the.

[0112] If this is not the case (branch N of the test), the iterative process resumes at step S1 with the obtaining of new evolutions Δ1, Δ2, Δ3 for a new set of points belonging to the domain of definition of the loss function L.

[0113] If a counter Ci reaches the given threshold S (branch Y of the test), the corresponding change Δi is applied to the value Vi. This is of course the case for all counters that reach the threshold and, in some embodiments, it can also be the case for counters close to the threshold.

[0114] Next, in this example, the set of counters C1, C2, C3 are reset to zero, which is indicated by the mention RAZ in the, and the process continues at step S1 with a new set of points belonging to the definition set of the loss function L.

[0115] Several criteria can be chosen to stop the iterative process. The process can stop when there have been no changes for a certain number of iterations, or after a fixed number of iterations, for example. The process can also stop after evaluating the function F to be optimized using the values ​​V1, V2, V3 obtained for the parameters P1, P2, P3 at a given time, but such an evaluation can take a significant amount of time.

[0116] This one, meanwhile, illustrates an example of implementing the process to obtain the parameter values ​​of a linear code.

[0117] In this example, the function to be optimized, F, is a linear encoding, and the parameters P1, P2, and P3 form, for example, a parity-checking matrix that defines the encoding. A decoding module, DEC, is provided, which allows the recovery of an encoded and then transmitted word, even in the presence of a number of transmission errors. The DEC decoding module can be, for example, an information-propagating decoder, so that the function F is optimized taking into account this very common type of decoding. The DEC decoding module uses a current state of the parity-checking matrix, represented by the COD element, to perform decoding. The current state of the parity matrix is ​​given by the values ​​V1, V2, and V3 of the parameters P1, P2, and P3.

[0118] To perform the iterative process, a set of measurement points must be defined. For this purpose, the word X, consisting only of zeros, is given. Such a word is a possible encoding result for any linear code, and, in a linear code, a possible encoding result represents all possibilities up to known transformations. The word X, consisting only of zeros, is therefore representative. Error noise (ERR) is added to this word, which is indicated by a circled + operator.

[0119] The DEC decoder will then attempt to correct the ERR error. The result of the DEC decoding for a point is noted If the DEC decoder manages to correct the introduced ERR error, the result noted will also be a word composed entirely of zeros. The encoding function F is optimized when the decoder DEC manages to correct the maximum number of errors. The loss function L to be minimized is defined as the binary cross entropy (BCE) between the original word X, composed entirely of zeros, and the resulting word denoted obtained after DEC decoding, which attempted to correct the introduced ERR error. The loss function L is indeed minimal when DEC manages to correct a maximum number of introduced ERR errors.

[0120] The processing device 100 therefore takes as input the partial derivatives D1, D2, D3 of the loss function L as seen previously. These partial derivatives D1, D2, D3 of the loss function L are obtained for a set of points of a given size, consisting only of zeros and to which errors ERR have been added. The two modules 101 and 102 then proceed respectively to obtain the evolutions Δ1, Δ2, Δ3 and then, if necessary, apply the evolutions Δ1, Δ2, Δ3 to obtain new values ​​V1, V2, V3 of the parameters P1, P2, P3. In some embodiments, the results of several sets are combined, for example, using a counter mechanism as seen previously.

[0121] If new values ​​V1, V2, V3 are obtained, they are integrated into the COD parity matrix. The function F to be optimized is thus potentially improved before a new iteration of the process on a new set of points which will be decoded DEC with the new values ​​V1, V2, V3 of the parameters P1, P2, P3 of the function F to be optimized.

[0122] Figure 1, meanwhile, illustrates the performance of a linear code whose parameter values ​​were obtained following a process according to the invention.

[0123] In this figure, the performance of three linear codes is represented by three broken lines formed by measurement points. These three codes are (128, 64) codes, meaning they encode 64-bit words by doubling their size to 128 bits. The matrix implementing these codes therefore has a size of 128 * 64 = 8192 bits.

[0124] These three linear codes are presented here at a size of 128 by 64 bits and are named ZC, CCSDS, and AR3A. The performance of the ZC code (acronym for Zero Coder) is represented by black triangles: this is the code obtained by implementing the method of the invention. The other two codes are named CCSDS and AR3A (acronyms for Consultative Committee for Space Data Systems and Accumulate Repeat-3 and Accumulate, respectively) and are represented by black and white circles, respectively. These two codes are constructed using traditional theoretical and heuristic methods and are known for their very good performance at a size of 128 by 64 bits. The performance of the CCSDS and AR3A codes as presented here is taken from the following article: Code Design for Short Blocks: A Survey, by Liva, Gaudio, Ninacs and Jerkovits, in Proceedings of EuCNC 2016, available at https: / arxiv.org / abs / 1610.00873.The CCSDS and AR3A codes have the best performance among the 14 codes listed in this article.

[0125] The performance of the three codes is measured as follows. The horizontal axis Eb / N0 is a measure related to noise in a communication channel. The higher this value, the less the communication is disrupted by noise in the channel. More precisely, Eb / N0 is a measure of the amount of energy expended per information symbol (Eb for energy per bit) normalized by the amount of noise affecting the signal carrying the information. This quantity is very commonly used in the field.

[0126] The vertical axis, meanwhile, measures the errors that remain uncorrected after decoding. The acronym BLER stands for Block Error Rate, which is the percentage of blocks (words) with errors after decoding.

[0127] The graphs are interpreted as follows: the lower the Eb / N0 ratio, the more noise disrupts communication and the higher the BLER uncorrected error rate will be. For an Eb / N0 ratio of 1, the uncorrected error rate is typically 0.5, or one uncorrected error out of two, whereas for an Eb / N0 of 4, the uncorrected error rate drops to one in a thousand or even one in ten thousand. The points on the graph are measured for a given Eb / N0 ratio by submitting a large number of words, encoding them according to the evaluated code, transmitting them over a channel with the given Eb / N0 ratio, decoding them with the same knowledge propagation decoder, and with the same number of iterations (200). The number of uncorrected errors for a given code then corresponds to a point on the curve representing that code.

[0128] To better assess the performance of the three codes, two theoretical curves are added to the graph. The Sphere Packing curve indicates the threshold below which no code can achieve its performance. The Random Coding Union curve indicates a zone below which at least one code is guaranteed to perform.

[0129] The ZC code was obtained by implementing the method of the invention. To obtain it, 8192 binary parameters had to be learned. The method was initialized with a random matrix where the density of 1s at initialization was 30%. The embodiment in this example consisted of measuring gradients on batches of size 8, i.e., 8 points on which the partial derivatives with respect to the parameters are measured. The partial derivatives are averaged over the eight points of the batch, and then only the signs are retained. These signs are used to increment or decrement a counter. The update threshold for a parameter was set at 20.

[0130] Thanks to this implementation of the process, several codes can be obtained and will be evaluated until the best one is selected: the ZC code, whose performance is shown in the graph. We can see that its performance is close to theoretically superior performance and equivalent to the best state-of-the-art codes.

[0131] Finally, it should be noted that, in this text, the term "module" can refer to a software component, a hardware component, or a set of hardware and software components. A software component itself corresponds to one or more computer programs or subprograms, or more generally, to any element of a program capable of implementing a function or set of functions as described for the modules in question. Similarly, a hardware component corresponds to any element of a hardware assembly capable of implementing a function or set of functions for the module in question (integrated circuit, smart card, memory card, etc.).

Claims

Optimization method for a function (F), the function (F) being a linear code to which a control matrix is ​​associated, the function to be optimized comprising parameters (P1, P2, P3) which are the elements of said control matrix, said parameters (P1, P2, P3) taking their values ​​(V1, V2, V3) from a set (E) of discrete values, the method using another function (L), called loss function, the method, implemented by a data processing device (100), comprising: – executing a gradient descent step of the loss function (L) allowing to obtain a value (V1, V2, V3) of parameter (P1, P2, P3), the values ​​(V1, V2, V3) obtained optimizing the function (F). Optimization method of a function (F) according to claim 1 characterized in that the gradient descent step comprises: – obtaining an evolution (Δ1, Δ2, Δ3) of a value (V1, V2, V3) of at least one parameter (P1, P2, P3) as a function of a partial derivative (D1, D2, D3) at at least one point of the loss function (L). A method for optimizing a function (F) according to claim 2 characterized in that an evolution (Δ1, Δ2, Δ3) obtained from a value (V1, V2, V3) of at least one parameter (P1, P2, P3) considered includes an increment by one or a decrement by one as an inverse function of the sign of the partial derivative (D1, D2, D3) at at least one point of the loss function (L) for the parameter (P1, P2, P3) considered as well as a zero evolution of a value of a parameter considered if this parameter already has the value of the minimum or the maximum of the set (E) of discrete values. A method for optimizing a function (F) according to one of claims 2 or 3 characterized in that the method is iterative and comprises, for one iteration, the selection of a set of points from the domain of definition of the loss function (L), said set of points being called the batch, and the step of obtaining an evolution (Δ1, Δ2, Δ3) of the value (V1, V2, V3) of a parameter (P1, P2, P3) considered comprises the calculation of an average of the loss function (L) for the points of the batch and the evolution (Δ1, Δ2, Δ3) of the value of the parameter considered is an inverse function of the partial derivative of the average of the loss function (L) for this parameter. A method for optimizing a function (F) according to one of claims 2 or 3 characterized in that the method is iterative and comprises, for one iteration, the selection of a set of points from the domain of definition of the loss function (L), said set of points being called the batch, and the step of obtaining an evolution (Δ1, Δ2, Δ3) of the value (V1, V2, V3) of a parameter (P1, P2, P3) considered comprises the calculation of partial derivatives (D1, D2, D3) of the loss function (L) for the parameter considered (P1, P2, P3) at the points of the batch, followed by the reduction of these partial derivatives to increments or decrements in inverse function of the sign of the partial derivatives, the evolution (Δ1, Δ2, Δ3) being obtained by combining these increments and decrements. A method for optimizing a function (F) according to any one of claims 2 to 5 characterized in that the partial derivative (D1, D2, D3) as a function of which an evolution (Δ1, Δ2, Δ3) of value (V1, V2, V3) of parameter (P1, P2, P3) is obtained is reduced to zero if its absolute value is less than a given threshold. A method for optimizing a function (F) according to any one of claims 2 to 6 characterized in that the step of obtaining a value of a parameter includes a step of combining the evolutions (Δ1, Δ2, Δ3) obtained with the evolutions (Δ1, Δ2, Δ3) obtained during the previous steps and a step of updating the value (V1, V2, V3) of a parameter (P1, P2, P3) as a function of the combined evolutions (Δ1, Δ2, Δ3). A method for optimizing a function (F) according to any one of claims 2 to 7 characterized in that the step of obtaining a value (V1, V2, V3) of a parameter (P1, P2, P3) according to the evolution (Δ1, Δ2, Δ3) obtained includes updating a counter (C1, C2, C3) associated with the parameter (P1, P2, P3) as a function of the partial derivatives of the loss function (L) with respect to the respective parameters (P1, P2, P3) and characterized in that the value (V1, V2, V3) obtained only changes when the counter (C1, C2, C3) associated with the parameter (P1, P2, P3) has exceeded a given threshold (S). A method for optimizing a function (F) according to claim 8 characterized in that, when a value (V1, V2, V3) of a parameter (P1, P2, P3) changes after the counter (C1, C2, C3) has exceeded the given threshold (S), all counters (C1, C2, C3) associated with parameters (P1, P2, P3) that are below another threshold, less than or equal to the given threshold (S), are reset to zero. A method for optimizing a function (F) according to one of claims 8 or 9 characterized in that the given threshold (S) is fixed at a value between 10 and 30. A method for optimizing a function (F) according to any one of claims 1 to 10 characterized in that the method includes a step of random initialization of the values ​​(V1, V2, V3) of the parameters (P1, P2, P3) of the function (F) according to a differentiated probability for the different values ​​of the set (E) of discrete values ​​that the parameters (P1, P2, P3) can take. A method for optimizing a function (F) according to any one of claims 4 or 5 characterized in that the loss function (L) is measured according to the cross entropy between a word (X) resulting from encoding by the function (F) and the decoding of this result word (X) by a decoder using the control matrix associated with the linear code of the function (F) and characterized in that a point from a batch is provided to the loss function (L) by applying random errors to the word formed only from the minimal symbols among the set (E) of discrete values. Data processing device (100) implementing a method for optimizing a function (F), the function (F) being a linear code to which a control matrix is ​​associated, the function (F) comprising parameters (P1, P2, P3) which are the elements of said control matrix, said parameters (P1, P2, P3) taking their values ​​(V1, V2, V3) from a set (E) of discrete values, the method using another function (L), called the loss function, said device (100) comprising the following module: A module (101) for executing a gradient descent step of the loss function (L) allowing to obtain a value (V1, V2, V3) of parameter (P1, P2, P3), the values ​​(V1, V2, V3) obtained optimizing the function (F). Computer program capable of being implemented by a data processing device (100) according to claim 13, the program comprising code instructions which, when executed by a processor, performs the steps of the optimization process according to any one of claims 1 to 12. Data carrier on which is recorded a computer program according to claim 14 comprising a sequence of instructions for implementing the optimization process according to any one of claims 1 to 12. linear code obtained by an optimization process according to one of claims 1 to 12 implemented by a data processing device (100) according to claim 13.

Images