Method for obtaining parameter values ​​of a function from a set of discrete values

Abstract

Method for Obtaining Parameter Values ​​of a Function from a Set of Discrete Values. The invention relates to a method for obtaining parameter values ​​of a function (F) implemented by a data processing device (100), said parameters taking their values ​​from a set (E) of discrete values. The method implements another function (L), called the loss function, such that optimizing the function (F) corresponds to minimizing the loss function (L). Said loss function (L) is differentiable with respect to the parameters (P1, P2, P3). The method comprises a step of obtaining an evolution (Δ1, Δ2, Δ3) of the value (V1, V2, V3) of a parameter (P1, P2, P3) as a function of a partial derivative (D1, D2, D3) of the loss function (L) obtained for the parameter in question, followed by a step of obtaining a value (V1, V2, V3) of the parameter. according to the evolution (Δ1, Δ2, Δ3) obtained,the value obtained being constrained to belong to the set (E) of discrete values. Figure 1,

Description

Title of the invention: Method for obtaining parameter values ​​of a function from a set of discrete values technical field

[0001] The technical field is that of the optimization of functions and in particular the search for efficient codes, codes used in telecommunications.

[0002] More specifically, the invention relates to a method for obtaining parameter values ​​of a function from a set of discrete values. The method in question is adapted to obtaining the parameter values ​​of a function that uses numerous parameters, all of which take their values ​​from a single set of discrete values.

[0003] In particular, the function in question can be a linear code. A linear code can be expressed as a matrix whose size depends on the coding scheme applied. The encoding of a word of a given length is then performed by taking a matrix product of the word in question and the matrix corresponding to the code. When the word to be encoded is composed of bits, the matrix forming the code is also composed of bits, and the matrix product corresponding to the encoding is performed according to the operations of adding and multiplying bits 0 and 1. These principles can be generalized to words to be encoded composed of discrete values ​​other than bits 0 and 1, and the matrix forming a code is then composed of these same discrete values. These discrete values ​​can be, for example, symbols corresponding to phase modulations used to represent signals in wireless communications.The set of values ​​in question can typically be formed from 4, 8 or 16 distinct symbols, and a matrix forming a linear code to directly encode words formed from these symbols will also be formed from these 4, 8 or 16 distinct symbols.

[0004] The typical size of a matrix forming a linear code will be, for example, 16 or 24 by 32. The linear code implemented by this matrix is ​​therefore a function having 24 * 32 = 768 parameters. In other classic examples, the size of a matrix forming a linear code is 200 by 256, or a quantity of 51,200 parameters. These parameters take binary values ​​(0 or 1) when the linear code encodes binary words, or take values ​​from a set of 4, 8, or 16 distinct symbols when the linear code encodes words from these sets. In all cases, the elements of the matrix forming a linear code are the parameters of a function (the linear code in question), and these parameters take values ​​from among a set of discrete values, which will most often be binary values ​​but may belong to a larger set of discrete values.

[0005] The matrix of a linear code is a rectangular matrix. In this way, when a word is encoded by a matrix multiplication, the resulting word is longer than the original word. For example, when the linear code is a 200 by 256 matrix, it will encode 200-bit words, and the resulting word, obtained by the matrix multiplication, is a 256-bit word. This resulting word is often called the code. It should be noted that, for a given linear code, in the sense of a function, only a subset of codes, in the sense of possible resulting words, can be obtained, and not all possible 256-bit words.

[0006] The additional bits in the result word contain redundant information that allows the original word to be recovered when it is transmitted and transmission errors alter the values ​​of elements in the transmitted word. This redundant information consists of relationships between different bits that enable error detection. For example, a very simple code is as follows: it consists of adding a single bit to a binary word of length 2, which encodes the parity of the original word. The word 00 will then be encoded as 000; the word 11 will be encoded as 110 because 11 has the same parity as 00; and the words 01 and 10 will be encoded as 011 and 101, respectively. When a word is transmitted, for example 000, and there is a transmission error, resulting in, for example, 010 at the receiver, the detection of a transmission error is immediate.By adding more redundant information, a correction more likely than others can be obtained, which this very simple code does not allow since, once an error has been detected, it is not possible to say which one is the most likely.

[0007] The relationships between redundant information can be defined in another matrix, called the parity check matrix. A linear code can be equivalently defined by the matrix that encodes a word by matrix multiplication or by the parity check matrix. When the linear code matrix is ​​of size K by N, where N is the length of the resulting word obtained after encoding, the parity check matrix is ​​of size N - K by N, that is, the size of a rectangular matrix describing the links between redundant information. In the case 200 by 256, the parity check matrix therefore has a size of 56 * 256 = 14,336. Even though the parity check matrix is ​​smaller than the complete matrix defining the linear code, the order of magnitude of the number of parameters that must be fixed to define the code is the same, namely several thousand, or even tens of thousands.

[0008] These different concepts on linear codes are well known to those skilled in the art and will not be described further. State of the art

[0009] Such linear codes, which allow for error detection and correction, are widely used to encode and decode information transmitted over unreliable channels. This is known as channel coding. These codes are very frequently used in telecommunications. To define a linear code such as those used in wireless digital communication systems (4G or 5G, for example), it is therefore necessary to define a matrix comprising several thousand or even tens of thousands of parameters. These parameters are the different elements of the matrix and take their values ​​from a set of discrete values, generally bits 0 or 1.

[0010] Several traditional techniques exist for constructing linear codes. These often have the drawback of focusing on codes as mathematical objects and neglecting practical aspects important for their use in telecommunications. One important practical aspect is the simplicity of the decoder associated with the code in question. A practical decoder, for example, uses belief propagation. This is then referred to as a belief propagation decoder. Another potentially important practical aspect is the code's composition: a code with more zeros is considered more advantageous than one with more ones.

[0011] Furthermore, we have seen that finding a new code involves finding several thousand, or even tens of thousands, of different parameters. Given the size of the problem to be solved, it is natural to seek to use machine learning techniques to search for new linear codes.

[0012] A well-known technique in machine learning for defining functions with a very large number of parameters is gradient descent. This technique is used in particular to learn the weights of an artificial neural network, representing the parameters of the function. In this situation, a neural network implements a function (for example, a pattern recognition function), and this function includes a very large number of parameters (the weights of the several thousand neurons present in the neural network). Gradient descent allows the values ​​of these parameters to be learned automatically.In other words, while the gradient descent technique is initially used to find points that are local or global minima of a function, it can also be used to optimize the parameters of a function, as we will see later.

[0013] The gradient descent technique works as follows. First, a function, called the loss function, is defined. This function must be minimized to optimize the function for which we are seeking parameter values. The learning objective is then to find parameter values ​​that minimize this loss function. The loss function is composed of a set of functions corresponding to the effect of each parameter on the overall loss function. To apply the gradient descent technique, one must be able to calculate partial derivatives for each of the functions corresponding to each parameter. Values ​​are given for all parameters, for example, random values, and the loss function is defined along with the set of partial derivatives for each parameter.

[0014] The definition of the loss function allows for the integration of important criteria into the learning process. For example, when learning a linear code, the loss function can incorporate a preferred decoder, such as an information propagation decoder. In this way, the code is learned by taking into account a suitable target decoder.

[0015] In a conventional approach, gradient descent seeks to find a point for which the loss function is minimal. To do this, a random point is given, the loss function is calculated, and the next point is obtained by moving in the opposite direction of the function's derivative, making a jump proportional to the magnitude of the derivative at that point. Visually, one descends a slope, moving faster as the slope increases, until a minimum is reached, which may be the global minimum of the function or, more often, a local minimum.

[0016] Here, we want to learn parameter values ​​to minimize a function as a whole, rather than searching for a specific point where the function is minimal. To achieve this, the learning process will proceed in batches. A batch of possible points for the loss function is defined, and the values ​​and partial derivatives of the loss function at these points are obtained. The parameter values ​​are then modified in the inverse direction of the partial derivative for that parameter, by a quantity proportional to that partial derivative. In other words, the steeper the slope of the loss function for the partial derivatives of a given parameter for a batch of calculation points, the more that parameter will be modified, in the inverse direction of the slope.The evolution of the parameter value following the evaluation of a batch can, for example, be obtained by averaging the different partial derivatives for all points in the batch. The idea is to minimize the loss function over all the points where it is defined by acting on the parameters. A stopping criterion is given, for example, when the evolution of the loss function falls below a given threshold or even when a certain number of batches have been provided for training.

[0017] This process is very computationally intensive. To be performed efficiently, the gradient descent algorithm will run on dedicated processors that were originally used in computers for graphics processing. These are GPUs (Graphics Processing Units) and not the CPUs (Central Processing Units) more commonly known in computing. Using GPUs to perform gradient descent calculations imposes an additional constraint. Indeed, when using GPUs, the format of the calculations must be defined in advance so that the processing unit can implement optimized chains of a given size. In practical terms, this means that, in the execution of a gradient descent algorithm, the batch size must be fixed in advance and must not be changed during the execution of the algorithm if a GPU is to be used instead of a CPU.Using a CPU instead of a GPU would provide more flexibility but at the cost of a tenfold or even a hundredfold decrease in computing performance.

[0018] This gradient descent technique is well established and widely used for training neural networks. It therefore allows for the automatic learning of values ​​for tens of thousands of parameters for a given function, in a reasonable computation time thanks to the use of GPU units.

[0019] However, the gradient descent technique cannot be applied to learn the parameter values ​​of a linear code. This is because this technique only applies to a differentiable function and to parameters that take real values. But the parameters of a linear code take binary or, more generally, discrete values. Therefore, this technique, originating from the mathematics of real functions, cannot be applied here, whereas it can indeed be applied to learn the weights of a neural network, which are real parameters.

[0020] Moreover, an additional difficulty arises if one seeks to learn the discrete parameters of a function which is a linear code.

[0021] Indeed, linear coding techniques are well established, and currently known codes are very efficient. This means that learning linear codes will be applicable to datasets for which very good results are already available. In other words, the events for which improvements are needed in the machine learning coding function occur very rarely. These events are cases in which a word is encoded, transmitted with one or more errors, and the decoding fails to recover the original word because the number of transmission errors has rendered the code inoperative. These are therefore uncorrected errors, which the code seeks to minimize. The potential training datasets must therefore be very large to be successful. to have representative events of uncorrected errors that can guide learning.

[0022] However, the number of typical parameters sought by learning is in the thousands. Since these are binary parameters, the set of possible combinations is 2 to the power of the number of parameters. Learning, by necessity, therefore takes place in a space much smaller than that of all possible combinations. Moreover, representative events that can guide learning (uncorrected errors) are rare and become increasingly rare as learning progresses and the learned code improves and becomes more efficient. When such representative events do occur, they will exert a potentially disproportionate influence on the actual learning results, since they are very rare. It is therefore difficult to achieve convergence in learning, especially with the parameters sought being discrete values.

[0023] The invention improves the situation. Description of the invention

[0024] According to a first functional aspect, the invention relates to a method implemented by a data processing device for obtaining parameter values ​​of a function, called the function to be optimized, said parameters taking their values ​​from a set of discrete values, the method implementing another function, called the loss function, such that the optimization of the function to be optimized corresponds to minimizing the loss function, said loss function being differentiable with respect to the parameters, the method comprising, for at least one parameter, a step of obtaining an evolution of the value of a parameter considered as a function of a partial derivative at at least one point of the differentiable loss function for the parameter considered; followed by a step of obtaining a parameter value according to the evolution obtained, the value obtained being constrained to belong to the set of discrete values.

[0025] Thanks to the invention, the gradient descent method, which has proven effective for learning large quantities of real-world function parameters, for example, for learning neural network weights, is modified so that it can be used in a context where the parameters to be learned take discrete values. For this purpose, a loss function, which is differentiable with respect to the parameters considered, is defined as in the classical gradient descent technique. The loss function is such that optimizing the function to be optimized is equivalent to minimizing the loss function. The various partial derivatives of the loss function with respect to the function parameters The values ​​to be optimized can be obtained for points in the set for which the loss function is defined. All these elements correspond to a classical gradient descent technique, but the method according to the invention differs in that the evolutions of parameter values ​​obtained from the partial derivatives, as well as the actual application of these evolutions to obtain values, are constrained so that the parameter values ​​do not leave the set of discrete values ​​they can take.

[0026] In a conventional gradient descent technique, the value evolves in the inverse direction of the slope for that parameter, and proportionally to the size of the slope. Such an evolution, proportional to a continuous quantity, would almost certainly result, when applied to a given parameter value, in a value that falls outside the set of discrete values. The constraint introduced by the invention, however, will guarantee that the use of the gradient descent technique, which is effective for learning a large number of parameter values, will not cause the values ​​to fall outside the set of discrete values ​​to which they are limited.

[0027] According to one embodiment, which can be carried out alternatively or cumulatively with the previous embodiment, an evolution obtained of a value of at least one parameter considered includes an increment by one or a decrement by one in inverse function of the sign of the partial derivative at at least one point of the loss function obtained for the parameter considered as well as a zero evolution of a value of a parameter considered if this parameter already has the value of the minimum or the maximum of the set of discrete values.

[0028] Thanks to this embodiment, the parameter values ​​evolve within the predetermined set of discrete values. If the partial derivative of the loss function for a parameter is positive, the value of that parameter will be decreased by one (and conversely increased by one if the partial derivative is negative), but if the minimum or maximum of possible values ​​is reached, there will be no further evolution beyond that point, even if the sign of the partial derivative dictates otherwise. Thus, the set of parameter values ​​is not exceeded. The values ​​evolve in the opposite direction to the signs of the partial derivatives because the optimization sought for the parameters of the function to be optimized corresponds to the minimization of the loss function.In contrast to the classical gradient descent technique, the changes in values ​​are not proportional to the amplitude of the partial derivatives but depend only on their signs and proceed by discrete jumps of one (or are zero) to ensure that the value of the parameter in question remains within the set of discrete values.

[0029] According to one embodiment, which can be implemented alternatively or cumulatively with the preceding embodiments, an evolution obtained from a value of at less a parameter considered includes a variation by a relative integer, the sign of the variation being inversely proportional to the sign of the partial derivative at at least one point of the loss function obtained for the parameter considered, and the absolute value of the variation being a function of the magnitude of the partial derivative, said absolute value being reduced if necessary to constrain the parameter not to exceed the minimum or maximum of the set of discrete values.

[0030] Thanks to this embodiment, the method can be adapted to the case where the set of discrete values ​​that the parameters can take is larger than the binary set of 0 and 1. In this case, it may be advantageous to consider large magnitudes of partial derivatives, which is not possible when the evolutions occur in steps of only one. The sign of the partial derivative will determine the direction of the evolution, as before, but the step size can be larger than one in the presence of large partial derivatives. However, to constrain the parameter values ​​obtained to remain within the set of discrete values, the evolutions must be constrained. The evolutions are integers, and their absolute value is constrained so as not to exceed the minimum or maximum, if necessary.

[0031] According to one embodiment, which can be carried out alternatively or cumulatively with the preceding embodiments, the process is iterative and includes, for one iteration, the selection of a set of points from the domain of definition of the loss function, said set of points being called lot, and the step of obtaining an evolution of the value of a parameter considered includes the calculation of an average of the loss function for the points of the lot and the evolution of the value of the parameter considered is an inverse function of the partial derivative of the average of the loss function for this parameter.

[0032] Thanks to this embodiment, it is possible to consider a large number of points of the loss function to train the parameters of the function to be optimized. To do this, sets of points are created, generally by random sampling. The evolution of a parameter value will then be a function of a partial derivative of the loss function averaged over the points in the set. In this way, it is possible to consider a larger number of points during the training, which aims to minimize the loss function, rather than working point by point of the domain.

[0033] In general, the calculations required for the process will be performed by a GPU unit and not by a CPU unit to achieve greater computational efficiency. Consequently, the point set size will be the same between the different iterations of the process in order to benefit from GPU optimizations.

[0034] According to one embodiment, which can be carried out alternatively or cumulatively with the preceding embodiments, the process is iterative and includes, for one iteration, the selection of a set of points from the domain of definition of the loss function, said set of points being called a batch, and the step of obtaining an evolution of the value of a parameter considered includes the calculation of partial derivatives of the loss function for the parameter considered at the points of the batch, followed by the reduction of these partial derivatives to increments or decrements in inverse function of the sign of the partial derivatives, and the combination of these increments and decrements into a single increment or decrement, the evolution of the value being obtained by combining these increments and decrements.

[0035] This embodiment provides an alternative calculation method to avoid calculating the average of the loss function over all points in a batch. Rather than considering the partial derivatives of the loss function averaged over the points in the batch, the various partial derivatives are obtained at each point in the batch and quantified as increments or decrements inversely proportional to their sign. These increments and decrements are then reduced to a single increment or decrement that can be applied to obtain the value of the parameter under consideration. In this embodiment, each point in the batch has the same potential influence on the possible evolution of the parameter value, whereas in the previous embodiment, the evolution depends on an average of the partial derivatives, and the points for which the partial derivative is higher will therefore have a greater influence.This alternative embodiment may be more suitable for equalizing the influences exerted by the different points in the batch.

[0036] According to one embodiment, which can be carried out alternatively or cumulatively with the preceding embodiments, the partial derivative as a function of which an evolution of parameter value is obtained is reduced to zero if its absolute value is less than a given threshold.

[0037] Thanks to this embodiment, learning is improved. Indeed, low values ​​of partial derivatives can have a disproportionate effect in the process according to the invention, since a very low value of partial derivative could lead to a discrete, and therefore large, change compared to minimal continuous changes. It is therefore preferable, below a certain threshold, to set the partial derivatives to zero rather than risk them triggering discrete changes.

[0038] According to one embodiment, which can be carried out alternatively or cumulatively with the preceding embodiments, the step of obtaining a value of a parameter includes a step of combining the changes obtained with the changes obtained during the previous steps and a step of updating the value of a parameter based on the combined changes.

[0039] Thanks to this embodiment, the performance of the learning algorithm is improved. We have seen that one difficulty with learning algorithms is that they operate in the presence of the curse of dimensionality, meaning that the elements of interest for learning are very rare among the set of possible points. For linear code learning, this can mean that the points corresponding to uncorrected errors, which are the ones of interest for learning better coding, will appear very rarely in a given set of points in the loss function used in an iteration of the process.To compensate for this scarcity, and given that the size of a point set is fixed once and for all due to the use of a GPU unit to perform the calculations, it is useful to combine and aggregate the results of the partial derivatives obtained over several sets before updating the parameter value based on the combination of these results. In this way, a larger number of points in the definition space are taken into account before performing an adjustment of the parameter value for a given set size.

[0040] According to one embodiment, which can be carried out alternatively or cumulatively with the preceding embodiments, the step of obtaining a value of a parameter according to the evolution obtained includes updating a counter associated with the parameter as a function of the partial derivatives of the loss function with respect to the respective parameters and the value obtained only changes when the counter associated with the parameter has exceeded a given threshold.

[0041] According to an embodiment which can be carried out cumulatively with the previous embodiment, the counters associated with the parameters add together the inverse values ​​of the partial derivatives of the loss function with respect to the respective parameters obtained during the previous iterations.

[0042] According to an embodiment which can be carried out alternatively with the previous mode, the counters associated with the parameters add together the inverse signs of the partial derivatives of the loss function with respect to the respective parameters obtained during the previous iterations.

[0043] Thanks to these embodiments, the performance of the learning algorithm is further improved. As seen previously, it is useful to combine the results from several batches before changing a parameter value. Using a counter associated with the parameter allows this combination to be performed simply and efficiently. These counters must take into account the partial derivatives in a manner inverse to the signs of the partial derivatives in order to properly perform gradient descent, which aims to minimize the loss function.

[0044] Two different modes of counter operation are possible according to the invention. In the first mode, the values ​​of the partial derivatives for a given parameter are added together and can be averaged to give them a maximum value. When the counter exceeds a certain threshold, the value of the parameter will change while remaining constrained by the set of discrete values ​​that the parameter can take. In the second mode, only the signs of the partial derivatives are added together in the counter. The counters can then increase or decrease by one depending on the sign of the partial derivative in question. Here again, when the counter exceeds a certain threshold, the value of the parameter will change while remaining constrained by the set of discrete values ​​that the parameter can take.The advantage of the counter that takes into account the average values ​​of partial derivatives is that it considers their magnitude, whereas the second method offers the advantage of simpler calculation. In both cases, the changes must be in the opposite direction to the sign of the partial derivatives.

[0045] In our experiments, the effect of the scarcity of points of interest for learning, a scarcity that increases as learning yields a better solution, is that their effect will tend to be disproportionate in the absence of a compensatory mechanism obtained by combining results from several batches. As soon as a point provides information useful for learning, this information will be used, even if excessively. This can trigger oscillations in parameter values ​​around target values ​​as soon as relevant information can be used for learning. This effect is amplified by the fact that the changes in values ​​are discrete. Indeed, in the traditional gradient descent technique, the changes in values ​​are proportional to the magnitude of the partial derivatives.This allows us to approach the minimums sought by the method more or less quickly and to avoid exceeding them and fluctuating around them as soon as informational points are used for learning.

[0046] To avoid this pitfall, the method according to the invention introduces sets of counters associated with the parameters. A parameter value will only change (in one direction or the other), for example, after its associated counter has recorded a number of changes to be made according to the signs of partial derivatives that are above a given threshold, or after the sum of the partial derivatives has reached a certain threshold, depending on the embodiment. This evolution of the method according to the invention makes it possible to avoid the phenomenon observed in some versions of the method, namely oscillations of values ​​and an excessive impact on the learning of isolated points. Changes are only implemented if they apply to several batches, which will cause the associated counters to exceed their threshold.

[0047] Furthermore, this combination of results from several batches before triggering a change in the value of a parameter makes it possible to perform calculations with a large amount of training data while controlling the complexity of the calculations. Let's take as an example a batch of calculation points with a size of 32 points. This size is fixed, as we have seen, because the calculations are performed on GPU units. If we set a threshold of 20 to trigger a change in the value of a parameter, this means that a change in the parameter value can only occur after considering 640 points (32 times 20), which corresponds to a very large batch, probably not achievable in a reasonable timeframe.

[0048] According to one embodiment, which can be implemented alternatively or cumulatively with the preceding embodiments, when a value of a parameter changes after the counter has exceeded a given threshold, the counters associated with parameters that are below another threshold, less than or equal to the given threshold, are reset to zero.

[0049] According to an embodiment which can be implemented cumulatively with the previous embodiment, said other threshold is equal to the given threshold.

[0050] Thanks to this embodiment, the use of counters to control changes in parameter values ​​is refined. In our experiments, it is preferable, when at least one change in parameter value is decided, to reset the counters associated with the parameters since subsequent calculation points of the loss function will be obtained with parameter values ​​that have changed. It can be decided to reset all counters below the given threshold, or only those below another threshold, smaller than the given threshold, and thus retain the counters located between the two thresholds, which have already accumulated a certain number of clues as to the direction in which they should change.

[0051] According to one embodiment, which can be realized alternatively or cumulatively with the preceding embodiments, when a value of a parameter changes after the counter has exceeded a given threshold, the values ​​of the parameters whose associated counters are close to the given threshold at a given distance are also changed.

[0052] Thanks to this embodiment, the use of counters is further refined. We have seen that the value of a parameter is changed when its associated counter reaches a certain threshold, and that the other counters are then reset to zero. It is also possible in this embodiment to modify the values ​​of parameters whose associated counters have not reached the threshold but are close to it, for example, at a distance of 1 to 5, so as not to reset information accumulated during learning. All counters can then be reset after these changes to the parameter values ​​have been applied.

[0053] According to one embodiment, which can be implemented alternatively or cumulatively with the preceding embodiments, the given threshold is set at a value between 10 and 30.

[0054] In our experiments, the threshold for a counter beyond which a parameter will change should not be chosen too low. Furthermore, counters that are too high will tend to delay certain changes in parameter values. The range of values ​​between 10 and 30 for this threshold gives good results, with a preference for a threshold set at 20.

[0055] According to one embodiment, which can be implemented alternatively or cumulatively with the preceding embodiments, the given threshold increases as a direct function of the number of parameter values ​​to be learned.

[0056] In our experiments, the larger the code we are trying to learn, the more it allows us to correct rare events. This means that the events useful for learning, the uncorrected errors, are very infrequent. The learning process will therefore need high thresholds to judge the relevance of these rare events.

[0057] According to one embodiment, which can be carried out cumulatively or alternatively with the preceding modes, the combination of the evolutions obtained includes the estimation of a confidence interval for the values ​​of the partial derivatives associated with the respective parameters using a binomial law, and the update of the parameter value is done when this estimation exceeds a certain threshold.

[0058] This embodiment provides another way to aggregate calculation results from multiple batches. Here, rather than using counters, the values ​​of the partial derivatives taken at the different points are used to estimate a confidence interval for the value of this partial derivative at all points in the domain. A method such as the Agresti-Coull method can be used for this. Once the confidence interval indicates that the value of the partial derivative for a given parameter has very likely exceeded a given threshold (positive or negative), the parameter in question can be increased or decreased depending on the sign of the threshold.

[0059] According to one embodiment, which can be carried out alternatively or cumulatively with the preceding embodiments, the method includes a step of random initialization of the values ​​of the parameters of the function to be optimized.

[0060] Thanks to this embodiment, a starting point is given to the parameter values ​​of the function to be optimized. In our experiments, a random choice is effective and does not prevent the efficient implementation of the process according to the invention.

[0061] According to one embodiment, which can be carried out alternatively or cumulatively with the preceding embodiments, the method includes a step of random initialization of the values ​​of the parameters of the function to be optimized according to a differentiated probability for the different values ​​of the set of discrete values ​​that the parameters can take.

[0062] In our experiments, it can be useful to differentiate the probabilities of obtaining a given discrete value for a parameter according to the values ​​belonging to the set of possible discrete values. For example, in a linear code for the binary case, a random selection of the code parameters with a probability of 70% for the value 0 and 30% for the value 1 gives good results because it initiates the process with a parameter density of 1 close to that of a high-performing code.

[0063] According to one embodiment, which can be realized alternatively or cumulatively with the preceding embodiments, the function to be optimized is a linear code and the parameters of the function to be optimized are the elements of the control matrix associated with said linear code.

[0064] Thanks to this embodiment, the invention is applied to the specific case of linear codes and allows for the efficient learning of a linear code. The parameters whose values ​​are learned are those of the control matrix in order to somewhat reduce the size of the learning problem compared to direct learning of the linear code matrix. The parameters of the function to be optimized are then the elements of the code's parity matrix and will generally be bits. The parameters thus take their values ​​from a set of discrete values, namely here the set of the values ​​0 and 1. In other examples, the linear code is applied to words formed from symbols that are not binary but belong to larger sets of values. These may be, for example, codes directly encoding phase modulations used in wireless communications.Depending on the wireless communication methods deployed, the phase modulations used can be, for example, 2, 4, 8 or 16. For example, in the cases 4, 8 or 16, the codes used will be matrices whose elements will belong to finite fields of size 4, 8 or 16 (respectively Z / 8Z and Z / 16Z) for which a minimum and a maximum are defined (respectively, 0 and 3, 0 and 7, 0 and 15) and the changes of parameter values ​​in these finite fields will be done by additions or subtractions of integers which will be limited where appropriate so as not to exceed the minimum or the maximum.

[0065] According to one embodiment, which can be implemented alternatively or cumulatively with the preceding embodiments, the function to be optimized is a linear code and the loss function is measured according to the cross-entropy between a result word of encoding by the function to be optimized and decoding of this result word by a decoder using the control matrix associated with the linear code of the function to be optimized and a point from a batch is provided to the loss function by applying random errors to the word formed only of minimal symbols from the set of discrete values.

[0066] Thanks to this embodiment, a loss function suitable for learning a linear code is defined. The loss function is defined as the cross-entropy measured between a known word, the result of encoding, and its decoding after random errors have been applied to the known word. If the encoding is efficient, the decoding will recover the known word even in the presence of errors, and the cross-entropy will be zero. Optimizing the encoding therefore amounts to minimizing this loss function. The known word must be a possible encoding result; for all possible linear codes, the word formed solely of minimal symbols, for example, consisting only of 0s for a binary code, is such a word.

[0067] According to a first material aspect, the invention relates to a data processing device implementing a method for obtaining parameter values ​​of a function, called the function to be optimized, said parameters taking their values ​​from a set of discrete values, the method implementing another function, called the loss function, such that the optimization of the function to be optimized corresponds to minimizing the loss function, said loss function being differentiable with respect to the parameters, said device comprising the following modules: • A module for obtaining, for at least one parameter, an evolution of the value of a parameter considered as a function of a partial derivative at at least one point of the differentiable loss function for the parameter considered; • A module for obtaining a parameter value according to the evolution obtained, the value obtained being constrained to belong to the set of discrete values.

[0068] According to another material aspect, the invention relates to a computer program suitable for implementation by a data processing device, the program comprising code instructions which, when executed by a processor, carries out the steps of the obtaining process defined above.

[0069] According to another material aspect, the invention relates to a data carrier on which is recorded a computer program comprising a sequence of instructions for the implementation of the obtaining process defined above.

[0070] According to another material aspect, the invention relates to a linear code obtained by a process according to the invention implemented by a data processing device according to the invention.

[0071] The data carrier can be any entity or device capable of storing programs. For example, the carrier can include a storage means, such as a ROM, for example a CD-ROM or a microelectronic circuit ROM, or a magnetic recording means such as a hard drive. Alternatively, the carrier can be a transmissible medium such as an electrical or optical signal, which can be transmitted via an electrical or optical cable, by radio, or by other means. The program according to the invention can, in particular, be uploaded to a network such as the Internet. Alternatively, the information carrier can be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the process in question.The program according to the invention can use any type of computer technology in terms of compiled programming languages, interpreted languages, or a combination of both, as well as in terms of operating systems. Brief description of the figures

[0072] The invention will be better understood upon reading the following description, given by way of example, and made with reference to the accompanying drawings in which:

[0073] [Fig. 1] represents a data processing device implementing a method for obtaining the parameter values ​​of a function from a set of discrete values.

[0074] [Fig.2] illustrates an example of a sequence of steps implemented in a process according to the invention.

[0075] [Fig.3] illustrates an example of implementation of the method for obtaining the parameter values ​​of a linear code.

[0076] [Fig.4] illustrates the performance of a linear code whose parameter values ​​have were obtained according to a process according to the invention. Detailed description

[0077] Fig. 1 represents an example of a data processing device 100 implementing a method for obtaining the values ​​VI, V2, V3 of parameters PI, P2, P3 of a function F from a set E of discrete values.

[0078] The method includes the prior selection of another function L, called the loss function.

[0079] The device 100 includes a module 101 for obtaining, for the parameters PI, P2, P3, evolutions Al, A2, A3 of the values ​​VI, V2, V3 of the parameters PI, P2, P3 as a function of the partial derivatives Dl, D2, D3 of the loss function L for the parameters PI, P2, P3 at a point of the domain of definition of the loss function L.

[0080] The device 100 includes a module 102 for obtaining new values ​​VI, V2, V3 of the parameters PI, P2, P3 according to the evolutions obtained Al, A2, A3, the new values ​​obtained being constrained to belong to the set E of discrete values.

[0081] The data processing device 100 has the hardware architecture of a conventional computer. It includes in particular at least one processor, random access memory of the RAM type and read-only memory such as Flash memory, ROM, (not shown in the figure) as well as input-output devices such as, in some cases, keyboards and / or screens (not shown in the figure), and network ports allowing communication with other entities and servers via a communication network such as the Internet not shown in the figure.

[0082] The processing units present in the processing device 100 and used to carry out the process according to the invention will be, in some embodiments of the graphics processing units, referred to as GPUs (not shown in the figure). In other examples, the processing units are CPUs (not shown in the figure). The use of a GPU processing unit to carry out the process according to the invention provides greater performance to the calculations required to perform the process according to the invention, and therefore allows for more advanced learning within a given computation time, at the expense of the flexibility in defining the calculations that can be performed by a CPU. Several embodiments of the invention are specifically adapted to the use of GPU processing units.

[0083] The device 100 will, for example, be a computer server that performs the process of obtaining a set of values ​​VI, V2, V3 of parameters PI, P2, P3 of a function F, said values ​​VI, V2, V3 belonging to a set E of discrete values. The function F is, for example, a linear code, and the parameters PI, P2, P3 of the code are the elements of the code's check matrix, i.e., bits that take the values ​​0 or 1. In some cases, the code can encode several symbols before transmission, and the values ​​VI, V2, V3 of the parameters PI, P2, P3 are no longer bits but still belong to a set E of discrete values.

[0084] In the case where the device 100 is a computer server, it will therefore perform a learning process which will then allow for the production of an optimized function F, for example a new linear code. The device 100 may also belong to a cloud architecture, and the calculations enabling the production process are then performed on computing, memory, and network resources provided on demand by the cloud system.

[0085] In other embodiments, the data processing device 100 belongs to equipment that will implement the function F whose values ​​VI, V2, V3 of parameters PI, P2, P3 are obtained by the process implemented by the Device 100. For example, Device 100 can be included in telecommunications equipment, such as a router, a mobile network base station, or any other telecommunications equipment. Device 100 can then learn a linear code and distribute it directly to the equipment in which it is included. In some embodiments, Device 100 can use actual traffic data passing through the telecommunications equipment in which it is included to improve the implementation of the acquisition method.

[0086] Module 101 of the data processing device 100 will allow to obtain evolutions Al, A2, A3 of the values ​​VI, V2, V3 of the parameters PI, P2, P3 as a function of the partial derivatives Dl, D2, D3 of the loss function L for the parameters PI, P2, P3 at a point of the domain of definition of the loss function L.

[0087] In an example of an embodiment of the process, module 101 will calculate all the partial derivatives of the loss function L for a set, called a batch, of points in the domain of definition of the loss function L with respect to the parameters of the function to be optimized F. From these partial derivatives, module 101 can, for example, calculate their average values ​​over all the points in the batch to obtain a single value. The average values ​​thus obtained can be directly considered as the possible evolutions Al, A2, A3 of the values ​​VI, V2, V3 of the parameters PI, P2, P3. Such possible evolutions Al, A2, A3, obtained as averages over the different points in the batch of the partial derivatives D1, D2, D3, will obviously not, in general, be discrete evolutions.

[0088] Another embodiment of the process will consist, for module 101 of device 100, in simply obtaining the signs of the partial derivatives of the loss function L for a set, called a batch, of points in the domain of definition of the loss function L with respect to the parameters of the function to be optimized F. From these signs of partial derivatives, module 101 can, for example, obtain possible changes A1, A2, A3 in the values ​​V1, V2, V3 of the parameters P1, P2, P3 by adding the number of positive signs of partial derivatives for a given parameter and subtracting the number of negative signs. This embodiment will yield changes in parameter values ​​that will be discrete but may be greater than +1 or -1.

[0089] In exemplary implementations, the partial derivatives D1, D2, D3 whose absolute values ​​are below a given threshold are considered to be zero, so as not to include in the training data values ​​that are too small, which would not be significant, and which are best ignored. This cancellation can take place before averaging the partial derivatives D1, D2, D3 over all the points in the dataset, or for the partial derivatives averaged over all the points in the dataset.

[0090] Other embodiment examples are possible which allow module 101 to obtain possible evolutions Al, A2, A3 of the values ​​VI, V2, V3 of the parameters PI, P2, P3 of the function to be optimized F as a function of the partial derivatives Dl, D2, D3 of the loss function L at one or more points of the domain of definition of the loss function L.

[0091] Module 102 of the data processing device 100, for its part, will allow obtaining new values ​​VI, V2, V3 of the parameters PI, P2, P3 according to the evolutions obtained Al, A2, A3 by module 101, the new values ​​obtained being constrained to belong to the set E of discrete values.

[0092] We have seen that the module 101 could obtain possible evolutions Al, A2, A3 of the values ​​VI, V2, V3 which are not discrete quantities or, even if they are discrete quantities in certain embodiment examples, which can be greater than +1 and -1. Such evolutions applied to discrete values, which for example in the case of learning binary parameters are the values ​​0 or 1, would have the consequence that the new values ​​VI, V2, V3 obtained after application of the evolutions Al, A2, A3 would no longer generally belong to the set E to which the values ​​VI, V2, V3 of the parameters PI, P2, P3 must belong.

[0093] Module 102 must therefore apply the changes obtained A1, A2, A3 while ensuring a constraint that the values ​​V1, V2, V3 remain within the set E of discrete values. Several embodiments are possible to achieve this.

[0094] For example, in some embodiments, the changes Al, A2, A3 are real quantities obtained by averaging the partial derivatives D1, D2, D3 over the different points of a batch. In these examples, module 102 can use a threshold system to transform the real changes Al, A2, A3 obtained by module 101 into discrete changes that ensure that the new values ​​VI, V2, V3 will always belong to the set E of discrete values.

[0095] Furthermore, if a discrete evolution is applied to a value that is already at the minimum or maximum of the set E of discrete values, or close to them, the module 102 will reduce the evolutions A1, A2, A3 in order to ensure that the minimum or maximum is not exceeded.

[0096] In examples where the changes A1, A2, A3 obtained by module 101 are already discrete quantities, module 102 can also use a threshold system. These thresholds can, on the one hand, serve to prevent excessively large changes that would push the values ​​V1, V2, V3 of the parameters P1, P2, P3 outside the set E of discrete values. These thresholds can also serve to ensure that a change only occurs when a large number of training points point towards a given change, in order to avoid a large number of oscillations caused by a small number of insignificant data points. For example, in a set of 32 For each point, the changes in A1, A2, and A3 can be obtained by summing the signs of the partial derivatives D1, D2, and D3, and are limited to changes of +1 or -1 if the parameters P1, P2, and P3 have binary values ​​V1, V2, and V3. Rather than performing these changes of +1 or -1 as soon as the changes in A1, A2, and A3 obtained by module 101 are positive or negative, respectively, it is possible to use a threshold, for example, +30 or -30, to achieve an effective change of +1 or -1 in the values ​​V1, V2, and V3. In this example, a change will only be effective if at least 30 points in the set of 32 have a partial derivative D1, D2, and D3 of the same sign, indicating that almost all the points in the set show the same indication in their partial derivative.

[0097] In embodiment examples, the process is iterative and module 102 performs the combination of the evolutions Al, A2, A3 obtained by module 101 during several successive steps before proceeding to a change of the values ​​VI, V2, V3 of the parameters PI, P2, P3.

[0098] This combination may include, for example, the use of counters. The changes A1, A2, A3 obtained by module 101 can, for example, be reduced to the sign of the partial derivatives D1, D2, D3 for the parameters P1, P2, P3, and therefore to increments or decrements of 1 in the opposite direction to the sign of the partial derivatives D1, D2, D3 averaged over all the points in the set. These changes A1, A2, A3 are then combined by module 102 in a counter associated with the parameters P1, P2, P3. The changes in the values ​​V1, V2, V3 will only be effective when one of the counters exceeds a given threshold, and will apply to the values ​​of the parameters for which the associated counter has reached the given threshold. The other counters will then be reset to zero in some embodiments. It is also possible not to reset all the counters to zero, but to keep those that are close to the given threshold.It is also possible to choose to adjust the values ​​of parameters associated with counters that have not reached the given threshold but are close to it, and then reset all counters to zero.

[0099] In another example of a combination, the values ​​of the evolutions Al, A2, A3 obtained by module 101 are equal to the partial derivatives D1, D2, D3 of the loss function L, possibly averaged over all the points of a calculation batch. These different evolutions Al, A2, A3 are then summed together in counters associated with the parameters and, as in the previous example, module 102 performs an actual evolution of the values ​​VI, V2, V3 of the parameters PI, P2, P3 only when these counters exceed a given threshold. Unlike the previous example, this combination takes into account the magnitude of the partial derivatives D1, D2, D3 and not just their sign. As in the previous mode, the actual evolutions of the values ​​VI, V2, V3 must be in the opposite direction to the signs of the partial derivatives D1, D2, D3 since the optimization of the function F that we seek to achieve by modifying the values ​​VI, V2, V3 of the parameters PI, P2, P3 corresponds to a minimization of the loss function L.

[0100] In another example of a combination, the process does not use counters but an estimation of the confidence interval for the values ​​of the partial derivatives D1, D2, D3. As before, the changes A1, A2, A3 obtained are directly related to the partial derivatives D1, D2, D3. Module 102 will then perform, as it receives the changes A1, A2, A3, an estimation of the confidence interval in which the partial derivatives for the parameters P1, P2, P3 lie for the entire domain. The Agresti-Coull method can be used to calculate this confidence interval. When sufficient certainty is reached, Module 102 can perform an actual change in the values ​​V1, V2, V3 of the parameters P1, P2, P3 for which sufficient confidence is present, and reset all the estimates for subsequent measurements.

[0101] Fig. 2, meanwhile, presents an example of a sequence of steps implemented in a process according to the invention.

[0102] In this embodiment, the process is iterative. An initialization step for the values ​​VI, V2, V3 of the parameters PI, P2, P3 has taken place, but is not shown in the figure. This initialization can be random or expert-directed. When it is random, different probabilities can be assigned to the different values ​​present in the set E of discrete values ​​to initialize each parameter, in order to provide a relevant starting point for the process.

[0103] The SI step corresponds to obtaining the possible evolutions Al, A2, A3 of the values ​​VI, V2, V3 by the module 101 as a function of the partial derivatives Dl, D2, D3 of the loss function L with respect to the parameters PI, P2, P3. These possible evolutions can be, for example, increments or decrements of one as an inverse function of the sign of the partial derivatives Dl, D2, D3 averaged over all the points of the batch.

[0104] Step S2 corresponds to a combination, over successive iterations, of the changes A1, A2, and A3. In the example presented here, this combination is performed using counters. The changes of +1 and -1 are added at each iteration to counters C1, C2, and C3 associated with the parameters P1, P2, and P3.

[0105] When the counters Cl, C2, C3 have been updated with the evolutions Al, A2, A3 obtained in step SI, it is tested whether one or more counters Ci among the counters Cl, C2, C3 has reached a given threshold S. This is represented by a test in [Fig.2].

[0106] If this is not the case (branch N of the test), the iterative process resumes at step SI with the obtaining of new evolutions Al, A2, A3 for a new set of points belonging to the domain of definition of the loss function L.

[0107] If a counter Ci reaches the given threshold S (branch Y of the test), the corresponding evolution Ai is applied to the value Vi. This is of course the case for all counters that reach the threshold and, in some embodiments, this may also be the case for counters close to the threshold.

[0108] Next, in this example, the set of counters Cl, C2, C3 are reset to zero, which is indicated by the mention RAZ in [Fig.2], and the process continues in step SI with a new set of points belonging to the definition set of the loss function L.

[0109] Several stopping criteria for the iterative process can be chosen. The process can stop when there have been no further changes for a certain number of iterations, or after a fixed number of iterations, for example. The process can also stop after an evaluation of the function F to be optimized using the values ​​obtained VI, V2, V3 of the parameters PI, P2, P3 at a given time, but such an evaluation can take a significant amount of time to perform.

[0110] Figure 3 illustrates an example of implementation of the process allowing to obtain the parameter values ​​of a linear code.

[0111] In this example, the function to be optimized F is a linear encoding, and the parameters PI, P2, P3 form, for example, a parity-check matrix that defines the encoding. A decoding module DEC is provided, which allows the recovery of an encoded and then transmitted word, even in the presence of a number of transmission errors. The decoding module DEC can be, for example, an information-propagating decoder, so that the function F is optimized taking into account this very frequent type of decoding. The decoding module DEC uses a current state of the parity-check matrix, represented by the element COD in [Fig. 3], to perform decoding. The current state of the parity matrix is ​​given by the values ​​VI, V2, V3 of the parameters PI, P2, P3.

[0112] To carry out the iterative process, a set of measurement points must be defined. For this purpose, the word X, consisting only of zeros, is given. Such a word is a possible encoding result for any linear code, and, in a linear code, a possible encoding result represents all possibilities up to known transformations. The word X, consisting only of zeros, is therefore representative. ERR noise is added to this word, which is indicated in [Fig. 3] by a circled + operator.

[0113] The DEC decoder will then attempt to correct the ERR error. The result of the DEC decoding for a point is denoted X. If the DEC decoder succeeds in correcting the introduced ERR error, the result denoted X will also be a word consisting only of zeros. The encoding function F is optimized when the DEC decoder manages to correct the maximum number of errors. The loss function L to be minimized is defined as the binary cross entropy (BCE) between the original word X and the original ERR error. The result word, denoted X, is obtained after decoding (DEC), which attempts to correct the introduced error (ERR). The loss function L is indeed minimal when the decoding (DEC) process manages to correct a maximum number of introduced errors (ERR).

[0114] The processing device 100 therefore takes as input the partial derivatives D1, D2, D3 of the loss function L as seen previously. These partial derivatives D1, D2, D3 of the loss function L are obtained for a batch of points of a given size, consisting only of zeros and to which errors ERR have been added. The two modules 101 and 102 then proceed respectively to obtain the evolutions A1, A2, A3 and then to possibly apply the evolutions A1, A2, A3 to obtain new values ​​V1, V2, V3 of the parameters P1, P2, P3. In exemplary embodiments, the results of several batches are combined, for example using a counter mechanism as seen previously.

[0115] If new values ​​VI, V2, V3 are obtained, they are integrated into the COD parity matrix. The function F to be optimized is thus potentially improved before a new iteration of the process on a new set of points which will be decoded DEC with the new values ​​VI, V2, V3 of the parameters PI, P2, P3 of the function F to be optimized.

[0116] Fig. 4, meanwhile, illustrates the performance of a linear code whose parameter values ​​were obtained following a process according to the invention.

[0117] In this figure, the performance of three linear codes is represented by three broken lines formed by measurement points. These three codes are (128, 64) codes, meaning that they encode 64-bit words by doubling their size to 128 bits. The matrix implementing these codes therefore has a size of 128 * 64 = 8192 bits.

[0118] These three linear codes are presented here in 128 by 64 bit size and are designated ZC, CCSDS, and AR3A. The performance of the ZC code (acronym for Zero Coder) is represented by black triangles: this is the code obtained by implementing the method of the invention. The other two codes are designated CCSDS and AR3A (acronyms for Consultative Committee for Space Data Systems and Accumulate Repeat-3 and Accumulate, respectively) and represented by black and white circles, respectively. These two codes are constructed using traditional theoretical and heuristic methods and are known to have very good performance in 128 by 64 bit size. The performance of the CCSDS and AR3A codes as presented here is taken from the following article: Code Design for Short Blocks: A Survey, by Liva, Gaudio, Ninacs and Jerkovits, in Proceedings of EuCNC 2016, available at https: / / arxiv.org / abs / 1610.00873.The CCSDS and AR3A codes have the best performance among the 14 codes listed in this article.

[0119] The performance of the three codes is measured as follows. The horizontal axis Eb / NO is a measure related to noise in a communication channel. More The higher this measurement, the less the communication is disrupted by noise in the channel. More precisely, Eb / NO is a measure of the amount of energy expended per information symbol (Eb for energy per bit) normalized by the amount of noise affecting the signal carrying the information. This quantity is very commonly used in the field.

[0120] The vertical axis, for its part, measures the errors that remain uncorrected after decoding. The acronym BLER stands for Block Error Rate, which is the rate of blocks (words) in error after decoding.

[0121] The graphs are interpreted as follows: the lower the Eb / NO ratio, the more noise disrupts communication and the higher the BLER uncorrected error rate will be. For an Eb / NO ratio of 1, the uncorrected error rate is typically 0.5, or one uncorrected error out of two, whereas for an Eb / NO of 4, the uncorrected error rate drops to one in a thousand or even one in ten thousand. The points on the graph are measured for a given Eb / NO ratio by submitting a large number of words, encoding them according to the evaluated code, transmitting them with a channel having the given Eb / NO ratio, decoding them with the same knowledge propagation decoder, and with the same number of iterations (200). The number of uncorrected errors for a given code then gives a point on the curve representing the code.

[0122] To better assess the performance of the three codes, two theoretical curves are added to the graph. The Sphere Packing curve indicates the limit below which no code exists that can achieve its performance. The Random Coding Union curve indicates an area below which we are certain that at least one code exists that performs as well.

[0123] The ZC code was obtained by implementing the method of the invention. To obtain it, 8192 binary parameters had to be learned. The method was initialized with a random matrix where the density of 1s at initialization was 30%. The embodiment in this example consisted of measuring the gradients on batches of size 8, i.e., 8 points on which the partial derivatives with respect to the parameters are measured. The partial derivatives are averaged over the eight points of the batch, and then only the signs are retained. These signs are used to increment or decrement a counter. The update threshold for a parameter was set at 20.

[0124] Thanks to this implementation of the process, several codes can be obtained and will be evaluated until the best one is selected, which is the ZC code whose performance is shown in the graph. Its performance can be seen to be close to the theoretical performance, superior to and equivalent to the best state-of-the-art codes.

[0125] Finally, it should be noted here that, in this text, the term "module" can refer to a software component, a hardware component, or a set of hardware and software components, a software component itself corresponding to A computer program or subprogram, or more generally any element of a program capable of implementing a function or set of functions as described for the modules concerned, is defined as follows: Similarly, a hardware component corresponds to any element of a hardware assembly capable of implementing a function or set of functions for the module concerned (integrated circuit, smart card, memory card, etc.).

Claims

Demands

1. A method for obtaining values ​​(VI, V2, V3) of parameters (PI, P2, P3) of a function (F) implemented by a data processing device (100), said parameters (PI, P2, P3) taking their values ​​(VI, V2, V3) from a set (E) of discrete values, the method implementing another function (L), called the loss function, such that the optimization of the function (F) corresponds to minimizing the loss function (L), said loss function (L) being differentiable with respect to the parameters (PI, P2, P3), the method comprising, for at least one parameter (PI, P2, P3), a step of obtaining an evolution (A1, A2, A3) of the value (VI, V2, V3) of a parameter (PI, P2, P3) considered as a function of a partial derivative (D1, D2, D3) at at least one point of the loss function (L) differentiable for the parameter (PI, P2, P3) considered;followed by a step of obtaining a value (VI, V2, V3) of parameter (PI, P2, P3) according to the evolution (Al, A2, A3) obtained, the value (VI, V2, V3) obtained being constrained to belong to the set (E) of discrete values.;

2. A method for obtaining values ​​(VI, V2, V3) of parameters (PI, P2, P3) of a function (F) according to claim 1 characterized in that an evolution (Al, A2, A3) obtained from a value (VI, V2, V3) of at least one parameter (PI, P2, P3) considered includes an increment by one or a decrement by one as an inverse function of the sign of the partial derivative (Dl, D2, D3) at at least one point of the loss function (L) for the parameter (PI, P2, P3) considered as well as a zero evolution of a value of a parameter considered if this parameter already has the value of the minimum or the maximum of the set (E) of discrete values.

3. A method for obtaining values ​​(VI, V2, V3) of parameters (PI, P2, P3) of a function (F) according to claim 1 or 2, characterized in that the method is iterative and comprises, for one iteration, the selection of a set of points from the domain of the loss function (L), said set of points being referred to as the batch, and the step of obtaining an evolution (A1, A2, A3) of the value (VI, V2, V3) of a parameter (PI, P2, P3) considered comprises the calculation of an average of the loss function (L) for the points of the batch and the evolution (A1, A2, A3) of the value of the parameter considered is an inverse function of the partial derivative of the mean of the loss function (L) for this parameter.

4. A method for obtaining values ​​(VI, V2, V3) of parameters (PI, P2, P3) of a function (F) according to claim 1 or 2, characterized in that the method is iterative and comprises, for one iteration, the selection of a set of points from the domain of the loss function (L), said set of points being called the batch, and the step of obtaining an evolution (A1, A2, A3) of the value (VI, V2, V3) of a parameter (PI, P2, P3) considered comprises the calculation of partial derivatives (D1, D2, D3) of the loss function (L) for the parameter considered (PI, P2, P3) at the points of the batch, followed by the reduction of these partial derivatives to increments or decrements inversely proportional to the sign of the partial derivatives, the evolution (A1, A2, A3) being obtained by combining these increments and decrements.

5. Method of obtaining values ​​(VI, V2, V3) of parameters (PI, P2, P3) of a function (F) according to any one of claims 1 to 4 characterized in that the partial derivative (D1, D2, D3) as a function of which an evolution (Al, A2, A3) of value (VI, V2, V3) of parameter (PI, P2, P3) is obtained is reduced to zero if its absolute value is less than a given threshold.

6. Method of obtaining values ​​(VI, V2, V3) of parameters (PI, P2, P3) of a function (F) according to any one of claims 1 to 5 characterized in that the step of obtaining a value of a parameter includes a step of combining the evolutions (Al, A2, A3) obtained with the evolutions (Al, A2, A3) obtained during the previous steps and a step of updating the value (VI, V2, V3) of a parameter (PI, P2, P3) as a function of the combined evolutions (Al, A2, A3).

7. A method for obtaining values ​​(VI, V2, V3) of parameters (PI, P2, P3) of a function (F) according to any one of claims 1 to 6, characterized in that the step of obtaining a value (VI, V2, V3) of a parameter (PI, P2, P3) according to the evolution (A1, A2, A3) obtained comprises updating a counter (C1, C2, C3) associated with the parameter (PI, P2, P3) as a function of the partial derivatives of the loss function (L) with respect to the respective parameters (PI, P2, P3), and characterized in that the value (VI, V2, V3) obtained does not change only when the counter (Cl, C2, C3) associated with the parameter (PI, P2, P3) has exceeded a given threshold (S).

8. Method of obtaining values ​​(VI, V2, V3) of parameters (PI, P2, P3) of a function (F) according to claim 7 characterized in that, when a value (VI, V2, V3) of a parameter (PI, P2, P3) changes after the counter (Cl, C2, C3) has exceeded the given threshold (S), all counters (Cl, C2, C3) associated with parameters (PI, P2, P3) that are below another threshold, less than or equal to the given threshold (S), are reset to zero.

9. Method of obtaining values ​​(VI, V2, V3) of parameters (PI, P2, P3) of a function (F) according to one of claims 7 or 8 characterized in that the given threshold (S) is fixed at a value between 10 and 30.

10. Method of obtaining values ​​(VI, V2, V3) of parameters (PI, P2, P3) of a function (F) according to any one of claims 1 to 9 characterized in that the method comprises a step of random initialization of the values ​​(VI, V2, V3) of the parameters (PI, P2, P3) of the function (F) according to a differentiated probability for the different values ​​of the set (E) of discrete values ​​that the parameters (PI, P2, P3) can take.

11. Method of obtaining values ​​(VI, V2, V3) of parameters (PI, P2, P3) of a function (F) according to any one of claims 1 to 10 characterized in that the function (F) is a linear code and in that the parameters (PI, P2, P3) of the function (F) are the elements of the control matrix associated with said linear code.

12. A method for obtaining values ​​(VI, V2, V3) of parameters (PI, P2, P3) of a function (F) according to claim 11 in combination with one of claims 3 or 4 characterized in that the loss function (L) is measured according to the cross entropy between a word (X) result of encoding by the function (F) and the decoding of this result word (X) by a decoder using the control matrix associated with the linear code of the function (F) and characterized in that a point of a batch is provided to the loss function (L) by applying random errors to the word formed only of the minimal symbols among the set (E) of discrete values.

13. A data processing device (100) implementing a method for obtaining values ​​(VI, V2, V3) of parameters (PI, P2, P3) of a function (F), said parameters (PI, P2, P3) taking their values ​​among a set (E) of discrete values, the process implementing another function (L), called loss function, such that the optimization of the function (F) corresponds to minimizing the loss function (L), said loss function (L) being differentiable with respect to the parameters (PI, P2, P3), said device (100) comprising the following modules: • A module (101) for obtaining, for at least one parameter (PI, P2, P3), an evolution (A1, A2, A3) of the value (VI, V2, V3) of a parameter (PI, P2, P3) considered as a function of a partial derivative (D1, D2, D3) at at least one point of the loss function (L) differentiable for the parameter (PI, P2, P3) considered; • A module (102) for obtaining a value (VI, V2, V3) of parameter (PI, P2, P3) according to the evolution (Al, A2, A3) obtained, the value (VI, V2, V3) obtained being constrained to belong to the set (E) of discrete values.

14. A computer program capable of being implemented by a data processing device (100), the program comprising code instructions which, when executed by a processor, carries out the steps of the process of obtaining according to any one of claims 1 to 12.

15. Data carrier on which is recorded a computer program according to claim 14 comprising a sequence of instructions for carrying out the method of obtaining according to any one of claims 1 to 12.

16. Linear code obtained by a method of obtaining according to one of claims 11 or 12 implemented by a data processing device (100) according to claim 13.

Images