Method for analyzing numerical data

Abstract

The invention is a method for analyzing data measured by a detector using an analysis algorithm comprising at least one multilayer perceptron neural network. The neural network comprises at least two successive layers, forming an upstream layer and a downstream layer, whose respective nodes are connected by a sparse connection matrix, at least 50% of whose terms are zero or negligible. The nodes of the downstream layer are segmented into groups. Within each group, a selection of activated nodes is established based on connection signals linking the nodes of the upstream layer to the nodes of the group. Figure 1.

Description

Description Title of the invention: Method for analyzing digital data technical field

[0001] — The invention relates to data processing using a neural network, featuring a multi-layered perceptron with a particular architecture. EARLIER ART

[0002] — The use of neural networks for the analysis of various numerical patterns, of Type images, sounds, videos is common.

[0003] — Some processing algorithms implement several neural networks, powered by shared data and operating in parallel. The aim is that neural networks operating in parallel may be dissimilar, for example in using a different number of layers, or a different number of nodes per layer different. When implementing convolutional neural networks, comprising convolutional layers designed to apply convolutional filters, each network of neurons operating in parallel can implement different filters.

[0004] — The invention described below addresses the issue of the diversity of networks of neurons, and in particular neural networks operating in parallel. It allows to achieve diversity while remaining simple to implement. Description of the invention

[0005] — A first object of the invention is a method for analyzing numerical data, taking the form of a vector or a matrix, the numerical data resulting from a measurement taken by a detector, and forming an input data point for a network of neurons, the neural network comprising several neural networks ele- mentals, each elementary neural network comprising a multi-perceptron ticouches extending between:

[0006] a first layer; an output layer, the output layer containing an analysis result - commentary, by the elementary neural network, of the numerical data;

[0007] — each multilayer perceptron being such that:

[0008] - each layer is assigned a rank, the rank of a layer being all the more raised as the layer is close to the output layer; at least two layers of successive rows, forming an upstream layer and a downstream layer, contain nodes, nodes of the two layers being connected, each node of the upstream layer addresses a connection signal to at least one node of the downstream layer to which it is connected, the connection of nodes of the upstream and downstream layers being defined by a connection matrix, each term of the connection matrix being associated with a node of the layer upstream and a node of the downstream layer, and quantifying a contribution from the node of the upstream layer for the node of the downstream layer; The process involves the following steps: a) use of the input data to feed the first layer of the multilayer perceptron of each elementary neural network; b) implementation of each elementary neural network to obtain a result of elemental analysis by the output layer of the mul- perceptron layers of each elementary neural network; the process being characterized in that: at least one multilayer perceptron connection matrix for each An elementary neural network is a sparse matrix, containing at least 50% of terms are zero or have a value at least ten times lower than minus one other term in the matrix: the multilayer perceptron of each elementary neural network comprises, respectively, between two successive layers of the same rank respective, different hollow connection matrices. In one embodiment, each elementary neural network includes an extraction block programmed to extract features from the input data, the extracted features forming the first layer of the multi-layer perceptron of said elementary neural network. The extraction block may include different successive convolution layers, each convolution layer resulting from the application of a convolution filter to the preceding convolution layer. Preferably, each multilayer perceptron has the same number of layers and the same number of nodes per layer. Each multilayer perceptron can have at least one intermediate layer, between the first layer and the output layer. Two successive layers of at least one multilayer perceptron can be such that: The nodes of the downstream layer are segmented into different groups. The process is then such that step b) comprises, for the multilayer perceptron, the following sub-steps: bi) from the connection signals resulting from the nodes of the layer upstream, connected to the same group in the downstream layer, node selection, belonging to said group, at least one node of said group not being se- selected; b-ii) activation of the selected nodes, at least one node of the group not being Not activated. According to one possibility, at least one multilayer perceptron comprises several pairs including two successive layers, each pair comprising an upstream layer and a downstream layer, a connection matrix being defined between the upstream layer and the downstream layer; each connection matrix contains at least 50% of zero terms or terms whose the value is at least ten times lower than at least one other term of said matrix; Substeps bi) and b-ii) are implemented for each pair of layers. Substep bi) may include, for each group of a downstream layer, at least one multilayer perceptron: calculation of an activation signal for each node in the group, based on connection signals emitted, towards said node, by each node of the layer upstream connected audit node; normalization of the activation signals calculated for each node of the group so that the normalized signals are between a minimum value and a predetermined maximum value; selection of group nodes based on activation signals standardized. Substep bi) may include: taking into account a selection function, the selection function being a increasing or strictly increasing function; application of the selection function to each standardized activation signal, in order to calculate, for each normalized activation signal, a signal of comparison , group node selection based on the calculated comparison signal for each node. The selection function can include a sigmoid or piecewise linear function. According to one embodiment, at least one or each connection matrix of a multilayer perceptron comprises at least 90% or at least 95% of terms that are zero or at least less than at least one other term of said matrix. According to one embodiment, for at least one multilayer perceptron, each connection signal between two nodes of two successive layers is positive. According to one embodiment, the process includes a step c) of combining the elementary analysis results, resulting from each elementary neural network, to form an analysis result. According to one embodiment, the process includes, prior to step a), measurement of a physical quantity by a detector; formation, by the detector, of an input signal from the quantity measured physics; formation of input data from the input signal. The result of the analysis can be a characterization of the measured physical quantity. The process may include, prior to step a): taking into account data stored in a database, the stored data forming the input signal; input data formation using the input signal; The result of the analysis can then be a characterization of the stored data. A second object of the invention is a measuring system, comprising: a detector, configured to measure a physical quantity and to establish a input signal from the measured physical quantity; a processing unit, programmed to implement the process according to the first object of the invention. The detector can be an image sensor, a sound sensor, or a magnetic sensor. A third object of the invention is a data carrier, readable by a computer, or configured to be connected to a computer, or integrated circuit, comprising instructions for implementing the process according to the first object of the invention, in particular from a measured data. A fourth object of the invention is a computer program comprising instructions which, when the program is executed by a computer, lead to the implementation of a neural network, configured to analyze input data, the neural network comprising several elementary neural networks, each elementary neural network comprising a multilayer perceptron extending between: a first layer; an output layer, the output layer containing an analysis result - commentary, by the elementary neural network, of the numerical data; each multilayer perceptron being such that: Each layer is assigned a rank, the rank of a layer being all the higher raised as the layer is close to the output layer; at least two layers of successive rows, forming an upstream layer and a downstream layer, contain nodes, nodes of the two layers being connected, each node of the upstream layer addresses a connection signal to at least one node of the downstream layer to which it is connected, the connection of nodes of the upstream and downstream layers being defined by a connection matrix, each term of the connection matrix being associated with a node of the layer upstream and a node of the downstream layer, and quantifying a contribution from the node of the upstream layer for the node of the downstream layer; the program being characterized in that: at least one connection matrix for each multilayer perceptron is a sparse matrix, containing at least 50% zero terms or whose value is at least ten times lower than at least one other term of the matrix; multilayer perceptrons respectively include, between two layers successive sparse connection matrices of the same respective ranks, different sparse connection matrices different. The computer program may include one of the following characteristics, taken individually or in technically feasible combinations: Each multilayer perceptron has at least one intermediate layer, between the input layer and the output layer; Each multilayer perceptron comprises several pairs, each consisting of two successive layers, each pair comprising an upstream layer and a downstream layer, a connection matrix being defined between the upstream layer and the downstream layer; each connection matrix comprises at least 50% or at least 80% of zero terms or terms whose value is at least ten times less than at least one another term of said matrix. The computer program may include instructions to combine analysis results resulting from each elementary neural network. A fifth object of the invention is a computer-readable data carrier, or one that can be connected to a computer, or a printed circuit board, in which the computer program according to the fourth object of the invention is recorded. A sixth object of the invention is a data annotation method, using a program according to the fourth object of the invention, the program comprising a first elementary neural network and a second elementary neural network, the first and second elementary neural networks having respectively the same number of layers, and the same number of nodes per layer, the method comprising the following steps: 1) taking into account initial annotated input data and initial unannotated input data; 11) consideration of second annotated input data, different from the first annotated input data, and second unannotated data; iii) learning: from the first elementary neural network using the first Annotated input data: and the second elementary neural network using the second annotated input data; iv) use of the first elementary neural network for: annotate the first unannotated data, so as to form initial pseudo-annotated data; or update an annotation of initial pseudo-annotated data resulting from a previous iteration; of the second elementary neural network for: annotate the second unannotated data, so as to form second pseudo-annotated data; or update a second pseudo-annotated data annotation resulting from a previous iteration; v) Learning update: from the first elementary neural network using the first annotated input data and second input data pseudo-annotated during step iv); of the second elementary neural network using the second annotated input data and initial pseudo- input data annotated during step iv); vi) repeating steps iv) to v) until a stopping criterion is reached of iterations, such that following the iterations, the annotations of the first pseudo-annotated input data and second data Pseudo-annotated input data are considered stabilized. The first annotated and unannotated data, as well as the second annotated and unannotated data, can be data measured by a detector. The detector can be an image sensor, a sound sensor, or, more generally, a sensor of a physical quantity. According to one embodiment, (during step 1), the number of first unannotated input data is greater than the number of first annotated input data; (during step 11), the number of unannotated second input data points is greater than the number of second annotated input data points. The computer program may include several first elementary neural networks, each first elementary neural network comprising the same number of layers, and the same number of nodes per layer; among the layers, two layers of successive ranks, forming respectively an upstream layer and a downstream layer, the connection matrix between the layer upstream and downstream layer comprising at least 50% or at least 80% of zero terms or terms whose value is at least ten times less than at least one Another term for the matrix is ​​the connection matrices between said layers being different in each first neural network. Step iii) may involve training each first elementary neural network using the first annotated input data. Step iv) may include iv-l) a use of each first elementary neural network for independently annotate the initial unannotated data or update independently annotation of the first pseudo-annotated data resulting from a previous iteration; iv-2) for each first pseudo-annotated data point, a combination of the an- notations resulting from substep iv-1) to define a single annotation for the said first piece of data; Step v) may involve updating the training of each first elementary neural network using the first annotated input data and the second pseudo-annotated input data from step iv). The computer program may include several elementary second neural networks, each elementary second neural network comprising the same number of layers, and the same number of nodes per layer; among the layers, two layers of successive ranks, forming respectively an upstream layer and a downstream layer, the connection matrix between the layer upstream and downstream layer comprising at least 50% or at least 80% of zero terms or terms whose value is at least ten times less than at least one In other words, the connection matrices between these layers are different. in every second elementary neural network. Step iii) may involve training each second elementary neural network using the second annotated input data. Step iii) may include: iv-1) a use of every second elementary neural network for independently annotate the second annotated data or update in- depending on an annotation of the second pseudo-annotated data resulting from a previous iteration; iv-2) for each second pseudo-annotated data point, combination of the an- notations resulting from substep v-1) to define a single annotation. Step v) may involve updating the training of each second elementary neural network using the second annotated input data and the first pseudo-annotated input data from step iv). The invention will be better understood upon reading the description of the exemplary embodiments presented later in this description, in connection with the figures listed below. FIGURES Figure 1 schematically illustrates the main elements enabling the implementation of the invention. Figure 2A represents an architecture of an analysis algorithm implementing the invention. The analysis algorithm is based on several elementary neural networks. Figure [Fig. 2B] schematically illustrates an extraction block implemented in each elementary neural network. Figure [Fig. 2C] schematically represents a multilayer perceptron of each elementary neural network. Figure [Fig. 3A] shows the main steps in the operation of a neural network as described in relation to Figure [Fig. 2C]. Figure 3B details the steps for selecting one or more nodes from a group of nodes in a layer of an elementary neural network described in connection with Figure 2C. Figure [3C] is a detail of a group, or cluster, of nodes shown in Figure [2C] Figures [Fig.3D] and [Fig.3E] illustrate examples of selection functions. Figure [Fig. 4A] shows a parallel between a first neural network and a second neural network that can be used for partially supervised learning. Figure 4B shows the main stages of partially supervised learning implementing the architecture described in relation to Figure 4A. Figure 5 schematically illustrates a variant of the parallel implementation of the first and second neural networks, described in relation to Figure 4A. According to this variant, the first and second neural networks are subdivided into several elementary neural networks operating in parallel. PRESENTATION OF SPECIFIC IMPLEMENTATION METHODS Figure 1 illustrates an example of a system implementing the invention. The system includes a detector 10, configured to detect a physical quantity 11. In this example, and without limitation, the detector 10 is an image sensor, the physical quantity being an electromagnetic wave. The detector can The sensor is an acoustic sensor, the physical quantity being an acoustic wave. The detector can be a magnetic or electrical sensor. Generally, the detector generates an input signal S that represents the physical quantity it has detected. The detector 10 is connected to a processing unit 12, configured to process measurements. The processing unit 12 includes a computer or a dedicated processor. The processing unit is connected to a memory 13 containing instructions, in the form of a computer program or a dedicated integrated circuit, to process the input signal S. The dedicated circuit can be an ASIC (Application-Specific Integrated Circuit). The input signal S is processed to characterize the detected physical quantity. In the example described, the processing performed by the processing unit aims to classify the image formed by the image sensor. The images contain alphanumeric characters, and the processing unit is programmed to identify these characters. The identification process consists of classifying each image into a predetermined character class. This is an image recognition application. Other types of characterization are conceivable. Thus, a characterization can be, in a non-limiting way, an identification, a recognition (image recognition, sound recognition), a classification among predetermined classes, a determination of a probability of belonging to a class, or an estimation of a parameter on which the measured physical quantity depends. Figure [Fig. 2A] schematically illustrates an architecture of an analysis algorithm 2 executed by the processing unit 12. The image acquired by the detector 10 forms an input data IN of the analysis algorithm 2 implemented by computer. The analysis algorithm 2 comprises a neural network NN. An important aspect of the invention is that the neural network includes several multilayer perceptrons, each multilayer perceptron operating in parallel. In the example described, the neural network comprises three multilayer perceptrons 31, 32, 33. Each multilayer perceptron extends from a first layer L1, L2, and L3, as described in relation to [Fig. 2B]. Each first layer can form an input layer of the neural network. This is particularly the case when the input data is a vector. In this example, the input layer of each multilayer perceptron is fed to an extraction block, the latter being configured to extract features from the image forming the input data. Thus, the NN neural network has three extraction blocks 21, 22, 23, each extraction block being configured to extract features from the image forming the input data.Each extraction block is positioned upstream of a multilayer perceptron, . so that the image features extracted by an extraction block form the first layer of a multilayer perceptron. In general, a neural network NN is formed from several elementary neural networks operating in parallel, for example, NN1, NN2, NN3, NN3; in the case of three elementary neural networks. Each elementary neural network includes a multilayer perceptron 31, 32, 33. Each elementary neural network may include a feature extraction block 21, 22, 23 feeding the multilayer perceptron 31, 32, 33. In the example described, the neural network comprises three elementary neural networks. By operating in parallel, we mean that each elementary neural network uses the same input data and performs processing, independently of any other elementary neural network, to produce output data. The respective output data of each elementary neural network can be combined to form the output data of the neural network. Figure 2B schematically represents the extraction block 21 of the first elementary neural network (NN), noting that extraction blocks 22 and 23 have a similar structure. Extraction block 21 extends between an input layer 21, which corresponds to the input IN of the neural network, and an output layer 21. Extraction block 21 comprises M successive convolution layers Cm, ..., Cm. When m > 1, each convolution layer Cm is obtained by convolving a previous layer with a convolution filter, for example, a 5x5 convolution filter. The number M of convolution layers Cm can, for example, be equal to 2. The output layer 21 is usually formed by concatenating the data from the last convolution layer, forming a feature vector. The content of output layer 21 constitutes the features extracted from the input data. The extraction block 21 is configured to extract relevant features from the image, which then serve as input data for a multilayer perceptron, described later. A pre-programmed extraction block 21 can be used to extract features from an image. Alternatively, or complementaryly, the extraction block 21 is trained, along with the multilayer perceptron it feeds, to determine the parameters of the convolution filters implemented in the convolution layers C. The features extracted by the extraction block 21 are then analyzed by a multilayer perceptron 31. An important aspect of the invention is that the neural network comprises different elementary neural networks. In the example shown in [Fig. 2A], the neural network NN comprises three elementary neural networks. The number of The number of elementary neural networks can be between 2 and 10. Fig. 2C represents an architecture of a multilayer perceptron 31 of an elementary neural network NN, knowing that each multilayer perceptron of each elementary neural network has a similar architecture. In the example shown, the multilayer perceptron 31 has three layers: Ly, L, and L;. As is common in this type of neural network architecture, each layer has *a p nodes. In Figure 2A, each *np node is represented by a circle. Each layer L is assigned a rank. The rank of layer L is higher the closer the layer is to the output layer OUT of the multilayer perceptron 31. In this example, the output layer OUT is layer L3. Layer L is a rank 1 layer, layer L; is a rank 2 layer, and layer L; is a rank 3 layer. Generally, every multilayer perceptron has at least two layers: the input layer and the output layer. It may have several intermediate layers between the input and output layers. In each layer of rank ", each node "", p is assigned an order / . The number of nodes *n, p in a layer of rank " is P,. P, is, for example, between a few tens and a few thousand. The respective numbers of nodes in two different layers can be different. The operation of a multilayer perceptron neural network is known to those skilled in the art. If we consider two successive layers, i.e., of successive ranks -1, 7, each node of layer L, of rank , called the downstream layer, receives an activation signal *, p corresponding to a linear combination of the connection signal *an-1, p, p of one or more nodes *x-t, p' of layer L, of rank N-1, called the upstream layer. Thus, the value %+, of the activation signal of each node *n, p' of a layer L, is such that: = 2 W s, ; bu) ( = w.: I pp 1, ; fl p'=1 pp Or Sn -1, p'. p is the connection signal addressed by each node *7 - 1. p° of order P' of layer L,,- ; of rank - 1 towards the node of order of layer L, of rank; b. tp is a bias associated with each node *a-1, p' of the Ly-1 layer. 1, is an activation function associated with the order node / of the layer rank *; Y >. p is a connection coefficient between each node *»- 1, p' of the previous layer and the node *n, p of order P of layer of rank n. is determined by the man of the RM MEME M ME OM EE The form of each activation function f np profession, It could for example be a function of the hyperbolic tangent or sigmoid type. The value of the nodes in the first L layer depends on the last layer of the extraction block. For L layers, with "! > 1, the parameters associated with each node *«, p, i.e. the terms, b, hp and "p. p' defined in relation to expression (1) can be determined during training. Each multilayer perceptron 31, 32, 33 of each elementary neural network NN,, NN,, NN> preferably has the same number of layers and the same number of nodes per layer, At least one connection matrix W,_ ;, linking two successive layers of ranks ft - 1, A is different for each multilayer perceptron. The respective outputs OUT, OUT, OUT, OUT of each multilayer perceptron also constitute the respective outputs of each elementary neural network NN, NN, NN, NN. These can be combined in a combination layer 40 to form an output OUT of neural network 2. The output layer OUT thus corresponds to a combination of the outputs of each elementary neural network. The combination can be obtained by taking the mean or median of the outputs of the multilayer perceptrons, or by other types of combination, for example, majority voting, as described below. A characteristic of at least one multilayer perceptron, and preferably of every multilayer perceptron, is that between at least two successive layers of rank 7 - 1, more than 50%, or even more than 80%, and even more than 90% of the connection coefficients are zero. The connection between the two successive layers can be represented by a connection matrix Wn of dimension (P, 1, Pn), where each term WPn, pn, is the connection coefficient between the node of order p' of the layer of rank n-1 and the node of order P of the layer of rank n. Each term 'p'. p represents a contribution from node *n - 1, p' to node “x, p”. The connection matrix W,,_;, » is a sparse matrix, in the sense that more than 50%, or even more than 80%, or more than 90% or 95%, are zero. By zero, it is understood to be equal to zero where terms can be considered zero. Terms that can be considered zero are those whose value is at least 10 times smaller than at least one other term in the matrix.The terms of the connection matrix are positive or zero. At least one multilayer perceptron, and preferably each multilayer perceptron, is established such that at least one connection matrix W,_, , de- ending the connections between two successive layers, or even each connection matrix, which is a sparse matrix as defined in the previous paragraph. Each connection matrix can be defined, prior to the learning phase, by random sampling. Using a multilayer perceptron with at least one sparse connection matrix improves learning diversity when training is performed using a parallel multi-network approach. In such an approach, several elementary neural networks with identical structures but different connection matrices are implemented in parallel. This leads to improved analysis performance. The use of a sparse connection matrix W,,,, , between two consecutive layers (upstream layer L, — downstream layer L, - 1) is preferably accompanied by structuring the higher-rank layer (downstream layer) into groups, or clusters. Each cluster X,, 5 of a layer L, of rank ” is assigned an order of 4. Each cluster X,, y 'egroups several nodes *n. p of the downstream layer. The number of nodes per cluster can, for example, be between 2 and 10. The downstream layer L, of rank ” is thus segmented into different clusters, which do not overlap. Thus, preferably, a node *n, p can only belong to a single cluster X,, R. [Fig. 3A] illustrates the implementation principles of the measurement system described in connection with [Fig. 1]. During a step 100, the detector detects a physical quantity and generates an input signal, for example, an image.In general, the input signal can be expressed in vector form, matrix form, or as a set of multiple matrices. In step 110, the input signal is transmitted to the processing unit 12. In step 120, the input signal is used to train the neural network NN as input data. In step 130, the neural network NN is implemented, and the result of the analysis of the input signal is obtained at the output layer OUT. Advantageously, the structuring of the nodes of a layer into clusters is accompanied by a process of selecting one or more nodes from each layer segmented into clusters, as described below, in connection with Figures 3B to 3E. Figure 3B illustrates the selection process as it is performed for each cluster. Figure 3C shows a detail from Figure 2C, which shows the connections of the nodes of the first cluster X. In this example, ' = 2 and Ÿ = 1. Cluster X has three nodes *n, 1, *n, 2, and *n. The nodes are activated by nodes of the upstream layer L, of rank '-1. Each node *n, p of layer L receives an activation signal '4, p from the upstream layer, as described in connection with (1). When node *a, p is connected to a single node of the upstream layer, the SR signal corresponds to the The connection signal *n, p. p° generated by said node, of order p', of the upstream layer. When the node *"," p is connected to several nodes of the upstream layer, the signal *, p corresponds to the sum of the connection signals generated by said nodes of the upstream layer. During step 131, we take into account the activation signals of each node * », p of the cluster. Preferably, during step 132, the activation signals of each node are normalized so as to fall between a predetermined minimum and maximum value. Preferably, the signals are normalized so as to be homogeneous to a probability, falling between the minimum value 0 and the maximum value 1. The normalization can be performed according to: FE p Sp pl 5 y 5 Xn ap ! where Ë 5. _ corresponds to the sum of the activation signals of the nodes of the *ag np 4th-order cluster of layer ” and 5 * corresponds to the normalized activation signal of Each node “x, p of order P of the cluster A, , of order 4. Step 132 is optional. During a step 133, the cluster activation signals are processed by a selection function / . The selection function is preferably a continuous and monotonic function. The selection function is, or preferably includes, a sigmoid function, as shown schematically in Figure 3D, or a function approximating a sigmoid function. Alternatively, the selection function can be a piecewise linear function, as shown in Figure 3E. The selection function enhances the discrimination between the values ​​of the normalized activation signals s° »- The activation signals / (6. a) processed by the selection function are comparison signals, intended to be compared with each other, or with respect to a threshold. This is the purpose of a comparison step 134. During the comparison, according to one possibility, the highest comparison signal f(5p) is selected. According to another possibility, each comparison signal / (5.") exceeding a predetermined threshold value is selected. Following the comparison: the nodes *x, p of cluster X, "Corresponding to the comparison signals selected are activated. The nodes Un, pu cluster corresponding to the comparison signals the nodes x, p of the cluster X, 7 dant d Unselected items are disabled. Thus, during step 135, the nodes *n, p of the cluster x ap are activated or deactivated based on the comparison. Activated nodes receive the activation signals “a. p” from the upstream layer. They generate, if necessary, a connection signal for one or more nodes in the next layer. Deactivated nodes do not generate a connection signal for the next layer. If the layer containing the nodes is the last layer of the perceptron, the deactivated nodes are ignored for subsequent processing. Steps 131 to 135 are repeated for each cluster defined on at least one layer of the multilayer perceptron. Preferably, clusters on the same layer have the same number of nodes. The selection function / can have the following analytical form: (3) > .-84 (m] FlOmp) = 6) ad P4 We note that, following normalization, when the values ​​of the activation signals 5; x are between 0 and 1, the values ​​of the comparison signals f (5;") are also between 0 and 1. 8 is a threshold and T is a parameter designated by the term "temperature". The values ​​of 9 and T are predetermined or can self-adjust. When the values ​​of #(s, ) are between 0 and 1, the threshold is also between np 0 and 1. When 4 > 0.5, only one neuron can be activated following the selection process. The temperature T determines the slope of the selection function. The lower the value of T, the steeper the slope of the sigmoid, which increases the number of occurrences for which f(sp) = 0+ or ​​at (5") = |. The designations 0* and 1- denote, respectively, tending towards 0 when greater than 0, and tending towards | when less than 1. A low value of 7 makes the sigmoid function tend towards a square-type function. It is preferable for the function f to be differentiable and strictly increasing. Indeed, during the learning phase, gradient descent algorithms are used. Implementing this type of algorithm can be difficult if the derivative of the function f is zero. According to one possibility, the function f has a sigmoid function, as explained in relation to (3), to which a linear term is added. An important aspect of the invention is the use of normalization of the 2π values ​​by the sum of the activation signals reaching the nodes of the cluster under consideration. Thus, The values ​​$, » can be considered probabilities. This allows for values ​​extending within a controlled interval, which is more appropriate prior to applying a sigmoid function. This probabilistic approach, coupled with a selection process performed in parallel on several small clusters of the same layer, is particularly interesting when considering a small number of activation signals, induced by a sparse connection matrix. It leverages a diversity principle, which consists of multiplying the competitions between small subsets of signals. Generally, the size of each cluster is between 2 and 10 nodes. The number of nodes selected in a cluster is 1 or 2 per cluster, representing approximately 10% to 50% of the nodes forming the cluster. As previously described, analysis algorithm 2 is designed to process input data resulting from the detection of a physical quantity, forming a vector, a matrix, or multiple matrices. Since each elementary neural network includes a convolutional layer extraction block, the data forming the input layer of the multilayer perceptron are positive, which facilitates the normalization described earlier. The fact that the values ​​are positive avoids the need for a complex function such as a normalized exponential function (usually referred to as softmax), and the associated memory cost. Preferably, the selection process as previously described is implemented on several layers of each multilayer perceptron arranged downstream of the first Ly layer. Using elementary neural networks, each containing a multilayer perceptron as previously described, arranged in parallel, allows us to leverage a certain diversity due to the structure of each perceptron: their structure is similar, differing by at least one connection matrix. This increases the robustness of the analysis performed by the analytical neural network, formed by combining the different multilayer perceptrons and combining the outputs from the multilayer perceptrons. Using sparse matrices makes it easy to achieve high diversity, especially when the connection matrices are defined by random sampling. The resulting connection matrices are highly uncorrelated with each other. It should be noted that using a neural network with one or more sparse connection matrices is not straightforward. Indeed, it is generally accepted that the performance of a neural network decreases when there are few connections between two successive layers. However, a key element of the invention is that the use of sparse matrices, combined with selection in Each cluster, as previously described, and / or a parallel arrangement of several identical elementary neural networks, up to the connection matrices, makes it possible to constitute a high-performing NN neural network, given the diversity of learning conferred by the connection matrices. The inventors tested an analysis method as described in relation to Figures 2A to 2C. The input data consisted of images from the MNIST database, selected from only 10% of them. The MNIST database is known to those skilled in the art. It contains 60,000 representative samples of ten digits ranging from 0 to 9. Each image represents a character encoded as 28 x 28 pixels, or 784 pixels. The input data IN of the neural network was a vector formed by these 784 pixels. In each elementary neural network, an extraction block was used, consisting of two convolutional layers: a first convolutional layer with 64 convolutional filters of size 5 x 5, and a second convolutional layer with 128 convolutional filters of size 5 x 5. The output of the extraction block was a feature vector of dimension (1, 4608).As previously described, the neural network comprised three elementary neural networks operating in parallel. Each elementary neural network included an extraction block (convolution layers) 21, 22, 23, as described in the preceding paragraph, feeding a multilayer perceptron 31, 32, 33, respectively. Each multilayer perceptron comprised, in addition to the first layer L, (* = 1), three layers L₁, L₂, and L₄, containing 1500 nodes, 1200 nodes, and 504 nodes, respectively. Each of these layers was segmented into clusters of 3 nodes each. The selection process, as described in relation to Figure 3B, was implemented on each cluster of each layer. We used a sigmoid function as given in (3), to which we added a linear term with a slope of 0.05, in order to form a strictly increasing function and avoid an excessively small derivative. The values ​​were 0.64, 0.79 and 0 respectively.4 for layers L3, La, L4. The value of T was equal to 0.05 for each layer. Given the linear term, the selection function was: (4) (8 5) = ae ant | ep PP tra (FE The training of the three elementary neural networks was carried out using annotated images, with a number of epochs limited to 10. An epoch corresponds to a number of times when all the images are submitted to the network, in a different order at each epoch. The output layer of each elementary neural network was a code word binary of length 504, structured into 168 clusters. Each word resulting from the output layer corresponded to the identified character. A test to assess classification quality was performed by analyzing the classifications resulting from the three elementary neural networks implemented independently, as well as a classification obtained by combining, according to a majority vote, the classifications established by the three elementary neural networks. Majority vote refers to the classification result corresponding to the majority classification among those resulting from each elementary neural network. Classification quality is quantified by the percentage of correctly recognized characters. Using the elementary neural networks, without combining the results, yielded percentages of 99.15%, 98.95%, and 99.11%, respectively. Combining the results using the majority vote approach resulted in a percentage of 99.23%, which corresponds to an increase of 0.16% compared to the average percentages obtained with elementary neural networks used independently. Thus, paralleling elementary neural networks, each containing at least one connection matrix, is different and improves analysis performance. We will now describe another advantage of the previously described selection approach, combined with the parallelization of multilayer perceptrons with identical structures, differing by at least one connection matrix. Training a neural network requires annotated data, that is, input data whose output label is known. This is referred to as supervised learning. Implementing neural networks operating in parallel allows us to increase the amount of training data by using unannotated data. We thus move from supervised learning, in which all training data is annotated, to partially supervised learning, in which some of the data used for training is not pre-annotated. Figure 4A represents an algorithm architecture similar to the structure shown in Figure 2A. A first multilayer perceptron 31 is placed downstream of a convolutional extraction block 21. The combination of the extraction block 21 and the multilayer perceptron 31 forms a first neural network NNa. In the first multilayer perceptron 31, at least two successive layers (upstream layer — downstream layer) are connected by a sparse connection matrix, the downstream layer being segmented into clusters of nodes implementing a selection process as previously described. The first neural network NNa is trained with first annotated training data D1, but also with first unannotated training data d1. In this example, the training data, whether annotated (10% of the training data) or not (90% of the training data), are images extracted from the MNIST database. We also have a second multilayer perceptron 32, connected to a convolutional extraction block 22, whose structure is identical to the first multilayer perceptron 31. The combination of the extraction block 22 and the multilayer perceptron 32 forms a second neural network NN. The second multilayer perceptron 32 is similar to the first multilayer perceptron 31, except for at least one connection matrix. Thus, between at least two layers, the first and second multilayer perceptrons have at least one connection matrix W, which is different from each other. The second neural network NN is trained with second annotated training data D, but also with second unannotated training data d. The first annotated data D and the second annotated data D are preferably disjoint sets. The same applies to the first unannotated data d, and the second unannotated data d,. Figure 4B illustrates the main steps in a learning process for the two neural networks NNa and NNb. During steps 200 and 300, the first and second neural networks, NNa and NNb, are respectively trained using the first and second annotated data points D1 and D2. Following training: The first neural network NN performs an annotation of the first unannotated data d,, so as to obtain initial pseudo- data annotated D', : step 210; The second neural network NN performs an annotation of the second unannotated data d, so as to obtain pseudo-second data annotated D', : step 310. The annotations of the pseudo-annotated data D' and D'; are provisional. They are called pseudo-annotations because they are not final and can be adjusted during the iterative process described below. The first pseudo-annotated data D', are transmitted to the second neural network NN, : step 220. Symmetrically, the second pseudo-annotated data D', are transmitted to the first neural network NN,: step 320. During step 230, the first neural network NN is retrained using the first annotated data D, and the second pseudo-annotated data D'. Symmetrically, the second neural network NN is retrained using the second annotated data D, and the first pseudo-annotated data D'. Steps 210 to 230 and 310 to 330 can be repeated. At each iteration: during steps 210 and 310, the first neural network NN, and the second NN neural networks perform an update of the pseudo-annotations respectively, the first pseudo-annotated data and the second pseudo-annotated data; during steps 220 and 320, the first and second pseudo- data annotated, respectively updated during steps 210 and 310 of the same iteration, are respectively transmitted to the second neural network NN, and to the first NN neural network; During steps 230 and 330, the first and second neural networks are retrained, using respectively: the first annotated data D, and the second pseudo- data annotated D', ; the second annotated data D, and the first pseudo- data annotated D',. Steps 210 to 230 and 310 to 330 are repeated until stability is reached in the pseudo-annotations, for example, when a predefined proportion of pseudo-annotated data no longer changes its annotation after two successive iterations. The pseudo-annotations are then considered stable. In case of instability in an annotation, for example when after 20 or 30 iterations a pseudo-annotation has not been stabilized, an annotation by a human user may be requested. Figure 5 illustrates a variant of the architecture described in relation to Figure 4A. According to this variant: The first neural network NN comprises three initial networks of elementary neurons NN, ,, NN,,, NN, operating in parallel. Each The first elementary neural network includes an extraction block (21,, 21, 213) coupled to a multilayer perceptron (31, 31, 314). Each first elementary neural networks have the same structure: the same number of layers, same number of nodes per layer. At least one matrix of The connection between two layers of each multilayer perceptron is different in each first elementary neural network. The second neural network, NN, comprises several second networks of elementary neurons NN,,1, NN,,3, NN, 3 operating in parallel. Each The second elementary neural network includes an extraction block (22,, 222, 223) coupled to a multilayer perceptron (32, 322, 323). Each The second elementary neural network has the same structure: same Number of layers, same number of nodes per layer. At least one connection matrix between two layers of each multilayer perceptron is different in every other elementary neural network. The multilayer perceptrons of the first and second neural networks are connected to a first combination layer 41 and a second combination layer 42, respectively. Each combination layer is configured to combine the outputs from the multilayer perceptrons to which it is connected. The combination can be based on a mean or a median. The output of each combination layer constitutes the output of the neural network. The first and second neural networks NN, NN, can be implemented, for learning purposes, as described in connection with [Fig.4B]. The inventors implemented the process described in relation to [Fig. 4B] using an architecture as described in relation to [Fig. 5]. The first and second neural networks, NNa and NNb, were trained using annotated first data (5% of the database), unannotated first data, annotated second data (5% of the database), and unannotated second data. The number of pseudo-annotated first data points, randomly selected at each iteration, was equal to the number of annotated first data points. Similarly, the number of pseudo-annotated second data points, randomly selected at each iteration, was equal to the number of annotated second data points. The annotated and pseudo-annotated first data points after 20 iterations were used to train a neural network as described in [Fig. 5]. The digit recognition rate reached 99.4%. Thus, using parallel and cross-referencing neural networks, it is possible to annotate data, creating pseudo-annotated data from a small number of initially annotated data points. The ratio between initially annotated and pseudo-annotated data can typically be less than 10%. Performance can be improved by increasing the number of neural networks working in parallel. The architectures described in the example included 3 networks operating in parallel. It is possible to increase this number, and to use 4 or 5 parallel networks, or even more. The invention can be used for processing measured data, for example, image processing (e.g., image recognition) or sound processing (e.g., sound classification or speaker identification). In the case of sound processing, each input data point can be represented as an image resulting from the evolution of a frequency spectrum over time. Thus, the invention applies to any data that can be represented as a multidimensional vector.

Claims

Demands

1. A method for analyzing numerical data, in the form of a vector or a matrix, the numerical data resulting from a measurement performed by a detector, and forming an input to a neural network, the neural network comprising several elementary neural networks (NN1, NN2, NN3), each elementary neural network comprising a multilayer perceptron extending between: a first layer (L,, L>, L3); an output layer (OUT, OUT, OUT), the output layer including an elemental analysis result, by the network of elementary neurons, of digital data; each multilayer perceptron being such that: Each layer is assigned a rank ("), the rank of a layer being higher the closer the layer is to the layer exit; at least two layers of successive rows, forming a an upstream layer and a downstream layer, contain nodes, nodes of the two layers (L, L, -1) being connected, each node of the upstream layer (*- 1. »°) addressing a signal of connection (9-1. z', p) to at least one node of the downstream layer ( *n. p) to which it is connected, the connection of the nodes of upstream and downstream layers being defined by a matrix of connection (W,. ;, ), Each term of the matrix of connection (""", p') being associated with a node of the upstream layer and a node of the downstream layer, and quantifying a contribution from the node of the upstream layer to the node of the downstream layer; The process involves the following steps: a) use of the input data to feed the first layer of the multilayer perceptron of each elementary neural network; b) implementation of each neural network elements to obtain an elemental analysis result commentary by the output layer of the multi-perceptron layers of each elementary neural network; the process being characterized in that: at least one connection matrix (W,,. ; ,,) of the perceptron multilayered, each elementary neural network is a sparse matrix, containing at least 50% zero terms or whose value is at least ten times less than at least another term of the matrix; the multilayer perceptron of each neural network mentary comprises respectively, between two successive layers cessives of the same respective ranks, connection matrices different hollows.

2. A method according to claim !, wherein each neural network elementary includes an extraction block (21, 22, 23), programmed to extracting characteristics from the input data, the characteristics extracted in this way, forming the first layer of the multilayer perceptron (31, 32, 33) of said elementary neural network.

3. Method according to claim 2, wherein the extraction block consists of different successive convolution layers, each convolutional layer resulting from the application of a filter convolution to the previous convolution layer.

4. A method according to any one of the preceding claims, in each multilayer perceptron has the same number of layers and the same number of nodes per layer.

5. A method according to any one of the preceding claims, in each multilayer perceptron comprises at least one layer intermediate, between the first layer and the output layer.

6. Method according to any one of the preceding claims, in which the two successive layers of at least one mul- perceptron The diapers are such that: the nodes of the downstream layer (*n. p) are segmented into different groups (X,, 4): the process being such that step b) involves, for the mul- perceptron ticouches, the following sub-steps: bi) from the resulting connection signals (9n - 1, p. p) nodes of the upstream layer, connected to the same group of the downstream layer, selection of nodes, belonging to the audit group, at least one node of said group not being se- selected; b-ii) activation of the selected nodes, at least one node of the group not activated.

7. The method according to claim 6, wherein: at least one multilayer perceptron comprises several pairs consisting of two successive layers, each pair comprising an upstream layer and a downstream layer, a matrix connection being defined between the upstream layer and the layer downstream; each connection matrix (W,, ;) contains at least 50 % of zero terms or terms whose value is at least ten times in- less than at least one other term of said matrix; Substeps bi) and b-ii) are implemented for each pair of diapers.

8. A method according to any one of claims 6 or 7, wherein substep bi) includes, for each group of a downstream layer of minus a multilayer perceptron: calculation of an activation signal (9x. p) for each node (%…. p) of the group (X,. g), depending on connection signals ( Sn-1, p°, p) emitted, towards said node, by each node of the upstream layer connected to audit node; normalization of activation signals calculated for each node of the group so that the normalized signals 6") are between a minimum value (0) and a value maximum (1) predetermined; group node selection based on signals 'activation. î normalized activation (£n »}

9. A method according to claim 8, wherein substep bi) includes: taking into account a selection function (f), the function of selection being an increasing or strictly increasing; application of the selection function (f) to each signal standardized activation, so as to calculate for each signal normalized activation, a comparison signal (f (E 2} selection of nodes in the group based on the communication signal parison calculated for each node.

10. A method according to claim 9, wherein the selection function includes a sigmoid or piecewise linear function.

11. | Method according to any one of the preceding claims, in which at least one or each connection matrix of a perceptron multilayer contains at least 90% or at least 95% zero terms or at least 10 times lower than at least one other term of said matrix.

12. A method according to any one of the preceding claims, in which, for at least one multilayer perceptron, each signal of The connection between two nodes of two successive layers is positive.

13. A method according to any one of the preceding claims, including step c) combination of the results of analysis e- comments, resulting from each elementary neural network, for to form an analysis result.

14. A method according to claim 13, comprising, prior to the step has), measurement of a physical quantity (11) by a detector (10); formation, by the detector, of an input signal (S) from the measured physical quantity; formation of input data from the input signal; the result of the analysis being a characterization of the physical quantity measured.

15. A method according to claim 13, comprising, prior to the step has): taking into account data stored in a database data, the stored data forming the input signal; input data formation using the input signal; The analysis result being a characterization of the stored data.

16. Measurement system, comprising: a detector (10), configured to measure a quantity physics (11) and to establish an input signal (S) from the measured physical quantity; a processing unit (12), programmed to put into implements the process according to any one of claims 1 at 15.

17. A measuring system according to claim 16, wherein the detector is an image sensor or a sound sensor or a magnetic sensor.

18. Computer-readable data carrier, or configured to be connected to a computer, or integrated circuit, containing instructions to implement a process according to any of the claims Instructions 1 to 15.