Data processing system with network, method and computer program product thereof

Abstract

The present disclosure relates to a method, a computer program product, and a data processing system (100) having one or more system inputs (110a, 110b, ..., 110z) containing data to be processed and a system output (120). The system includes a network, NW, (130) having a plurality of nodes (130a, 130b, ..., 130x), each node configured to have a plurality of inputs (132a, 132b, ..., 132y), each node (130a, 130b, ..., 130x) having a weight (Wa, ..., Wy) for each input (132a, 132b, ..., 132y), each node configured to generate an output (134a, 134b, ..., 134x), and during a learning mode, the weight (Wa, ..., Wy) of the node (130a) is calculated. The system comprises an update unit (150) configured to update the weights (Wa, ..., Wy) of each node based on a correlation between each of its inputs (132a, ..., 132c) and a corresponding output (134a), and one or more processing units (140x) configured to receive the processing unit inputs and generate processing unit outputs, each processing unit (140x) providing the processing unit outputs as inputs (132b, 132e, ...) to one or more other nodes (130a, 130b, ...).

Description

[Technical field]

[0001] The present disclosure relates to a data processing system with a network, a method therefor, and a computer program product therefor. More specifically, the present disclosure relates to a data processing system with a network, a method, and a computer program product therefor as defined in the preambles of the independent claims. [Background technology]

[0002] Artificial intelligence (AI) is known. One example of AI is the artificial neural network (ANN). ANNs can suffer from rigid representations, which make the network appear to focus on limited features for discrimination. Such rigid representations can lead to inaccuracies in predictions. It can therefore be advantageous to create networks / data processing systems that do not rely on rigid representations, e.g., networks / data processing systems where inference is based on broad representations across all nodes / elements and / or where individual features are prevented from becoming too dominant, thereby providing more accurate predictions and / or more accurate data processing systems. A network where all nodes contribute to all representations is known as a dense coding network. So far, the implementation of dense coding networks has been hindered by the lack of rules for autonomous network formation, which makes it difficult to build functional networks with high capacity / diversity.

[0003] Thus, there may be a need for AI systems with improved processing power. Preferably, such AI systems provide or enable one or more of: improved performance, improved reliability, improved efficiency, faster learning, less use of computer power, less use of training data, less use of storage space, less complexity, and / or less use of energy.

[0004] SE 2051375 A1 alleviates some of the above issues, but there may still be a need for more efficient AI / data processing systems and / or alternative approaches. DISCLOSURE OF THEINVENTION [Problem to be solved by the invention]

[0005] [Means for solving the problem]

[0006] It is an object of the present disclosure to mitigate, alleviate or eliminate one or more of the above-mentioned deficiencies and disadvantages in the prior art, or at least to solve the above-mentioned problems.

[0007] According to a first aspect, a data processing system is provided. The data processing system is configured to have one or more system inputs including data to be processed and a system output. The data processing system comprises: a network, NW, comprising a plurality of nodes, each node configured to have a plurality of inputs, each node comprising a weight for each input, each node configured to generate an output; one or more update units configured to update the weights of each node based on a correlation between each input and a corresponding output of the node during a learning mode; and one or more processing units configured to receive a processing unit input and configured to generate a processing unit output by changing a sign of the received processing unit input. The system output comprises the output of each node. Furthermore, the nodes of a first group of the plurality of nodes are configured to excite one or more other nodes of the plurality of nodes by providing the respective outputs of the nodes of the first group of nodes as inputs to one or more other nodes. Further, the nodes of a second group of the plurality of nodes are configured to inhibit one or more other nodes of the plurality of nodes by providing the output of each of the nodes of the second group as a processing unit input to a respective processing unit, and each processing unit is configured to provide the processing unit output as an input to the one or more other nodes. Each node of the plurality of nodes belongs to one of the first and second groups of nodes.

[0008] According to some embodiments, the system input includes sensor data for multiple contexts / tasks.

[0009] According to some embodiments, the update unit comprises, for each weight, a probability value for increasing said weight, and during the learning mode, the data processing system is configured to limit the ability of a node to inhibit or excite the one or more other nodes by providing a first set point for the sum of all weights associated with the inputs to the one or more other nodes, comparing the first set point with the sum of all weights associated with the inputs to the one or more other nodes, decreasing the probability value associated with the weights associated with the inputs to the one or more other nodes if the first set point is smaller than the sum of all weights associated with the inputs to the one or more other nodes, and increasing the probability value associated with the weights associated with the inputs to the one or more other nodes if the first set point is larger than the sum of all weights associated with the inputs to the one or more other nodes, thereby improving the uniqueness of each node, improving / speeding up learning, and / or improving / increasing precision / accuracy.

[0010] According to some embodiments, during the learning mode, the data processing system is configured to limit the ability of system inputs to inhibit or excite one or more nodes by providing the first set point for the sum of all weights associated with the inputs to the one or more nodes, comparing the first set point to the sum of all weights associated with the inputs to the one or more nodes, decreasing the probability value associated with the weights associated with the inputs to the one or more nodes if the first set point is smaller than the sum of all weights associated with the inputs to the one or more nodes, and increasing the probability value associated with the weights associated with the inputs to the one or more nodes if the first set point is larger than the sum of all weights associated with the inputs to the one or more nodes, thereby improving / speeding up learning and / or increasing precision / accuracy.

[0011] According to some embodiments, each of the inputs to the one or more other nodes has a coordinate in network space, and the amount by which to decrease / increase the weight of the input to the one or more other nodes is based on the distance between the coordinates of the inputs associated with their weights in the network space.

[0012] According to some embodiments, the system is further configured to set the weight to zero if the weight does not increase for a pre-set period of time.

[0013] According to some embodiments, the system is further configured to increase the probability value of a weight having a zero value if the sum of all weights associated with the input to the one or more other nodes does not exceed a first set point for a pre-set period of time.

[0014] According to some embodiments, during the learning mode, the data processing system is configured to increase the relevance of the output of the node to the one or more other nodes by providing a first set point for the sum of all weights associated with the inputs to the one or more other nodes, comparing the first set point to the sum of all weights associated with the inputs to the one or more other nodes over a first time period, increasing the probability of modifying the weight of the input to the node if the first set point is smaller than the sum of all weights associated with the inputs to the one or more other nodes over the entire first time period, and decreasing the probability of modifying the weight of the input to the node if the first set point is larger than the sum of all weights associated with the inputs to the one or more other nodes over the entire first time period, thereby improving / speeding up learning and / or increasing precision / accuracy.

[0015] According to some embodiments, the update unit comprises, for each weight, a probability value for increasing said weight, and during said learning mode, the data processing system is configured to provide a second set point for the sum of all weights associated with the inputs to the node, calculate the sum of all weights associated with the inputs to the node, compare the calculated sum with the second set point, decrease the probability value associated with the weights associated with the inputs to the node if the calculated sum is greater than the second set point, and increase the probability value associated with the weights associated with the inputs to the node if the calculated sum is less than the second set point, thereby improving / speeding up learning and / or increasing precision / accuracy.

[0016] According to some embodiments, each node comprises multiple partitions, each partition is configured to have multiple partition inputs, each partition comprises a partition weight for each partition input, each partition is configured to generate a partition output, where each partition comprises an update unit configured to update the partition weight based on correlation during said learning mode, and the partition output of each partition is utilized to adjust the output of the node for which the partition is configured based on a transfer function, thereby making each single node more useful / powerful (e.g., with increased capacity), improved / faster learning, and / or improved / increased precision / accuracy.

[0017] According to some embodiments, during the learning mode, the data processing system is configured to: detect whether the network is sparsely connected by comparing a cumulative weight change of the one or more system inputs to a threshold over a second time period; and if the data processing system detects that the network is sparsely connected, increase an output of one or more of the plurality of nodes by adding a predetermined waveform to the output of the one or more of the plurality of nodes for a third time period. This results in a more efficient data processing system that can process a wider range of contexts / tasks per given amount of network resources, resulting in reduced power consumption.

[0018] According to some embodiments, each node comprises an update unit, each update unit configured to update the weights of the respective node based on a correlation between the respective inputs of the node and the output of the node, each update unit configured to apply a first function to the correlation if the associated node belongs to the first group of the plurality of nodes and a second function different from the first function to the correlation if the associated node belongs to the second group of the plurality of nodes, in order to update the weights during a learning mode. By updating the weights of the respective nodes based on a correlation between the respective inputs of the node and the output of the (same) node, and applying a first function to the correlation if the associated node belongs to the first group of the plurality of nodes and a second function different from the first function to the correlation if the associated node belongs to the second group of the plurality of nodes, in order to update the weights (during a learning mode), each node is more independent from other nodes and a higher accuracy is obtained (compared to prior art, e.g. backpropagation, etc.). Thus, there is a technical effect that a higher accuracy / precision is achieved / obtained.

[0019] According to some embodiments, the data processing system is configured to calculate population variances of the outputs of the nodes of the network after the weight updates have been performed, compare the calculated population variances with a power law, and minimize the error or mean square error between the population variances and the power law by adjusting the parameters of the network, so that each node becomes more independent from the other nodes (and a measure of how independent the nodes are from each other can be obtained). This results in a more efficient data processing system, capable of processing a wider range of contexts / tasks per given amount of network resources, resulting in reduced power consumption.

[0020] According to some embodiments, the data processing system is configured to learn to identify one or more entities from the sensor data during a learning mode and thereafter identify the one or more entities during a performance mode, the identified entities being one or more of a speaker, a spoken character, a syllable, a phoneme, a word, a phrase, an object or feature of an object present in the sensor data, a new touch event, an end of a touch event, a gesture, or an applied pressure present in the sensor data. In some embodiments, a higher precision / accuracy in identifying one or more entities or measurable characteristics thereof is achieved / obtained.

[0021] According to some embodiments, the network is a recurrent neural network.

[0022] According to some embodiments, the network is a recurrent neural network.

[0023] According to a second aspect, a computer-implemented or hardware-implemented method for processing data is provided, the method comprising: a) receiving one or more system inputs comprising data to be processed; b) providing a plurality of inputs to a network, NW, comprising a plurality of first nodes, at least one of the plurality of inputs being a system input; c) receiving outputs from each first node; d) providing a system output comprising the outputs of each first node; e) exciting, by a node of a first group of the plurality of nodes, one or more other nodes of the plurality of nodes by providing, as inputs, the outputs of each of the nodes of the first group of nodes to one or more other nodes; f) exciting, by a node of a second group of the plurality of nodes, one or more other nodes of the plurality of nodes. and g) optionally updating weights based on the correlation by one or more update units; h) optionally repeating steps a) to g) until a learning criterion is met; and i) repeating steps a) to f) until a stopping criterion is met, wherein each node of the plurality of nodes belongs to one of the first and second groups of nodes.

[0024] According to some embodiments, the method further includes: initializing the weights by setting the weights to zero; and applying a predetermined waveform to the output of one or more nodes of the plurality of nodes for a third time period, wherein the third time period begins simultaneously with the start of the step of receiving one or more system inputs comprising data to be processed.

[0025] According to some embodiments, the method further includes: initializing the weights by randomly assigning values ​​between 0 and 1 to the weights; and applying a predetermined waveform to the outputs of one or more nodes of the plurality of nodes for a third time period.

[0026] According to a third aspect there is provided a computer program product comprising a non-transitory computer readable medium storing a computer program comprising program instructions, the computer program being loadable into a data processing unit and configured to cause, when the computer program is executed by the data processing unit, to perform a method according to the third aspect or any of the above embodiments.

[0027] The advantages and features of the second and third aspects are largely similar to those described above in relation to the first aspect, and vice versa. The embodiments described in relation to the first aspect generally apply to the second and third aspects, and vice versa.

[0028] An advantage of some embodiments is that they process data / information more efficiently, especially during learning / training mode.

[0029] A further advantage of some embodiments is that a more efficient network is provided, eg, maximizing utilization of available network capacity, resulting in a more efficient data processing system.

[0030] Another advantage of some embodiments is that the system / network may be less complex, e.g., having a smaller number of nodes (at the same precision and / or for the same context / input range).

[0031] Yet another advantage of some embodiments is that data may be utilized more efficiently.

[0032] A further advantage of some embodiments is that utilization of available network capacity is improved (eg, maximized), resulting in a more efficient data processing system.

[0033] A further advantage of some embodiments is that the system / network becomes more efficient and / or training / learning becomes shorter / faster.

[0034] Another advantage of some embodiments is that they provide a less complex network.

[0035] A further advantage of some embodiments is improved / enhanced generalization (eg, across different tasks / contexts).

[0036] A further advantage of some embodiments is that the system / network is less susceptible to noise.

[0037] Other advantages of some embodiments are improved performance, improved reliability, improved accuracy, improved efficiency (of training and / or performance), faster / shorter training / learning, less computer power required, less training data required, less storage capacity required, reduced complexity, and / or reduced energy consumption.

[0038] In some embodiments, each node becomes more independent from other nodes, which results in an increased total capacity for representing information in the data processing system (and thus, for example, more information can be represented in the data processing system or for identification of one or more entities / objects and / or one or more features of one or more objects), and thus, a higher accuracy (e.g., compared to prior art techniques such as backpropagation).

[0039] The present disclosure will become apparent from the detailed description set forth below. The detailed description and specific examples disclose preferred embodiments of the present disclosure for illustrative purposes only. Those skilled in the art will understand that, from the guidance of the detailed description, changes and modifications may be made within the scope of the present disclosure.

[0040] Thus, it is to be understood that the disclosure disclosed herein is not limited to the particular components of the disclosed apparatus or steps of the disclosed methods, as such apparatus and methods may vary. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting. It is to be noted that, as used in this specification and the appended claims, the articles "a," "an," "the," and "said" are intended to mean that there is one or more elements, unless the context clearly dictates otherwise. Thus, for example, reference to a "unit" or "the unit" may include a plurality of apparatuses, etc. Furthermore, words such as "comprising," "including," "containing," etc. do not exclude other elements or steps.

[0041] The above objects, as well as additional objects, features, and advantages of the present disclosure will be more fully understood by reference to the following illustrative and non-limiting detailed description of exemplary embodiments thereof, when considered in conjunction with the accompanying drawings. [Brief description of the drawings]

[0042] [Figure 1] FIG. 1 is a schematic block diagram illustrating a data processing system according to some embodiments. [Diagram 2] FIG. 2 is a schematic block diagram illustrating a data processing system according to some embodiments. [Diagram 3] FIG. 3 is a flow chart illustrating method steps according to some embodiments. [Figure 4] FIG. 4 is a schematic diagram illustrating an example computer-readable medium according to some embodiments. [Diagram 5] FIG. 5 is a schematic diagram illustrating an update unit according to some embodiments. [Figure 6] FIG. 6 is a schematic diagram illustrating a compartment according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0043] The present disclosure will now be described with reference to the accompanying drawings, in which preferred exemplary embodiments of the present disclosure are shown. However, the present disclosure may be embodied in other forms and should not be construed as being limited to the embodiments disclosed herein. The disclosed embodiments are provided to fully convey the scope of the present disclosure to those skilled in the art.

[0044] Glossary In the following, reference will be made to a "node." The term "node" may refer to a neuron, such as a neuron of an artificial neural network, another processing element, such as a processor of a network of processing elements, or a combination thereof. Thus, the term "network" (NW) may refer to an artificial neural network, a network of processing elements, or a combination thereof.

[0045] In the following, reference is made to a "processing unit." A processing unit may be referred to as a synapse, e.g., an input unit (having a processing unit) of a node. However, in some embodiments, the processing unit is a (non-synaptic) (general purpose) processing unit associated with (connected, connectable, or composed of) a node of the NW, or a (general purpose) processing unit located between two different nodes of the NW.

[0046] In the following, reference is made to "context." A context is a situation or state of affairs. Context relates to what type of (input) data is expected, for example when there are different types of tasks and each different task has its own context. As an example, if the system input is pixels from an image sensor and the image sensor is exposed to different lighting conditions, each different lighting condition may be a different context for an object such as a ball, a car, a tree, etc. imaged by the image sensor. As another example, if the system input is audio frequency bands from one or more microphones, each different speaker may be a different context for phonemes present in one or more of the audio frequency bands.

[0047] In the following, reference will be made to "measurable". The term "measurable" is to be interpreted as something that can be measured or detected, i.e., detectable. The terms "measure" and "sense" are to be interpreted as synonyms.

[0048] In the following, reference is made to an "entity". The term entity may be interpreted as a physical entity or as a more abstract entity such as a financial entity (e.g., one or more financial data sets). The term "physical entity" may be interpreted as an entity that has a physical existence, such as an object, a feature (of an object), a gesture, an applied pressure, a speaker, a spoken letter, a syllable, a phoneme, a word, a phrase, etc.

[0049] In the following, reference is made to an “update unit.” An update unit may be an update module or an update object.

[0050] In the following, an embodiment of the present disclosure is described, in which FIG. 1 is a schematic block diagram of a data processing system 100 according to some embodiments, and FIG. 2 is a schematic block diagram of a data processing system 100 according to some embodiments. In some embodiments, the data processing system 100 is a network (NW) or includes a NW. In some embodiments, the data processing system 100 is a deep neural network, a deep belief network, a deep reinforcement learning system, a recurrent neural network, or a convolutional neural network.

[0051] Data processing system 100 has or is configured to have one or more system input(s) 110a, 110b, ..., 110z. One or more system inputs 110a, 110b, ..., 110z include data to be processed. This data may be multi-dimensional. For example, multiple signals are provided in parallel. In some embodiments, system inputs 110a, 110b, ..., 110z comprise or consist of time-continuous data. In some embodiments, the data to be processed comprises data from sensors, such as image sensors, touch sensors, and / or sound sensors (e.g., microphones). Furthermore, in some embodiments, system inputs 110a, 110b, ..., 110z include sensor data for multiple contexts / tasks, for example, while data processing system 100 is in a learning mode and / or while data processing system 100 is in a performance mode. That is, in some embodiments, data processing system 100 may be in a performance mode and a learning mode at the same time.

[0052] Further, the data processing system 100 has or is configured to have a system output 120. The data processing system 100 comprises a network (NW) 130. The NW 130 comprises a plurality of nodes 130a, 130b, ..., 130x. Each node 130a, 130b, ..., 130x has or is configured to have a plurality of inputs 132a, 132b, ..., 132y. In some embodiments, at least one of the plurality of inputs 132a, 132b, ..., 132y is a system input 110a, 110b, ..., 110z. Further, in some embodiments, all of the system inputs 110a, 110b, ..., 110z are used as inputs 132a, 132b, ..., 132y to one or more of the nodes 130a, 130b, ..., 130x. Further, in some embodiments, each of the nodes 130a, 130b, ..., 130x has one or more system inputs 110a, 110b, ..., 110z as input(s) 132a, 132b, ..., 132y. Each node 130a, 130b, ..., 130x has or is provided with a weight Wa, Wb, ..., Wy for each input 132a, 132b, ..., 132y. That is, each input 132a, 132b, ..., 132y is associated with a respective weight Wa, Wb, ..., Wy. In some embodiments, each weight Wa, Wb, ..., Wy has a value in the range from 0 to 1. Furthermore, the NW 130 or each node thereof generates or is configured to generate outputs 134a, 134b, ..., 134x. In some embodiments, each node 130a, 130b, ..., 130x computes a combination, such as a (linear) sum, sum of squares, or average, of its inputs 132a, 132b, ..., 132y (to that node) multiplied by respective weights Wa, Wb, ..., Wy to generate an output 134a, 134b, ..., 134x.

[0053] The data processing system 100 comprises one or more update units 150 configured to update the weights Wa, ..., Wy of each node during the learning mode based on (in response to) the correlation of each respective input 132a, ..., 132c of a node (e.g., 130a) with the corresponding output (e.g., 134a), i.e. with the output of the same node (e.g., 130a). In some embodiments, no weights are updated during the performance mode. In one example, the update of the weights Wa, Wb, Wc is based on (according to) the correlation of each respective input 132a, ..., 132c to the node 130a with the combined activity of all inputs 132a, ..., 132c to that node 130a, i.e., the correlation of each respective input 132a, ..., 132c to the node 130a with the output 134a of that node 130a (as an example of the node 130a, but applicable to all other nodes 130b, ..., 130x). Thus, a correlation (value) between the first input 132a and the respective output 134a is calculated, a correlation (value) between the second input 132b and the respective output 134a is calculated, and a correlation (value) between the third input 132c and the respective output 134a is calculated. In some embodiments, these different calculated correlation (sequence) values ​​are compared with each other and the update of the weights is based on (according to) this comparison. In some embodiments, updating the weights Wa, ..., Wy of each node based on (according to) the correlation between each input (e.g., 132a, ..., 132c) of the node (e.g., 130a) and the corresponding output (e.g., 134a) includes evaluating each input (e.g., 132a, ..., 132c) of the node (e.g., 130a) based on (according to) a score function. The score function provides an indication of how useful each input (e.g., 132a, ..., 132c) of the node (e.g., 130a) is spatially, e.g., relative to the corresponding output (e.g., 134a) in comparison to other inputs (e.g., 132a, ..., 132c) to the node, and / or temporally, e.g., over the time that the data processing system (100) processes the input (e.g., 132a).As mentioned above, the update of the weights Wa, ..., Wy of each node is based on or according to the correlation of each respective input 132a, ..., 132c of a node (e.g., 130a) with the corresponding output (e.g., 134a), i.e., with (only) outputs of the same node. Thus, the update of the weights of each node is independent of the updates / learning of other nodes, i.e., each node has independent learning.

[0054] Further, the data processing system 100 comprises one or more processing units 140x configured to receive the processing unit inputs 142x and generate the processing unit outputs 144x by changing the sign of the received processing unit inputs 142x. In some embodiments, the sign of the received processing unit inputs 142x is changed by multiplying the processing unit inputs 142x by −1. However, in other embodiments, the sign of the received processing unit inputs 142x is changed by phase shifting the received processing unit inputs 142x by 180 degrees. Alternatively, the sign of the received processing unit inputs 142x is changed by inverting the sign, for example, from positive to negative or from negative to positive. The system output 120 includes the outputs 134a, 134b, ..., 134x of each node 130a, 130b, ..., 130x. In some embodiments, the system output 120 is an array of outputs 134a, 134b, ..., 134x. Additionally, in some embodiments, the system output 120 is utilized to identify one or more entities or a measurable characteristic (or multiple measurable characteristics) thereof while in the performance mode, for example from sensor data.

[0055] In some embodiments, the NW 130 is composed of only a first group 160 of nodes 130a, 130b, ..., 130x (as shown in FIG. 1). However, in some embodiments, the NW 130 is composed of a first group 160 of nodes 130a, 130b, ..., 130x and a second group 162 of nodes 130a, 130b, ..., 130x (as shown in FIG. 2). Each of the nodes (e.g., 130a, 130b) of the first group 160 of the plurality of nodes (i.e., an excitable node) is configured to excite one or more other nodes (e.g., 130x) of the plurality of nodes 130a, 130b, ..., 130x, e.g., all the other nodes 130b, ..., 130x, by (directly) supplying an output (e.g., 134a, 134b) of each of the nodes (e.g., 130a, 130b) of the first group 160 of the nodes as an input (132d, ..., 132y) to one or more other nodes (e.g., 130x), such as all the other nodes 130b, ..., 130x.

[0056] Further, the nodes (e.g., 130x) of the second group 162 of the plurality of nodes are configured to suppress one or more other nodes 130a, 130b, ..., 130x, such as all other nodes 130a, 130b, ..., of the plurality of nodes 130a, 130b, ..., 130x, by providing an output (e.g., 134x) of each node (e.g., 130x) of the second group 162 as a processing unit input 142x to a respective processing unit (e.g., 140x). Each processing unit (e.g., 140x) is configured to provide a processing unit output 144x as an input (e.g., 132b, 132e) to one or more other nodes (e.g., 130a, 130b). Each node of the plurality of nodes 130a, 130b, ..., 130x belongs to one of the first and second groups (160, 162) of nodes. Further, as indicated above, in some embodiments, all of the nodes 130a, 130b, ..., 130x belong to the first group of nodes 160. In some embodiments, each node 130a, 130b, ..., 130x is configured to inhibit or excite some / all of the other nodes 130b, ..., 130x of the plurality of nodes 130a, 130b, ..., 130x by multiplying the output 134a, 134b, ..., 134x (of each node 130a, 130b, ..., 130x) by -1 or providing it directly as an input 132d, ..., 132y to one or more of the other nodes 130b, ..., 130x. By configuring one group of nodes to inhibit other nodes and another group of nodes to excite other nodes, and performing updates based on (and therefore) correlations during the learning mode, a more efficient network may be provided, e.g., utilization of available network capacity may be maximized, and thus a more efficient data processing system may be provided.

[0057] In some embodiments, the update unit(s) 150 comprises, for each weight Wa, ..., Wy, a probability value Pa, ..., Py for increasing the weight (and possibly a probability value Pad, ..., Pyd for decreasing the weight, which in some embodiments is 1-Pa, ..., 1-Py, i.e. Pad=1-Pa, Pbd=1-Pb, etc.). In some embodiments, the update unit(s) 150 comprises a look-up table (LUT) for storing the probability values ​​Pa, ..., Py. During the learning mode, the data processing system 100 is configured to limit (control) the ability of a node (e.g., 130a) to inhibit or excite one or more other nodes (e.g., 130b, ..., 130x). It provides a first set point for the sum of all weights (e.g., Wd, Wy) associated with the inputs (e.g., 132d, 132y) to one or more other nodes (e.g., 130b, ..., 130x), compares the first set point to the sum of all weights (e.g., Wd, Wy) associated with the inputs (e.g., 132d, 132y) to the one or more other nodes (e.g., 130b, ..., 130x), and if the first set point is less than the sum of all weights (e.g., Wd, Wy) associated with the inputs (e.g., 132d, 132y) to the one or more other nodes (e.g., 130b, ..., 130x), selects one or more of the other nodes as the first set point. This is achieved by decreasing the probability values ​​(e.g., Pd, Py) associated with the weights (e.g., Wd, Wy) of the inputs (e.g., 132d, 132y) for a node (e.g., 130b, ..., 130x) and increasing the probability values ​​(e.g., Pd, Py) associated with the weights (e.g., Wd, Wy) of the inputs (e.g., 132d, 132y) for one or more other nodes (e.g., 130b, ..., 130x) if the first set point is greater than the sum of all weights (e.g., Wd, Wy) associated with the inputs (e.g., 132d, 132y) for one or more other nodes (e.g., 130b, ..., 130x).

[0058] Further, in some embodiments, the data processing system 100 is configured to limit (control) the ability of the system inputs (e.g., 110z) during the learning mode to inhibit or excite one or more nodes (e.g., 130b, 130x) by providing a first set point for the sum of all weights (e.g., Wg, Wx) associated with the inputs (e.g., 132g, 132x) to one or more nodes (e.g., 130b, 130x), comparing the first set point to the sum of all weights (e.g., Wg, Wx) associated with the inputs (e.g., 132g, 132x) to one or more nodes (e.g., 130b, 130x), and inhibiting one or more nodes (e.g., 130b, 130x) if the first set point is less than the sum of all weights (e.g., Wg, Wx) associated with the inputs (e.g., 132g, 132x) to one or more nodes (e.g., 130b, 130x). This is achieved by decreasing probability values ​​(e.g., Pg, Px) associated with weights (e.g., Wg, Wx) associated with inputs (e.g., 132g, 132x) for one or more nodes (e.g., 130b, 130x) and increasing probability values ​​(e.g., Pg, Px) associated with weights (e.g., Wg, Wx) associated with (associated with) inputs (e.g., 132g, 132x) for one or more nodes (e.g., 130b, 130x) if the first set point is greater than the sum of all weights (e.g., Wg, Wx) associated with inputs (e.g., 132g, 132x) for one or more nodes (e.g., 130b, 130x).

[0059] Further, in some embodiments, each of the inputs (e.g., 132d, 132y) for one or more other nodes (e.g., 130b, 130x) has a coordinate in network space, and the amount by which to decrease / increase the weight (e.g., Wd, Wy) of the input (e.g., 132d, 132y) for one or more other nodes (e.g., 130b, 130x) is determined based on (in response to) the distance between the coordinates of the inputs (e.g., 132d, 132y) associated with the weight (e.g., Wd, Wy) in network space. In these embodiments, the decrease / increase in the weight is based on (in response to) the probability (indicated by a probability value) of decreasing / increasing the weight, and based on the amount by which to decrease / increase the weight (which is calculated based on the distance in network space between the coordinates of the inputs).

[0060] In some embodiments, the data processing system 100 is (further) configured to set a weight Wa, ..., Wy (e.g., any of the one or more weights) to zero if the weight Wa, ..., Wy does not increase for a (first) preset time period. Furthermore, in some embodiments, the data processing system 100 is (further) configured to increase a probability value Pa, ..., Py of a weight Wa, ..., Wy having a zero value if the sum of all weights (e.g., Wd, Wy) associated with an input (e.g., 132d, 132y) for one or more other nodes (e.g., 130b, 130x) does not exceed a first set point for a (second) preset time period.

[0061] In some embodiments, the data processing system 100 is configured to increase the relevance of an output (e.g., 134a) of a node (e.g., 130a) to one or more other nodes (e.g., 130b, 130x) during the learning mode. It provides a first set point for the sum of all weights (e.g., Wd, Wy) associated with the inputs (e.g., 132d, 132y) for one or more other nodes (e.g., 130b, 130x), compares the first set point to the sum of all weights (e.g., Wd, Wy) associated with the inputs (e.g., 132d, 132y) for the one or more other nodes (e.g., 130b, 130x) over a first time period, and determines whether the first set point is less than the sum of all weights (e.g., Wd, Wy) associated with the inputs (e.g., 132d, 132y) for the entire first time period. This is achieved by increasing the probability of changing the weights (e.g., Wa, Wb, Wc) of the inputs (e.g., 132a, 132b, 132c) to the node (e.g., 130a) when the first set point is greater than the sum of all weights (e.g., Wd, Wy) associated with inputs (e.g., 132d, 132y) to one or more other nodes (e.g., 130b, 130x) over the entire first time period (and leaving the probability of changing weights unchanged in the rare case where the first set point is neither less than nor greater than the sum of all weights over the entire first time period).

[0062] Further, in some embodiments, the update unit(s) 150 comprises, for each weight Wa, ..., Wy, a probability value Pa, ..., Py for increasing the weight (and possibly a probability value Pad, ..., Pyd for decreasing the weight, which in some embodiments is 1-Pa, ..., 1-Py, i.e. Pad=1-Pa, Pbd=1-Pb, etc.). In these embodiments, during the learning mode, the data processing system 100 is configured to provide a second set point for the sum Wa, Wb, Wc of all weights associated with the inputs 132a, 132b, 132c for the node 130a, to calculate the sum Wa, Wb, Wc of all weights associated with the inputs 132a, 132b, 132c for the node 130a, to compare the calculated sum with the second set point, and to update the node 130a if the calculated sum is greater than the second set point. and configured to decrease the probability values ​​Pa, Pb, Pc associated with the weights Wa, Wb, Wc of the inputs 132a, 132b, 132c for node 130a if the calculated sum is less than a second set point, and configured to increase the probability values ​​Pa, Pb, Pc associated with the weights Wa, Wb, Wc of the inputs 132a, 132b, 132c for node 130a (as an example for node 130a, but also applicable to all other nodes 130b, ..., 130x).

[0063] Further, in some embodiments, during the learning mode, the data processing system 100 is configured to detect whether the network 130 is sparsely connected (loosely coupled) by comparing a cumulative weight change of one or more system inputs 110a, 110b, ..., 110z to a threshold value over a second time period. The cumulative weight change is the change in weights Wa, Wf, Wg, Wx associated with one or more system inputs 110a, 110b, ..., 110z over the second time period. The second time period may be a predetermined period. If the cumulative weight change is greater than the threshold value, the network 130 is determined to be sparsely connected. Furthermore, the data processing system 100 is configured to increase the output 134a, 134b, ..., 134x of one or more nodes of the plurality of nodes 130a, 130b, ..., 130x by applying a predetermined waveform to the output 134a, 134b, ..., 134x of one or more nodes of the plurality of nodes 130a, 130b, ..., 130x during a third time period if the data processing system 100 detects that the network 130 is loosely coupled. The third time period may be a predetermined period. By applying a predetermined waveform to the output 134a, 134b, ..., 134x of one or more nodes of the plurality of nodes 130a, 130b, ..., 130x during the third time period, the plurality of nodes may be better grouped together.

[0064] Furthermore, in some embodiments, each node comprises an update unit 150. Each update unit 150 is configured to update the weights Wa, Wb, Wc of the respective node 130a based on (according to) the correlation between each respective input 132a, ..., 132c of the node 130a and the output 134a of said node 130a. Furthermore, in order to update the weights Wa, Wb, Wc during the learning mode, each update unit 150 is configured to apply a first function to the correlation if the associated node belongs to a first group 160 of the plurality of nodes and to apply a second function, different from the first function, to the correlation if the associated node belongs to a second group 162 of the plurality of nodes (as an example for node 130a, but also applicable to all other nodes 130b, ..., 130x). In some embodiments, the first (learning) function is a function such that if the input, i.e., correlation (value), increases, the output, i.e., change in weight (value), increases exponentially, and vice versa (decreasing input gives exponentially decreasing output). In some embodiments, the second (learning) function is a function such that if the input, i.e., correlation (value), increases, the output, i.e., change in weight (value), decreases exponentially, and vice versa (decreasing input gives exponentially increasing output).

[0065] In some embodiments, the data processing system 100 is configured to calculate the population variance of the outputs 134a, 134b, ..., 134x of the nodes 130a, 130b, ..., 130x of the network after the updates of the weights Wa, ..., Wy have been performed, compare the calculated population variance with a power law, and adjust the parameters of the network to minimize an error, such as the mean absolute error or the mean square error, between the population and the power law. Thus, the population variance of the outputs 134a, 134b, ..., 134x of the nodes 130a, 130b, ..., 130x of the network may be distributed close to a power law, thereby achieving optimal resource utilization and / or allowing all nodes to contribute optimally, thus providing a more efficient utilization of the data. The power law may, for example, be based on (according to) the logarithm of the amount of variance explained relative to the logarithm of the number of components obtained from a principal component analysis. In another example, the power law is based on a principal component analysis of a time-limited vector of activity / output across all neurons, where each principal component number on the horizontal axis is replaced by a node number. The input data to which the system is exposed is assumed to have a greater number of principal components than the number of nodes. In such a case, following the power law potentially expands the maximum capacity of the system with each additional node added to the system. Examples of network parameters (which can be adjusted) include: the type of scaling of the learning (how the weights are configured, the range of weights, etc.), the induced change in synaptic weights when updated (e.g., exponential, linear), the amount of gain in learning, one or more time constants for the state memory of the node or each node, a particular learning function (e.g., first and / or second functions), the transfer function of each node, the total capacity of the connections between the nodes and the sensors, and the total capacity of the nodes across all nodes.

[0066] Further, in some embodiments, the data processing system 100 is configured to learn to identify one or more (unidentified) entities or their (unidentified) measurable characteristics (or measurable characteristics) from the sensor data during a learning mode, and then configured to identify one or more entities or their measurable characteristics (or measurable characteristics) from the sensor data during a performance mode, for example. In some embodiments, the identified entity is one or more of a speaker, spoken characters, syllables, phonemes, words, or phrases present in the (speech) sensor data. Alternatively or additionally, the identified entity is one or more objects, or one or more features (e.g., pixels) of an object present in the sensor data. As another alternative or additionally, the identified entity is a new contact event, an end of a contact event, a gesture, or an applied pressure present in the (touch) sensor data. In some embodiments, all sensor data is a particular type of sensor data, such as audio sensor data, image sensor data, or touch sensor data, while in other embodiments, the sensor data is a mixture of different types of sensor data, such as audio sensor data, image sensor data, and touch sensor data, i.e., the sensor data is composed of different modalities. In some embodiments, the data processing system 100 is configured to learn to identify a measurable characteristic (or measurable characteristics) of an entity from the sensor data. The measurable characteristic may be a feature of an object, a part of a feature, a time-evolving trajectory of a position, a trajectory of applied pressure, or a frequency signature or a time-evolving frequency signature of a particular speaker when speaking a particular letter, syllable, phoneme, word, or phrase. Such measurable characteristics may be mapped to an entity.For example, object features may be mapped to objects, parts of features may be mapped to features (of objects), trajectories of positions may be mapped to gestures, trajectories of applied pressure may be mapped to (maximum) applied pressure, the frequency signature of a particular speaker may be mapped to that speaker, and spoken letters, syllables, phonemes, words or phrases may be mapped to actual letters, syllables, phonemes, words or phrases. Such mapping may simply be a lookup in a memory, a lookup table, or a database. The lookup may be based on (according to) finding among a number of physical entities an entity whose characteristics are closest to the identified measurable characteristics. From such a lookup, an actual entity may be identified. Furthermore, the data processing system 100 may be part of a warehouse, e.g. a fully automated warehouse (machine), connected to robotics, e.g. robot actuators (or robotics control circuits) via middleware (to connect the data processing system 100 to the actuators), or may be utilized in a system with low-complexity event-based cameras, whereby trigger data from the event-based cameras may be directly fed / transmitted to the data processing system 100.

[0067] FIG. 3 is a flow chart illustrating exemplary method steps according to some embodiments. FIG. 3 illustrates a computer-implemented or hardware-implemented method 300 for processing data. The method can be implemented in analog hardware / electronic circuits, digital circuits, such as gates and flip-flops, mixed-signal circuits, software, and any combination thereof. In some embodiments, the method 300 includes entering a learning mode. Alternatively, the method 300 includes providing an already trained data processing system 100. In this case, steps 370 and 380 (steps g and h) are not performed. The method 300 includes receiving 310 one or more system inputs 110a, 110b, ..., 110z including data to be processed. The method 300 further includes providing 320 a plurality of inputs 132a, 132b, ..., 132y, at least one of the plurality of inputs 132a, 132b, ..., 132y being a system input, to a network, NW, 130 comprising a plurality of first nodes 130a, 130b, ..., 130x. The method 300 further includes receiving 330 outputs 134a, 134b, ..., 134x from each of the first nodes 130a, 130b, ..., 130x. The method 300 further includes providing 340 a system output 120 comprising outputs 134a, 134b, ..., 134x of each of the first nodes 130a, 130b, ..., 130x. Further, the method 300 includes exciting 350 one or more other nodes ..., 130x of the plurality of nodes 130a, 130b, ..., 130x by the nodes 130a, 130b of the first group 160 of the plurality of nodes providing outputs 134a, 134b of each of the nodes 130a, 130b of the first group 160 of the nodes as inputs 132d, ..., 132y to the one or more other nodes ..., 130x.Furthermore, the method 300 includes suppressing 360, by the nodes 130x of the second group 162 of the plurality of nodes, one or more other nodes 130a, 130b, ..., of the plurality of nodes 130a, 130b, ..., by providing an output 134x of each of the nodes 130x of the second group 162 as a processing unit input 142x to a respective processing unit 140x. Each processing unit 140x is configured to provide a processing unit output 144x as an input 132b, 132e, ... to one or more other nodes 130a, 130b, .... The method 300 includes updating 370, by the one or more update units 150, the weights Wa, ..., Wy based on (according to) the correlations (as described during the learning mode and in relation to Figures 1 and 2 above). Further, the method 300 includes repeating 380 steps 310, 320, 330, 340, 350, 360 and 370 (discussed above) until a learning criterion is met (during the learning mode) (thus terminating the learning mode when the learning criterion is met). In some embodiments, the learning criterion is that the data processing system 100 is fully trained. In some embodiments, the learning criterion is that the weights Wa, Wb, ..., Wy converge and / or that the overall error is below an error threshold. In some embodiments, the method 300 includes entering a performance / identification mode. Further, the method 300 includes repeating 390 steps 310, 320, 330, 340, 350 and 360 (discussed above) (during the performance / identification mode) until a stopping criterion is met (thus terminating the performance / identification mode when the stopping criterion is met). The stopping criterion / condition may be that all data to be processed has been processed or that a certain amount of data / number of loops has been processed / performed. Alternatively, the stopping criterion is that the entire data processing system 100 is turned off. As another alternative, the stopping criterion is that the data processing system 100 (or a user of the system 100) discovers that the data processing system 100 needs further training.In this case, the data processing system 100 enters / re-enters a learning mode (and performs steps 310, 320, 330, 340, 350, 360, 370, 380 and 390). Each node of the plurality of nodes 130a, 130b, ..., 130x belongs to one of the first and second groups 160, 162 of nodes.

[0068] In some embodiments, the method 300 includes initializing 304 the weights Wa, ..., Wy by setting the weights Wa, ..., Wy to zero. Alternatively, the method 300 includes initializing 306 the weights Wa, ..., Wy by randomly assigning values ​​between 0 and 1 to the weights Wa, ..., Wy. Furthermore, in some embodiments, the method 300 includes adding 308 a predetermined waveform to outputs 134a, 134b, ..., 134x of one or more of the plurality of nodes 130a, 130b, ..., 130x during a third time period. In some embodiments, the third time period begins simultaneously with receiving 310 one or more system inputs 110a, 110b, ..., 110z including data to be processed.

[0069] In some embodiments, the computer program product is comprised of a non-transitory computer readable medium 400, such as, for example, a Universal Serial Bus (USB) memory, a plug-in card, an embedded drive, a digital versatile disk (DVD) or a read-only memory (ROM). FIG. 4 shows an example of a computer readable medium in the form of a compact disk (CD) ROM 400. The computer readable medium has stored thereon a computer program comprising program instructions. The computer program is loadable, for example, into a data processing processor (PROC) 420 configured in a computer or computing device 410. When loaded into the data processing device, the computer program may be stored in a memory (MEM) 430 associated with or configured in the data processing unit. According to some embodiments, the computer program, when loaded into the data processing unit and executed by the data processing unit, may cause the execution of the method steps, for example according to the method illustrated in FIG. 3 described herein. Furthermore, in some embodiments, a computer program product is provided, comprising instructions, when executed on at least one processor of the processing device, that cause the processing device to execute the method illustrated in FIG. 3. Further, in some embodiments, a non-transitory computer-readable storage medium is provided that stores one or more programs configured to be executed by one or more processors of a processing device, the one or more programs including instructions that, when executed by the processing device, cause the processing device to perform the method illustrated in FIG.

[0070] Fig. 5 shows an update unit according to some embodiments. An update unit 150a is provided for node 130a. However, all update units 150, 150a (for all nodes) are the same or similar. The update unit 150a receives the inputs 132a, ..., 132c of each of the node 130a (all nodes in case of a central update unit 150). Furthermore, the update unit 150a receives the output 134a of the node 130a (all nodes in case of a central update unit 150). Furthermore, the update unit 150a comprises a correlator 152a. The correlator 152a calculates during a learning mode a correlation between each of the inputs 132a, ..., 132c of the node 130a and the corresponding output (134a), thereby generating a (series of) correlation values ​​for each of the inputs 132a, ..., 132c. In some embodiments, the different calculated (series of) correlation values ​​are compared with each other (to generate a correlation ratio value) and the weight update is based on (according to) this comparison. Furthermore, in some embodiments, the update unit 150a is configured to apply a first function 154 to the correlation (value, ratio value) if the node (130a) belongs to a first group 160 of the plurality of nodes and to apply a second function 156, different from the first function, to the correlation (value, ratio value) if the node (130a) belongs to a second group 162 of the plurality of nodes, in order to update the weights Wa, Wb, Wc during the learning mode. In some embodiments, the update unit 150a keeps track of whether the node belongs to the first or second group 160, 162 by utilizing a look-up table (LUT). Furthermore, in some embodiments, the update unit 150a comprises, for each weight Wa, Wb, Wc, a probability value Pa, Pb, Pc for increasing the weight. In some embodiments, the update unit 150a provides, for each weight Wa, Wb, Wc, a probability value Pad, Pbd, Pcd) for reducing the weight, which in some embodiments is 1-Pa, 1-Pb, 1-Pc, i.e., Pad=1-Pa, Pbd=1-Pb, Pcd=1-Pc.In some embodiments, the probability values ​​Pa, Pb, Pc, and optionally the probability values ​​Pad, Pbd, Pcd are configured in a memory unit 158a of the update unit 150a. In some embodiments, the memory unit 158a is a look-up table (LUT). In some embodiments, the update unit 150a applies one of the first and second functions and / or the probability values ​​Pa, Pb, Pc, and optionally the probability values ​​Pad, Pbd, Pcd to the calculated (series of) correlation values ​​(or generated correlation ratio values) to obtain an update signal 159, which it applies to the weights Wa, Wb, Wc, thereby updating the weights Wa, Wb, Wc. The function / structure of the update units of the other nodes 150b, ..., 150x is the same as that of the node 150a. Furthermore, in some embodiments, the central update unit 150 configures each of the update units for each of the nodes 130a, 130b, ..., 130x.

[0071] FIG. 6 illustrates a compartment according to some embodiments. In some embodiments, each node 130a, 130b, ..., 130x comprises multiple compartments 900. Each compartment is configured to have multiple compartment inputs 910a, 910b, ..., 910x. Additionally, each compartment 900 comprises a compartment weight 920a, 920b, ..., 920x for each compartment input 910a, 910b, ..., 910x. Additionally, each compartment 900 is configured to generate a compartment output 940. The compartment output 940 is calculated by the compartment in some embodiments as a combination, such as the sum of all weighted compartment inputs 930a, 930b, ..., 930x for that compartment. To calculate the sum / combination, the compartment may comprise an adder (or adder / adder device) 935. Each partition 900 comprises an update unit 995 configured to update partition weights 920a, 920b, ..., 920x based on (in response to) correlations during a learning mode (in the same manner as described for update unit 150a above in relation to FIG. 5, etc.), which may include, for one or more partitions, evaluating each input of the node based on a score function. Furthermore, the partition output 940 of each partition is utilized to adjust the output 134a, 134b, ..., 134x (e.g., 134a) of the node 130a, 130b, ..., 130x (e.g., 130a) of which the partition 900 is configured based on (in accordance with) a transfer function. Examples of transfer functions that may be utilized are one or more of a time constant such as an RC filter, a resistor, a spike generator, and an active element such as a transistor or an operational amplifier. Partitions 900a, 900b, ..., 900x may comprise sub-partitions 900aa, 900ab, ..., 900ba, 900bb, ..., 900xx. Thus, each partition 900a, 900b, ..., 900x may have sub-partitions 900aa, 900ab, ..., 900ba, 900bb, etc. that function similarly to the partition, i.e., multiple partitions may be cascaded. The number of partitions (and sub-partitions) for a particular node is determined based on the type of input, such as inhibitory input, sensor input, excitatory input, etc., to that particular node.Additionally, the partitions 900 of the node are arranged such that each partition 900 has a majority of one of the types of inputs (e.g., inhibitory inputs, sensory inputs, excitatory inputs) such that no type of input (e.g., inhibitory inputs, sensory inputs, or excitatory inputs) becomes too dominant.

[0072] 6, in some embodiments, the update unit 995 of each partition 900 comprises, for each partition weight 920a, 920b, ..., 920x, a probability value PCa, ..., PCy for increasing the weight (and possibly a probability value PCad, ..., PCyd for decreasing the weight, which in some embodiments is 1-PCa, ..., 1-PCy, i.e. PCad=1-PCa, PCbd=1-PCb, etc.). In these embodiments, during the learning mode, the data processing system 100 is configured to provide a third set point for the sum of all partition weights 920a, 920b, ..., 920x associated with the partition inputs 910a, 910b, ..., 910x for the partition 900, to calculate a sum of all partition weights 920a, 920b, ..., 920x associated with the partition inputs 910a, 910b, ..., 910x for the partition 900, to compare the calculated sum to the third set point, and to calculate the third set point for the sum of all partition weights 920a, 920b, ..., 920x associated with the partition inputs 910a, 910b, ..., 910x for the partition 900. If the sum is greater than the third set point, it is configured to decrease the probability values ​​PCa, ..., PCy associated with (associated with) partition weights 920a, 920b, ..., 920x of partition inputs 910a, 910b, ..., 910x to partition 900, and if the calculated sum is less than the third set point, it is configured to increase the probability values ​​PCa, ..., PCy associated with (associated with) partition weights 920a, 920b, ..., 920x of partition inputs 910a, 910b, ..., 910x to partition 900. The third set point is based on (according to) a type of input, such as a system input (sensor input), an input from a node of the first group 160 of the plurality of nodes (excitatory input), or an input from a node of the second group 162 of the plurality of nodes (inhibitory input).

[0073] In some embodiments, the data processing system 100 is a time-continuous data processing system, i.e., all signals, including signals between different nodes within the data processing system 100, including one or more system inputs 110a, 110b, ..., 110z and the system output 120, are time-continuous (e.g., spike-free).

[0074] List of Examples Example 1: A data processing system (100) configured to have one or more system inputs (110a, 110b, ..., 110z) containing data to be processed and a system output (120), said data processing system comprising: a network, NW, (130) comprising a plurality of nodes (130a, 130b, ..., 130x), each configured to have a plurality of inputs (132a, 132b, ..., 132y), each node (130a, 130b, ..., 130x) comprising a weight (Wa, ..., Wy) for each input (132a, 132b, ..., 132y), each node configured to generate an output (134a, 134b, ..., 134x); one or more update units (150) configured to update the weights (Wa, ..., Wy) of each node based on a correlation between each of the inputs (132a, ..., 132c) of the node (130a) and the corresponding output (134a) during a learning mode; and one or more processing units (140x) configured to receive a processing unit input and configured to generate a processing unit output by changing a sign of the received processing unit input; wherein the system output (120) comprises the outputs (134a, 134b, ..., 134x) of each node (130a, 130b, ..., 130x); wherein the nodes (130a, 130b) of the first group (160) of nodes are configured to excite the one or more other nodes (..., 130x) of the plurality of nodes (130a, 130b, ..., 130x) by providing the outputs (134a, 134b) of each of the nodes (130a, 130b) of the first group (160) of nodes as inputs (132d, ..., 132y) to the one or more other nodes (..., 130x), wherein a node (130x) of a second group (162) of the plurality of nodes is configured to inhibit one or more other nodes (130a, 130b, ...) of the plurality of nodes (130a, 130b, ...) by providing the output (134x) of each of the nodes (130x) of the second group (162) as a processing unit input to a respective processing unit (140x), each processing unit (140x) being configured to provide the processing unit output as an input (132b, 132e, ...) to the one or more other nodes (130a, 130b, ...); A data processing system, wherein each node of said plurality of nodes (130a, 130b, ..., 130x) belongs to one of said first and second groups of nodes (160, 162).

[0075] Example 2: The data processing system of example 1, wherein the one or more system inputs include sensor data for multiple contexts / tasks.

[0076] Example 3: A data processing system according to Example 1 or 2, wherein the update unit 150 includes probability values (Pa,..., Py) for increasing the weights for each weight (Wa,..., Wy), and during the learning mode, the data processing system supplies a first set point for the sum of all weights (Wd, Wy) associated with the inputs (132d,..., 132y) for the one or more other nodes (130b,..., 130x), compares the first set point with the sum of all weights (Wd, Wy) associated with the inputs (132d,..., 132y) for the one or more other nodes (130b,..., 130x), and when the first set point is smaller than the sum of all weights (Wd, Wy) associated with the inputs (132d,..., 132y) for the one or more other nodes (130b,..., 130x), decreases the probability values (Pd, Py) associated with the weights (Wd, Wy) associated with the inputs (132d,..., 132y) for the one or more other nodes (130b,..., 130x), and when the first set point is larger than the sum of all weights (Wd, Wy) associated with the inputs (132d,..., 132y) for the one or more other nodes (130b,..., 130x), increases the probability values (Pd, Py) associated with the weights (Wd, Wy) associated with the inputs (132d,..., 132y) for the one or more other nodes (130b,..., 130x), thereby restricting the ability of the node (130a) and configured to suppress or excite the one or more other nodes (130b,..., 130x).

[0077] Example 4: The data processing system according to any of Examples 1 to 3, wherein during the learning mode, the data processing system provides the first set point for a sum of all weights (Wg, Wx) associated with the inputs (132g, 132x) to the one or more nodes (130a, ..., 130x), compares the first set point to the sum of all weights (Wg, Wx) associated with the inputs (132g, 132x) to the one or more nodes (130a, ..., 130x), and if the first set point is less than the sum of all weights (Wg, Wx) associated with the inputs (132g, 132x) to the one or more nodes (130a, ..., 130x), selects the one or more nodes (130a, ..., 130x) as the learning mode. 130x)と,if the first set point is greater than the sum of all weights (Wg, Wx) associated with the inputs (132g, 132x) to the one or more nodes (130a, ..., 130x), the first set point being greater than the sum of all weights (Wg, Wx) associated with the inputs (132g, 132x) to the one or more nodes (130a, ..., 130x).

[0078] Example 5: A data processing system as described in Example 3 or 4, wherein each of the inputs (132d, ..., 132y) for the one or more other nodes (130b, ..., 130x) has coordinates in a network space, and the amount by which the weights (Wd, Wy) of the inputs (132d, 132y) for the one or more other nodes (130b, ..., 130x) are decreased / increased is based on the distance between the coordinates of the inputs (132d, 132y) associated with the weights (Wd, Wy) in the network space.

[0079] Example 6: A data processing system as described in any of Examples 3 to 5, wherein the system is further configured to set the weights (Wa, ..., Wy) to zero if the weights (Wa, ..., Wy) do not increase for a preset period of time, and / or the system is further configured to increase the probability value (Pa, ..., Py) of the weights (Wa, ..., Wy) having a zero value if the sum of all weights (Wd, Wy) associated with the inputs (132d, ..., 132y) to the one or more other nodes (130b, ..., 130x) does not exceed the first set point for a preset period of time.

[0080] Example 7: The data processing system according to example 1 or 2, wherein during the learning mode, the data processing system provides a first set point for a sum of all weights (Wd, Wy) associated with the inputs (132d, ..., 132y) to the one or more other nodes (130b, ..., 130x), compares the first set point to the sum of all weights (Wd, Wy) associated with the inputs (132d, ..., 132y) to the one or more other nodes (130b, ..., 130x) over a first time period, and determines whether the first set point is greater than the sum of all weights (Wd, Wy) associated with the inputs (132d, ..., 132y) to the one or more other nodes (130b, ..., 130x) over the entire length of the first time period. and increasing the probability of changing the weights (Wa, Wb, Wc) of the inputs (132a, 132b, 132c) to the node (130a) when the first set point is smaller than the sum of all weights (Wd, Wy) associated with the inputs (132d, ..., 132y) to the one or more other nodes (130b, ..., 130x) over the first time period, by decreasing the probability of changing the weights (Wa, Wb, Wc) of the inputs (132a, 132b, 132c) to the node (130a).

[0081] Example 8: The data processing system according to example 1 or 2, wherein the update unit (150) comprises, for each weight (Wa, ..., Wy), a probability value (Pa, ..., Py) for increasing said weight, and during the learning mode, the data processing system provides a second set point for the sum of all weights (Wa, Wb, Wc) associated with the inputs (132a, 132b, 132c) to the node (130a), calculates the sum of all weights (Wa, Wb, Wc) associated with the inputs (132a, 132b, 132c) to the node (130a), and a second set point; and if the calculated sum is greater than the second set point, decreasing the probability values ​​(Pa, Pb, Pc) associated with the weights (Wa, Wb, Wc) associated with the inputs (132a, 132b, 132c) to the node (130a), and if the calculated sum is less than the second set point, increasing the probability values ​​(Pa, Pb, Pc) associated with the weights (Wa, Wb, Wc) associated with the inputs (132a, 132b, 132c) to the node (130a).

[0082] Example 9: The data processing system according to example 1 or 2, wherein each node (130a, 130b, ..., 130x) is configured to have a plurality of partitions (900), each partition having a plurality of partition inputs (910a, 910b, ..., 910x), each partition (900) having a partition weight (920a, 920b, ..., 920x) for each partition input (910a, 910b, ..., 910x), and each partition (900) generates a partition output (940). , wherein each partition (900) comprises an update unit (995) configured to update the partition weights (920a, 920b, ..., 920x) based on correlation during the learning mode, and the partition outputs (940) of each partition are utilized to adjust the outputs (134a, 134b, ..., 134x) of the nodes (130a, 130b, ..., 130x) on which the partition is configured based on a transfer function.

[0083] Example 10: The data processing system according to example 9, wherein the update unit (995) of each partition (900) comprises, for each partition weight (920a, 920b, ..., 920x), a probability value (PCa, ..., PCy) for increasing said weight, and during the learning mode, the data processing system is configured to provide a third set point for the sum of all partition weights (920a, 920b, ..., 920x) associated with the partition inputs (910a, 910b, ..., 910x) for the partition (900), to calculate the sum of partition weights (920a, 920b, ..., 920x) associated with the partition inputs (910a, 910b, ..., 910x) for the partition (900), to compare the calculated sum with the third set point, and ... the data processing system is configured to decrease the probability values ​​(PCa, ..., PCy) associated with the partition weights (920a, 920b, ..., 920x) associated with the partition inputs (910a, 910b, ..., 910x) for the partition (900) if the calculated sum is greater than a fixed point, and to increase the probability values ​​(PCa, ..., PCy) associated with the partition weights (920a, 920b, ..., 920x) associated with the partition inputs (910a, 910b, ..., 910x) for the partition (900) if the calculated sum is less than the third set point, the third set point being based on a type of input, such as a system input, an input from a node of the first group (160) of the plurality of nodes, or an input from a node of the second group (162) of the plurality of nodes.

[0084] Example 11: The data processing system of example 1 or 2, wherein during the learning mode, the data processing system is configured to include: detecting whether the network (130) is sparsely connected by comparing a cumulative weight change of the one or more system inputs (110a, 110b, ..., 110z) to a threshold over a second time period; and and increasing an output (134a, 134b, ..., 134x) of one or more nodes of the plurality of nodes (130a, 130b, ..., 130x) by applying a predetermined waveform to the output (134a, 134b, ..., 134x) of one or more nodes of the plurality of nodes (130a, 130b, ..., 130x) when the data processing system detects that the network (130) is sparsely connected for a third time period.

[0085] Example 12: A data processing system according to any one of Examples 1 to 11, wherein each node comprises an update unit (150), each update unit (150) configured to update the weights (Wa, Wb, Wc) of each node (130a) based on a correlation between each input (132a, ..., 132c) of the node (130a) and an output (134a) of the node (130a), wherein each update unit (150) is configured to apply a first function to the correlation if the associated node belongs to the first group (160) of the plurality of nodes, and to apply a second function, different from the first function, to the correlation if the associated node belongs to the second group (162) of the plurality of nodes, in order to update the weights (Wa, Wb, Wc) during a learning mode.

[0086] Example 13: A data processing system according to any one of Examples 1 to 12, wherein the data processing system is configured to calculate a population variance of the outputs (134a, 134b, ..., 134x) of the nodes (130a, 130b, ..., 130x) of the network (130) after the update of the weights (Wa, ..., Wy) is performed, compare the calculated population variance with a power law, and minimize the error or mean square error between the population variance and the power law by adjusting parameters of the network (130).

[0087] Example 14: A data processing system as described in any of Examples 2 to 13, wherein the data processing system is configured to learn to identify one or more entities from the sensor data during a learning mode, and thereafter identify the one or more entities during a performance mode, wherein the identified entities are one or more of a speaker, spoken characters, syllables, phonemes, words, phrases, features of an object or object present in the sensor data, a new contact event, an end of a contact event, a gesture, or an applied pressure present in the sensor data.

[0088] Example 15: A computer-implemented or hardware-implemented method for processing data, comprising: a) receiving (310) one or more system inputs (110a, 110b, ..., 110z) containing data to be processed; b) providing (320) a plurality of inputs (132a, 132b, ..., 132y) to a network, NW, (130) comprising a plurality of first nodes (130a, 130b, ..., 130x), at least one of the plurality of inputs being a system input; c) receiving (330) outputs (134a, 134b, ..., 134x) from each first node (130a, 130b, ..., 130x); d) providing (340) a system output (120) comprising said outputs (134a, 134b, ..., 134x) of each first node (130a, 130b, ..., 130x); e) exciting one or more other nodes (..., 130x) of the plurality of nodes (130a, 130b) by the nodes (130a, 130b) of the first group (160) of nodes, by providing the outputs (134a, 134b) of each of the nodes (130a, 130b) of the first group (160) of nodes as inputs (132d, ..., 132y) to one or more other nodes (..., 130x); f) suppressing (360) one or more other nodes (130a, 130b, ...) of the plurality of nodes (130a, 130b, ...) by a node (130x) of a second group (162) of the plurality of nodes (130x) by providing the output (134x) of each of the nodes (130x) of the second group (162) as a processing unit input to a respective processing unit (140x), each processing unit (140x) being configured to provide a processing unit output as an input (132b, 132e, ...) to the one or more other nodes (130a, 130b, ...); and g) updating (370) the weights (Wa,...,Wy) as necessary based on the correlations by one or more update units (150); h),repeat steps a) through g) as necessary until the learning criterion is met; i) repeating steps a) through f) until a stopping criterion is met, where: Each node of the plurality of nodes (130a, 130b, ..., 130x) belongs to one of the first and second groups of nodes (160, 162).

[0089] Example 16: The method of Example 15, further comprising: initializing (304) the weights (Wa,...,Wy) by setting them to zero; and applying (308) a predetermined waveform to the outputs (134a, 134b, ..., 134x) of one or more nodes of the plurality of nodes (130a, 130b, ..., 130x) for a third time period, wherein the third time period begins simultaneously with the start of a step of receiving (310) one or more system inputs (110a, 110b, ..., 110z) comprising data to be processed.

[0090] Example 17: The method of Example 15, further comprising: initializing 306 the weights (Wa,...,Wy) by randomly assigning values ​​between 0 and 1 to the weights (Wa,...,Wy); and applying (308) a predetermined waveform to the outputs (134a, 134b, ..., 134x) of one or more nodes of the plurality of nodes (130a, 130b, ..., 130x) during a third time period.

[0091] Example 18: A computer program product comprising a non-transitory computer-readable medium (400) storing a computer program including program instructions, the computer program being loadable into a data processing unit (420) and configured to cause the computer program to perform the method of any of Examples 15 to 17 when the computer program is executed by the data processing unit (420).

[0092] Those skilled in the art will understand that the present disclosure is not limited to the preferred embodiments described above. Those skilled in the art will further understand that modifications and variations are possible within the scope of the appended claims. For example, signals from other sensors, such as aroma sensors and flavor sensors, may be processed by the data processing system. Furthermore, the described data processing system can be similarly utilized for unsegmented and connected handwriting recognition, speech recognition, speaker recognition, anomaly detection in network traffic or intrusion detection systems (IDS). Furthermore, variations to the disclosed embodiments can be understood and effected by those skilled in the art practicing the claimed disclosure from a study of the drawings, the disclosure, and the appended claims.

Claims

1. A data processing system (100) configured to have one or more system inputs (110a, 110b, ..., 110z) containing data to be processed and a system output (120), said data processing system comprising: a network (130) comprising a plurality of nodes (130a, 130b, . . . , 130x), each configured to have a plurality of inputs (132a, 132b, . . . , 132y), each node (130a, 130b, . . . , 130x) having a weight (Wa, . . . , Wy) for each input (132a, 132b, . . . , 132y), and each node configured to generate an output (134a, 134b, . . . , 134x); and one or more processing units (140x) configured to receive a processing unit input and to generate a processing unit output by changing a sign of the received processing unit input; wherein the system output (120) comprises the outputs (134a, 134b, . . . , 134x) of each node (130a, 130b, . . . , 130x); wherein the nodes (130a, 130b) of the first group (160) of nodes are configured to excite one or more other nodes of the plurality of nodes (130a, 130b, ..., 130x) by providing the outputs (134a, 134b) of each of the nodes (130a, 130b) of the first group (160) of nodes as inputs (132d, ..., 132y) to one or more other nodes; wherein a node (130x) of a second group (162) of the plurality of nodes is configured to inhibit one or more other nodes of the plurality of nodes (130a, 130b, ..., 130x) by providing the output (134x) of each of the nodes (130x) of the second group (162) as a processing unit input to a respective processing unit (140x), and each processing unit (140x) is configured to provide the processing unit output as an input (132b, 132e, ...) to the one or more other nodes; wherein each node of said plurality of nodes (130a, 130b, . . . , 130x) belongs to one of said first and second groups of nodes (160, 162); Here, each node comprises an update unit (150), each update unit (150) having a respective input (132a) of said node (130a). , . . . , 132c) and the output (134a) of the node (130a), and wherein each update unit (150) is configured to apply a first function to the correlation if the associated node belongs to the first group (160) of the plurality of nodes, and to apply a second function, different from the first function, to the correlation if the associated node belongs to the second group (162) of the plurality of nodes, in order to update the weights (Wa, Wb, Wc) during a learning mode. A data processing system comprising:

2. 2. The data processing system of claim 1, the one or more system inputs include sensor data for multiple contexts / tasks; A data processing system comprising:

3. 3. A data processing system according to claim 1, said update unit (150) comprises, for each weight (Wa,...,Wy), a probability value (Pa,...,Py) for increasing said weight; During the learning mode, the data processing system provides a first set point for the sum of all weights (Wd, Wy) associated with the inputs (132d, . . . , 132y) for the one or more other nodes (130b, . . . , 130x), compares the first set point with the sum of all weights (Wd, Wy) associated with the inputs (132d, . . . , 132y) for the one or more other nodes (130b, . . . , 130x), and if the first set point is less than the sum of all weights (Wd, Wy) associated with the inputs (132d, . . . , 132y) for the one or more other nodes (130b, . . . , 130x), and limiting the ability of the node (130a) to inhibit or excite the one or more other nodes (130b,...,130x) by decreasing the probability values ​​(Pd,Py) associated with the weights (Wd,Wy) associated with the inputs (132d,...,132y) for the one or more other nodes (130b,...,130x) and increasing the probability values ​​(Pd,Py) associated with the weights (Wd,Wy) associated with the inputs (132d,...,132y) for the one or more other nodes (130b,...,130x) if the first set point is greater than the sum of all weights (Wd,Wy) associated with the inputs (132d,...,132y) for the one or more other nodes (130b,...,130x). A data processing system comprising:

4. 3. A data processing system according to claim 1, During the learning mode, the data processing system provides the first set point for the sum of all weights (Wg, Wx) associated with the inputs (132g, 132x) for the one or more nodes (130a, . . . , 130x), compares the first set point with the sum of all weights (Wg, Wx) associated with the inputs (132g, 132x) for the one or more nodes (130a, . . . , 130x), and if the first set point is less than the sum of all weights (Wg, Wx) associated with the inputs (132g, 132x) for the one or more nodes (130a, . . . , 130x), and limiting the ability of a system input (110z) to inhibit or excite one or more nodes (130a,...,130x) by decreasing the probability values ​​(Pg,Px) associated with the weights (Wg,Wx) associated with the inputs (132g,132x) for the one or more nodes (130a,...,130x) when the first set point is greater than the sum of all weights (Wg,Wx) associated with the inputs (132g,132x) for the one or more nodes (130a,...,130x). A data processing system comprising:

5. 4. The data processing system of claim 3, each of the inputs (132d,...,132y) to the one or more other nodes (130b,...,130x) having coordinates in network space; the amount by which the weights (Wd, Wy) of the inputs (132d, 132y) relative to the one or more other nodes (130b, . . . , 130x) are decreased / increased is based on the distance between the coordinates of the inputs (132d, 132y) associated with the weights (Wd, Wy) in the network space; A data processing system comprising:

6. 4. The data processing system of claim 3, the system is further configured to set the weights (Wa, . . . , Wy) to zero if the weights (Wa, . . . , Wy) do not increase for a preset period of time; and / or the system is further configured to increase the probability values ​​(Pa,...,Py) of the weights (Wa,...,Wy) having a zero value when the sum of all weights (Wd, Wy) associated with the inputs (132d,...,132y) for the one or more other nodes (130b,...,130x) does not exceed the first set point for a preset period of time. A data processing system comprising:

7. 3. A data processing system according to claim 1, During the learning mode, the data processing system provides a first set point for the sum of all weights (Wd, Wy) associated with the inputs (132d, ..., 132y) to the one or more other nodes (130b, ..., 130x), compares the first set point to the sum of all weights (Wd, Wy) associated with the inputs (132d, ..., 132y) to the one or more other nodes (130b, ..., 130x) over a first time period, and determines whether the first set point is greater than the sum of all weights (Wd, Wy) associated with the inputs (132d, ..., 132y) to the one or more other nodes (130b, ..., 130x) over the entire first time period. , 130x) is configured to increase the relevance of the output (134a) of the node (130a) to the one or more other nodes (130b,...,130x) by increasing the probability of changing the weights (Wd, Wy) of the inputs (132d,...,132y) to the node (130a) when the first set point is greater than the sum of all weights (Wd, Wy) associated with the inputs (132d,...,132y) to the one or more other nodes (130b,...,130x) over the first time period. A data processing system comprising:

8. 3. A data processing system according to claim 1, said update unit (150) comprises, for each weight (Wa,...,Wy), a probability value (Pa,...,Py) for increasing said weight; During the learning mode, the data processing system provides a second set point for the sum of all weights (Wa, Wb, Wc) associated with the inputs (132a, 132b, 132c) to the node (130a), calculates the sum of all weights (Wa, Wb, Wc) associated with the inputs (132a, 132b, 132c) to the node (130a), compares the calculated sum to the second set point, and determines whether the calculated sum is greater than the second set point. and increasing the probability values ​​(Pa, Pb, Pc) associated with the weights (Wa, Wb, Wc) associated with the inputs (132a, 132b, 132c) to the node (130a) if the calculated sum is less than the second set point. A data processing system comprising:

9. 3. A data processing system according to claim 1, Each node (130a, 130b, ..., 130x) is configured to have a plurality of partitions (900), each partition having a plurality of partition inputs (910a, 910b, ..., 910x), each partition (900) having a partition weight (920a, 920b, ..., 920x) for each partition input (910a, 910b, ..., 910x), and each partition (900) is configured to generate a partition output (940); wherein each partition (900) comprises an update unit (995) configured to update said partition weights (920a, 920b, . . . , 920x) based on correlation during said learning mode; The partition output (940) of each partition is used to adjust the outputs (134a, 134b, ..., 134x) of the nodes (130a, 130b, ..., 130x) that make up the partition based on a transfer function. A data processing system comprising:

10. 10. The data processing system of claim 9, The update unit (995) of each partition (900) calculates, for each partition weight (920a, 920b, . . . , 920x), a probability value (PCa , . . . , PCy), During the learning mode, the data processing system is configured to provide a third set point for the sum of all partition weights (920a, 920b, . . . , 920x) associated with the partition inputs (910a, 910b, . . . , 910x) for the partition (900), calculate the sum of partition weights (920a, 920b, . . . , 920x) associated with the partition inputs (910a, 910b, . . . , 910x) for the partition (900), compare the calculated sum with the third set point, and determine whether the calculated sum is greater than the third set point. configured to decrease the probability values ​​(PCa, . . . , PCy) associated with the partition weights (920a, 920b, . . . , 920x) associated with the partition inputs (910a, 910b, . . . , 910x) for the partition (900) if the calculated sum is greater than the third set point, and configured to increase the probability values ​​(PCa, . . . , PCy) associated with the partition weights (920a, 920b, . . . , 920x) associated with the partition inputs (910a, 910b, . . . , 910x) for the partition (900) if the calculated sum is less than the third set point; the third set point is based on a type of input, such as a system input, an input from a node in the first group (160) of the plurality of nodes, or an input from a node in the second group (162) of the plurality of nodes; A data processing system comprising:

11. 3. A data processing system according to claim 1 or 2, wherein during the learning mode the data processing system is configured to: detecting whether the network (130) is sparsely connected by comparing a cumulative weight change of the one or more system inputs (110a, 110b, . . . , 110z) to a threshold value over a second time period; and if the data processing system detects that the network (130) is sparsely connected, increasing the outputs (134a, 134b, ..., 134x) of one or more nodes of the plurality of nodes (130a, 130b, ..., 130x) by applying a predetermined waveform to the outputs (134a, 134b, ..., 134x) of one or more nodes of the plurality of nodes (130a, 130b, ..., 130x) for a third time period; A data processing system comprising:

12. 3. A data processing system according to claim 1, the data processing system is configured to calculate a population variance of the outputs (134a, 134b, ..., 134x) of the nodes (130a, 130b, ..., 130x) of the network (130) after the weights (Wa, ..., Wy) have been updated, compare the calculated population variance with a power law, and adjust parameters of the network (130) to minimize the error or mean square error between the population variance and the power law. A data processing system comprising:

13. 13. The data processing system of claim 12, wherein adjusting parameters of the network (130) comprises adjusting one or more of the following: the type of scaling of the learning, such as the range of the weights; Induced changes in synaptic weights during updates, such as exponential or linear; the amount of gain in said learning; one or more time constants for the state memory of each of said nodes; one or more learning functions, such as the first and second functions; Transfer function of each node; the total capacity of the connections between the nodes and the sensors; and The total capacity of all nodes, A data processing system comprising:

14. 3. The data processing system of claim 2, the data processing system is configured to learn to identify one or more entities from the sensor data during a learning mode, and thereafter identify the one or more entities during a performance mode; A data processing system comprising:

15. 15. The data processing system of claim 14, the identified entity is one or more of a speaker, a spoken character, a syllable, a phoneme, a word, or a phrase present in the sensor data; A data processing system comprising:

16. 15. The data processing system of claim 14, the identified entities are features of objects or objects present in the sensor data; A data processing system comprising:

17. 15. The data processing system of claim 14, the identified entity is a new touch event, an end of a touch event, a gesture, or an applied pressure present in the sensor data; A data processing system comprising:

18. 3. A data processing system according to claim 1, the network (130) is a recurrent neural network; A data processing system comprising:

19. 3. A data processing system according to claim 1, the network (130) is a recurrent neural network; A data processing system comprising:

20. A computer-implemented or hardware-implemented method for processing data, comprising: receiving (310) one or more system inputs (110a, 110b, . . . , 110z) containing data to be processed; providing (320) a plurality of inputs (132a, 132b, . . . , 132y) to a network (NW) (130) comprising a plurality of first nodes (130a, 130b, . . . , 130x), at least one of the plurality of inputs being a system input; receiving (330) outputs (134a, 134b, . . . , 134x) from each first node (130a, 130b, . . . , 130x); providing (340) a system output (120) including the outputs (134a, 134b, . . . , 134x) of each first node (130a, 130b, . . . , 130x); Exciting (350) one or more other nodes (...., 130x) of the plurality of nodes (130a, 130b,..., 130x) by the nodes (130a, 130b) of the first group (160) of nodes providing the outputs (134a, 134b) of each of the nodes (130a, 130b) of the first group (160) of nodes as inputs (132d,..., 132y) to one or more other nodes (...., 130x); suppressing (360) one or more other nodes (130a, 130b, ...) of the plurality of nodes (130a, 130b, ...) by a node (130x) of a second group (162) of the plurality of nodes by providing the output (134x) of each of the nodes (130x) of the second group (162) as a processing unit input to a respective processing unit (140x), each processing unit (140x) configured to provide a processing unit output as an input (132b, 132e, ...) to the one or more other nodes (130a, 130b, ...); and For each node, update (370) weights (Wa, Wb, Wc) based on a correlation between each input (132a, . . . , 132c) of the node (130a) and the output (134a) of the node (130a), and during the learning mode, apply a first function to the correlation if the associated node belongs to the first group (160) of the plurality of nodes, and apply a second function, different from the first function, to the correlation if the associated node belongs to the second group (162) of the plurality of nodes, wherein: each node of said plurality of nodes (130a, 130b, . . . , 130x) belongs to one of said first and second groups of nodes (160, 162); A method characterized by:

21. 21. The computer-implemented or hardware-implemented method of claim 20, further comprising: repeating (380) the steps of receiving one or more system inputs (310), providing a plurality of inputs (320), receiving an output (330), providing a system output (340), exciting (350), inhibiting (360), and updating (370) until a learning criterion is met; A method characterized by:

22. 22. The computer- or hardware-implemented method of claim 20 or 21, further comprising: repeating (390) the steps of receiving one or more system inputs (310), providing a plurality of inputs (320), receiving an output (330), providing a system output (340), exciting (350), and inhibiting (360) until a stopping criterion is met; A method characterized by:

23. 22. The method of claim 20 or 21, further comprising: Initializing the weights (Wa, . . . , Wy) by setting them to zero (304); and applying (308) a predetermined waveform to the outputs (134a, 134b, ..., 134x) of one or more nodes of the plurality of nodes (130a, 130b, ..., 130x) for a third time period, wherein the third time period begins simultaneously with the start of the step (310) of receiving one or more system inputs (110a, 110b, ..., 110z) comprising data to be processed; A method characterized by:

24. 22. The method of claim 20 or 21, further comprising: Initializing 306 the weights (Wa, . . . , Wy) by randomly assigning values ​​between 0 and 1 to the weights (Wa, . . . , Wy); and applying (308) a predetermined waveform to the outputs (134a, 134b, . . . , 134x) of one or more nodes of the plurality of nodes (130a, 130b, . . . , 130x) for a third time period; A method characterized by:

25. A computer program comprising instructions that, when executed on at least one processor of a processing device, cause said processing device to perform the method according to claim 20 or 21.

26. 22. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a processing device, the one or more programs comprising instructions that, when executed by the processing device, cause the processing device to perform the method of claim 20 or 21.

Images