Data processing system comprising a first and a second network, the second network connectable to the first network, method therefor, and computer program product therefor - Patents.com

Abstract

The present disclosure relates to a data processing system (100) comprising one or more system inputs containing data to be processed and a system output (120). The data processing system comprises a first network NW (130) having a plurality of inputs and configured to generate an output, and a second network NW (140) having as inputs outputs of one or more first nodes and configured to generate an output, where the system output (120) comprises the output of each first node, and an output (144a) of a second node (140a) of a first set (146) of nodes (140a) is used as an input to one or more processing units (136a3, 136b1), each processing unit (136a3, 136b1) configured to provide a negative or positive feedback to a respective first node.

Description

[Technical field]

[0001] The present disclosure relates to a data processing system comprising a first and a second network, a second network connectable to the first network, a method and a computer program product. More specifically, the present disclosure relates to a data processing system comprising a first and a second network, a second network connectable to the first network, a method and a computer program product as defined in the introduction of the independent claims. [Background technology]

[0002] Artificial intelligence (AI) is well known. However, today's AI models are typically trained to do only one thing. Therefore, AI systems are often trained from scratch for each new problem, in other words, from a baseline of zero knowledge. Moreover, learning each new task often takes a fairly long time. Moreover, training requires a large amount of training data, for example, because every new task is learned from scratch. Moreover, most of the current models only process information in one modality at a time. For example, they can take in text, images, and audio, but typically cannot take in all three at the same time. Moreover, most of the current models cannot handle data in abstract formats. Most of the current models also have a fairly high energy consumption.

[0003] Thus, there may be a need for an AI system that can handle many distinct tasks. Additionally, there may be a need for an AI system that leverages existing skills to learn new tasks faster and more effectively. Additionally, there may be a need for an AI system that requires only a limited amount of training data. Additionally, there may be a need for an AI system that allows for multimodal models that simultaneously encompass different modalities such as vision, hearing, and language understanding. Additionally, there may be a need for an AI system that performs new, more complex tasks. Additionally, there may be a need for an AI system that generalizes across tasks. Additionally, there may be a need for an AI system that handles data in more abstract forms. Additionally, there may be a need for an AI system that is sparse, efficient, and utilizes all relevant information, thereby enabling more energy-efficient data processing. Preferably, such an AI system provides or enables one or more of improved performance, improved reliability, improved efficiency, faster training, use of less computer power, use of less training data, use of less storage capacity, less complexity, and / or use of less energy.

[0004] Google Pathways (https: / / www.searchenginejournal.com / google-pathways-ai / 428864 / #close) mitigates the above issues to some extent, but there may be a need for more efficient AI systems and / or alternative approaches. DISCLOSURE OF THEINVENTION [Means for solving the problem]

[0005] The present disclosure seeks to mitigate, alleviate, or eliminate one or more of the above-mentioned deficiencies and drawbacks in the prior art, for example by at least attempting to address the above-mentioned problems and limitations of known AI systems. According to a first aspect, a data processing system is provided. The data processing system is configured to have one or more system inputs including data to be processed and a system output. The data processing system comprises a first network (NW) including a plurality of first nodes, each first node configured to have a plurality of inputs, at least one of the plurality of inputs being a system input, and configured to generate an output. Further, the data processing system comprises a second NW including a first and a second set of second nodes, each second node configured to have as inputs the outputs of one or more of the first nodes, and configured to generate an output. Further, the system output includes the output of each first node. The outputs of the second nodes of the first set of nodes are used as inputs to one or more processing units, each configured to provide negative feedback to the respective first node, and / or the outputs of the second nodes of the second set of nodes are used as inputs to one or more processing units, each configured to provide positive feedback to the respective first node. By providing negative and / or positive feedback from the nodes of the second network to the first network, the context / task at hand can be processed more accurately and / or efficiently by using only, or primarily, the nodes (of the first network) that are best suited for data processing for that particular context / task. Thus, a more efficient data processing system is achieved that can process a wider range of contexts / tasks per given amount of network resources, resulting in reduced power consumption.

[0006] According to some embodiments, each of the plurality of first nodes comprises a processing unit for each of the plurality of inputs, each processing unit comprising an amplifier and a leaky integrator having a time constant.

[0007] According to some embodiments, the time constant of the processing unit having as input the output of a node of the first or second set of nodes is larger than the time constant of the other processing units. By setting the time constant of the processing unit influenced by the node of the second network (node ​​of the first or second set of nodes) larger than the time constant of the other processing units, e.g., the processing units influenced by the system input, an increase / improvement in the dynamic performance of the data processing system, and therefore a higher reliability, is achieved, e.g., by providing a smoother transition from one context / task to another and / or by avoiding / reducing flip-flops / oscillations between a first processing mode associated with a first context / task and a second processing mode associated with a second (different) context / task.

[0008] According to some embodiments, while the data processing system is in the learning mode, the output of each node of the first and / or second set of nodes is suppressed.

[0009] According to some embodiments, each processing unit comprises a suppression unit configured to suppress an output of each node of the first and / or second set of nodes while the data processing system is in the learning mode.

[0010] According to some embodiments, each node of the first and second sets of nodes comprises an enable unit, each enable unit directly connected to an output of the respective node, and the enable unit(s) are configured to suppress the output while the data processing system is in the learning mode.

[0011] According to some embodiments, the data processing system comprises a comparison unit, the comparison unit being configured to compare the system output with an adaptive threshold while the data processing system is in the learning mode.

[0012] According to some embodiments, the output of each node in the first or second set of nodes is suppressed only if the system output is greater than an adaptive threshold.

[0013] According to some embodiments, the system input(s) include sensor data for multiple contexts / tasks.

[0014] According to some embodiments, the data processing system is configured to learn to identify one or more entities from the sensor data while in the learning mode, and thereafter to identify the one or more entities while in the performance mode.

[0015] According to some embodiments, the identified entity is one or more of a speaker, a spoken character, syllable, phoneme, word or phrase, or an object or feature of an object present in the sensor data, or a new touch event, an end of a touch event, a gesture or applied pressure present in the sensor data.

[0016] According to some embodiments, the data processing system is configured to learn to identify one or more (previously unidentified) entities or measurable characteristics (or measurable characteristics) thereof from the sensor data while in the learning mode, and then configured to identify one or more entities or measurable characteristics (or measurable characteristics) from newly acquired sensor data that is not included in the corpus of sensor data that the data processing system was originally trained on while in the performance mode. Such sensor data may include one or more types of fused sensor data, e.g., audio and visual data feeds from audio and image sensors may be fused. In some embodiments, this allows both visual and audio characteristics of, e.g., an image of a human entity speaking, to be used to identify the entity.

[0017] In some embodiments, depending on the level of metadata used for training in the training data, an entity may be identified in more than one way, for example, as a type of entity, a classification or category of entity, or an individual entity. In other words, an object may be recognized as a "car", or as a particular brand, color, or body style of car, or as an individual car with a particular registration number. An entity may be an object, or may be a living thing, such as a human or animal, or part thereof.

[0018] Applications of the data processing system include, but are not limited to, processing images of tissue samples to classify cells or microorganisms, determining drug and medication and dosages for individual patient treatment and / or therapy for personalized medication, etc. However, a wide range of other contexts of use of the data processing system are possible, and the present embodiments may be useful in fields as diverse as, for example, facial recognition and biometric security, dynamic spectrum allocation in wireless networks, and robotics.

[0019] According to some embodiments, each input of the second node is a weighted version of the output of one or more of the first nodes.

[0020] According to some embodiments, the training and / or updating of the weights of the first and / or second network during the training mode is based on correlation.

[0021] According to a second aspect, a second network, NW, is provided that is connectable to the first NW, the first NW including a plurality of first nodes, each first node configured to have a plurality of inputs and configured to generate an output. The second NW includes a first and a second set of second nodes, each second node configurable to have as inputs the outputs of one or more of the first nodes and configured to generate an output. The outputs of the nodes of the first node set are used as inputs to one or more processing units, each processing unit configured to provide negative feedback to a respective first node of the first NW, and / or the outputs of the nodes of the second node set are used as inputs to one or more processing units, each processing unit configured to provide positive feedback to a respective first node.

[0022] According to a third aspect, a computer-implemented or hardware-implemented method for processing data is provided. The method includes: receiving one or more system inputs including data to be processed; providing the multiple inputs to a first network, NW, comprising multiple first nodes, where at least one of the multiple inputs is a system input; receiving an output from each first node; providing a system output including an output of each first node; providing the output of each first node to a second NW comprising a first and second set of second nodes; and receiving an output of each second node. The method further includes: using an output of a second node of the first set of nodes as an input to one or more processing units, where each processing unit is configured to provide a negative feedback to the respective first node; and / or using an output of a second node of the second set of nodes as an input to one or more processing units, where each processing unit is configured to provide a positive feedback to the respective first node.

[0023] According to a fourth aspect there is provided a computer program product comprising a non-transitory computer readable medium having stored thereon a computer program comprising program instructions, the computer program being loadable into a data processing unit and configured to cause, when executed by the data processing unit, to perform the method of the third aspect or any of the embodiments described above.

[0024] The advantages and features of the second, third and fourth aspects are largely similar to those described above in relation to the first aspect and vice versa. The embodiments described in relation to the first aspect generally apply to the second, third and fourth aspects and vice versa.

[0025] An advantage of some embodiments is that they process data / information more efficiently, especially during the learning / training mode: for example, training from one training context, in other words training on one data corpus, can be more or less transferred to other new training contexts, so that the training phase for the new training context can be significantly reduced and / or a smaller corpus of learning data can be utilized than might otherwise be required.

[0026] Another advantage of some embodiments is that the system / network may be less complex, e.g., having a smaller number of nodes (at the same precision and / or for the same context / input range).

[0027] Yet another advantage of some embodiments is that data may be utilized more efficiently.

[0028] A further advantage of some embodiments is that the system / network can handle a larger / wider input range and / or a larger context range (with the same size of the system / network, e.g., the same number of nodes, and / or the same precision).

[0029] A further advantage of some embodiments is that the system / network becomes more efficient and / or training / learning becomes shorter / faster.

[0030] Another advantage of some embodiments is that they provide a less complex network.

[0031] A further advantage of some embodiments is improved / enhanced generalization (eg, across different tasks / contexts).

[0032] A further advantage of some embodiments is that the system / network is less susceptible to noise.

[0033] Yet another advantage of some embodiments is that the system / network can learn new tasks / contexts more quickly and effectively.

[0034] Yet another advantage of some embodiments is that the system / network enables multi-modal discrimination encompassing vision, hearing and language understanding simultaneously.

[0035] Yet another advantage of some embodiments is that the system / network can handle data in a more abstract form.

[0036] Yet another advantage of some embodiments is that the system / network can be "sparsely" activated, making it faster, more energy efficient, and more accurate.

[0037] Yet another advantage of some embodiments is that the system / network understands / interprets different types (or modalities) of data more efficiently.

[0038] Other advantages of some embodiments are improved performance, improved reliability, improved accuracy, improved efficiency (of training and / or performance), faster / shorter training / learning, less computer power required, less training data required, less storage capacity required, reduced complexity, and / or reduced energy consumption.

[0039] The present disclosure will become apparent from the detailed description set forth below. The detailed description and specific examples disclose preferred embodiments of the present disclosure for illustrative purposes only. Those skilled in the art will understand that, from the guidance of the detailed description, changes and modifications may be made within the scope of the present disclosure.

[0040] Thus, it is to be understood that the disclosure disclosed herein is not limited to the particular components of the disclosed apparatus or steps of the disclosed methods, as such apparatus and methods may vary. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting. It is to be noted that, as used in this specification and the appended claims, the articles "a," "an," "the," and "said" are intended to mean that there is one or more elements, unless the context clearly dictates otherwise. Thus, for example, reference to a "unit" or "the unit" may include a plurality of apparatuses, etc. Furthermore, words such as "comprising," "including," "containing," etc. do not exclude other elements or steps.

[0041] The above objects, as well as additional objects, features, and advantages of the present disclosure will be more fully understood by reference to the following illustrative and non-limiting detailed description of exemplary embodiments thereof, when considered in conjunction with the accompanying drawings. [Brief description of the drawings]

[0042] [Figure 1] FIG. 1 is a schematic block diagram illustrating a data processing system according to some embodiments. [Diagram 2] FIG. 2 is a schematic block diagram illustrating a second network according to some embodiments. [Diagram 3] FIG. 3 is a flow chart illustrating method steps according to some embodiments. [Figure 4] FIG. 4 is a schematic diagram illustrating an example computer-readable medium according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0043] The present disclosure will now be described with reference to the accompanying drawings, in which preferred exemplary embodiments of the present disclosure are shown. However, the present disclosure may be embodied in other forms and should not be construed as being limited to the embodiments disclosed herein. The disclosed embodiments are provided to fully convey the scope of the present disclosure to those skilled in the art.

[0044] Glossary In the following, reference will be made to a "node." The term "node" may refer to a neuron, such as a neuron of an artificial neural network, another processing element, such as a processor of a network of processing elements, or a combination thereof. Thus, the term "network" (NW) may refer to an artificial neural network, a network of processing elements, or a combination thereof.

[0045] In the following, reference is made to a "processing unit." A processing unit may be referred to as a synapse, e.g., an input unit (having a processing unit) of a node. However, in some embodiments, the processing unit is a (non-synaptic) (general purpose) processing unit associated with (connected to, connectable to, or composed of) a node of a NW (e.g., a first NW or a second NW) or a (general purpose) processing unit located between a node of the first NW and a node of the second NW.

[0046] In the following we refer to "negative feedback". Negative feedback (or balanced feedback) is or occurs when some function of the output, such as the output of a second NW, is fed back (in a feedback loop) in a way that tends to reduce the amplitude and / or fluctuations of the output, i.e. when the (total) loop gain (of the feedback loop) becomes negative.

[0047] In the following reference will be made to "positive feedback". Positive feedback (or augmented feedback) is or occurs when some function of the output, such as the output of a second NW, is fed back (in a feedback loop) in a way that tends to increase the amplitude and / or fluctuations of the output, i.e. when the (total) loop gain (of the feedback loop) becomes positive.

[0048] In the following we refer to a "leaky integrator" (LI). An LI is a component that has an input, takes / calculates the integral of the input (and provides the calculated integral as an output), and gradually leaks small amounts of the input over time (thereby decreasing the output over time).

[0049] In the following, reference is made to "context." A context is a situation or state of affairs. Context relates to what type of (input) data is expected, for example when there are different types of tasks and each different task has its own context. As an example, if the system input is pixels from an image sensor and the image sensor is exposed to different lighting conditions, each different lighting condition may be a different context for an object such as a ball, a car, a tree, etc. imaged by the image sensor. As another example, if the system input is audio frequency bands from one or more microphones, each different speaker may be a different context for phonemes present in one or more of the audio frequency bands.

[0050] In the following, reference will be made to "measurable". The term "measurable" is to be interpreted as something that can be measured or detected, i.e., detectable. The terms "measure" and "sense" are to be interpreted as synonyms.

[0051] In the following, reference is made to an "entity". The term entity may be interpreted as a physical entity or as a more abstract entity such as a financial entity (e.g., one or more financial data sets). The term "physical entity" may be interpreted as an entity that has a physical existence, such as an object, a feature (of an object), a gesture, an applied pressure, a speaker, a spoken letter, a syllable, a phoneme, a word, a phrase, etc.

[0052] One of the ideas behind the present invention is a system / network in which all nodes are activated, but only some of them are activated to a greater extent (or only some of the nodes are activated) for each specific context / task. Moreover, during learning / training, this system / network dynamically learns which parts (nodes) of the network are suitable for which context / task. Thus, this system / network has a greater capacity to learn various contexts / tasks and / or modalities, and at the same time, learning is faster and more energy efficient (e.g., because the entire network is not activated for each context / task / modality). Since each node can in principle contribute to each task, albeit to a different relative degree, skills learned in one task can be utilized when learning other tasks. This increases the generalizability of learning across different tasks.

[0053] In the following, embodiments of the present disclosure are described, and FIG. 1 is a schematic block diagram illustrating a data processing system 100 according to some embodiments. In some embodiments, the data processing system 100 is a network or comprises a first network and a second network. In some embodiments, the data processing system 100 is a deep neural network, a deep belief network, a deep reinforcement learning system, a recurrent neural network, or a convolutional neural network.

[0054] The data processing system 100 has or is configured to have one or more system input(s) 110a, 110b, ..., 110z. The one or more system inputs 110a, 110b, ..., 110z include data to be processed. This data may be multi-dimensional. For example, multiple signals are provided in parallel. In some embodiments, the system inputs 110a, 110b, ..., 110z comprise or consist of time-continuous data. In some embodiments, the data to be processed comprises data from sensors, such as image sensors, touch sensors, and / or sound sensors (e.g., microphones). Furthermore, in some embodiments, the system input(s) include sensor data for multiple contexts / tasks, for example, while the data processing system 100 is in a learning mode and / or while the data processing system 100 is in a performance mode.

[0055] Further, the data processing system 100 has or is configured to have a system output 120. The data processing system 100 comprises a first network (NW) 130. The first NW 130 comprises a plurality of first nodes 130a, 130b, ..., 130x. Each first node 130a, 130b, ..., 130x has or is configured to have a plurality of inputs 132a, 132b, ..., 132y. At least one of the plurality of inputs 132a, 132b, ..., 132y is a system input 110a, 110b, ..., 110z. In some embodiments, all of the system inputs 110a, 110b, ..., 110z are used as inputs 132a, 132b, ..., 132y to one or more of the first nodes 130a, 130b, ..., 130x. Further, in some embodiments, each of the first nodes 130a, 130b, ..., 130x has one or more system inputs 110a, 110b, ..., 110z as input(s) 132a, 132b, ..., 132y. Furthermore, the first NW 130 generates or is configured to generate outputs 134a, 134b, ..., 134x. In some embodiments, each of the first nodes 130a, 130b, ..., 130x calculates a combination, such as a (linear) sum, sum of squares, or average, of the inputs 132a, 132b, ..., 132y (to that node) multiplied by first weights Wa, Wb, ..., Wy to generate the outputs 134a, 134b, ..., 134x. Furthermore, the data processing system 100 comprises a second NW 140. The second NW 140 comprises a first set 146 of second nodes 140a. Additionally, the second NW 140 comprises a second set 148 of second nodes 140b, ..., 140u. Each of the second nodes 140a, 140b, ..., 140u has or is configured to have as input(s) 142a, 142b, ..., 142u an output 134a, 134b, ..., 134x of one or more of the first nodes 130a, 130b, ..., 130x.In some embodiments, each second node 140a, 140b, ..., 140u has or is configured to have as input(s) 142a, 142b, ..., 142u all outputs 134a, 134b, ..., 134x of the first node(s) 130a, 130b, ..., 130x. Additionally, each second node 140a, 140b, ..., 140u generates or is configured to generate outputs 144a, 144b, ..., 144u. In some embodiments, each of the second nodes 140a, 140b, ..., 140u computes a combination, such as a (linear) sum, sum of squares, or average, of its inputs 142a, 142b, ..., 142u multiplied by second weights Va, Vb, ..., Vu to generate an output 144a, 144b, ..., 144u. The system output 120 includes the outputs 134a, 134b, ..., 134x of each first node 130a, 130b, ..., 130x. In some embodiments, the system output 120 is an array of the outputs 134a, 134b, ..., 134x of each first node 130a, 130b, ..., 130x. Furthermore, the output 144a of the (or each) second node 140a of the first set 146 of nodes 140a is used as an input to one or more processing units 136a3, 136b1, each processing unit 136a3, 136b1 configured to provide negative feedback to the respective first node 130a, 130b. In some embodiments, the negative feedback is provided as a direct input 132c, 132d (weighted with respective weights Wc, Wd) and / or as a linear or (frequency-dependent) non-linear gain control (e.g., gain reduction) of the other input 132a, 132b, 132e, 132f (not shown).That is, in some embodiments, the processing units 136a3, 136b1 are not separate inputs to one or more nodes 130a, 130b, but instead control (e.g., reduce) the gains of other inputs 132a, 132b, 132e, 132f of one or more nodes 130a, 130b, for example, via adjustment of first weights Wa, Wb (associated with one or more nodes 130a, 130b) or by controlling the gains of amplifiers associated with the inputs 132a, 132b, 132e, 132f. Additionally or alternatively, outputs 144b, ..., 144u of one / each second node 140b, ..., 140u of the second set 148 of nodes 140b, ..., 140u are used as inputs to one or more processing units 136x3, each configured to provide positive feedback to a respective first node 130x. In some embodiments, the positive feedback is provided as a direct input 132y (weighted with a respective weight Wy) and / or as a linear or (frequency-dependent) non-linear gain control (e.g., gain increase) of the other inputs 132v, 132x (not shown). That is, in some embodiments, the processing unit 136x3 is not a separate input to one or more nodes 130x, but instead controls (e.g., increases) the gain of the other inputs 132v, 132x of one or more nodes 130x, for example, via adjustment of the first weights Wv, Wx (associated with one or more nodes 130x) or by controlling the gain of an amplifier associated with the input 132v, 132x. By providing negative and / or positive feedback from the nodes of the second network to the first network, the context / task at hand can be processed more accurately and / or efficiently by utilizing only or preferentially utilizing the nodes (of the first network) that are best suited for processing data for that particular context / task. This results in a more efficient data processing system capable of handling a wider range of contexts / tasks, resulting in reduced power consumption.

[0056] In some embodiments, each of the plurality of first nodes 130a, 130b, ..., 130x comprises a processing unit 136a1, 136a2, ..., 136x3 for each of the plurality of inputs 132a, 132b, ..., 132y. Each processing unit 136a1, 136a2, ..., 136x3 comprises an amplifier and a leaky integrator (LI) with a time constant A1, A2. The equation for each LI is of the form dX / dt=-Ax+C, where C is the input and A is the time constant, i.e., the rate of leakage. The time constant A1 for the LI of the processing units 136a3, 136b1, ..., 136x3 having as input the outputs of the nodes of the first or second set 146, 148 of nodes 140a, ..., 140u is larger than the time constant A2 for the LI of (all) other processing units 136a1, 136a2, ... (e.g. all processing units processing the system inputs), e.g. at least 10 times, preferably at least 50 times, more preferably at least 100 times larger. By making A1 larger than A2, i.e. by setting the time constant of the processing units affected by the nodes of the second network (first or second set of nodes) larger than the time constants of the other processing units, e.g. processing units affected by the system inputs, the context may be clarified or emphasized. This results in better / improved dynamic performance, and thus greater reliability, of the data processing system, for example by providing a smoother transition from one context / task to another and / or by avoiding / reducing flip-flops / oscillations between a first processing mode associated with a first context / task and a second processing mode associated with a second (different) context / task.

[0057] In some embodiments, while the data processing system is in the learning mode, the outputs 144a, 144b, ..., 144u of one or more, e.g., each, of the nodes of the first and / or second node sets 146, 148 are suppressed. Furthermore, in some embodiments, each processing unit 136a1, 136a2, ..., 136x3 comprises a suppression unit. Each suppression unit is configured to suppress (at least part of the time) the outputs 144a, 144b, ..., 144u of the respective nodes 140a, 140b, ..., 140u of the first and / or second node sets 146, 148 while the data processing system is in the learning mode. The suppression unit may suppress the outputs 144a, 144b, ..., 144u by setting the gain of the amplifier (of the processing unit it is configured with) to zero or by setting the output (of the processing unit it is configured with) to zero. Alternatively or additionally, each node 140a, 140b, ..., 140u of the first and second sets of nodes 146, 148 comprises an enable unit, each enable unit being directly connected to the output 144a, 144b, ..., 144u of the respective node 140a, 140b, ..., 140u. Each enable unit is configured to inhibit (or make available) the output 144a, 144b, ..., 144u (at least part of the time) while the data processing system is in the learning mode. The enable unit may inhibit the output 144a, 144b, ..., 144u by setting the output 144a, 144b, ..., 144u to zero. In some embodiments, the data processing system 100 comprises a comparison unit 150. The comparison unit 150 is configured to compare the system output 120 to an adaptive threshold, for example, while the data processing system 100 is in the learning mode. In these embodiments, the output 144a, ..., 144u of each node 140a, 140b, ..., 140u of the first or second node set 146, 148 is suppressed only if the system output 120 is greater than an adaptive threshold.In some embodiments, the suppression unit and / or the enablement unit are provided with information, such as a flag, regarding the result of the comparison between the system output 120 and the adaptive threshold. Furthermore, in some embodiments, comparing the system output 120 with the adaptive threshold includes comparing the activity of each first node 130a, ..., 130x, e.g., an average value of the outputs 134a, 134b, ..., 134x of each first node 130a, 130b, ..., 130x, to the adaptive threshold. Alternatively or additionally, comparing the system output 120 with the adaptive threshold includes comparing the activity of one or more particular first nodes 130b, e.g., outputs 134a, 134b, ..., 134x (or an average of the activity) to the adaptive threshold. Further, alternatively or additionally, comparing the system output 120 to the adaptive threshold includes comparing the activity, e.g., outputs 134a, 134b, ..., 134x, of all first nodes 130a, 130b, ..., 130x to the adaptive threshold. Furthermore, in some embodiments, the adaptive threshold is a set of multiple adaptive thresholds, one adaptive threshold for each (or each particular of one or more) first nodes 130a, 130b, ..., 130x. In some embodiments, the adaptive threshold is adapted based on the total energy / activity / level of all system inputs 110a, 110b, ..., 110z or all inputs 132a, 132b, ..., 132y to the first nodes 130a, 130b, ..., 130x. As an example, the higher the total energy of the (system) inputs, the higher the threshold (level) is set. Additionally or alternatively, at the start of the learn mode, the threshold (level) may be higher than at the end of the learn mode.

[0058] In some embodiments, the data processing system 100 is configured to learn to identify one or more (previously unidentified) entities or measurable characteristics (or measurable characteristics) thereof from the sensor data while in the learning mode, and then configured to identify one or more entities or measurable characteristics (or measurable characteristics) thereof from the sensor data while in the performance mode. In some embodiments, the identified entities are one or more of a speaker, spoken character, syllable, phoneme, word, or phrase present in the (audio) sensor data, or an object or feature of an object present in the sensor data (e.g., pixels), or a new contact event, end of a contact event, gesture, or applied pressure present in the (touch) sensor data. In some embodiments, all the sensor data is a specific type of sensor data, such as audio sensor data, image sensor data, or touch sensor data, while in other embodiments the sensor data is a mixture of different types of sensor data, such as audio sensor data, image sensor data, and touch sensor data, i.e. the sensor data is composed of different modalities. In some embodiments, the data processing system 100 is configured to learn to identify a measurable characteristic (or a set of measurable characteristics) of an entity from the sensor data. The measurable characteristic may be a feature of an object, a part of a feature, a time-evolving trajectory of a position, a trajectory of applied pressure, or a frequency signature or a time-evolving frequency signature of a particular speaker when speaking a particular character, syllable, phoneme, word, or phrase. Such measurable characteristics may be mapped to an entity. For example, an object feature may be mapped to an object, a part of a feature may be mapped to a feature (of an object), a trajectory of a position may be mapped to a gesture, a trajectory of applied pressure may be mapped to a (maximum) applied pressure, a frequency signature of a particular speaker may be mapped to said speaker, and a spoken character, syllable, phoneme, word, or phrase may be mapped to an actual character, syllable, phoneme, word, or phrase.Such a mapping may simply be a lookup in memory, a lookup table or a database. The lookup may be based on finding an entity among a plurality of physical entities that has a closest characteristic to the identified measurable characteristic. From such a lookup, a real entity may be identified, e.g., an unidentified entity is identified as the entity among a plurality of entities having one or more stored characteristics that has a closest match to the one or more identified characteristics. By utilizing the methods described herein to identify one or more unidentified entities or their measurable characteristics (or measurable characteristics), improved performance of entity identification is achieved, more reliable entity identification is provided, a more efficient entity identification method is provided, and / or a more energy efficient entity identification method is provided as the method reduces computational power and / or storage capacity.

[0059] In some embodiments, each input 142a, 142b, ..., 142u of the second nodes 140a, 140b, ..., 140u is a weighted version of the output 134a, 134b, ..., 134x of one or more of the first nodes 130a, 130b, ..., 130x. Furthermore, in some embodiments, each of the second nodes 140a, 140b, ..., 140u comprises a (second) processing unit (not shown) for each of the multiple inputs 142a, 142b, ..., 142u. In these embodiments, each of the multiple inputs 142a, 142b, ..., 142u may be processed by the respective (second) processing unit, for example, before being weighted by the respective second weights Va, Vb, ..., Vu.

[0060] In some embodiments, the learning and / or updating of the weights Wa, Wb, ..., Wy, Va, Vb, ..., Vu during the learning mode of the first network 130 and / or the second network 140 is based on correlation, for example between each respective input 142a, ..., 142c to the node 140a and the combined activity of all inputs 142a, ..., 142c to that node 140a, i.e., between each respective input 142a, ..., 142c to the node 140a and the output 144a of that node 140a (example of node 140a, applicable to all other nodes 130b, ..., 130x, 140a, ..., 140u). In order for the system to learn during the learning mode, updating of the weights Wa, Wb, ..., Wy, Va, Vb, ..., Vu for the first and / or second network 130, 140 may be performed. For this purpose, the data processing system 100 may comprise an update / learning unit 160. In some embodiments, the negative and positive feedback loop from the second network 140 back to the first network 130 may occur with fixed weights, i.e. the first weights Wa, Wb, ..., Wy are fixed (e.g. set to a fixed value in a first step based on correlation), while the weights of the connections from the first network 130 to the second network 140, i.e. the second weights Va, Vb, ..., Vu, are changeable by learning based on correlation. This helps to identify which nodes 130a, 130b, ..., 130x provide relevant information for a given context (e.g., information that is important to that context) and then helps these particular nodes (i.e., the nodes that provide information that is important to that context) cooperatively increase each other's activity / output for that context (through positive feedback).At the same time, these cooperative nodes, through learning based on correlations in the nodes of the negative feedback loop, automatically identify other nodes that provide the least relevant information (e.g., unimportant information) for the context and suppress the activity (e.g., output) of these nodes through negative feedback. In another context, another subset of nodes (possibly only partially overlapping with the previous subset of nodes) may learn to provide relevant information and cooperatively increase or amplify each other's activity for the context (and suppress the activity of other nodes that provide less relevant information for the context). This contributes to generating a first (data processing) network 130 in which many nodes learn to participate across many different contexts, albeit with different (relative) specializations. The connections from the first network 130 to the second network 140 may be learned while the first network 130 is not in a learning mode, or the second network 140 may be learned simultaneously with the learning in the first network 130. That is, the second weights Va, Vb, ..., Vu may be updated / modified during the second learning mode where the first weights Wa, Wb, ..., Wy are fixed (e.g., after the first learning mode where the first weights Wa, Wb, ..., Wy are updated / modified / set). In some embodiments, the first and second learning modes may be repeated a number of times, e.g., 2, 3, 4, 5 or 10 times, i.e., iterations of the first and second learning modes may be performed. Alternatively, both the first weights Wa, Wb, ..., Wy and the second weights Va, Vb, ..., Vu are updated / modified during the learning mode. In some embodiments, the data processing system 100 comprises an update / learning unit 160 for updating, combining and / or correlating. In some embodiments, the update / learning unit 160 has the system output 120 (or the desired system output) as a direct input. Alternatively, the update / learn unit 160 has the output of the comparison unit 150 as an input.Alternatively or additionally, the update / learn unit 160 has as input the state / values ​​of each of the first weights Wa, Wb, ..., Wy and / or each of the second weights Va, Vb, ..., Vu. In some embodiments, the update / learn unit 160 applies a correlation learning rule to the actual (or desired) outputs and inputs of the (each) first node 130a, 130b, ..., 130x and / or each second node 140a, 140b, ..., 140u to find the differential weight(s) to apply to the weights Wa, Wb, ..., Wy, Va, Vb, ..., Vu (for updating). In some embodiments, the update / learn unit 160 generates an update signal (e.g., including the differential weights), which is used to update each of the first weights Wa, Wb, ..., Wy and / or each of the second weights Va, Vb, ..., Vu. In some embodiments, the data processing system 100 comprises a first update / learning unit configured to update each respective first weight Wa, Wb, ..., Wy and a second update / learning unit configured to update each respective second weight Va, Vb, ..., Vu. In some embodiments, the learning is based on correlation. That is, for a first node (e.g., 130a) that is not correlated with the activity, e.g., output (e.g., 144a), of a particular second node (e.g., 140a), the second weight (e.g., Va) associated with the connection between the particular first node (e.g., 130a) and the particular second node (e.g., 140a) is gradually decreased, while for a first node (e.g., 130b) that is correlated with the activity, e.g., output (e.g., 144a), of a second node (e.g., 140a), the second weight (e.g., Vb) associated with the connection between the particular first node (e.g., 130b) and the particular second node (e.g., 140a) is gradually increased. Once the learning mode is completed, in some embodiments, the first and second weights Wa, Wb, ..., Wy, Va, Vb, ..., Vu are not updated, e.g., during the performance mode.

[0061] In some embodiments, the first node 130a comprises a plurality of processing units 136a1, ..., 136a3 configured to provide negative feedback and / or a plurality of processing units 136a1, ..., 136a3 configured to provide positive feedback. Thus, the first nodes 130a, 130b, ..., 130x may have a plurality of processing units providing negative feedback and a plurality of processing units providing positive feedback (although no processing unit may provide both negative and positive feedback). In these embodiments, the negative / positive feedback may be provided as weighted direct inputs 132c, and the first weights Wa, Wb, Wc associated with (connected to) the processing units 136a1, ..., 136a3 may be different from each other.

[0062] 2 illustrates a second network 140 according to some embodiments. The second network, NW, 140, is connectable to a first NW 130, which comprises a plurality of first nodes 130a, 130b, ..., 130x. Each of the first nodes 130a, 130b, ..., 130x has or is configured to have a plurality of inputs 132a, 132b, ..., 132x. Furthermore, each of the first nodes 130a, 130b, ..., 130x generates or is configured to generate an output 134a, 134b, ..., 134x. Furthermore, each of the first nodes 130a, 130b, ..., 130x comprises at least one processing unit 136a3, ..., 136x3. The second NW 140 comprises a first and second set 146, 148 of second nodes 140a, 140b, ..., 140u. Each second node 140a, 140b, ..., 140u is configurable to have as input(s) 142a, 142b, ..., 142u an output 134a, 134b, ..., 134x of one or more first nodes 130a, 130b, ..., 130x. Additionally, each second node 140a, 140b, ..., 140u generates or is configured to generate an output 144a, 144b, ..., 144u. An output 144a of a / each second node 140a of the first set 146 of nodes 140a is available as an input to one or more processing units 136a3, each processing unit 136a3 providing or configured to provide negative feedback to each first node 130a (of the first NW 130). Additionally or alternatively, an output 144u of a / each second node 140u of the second set 148 of nodes 140b, ..., 140u is available as an input to one or more processing units 136x3, each processing unit 136x3 providing or configured to provide positive feedback to a respective first node 130x (of the first NW 130).The second NW 140 may be utilized to increase the capacity of the first NW 130 (or make the first NW more efficient), for example, by identifying an apparent (existing) context of the first NW 130 (and facilitating adaptation of the first NW 130 according to the identified context).

[0063] FIG. 3 is a flow chart illustrating exemplary method steps according to some embodiments. FIG. 3 illustrates a computer-implemented or hardware-implemented method 300 for processing data. The method can be implemented in analog hardware / electronic circuits, digital circuits, such as gates and flip-flops, mixed-signal circuits, software, and any combination thereof. The method includes receiving 310 one or more system input(s) 110a, 110b, ..., 110z including data to be processed. Additionally, the method 300 includes providing 320 a plurality of inputs 132a, 132b, ..., 132y, at least one of the plurality of inputs being a system input, to a first network NW(30) comprising a plurality of first nodes 130a, 130b, ..., 130x. The method 300 further includes receiving 330 the outputs 134a, 134b, ..., 134x from each of the first nodes 130a, 130b, ..., 130x. The method 300 further includes providing 340 the system output 120. The system output 120 includes the outputs 134a, 134b, ..., 134x of each of the first nodes 130a, 130b, ..., 130x. The method 300 further includes providing 350 the outputs 134a, 134b, ..., 134x of each of the first nodes 130a, 130b, ..., 130x to the second NW 140. The second NW 140 includes first and second sets 146, 148 of second nodes 140a, 140b, ..., 140u. Further, the method 300 includes receiving 360 the output 144a, 144b, ..., 144u of each second node 140a, 140b, ..., 140u. The method 300 includes using 370 the output 144a of a / each second node 140a of the first set 146 of nodes 140a as an input to one or more processing unit(s) 136a3, each processing unit 136a3 configured to provide negative feedback to a respective node 130a of the first NW 130 (based on the input).Additionally or alternatively, the method 300 includes using 380 the output 144u of some / each second node 140u of the second set 148 of nodes 140b, ..., 140u as input to one or more processing unit(s) 136x3, each processing unit configured to provide positive feedback (based on the input) to a respective node 130x of the first NW 130. In some embodiments, steps 310-380 are repeated until a stopping condition is met. The stopping condition may be that all data to be processed has been processed, or that a certain amount of data / number of loops has been processed / performed.

[0064] In some embodiments, the computer program product is comprised of a non-transitory computer readable medium 400, such as, for example, a Universal Serial Bus (USB) memory, a plug-in card, an embedded drive, a digital versatile disk (DVD) or a read-only memory (ROM). FIG. 4 shows an example of a computer readable medium in the form of a compact disk (CD) ROM 400. The computer readable medium has stored thereon a computer program comprising program instructions. The computer program can be loaded, for example, into a data processor (PROC) 420 configured in a computer or computing device 410. When loaded into a data processing device, the computer program can be stored in a memory (MEM) 430 associated with or configured in the data processing unit. According to some embodiments, the computer program, when loaded into the data processing unit and executed by the data processing unit, can cause the execution of method steps, for example according to the method illustrated in FIG. 3 described herein.

[0065] Those skilled in the art will understand that the present disclosure is not limited to the preferred embodiments described above. Those skilled in the art will further understand that modifications and variations are possible within the scope of the appended claims. For example, signals from other sensors, such as aroma sensors and flavor sensors, may be processed by the data processing system. Furthermore, the described data processing system can be similarly utilized for unsegmented and connected handwriting recognition, speech recognition, speaker recognition, anomaly detection in network traffic or intrusion detection systems (IDS). Furthermore, variations to the disclosed embodiments can be understood and effected by those skilled in the art practicing the claimed disclosure from a study of the drawings, the disclosure, and the appended claims.

Claims

1. A data processing system (100) configured to have one or more system inputs (110a, 110b, ..., 110z) containing data to be processed and a system output (120), said data processing system comprising: A first network, NW, (130) comprising a plurality of first nodes (130a, 130b, ..., 130x), each first node configured to have a plurality of inputs (132a, 132b, ..., 132y), at least one of the plurality of inputs being a system input, and configured to generate outputs (134a, 134b, ..., 134x); and a second NW (140) comprising first and second sets (146, 148) of second nodes (140a, 140b, ..., 140u), each second node having as input (142a, 142b, ..., 142u) an output (134a, 134b, ..., 134x) of one or more first nodes (130a, 130b, ..., 130x) and configured to generate outputs (144a, 144b, ..., 144u); wherein the system output (120) includes the outputs (134a, 134b, . . . , 134x) of each first node (130a, 130b, . . . , 130x); and the output (144a) of a second node (140a) of the first set (146) of nodes (140a) is used as an input to one or more processing units (136a3, 136b1), each processing unit (136a3, 136b1) configured to provide negative feedback to a respective first node (130a, 130b); and / or the output (144) of a second node (140u) of the second set (148) of nodes (140b, . . . , 140u) is used as an input to one or more processing units (136x3), each processing unit configured to provide positive feedback to a respective first node (130x); A data processing system comprising:

2. 2. The data processing system of claim 1, Each of the plurality of first nodes (130a, 130b, ..., 130x) comprises a processing unit (136a1, 136a2, ..., 136x3) for each of the plurality of inputs (132a, 132b, ..., 132y), each processing unit (136a1, 136a2, ..., 136x3) comprising an amplifier and a leaky integrator having a time constant (A1, A2), where: the time constant (A1) of the processing unit (136a3, . . . , 136x3) having as input an output of a node of said first or second set (146, 148) of nodes (140a, . . . , 140u) is greater than the time constant (A2) of the other processing units; A data processing system comprising:

3. 3. A data processing system according to claim 1, further comprising: While the data processing system (100) is in a learning mode, the outputs (144a, 144b, ..., 144u) of each node of the first and / or second sets of nodes (146, 148) are suppressed; A data processing system comprising:

4. 4. A data processing system according to claim 1, each of said processing units comprises a suppression unit configured to suppress said output (144a, 144b, ..., 144u) of each node of said first and / or second node sets (146, 148) while said data processing system is in a learning mode; A data processing system comprising:

5. 4. A data processing system according to claim 1, each node (140a, 140b, ..., 140u) of said first and second sets of nodes (146, 148) comprising an enable unit, each enable unit being directly connected to said outputs (144a, 144b, ..., 144u) of a respective one of said nodes (140a, 140b, ..., 140u), said enable unit being configured to inhibit said outputs (144a, 144b, ..., 144u) while said data processing system is in said learning mode; A data processing system comprising:

6. 6. A data processing system according to claim 3, further comprising: the data processing system (100) comprises a comparison unit (150), the comparison unit (150) being configured to compare the system output (140) with an adaptive threshold while the data processing system (100) is in a learning mode, and the output (144a, ..., 144u) of each node (140a, 140b, ..., 140u) of the first or second node set (146, 148) is inhibited only if the system output (140) is greater than the adaptive threshold; A data processing system comprising:

7. 7. A data processing system according to claim 1, The system input includes sensor data for multiple contexts / tasks. A data processing system comprising:

8. 8. A data processing system according to claim 1, the data processing system is configured to learn to identify one or more entities from the sensor data while in a learning mode, and thereafter to identify the one or more entities while in a performance mode, the identified entities being one or more of a speaker, a spoken character, a syllable, a phoneme, a word or phrase, or an object or feature of an object present in the sensor data, or a new touch event, an end of a touch event, a gesture, or an applied pressure present in the sensor data; A data processing system comprising:

9. 9. A data processing system according to claim 1, each input (142a, 142b, ..., 142u) of said second node (140a, 140b, ..., 140u) is a weighted version of an output (134a, 134b, ..., 134x) of said one or more first nodes (130a, 130b, ..., 130x); A data processing system comprising:

10. 10. A data processing system according to claim 3, further comprising: training during said training mode and / or updating of weights of said first and / or second networks (130, 140) is based on correlations; A data processing system comprising:

11. A second network, NW, (140) connectable to a first NW (130), the first NW (130) comprising a plurality of first nodes (130a, 130b, ..., 130x), each of the first nodes (130a, 130b, ..., 130x) having a plurality of inputs (132a, 132b, ..., 132x) and configured to generate outputs (134a, 134b, ..., 132x), and comprising at least one processing unit (136a3, ..., 136x3); The second NW (140) comprises: a first and second set (146, 148) of second nodes (140a, 140b, ..., 140u), where each second node (140a, 140b, ..., 140u) is configurable to have as inputs (142a, 142b, ..., 142u) the outputs (134a, 134b, ..., 134x) of one or more first nodes (130a, 130b, ..., 130x) and is configured to generate outputs (144a, 144b, ..., 144u); and wherein the output (144a) of a second node (140a) of the first set (146) of nodes (140a) is available as an input to one or more processing units (136a3, 136b1), each processing unit (136a3, 136b1) configured to provide negative feedback to a respective first node (130a, 130b), and / or the output (144u) of a second node (140u) of the second set (148) of nodes (140b, ..., 140u) is available as an input to one or more processing units (136x3), each processing unit configured to provide positive feedback to a respective first node (130x), A second network.

12. A computer-implemented or hardware-implemented method (300) for processing data, comprising: Receive (310) one or more system inputs (110a, 110b, ..., 110z) containing data to be processed; providing (320) a plurality of inputs (132a, 132b, . . . , 132y), at least one of the plurality of inputs being a system input, to a first network, NW, (130), comprising a plurality of first nodes (130a, 130b, . . . , 130x); receiving (330) outputs (134a, 134b, . . . , 134x) from each first node (130a, 130b, . . . , 130x); providing (340) a system output (120) including the outputs (134a, 134b, . . . , 134x) of each first node (130a, 130b, . . . , 130x); providing (350) the outputs (134a, 134b, ..., 134x) of each first node (130a, 130b, ..., 130x) to a second NW (140) comprising first and second sets (146, 148) of second nodes (140a, 140b, ..., 140u); receiving (360) an output (144a, 144b, ..., 144u) of each second node (140a, 140b, ..., 140u); and using (370) the output (144a) of a second node (140a) of the first set (146) of nodes (140a) as an input to one or more processing units (136a3, 136b1), each processing unit (136a3, 136b1) configured to provide negative feedback to a respective first node (130a, 130b); and / or using (380) the output (144) of a second node (140u) of the second set (148) of nodes (140b, . . . , 140u) as an input to one or more processing units (136x3), each processing unit (136x3) configured to provide positive feedback to a respective first node (130x); A method comprising:

13. 13. A computer program product comprising a non-transitory computer readable medium (400) having stored thereon a computer program comprising program instructions, said computer program being loadable into a data processing unit (420) and configured to cause the execution of a method according to any of claims 12 to 13 when said computer program is executed by said data processing unit (420).

Images