Computer-implemented methods, computer-controlled systems, and computer programs for solving cognitive tasks, including learning abstract properties (Neural network systems for abstract reasoning)

Abstract

To provide a computer-implemented method, system, and computer program product to solve a cognitive task that includes learning abstract properties.SOLUTION: One embodiment comprises accessing datasets that characterize abstract properties. The accessed datasets are then inputted into a first neural network to generate first embeddings. Generated pairs of the first embeddings can be formed, which correspond to pairs of the datasets. Data corresponding to the formed pairs are then inputted into a second neural network, which can be executed to generate second embeddings. The latter can capture relational properties of the pairs of the datasets. A third neural network is subsequently executed based on the generated second embeddings to obtain output values. One or more abstract properties of the datasets are learned based on the obtained output values in order to solve a cognitive task.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates generally to the field of computer-implemented methods and systems for solving cognitive tasks that require learning abstract properties. [Background technology]

[0002] In particular, the present disclosure is directed to methods that rely on combined neural networks to generate embeddings that capture relevant properties of pairs of input datasets (e.g., frames), where such datasets characterize the abstract properties to be learned. In some embodiments, the neural networks may be advantageously regularized using mutual information-based regularizers.

[0003] The development of the EDVAC system in 1948 is often cited as the beginning of the computer age. Since that time, computer systems have evolved into extremely complex devices. Today's computer systems typically include a combination of sophisticated hardware and software components, application programs, operating systems, processors, buses, memory, input / output devices, etc. As advances in semiconductor processing and computer architectures push performance ever higher, more advanced computer software has evolved to take advantage of these higher performance capabilities, resulting in today's computer systems that are much more powerful than they were just a few years ago.

[0004] One application of these new capabilities is machine learning. Machine learning mostly relies on artificial neural networks (ANNs), which generally refer to computational models inspired by biological neural networks in human or animal brains. Such systems usually learn tasks by example, progressively and autonomously; that is, they have been successfully applied to speech recognition, text processing, and computer vision, among many other tasks.

[0005] Many types of ANNs are known, starting from feedforward neural networks (FFNNs), to multi-layer perceptrons, deep neural networks (DNNs), and convolutional neural networks (CNNs). ANNs are typically implemented in software, and the majority of implementations of ANNs involve software implementations. However, ANNs can also be implemented in hardware, for example as resistive processing units or optical neuromorphic systems. Summary of the Invention [Problem to be solved by the invention]

[0006] Visual oddities tasks generally refer to a set of tasks designed to test fundamental concepts of geometry, such as points, lines, parallelism, and symmetry. They are first introduced as part of a neuroscience experiment to test knowledge of conceptual principles of geometry. Each task has six frames, five of which contain the geometric concept being tested. One of the frames disrupts the geometric concept and is called an oddity. The goal of the task is to classify how many of the six frames are oddities. [Means for solving the problem]

[0007] According to a first aspect, there is a computer-implemented method for solving a cognitive task that requires learning abstract properties. The method includes first accessing a dataset (e.g., image frames) that characterizes the abstract properties. The accessed dataset is then input to a first neural network (e.g., CNN), which is then executed to generate a first embedding. Then, a pair of the generated first embeddings is formed, which corresponds to the pair of datasets. Data corresponding to the formed pair (e.g., data obtained by concatenating the pair of first embeddings) is then input to a second neural network, which is executed to generate a second embedding. The latter captures the associative properties of the pair of datasets. A third neural network is then executed to obtain an output value, which is typically interpretable as a distance or similarity between the accessed datasets, based on the generated second embedding. Each of the second and third neural networks may be a fully connected neural network. Finally, one or more abstract properties of the datasets are learned based on the obtained output value to solve the cognitive task.

[0008] In some embodiments, the method further includes regularizing at least one of the first neural network and the second neural network using a mutual information based regularizer, e.g., by applying a regularization term determined by minimizing a mutual information based function, e.g., according to a variational approximation to an information bottleneck function.

[0009] In some embodiments, the accessed data sets each correspond to a frame characterizing an abstract property, and the cognitive task is a visual oddity task, whereby one of the accessed frames represents an oddity.

[0010] In some embodiments, each of the first, second, and third neural networks executes based on parameters that were learned during one or more previous phases while solving one or more previous different cognitive tasks.

[0011] According to another aspect, a computerized system is provided for solving cognitive tasks including learning abstract properties consistent with the method. During operation, the system is adapted to operate an input unit, a neural network module, and an output unit. The input unit is adapted to access a data set characterizing the abstract property. The neural network module is connected to the input unit. The modules include a first neural network, a structural operations unit, a second neural network, and a third neural network. The first neural network is configured to generate a first embedding upon execution based on the data set as accessed by the input unit. The structural operations unit is configured to form pairs of the first embeddings generated by the first neural network, the pairs being formed corresponding to said data set pairs. The second neural network is configured to generate a second embedding capturing the associative properties of the data set pairs upon execution based on data corresponding to the pairs as formed by the structural operations unit during operation. The third neural network is configured to generate an output value (e.g., interpretable as a distance or similarity of the accessed dataset) upon execution based on the second embedding as generated by the second neural network. Finally, an output unit is connected to the neural network module and configured to learn one or more abstract properties of the dataset based on the output value as obtained from the third neural network to solve a cognitive task.

[0012] According to another aspect, there is a computer program product (again for solving cognitive tasks involving learning abstract properties) that includes a computer-readable storage medium having program instructions embodied therein, the program instructions being executable by one or more processors to cause operations implementing the method.

[0013] Computer controlled systems, methods and computer program products in which the present disclosure is embodied will now be described, by way of non-limiting example and with reference to the accompanying drawings, in which: [Brief description of the drawings]

[0014] In the accompanying drawings, like reference characters refer to identical or functionally similar elements throughout the individual figures, and together with the following detailed description, which are incorporated in and form a part of this specification, serve to further illustrate various embodiments and explain various principles and all advantages of the present disclosure.

[0015] [Figure 1] 1 is a flow chart illustrating high-level components of an embodiment of a computerized system, as well as the operation of an embodiment of a method of operating the system to solve a cognitive task. [Figure 2A] FIG. 2 illustrates, in an embodiment, different sets of frames (i.e., images) that may be used as input to a neural network module of a system such as that illustrated in FIG. 1 to solve a visual oddity task. [Figure 2B] FIG. 2 illustrates, in an embodiment, different sets of frames (i.e., images) that may be used as input to a neural network module of a system such as that illustrated in FIG. 1 to solve a visual oddity task. [Figure 2C]FIG. 2 illustrates, in an embodiment, different sets of frames (i.e., images) that may be used as input to a neural network module of a system such as that illustrated in FIG. 1 to solve a visual oddity task. [Figure 2D] FIG. 2 illustrates, in an embodiment, different sets of frames (i.e., images) that may be used as input to a neural network module of a system such as that illustrated in FIG. 1 to solve a visual oddity task.

[0016] [Diagram 3] FIG. 2 illustrates, in an embodiment, a schematic diagram of how the inputs to, and outputs obtained from, a neural network of a system such as that illustrated in FIG. 1 and the final output are combined to solve a visual oddity task.

[0017] [Figure 4] FIG. 1 illustrates a schematic representation of a general-purpose computerized system suitable for implementing operations of one or more methods associated with embodiments of the present disclosure.

[0018] The accompanying drawings depict simplified representations of computer-controlled systems or portions thereof as they relate to embodiments, and in which like or functionally similar elements are assigned the same reference numbers unless otherwise indicated. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] This disclosure describes a solution to the task of visual oddity, and more generally, to cognitive tasks that require the formation of metaphors.

[0020] An ANN generally includes a set of connected units or nodes, which are considered to be equivalent to biological neurons in an animal's brain and are therefore called artificial neurons. Signals can be transmitted along connections (also called edges), similar to synapses, between artificial neurons. Taken together, an artificial neuron that receives a signal can process it and then send a signal to the connected neurons.

[0021] Connection weights (also called synaptic weights) may be associated with the connections, and such weights may adapt as learning progresses. Each neuron may have several inputs, and a connection weight is attributed to each input (the weight of a particular connection). The connection weights may be learned and updated by a training algorithm during a training phase. The learning process is iterative, and data cases may be presented to the network, usually one at a time, and the weights associated with the input values ​​may be adjusted at each time step.

[0022] With reference to Figures 1-3, aspects of the disclosure related to a computer-implemented method for solving cognitive tasks that require learning of abstract properties are described. It should be noted that the method and its variations are collectively referred to as "the method". All references Sij refer to operations of the method illustrated in Figures 1 and 3, and reference numbers refer to components of the system 1 and digital objects consumed or created thereby.

[0023] As shown in Figure 1, the method may first include accessing S10 a dataset 10. The dataset 10 may include, for example, frames (e.g., images). More generally, the dataset may include data files or data characterizing abstract properties to be learned to solve a given cognitive task. In the following, however, the dataset 10 is assumed for purposes of representation to be a frame, such as those illustrated in Figures 2A-2D.

[0024] The accessed frames 10 may be input S20 to a first neural network 21, which may be a CNN. This network may be run based on the input frames 10 to generate S30 a first embedding 31. The first embedding 31 may be output from the first neural network 21. Such an embedding typically characterizes a vector, e.g. a one-dimensional (1D) array of numerical values.

[0025] Next, pairs 32 of first embeddings 31 may be formed S40. The embedding pairs 32 may correspond to pairs of frames 10. This task may be performed by structural operations unit 21a, which forms part of neural network modules 21-23 implementing a pipeline of three neural networks, as described below. The pairs 32 may be formed S40 simply by concatenating data corresponding to each pair of embeddings 31, e.g. the data corresponding to the formed pairs may include concatenated values ​​that ultimately form a single vector 32 for each pair of embeddings 31. It should be noted that the pairs 32 of embeddings 31 illustrated in FIG. 3 may therefore each consist of a single array.

[0026] Data corresponding to the formed pairs 32 are then input S40 into a second neural network 22, which may be a fully connected network. As mentioned, the input data may simply consist of the concatenated data of the paired embeddings 31, whereby a one-dimensional array of data may be input S40 into the second network 22. The network 22 may then be run based on such data to generate S50 second embeddings 33, 34. The second embeddings 33, 34 may be output from the second neural network 22, which captures the associative properties of the pairs of frames 10.

[0027] Next, based on the second embeddings 33, 34, the third neural network 23 is executed. The direct output 33 from the network 22 may be subjected to several mathematical operations resulting in transformed data 34 before being fed S50 to the third network 23. In some embodiments, all outputs 33 from the second neural network 22 corresponding to the same frame may be summed before being input S50 to the third neural network 23 for their execution. The third network 23 may also be a fully connected network. The execution of the third network 23 may provide S60 an output value 35, which may typically be interpreted as a distance or similarity between the accessed frames 10.

[0028] Next, one or more abstract characteristics of the frames 10 may be learned based on the output values ​​35 obtained in operation S60 to solve a cognitive task. The cognitive task may, for example, be a visual oddity task, whereby one of the frames 10 accessed in operation S10 represents an oddity when compared to other frames, e.g., see Figures 2A to 2D. In that case, learning the abstract characteristics of the frames (based on the output values ​​35 obtained in operation S60) allows solving the cognitive task by discriminating one of the input frames based on characteristics determined from each frame.

[0029] Beyond the task of visual oddity and other visual tasks, the embodiments of the present disclosure may enable the solution of certain cognitive tasks that may be characterized by an input data set 10 to be efficiently solved by a relational network formed by the second network 22 and the third neural network 23. The idea behind these embodiments is to first calculate an embedding 31 from the input data set 10, then form an embedding pair 32, obtaining data 32 (e.g., a concatenated vector) that can be fed to the second neural network 22. In this way, a second embedding 33, 34 can be generated that can capture the relational properties of the pair of frames 10. Using a network pipeline 22, 23 designed to handle such relational properties, the data 34 obtained from the second embedding 33 can then be fed to the third network 23 to obtain an output 35, which can simply be evaluated to learn the desired abstract properties, thereby solving the cognitive task, as will be explained in more detail later.

[0030] In some embodiments, the method may further include regularizing at least one (possibly each) of the first neural network 21 and the second neural network 22. The network regularization may be performed using a mutual information based regularization means. In general, mutual information refers to a measure of the mutual dependency of two random variables and may essentially be used to measure the relationship between such variables. Mutual information may be considered as a phenomenon of uncertainty in a random variable given the available knowledge about the other variable. It may be measured in bits and is also called Shannon. In this context, mutual information may include information between hidden layers and the inputs and outputs of the neural network.

[0031] In some embodiments, regularization may be achieved by employing mutual information estimates as additional terms in the loss function characteristic of the network. The regularization term may be added, along with cross-entropy, as an additional loss contribution to the overall loss function. Tests performed by the authors of this disclosure have shown that such regularization may stabilize test accuracy and significantly reduce variance. Comparison of the accuracies achieved on individual tasks shows that the network pipeline 21-23, when augmented by a mutual information-based regularizer, may achieve overall performance on the task of visual oddity that substantially exceeds human accuracy.

[0032] The regularization term may, in some embodiments, be determined by minimizing a mutual information based function. For example, the regularization term may be determined according to a variational approximation to the known information bottleneck (IB) method.

[0033] The goal of the IB method is to find a maximally compressed representation of an input random variable X, where X may be obtained as a function of the associated random variables Y, thereby preserving as much information as possible on the associated random variables Y. An alternative mutual information estimation method known in neural networks is a variational approximation to the IB method, called the variational information bottleneck (VIB), which may be utilized in this context. The VIB method may be applied to multiple layers of a convolutional neural network for network compression. An approximation of the mutual information between successive layers may be added as part of the objective function as a regularizer.

[0034] For example, a VIB-based regularizer may be added to the convolutional neural network and the convolutional layers (e.g., the four fully connected layers that compose it) of the second neural network 22. The VIB-based regularizer may improve and stabilize the accuracy obtained by the network structure, especially with regard to the task of visual oddity.

[0035] As induced earlier, the first network may be a CNN, followed by fully connected layers up to the final output. One feature and advantage of a CNN is the benefit of sensor image data. In some embodiments, the first network 21 is a CNN and may include 4, 5, or more layers. The second network 22 or the third network 23, or both, may then be fully connected networks. The network 22 may include, for example, four layers, and the third neural network may include three layers. The four layers of the second network may include, for example, 512 neurons each, and the three layers of the third network may include, for example, 256, 256, and 1 neurons, respectively. More generally, successive networks may be designed to accommodate successively compressed data (e.g., from the input data set 10 to the first embedding 31 to the second embeddings 33, 34 to the final output 35).

[0036] The second and third networks 22, 23 may be implemented as related network modules, which form part of the overall network modules 21-23, as will be described later with reference to other aspects of the disclosure.

[0037] In a first class of embodiments, each initial image frame 10 is input S20 to a four-layer CNN 21. Each convolutional layer may have, for example, 32 filters of kernel size 5×5. The convolved outputs S30 from the convolutional neural network for each image may then be concatenated S40 according to channel, thereby flattening them into a vector (i.e., a one-dimensional array). The flattened outputs may be input S40 to a four-layer fully connected network 22.

[0038] In some of this first class of embodiments, batch normalization may be used for all convolutional layers. In addition, dropout regularization may be applied after all convolutional layers that do not precede a pooling layer (e.g., with a dropout rate of 0.3) and after the first fully connected layer of the second network 22 (e.g., with a dropout rate of 0.5).

[0039] Overall, the neural network modules 21-23 implementing the three networks 21-23 may be configured as relational networks to solve the task of visual oddity. Such a network may advantageously be designed to extend the known Wild Relation Network (WReN), which is brought to solve Raven's progressive matrix RPM. However, the WReN model directly calculates the pairwise relations of the RPM panels, i.e., determines the pairwise relations between the context panel and the selection panel. For a cognitive task such as the task of visual oddity, it is not possible that any context panel or selection panel as all frames can be classified as oddity.

[0040] In a second class of embodiments, each frame 10 of the visual oddity task may be input S20 to a five-layer CNN 21, which may output a corresponding frame embedding 31. As shown in FIG. 3, this task may involve, for example, identifying non-quads from a given set of six frames. See also FIG. 2A. For example, for every frame k ∈ [1, 2, ..., 6], the CNN 21 may output a frame embedding y k Then, for each frame embedding y kFor a frame, as illustrated in FIG. 3 using individually patterned shapes 31, 32, a pair 32 of embeddings 31 may be generated S40 by sequentially concatenating them with the embeddings of the other five frames. By way of example, for the first frame (frame 1), the pairs are {y1, y2}, {y1, y3}, ..., and {y1, y6}. Thus, a total of 30 pairs 32 may be generated. Each pair 32 may then be fed S40 into a second network 22. Stated differently, the architecture of FIG. 3 may be such that each pair 32 is computed using a function g parameterized by the neural network 22. θ It is assumed that this can be done frame by frame. θ The outputs 33 of the summed outputs 34 are summed to result in an array of data 34, and a similar type of calculation can be performed for each frame. As shown in FIG. 3, the vector of summed outputs 34 corresponding to each frame 10 is then fed to a third neural network 23 (e.g., a second function f φ (used as an argument to S50) to calculate a final score 35 and then identify oddities based on all the final scores obtained.

[0041] In an embodiment, the learning of the abstract features is performed by determining the probability that each frame is an oddity, as further envisioned in FIG. φ This can be accomplished simply by applying a softmax function over all output values ​​35 obtained from, oddity, which in this example corresponds to a maximum value of 0.970.

[0042] Other possible visual oddity tasks may include, for example, distinguishing non-convex shapes from convex shapes (FIG. 2B), identifying pairs of shapes that cannot be captured by rotation from others that can (FIG. 2C), or distinguishing scalene from equilateral triangles (FIG. 2D). Each oddity frame is bounded by a dashed line in the data sets 10a-10d shown in FIGS. 2A-2D.

[0043] The networks 21, 22, 23 may be calculated for training or inference purposes, and may be trained before making inferences. Network parameters learned for a given task may be transferred to further tasks in some cases. In particular, in the method, each of the first neural network 21, the second neural network 22, and the third neural network 23 may be executed based on the network parameters as learned when executing the same network to solve one or more previous different cognitive tasks. For example, by initially training on 44 of the 45 tasks, with the limited amount of data provided per task, data transfer learning has been studied prominently for the task of visual oddity, the average test accuracy of models trained on previously unseen tasks with a small number of samples increases significantly. For example, when only 50 training samples per task are provided, the accuracy of the transfer learning model increases from 45.3% to 73.8%.

[0044] The above embodiments have been briefly described with reference to the accompanying drawings, and may accommodate many variations. Several combinations of the above features may be envisaged.

[0045] In these embodiments, neural network architectures can be utilized to solve cognitive tasks that additionally benefit from network regularization, particularly for large numbers of training epochs, by employing mutual information estimates as additional terms in the loss function characteristic of the network. Such regularization can provide benefits by stabilizing test accuracy and significantly reducing its variance. The neural network structure in some embodiments can be further extended by elements that mediate by mutual information estimates for the resolution of abstract reasoning obstacles. Supervised learning techniques can be used to solve visual oddity tasks, achieving an average task accuracy of 96.5%. Regularization based on mutual information estimates further improves the accuracy to 97.1%. Finally, the proposed model benefits from and / or enables transfer learning.

[0046] Next, an embodiment of the present disclosure according to another aspect may be embodied as a computerized system 1, which is generally designed to solve cognitive tasks that may learn abstract properties. The operational aspects of this system 1 have already been implicitly described with reference to the present method. System 1 will be described only briefly below.

[0047] Consistent with some embodiments, system 1 may comprise a processor and memory, with reference to Figures 1 and 4, the latter including computer-controlled methods that enable system 1 to form input unit 20, neural network modules 21-23, and output unit 24 during operation.

[0048] The input unit 20 may be designed to access the frame 10 characterizing the abstract property. The neural network modules 21-23 may be connected to the input unit 20. The neural network modules 21-23 may include a first neural network 21, a structural operation unit 21a, a second neural network 22, and a third neural network 23, as previously described.

[0049] More specifically, the first neural network 21 may be configured to generate a first embedding 31 when executed based on the frames 10 as accessed by the input unit 20. The structural operations unit 21a may be designed to form pairs 32 of the first embeddings 31 generated by the first neural network 21. The formed pairs 32 may correspond to pairs of frames 10. The second neural network 22 may be configured to generate second embeddings 33, 34 when executed based on data corresponding to the pairs 32 as formed by the structural operations unit 21a, which may capture the associative properties of the pairs 32 of frames 10. The third neural network 23 may be designed to create an output value 35 (which may be interpretable as a distance or similarity between the accessed frames 10) when executed based on the second embeddings 33, 34 as generated by the second neural network 22. Finally, the output unit 24 may be connected to the neural network modules 21-23 and configured to learn one or more abstract characteristics of the frame 10 based on the output values ​​35, as may be obtained from the third neural network 23, to solve a cognitive task.

[0050] In an embodiment, the system 1 may further be operable to form the first neural network 21 as a CNN, and the second neural network 22, or the third neural network 23, or both, may be formed as fully-connected neural networks, and, as previously described, the neural network modules 21-23 may further be configured to regularize one or each of the first neural network 21 and the second neural network 22 using a mutual information based regularization means, e.g., by applying a regularization term determined by the VIB method.

[0051] A computerized system may be suitably designed to implement the embodiments of the present disclosure as described herein. In that regard, some embodiments described herein are largely non-interactive and automatic. However, the methods described herein may also be implemented in some embodiments in either interactive, partially interactive, or non-interactive systems. Additionally, the methods described herein may be implemented in software (e.g., firmware), general-purpose hardware, dedicated hardware (e.g., graphical processing units), or a combination thereof. In some embodiments, the methods described herein may be implemented in software as executable programs, the latter executed by a suitable digital processing device. More generally, the embodiments of the present disclosure may be implemented using a general-purpose digital computer, such as a personal computer, a workstation, or the like.

[0052] For example, the system 1 illustrated in FIG. 4 generally represents a computerized unit 101, e.g., a general-purpose computer. In some embodiments, with respect to the hardware architecture, as shown in FIG. 4, the unit 101 may include a processor 105, a memory 110 coupled to a memory controller 115, and one or more input and / or output (I / O) devices 145, 150, 155 (or peripherals) communicatively coupled via an input / output controller 135. The input / output controller 135 may be, but is not limited to, one or more buses or other wired or wireless connections. The input / output controller 135 may also have additional elements such as controllers, buffers (caches), drivers, repeaters, and receivers that enable communication, which are omitted for simplicity. Furthermore, the local interface may include address, control, or data connections, or combinations thereof, that enable appropriate communication between the above components.

[0053] In some embodiments, the processor 105 may be a hardware device for executing software, particularly stored in the memory 110. The processor 105 may be any custom-made or commercially available processor, a central processing unit (CPU), a coprocessor among several processors associated with the computer 101, a semiconductor-based microprocessor (in the form of a microchip or chipset), or generally any device for executing software instructions.

[0054] Memory 110 may be any one or combination of volatile memory elements (e.g., random access memory) and non-volatile memory elements in some embodiments. Memory 110 may also incorporate electronic, magnetic, optical, or other types of storage media, or combinations thereof. It is noted that memory 110 may be a distributed architecture in which various components may be located remotely from one another but accessible by processor 105.

[0055] The software in memory 110 may include one or more individual programs, each of which includes an executable ordered listing of program instructions for implementing logical functions. In the example of Figure 4, the software in memory 110 includes some or all of the methods described herein according to some embodiments, as well as a suitable operating system (OS) 111. OS 111 may essentially control the execution of other computer programs and may provide scheduling, input-output control, file and data management, memory management, and communication control and related services.

[0056] The methods described herein may be in the form of a source program, an executable program (object code), a script, or any other entity that includes a set of program instructions to be executed. When in the form of a source program, the program may then be translated via a compiler, assembler, interpreter, etc., which may or may not also be contained within the memory 110, but is connected to the OS 111 to operate correctly. Furthermore, the methods may be written as an object-oriented programming language, which has classes of data and methods, or as a procedure-oriented programming language, which has routines, subroutines, or functions, or a combination thereof.

[0057] In some cases, a conventional keyboard 150 and mouse 155 may be coupled to the input / output controller 135. Other I / O devices 145-155 may include other hardware devices such as cameras, microphones, printers, and the like.

[0058] Additionally, I / O devices 145-155 further include devices that communicate both input and output. System 100 may further include a display controller 125 coupled to a display 130. In some embodiments, system 100 may further include a network interface or transceiver 160 for coupling to a network (not shown).

[0059] In some embodiments, the network transmits and receives data between the unit 101 and external systems. The network may be implemented in a wireless manner, for example, using wireless protocols and technologies such as WiFi, WiMax, etc. The network may be a fixed wireless network, a wireless local area network (LAN), a wireless wide area network (WAN), a personal area network (PAN), a virtual private network (VPN), the Internet, or other suitable network system, including devices that receive and transmit signals.

[0060] The network may also be an IP-based network for communicating over a broadband connection between the unit 101 and any external servers, clients, etc. In some embodiments, the network may be a managed IP network operated by a service provider. The network may also be a packet-switched network such as a LAN, a WAN, an Internet network, etc.

[0061] If the unit 101 is a personal computer (PC), a workstation, an intelligent device, etc., the software in the memory 110 may further include an input / output system (BIOS). The BIOS may be stored in a read-only memory (ROM) structure such that the BIOS may be executed when the computer 101 is started.

[0062] When the unit 101 is in operation, the processor 105 may be configured to execute program instructions stored in the memory 110, communicate data to and from the memory 110, and generally control the operation of the computer 101 in accordance with the software and the OS 111. The methods described herein and the OS 111 are loaded, in whole or in part, by the processor 105, typically buffered within the processor 105, and then executed. When the methods described herein are implemented in software, the methods may be stored on any computer-readable medium, such as the storage 120, for use by or in conjunction with any computer-related system or method.

[0063] Aspects of the present disclosure may also be embodied as a computer program product. Generally speaking, the computer program product may include a computer-readable storage medium having program instructions embodied therein, which may be executable by one or more processors to cause the computer system 1 to implement the operations described above, such as accessing the frames 10, executing the first neural network 21 to generate a first embedding 31 based on such input frames, forming a pair 32 of the resulting embeddings 31, inputting corresponding data into the second neural network 22, and executing the latter to generate a second embedding 33, 34, etc. The embeddings 33, 34 may capture the associative properties of the pair 32 of the frames 10. Finally, the third neural network 23 is calculated based on the second embeddings 33, 34 to obtain output values ​​35, based on which one or more abstract properties of the frames 10 may be learned to solve a cognitive task.

[0064] Thus, aspects of the disclosure may include a system, method, or computer program product, or combinations thereof, at any possible level of technical detail of integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions to cause a processor to execute aspects of the disclosure. The computer-readable storage medium may be a tangible device that can hold and store program instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, a magneto-optical storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer readable storage media includes: portable computer diskettes, hard disks, solid state drives (SSDs), random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or raised groove structures having instructions recorded thereon, and any suitable combination of the above.

[0065] A computer-readable storage medium as used herein should not be construed to include signals that are themselves transitory, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted over wires, but is not intended to otherwise limit the present disclosure.

[0066] The program instructions and data stored on the computer-readable storage medium in a non-transitory form may further be transmitted by a transmission medium or signal, such as an electrical, electromagnetic, or digital signal, which may be conveyed over a communication medium, such as a network or a wireless link, or a combination thereof. The computer-readable program instructions described herein may be downloaded from the computer-readable storage medium to the respective computing / processing device, or to an external computer or external storage device, over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions to the computer-readable storage medium in the respective computing / processing device for storage.

[0067] The computer readable program instructions for carrying out the operations of the present disclosure may be either source or object code written in any combination of one or more programming languages, such as assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state set data, configuration data for an integrated circuit, or object oriented programming languages ​​such as Smalltalk, C++, and procedure oriented programming languages ​​such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be made to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may execute computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry to perform aspects of the present disclosure.

[0068] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to the disclosed embodiments. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0069] These computer readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus that causes a machine to generate means for implementing the function / operation identified in a block or blocks of the flowcharts and / or block diagrams, and these computer readable program instructions may be stored on a computer readable storage medium that may cause a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer readable storage medium having stored thereon instructions has an article of manufacture including instructions that implement aspects of the function / operation identified in a block or blocks of the flowcharts and / or block diagrams.

[0070] Furthermore, the computer readable program instructions may be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be executed on the computer, other programmable apparatus, or other device to create a computer-implemented process, such that the instructions executed on the computer, other programmable apparatus, or other device implement the function / act identified in a block or blocks of the flowcharts and / or block diagrams.

[0071] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of the systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions that includes one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions described in the blocks may occur out of the order described in the figures. For example, two blocks shown in succession may in fact be performed as one operation or may be performed simultaneously, substantially simultaneously, partially, or fully overlapping in time, or the blocks may be performed in reverse order depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, as well as combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs the specified functions or acts or a combination of dedicated hardware and computer instructions.

[0072] Although the present disclosure has been described with reference to a limited number of embodiments, variants, and accompanying drawings, it will be understood by those skilled in the art that various modifications may be made and equivalents may be substituted without departing from the scope of the present disclosure. In particular, features (such as devices or methods) described in a given embodiment, variant, or illustrated in a drawing may be combined or substituted with other features in another embodiment, variant, or drawing without departing from the scope of the present disclosure. Thus, various combinations of the features described with respect to any of the above embodiments or variants are contemplated and remain within the scope of the appended claims. Also, many minor modifications may be made to adapt a particular situation or material to the teachings of the present disclosure without departing from its scope. Thus, it is not intended that the present disclosure be limited to the particular embodiments disclosed, but rather that the present disclosure will include all embodiments that are within the scope of the appended claims. Also, many other variations may be contemplated other than those expressly mentioned above.

Claims

1. 1. A computer-implemented method for solving cognitive tasks, including learning abstract properties, the method comprising: accessing a dataset from an input unit, said dataset characterizing said abstract property, said dataset including a plurality of data; inputting the accessed data set into a first neural network of a neural network module; executing the first neural network to generate a first embedding for each datum of the plurality of datum; forming pairs of the generated first embeddings, the formed pairs being pairs of the first embeddings corresponding to pairs of each data of the plurality of data with each other data of the plurality of data; inputting data corresponding to the formed pairs into a second neural network of the neural network module; running the second neural network to generate, for each datum of the plurality of data, a second embedding that captures the pairwise associative properties of the first embedding; executing a third neural network of the neural network module based on the second embedding to obtain an output value for each data of the plurality of data; learning one or more abstract properties of the data set based on the output values ​​for each data of the plurality of acquired data to solve a cognitive task; A method comprising:

2. The method of claim 1 , wherein the first neural network is a convolutional neural network.

3. The method of claim 2 , wherein each of the second neural network and the third neural network is a fully-connected neural network.

4. 4. The method of claim 3, further comprising regularizing at least one of the first neural network and the second neural network using a mutual information based regularizer.

5. The method of claim 4 , wherein using the mutual information based regularizer comprises applying a regularization term determined by minimizing a mutual information based function.

6. The method of claim 5 , wherein the regularization term applied is determined according to a variational approximation to an information bottleneck function.

7. 7. The method of claim 2, wherein the convolutional neural network includes at least five layers.

8. 8. The method of claim 3, wherein the second neural network includes four layers.

9. The method of claim 3 , wherein the third neural network includes three layers.

10. forming the pairs of the first embedding comprises concatenating the pairs of the first embedding; the data corresponding to the formed pairs includes concatenated values ​​forming a single vector for each of the pairs of the first embedding.

10. The method according to any one of claims 1 to 9.

11. 11. The method of claim 1, wherein all outputs from the second neural network corresponding to the same one of the data sets are summed before being input to the third neural network for their execution.

12. The method of claim 1 , wherein the accessed data sets each correspond to a frame characterizing the abstract property.

13. The method of claim 12 , wherein the cognitive task is a visual oddity task and one of the accessed frames represents an oddity.

14. 14. The method of claim 13, wherein learning the one or more abstract characteristics comprises applying a softmax function over the obtained output values ​​to determine a probability that each frame is an oddity.

15. 15. The method of claim 1, wherein each of the first neural network, the second neural network, and the third neural network is executed based on neural network parameters as learned during execution of the first neural network, the second neural network, and the third neural network to solve one or more previous different cognitive tasks.

16. 1. A computerized system for solving cognitive tasks involving learning abstract properties, the system comprising: an input unit adapted to access a data set characterizing said abstract property; a neural network module connected to the input unit, the neural network module including a first neural network, a structural operation unit, a second neural network, and a third neural network; The dataset includes a plurality of data; the first neural network is configured to generate a first embedding for each datum of the plurality of datum upon execution based on a data set as accessed by the input unit; the structural operations unit is configured to form pairs of the first embeddings generated by the first neural network, the pairs being the first embedding pairs corresponding to pairs of each datum of the plurality of data with each other datum of the plurality of data; the second neural network is configured to generate, for each datum of the plurality of data, a second embedding that captures associative properties of the pairs of the data set, when executed on data corresponding to the pairs as formed by the structural operations unit; a neural network module, the third neural network configured to generate an output value for each datum of the plurality of data upon execution based on the second embedding as generated by the second neural network; and an output unit, coupled to the neural network module and configured to learn one or more abstract properties of a data set based on the output values ​​for each data of the plurality of data as obtained from the third neural network, to solve the cognitive task; The system is operable to form

17. 17. The system of claim 16, wherein the system is further operable to form the first neural network as a convolutional neural network and to form each of the second neural network and the third neural network as a fully connected neural network.

18. 20. The system of claim 17, wherein the neural network module is further configured to regularize at least one of the first neural network and the second neural network using a mutual information based regularizer.

19. 20. The system of claim 18, wherein the neural network module is further configured to apply a regularization term determined according to a variational approximation to an information bottleneck function in regularizing at least one of the first neural network and the second neural network.

20. 1. A computer program for solving cognitive tasks, including learning abstract properties, the computer program including program instructions embodied thereby, the program instructions being executable by one or more processors; accessing a dataset characterizing the abstract property, the dataset including a plurality of data; inputting the accessed data set into a first neural network and executing the first neural network to generate a first embedding for each datum of the plurality of data; forming pairs of the generated first embeddings, the formed pairs being pairs of the first embeddings corresponding to pairs of each data of the plurality of data with each other data of the plurality of data; inputting data corresponding to the formed pairs into a second neural network and executing the second neural network to generate, for each datum of the plurality of data, a second embedding that captures associative properties of the pairs in the data set; executing a third neural network based on the second embedding to obtain an output value for each data of the plurality of data; learning one or more abstract properties of the dataset based on the output values ​​for each data of the acquired plurality of data to solve the cognitive task; A computer program that produces the following:

Images