A neural inverse technique for pattern classification and memory recall by ai systems
Patent Information
- Application Number
- PCT/IN2024/050573
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-23
- Filing Date
- 2024-05-17
- Publication Date
- 2024-11-28
Smart Images

Figure IN2024050573_28112024_PF_FP_ABST
Abstract
Description
[0001] A NEURAL INVERSE TECHNIQUE FOR PATTERN CLASSIFICATION AND MEMORY RECALL BY Al SYSTEMS
[0002] In an earlier patent [1], we described a method to develop a system which can be used for Automatic Learning. In this invention, we will now describe the development of a system that can facilitate easy classification of data and memory recall, so that Al systems that can leam from data can be developed and deployed, the speed of the systems developed by this invention, is very high and therefore can be used for real time classification.
[0003] The description of this new invention relies on the techniques described earlier to by the present author and hence these references can be consulted so that this write up is better understood, furthermore an additional concept which is known as BAM, First introduced by Bart Koso , is also used in this invention. The process of BAM is used to recall the similarity of data which is currently presented, to some other pattern of data which is available in memory. For example, the present data may be the word 0 botleD , this word would be compared with other words by a procedure called BAM processing and the word closest to this word say 0 bottleD , would be extracted, this word bottle would be called the 'recalled data' or the word recalled from memory. A similar process is extracting an original image of the face a person from the memory when given a slightly different face image or distorted face image of the said person. This extraction of the original image is done by producing a sequence of images starting from the given image in such a manner that the original images in the sequence is close to the recalled, original image. This process is called BAM processing. However, when the number of patterns are very numerous and the dimensions of the data are huge (e.g. in images / video) and where the classification is being done by using Artificial Neural Networks (ANN) using Deep learning systems involving numerous layers of processing elements, BAM processing is simply not possible. This is also because of the presence of activation functions such as Rel and sigmoid functions which makes the mappings used in conventional ANN and Deep learning non-linear and thus not amenable to BAM processing. In this invention because of the use of partitioning by hyper planes used in ref [1], where we use only linear mappings, it is shown that one can use the method akin to BAM processing and it is possible to recall and memory in huge data. Essentially this invention does the inverse of what the original network does. The original ANN does a forward process which accepts an input, processes this input and then gives a desired output. But in the inverse process we are given a 0 desired outputD and then we try to answer the question. What should the input be, so that the Neural Network gives this 0 desired output
[0004] This invention is concerned with demonstrating how this inverse process, which we term as the 0 Neural Inversion!] algorithm can be performed without any iterations. The discovery of this algorithm enabled us to invent an Al system which very e-ciently and speedily (using minimal of memory resources) handles and works on huge amounts of data and hence can be use by ANN based learning systems. This invention can be used for pattern classification involving huge amount of data, for memory recall, memory retrieval in video and audio images (by adopting techniques described in [4] by this author) and also can be used for other applications called generative Al and is thus a major innovative step in the field of Al concerned with Artificial Neural Networks (ANN). Before we proceed to describe the present invention concerning Neural Inversion, it is necessary that we briefly describe this author's previous invention namely the automated non-iterative neural learning system, which we proceed to do in the next section.
[0005] Brief Description of Previous Invention and method adopted for Automated learning by an ANN
[0006] It was shown in our invention [1] that the architecture of an ANN can be systematically determined when we are given a sufficient number of training samples. The method uses the technique of separation of points in high n-dimension space by using the algorithm, described in Refs [2] to
[0004] , which we call the KE sieve algorithm or as the KE's algorithm.
[0007] An ANN classi_cation system requires as input a set of numbers (or vectors) which is fed to the system; the system is then supposed to output another set of numbers (or vectors) which determine the class the input vector belongs to. For example, if the input numbers (vector) represent the image of the face of an individual (in a digitized format) then the correct output vector would be the ID or the name of the individual's face. The main task of formulating or designing a good Al system is to ensure that after sufficient training and iterations the Al program accurately outputs the correct vector for every input vector that it is presented with. For the purpose of our understanding let us assume that during the training of the ANN we are given a very large number of training samples (say, each sample is a face) and each sample is a vector of n dimensions (xl, x2, x3, xn) and the output of the network should be another k dimensional vector (vl, v2.v3, vk), for example, the input vector could be the pixels values of a face and the output could be the name of the individual whose face was given as input or equivalently the passport photograph. Each of the input samples can be treated as points in n- dimensional X-space and similarly each of the corresponding output vectors can be treated as points in dimensional V-space.
[0008] A typical ANN architecture is shown in the Figure 1. This architecture can be represented as a sequence of mappings:
[0009] X - space — S - space — U - space — V - space It may be noted that during training we are given, for every point in X- space, the corresponding image point in V-space. The task of the algorithm is to completely determine the intermediate mappings i.e, the corresponding image points in S-space and U-space and nally the correct image point in V- space. The determination of this mapping sequence has occupied researchers for more than 40 years and a number of approximate techniques such as the back-propagation algorithm and deep learning methods have been formulated, however all these methods are highly iterative and do not determine (ab initio) the architecture (the number of processing elements and layers) and occupy enormous amounts of memory and computer time for training. However we have shown in [1], that our invention circumvents all these di-culties and also is non-iterative and deterministic.
[0010] Non-iterative Neural Processing: In this method the ensemble of training data is assumed to be in the form of n-dimensional sample vectors. We assume that we are given a very large number of training samples, say N, and each of these N samples is a n-dimensional vector which is represented by (xl, x2, x3, ..., xn) as stated above, and can be considered as a point in n- dimensional space. Therefore the entire ensemble of train-data constitute N points in the X- space of n-dimensions.(Note: N and n are different numbers unrelated to each other). We then use the KE sieve algorithm to separate i.e. partition each of the N points from every other point by hyperplanes, See Fig 2 and Fig 3. Let us say that by making su-cient number of passes in KE's algorithm we have generated q hyperplanes. The algorithm determines the equations of each of these hyper-planes which can be represented in the form: am, 1x1 + am, 2x2 + ... + am,nxn + 1 = 0 where m = 1, 2, q. All the coe-cients am,j(m = 1, 2, ,.q)and (j = 1, 2, .., n) are determined by the algorithm. (We have 'normalised' the above equations by divided out each equation by its constant term, so all the constants become 1). These coe-cients become the weights of the q processing elements in the rst layer and hence are determined without iterations. If one chooses any point P in X-Space, whose coordinates are (xl, x2, x3, ..., xn) and if this point is substituted into the q Eqs (1) above, we would get the following relationships: am, 1x1 + am, 2x2 + ... + am,nxn + 1 = sm where m = 1, 2, .., q. . Because of the way the coe-cients of the hyperplanes have been normalized, see Ref
[0001] , it so turns out that smis the perpendicular distance of the point P from the mth hyperplane, see Fig 2 and Fig 3. The above Equations thus determine the output (si, s2, s3, .., sq) of the rst layer of the processing elements shown in Fig 1, when the input is (xl, x2, x3, ..., xn) which are the coordinates of point P. One can write Eq(2) in matrix form as: A x= s where A is a q * (n + 1) matrix with coe-cients am,j as de ned in Eq (1),
[0011] (the last colunmn of A is 1 because of our normalization), xe and se are (n + 1)- dimensional and q-dimensional column vectors, representing the rhs of Eqs. (1) or (2). We can also consider Eq (3) as a mapping of a point P with coordinates (xl, x2, x3, ..., xn) to a point P' whose coordinates are (si, s2, s3, .., sq) which are represented as outputs of the rst layer of processing elements when the input is the coordinates of point P viz. (xl, x2, x3, ..., xn).
[0012] To obtain the weights of the processing elements in the second and third layer we proceed similarly. Now, just as all possible inputs P, each represented by n- dimensional points in X-space, were separated by q planes; we can visualize another mapping wherein all the image points P' in S-space can be similarly mapped to U-space, by partitioning all image points such as P' by using hyperplanes in q dimensional S-space generated by once again using KE's algorithm in S-space; suppose that this algorithm determines that r hyperplanes partition all the P' points then we can similarly perform a mapping of every point P' to PO from S-space to U-space viz. (si, s2, sq) — (ul, u2, ur). Where uj is the perpendicular distance of the image point P' in S-space from the jth hyperplane (j = 1, 2, r). This mapping can now be represented by an ANN architecture, which is similar to that shown in the Figure above to the second layer with q inputs (si, s2, s3, sq) and r outputs (ul, u2, ur). Even in this case all the weight of the processing elements in the new layer will be known as they are nothing but the coe-cients of the hyper planes drawn in the S- space by KE's algorithm. This is shown in Fig 4:
[0013] This is the general scheme of automatically determining the architecture of a NN network given all feasible data points each of which is n-dimension in X-space. Just for a further illustration, one can extend the architecture (if necessary) by once again using KE's algorithm to separate points in U-space, this would result in (say) k-hyperplanes, we would again have a well defined mapping of Points PO in r-dimensional U-space to a point PO ' in k-dimensional V-space. This mapping can again be represented by a suitable ANN. Putting all the mappings together i.e. X=Space 0 S=space 0 U=space 0 V = space The resulting Final ANN architecture would be as shown above: Fig 1.
[0014] Mathematical Details of Present Invention
[0015] It is now possible to describe the present invention, which is an 0 Inverse Neural Network System!] . This invention affirmatively answers the question Can one make an Neural Network system which given the out put of a neural network it can find out what the input to the Neural Network should be such that it can generate a given output. Such a system has many uses in BAM processing that is used in Al learning systems.
[0016] We will now describe the Invention of a system that performs this task which we call 0 Neural Inversion!] . To do this we will work step by step: In Eq (3) viz. A xe = s e
[0017] , suppose we are given the right hand side vector s e is it possible to nd the input vector x e which can ensure that Eq (3) is satis ed? If A was a square matrix this question has a simple answer, we can invert A and write: xe
[0018] = A-lse
[0019] . But the problem is that A will most often be a rectangular matrix and therefore we cannot perform this inverse. We therefore employ a technique called Moore-Penrose inversion, we multiply both sides of Eq (3), from the left, by the transpose of AT to get:
[0020] AT A x e
[0021] = ATse
[0022] (4)
[0023] Now since A. AT is a square matrix, we can can invert it and mutiply both sides of Eq. (4) to obtain: xe
[0024] = [A.A]-lATse
[0025] (5 this is the given expression which determines the vector x e when the output vector s e is known. It may be noted since the Neural networrk is known and well de ned, all the components of the matriz A are known and therefore Eq(5) can be used. The only requirement for getting a proper Moore-Penrose inverse is that n < q.
[0026] We will now 0 invertD the mapping (s 1 , s2, .. , sq) — (ul , u2, .... , ur), which is the calculation processes represented by the second layer. The neural calculation in the second layer is similar to Eq (2) except that the rectangular coe-cient matrix B has 0i,j as coe-cients of the hyperplane (i = 1, 2, r) and (j =
[0027] 1, 2, q). The relationship is:
[0028] B s e
[0029] = u e
[0030] (6) we can now 0 invertD Eq(6) similarly by using the Moore-Penrose technique to get: se
[0031] = [B BT ]-lBTue
[0032] (7) in this case for a proper 0 inverese we must have q < r Putting together equations (7) and (5) we obtain the expression : xe
[0033] = [A.A]-1AT [B.BT ]-lBTue
[0034] (8)
[0035] This is the wanted Neural Inverse of a two layer Neural network which determines the input x e when the output u e is given. We must have n < q < r for an accurate inverse.
[0036] The General Case of Neural Inverse: When there are of more than two layers
[0037] The above equation states the Neural Inverse when there are two layers of processing elements, if there are three layers then we need to make another mapping such as: (ul, u2, ...., ur) — (vl, v2, ..., vk) involving a coe-cent rectangular k * (r + 1) matrix C with coe-cients yij , (i=l,2,..,k), (j=l,2,..r). A similar treatment gives a relationships e
[0038] = [C.CT ]-lCTve. Thus giving the nal Neural
[0039] Inverse of a three layer neual network as : xe
[0040] = [A.A]-1AT [B BT ]-lBT [C.CT ]-lCTve
[0041] (9) which determines the input x e when given the output v e
[0042] . determines the input xe when given the output v e
[0043] .viz. n < q < r < k.
[0044] We can write down the Neural Inverse for a general m layer neural network wherin the coe-cient of the jth layer denoted byAjwith (j = I, 2, ,.,m). This is given by: xe
[0045] = [A1.A1]-1AT1
[0046] [A2.AT2
[0047] ]-lAT2
[0048] > [Am. AT m]-lAT m w e
[0049] (10) an expression which determines the input x e when given the output w e
[0050] . Of course, the condition that the number of hyperplanes drawn in the space associated with the ith layer shown be equal or greater than the number of hyperplanes drawn in the previous (i-l)th layer must still hold. This condition can be easily ful lied by making an adequate number of passes while using KE's algorithm for the creation of partitioning hyperplanes in each space.
[0051] With this the technical description of the invention is complete. The various steps delineated above can be followed to develop an Al system, to automatically build a Neural Al system without iterations, where in the architecture of the ANN system is completely determined, along with the number of processing elements in each layer and all the weights of the processing elements. This neural system by the above described method of Neural Inversion can be used to not only classify huge amounts of data but also facilitate easy recall of data from memory. Because of its non-iterative and deterministic techniques that is used by this invention and also because of its speed and far less use of memory, it is an alternative to Back propagation and Deep learning algorithms that are used in Al in the present times.
[0052] Usage of Present Invention
[0053] We now brie y describe the usage of the present invention, viz. the Neural Inverse.
[0054] The method of Bidirectional Associative Memory (BAM), is very useful in recognizing similar patterns say associating two diO erent images of the same person or by associating two descriptions of the same object for example associating the image of a telephone with the voice or the uttered word 0 telephone!! . Suppose we are given a huge amount of training data (say images), and we need to train an ANN, to recognize patterns. The steps taken are as follows: Step 1 : An ANN model is rst formulated using the train data by using the method of the previous invention [1] and the complete architecture with all the necessary layers and the weight for each processing element are determined by using KE's algorithm, Step 2: Then by using the Moore Penrose technique the matrices that de ne the Neural Inverse are calculated. By this method the Neural Inverse expressions are fully determined.
[0055] For a two layer ANN architecture the Neural Inverse expression connecting the output vector to the plausible input vector is given by Eq. (8).
[0056] Similarly for a three layer the corresponding expression is Eq. (9) and for a general m-layer network it is Eq. (10).
[0057] Using the Neural Inverse for BAM Processing:
[0058] Suppose we have completed Steps 1 and 2, and now we are given an input vector and we want to nd that test vector which is close to the input vector so that we can know its class. We will illustrate the method for a two layer ANN the procedure for the general case is similar.
[0059] Let us think of the input vector say a face image and we wish to nd the closest face in the data base. We proceed with the BM process as follows:
[0060] (a)We rst use the ANN obtained in Step 1 above as a feed forward network. Now we feed forward every train image we nd corresponding output image.
[0061] That is for every train image x e
[0062] (p) we know the output vector u e
[0063] (p) , where p = 1, 2, 3, ..N) where the total number of train images is N.We need to do all this only once and store all the vectors ue
[0064] (p) in memory storage. To save memory the Orientation Vectors (OV's) can be stored see [1], instead of the vector itself.
[0065] (b) Let us call the input image as x e
[0066] , we rst use the ANN obtained in Step
[0067] 1 above as a feed forward network and for the given input we nd the output vector u e
[0068] . We then nd the nearest image vector among all the vectors ue (р) which is closest to u e
[0069] . This can be done by using a k-nearest neighbor search or by nding the nearest OV's, let us say that the nearest vector is u e
[0070] (m) •
[0071] (с) Having found the nearest image vector u e
[0072] (m) , we now use Neural Inverse to nd the image z e given by Eq(8): ze
[0073] = [A.A]-1AT [B.BT ]-lBTue
[0074] (m) (ll)
[0075] We now interpret z e as the closest image to the input image x e
[0076] Perhaps it is perhaps worthwhile to mention, here that at step (b) instead of taking the nearest vector u e
[0077] (m)among k nearest neighbors, we had taken the average vector u e
[0078] (A), and substituted this vector on the rhs of (11) instead of ue
[0079] (m)we would get another vector (say) z e (N) which is a new face which is not among the train samples. We thus use this technique to generate new samples called generative Al.
[0080] The above calculation process is called BAM processing by using Neural Inverse.
[0081] In the next section, for the purpose of demonstrating the e-cacy and accuracy of this invention we present the results applied to an actual problem involving images in video.
[0082] 3.4 Other Uses of the Present Invention
[0083] This invention can be used in a variety of ways and has many applications, for example classi cation problems involving satellite imagery and images in the the medical eld such as x-ray radiography and MRI images; classi cation in disease diagnostics, classi cation involving speech processing. Even in neuroscience where experiments and simulation studies are being done on the human and mammal brains ANN techniques are used see [8] to [9],
[0084] 4 EXAMPLE PROBLEM
[0085] Dataset:
[0086] To illustrate the application of our Invention of Neural Inversion, we consider a cartoon video data set. This data set includes ve cartoons. The description of the data set is shown in Table 1.1. We have collected the Oth, 10th, 20th, 30th... (Every 10th frame or approx ten percent of frames) frames from each cartoon video and consider them as train frames and the remaining all frames we consider as test frames. There are 14,179 train frames and 1,27,594 test frames. We have reduced each frame image to 30 x 30 pixels hence the number of attributes is 900, hence this is the dimension of X-space. The data set is split in the ratio of 10:90 as train and test. All train and test data frames pixel values are normalized (values range -1 to 1 oat). We rst construct the NN model by using KE's algorithms using only the train frames and KE's algorithms draws adequate number of hyperplanes in X-space, S-space and U-space.The data below shows the number of Train and Test frames: (as shown in table 1) The Details of the videos are available in th following les:
[0087] Cartoon Dataset: 1 Videol (Mickey Mouse):https: / / youtu.be / yH58Xv204sM, 2 Video2 (Tom and Jerry):https: / / www.youtube.com / watch?v=QM2-MXlLz-A& t=Os, 3 Video3(Donald Duck): https: / / youtu.be / J61Oq922JII, .4. Video4(Mr Bean): https: / / youtu.be / EGlLMGUCUos, 5 Video5 (Motu Patlu): https: / / youtu.be / ry4mm6R9tNMhttps: / / youtu.be / yH58Xv204sM
[0088] Details of the Steps followed for Neural Inverse:
[0089] The above data set was used to rst construct the NN model by using a total of 14,179 images (frames). We consider each of the rest of the images namely 1,27,594 frames as test images. That is we take one of test images and perform a Neural Inverse on this image and get back the input image and then examine if it is close to the actual images. For every test image the input image is fairly closely recovered as can be seen from the sample results, thus demonstrating that Neural Inverse is very accurate.
[0090] The KE's algorithm was used the total number of 14,179 train points (test frames) by hyper planes, in X-space and S -space. The neural network architecture consisted of an input layer of 900 numbers (xl, x2, x3, ..., xn) and the out put from the U-space the following data gives the details: ( table 2)
[0091] From the above deployment of KE's algorithm matrices A and B were constructed and the Neural Inverse given by Eq: (8) viz. xe=
[0092] [
[0093] A.A]-1AT [
[0094] B BT ]-lBTue
[0095] (8') which connects the output image u e assumed given, to the input vector x e calculated.
[0096] We considered one of the test images as u e
[0097] ,and substituted it in the above Eq.
[0098] (8') and calculated the vector x e
[0099] , this vector was compared with expected original image, it was found that it was very close to the original image. This exercise was done for all the 1,27,594 test images and in every case the 0 inversed ImageO was very close to the expected original image thus proving the e-ciency and e-cacy of the invention. Some typical images are shown below in the Figure 5, and summarizes our results.
[0100] 5 REFERENCES
[0101] 1. K.Eswaran (2023): 0 A Non-iterative Automated Neural Learning Al System!] , Patent led on 8 April 2023, Docket No: 35577; Ref. Application No.: 202341026335; App. Number: TEMP / E- 1 / 29847 / 2023 -CHE, https: / / www. researchgate.net / publication / 369912150_A_NON-ITERATIVE_AUTOMATED_NEURAL LEARNING AI SYSTEM
[0102] 2. K. Eswaran (2016)): An Universal Classi er for Learning and Classi - cation of Data with Uses in Machine Learning, International Patent led under PCT / IN2017 / 000096, on 28th of April 2017, with our the Indian Patent Application No. 201641015775, with Priority date of 05 / 05 / 2016
[0103] 3. K. Eswaran (2015) : A Neural Architecture to be used for Pattern Recognition and Deep Learning, Indian Patent Status: Filed, Indian Patent App.
[0104] Nos. 2669 / CHE / 2015 15
[0105] 4.K. Eswaran (2017): Robotic Device driven by Articial Intelligence (Al) capable of correlating historic events with that of present events for Indexing of imagery captured in Camcorder device and retrieval. Indian Patent Filed on 7th August 2018 No: 2017- 4102-8240 with Priority date9 / 8 / 2017.K. Eswaran (2017): Robotic Device driven by Articial Intelligence (Al) capable of correlating historic events with that of present events for Indexing of imagery captured in Camcorder device and retrieval. Indian Patent Filed on 7th August 2018 No: 2017- 4102-8240 with Priority date9 / 8 / 2017.
[0106] 5. Bart Kosko (1988): Bidirectional associative memories. IEEE Transactions on Systems, Man, and Cybernetics, 18(1), 49-60.
[0107] 6. J. Kim and C. Lee (2018): "Bidirectional Associative Memory Using Recurrent Neural Networks." IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 8, pp. 3459-3467, Aug. 2018.
[0108] 7. X. Huang, J. Li, and X. Li. (2019): "Deep Bidirectional Associative Memory Networks for Complex Pattern Recognition." IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 7, pp. 2178-2187, July 2019
[0109] 8. S. Srinivasan, A. Banerjee & A. Basu (2018): A survey of biologically- inspired models of episodic memory. Neural Networks, 107, 54-85.
[0110] 9. Graves, A., Wayne, G., & Danihelka, I. (2016): Hybrid computing using a neural network with dynamic external memory. Nature, 538(7626), 471-476
Claims
CLAIMSI claim1) A Neural Network based algorithm that can be used for a large variety of classi cation problems wherein It is non-iterative, unlike other Neural Network (NN) algorithms like Deep Learning and Back-propagation algorithms which are highly iterative and require huge amounts of training and testing time.2) The Neural Network based algorithm as claimed in claim 1, automatically discovers the architecture of the neural network: the number of processing elements necessary for each layer, the weights of each layer, all these characteristics pertaining to the architecture of the NN, are determined once the training samples, their number and and dimension are known. There is no other NN algorithms that has this property; all other ANN systems have to use trial and error techniques for finding a suitable architecture.3) The neural network as claimed in claim 1, posseses an architecture which is compact, the number of processing elements and layers are much less than any other NN, this is because we require much less hyper-planes, therefore fewer processing elements to segregate the data and also the number of layers necessary, even for very good accuracy is very few.4) A Neural network as claimed in claim 1, with the additional capability of performing BAM processing by adopting a procedure which has a solid mathematical proof and derivation (see subsection 3.2)5) A Neural network as claimed in claim 4, with BAM processing that is new unclassi ed data can be fed in a feed forward manner or feed backward manner and thus making the ANN useful as an auxiliary memory, (see subsection 3.3) (6) The Neural network as claimed in claim 1 and claim 4 takes much less time for training than other NN algorithms.Dated this 23rd day of May 2023
Citation Information
Patent Citations
Discrete BAM neural network system based on FPGA
CN111091187A
Method and electronic system for a non-iterative training of a neural network
WO2022194757A2
