Similarity determination method, similarity calculation unit, divergent learning network, and neural network execution program.

Abstract

In the present invention, one or more inputs occur, with one of two different values L and H being inputted as each input, and when a value for the i'th input of a learning phase is expressed by xi and a value for the ith input of a similarity assessment phase is expressed by yi, the i'th input is assigned a value wi, with one of the two values L and H being set to the value wi. In the learning phase, the weight value wi assigned to the i'th input is set to the value of xi. In the similarity assessment phase, the number of inputs where xi equals the value H, the number of inputs where wi and yi both equal the value H, and the number of inputs where the value of yi equals the value H are calculated. A value obtained by dividing the number of inputs where wi and yi both equal the value H by the sum of the number of inputs where wi equals the value H and the number of inputs where yi equals the value H is calculated and outputted as a similarity degree that represents the extent of similarity.

Description

[Technical Field]

[0001] This invention relates to a similarity determination method, a similarity calculation unit, a diffusive learning network, and a neural network execution program. [Background technology]

[0002] In recent years, artificial intelligence technology using artificial neural networks has developed, leading to various industrial applications. These neural networks are characterized by the use of a network of perceptrons, which are models of nerve cells. Neural networks perform calculations based on input to the entire network and output the calculation results.

[0003] The perceptrons used in artificial neural networks are based on advancements of early models of nerve cells.

[0004] Figure 51 shows the operation of perceptron 200 with variable constant input. As shown in Figure 51, the N+1 input values ​​are b, x1, x2, ... x N The following is input to the perceptron 200. Of these, N are external inputs to the entire neural network, and input i is input value x i The input is . b is a constant value held within the neural network. Also, as the output of the neural network, one output y is output from the perceptron. For input i (i=1,2,...N), a value called w is output. i These are assigned (hereinafter referred to as synaptic weights). In this case, the output y is expressed by equation (1).

[0005]

number

[0006] Here, f(·) represents an activation function. As activation functions, non-linear functions such as the sigmoid function and the tanh function, and ReLU (Rectified Linear Unit function) are often used. In Equation (1), w i x i To eliminate the notation difference between w and b and make the equation easier to read, for the constant input being 1, a circuit as shown in FIG. 52 where the synaptic weight w0 for it is b, and the following Equation (2) are often used. FIG. 52 is a diagram showing the operation of the perceptron 200 that generalizes the expression of the input · synaptic weight.

[0007]

Number

[0008] As shown in Equation (2), the value passed to the activation function is calculated based on the input value, and the value that becomes the output is calculated by the activation function. In the following description, the value passed to the activation function will be called the activation degree. When the activation function is represented by f(a), a is the activation degree. Usually, when performing machine learning using an artificial neural network, a network as shown in FIG. 53, where one or more perceptrons 200 are hierarchically connected, is used. FIG. 53 is a diagram showing a multi-layered artificial neural network.

[0009] An artificial neural network has a plurality of combinations of input values x i (i = 1, 2,..., N). Represent one combination by j, and when considering each of the input values x i (i = 1, 2,..., N) as a component of a vector, the vector composed of x i (i = 1, 2,..., N) will be represented as x j Here, the components of x j are x j =(x j1 , x j2 ,..., x jN ) T And (x j =(x j1,x j2 ,…,x jN ) T The T included in this will be represented as (meaning that the vector is converted to a column vector).

[0010] Next, each x j For this, the target value l j Prepare multiple sets with assigned values, and use these as training data, w i The value of is determined. This determination is performed by minimizing the error across the entire training data, using the difference between the value calculated by the neural network and the target value as the error.

[0011] In machine learning methods using this type of artificial neural network, the training data itself is not stored within the neural network. On the other hand, there is a machine learning method called the k-nearest neighbors method, which stores training data, calculates the similarity between the input and the stored patterns, and outputs labels using the k most similar stored patterns. This k-nearest neighbors method has been shown to enable relatively stable learning even with a small amount of training data, and has advantages depending on the application.

[0012] Furthermore, as described in Non-Patent Literature 4, the brain is thought to possess a pattern completion function that, when multiple external inputs are received, allows for the complete recall of a similar memory already established in the brain, even if a perfectly matching input pattern is not memorized. Finding a memory similar to an external input pattern is one of the functions of human intelligence, and calculating the similarity between input and memory patterns provides fundamental information for finding the most similar memory. Therefore, the technology for calculating the similarity between input and memory patterns is important as a component technology for realizing this pattern completion.

[0013] As described above, neural networks are a fundamental technology for artificially realizing intelligent functions that are thought to be inherent in humans, such as machine learning and the recall of similar memories.

[0014] In neurons and neural networks that form the basis of perceptrons and artificial neural networks, techniques exist for learning previously inputted information, storing that information, and comparing that memory with current input to determine similarity. These techniques include the Associative Network described in Non-Patent Documents 1, 2, and 3. The neurons used in the Associative Network and examples of the Associative Network are shown in Figures 54 and 55, respectively.

[0015] Figure 54 shows an example of a simple Associative Network. In Figure 54, neuron 300 is represented by a combination of an arrow and a black triangle. The upper side of the triangle (the side without the arrowhead) is the input of the neuron, and the lower side of the triangle (the side with the arrowhead) is the output of the neuron.

[0016] Suppose we have a neuron 300 in a neural network that changes to a firing state (meaning the membrane potential of the nerve cell rises and exceeds a threshold) when a certain input A is applied. If we repeatedly apply input B simultaneously with input A, we will eventually reach a point where neuron 300 changes to a firing state when only input B is applied. This phenomenon is explained by Hebb's rule, which states that the synaptic connection between input B and neuron 300 is strengthened when the neuron generating input B and neuron 300 fire simultaneously. This phenomenon, where neuron 300 fires when only input B is applied, is called classical conditioning, and input A and input B are called the unconditioned stimulus and conditioned stimulus, respectively.

[0017] Figure 55 shows an example of an Associative Network that includes multiple unconditioned stimuli. Figure 55 illustrates the case where different unconditioned stimuli P, Q, and R are related to a single conditioned stimulus C through classical conditioning. Unconditioned stimulus P and conditioned stimulus C are input to neuron 301. Unconditioned stimulus Q and conditioned stimulus C are input to neuron 302. Unconditioned stimulus R and conditioned stimulus C are input to neuron 303.

[0018] Next, we will explain the technique for determining similarity using an Associative Network. Figure 56 illustrates the 300 neurons that make up the technology for determining similarity using an Associative Network. Figure 56 shows the setting of synaptic weights in a simple Associative Network. Neuron 300 in Figure 56 receives four input values ​​x1, x2, x3, and x4. Here, input i receives input value x i The following inputs are received. These input values ​​are either 0 or 1. This relates to the state of the preceding neuron that generates each input, with 0 representing the non-firing state of the preceding neuron (a state in which the membrane potential of the nerve cell has not reached the threshold membrane potential) and 1 representing the firing state of the preceding neuron. This corresponds to the fact that in the non-firing state, neurotransmitters do not reach the connected neuron, while in the firing state, neurotransmitters do reach it. Since the combination of input values ​​to a neuron can be considered as a vector with each of these as components, we can represent the vector with components x1, x2, x3, and x4 as x, and then x = (x1, x2, x3, x4). T This is expressed as follows. From now on, we will refer to this x as the input vector.

[0019] Synapses, which are the points where inputs connect to neurons, are assigned synaptic weights. Let w1, w2, w3, and w4 be assigned to inputs 1, 2, 3, and 4, respectively. Since this combination of synaptic weights can also be considered as a vector, we will use the same notation as for inputs, denoting the synaptic weight vector w as w=(w1,w2,w3,w4). T This is how it is expressed.

[0020] Figures 57A to 57F illustrate the similarity calculation method in the prior art. Figure 57A shows the state of the Associative Network during training. Neuron 300 in Figure 57A has six inputs connected to it. In Figure 57A, the input vector x l is, x l =(1,0,0,1,0,1) T This learning process sets up the synaptic weight vector as shown in Figure 57B. This is because when neuron 300, shown in Figure 57A, is firing, the input vector x l =(1,0,0,1,0,1) T This is added, and it indicates that for inputs with a value of 1 among the components of this input vector, the weight of the corresponding synapse is set to 1 according to Hebb's rule. That is, w=x l This is the result.

[0021] As an example of the first similarity determination, as shown in Figure 57C, the input vector x1 is x1 = (1,0,0,1,0,1) T Let's assume that the following is input. That is, the same input vector used during training is also added during similarity determination. In an Associative Network, in this case, x1 and the input x from training are used. l The similarity between the two vectors is calculated as the dot product of the two vectors. That is, the dot product is x l x = x l Therefore, the dot product can be rewritten as w·x1. The degree of similarity calculated in this way (hereinafter referred to as the dot product similarity) is 3. At this time, we consider the activity level of the neuron in Figure 57C, that is, the value passed to the neuron's activation function to determine its output, to be equal to the dot product similarity. If neuron 300 in Figure 57C has a step function with a threshold of 3 as its activation function, then neuron 300 will output 1.

[0022] As a second example of similarity determination, as shown in Figure 57D, the input vector x2 is x2 = (1,0,0,1,1,0) TSuppose the following is input. In this case, the dot product similarity is 2, and the input vector x during training is... l This indicates that there is one less input that results in a value of 1. If neuron 300 in Figure 57D has the same activation function as when the above input vector x2 was input, then this dot product similarity does not reach the threshold of 3, and therefore outputs 0.

[0023] As a third example of similarity determination, as shown in Figure 57E, the input vector x3 is x3 = (1,0,0,1,0,0) T Let's assume that the following is input. In this case as well, the dot product similarity will be 2, and the input vector x during training will be... l This indicates that there is one less input that results in a value of 1. In this case, as in Figure 57D, the output will be 0.

[0024] Now, looking at the difference between input vectors x2 and x3, in x2 there is one input where the training input is 0 and the similarity judgment input is 1, and one input where the training input is 1 and the similarity judgment input is 0. In other words, there are two inputs that show a difference. In contrast, in x3 there is only one input where the training input is 1 and the similarity judgment input is 0. In other words, there is only one input that shows a difference. Therefore, in reality, x3 is better than x l They are close, but the dot product similarity ends up being the same value.

[0025] As a fourth example of similarity determination, as shown in Figure 57F, the input vector x4 is x4 = (1,1,1,1,0,1) T Suppose the following is input. In this case, the dot product similarity is 3, and the input vector x during training is... l The value will be the same as in the first similarity judgment example where the input is left as is. However, x1 is x l In contrast to the case of x1, x4 yields the same result even though there are two inputs where the training input is 0 and the similarity judgment input is 1. [Prior art documents] [Non-patent literature]

[0026] [Non-Patent Document 1] BL McNaughton, RGM Morris, "Hippocampal synaptic enhancement and information storage within a distributed memory system," Trends in Neuroscience, volume 10, Issue 10, pp. 408-415, 1987. [Non-Patent Document 2] Thomas Trappenberg, Fundamentals of Computational Neuroscience, Oxford University Press, 2010. [Non-Patent Document 3] Edmund T. Roll, Cerebral Cortex: Principles of Operation, Oxford University Press, 2016. [Non-Patent Document 4] Eric R. Kandel, James H. Schwartz, Thomas M. Jessell, Steven A. Siegelbaum, and AJ Hudspeth, "PRINCIPLES OF NEURAL SCIENCE: Fifth Edition," McGraw-Hill Education, 2012. [Non-Patent Document 5] David J. Heeger, "Normalization of cell responses in cat striate cortex," Visual Neuroscience, vol. 9, pp. 181-197, 1992. [Non-Patent Document 6] T. Tanimoto, "An elementary mathematical theory of classification and prediction.", Technical report, International Business Machines Corporation, New York, 1958. [Non-Patent Document 7] P. Jaccard, "The distribution of the flora in the alpine zone", Phytologist, 1912;11(2):37-50. https: / / doi.org / 10.1111 / j.1469-8137.1912.tb05611.x. [Non-Patent Document 8] GA Carpenter, S. Grossberg, N. Markuzon, JH Reynolds, and DB Rosen, "Fuzzy ARTMAP: A Neural Network Architecture for Incremental Supervised Learning of Analog Multidimensional Maps," IEEE Transactions of Neural Networks, Vol. 3, No. 5, pp. 698-713, 1992. [Non-Patent Document 9] L. Zadeh, "Fuzzy sets", Information and Control, Vol. 8, No. 3, pp. 338-353, 1965. [Overview of the Initiative] [Problems that the invention aims to solve]

[0027] In an Associative Network, the neural network's inputs are treated as vectors (input vectors), and the dot product of the input vector used for training and the input vector being compared is calculated to determine similarity. In practice, even if there is a difference in the distance between the two input vectors being compared and the training input vector, the dot product similarity may still be the same. For example, as shown in the third similarity determination example in Figure 57E, in reality, x3 is better than x l Although the results are close, the dot product similarity may end up being the same value, or, as in the example of the fourth similarity determination shown in Figure 57F, x4 may produce the same result as x1, even though there are two inputs where the training input is 0 and the similarity determination input is 1. Thus, in conventional similarity calculation techniques, the dot product similarity sometimes has the problem of not being able to accurately determine the difference between the input vector during training and the input vector during similarity determination.

[0028] This invention has been made in view of these circumstances, and aims to accurately determine the difference between the input vector during training and the input vector during similarity determination when determining the dot product similarity. [Means for solving the problem]

[0029] To solve the aforementioned problems, a similarity determination method is provided in which the degree of similarity between the inputs of the learning phase and the inputs of the similarity determination phase is calculated using a perceptron modeled after nerve cells, wherein there is one or more inputs, each input is one of two different values, L or H, and the value of the i-th input of the learning phase is x i The value of the i-th input of the similarity determination phase is expressed as y i When expressed as such, the i-th input has the value w i It is assigned the value w i One of two values, L or H, is set for this, and the weight w is the value assigned to the i-th input during the learning phase. i to x i Set to the value of and in the similarity determination phase, xi The number of inputs whose value is H, w i and y i The number of inputs where both values ​​are H, y i The three values ​​of the number of inputs whose value is H are calculated, w i and y i The value representing the number of inputs where both are value H is w i The value of y represents the number of inputs whose value is H. i The similarity determination method is characterized by calculating and outputting a similarity score, which is obtained by dividing the value of by a value representing the number of inputs whose value is H, by the sum of the two values. [Effects of the Invention]

[0030] According to the present invention, when determining the dot product similarity, it is possible to accurately determine the difference between the input vector during training and the input vector during similarity determination. [Brief explanation of the drawing]

[0031] [Figure 1] This shows an example of a neural circuit that performs the division normalization operation in the division normalization type similarity determination method according to the first embodiment of the present invention. [Figure 2] This figure shows an example of a circuit for performing the division-normalized similarity determination method according to the first embodiment of the present invention. [Figure 3] This figure shows the setting of synaptic weights in the division-normalized similarity determination method according to the first embodiment of the present invention. [Figure 4] This figure shows the similarity determination phase in the division-normalized similarity determination method according to the first embodiment of the present invention. [Figure 5] This figure shows an example of a spreading learning network in the division-normalized similarity determination method according to the first embodiment of the present invention. [Figure 6] This figure shows a spread learning network obtained by removing the perceptron that adds up the outputs of each perceptron from the spread learning network shown in Figure 5. [Figure 7]Figure 6 illustrates the <learning phase> of Example 1 (step function) of a spreading learning network. [Figure 8] This figure illustrates Example 1 of the <Similarity Judgment Phase> of the Diffusion Learning Network Operation Example 1 (Step Function) shown in Figure 6. [Figure 9] This figure illustrates Example 2 of the <similarity determination phase> of Example 1 (step function) of the spread learning network shown in Figure 6. [Figure 10] This figure illustrates Example 3 of the <similarity determination phase> of Example 1 (step function) of the spread learning network shown in Figure 6. [Figure 11] Figure 6 illustrates the learning phase of Example 2 (linear function) of a spreading learning network. [Figure 12] This figure illustrates Example 1 of the <similarity determination phase> of Example 2 (linear function) of the spread learning network shown in Figure 6. [Figure 13] This figure illustrates Example 2 of the <Similarity Judgment Phase> of the Diffusion Learning Network Operation Example 2 (Linear Function) shown in Figure 6. [Figure 14] This figure illustrates Example 3 of the <similarity determination phase> of Example 2 (linear function) of the spread learning network shown in Figure 6. [Figure 15] This is a flowchart showing the processing in the learning phase of the division-normalized similarity calculation unit of the division-normalized similarity determination method according to the first embodiment of the present invention. [Figure 16] This is a flowchart showing the processing in the similarity determination phase of the division-normalized similarity calculation unit of the division-normalized similarity determination method according to the first embodiment of the present invention. [Figure 17] This is a flowchart showing the processing in the learning phase of the division-normalized similarity calculation unit of the division-normalized similarity determination method according to the first embodiment of the present invention. [Figure 18] This is a flowchart showing the processing in the similarity determination phase of the division-normalized similarity calculation unit of the division-normalized similarity determination method according to the first embodiment of the present invention. [Figure 19] This figure shows a neural network when the division-normalized similarity determination method according to the first embodiment of the present invention is combined with a spreading learning network. [Figure 20] This is a flowchart showing the processing in the learning phase of Example 3 of the division-normalized similarity determination method according to the first embodiment of the present invention. [Figure 21] This is a flowchart showing the processing in the similarity determination phase of Example 3 of the division-normalized similarity determination method according to the first embodiment of the present invention. [Figure 22] This is a flowchart showing the processing in the learning phase of Example 4 of the division-normalized similarity determination method according to the first embodiment of the present invention. [Figure 23] This is a flowchart showing the processing in the similarity determination phase of Example 4 of the division-normalized similarity determination method according to the first embodiment of the present invention. [Figure 24] This figure shows the effect of a spreading information network when m is varied, with the activation function of the perceptron in the division-normalized similarity calculation unit of the division-normalized similarity determination method according to the first embodiment of the present invention being a step function, N=100, p=0.05, k=0. [Figure 25] This figure shows the effect of a diffuse information network when p = 1.0, relative to Figure 24. [Figure 26] Figure 24 shows the effect of the diffusive learning network when the value of k is varied, with m=0. [Figure 27] Figure 25 shows the effect of the diffusive learning network when the value of k is varied, with m=0. [Figure 28] Figure 24 shows the effect of a diffusive learning network when the values ​​of m and k are changed simultaneously, with m = k. [Figure 29] Figure 25 shows the effect of a diffusive learning network when the values ​​of m and k are changed simultaneously, with m = k. [Figure 30]This figure shows the effect of the spreading learning network of the division-normalized similarity determination method according to the first embodiment of the present invention (linear function, p=0.05 and k=0). [Figure 31] This figure shows the effect of a spreading learning network in the division-normalized similarity determination method according to the first embodiment of the present invention (linear function, p=1.0 and k=0). [Figure 32] This figure shows the effect of the spreading learning network of the division-normalized similarity determination method according to the first embodiment of the present invention (linear function, p=0.05 and m=0). [Figure 33] This figure shows the effect of a spreading learning network in the division-normalized similarity determination method according to the first embodiment of the present invention (linear function, p=1.0 and m=0). [Figure 34] This figure shows the effect of the spreading learning network of the division-normalized similarity determination method according to the first embodiment of the present invention (for a linear function, p=0.05 and m=k). [Figure 35] This figure shows the effect of the diffusion learning network of the division-normalized similarity determination method according to the first embodiment of the present invention (for a linear function, p=1.0 and m=k). [Figure 36] This figure shows the division-normalized similarity calculation method according to the second embodiment of the present invention, and the activity level (N=100) of the perceptron that outputs the diffusion information network when only a diffusion learning network is used. [Figure 37] This figure shows the division-normalized similarity calculation method according to the second embodiment of the present invention, and the activity level (N=1000) of the perceptron that outputs the diffusion information network when only a diffusion learning network is used. [Figure 38] This figure shows the activity level of the perceptron that outputs the diffusion information network when the number of inputs, where the input value is 1 during learning and 0 during similarity determination, is changed, according to the second embodiment of the present invention. [Figure 39]This figure shows the activity level of the perceptron that outputs the diffusion information network when the number of inputs, which are 0 during learning and 1 during similarity determination, is changed, according to the second embodiment of the present invention. [Figure 40] This figure compares the activity level of a perceptron that outputs a diffusion information network when the number of inputs, where the input value is 1 during learning and 0 during similarity determination, is changed, with the raised Tanimoto similarity when using the division-normalized similarity calculation method, diffusion learning network, and noise-added sensitivity characteristic improvement method according to the second embodiment of the present invention. [Figure 41] This figure compares the activity level of a perceptron that outputs a diffusion information network when the number of inputs, where the input value is 0 during training and 1 during similarity determination, is changed, with the raised Tanimoto similarity when using the division-normalized similarity calculation method, diffusion learning network, and noise-added sensitivity characteristic improvement method according to the second embodiment of the present invention. [Figure 42] This is a flowchart showing the processing in the similarity determination phase of the division-normalized similarity calculation unit of the second embodiment of the present invention. [Figure 43] This is a flowchart showing the processing in the similarity determination phase of the division-normalized similarity calculation unit of the second embodiment of the present invention. [Figure 44] This is a flowchart showing the processing in the similarity determination phase of the division-normalized similarity calculation unit of the second embodiment of the present invention. [Figure 45] This is a flowchart showing the processing in the similarity determination phase of the division-normalized similarity calculation unit of the second embodiment of the present invention. [Figure 46] This figure illustrates an example of similarity calculated using a division-normalized similarity calculation method with fuzzy logic according to the third embodiment of the present invention. [Figure 47]This is a flowchart showing the processing of the learning phase using a division-normalized similarity calculation method with fuzzy logic according to the third embodiment of the present invention. [Figure 48] This is a flowchart showing the processing in the similarity determination phase of the division-normalized similarity calculation unit when the noise-adding sensitivity characteristic improvement method according to the third embodiment of the present invention is not used. [Figure 49] This is a flowchart showing the processing in the similarity determination phase of a division-normalized similarity calculation unit when using the noise-added sensitivity characteristic improvement method according to the third embodiment of the present invention. [Figure 50] This is a hardware configuration diagram showing an example of a computer that implements the functions of the division-normalized similarity calculation unit of the division-normalized similarity determination method according to the first to third embodiments of the present invention. [Figure 51] This diagram shows the operation of a perceptron with variable constant inputs. [Figure 52] This figure shows the operation of a perceptron, which generalizes the representation of input and synaptic weights. [Figure 53] This is a diagram of a multi-layered artificial neural network. [Figure 54] This figure shows an example of a simple Associative Network. [Figure 55] This figure shows an example of an Associative Network that includes multiple unconditioned stimuli. [Figure 56] This diagram illustrates the neurons that make up the technology used to determine similarity by employing an Associative Network. [Figure 57A] This diagram illustrates the similarity calculation method used in conventional technology. [Figure 57B] This diagram illustrates the similarity calculation method used in conventional technology. [Figure 57C] This diagram illustrates the similarity calculation method used in conventional technology. [Figure 57D] This diagram illustrates the similarity calculation method used in conventional technology. [Figure 57E]This diagram illustrates the similarity calculation method used in conventional technology. [Figure 57F] This diagram illustrates the similarity calculation method used in conventional technology. [Modes for carrying out the invention]

[0032] The following describes, with reference to the drawings, a similarity determination method, a similarity calculation unit, a diffusive learning network, and a neural network execution program in an embodiment for carrying out the present invention (hereinafter referred to as the "first embodiment"). (First Embodiment) This invention is realized by combining a [division-normalized similarity determination method] and a [diffusion-type learning network method]. [Division normalization type similarity judgment method] First, let's explain the division-normalized similarity determination method (similarity determination method). In the existing technology of Associative Networks, similarity is calculated by the dot product of the input vector during training and the input vector during similarity determination. Therefore, each neuron has the ability to calculate the product (i.e., multiplication) of the input value and the synaptic weight value for each input, and to add up the values ​​of the products for all inputs. Generally speaking, if the input value can take any real value, then the input value and synaptic weight value can also be negative, so in practice, it has the ability of multiplication, addition, and subtraction.

[0033] In contrast, the division-normalized similarity determination method incorporates operations caused by a phenomenon called the shunt effect (Non-Patent Literature 4) of nerve cells (neurons) into the perceptron model, in addition to multiplication, addition, and subtraction. The shunt effect arises from inhibitory synapses formed near the cell body within nerve cells. The shunt effect is the effect in which the entire sum of signals transmitted to a neuron is divided by signals transmitted via inhibitory synapses formed near the cell body. This division caused by the shunt effect is also used in a model called division normalization that explains the regulation of visual sensitivity, as described in Non-Patent Literature 5.

[0034] Figure 1 shows an example of a division-normalization similarity calculation unit for division normalization, representing an example of a neural circuit that performs division normalization operations. In Figure 1, neurons 001, 002, and 003, which contain black triangles, form excitatory synapses with 005, 006, and 007, respectively, while neuron 004, which contains a white triangle (△), forms inhibitory synapses 008, 009, and 010. Here, an excitatory synapse is a synapse that directs the activation state of the neuron receiving the synapse toward firing. Conversely, an inhibitory synapse is a synapse that directs the activation state toward rest. In Figure 1, the inhibitory synapses 008, 009, and 010 formed by neuron 004 are connected to the black triangles, which represents the shunt effect of the inhibitory synapses 008, 009, and 010.

[0035] Neurons 001, 002, and 003 in Figure 1 receive inputs 1 and 2, 3 and 4, and 5 and 6, respectively, and receive input values ​​x1 and x2, x3 and x4, and x5 and x6, respectively. Assuming that these inputs result in output values ​​e1, e2, and e3 for neurons 001, 002, and 003, respectively, these output values ​​e1, e2, and e3 are sent to neurons 005, 006, and 007, respectively. Here, these output values ​​are directly transmitted to neurons 005, 006, and 007, becoming their respective activity levels. Neuron 004 also receives e1, e2, and e3 directly, and its activity level is calculated as Σ 3 j=1 e j Let's assume the value is set to . Then, the activity level of neuron 004 is output directly and sent to neurons 005, 006, and 007, causing a shunt effect at synapses 008, 009, and 010. In this case, the effect of division normalization is expressed by the following formula, and the activity levels of neurons 005, 006, and 007 are expressed by this formula (3). Here, k is 1, 2, or 3.

[0036]

number

[0037] In this case, the activity levels of neurons 005, 006, and 007 are the values ​​obtained by setting the numerators in equation (3) above to e1, e2, and e3, respectively. Thus, in division normalization, the activity level of a given neuron is divided by the sum of the outputs of multiple neurons called a neuron pool (in the example in Figure 1, neurons 001, 002, and 003). This effect explains the regulation of visual sensitivity. However, in the division normalization model, changes in synaptic weights due to learning are not taken into account, and furthermore, the value of C is determined experimentally so that the current visual input does not saturate, so there is no clear method for determining it according to the input during learning, etc.

[0038] The [division normalization type similarity determination method] of the present invention is realized by the following (A) synaptic weight determination method, (B) division normalization constant C determination method, and (C) perceptron set corresponding to the neuron pool in division normalization (hereinafter referred to as perceptron pool) determination method.

[0039] FIG. 2 is a diagram showing an example of a division normalization type similarity calculation unit (similarity calculation unit) that performs the division normalization type similarity determination method, and represents the learning phase in an example of the division normalization type similarity determination method. Hereinafter, a module that executes the process of the division normalization type similarity determination method will be referred to as a division normalization type similarity calculation unit 100 (similarity calculation unit). The input values x1, x2, x3, x4, x5, x6 to the inputs 1, 2, 3, 4, 5, 6 shown in FIG. 2 represent the input values to the division normalization type similarity calculation unit 100. These are equally input to the perceptrons 001 and 002. In this way, in the division normalization type similarity determination method, as the (C) perceptron pool in division normalization, only all the inputs to the division normalization type similarity calculation unit are used. Each input takes two types of values when the previous perceptron is in a stationary state and when it is in a firing state. In this specification, these will be represented by 0 and 1, respectively. That is, x i ∈{0, 1} (i = 1, 2, 3, 4, 5, 6).

[0040] FIG. 3 is a diagram showing the setting of synaptic weights in the division normalization type similarity determination method. FIG. 3 shows that as a result of the learning phase in FIG. 2, the synaptic weights formed on the perceptron 001 by the input values x1, x2, x3, x4, x5, x6 become w1, w2, w3, w4, w5, w6.

[0041] In the (A) synaptic weight determination method of the division normalization type similarity determination method, w i = x iSet the synaptic weight as follows. That is, in the learning phase, the weight of the synapse that receives the input signal corresponding to the firing state is 1, and the synaptic weight that receives the input signal corresponding to the resting state is 0.

[0042] Figure 4 is a diagram showing the similarity determination phase in the division normalization type similarity determination method. Figure 4 represents the similarity determination phase when the input values y1, y2, y3, y4, y5, and y6 arrive. At this time, the input to the perceptron 001 is Σ 6 j=1 y j ·w j which is calculated. On the other hand, there is no change in the synaptic weight for the perceptron 002, and Σ 6 j=1 y j is input. The output of the perceptron 002 generates a shunting effect on the perceptron 001 through the synapse 003 formed between the perceptron 001 and the perceptron 002, and calculates the following operation.

[0043]

Number

[0044] Furthermore, as a method for determining the constant C in (B) division normalization, in the learning phase, set the value calculated as follows.

[0045]

Number

[0046] However, x = (x1, x2, x3, x4, x5, x6) T and ||x|| represents the norm of the vector x. Substituting Equation (5) into Equation (4), Equation (4) is converted as follows in Equation (6).

[0047]

Number

[0048] However, y = (y1, y2, y3, y4, y5, y6) T and w = (w1, w2, w3, w4, w5, w6) T is. Equation (6) includes the square of the norm and the inner product of two vectors as vector operations. Generally, for vectors v = (v1, v2, …, v N ) T and vector u = (u1, u2, …, u N ) T when there are, ||u|| 2 = u1 2 + u2 2 + … + u N 2 and u·v = u1v1 + u2v2 + … + u N v N is.

[0049] Now, if u i ∈ {0, 1} and v i ∈ {0, 1}, then ||u|| 2 = u1 2 + u2 2 + … + u N 2 = u1 + u2 + … + u N and u v = u1v1 + u2v2 + … + u N v N = Σ N i=1 u i v i = Σ N i=1 (u i AND v i ) can also be calculated as. u i AND v i represents the logical product operation of u i and v i .

[0050] Here, n 11 , n 10 , n 01 , and n 00 are, respectively, x i = 1 and y iThe number of inputs for which = 1, x i =1 and y i The number of inputs for which x = 0. i =0 and y i The number of inputs for which = 1, and x i =0 and y i Let N = n be the number of inputs for which = 0. 11 +n 10 +n 01 +n 00 We assume that is constant because it represents the total number of inputs. Equation (6) above can be transformed as follows.

[0051]

number

[0052] In the calculation of equation (7), if the denominator is 0, n 11 , n 10 , n 01 Since all of them become 0, the numerator also becomes n 11 Therefore, its value is also 0. In this case, the result of equation (7) is calculated as 0 because the two vectors are not similar. Now, when the same input is received in the learning phase and the similarity determination phase, n 10 =n 01 Since = 0, we get equation (8).

[0053]

number

[0054] Next, let's consider the case where the input differs between the learning phase and the similarity determination phase. f =n 11 +n 10 This N is the number of inputs that is 1 during training, and remains constant in the similarity determination phase after the training phase. f Using this, equation (7) can be transformed as follows:

[0055]

number

[0056] From this equation (9), it can be seen that the value calculated by equation (9) changes only depending on n 10 and n 01 Herefrom, it can be seen that how the value of equation (9) changes due to the changes in n 10 and n 01 will be explained.

[0057] <n 10 Change> First, consider the change in equation (9) with respect to the change in n 10 Equation (9) is transformed as the following equation (10).

[0058]

Number

[0059] In equation (10), when n 01 is constant, it can be seen that the value of the above equation monotonically decreases with the increase in n 10

[0060] <n 01 Change> Second, consider the change in equation (9) with respect to the change in n 01 In equation (9), when n 10 is constant, it can be seen that the value of equation (9) monotonically decreases with the increase in n 01 From the above, equation (7) has a value of 1 when n 10 =n 01 =0, and monotonically decreases with the increase in n 10 and n 01 , representing the degree of similarity, and it can be seen that it solves the problem that the degree of similarity does not change even when n 10 and n 01 which were problems in the existing technology change.

[0061] <The exact meaning of the value calculated by the division normalization type similarity calculation method> ​​Next, we will explain the precise meaning of the values ​​calculated by the division-normalized similarity method. The following two equations, S d , and, S c Let's consider this.

[0062]

number

[0063] Equation (11) shows that c1 is n 11 +n 10 When this is the case, the formula becomes the division-normalized similarity calculation method of the present invention.

[0064]

number

[0065] Equation (12) shows that c2 is n 11 +n 10 This represents the cosine similarity between vectors x and y. Cosine similarity indicates how similar two vectors are. Specifically, it is the cosine value of the angle between two vectors in a vector space. This value is calculated by dividing the dot product of the two vectors (an operation in which the products of corresponding components of the two vectors are added together over all components) by the product of the magnitudes (norms) of the two vectors.

[0066] First, let u and v be n, respectively. 11 , and, n 01 Substituting these into equations (11) and (12) above, we get S d , and, S c This can be expressed as a function of u and v, as follows:

[0067]

number

[0068]

number

[0069] Currently, generally speaking, when considering the Taylor expansion of a function f(u,v) around (u,v), if we consider up to the first-order term, the Taylor series f up to the first-order term is (1) (u+h,v+k) can be expressed as follows:

[0070]

number

[0071] Using this, S d (u,v), and S c Taylor series S up to the first linear term with respect to (u,v) d (1) (u+h,v+k), and S c (1) The solution to (u+h,v+k) is as follows:

[0072]

number

[0073]

number

[0074] In equations (16) and (17) above, c1 = c2 = n 11 +n 10 =N f u=N f Substituting v=0 into the equation yields the following:

[0075]

number

[0076]

number

[0077] Therefore, c1 = c2 = n 11 +n 10 =N f u=N f And, when v=0, the following equality holds.

[0078]

number

[0079] From the above, it can be seen that the value calculated by the division-normalized similarity determination method of the present invention is an approximation of the cosine similarity. As a result, the similarity calculated by the division-normalized similarity determination method can calculate the similarity more accurately than existing technologies.

[0080] [Diffusion learning network method] Next, we will explain the diffusive learning network method. Figure 5 shows an example of a diffusive learning network. As shown in Figure 5, in the diffusive learning network 1000, multiple division-normalized similarity calculation units 100 are connected to the input (the part in Figure 5 where input values ​​x1, x2, x3, x4, x5, x6, etc. are input), and each division-normalized similarity calculation unit 100 outputs output values ​​z1, z2, z3, z4, z5, z6, which are input to the perceptron 013.

[0081] As a result, in the diffusive learning network 1000, the output values ​​z1, z2, z3, z4, z5, and z6 are added together by the perceptron 013, and then the output value corresponding to the activation function of the perceptron 013 is output from z7 onwards.

[0082] Below, we will explain the operation of the network other than the perceptron 013, using Figure 6, which shows the diffusive learning network 1000 with the perceptron 013 removed. Figure 6 shows a diffuse learning network obtained by removing the perceptron that adds up the outputs of each perceptron from the diffuse learning network in Figure 5. For the sake of explanation, the diffuse learning network 1000 in Figure 6, which is obtained by removing perceptron 013 from diffuse learning network 1000, will also be denoted by the same symbols.

[0083] There are two examples of how a diffusive learning network works: Example 1 using a step function (Figures 7-10) and Example 2 using a linear function (Figures 11-14). Each of Examples 1 and 2 can be further divided into a learning phase (Figures 7 and 11), a similarity determination phase using a step function (Figures 8-10), and a similarity determination phase using a linear function (Figures 12-14). These will be explained in order below.

[0084] <Example of operation 1 (step function)> First, let's explain the operation example 1 of a diffusive learning network (step function). Figure 7 illustrates the <learning phase> of Example 1 (step function) of the spread learning network shown in Figure 6. In Figure 7, the <Learning Phase> is given as x=(x1,x2,x3,x4,x5,x6) T =(1,0,1,1,0,1) T This indicates the state when the input was entered. At this time, the activation function of perceptrons 001, 002, 003, 004, 005, and 006 is set to a step function with a threshold of 0.6.

[0085] During this learning phase, the synaptic weights of perceptrons 001, 002, 003, 004, 005, and 006 change according to the learning phase of the division-normalized similarity determination method. That is, when the input during learning is 1, the synaptic weight associated with that input is set to 1, and when the input is 0, the synaptic weight is set to 0. As a result, perceptrons 001, 002, 003, 004, 005, and 006 will each have two, one, one, one, one, and two synapses with a weight of 1, respectively.

[0086] Figure 8 illustrates Example 1 of the <Similarity Judgment Phase> of the Diffuse Learning Network Operation Example 1 (Step Function) shown in Figure 6. In Example 1 of the <Similarity Determination Phase> in Figure 8, (y1, y2, y3, y4, y5, y6) T =(1,0,1,1,0,1) T This shows the state when the input is received. This input is the same as the input in the <Learning Phase> in Figure 7. At this time, perceptrons 001 to 006 calculate the similarity as follows, according to the synaptic weights that have changed according to the input values ​​in the <Learning Phase> and the input values ​​in the similarity determination phase.

[0087] (1) In the case of perceptrons 001 and 006 The value calculated by the division-normalized similarity determination method is as follows. In the following formula, the reason for comparing with 0.6 at the end is that 0.6 is set as the threshold for the activation function of the perceptron.

[0088]

number

[0089] (2) In the case of perceptrons 002, 003, 004, and 005 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0090]

number

[0091] As described above, all perceptrons will have inputs that exceed the threshold, and since their activation function is a step function, their output will be 1. Therefore, all perceptrons 001, 002, 003, 004, 005, and 006 will output 1. As shown in Figure 5, when the outputs of perceptrons 001, 002, 003, 004, 005, and 006 are input to perceptron 013, and the activity level of this perceptron is represented by the sum of the input values, and its activation function is represented by a linear function with a threshold of 0, perceptron 013 will output 6.

[0092] Figure 9 illustrates Example 2 of the <similarity determination phase> of Example 1 (step function) of the spread learning network shown in Figure 6. In Example 2 of the <Similarity Determination Phase> in Figure 9, the similarity determination phase includes (y1, y2, y3, y4, y5, y6) T =(1,1,0,0,0,1) T This is the case when the following input is given.

[0093] (1) In the case of perceptron 001 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0094]

number

[0095] (2) In the case of perceptron 002 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0096]

number

[0097] (3) In the case of perceptrons 003 and 005 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0098]

number

[0099] (4) In the case of Perceptron 004 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0100]

number

[0101] (5) In the case of Perceptron 006 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0102]

number

[0103] As a result, the outputs of the three perceptrons 001, 002, and 006 become 1. As shown in Figure 5, when the outputs of perceptrons 001, 002, 003, 004, 005, and 006 are input to perceptron 013, and the activity level of this perceptron is represented by the sum of the input values, and the activation function is represented by a linear function with a threshold of 0, perceptron 013 outputs 3. Now, if we consider the case where all inputs are connected to a single perceptron, the value calculated by the division-normalized similarity determination method will be as follows:

[0104]

number

[0105] In this case, the similarity score could not be calculated without a diffusive learning network. On the other hand, in the example in Figure 9, the effect of the diffusive learning network causes a bias in the situation where the input is 1 for some perceptrons both during training and similarity determination. As a result, three perceptrons become fired, enabling the similarity to be determined.

[0106] Figure 10 illustrates Example 3 of the <similarity determination phase> of Example 1 (step function) of the spread learning network shown in Figure 6. In Example 3 of the <Similarity Determination Phase> in Figure 10, the similarity determination phase includes (y1, y2, y3, y4, y5, y6) T =(1,0,1,1,1,0) T This is the case when the following input is given.

[0107] (1) In the case of perceptron 001 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0108]

number

[0109] (2) In the case of perceptron 002 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0110]

number

[0111] (3) In the case of Perceptron 003 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0112]

number

[0113] (4) In the case of Perceptron 004 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0114]

number

[0115] (5) In the case of Perceptron 005 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0116]

number

[0117] (6) In the case of Perceptron 006 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0118]

number

[0119] As a result, the outputs of the five perceptrons 001, 003, 004, 005, and 006 become 1. As shown in Figure 5, when the outputs of perceptrons 001, 002, 003, 004, 005, and 006 are input to perceptron 013, and the activity level of this perceptron is represented by the sum of the input values, and the activation function is represented by a linear function with a threshold of 0, perceptron 013 outputs 5. Now, if we consider the case where all inputs are connected to a single perceptron, the value calculated by the division-normalized similarity determination method will be as follows:

[0120]

number

[0121] In this case, similarity can be determined even without a diffusive learning network. On the other hand, in this example, the output is 5, while in the previous example, the output was 3. This is because, with a so-called sparse distributed learning network, only a portion of the total input is fed into the division-normalized similarity determination method, and the output changes depending on the degree of bias. Therefore, the higher the similarity, the more likely it is that even with a small bias, the input to the division-normalized similarity determination method will exceed the threshold of the activation function in terms of activation. Hence, the output in this example is larger. From this, it can be seen that diffusive learning networks enable the determination of similarity for a wide range of inputs.

[0122] The above describes the operation with a step function with a threshold of 0.6 as the activation function. From here, we will explain the operation with a linear function with a threshold of 0.6 using Figure 11.

[0123] Figure 11 is a diagram illustrating the <learning phase> of Example 2 (linear function) of the spread learning network shown in Figure 6. In Figure 11, the <Learning Phase> is given as x=(x1,x2,x3,x4,x5,x6) T =(1,0,1,1,0,1) T This indicates the state when the input was entered. In this case, the activation functions of perceptrons 001, 002, 003, 004, 005, and 006 are linear functions with a threshold of 0.6 and a slope of 1.

[0124] During this learning phase, the synaptic weights of perceptrons 001, 002, 003, 004, 005, and 006 change according to the learning phase of the division-normalized similarity determination method. That is, when the input during learning is 1, the synaptic weight associated with that input changes to 1, and when the input is 0, the synaptic weight is 0. As a result, perceptrons 001, 002, 003, 004, 005, and 006 will each have two, one, one, one, one, and two synapses with a weight of 1, respectively.

[0125] Figure 12 illustrates Example 1 of the <similarity determination phase> of Example 2 (linear function) of the spread learning network shown in Figure 6. In Example 1 of the <Similarity Determination Phase> in Figure 12, (y1, y2, y3, y4, y5, y6) T =(1,0,1,1,0,1) T This indicates the state when the input was received. This input is the same as the input in the learning phase.

[0126] At this time, perceptrons 001 to 006 calculate the similarity and output according to the synaptic weights that have changed due to the input values ​​of the learning phase and the input values ​​of the similarity determination phase, as follows. Below, a linear function with a threshold of 0.6 and a slope of 1 is used for f. l (a) will be used to represent this.

[0127] (1) In the case of perceptrons 001 and 006 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0128]

number

[0129] Therefore, f l (S d )=f l (1) = 0.4 is the output.

[0130] (2) In the case of perceptrons 002, 003, 004, and 005 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0131]

number

[0132] Therefore, f l (S d )=f l (1) = 0.4 is the output.

[0133] As described above, all perceptrons will have inputs that exceed the threshold and will generate outputs proportional to the similarity. As shown in Figure 5, when the outputs of perceptrons 001, 002, 003, 004, 005, and 006 are input to perceptron 013, and the activity level of this perceptron is represented by the sum of the input values, and the activation function is represented by a linear function with a threshold of 0, perceptron 013 outputs 2.4.

[0134] Figure 13 illustrates Example 2 of the <Similarity Judgment Phase> of the Diffuse Learning Network Operation Example 2 (Linear Function) shown in Figure 6. In Example 2 of the <Similarity Determination Phase> in Figure 13, (y1, y2, y3, y4, y5, y6) T =(1,1,0,0,0,1) T This indicates the state when the input was entered.

[0135] (1) In the case of perceptron 001 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0136]

number

[0137] Therefore, f l (S d )=f l The output is (2 / 3) = 2 / 3 - 0.6.

[0138] (2) In the case of perceptron 002 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0139]

number

[0140] Therefore, f l (S d )=f lThe output is (2 / 3) = 2 / 3 - 0.6.

[0141] (3) In the case of perceptrons 003 and 005 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0142]

number

[0143] Therefore, f l (S d )=f l (0)=0 is the output.

[0144] (4) In the case of Perceptron 004 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0145]

number

[0146] Therefore, f l (S d )=f l (0)=0 is the output.

[0147] (5) In the case of Perceptron 006 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0148]

number

[0149] Therefore, f l (S d )=f l (1) = 0.4 is the output.

[0150] As shown in Figure 5, the outputs of perceptrons 001, 002, 003, 004, 005, and 006 are input to perceptron 013. When the activity level of this perceptron is represented by the sum of the input values, and the activation function is a linear function with a threshold of 0, perceptron 013 outputs 4 / 3 - 0.8 ≈ 0.53.

[0151]

number

[0152] Figure 14 illustrates Example 3 of the <similarity determination phase> of Example 2 (linear function) of the spread learning network shown in Figure 6. In Example 3 of the <Similarity Determination Phase> in Figure 14, (y1, y2, y3, y4, y5, y6) T =(1,0,1,1,1,0) T This indicates the state when the input was entered.

[0153] (1) In the case of perceptron 001 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0154]

number

[0155] Therefore, f l (S d )=f l (1) = 0.4 is the output.

[0156] (2) In the case of perceptron 002 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0157]

number

[0158] Therefore, f l(S d )=f l (0)=0 is the output.

[0159] (3) In the case of Perceptron 003 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0160]

number

[0161] Therefore, f l (S d )=f l The output is (2 / 3) = 2 / 3 - 0.6.

[0162] (4) In the case of Perceptron 004 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0163]

number

[0164] Therefore, f l (S d )=f l (1) = 0.4 is the output.

[0165] (5) In the case of Perceptron 005 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0166]

number

[0167] Therefore, f l (S d )=f l The output is (2 / 3) = 2 / 3 - 0.6.

[0168] (6) In the case of Perceptron 006 The values ​​calculated by the division-normalized similarity determination method are as follows:

[0169]

number

[0170] Therefore, f l (S d )=f l The output is (2 / 3) = 2 / 3 - 0.6.

[0171] As shown in Figure 5, the outputs of perceptrons 001, 002, 003, 004, 005, and 006 are input to perceptron 013, and the activity level of this perceptron is represented by the sum of the input values, and the activation function is represented by a linear function with a threshold of 0, then perceptron 013 is as follows.

[0172]

number

[0173] The above describes the [division-normalized similarity determination method] and the [diffusion learning network method]. Below, we will describe the division-normalized similarity calculation unit of the diffusion learning network.

[0174] [Division-Normalized Similarity Calculation Unit for Diffuse Learning Networks] A divergent learning network contains one or more division-normalized similarity calculation units. The following description explains how the input to the divergent learning network is connected to these division-normalized similarity calculation units and what the average output value of these units will be as a result.

[0175] Firstly, the input to the diffusive learning network (in the example in Figure 5, input i is the input value x) i The following six sets, I, are input, or consist of some of the inputs. N, I k , I m , I n , I d , I l Let's consider this.

[0176]

number

[0177]

number

[0178]

number

[0179]

number

[0180]

number

[0181]

number

[0182] I N This is the set of inputs whose input value is 1 during the learning phase. k This is the set of inputs whose input values ​​in the learning phase and the similarity determination phase are 0 and 1, respectively. m This is the set of inputs whose input values ​​in the learning phase and the similarity determination phase are 1 and 0, respectively. n This is the set of inputs connected to the division-normalized similarity calculation unit. d is set I n and I m This is the set of inputs that are included in both. l is set In and I k This is the set of inputs that are included in both.

[0183] Now, let N, k, m, n, d, and l be in set I, respectively. N , I k , I m , I n , I d , and, I l This is the number of elements contained in the formula. In this case, the number of inputs whose input value is 1 in at least one of the learning phase or the similarity determination phase is N+k. In the division-normalized similarity determination method, as can be seen from equation (7), only these N+k inputs affect the similarity. Therefore, we focus on these N+k inputs and analyze the connection status of the inputs to the division-normalized similarity calculation unit. Since the number of inputs connected to the division-normalized similarity calculation unit is n, the number of patterns when n out of N+k inputs are connected is expressed by the following formula.

[0184]

number

[0185] Secondly, the number of inputs where both the learning phase and the similarity determination phase input values ​​are 1 is Nm. Of these, ndl are input to the division-normalized similarity calculation unit. Therefore, the number of patterns is expressed by the following formula.

[0186]

number

[0187] Thirdly, there are m inputs where the input values ​​for the learning phase and the similarity determination phase are 1 and 0, respectively, and of these, d are input to the division-normalized similarity calculation unit. Therefore, the number of patterns is expressed by the following formula.

[0188]

number

[0189] Fourthly, there are k inputs where the input values ​​for the learning phase and the similarity determination phase are 0 and 1, respectively, and of these, l are input to the division-normalized similarity calculation unit. Therefore, the number of patterns is expressed by the following formula.

[0190]

number

[0191] Then, among the input patterns connected to the division-normalized similarity calculation unit, set I m , I n , and, I d The probabilities that the number of elements is m, n, and d, respectively, are given by the following formula.

[0192]

number

[0193] In this case, the similarity score calculated using the division-normalized similarity determination method is as follows:

[0194]

number

[0195] If we denote this value as the activity level S(n,d,l) and the activation function as f(a), then the output can be calculated as f(S(n,d,l)). From the above, the output of the division-normalized similarity calculation unit is expressed by the following formula.

[0196]

number

[0197] Here, C (representing the summation range written below the symbol Σ) is the set of combinations of n, d, and l that simultaneously satisfy the following conditions, with τ being the threshold of the activation function. The number of inputs that have a value of 1 during the learning phase is N, and some of these become 0 during the similarity determination phase. Since the number of these is m, the following inequality holds.

[0198]

number

[0199] The total number of inputs where the learning phase value is 1 and the similarity determination phase value is 0 is m. A portion of these inputs are connected to division-normalized similarity calculation units, and the number of such units is d, so the following inequality holds.

[0200]

number

[0201] The total number of inputs where the learning phase value is 0 and the similarity determination phase value is 1 is k. A portion of these inputs are connected to division-normalized similarity calculation units, and since the number of such units is l, the following inequality holds.

[0202]

number

[0203] The number of inputs connected to the division-normalized similarity calculation unit is n. Since some of these inputs are d, l, and d+l, the following three inequalities hold.

[0204]

number

[0205]

number

[0206]

number

[0207] The total number of inputs for which the learning phase value is 1 and the similarity determination phase value is 1 is Nm. A portion of these are connected to division-normalized similarity calculation units, and the number of such units is ndl, so the following inequality holds.

[0208]

number

[0209] For the division-normalized similarity calculation unit to fire and produce an output value greater than 0, its activity level must exceed the threshold τ. Therefore, the following inequality holds.

[0210]

number

[0211] In the above discussion regarding the expected value of the output calculated by the division-normalized similarity calculation unit, we calculated the expected value with n as a constant. Now, we will find the expected value of the output when each input connects to the division-normalized similarity calculation unit with a certain probability p. The inputs that we have focused on in this discussion are inputs whose value is 1 in at least one of the learning phase and the similarity determination phase, and the total number of such inputs is N+k. Of these, the probability that n inputs connect to the division-normalized similarity calculation unit is expressed by the following formula.

[0212]

number

[0213] Therefore, from equations (63) and (72), the expected value of the output of the division-normalized similarity calculation unit is expressed by the following equation.

[0214]

number

[0215] Since equation (73) represents the expected value of the output of the division-normalized similarity calculation unit, the activity level of the perceptron (013 in Figure 5) that outputs the diffusion information network is proportional to equation (73) because it is the sum of the outputs of the division-normalized similarity calculation unit. The effect of this diffusion information network will be described later using Figures 24 to 35.

[0216] [Processing of the learning phase and similarity determination phase of a diffusive learning network] The learning phase and similarity determination phase of the diffusive learning network will be explained below with reference to Figures 15 to 23.

[0217] <Example 1> <Example 1> describes Example 1 of the division-normalized similarity determination method. First, let's discuss the learning phase processing of a diffusive learning network. Figure 15 is a flowchart showing the processing during the learning phase of the division-normalized similarity calculation unit. In step S1, the division-normalized similarity calculation unit 100 (Figures 2-14) calculates the input vector x=(x1,x2,…,x) during the learning phase. N ) T Receive.

[0218] In step S2, the division-normalized similarity calculation unit 100 calculates the synaptic weight vector w=(w1,w2,…,w N ) T w i =x i Set the parameters as (i=1,2,…,N).

[0219] In step S3, the division-normalized similarity calculation unit 100 determines the parameter C used in the similarity determination phase as C = ||x|| 2 Calculate and set it as follows. After the learning phase shown in Figure 15, the similarity determination phase shown in Figure 16 is performed.

[0220] Next, we will discuss the similarity determination phase processing of a diffusive learning network. Figure 16 is a flowchart showing the processing in the similarity determination phase of the division-normalized similarity calculation unit.

[0221] In step S11, the division-normalized similarity calculation unit 100 calculates the input vector y=(y1,y2,…,y) for the similarity determination phase. N ) T Receive.

[0222] In step S12, the division-normalized similarity calculation unit 100 calculates the similarity using Y=||y|| 2 Calculate.

[0223] In step S13, the division-normalized similarity calculation unit 100 calculates Z = w·y, which is necessary to calculate the similarity.

[0224] In step S14, the division-normalized similarity calculation unit 100 uses the calculated Y and Z, as well as the parameter C calculated in step S3 of Figure 15, to calculate the similarity s according to the following equation (74).

[0225]

number

[0226] In step S15, the division-normalized similarity calculation unit 100 inputs the calculated similarity s into the activation function f(a) to obtain the output value f(s). This output value f(s) becomes the output of the division-normalized similarity calculation unit 100. Here, the activation function can be the commonly used ReLU or a step function. Alternatively, it can be a simple linear function, a threshold-linear function, a sigmoid function, or a radial-basis function, as described in Non-Patent Document 2. Furthermore, for these functions where the threshold is 0, any other arbitrary value may be used as the threshold.

[0227] <Example 2> Example 2 describes an example of a division-normalized similarity determination method. In Example 2, the dot product between vectors (w·y in equation (74)) and the square of the norm (C=||x|| in equation (74)) are included in equation (74). 2 , and, ||y|| 2 An example of efficiently calculating )) will be described.

[0228] Now, the vector v = (v1, v2, ..., v N ) T , and u=(u1,u2,…,u N ) T Let's assume that there was a v i ∈{0,1}, and u i For ∈{0,1}, the inner product (u·v) is (u·v) = v1u1 + v2u2 + ... + u N v N It is. i ∈{0,1}, and u i Since ∈{0,1}, v i u i is, v i and u i This is equivalent to the logical AND of v, therefore (u·v) is v i and u i This value is obtained by adding the logical AND of the two values ​​over all i. Also, the vector v = (v1, v2, ..., v N ) T The square of the norm of is ||v|| 2 =v1v1+v2v2+…+v N v N and v i Since ∈{0,1}, ||v|| 2 =v1+v2+…+v N Therefore, ||v|| 2 is, v i This is the value obtained by adding up all i values. Example 2 is an example of applying the above methods for calculating the dot product between vectors and the square of the norm of a vector.

[0229] First, let's discuss the learning phase processing of a diffusive learning network. Figure 17 is a flowchart showing the processing during the learning phase of the division-normalized similarity calculation unit. Steps that perform the same processing as in Figure 15 are denoted by the same reference numerals and their explanations are omitted.

[0230] In step S21, the division-normalized similarity calculation unit 100 calculates the input vector x=(x1,x2,…,x) in the learning phase. N ) T Receive.

[0231] In step S22, the division-normalized similarity calculation unit 100 calculates the synaptic weight vector w=(w1,w2,…,w N ) T w i =x i Set the parameters as (i=1,2,…,N).

[0232] In step S23, the division-normalized similarity calculation unit 100 calculates the parameter C = ||x|| used in the similarity determination phase. 2 C = Σ N i=1 x i Calculate as follows. After the learning phase shown in Figure 17, the similarity determination phase shown in Figure 18 is performed.

[0233] Next, we will discuss the similarity determination phase processing of a diffusive learning network. Figure 18 is a flowchart showing the processing in the similarity determination phase of the division-normalized similarity calculation unit.

[0234] In step S31, the division-normalized similarity calculation unit 100 calculates the input vector y=(y1,y2,…,y) for the similarity determination phase. N ) T Receive.

[0235] In step S32, the division-normalized similarity calculation unit 100 calculates the similarity using Y=||y|| 2 Calculate. At this time, Y = Σ N i=1 y i Calculate as follows.

[0236] In step S33, the division-normalized similarity calculation unit 100 calculates Z=w·y, which is necessary to calculate the similarity. At this time, Z=Σ N i=1 w i Newly i We calculate as follows: Here, w i Newly i is, w i and y i This represents a logical AND operation.

[0237] In step S34, the division-normalized similarity calculation unit 100 calculates the similarity s according to equation (74), using the calculated Y and Z, as well as the parameter C calculated in step S23 of Figure 17.

[0238] In step S35, the division-normalized similarity calculation unit 100 inputs the calculated similarity s into the activation function f(a) to obtain the output value f(s). This output value f(s) becomes the output of the division-normalized similarity calculation unit 100.

[0239] Here, the activation function can be the commonly used ReLU or a step function. Alternatively, it can be a simple linear function, a threshold-linear function, a sigmoid function, or a radial-basis function, as described in Non-Patent Document 2. Furthermore, for these functions where the threshold is 0, any other arbitrary value may be used as the threshold.

[0240] <Example 3> Example 3 describes Example 3 of the division-normalized similarity determination method. Example 3 describes a method for implementing a combination of a division-normalized similarity calculation method and a diffusive learning network. Figure 19 shows a neural network that combines a division-normalized similarity method with a diffusive learning network. A diffusive learning network contains one or more division-normalized similarity calculation units. First, it is determined whether or not to connect inputs to each division-normalized similarity calculation unit. When determining whether or not to connect inputs, the combination of inputs to each division-normalized similarity calculation unit should be as different as possible. For example, the connection status may be determined with a certain probability for each combination of input and division-normalized similarity calculation unit. In Figure 17, there are six division-normalized similarity calculation units 101 to 106 (hereinafter referred to as units). Each unit 101 to 106 is connected to all or part of the total inputs. Therefore, generally, each unit 101 to 106 receives different combinations of inputs as inputs.

[0241] Therefore, in Example 3, the learning phase processing of Example 1 and Example 2 (Figures 15 and 17) is performed only on the connected components of the input vector of the learning phase, synaptic weight vector, and input vector of the similarity determination phase for each unit. This processing will be explained using Figure 20.

[0242] Figure 20 is a flowchart showing the processing during the learning phase of <Example 3>. In the unit 101 shown in Figure 19, only inputs 1 and 3 are connected to inputs 1, 2, 3, 4, 5, and 6. In step S41, the learning phase of the division-normalized similarity determination method is executed for each division-normalized similarity calculation unit. Specifically, this is done as follows: In the learning phase, the overall input vector is x=(x1,x2,x3,x4,x5,x6) T In this case, the input vector x1 for the learning phase to unit 101 is x1=(x1,x3) TTherefore, the synaptic weight vector w1 becomes w1=(w1,w3) T = x1. Also, if the constant C of unit 101 is C1, then C1 = ||x1||, similar to <Example 1> and <Example 2>. 2 This is how it is determined. Similarly, for units 102 to 106, the synaptic weight vectors w2, w3, w4, w5, w6 and constants C2, C3, C4, C5, C6 are determined. After the learning phase shown in Figure 20, the similarity determination phase shown in Figure 21 is performed.

[0243] Next, we will describe the similarity determination phase processing in Example 3. Figure 21 is a flowchart showing the process in the similarity determination phase of <Example 3>. In step S51, the process of the similarity determination phase of each division-normalized similarity calculation unit is executed for each division-normalized similarity determination method, and the output value of each division-normalized similarity calculation unit i is f(s i Set it as follows. Specifically, it is as follows: Let's explain using Unit 101 as a representative example. In the similarity determination phase, the overall input vector is given by y=(y1,y2,y3,y4,y5,y6) T In this case, the input vector y1 for the similarity determination phase to unit 101 is y1=(y1,y3) T This is the result. Using these vectors, the similarity s1 of unit 101 is calculated using the following formula, in the same manner as in <Example 1> and <Example 2>.

[0244]

number

[0245] Similarly, the similarity is calculated for units 102 to 106. Next, the output value of the unit is calculated as f(s i Calculate as follows: Here, f(x) represents the activation function. The activation function can be the commonly used ReLU, or a step function. Alternatively, it can be a simple linear function, a threshold-linear function, a sigmoid function, or a radial-basis function, as described in Non-Patent Document 2. Furthermore, for these functions where the threshold is 0, any other arbitrary value may be used as the threshold.

[0246] In step S52, the total output S of the full division normalization similarity calculation unit (the aggregated value of the output calculated by each unit) is calculated as follows.

[0247]

number

[0248] In step S53, the obtained S is input to the activation function g(·) to calculate the output value V=g(S) of the diffusive learning network. Here, the activation function can be the commonly used ReLU or a step function. Alternatively, it can be a simple linear function, a threshold-linear function, a sigmoid function, or a radial-basis function, as described in Non-Patent Literature 2. In addition, the activation function can be the k-Winner-Take-All (kWTA) or Winner-Take-All (WTA) as described in Non-Patent Literature 3. Furthermore, for these functions, if the threshold is 0, any other arbitrary value can be used as the threshold.

[0249] <Example 4> Example 4 describes an example of a division-normalized similarity determination method. Example 4 describes a method for implementing a combination of a division-normalized similarity calculation method and a diffusive learning network. In Example 4, instead of creating separate input vectors for the learning phase, synaptic weight vectors, and similarity determination phase for each unit and calculating the similarity for each unit, as described in Example 3, the similarity is calculated using the input vectors for the learning phase, synaptic weight vectors, and similarity determination phase for the entire input.

[0250] First, determine whether or not to connect the inputs to each division-normalized similarity calculation unit. When determining whether or not to connect the inputs, ensure that the combinations of inputs to each division-normalized similarity calculation unit are as different as possible. For example, the presence or absence of connection may be determined with a certain probability for each combination of input and division-normalized similarity calculation unit.

[0251] Secondly, we create a matrix that shows which input is connected to which division-normalized similarity calculation unit. This matrix will be referred to as the connection matrix from now on. The element in the i-th row and j-th column of the connection matrix is ​​X. ij This is expressed as follows, and this component represents whether or not input i is connected to unit j. ij =1, and X ij When = 0, it represents that input i is connected to unit j, and that input i is not connected to unit j, respectively. The connection matrix X is expressed as follows:

[0252]

number

[0253] Here, for the purpose of the following explanation, the vector consisting of the components of column j of the connection matrix is ​​X. j It is represented as follows. Thirdly, the input vector for the learning phase is x=(x1,x2,x3,x4,x5,x6) T In this case, we define this as the synaptic weight vector w, and set w = x. w = (w1, w2, w3, w4, w5, w6) T That is the case. Here, generally, two vectors v = (v1, v2, ..., v N )T , and u=(u1,u2,…,u N ) T Regarding this, if we denote the Hadamard product of vectors v and u as v○u, then v○u=(v1u1,v2u2,…,v N u N ) T This is the result. Now, when the components of vectors v and u are represented by the binary values ​​0 and 1, focusing on each component i, we get the Hadamard product v i u i is, v i , and, u i This can be considered as a logical AND operation when we consider the variables as logical units. Therefore, the Hadamard product operation described below can also be calculated as a logical AND operation for each component. Using this Hadamard product expression, we get w1·y1, C1, and ||y1|| in equation (75). 2 These are (w○X1)·y and C1=||x1|| respectively. 2 =||x○X1|| 2 , and, ||y1|| 2 =||y○X1|| 2 Therefore, fourthly, in the similarity determination phase, the similarity s calculated by unit i i It can be calculated as follows:

[0254]

number

[0255] The learning phase and the similarity determination phase, based on the above, are shown in Figures 22 and 23, respectively.

[0256] Figure 22 is a flowchart showing the processing during the learning phase of <Example 4>. In the learning phase, as described above, the input vector x of the learning phase is used to set the synaptic weight vector w as w=x (step S61).

[0257] In step S62, the parameter C of each division-normalized similarity calculation unit ii Regarding C i =||x i || 2 =||x○X i || 2 Calculate and set it as follows.

[0258] Next, we will describe the similarity determination phase processing in <Example 4>. Figure 23 is a flowchart showing the process in the similarity determination phase of <Example 4>. In step S71, for each division-normalized similarity calculation unit i, the similarity s i We find this using equation (78).

[0259] In step S72, the sum S of the output of the full division normalization similarity calculation unit is calculated as shown in equation (76).

[0260] In step S73, the obtained S is used as input to the activation function g(·) to calculate the output value V=g(S) of the diffusive learning network. Note that steps S72 and S73 in Figure 23 are the same as steps S52 and S53 in Figure 21 of <Example 3>.

[0261] Figures 22 and 23 illustrate the relationship between the "learning phase" and the "similarity determination phase" in the division-normalized similarity calculation unit i. In the first embodiment, the diffusive learning network 1000 has multiple division-normalized similarity calculation units i connected to multiple inputs of the diffusive learning network, each i having some or all of the inputs. Furthermore, the output of each division-normalized similarity calculation unit i is input to a perceptron. The division-normalized similarity calculation unit i accepts one or more input values, with each input receiving either value L or value H, and the value of the i-th input in the learning phase is x i Represented as, the value of the i-th input in the similarity determination phase is y i When expressed as such, the i-th input has the value w i It is assigned the value w iOne of two values, L or H, is set for this, and the weight value w is assigned to the i-th input during the learning phase. i to x i Set the value to x, and in the similarity determination phase, i The number of inputs where the value of is H, w i and y i The number of inputs where both values ​​are H, y i Calculate the number of inputs where the value of is H, and w i and y i The number of inputs where both values ​​are H is given by w i The input number y is given by value H. i A similarity calculation is performed by dividing the input number, which has a value of H, by the input number, and using that value as the similarity score, which represents the degree of similarity.

[0262] Furthermore, in this division-normalized similarity determination method, in the similarity determination phase, the division-normalized similarity score is calculated using equation (6) above, which incorporates the operation caused by a phenomenon called the shunt effect of nerve cells into a perceptron model.

[0263] Here, the "learning phase" corresponds to steps S1 and S2 in Figure 15, and the "similarity determination phase" corresponds to step S3 in Figure 15 and steps S11 to S15 in Figure 16. In other words, the "learning phase" is calculated in steps S1 and S2 in Figure 15, and the "similarity determination phase" is calculated in step S3 in Figure 15 and steps S11 to S15 in Figure 16.

[0264] By transforming equation (6) by considering different cases, we obtain equations (7) to (10). By analyzing these equations, it can be seen that the value calculated by the division-normalized similarity calculation method is an approximation of the cosine similarity. In other words, the similarity calculated by the division-normalized similarity calculation method can calculate a more accurate confirmed similarity than existing technologies. As a result, by accurately measuring the similarity between the information memorized in the learning phase and the information input in the similarity determination phase using the division-normalized similarity calculation method, it becomes possible to eliminate the discrepancies in information differences and the degree of similarity calculated in prior art, and to calculate similarity based on the degree of similarity. A detailed explanation follows below.

[0265] [Effects of Diffuse Learning Networks] The effects of the diffusive learning networks in Examples 1 through 4 will be explained. Since equation (73) above represents the expected value of the output of the division-normalized similarity calculation unit, the activity level of the perceptron (013 in Figure 5) that outputs the diffusion information network is proportional to equation (73) because it is the sum of the outputs of the division-normalized similarity calculation unit. The effect of this diffusion information network will be explained using Figures 24 to 35.

[0266] Figure 24 shows the effect of a diffusive learning network (step function, p=0.05 and k=0), Figure 25 shows the effect of a diffusive learning network (step function, p=1.0 and k=0), Figure 26 shows the effect of a diffusive learning network (step function, p=0.05 and m=0), Figure 27 shows the effect of a diffusive learning network (step function, p=1.0 and m=0), Figure 28 shows the effect of a diffusive learning network (step function, p=0.05 and m=k), and Figure 29 shows the effect of a diffusive learning network (step function, p=1.0 and m=k). Figure 30 shows the effect of a diffusive learning network (linear function, p=0.05 and k=0), Figure 31 shows the effect of a diffusive learning network (linear function, p=1.0 and k=0), Figure 32 shows the effect of a diffusive learning network (linear function, p=0.05 and m=0), Figure 33 shows the effect of a diffusive learning network (linear function, p=1.0 and m=0), Figure 34 shows the effect of a diffusive learning network (linear function, p=0.05 and m=k), and Figure 35 shows the effect of a diffusive learning network (linear function, p=1.0 and m=k).

[0267] Figure 24 shows the effect of the diffuse information network when m is varied, with the activation function of the perceptron in the division-normalized similarity calculation unit 100 being a step function, N=100, p=0.05, and k=0, as described above. The activation function thresholds shown are 0.9, 0.8, and 0.7. The vertical axis of Figure 24 represents the normalized value of the perceptron activity that outputs the diffusive learning network (the value obtained by dividing the activity of the perceptron that outputs the diffusive learning network by the number of division-normalized similarity calculation units, 100, and calculated using the above formula (73)). The horizontal axis of Figure 24 represents the number of inputs whose value is 1 during training and 0 during similarity determination (value of m). That is, when the horizontal axis is 0, it means that the same input as during training is received during similarity determination, and as the value of the horizontal axis increases, it means that the difference between the inputs during training and during similarity determination becomes larger.

[0268] As can be seen from Figure 24, as the difference between the inputs during training and the inputs during similarity determination increases, the activity level of the perceptron that outputs the diffusive learning network gradually decreases, indicating that the similarity between the inputs during training and the inputs during similarity determination is being determined with high accuracy.

[0269] Figure 25 shows the effect of the diffusive information network when p=1.0, compared to Figure 24. In this case, all inputs are connected to all division-normalized similarity calculation units in the same way, resulting in the same situation as when the diffusive learning network is not used. The vertical and horizontal axes in Figure 25 are the same as in Figure 24. As can be seen from Figure 25, the vertical axis is 1 from 0 on the horizontal axis up to a value determined by the threshold of the activation function of the perceptron in the division-normalized similarity calculation unit, and is 0 thereafter. Therefore, compared to when the diffusive information network is used with p<1.0, the range in which the similarity of inputs can be judged during learning and similarity judgment becomes narrower, and the degree of similarity can only be judged with two values, 1 and 0, resulting in a coarse judgment.

[0270] Figures 26 and 27 show the results of changing the value of k while keeping m=0 in Figures 24 and 25, respectively. The horizontal axis represents the number of inputs whose value is 0 during training and 1 during similarity determination (the value of k). That is, when the horizontal axis is 0, it means that the same inputs are received during similarity determination as during training, and as the value of the horizontal axis increases, it means that the difference between the inputs during training and similarity determination increases. In Figures 26 and 27, as with the comparison between Figures 24 and 25, it can be seen that when a spread-through information network is used with p<1.0, the similarity between the inputs during training and similarity determination can be determined with high accuracy.

[0271] Figures 28 and 29 show the results when the values ​​of m and k are changed simultaneously, with m=k, in Figures 24 and 25, respectively. The horizontal axis represents the situation where the input is the same as during training when the horizontal axis is 0, and increases as the value of the horizontal axis increases, indicating a greater difference between the inputs during training and during similarity determination. In Figures 55 and 29, as with the comparison between Figures 24 and 25, it can be seen that when a spread-through information network is used with p<1.0, the similarity between the inputs during training and during similarity determination can be determined with high accuracy.

[0272] Figures 30 to 35 show the case where the activation function of the perceptron in the division-normalized similarity calculation unit is a linear function, as shown in Figures 24 to 29. The major difference between using a linear activation function and a step function is the effect of the diffuse information network when p=1.0. In other words, there is a significant difference in a situation that is essentially the same as when not using a diffuse information network. With a step function, the output is 0 below the threshold and becomes 1 above the threshold. On the other hand, with a linear function, the output is a value proportional to the activation level when the threshold is exceeded. Therefore, as shown in Figures 25, 27, 29, 31, 33, and 35, similarity can be determined accurately when the threshold is exceeded. On the other hand, similarity cannot be determined below the threshold. From the above, it can be seen that even when the activation function is a linear function, when a spread information network is used with p<1.0, the similarity of the inputs during training and similarity determination can be determined with high accuracy.

[0273] [Effects of the First Embodiment] As described above, the similarity determination method (division-normalized similarity calculation method) according to the first embodiment (Figures 15-18) is a similarity determination method that calculates the degree of similarity between the input of the learning phase and the input of the similarity determination phase using a perceptron modeled after nerve cells, wherein there is one or more inputs, and each input is input to either two different values, L or H, and the value of the i-th input of the learning phase is x i Represented as, the value of the i-th input in the similarity determination phase is y iWhen expressed as such, the i-th input has the value w i It is assigned the value w i One of two values, L or H, is set for this, and the weight value w is assigned to the i-th input during the learning phase. i to x i Set the value to x, and in the similarity determination phase, i The number of inputs where the value H is w i and y i The number of inputs where both values ​​are H, y i Calculate the number of inputs for which the value H is w i and y i The number of inputs where both values ​​are H is given by w i The number of inputs and y whose value is H i The similarity score, representing the degree of similarity, is calculated and output by dividing the input number (where H is the value of the input) by the number of inputs where H is the value of the input.

[0274] In this way, the similarity between the information memorized in the learning phase and the information input in the similarity determination phase can be accurately measured using the division-normalized similarity calculation method (Figures 15-18). This eliminates the discrepancies in information differences and the calculated degree of similarity found in prior art, enabling similarity calculation based on the degree of similarity. The value calculated by the division-normalized similarity calculation method is an approximation of cosine similarity. As a result, the similarity calculated by the division-normalized similarity calculation method can calculate similarity more accurately than existing technologies, as described in Figures 26 to 35. Consequently, in an artificial neural network composed of perceptrons modeled after nerve cells, the similarity between information memorized in the network and newly input information can be accurately determined.

[0275] Furthermore, the similarity determination method according to the first embodiment (Figures 15-18) is a similarity determination method that calculates the degree of similarity between the input of the learning phase and the input of the similarity determination phase using a perceptron modeled after nerve cells. In the similarity determination phase, the division-normalized similarity is calculated using equation (6), which incorporates the operation caused by a phenomenon called the shunt effect of nerve cells into the perceptron model.

[0276] Here, the division-normalized similarity calculation method is implemented by (A) a method for determining synaptic weights, (B) a method for determining the division-normalization constant C, and (C) a method for determining the perceptron set corresponding to the neuron pool in division-normalization. Figures 1 and 2 show examples of circuits that perform the division-normalized similarity calculation method. Examples of operation of a diffusive learning network include Example 1 using a (step function) (Figures 7-10) and Example 2 using a (linear function) (Figures 11-14). Each of Examples 1 and 2 is further divided into a <learning phase> (Figures 7, 11) and a <similarity determination phase> (Figures 7-10).

[0277] The output of the perceptron is calculated using equation (4), and the division normalization constant C is calculated using equation (5). Substituting equation (5) into equation (4) and rearranging it yields equation (6). Rearranging equation (6) by considering different cases yields equations (7) to (10). By analyzing these equations, it can be seen that the value calculated by the division normalization similarity calculation method is an approximation of the cosine similarity. In other words, the similarity calculated by the division normalization similarity calculation method can calculate recognition similarity more accurately than existing technologies. As a result, the similarity between the information memorized in the learning phase and the information input in the similarity judgment phase can be accurately measured using the division normalization similarity calculation method. Consequently, the discrepancies in information differences and the degree of similarity calculated in prior art are eliminated, and similarity calculation based on the degree of similarity becomes possible.

[0278] In the similarity determination method (division-normalized similarity calculation method) according to the first embodiment (Figures 15-18), the input value L is set to 0 and the value H is set to 1, and in the similarity determination phase, x i The number of inputs for which x is value H is given for all i. i Calculate as the sum of w i and y i The number of inputs where both values ​​are H, for all i, w i and y i The sum of the products of, or w i and y i It is calculated as the sum of the logical ANDs of y i The number of inputs for which y is value H is the number of inputs for all i. i Calculate it as the sum of the two.

[0279] In this way, the value calculated by the division-normalized similarity calculation method is an approximation of the cosine similarity. As a result, the similarity between the information memorized in the learning phase and the information input in the similarity determination phase can be accurately measured using the division-normalized similarity calculation method.

[0280] In the similarity determination method (division-normalized similarity calculation method) according to the first embodiment (Figures 15-18), the calculated similarity is used as an input value to the activation function that defines the operation of the perceptron and the neuron, and the value calculated by the resulting activation function is output as a value representing the degree of similarity.

[0281] In this way, the value calculated by the division-normalized similarity calculation method is an approximation of the cosine similarity. Note that the value of the activation function with similarity as input is not the cosine similarity. As a result, the similarity between the information memorized in the learning phase and the information input in the similarity judgment phase can be accurately measured using the division-normalized similarity calculation method.

[0282] In the similarity determination method (division-normalized similarity calculation method) according to the first embodiment, the input value L is set to 0, the value H is set to 1, and the value X indicates that the i-th input is connected to the j-th similarity calculation unit among the multiple similarity calculation units (division-normalized similarity calculation unit 100) (Figures 1-14) that perform similarity calculation processing. ij It is represented as such, and the value X when connected. ij Set to 1, and if not connected, the value X ij Set X to 0. 1j , X 2j A vector whose components are X j When x represents a vector whose components are the inputs x1, x2, ... of the learning phase, and y represents a vector whose components are the inputs y1, y2, ... of the similarity determination phase, the vector w whose components are w1, w2, ... is set as w=x, and when the j-th similarity calculation unit calculates the similarity using the similarity determination method described in any one of claims 1 to 4, x i The number of inputs for which the value of is 1 is given by vector x and X. j The Hadamard product of x○X j Calculated as the square of the norm of w i and y i The number of inputs where both are 1 is measured by the Hadamard product w○X j The dot product of the vector represented by and vector y is calculated, and y i The number of inputs whose value is 1 is measured using the Hadamard product y○X j It is calculated as the square of the norm of .

[0283] In conventional techniques, when calculating the dot product of the input vector used for training and the input vector used for similarity determination, the dot product similarity may be the same even if there is a difference in the distance between the two input vectors used for training and the input vector used for similarity determination. In other words, a problem with the similarity calculation in conventional techniques is that the dot product similarity may not accurately determine the difference between the input vector used for training and the input vector used for similarity determination.

[0284] In contrast, in the division-normalized similarity calculation method according to the first embodiment, when calculating similarity, in addition to multiplication, addition, and subtraction, division is performed using an operation caused by a phenomenon called the shunt effect of nerve cells (neurons), and this division is x i The number of inputs for which the value of is 1 is given by vector x and X. j The Hadamard product of x○X j Calculated as the square of the norm of w i and y i The number of inputs where both are 1 is measured by the Hadamard product w○X j The dot product of the vector represented by and vector y is calculated, and y i The number of inputs whose value is 1 is measured using the Hadamard product y○X j It is calculated as the square of the norm of . As a result, the value calculated by the division-normalized similarity calculation method becomes an approximation of the cosine similarity, and the similarity calculated by the division-normalized similarity calculation method can calculate the similarity more accurately than existing technologies. In addition, by using the Hadamard product, the calculation speed can be greatly improved or the circuit size can be greatly reduced.

[0285] The similarity calculation unit according to the first embodiment performs similarity calculation based on the similarity determination method described above (Figures 15-18).

[0286] By doing so, it is possible to realize a unit circuit device that can calculate similarity more accurately than existing technologies.

[0287] Multiple similarity calculation units (division-normalized similarity calculation units 100) (Figures 1-14) are connected to each other, each having some or all of the inputs, and the output of each similarity calculation unit is input to a perceptron in a spread learning network 1000 (Figures 5-14), where each similarity calculation unit has one or more inputs, and each input has one of two different values, L or H, and the value of the i-th input in the learning phase is x i Represented as, the value of the i-th input in the similarity determination phase is y i When expressed as such, the i-th input has the value wi It is assigned the value w i One of two values, L or H, is set for this, and the weight value w is assigned to the i-th input during the learning phase. i to x i Set the value to x, and in the similarity determination phase, i The number of inputs where the value H is w i and y i The number of inputs where both values ​​are H, y i Calculate the number of inputs where the value is H, w i and y i The number of inputs where both values ​​are H is given by w i The input number y is given by value H. i A similarity calculation is performed by dividing the input number, which has a value of H, by the input number, and using that value as the similarity score, which represents the degree of similarity.

[0288] By doing so, it becomes possible to realize a diffusive learning network that can calculate similarity more accurately than existing technologies. For example, if applied to an artificial neural network composed of perceptrons modeled after nerve cells, it becomes possible to accurately determine the similarity between information stored in the network and newly input information.

[0289] In the diffusive learning network 1000 (Figures 5-14), multiple similarity calculation units 100 are combined, with one or more inputs from the entire input being used as input to each similarity calculation unit. Each similarity calculation unit calculates the similarity, and the sum of the similarities calculated by all similarity calculation units is output as the final similarity.

[0290] By doing so, it is possible to realize a diffusive learning network that can calculate similarity more accurately than existing technologies.

[0291] Here, in calculating the Hadamard product of two vectors, if we let u be the vector whose components are u1, u2, ... and v be the vector whose components are v1, v2, ..., then the i-th component of the Hadamard product u○v is v i and u iBy performing calculations using logical AND, the calculation speed can be dramatically improved, or the circuit size can be significantly reduced.

[0292] In the diffuse learning network 1000 (Figures 5-14), instead of using the sum of the similarity scores calculated by all similarity calculation units as the similarity score, the similarity score is either the sum of the similarity scores divided by the total number of similarity calculation units, or the sum of the values ​​obtained by pre-dividing each of the similarity scores calculated by all similarity calculation units by the total number of similarity calculation units.

[0293] By doing so, it is possible to realize a diffusive learning network that can calculate similarity more accurately than existing technologies. Furthermore, by dividing the sum of the similarity scores by the number of each similarity calculation unit to obtain the similarity score, or by pre-dividing each of the similarity scores calculated by all similarity calculation units by the total number of similarity calculation units and using that as the sum of the values, it is possible to prevent overflow when using a computer, for example, by pre-dividing by the total number of similarity calculation units. Moreover, when comparing values ​​calculated by multiple diffusive learning networks, if there are a sufficient number of similarity calculation units, the values ​​calculated by each diffusive learning network will converge to equation (73), which shows that it is sometimes possible to compare the values ​​of multiple diffusive learning networks even if the number of similarity calculation units differs by a certain number among them.

[0294] The present invention is not limited to the first embodiment described above, and includes other modifications and applications, as long as they do not depart from the spirit of the invention as described in the claims. For example, instead of logic gates as multiplication circuits, LUTs (Look-Up Tables) may be used. LUTs are a fundamental component of FPGAs (Field Programmable Gate Arrays), which are accelerators, and have high compatibility with FPGA synthesis, making FPGA implementation easy. Alternatively, a GPU (Graphics Processing Unit) or ASIC (Application Specific Integrated Circuit) may be used as the accelerator.

[0295] (Second Embodiment) In the first embodiment, a [division-normalized similarity determination method] and a [diffusion-type learning network method] were combined. In the second embodiment, the [division-normalization type similarity determination method] and the [diffusion-type learning network method] are further combined with the [noise-added type sensitivity characteristic improvement method].

[0296] [Division normalization type similarity judgment method] First, I will explain the noise-added method for improving sensitivity characteristics. Generally, the sensitivity of a measuring instrument is expressed as the ratio of the instrument's indication to the observed value. On the other hand, the [division-normalized similarity determination method] and the [diffusion-type learning network method] described in the first embodiment can be considered as measuring instruments for measuring the similarity of data in the learning phase and the similarity determination phase. Figures 36 and 37 are used to explain the characteristics of these as measuring instruments.

[0297] Figure 36 shows the activity level (N=100) of the perceptron that outputs the diffusion information network when using only the division-normalized similarity calculation method and the diffusion learning network. Figure 37 shows the activity level (N=1000) of the perceptron that outputs the diffusion information network when using only the division-normalized similarity calculation method and the diffusion learning network. Figures 36 and 37 show that the difference between the data in the learning phase and the similarity determination phase increases as the horizontal axis moves to the right. The vertical axis represents the similarity calculated when using the [division-normalized similarity determination method] and the [diffusion learning network method], and is the value calculated by equation (73). The activation function included in equation (73) used in Figures 36 and 37 is the sigmoid function. The sigmoid function is expressed by the following equation (79). In this equation, β and τ are parameters representing the slope and threshold, respectively.

[0298]

number

[0299] The parameters included in equations (73) and (79) are p = 0.05 and β = 1.0 × 10⁻⁶. 4 τ = 0.9. The value of N is 100 and 1000 in Figures 36 and 37, respectively. As shown in dashed box a in Figure 36 and dashed boxes b and c in Figure 37, the slope of the curve is almost zero and nearly horizontal where the perceptron activity is close to 0.0 and close to 1.0.

[0300] The fact that the slope of the curve shown in Figure 36 is horizontal means that the similarity calculated does not change due to the difference in data between the learning phase and the similarity determination phase, resulting in poor sensitivity. Thus, when using only the [division-normalized similarity determination method] and the [diffusion-type learning network method] of the first embodiment, a problem arises in which there are parts where the sensitivity for measuring similarity is poor (Note 1).

[0301] Furthermore, comparing Figures 36 and 37, we can see that the curves differ due to the difference in N, which represents the square of the norm of the training data. For example, when the value on the horizontal axis is 0.3, the values ​​on the vertical axis are 0.302 and 0.0287 in Figures 36 and 37, respectively. Therefore, when various training data have different values ​​of N, even if the proportion of the difference is roughly the same relative to the training data, different similarity scores will be output. This creates a problem where it becomes difficult to compare the similarity between different training data with different values ​​of N (Note 2).

[0302] Furthermore, equation (7), used in the [division-normalized similarity determination method] of the first embodiment, is an approximation of cosine similarity, which is mathematically defined, has had its characteristics thoroughly analyzed, and whose effectiveness has been demonstrated. However, after calculating the activity using equation (7), a transformation is performed using an activation function, and in addition, processing is carried out using the [diffuse learning network method] of the first embodiment, which leads to the problem that the mathematically defined characteristics become unclear (Note 3).

[0303] The noise-adding sensitivity characteristic improvement method described in the second embodiment below is a technology that solves these points 1 to 3. The [Noise-Adding Sensitivity Characteristic Improvement Method] calculates the similarity Sd, which is expressed by equation (7) used in the [Division-Normalized Similarity Determination Method] and the [Diffusion-Type Learning Network Method] of the first embodiment, and then calculates the similarity Sg, which is obtained by adding noise to Sd, as shown in the following equation (80).

[0304]

number

[0305] Here, G is the value of a random variable X that is randomly generated according to the probability density function P(X) that generates the random variable X. This value is generated anew each time Sg is calculated. Furthermore, after calculating Sg, Sg will be used instead of Sd when processing the [division-normalized similarity determination method] and the [diffusion learning network method].

[0306] Thus, we consider the expected value of the output of the division-normalized similarity calculation unit when Sg is used instead of Sd. In a given division-normalized similarity calculation unit, the probability of the random variable X occurring is P(X)dX. As shown in equation (73), if the activation degree and activation function when noise is not added are S(n,d,l) and f(·), respectively, then when S is used, the output of this division-normalized similarity calculation unit becomes f(S(n,d,l)+X). In this equation, the value G of the randomly generated random variable mentioned above is represented by X.

[0307] If there are a sufficiently large number of division-normalized similarity calculation units, then we can assume that there are also a sufficient number of division-normalized similarity calculation units with the same activity level S(n,d,l). Therefore, the expected value of the output of a division-normalized similarity calculation unit with activity level S(n,d,l) is given by equation (81).

[0308]

number

[0309] Furthermore, since the probability that the activity level is S(n,d,l) is calculated in the process of deriving equation (73), the expected value of the output of the division-normalized similarity calculation unit can be expressed as equation (82) below by using the probability that the activity level is S(n,d,l).

[0310]

number

[0311] The characteristics of the similarity calculated by the division-normalized similarity calculation unit using equation (82) are explained with reference to Figures 38 and 39.

[0312] Figure 38 shows the activity level of the perceptron that outputs the diffusion information network when using a division-normalized similarity calculation method, a diffusion learning network, and a noise-added sensitivity enhancement method (changes in output when the number of inputs, where the input value is 1 during training and 0 during similarity determination, is changed). Figure 39 shows the activity level of the perceptron that outputs the diffusion information network when using a division-normalized similarity calculation method, a diffusion learning network, and a noise-added sensitivity enhancement method (changes in output when the number of inputs, where the input value is 0 during training and 1 during similarity determination, is changed). In Figures 38 and 39, the vertical axis represents the activity level of the perceptron that outputs the diffusive learning network, and the horizontal axis represents the percentage of data in the similarity judgment phase that differs from the data in the learning phase.

[0313] In Figures 38 and 39, the sigmoid function is used as the activation function, and the parameters included in equation (82) and the parameters included in equation (79) representing f(·) in equation (82) are p=0.05 and β=1.0×10 4 τ = 0.9. Furthermore, the values ​​of N are shown for 25, 50, 100, and 1000. Additionally, the probability density function P(X) in equation (82) uses the probability density function of a Gaussian distribution with mean and standard deviation of 0.01 and 0.5, respectively.

[0314] Figures 38 and 39 show that the difference between the data in the learning phase and the similarity judgment phase increases as the horizontal axis moves to the right. The vertical axis represents the activity level of the perceptron that outputs the diffusive learning network, calculated by equation (82).

[0315] As can be seen from Figures 38 and 39, the activity level of the perceptron that outputs the diffusive learning network always has a negative slope as the value on the horizontal axis increases. From this, it can be seen that by using the activity level of the perceptron that outputs the diffusive learning network as the similarity, the problem of poor sensitivity in measuring similarity in parts (Note 1) is solved. Furthermore, in Figures 38 and 39, it can be seen that for N=100 or more, it is almost independent of N, and the problem of difficulty in comparing similarity with different training data with different N values ​​(Note 2) is solved.

[0316] To explain that (Note 3) has been resolved, we will describe a method for representing the degree of similarity between two sets, called the Tanimoto similarity or Jaccard similarity, as described in Non-Patent Document 6 and Non-Patent Document 7. In this specification, these equivalent definitions of similarity will be abbreviated as Tanimoto similarity. Now consider two sets A and B. Tanimoto similarity S T This can be expressed by the following equation (83).

[0317]

number

[0318] In equation (83), |A| represents the number of elements in set A. Here, using the notation used in equation (7), we get the Tanimoto similarity S T Let's consider how to represent this. In this case, if we consider the two sets as the set of components whose value is 1 in the input vector w of the learning phase, and the set of components whose value is 1 in the input vector y of the similarity determination phase, then using the notation used in equation (7), |A∩B|=n 11 |A|=n 11 +n 10 |B|=n 11 +n 01 This is the result. Substituting these into equation (83), we get equation (84) below.

[0319]

number

[0320] The number of components N in w whose value is 1 is given by N = n 11 +n 10 Therefore, when this equation is rearranged, n 11 =Nn 10 Substituting this into equation (84), we get the Tanimoto similarity S T The result is as shown in equation (85) below.

[0321]

number

[0322] Here, by introducing a constant C, S can be expressed by the following equation (86). RT Define.

[0323]

number

[0324] S in equation (86) RT From now on, we will refer to this as the raised Tanimoto similarity. Now, let's call the Tanimoto similarity included in the two raised Tanimoto similarities S T (1) and S T (2) Let's assume that the difference in raised Tanimoto similarity calculated from these is given by the following equation (87).

[0325]

number

[0326] From the above, it can be seen that the difference in raised Tanimoto similarity is a constant multiple of the difference in Tanimoto similarity. From this, it can be seen that when comparing the magnitude of the difference between two sets, both Tanimoto similarity and raised Tanimoto similarity can be used for comparison. The Tanimoto similarity measure is a mathematically defined and widely applied similarity measure, and its effectiveness has been demonstrated in various fields.

[0327] Figure 40 compares the activity level of the perceptron that outputs the diffusion information network (change in output when the number of inputs, where the input value is 1 during training and 0 during similarity determination, is changed) and the raised Tanimoto similarity when using the division-normalized similarity calculation method, the diffusion learning network, and the noise-added sensitivity enhancement method. Figure 41 compares the activity level of the perceptron that outputs the diffusion information network (change in output when the number of inputs, where the input value is 0 during training and 1 during similarity determination, is changed) and the raised Tanimoto similarity when using the division-normalized similarity calculation method, the diffusion learning network, and the noise-added sensitivity enhancement method.

[0328] In Figures 40 and 41, the value of C in equation (86) for the raised Tanimoto is 0.03. In Figures 40 and 41, the raised Tanimoto similarity is represented as Raised-Tanimoto. For comparison, the above raised Tanimoto similarity value is also used for the Tanimoto similarity S included in equation (86). T The coefficient (1-C) is calculated as (DC). Here, D is the activity level of the perceptron that outputs the diffusive learning network when the horizontal axis is 0.

[0329] As can be seen from Figures 40 and 41, the slope of the activity of the perceptron that outputs the diffusive learning network is always a negative value, thus resolving (Note 1). Furthermore, even when there are different values ​​of N in the training data, for N=100 or greater, the activity of the perceptron that outputs the diffusive learning network is close to a similar value, thus resolving (Note 2). In addition, the fact that the activity of the perceptron that outputs the diffusive learning network is close to the raised Tanimoto similarity value indicates that (Note 3) is resolving.

[0330] <Example 5> Example 5 describes an example of the processing in the similarity determination phase. The learning phase of Example 5 of the second embodiment is the same as that of Example 1 of the first embodiment, and the process described in Figure 15 is performed. After the learning phase shown in Figure 15, the similarity determination phase shown in Figure 42 is performed. Figure 42 is a flowchart showing the processing in the similarity determination phase of the division-normalized similarity calculation unit of the second embodiment. Steps that perform the same processing as in Figure 16 are denoted by the same reference numerals.

[0331] In step S11, the division-normalized similarity calculation unit 100 calculates the input vector y=(y1,y2,…,y) for the similarity determination phase. N ) T Receive.

[0332] In step S12, the division-normalized similarity calculation unit 100 calculates the similarity using Y=||y|| 2 Calculate.

[0333] In step S13, the division-normalized similarity calculation unit 100 calculates Z = w·y, which is necessary to calculate the similarity.

[0334] In step S14, the division-normalized similarity calculation unit 100 calculates the similarity s according to formula (74) using the calculated Y and Z, as well as the parameter C calculated in step S3 of Figure 15.

[0335] After performing the processes in steps S11 to S14 above, a random variable X that follows the probability density function P(X) is randomly generated, and the generated random variable is called G (step S81).

[0336] In other words, in step S81, the division-normalized similarity calculation unit 100 generates a random variable X that follows the probability density function P(X), and this is called G.

[0337] In step S82, the division-normalized similarity calculation unit 100 inputs the calculated similarity s and G generated from the random variable X into the activation function f(a) to obtain the output value f(s+G). This output value f(s+G) becomes the output of the division-normalized similarity calculation unit 100.

[0338] The probability density function used here is not limited to a specific distribution; it can be a Gaussian distribution, a normal distribution, a Poisson distribution, a Weibull distribution, or any other distribution. Then, using G, the similarity s calculated in step S14, and the activation function f(a), f(s+G) is calculated, and this value is output.

[0339] The activation function can be the commonly used ReLU or a step function. Alternatively, it can be a simple linear function, a threshold-linear function, a sigmoid function, or a radial-basis function, as described in Non-Patent Literature 2. Furthermore, for functions with a threshold of 0, any other arbitrary value can be used as the threshold.

[0340] <Example 6> Example 6 describes an example of the processing in the similarity determination phase. The learning phase of Example 6 of the second embodiment is the same as that of Example 2 of the first embodiment, and the process described in Figure 17 is performed. After the learning phase shown in Figure 17, the similarity determination phase shown in Figure 43 is performed. Figure 43 is a flowchart showing the processing in the similarity determination phase of the division-normalized similarity calculation unit of the second embodiment. Steps that perform the same processing as in Figure 18 are denoted by the same reference numerals.

[0341] In step S31, the division-normalized similarity calculation unit 100 calculates the input vector y=(y1,y2,…,y) for the similarity determination phase. N ) T Receive.

[0342] In step S32, the division-normalized similarity calculation unit 100 calculates the similarity using Y=||y|| 2 Calculate. At this time, Y = Σ N i=1 y i Calculate as follows.

[0343] In step S33, the division-normalized similarity calculation unit 100 calculates Z=w·y, which is necessary to calculate the similarity. At this time, Z=Σ N i=1 (w i Newly i ) is calculated as follows: Here, w i Newly i is, w i and y i This represents a logical AND operation.

[0344] In step S34, the division-normalized similarity calculation unit 100 calculates the similarity s according to equation (74), using the calculated Y and Z, as well as the parameter C calculated in step S23 of Figure 17.

[0345] After performing the processes in steps S31 to S34 above, a random variable following the probability density function P(X) Let G be a randomly generated random variable.

[0346] In other words, in step S91, the division-normalized similarity calculation unit 100 generates a random variable X that follows the probability density function P(X), and this is called G.

[0347] In step S92, the division-normalized similarity calculation unit 100 inputs the calculated similarity s and G generated from the random variable X into the activation function f(a) to obtain the output value f(s+G). This output value f(s+G) becomes the output of the division-normalized similarity calculation unit 100.

[0348] The probability density function used here is not limited to a specific distribution; it can be a Gaussian distribution, a normal distribution, a Poisson distribution, a Weibull distribution, or any other distribution. Then, using G, the similarity s calculated in step S34, and the activation function f(?), f(s+G) is calculated, and this value is output.

[0349] The activation function can be the commonly used ReLU or a step function. Alternatively, it can be a simple linear function, a threshold-linear function, a sigmoid function, or a radial-basis function, as described in Non-Patent Literature 2. Furthermore, for functions with a threshold of 0, any other arbitrary value can be used as the threshold.

[0350] <Example 7> Example 7 describes an example of the processing in the similarity determination phase. The learning phase of Example 7 of the second embodiment is the same as that of Example 3 of the first embodiment, and the process described in Figure 20 is performed. After the learning phase shown in Figure 20, the similarity determination phase shown in Figure 44 is performed. Figure 44 is a flowchart showing the processing in the similarity determination phase of the division-normalized similarity calculation unit of the second embodiment. Steps that perform the same processing as in Figure 21 are denoted by the same reference numerals. In step S101, each division-normalized similarity calculation unit i generates a random variable X that follows the probability density function P(X), and this G i Let's assume that.

[0351] In step S102, each division-normalized similarity calculation unit i calculates similarity s i Calculate the output of each division-normalized similarity calculation unit i, and take the output of f(s i +G i ) . That is, in step S102, each division-normalized similarity calculation unit i performs the processing of the similarity determination phase of each division-normalized similarity calculation method and sets the output value of each division-normalized similarity calculation unit i to f(s i +G i Set it as ).

[0352] In step S103, each division-normalized similarity calculation unit i calculates the sum of the outputs of all division-normalized similarity calculation units S = Σ i f(S i +G i Calculate (Equation (83)).

[0353] In step S53, the obtained S is used as input to the activation function g(·) to calculate the output value V=g(S) of the diffusive learning network.

[0354] Here, the activation function can be the commonly used ReLU or a step function. Alternatively, it can be a simple linear function, a threshold-linear function, a sigmoid function, or a radial-basis function, as described in Non-Patent Document 2. In addition, the activation function can be the k-Winner-Take-All (kWTA) or Winner-Take-All (WTA) as described in Non-Patent Document 3. Furthermore, for these functions, if the threshold is 0, any other arbitrary value can be used as the threshold.

[0355] <Example 8> Example 8 describes an example of the processing in the similarity determination phase. The learning phase of Example 8 of the second embodiment is the same as that of Example 4 of the first embodiment, and the process described in Figure 22 is performed. After the learning phase shown in Figure 22, the similarity determination phase shown in Figure 45 is performed. Figure 45 is a flowchart showing the processing in the similarity determination phase of the division-normalized similarity calculation unit of the second embodiment. Steps that perform the same processing as in Figure 23 are denoted by the same reference numerals. In step S111, each division-normalized similarity calculation unit i generates a random variable X that follows the probability density function P(X), and this G i Let's assume that.

[0356] In step S71, each division-normalized similarity calculation unit i calculates similarity s i This is calculated using equation (78) above.

[0357] In step S112, each division-normalized similarity calculation unit i calculates the sum of the outputs of all division-normalized similarity calculation units S = Σ i f(S i +G i Calculate (Equation (83)).

[0358] In step S73, the obtained S is used as input to the activation function g(·) to calculate the output value V=g(S) of the diffusive learning network.

[0359] [Effects of the second embodiment] In the similarity determination method according to the second embodiment (Figures 36-45), a similarity is obtained by adding a predetermined noise to the calculated similarity, and thereafter, calculations are performed using the similarity with the added noise.

[0360] In other words, in the second embodiment, after calculating the similarity Sd represented by (1) a division-normalized similarity calculation method and (2) a diffusion-type learning network method, a similarity Sg is obtained by adding noise, and thereafter calculations are performed using Sg instead of Sd.

[0361] When using only (1) the division-normalization similarity calculation method and (2) the diffusion learning network method of the first embodiment, there were some areas where the sensitivity for measuring similarity was poor (Note 1), it was difficult to compare the similarity with different training data having different N (number of inputs) values ​​(Note 2), and the mathematically defined characteristics became unclear as a result of performing the above processing (1) and (2) (Note 3).

[0362] In the second embodiment, by performing calculations using the noise-added similarity Sg, the parts where the sensitivity for measuring similarity is poor in some areas are eliminated, as can be seen by comparing Figures 36 and 38, and Figures 37 and 39 (resolution of Note 1). Also, as shown in Figures 40 and 41, the activity levels of the perceptrons that output the diffusive learning network are close to the same value (resolution of Note 2). Furthermore, the activity level of the perceptrons that output the diffusive learning network is close to the raised Tanimoto similarity (resolution of Note 3).

[0363] In the similarity determination method according to the second embodiment (Figures 36-45), a similarity Sg is obtained by adding a predetermined noise to the calculated similarity Sd, and the final similarity calculation is performed using the similarity Sg with the added noise.

[0364] By doing so, the above points (1) to (3) can be resolved.

[0365] In the similarity determination method according to the second embodiment (Figures 36-45), the noise is randomly generated random numbers.

[0366] In this way, randomly generated numbers can be easily produced, for example, by a random number generation circuit, and by using these random numbers as noise, the computational complexity of similarity calculations can be reduced.

[0367] (Third embodiment) The third embodiment is an example of applying a division-normalized similarity calculation method using fuzzy logic. In the first and second embodiments, the vector w = (w1, w2, w3, ...) represents the synaptic weights set by the input in the learning phase. T and the vector y=(y1,y2,y3,…) representing the input for the similarity determination phase. T To calculate the similarity, we have used equations (6) and (7) above. In equations (6) and (7), we have explained that we use the following equation (88), assuming that each component of vectors w and y can only take values ​​of 0 or 1.

[0368]

number

[0369] Here, (y·w) in equation (88) represents the inner product, Σ i w i y i Therefore, when using equation (88), the input value can only be either 0 or 1. Consequently, it cannot be applied to applications that deal with multi-level values ​​rather than just two levels of brightness, such as the brightness of an image, or to applications that deal with non-stepless values ​​such as real numbers. To solve this problem, from here on we will use the Fuzzy logic described in Non-Patent Document 9, as in Non-Patent Document 8, so that the input value can be any real number from 0 to 1. In this way, for example, the input value x i When the value x is within the range from the minimum value L to the maximum value H, i (x i By replacing it with -L) / (HL), it becomes possible to convert from 0 to a real number of 1, and thus the above problem can be solved using fuzzy logic.

[0370] Let me explain this substitution. 0≦w i ≤1, 0 ≤ y i Assuming ≤ 1, the input x during learning when w is determined is (x1, x2, x3, ...). T Regarding the components, 0≦x i Let ≤ 1, and further Σi w i y i Σ i w i ∧ F y i Rewrite it as follows. Here, w i ∧ F y i ∧ inside F is an operator, and p∧ F The value of q is the smaller of p and q. More specifically, when p ≥ q, p ∧ F The value of q is q. With this swap, equation (88) becomes equation (89).

[0371]

number

[0372] In equation (89), z i =w i ∧ F y i That is the case. Regarding the characteristics of equation (89), we will explain the range of values ​​that equation (89) can take, the conditions under which the value of equation (89) reaches its maximum, and how the value of equation (89) changes when it deviates from the conditions for reaching the maximum. First, let's explain the range of possible values ​​for equation (89). The range of possible values ​​for the variable used in equation (89) is 0 ≤ w i ≤1, 0 ≤y i ≤ 1, and 0 ≤ z i Since ≤ 1, equation (89) can never take a negative value. Also, for any i, z i Since the value of equation (89) is 0 when = 0, we can conclude that the value of equation (89) is greater than or equal to 0. Next, using equation (89), we get equation (90) and the maximum value is 1.

[0373]

number

[0374] From the above discussion, it can be seen that the value of equation (89) is greater than or equal to 0 and less than or equal to 1. Secondly, we will explain the conditions under which the value of equation (89) reaches its maximum value. Since the maximum value of equation (89) is 1, we obtain the following conditional equation (91).

[0375]

number

[0376] Transforming this gives us equation (92).

[0377]

number

[0378] Further transformation yields equation (93).

[0379]

number

[0380] In equation (93), w i -z i ≥ 0, and y i -z i Since ≥ 0, the condition for satisfying equation (93) is that for any i, w i =z i , and y i =z i That is the case. Therefore, w i =y i =z i Therefore, the condition for the value of equation (89) to take its maximum value is that for any i, w i =y i This is when it happens. Thirdly, we will explain how the value of equation (89) changes when it deviates from the condition under which the value of equation (89) is at its maximum. In equation (89), w iThis is determined in the learning phase and is a constant in the similarity determination phase. Therefore, equation (89) is changed to equation (94), y k We perform partial differentiation using this method.

[0381]

number

[0382] First, w k <y k Considering the time, z k w k Therefore, equation (94) becomes equation (95) below.

[0383]

number

[0384] Here, w i , and, y i If not all of them are zero, the denominator of the above equation is clearly a positive value, and the numerator of equation (95) is clearly a negative value. From this, w k <y k Within the range, the value of equation (89) is y k It can be seen that it decreases monotonically as the value increases. Next, w k ≧y k Considering the time, z k =y k Therefore, equation (95) becomes equation (96) below.

[0385]

number

[0386] Here, w k , and, y k If not all of them are zero, then the denominator of equation (96) is clearly a positive value, and the numerator of equation (96) is also clearly a positive value. From this, w k≧y k Within the range, the value of equation (89) is y k It can be seen that it is monotonically increasing as increases. From the above discussion, it can be seen that as the value of equation (89) moves away from the condition in which it is the maximum value, the value of equation (89) decreases monotonically as it moves away.

[0387] Figure 46 illustrates an example of similarity calculated using a division-normalized similarity method with fuzzy logic. Figure 46 shows the change in similarity when y=(y1,y2) is varied, given that w=(w1,w2)=(0.5,0.5). In other words, it shows the similarity calculation results when fuzzy logic is applied to w=(w1,w2)=(0.5,0.5). In Figure 46, y=(y1,y2) is varied. The similarity in Figure 46 is calculated based on equation (89). As can be seen from Figure 46, the similarity decreases as y=(y1,y2) moves away from y=(0.5,0.5).

[0388] Here, in the case where Fuzzy logic is not used, equations (9) and (10) were used to explain that as the change in vector y from vector w increases, the similarity expressed by equation (7) decreases. In the above explanation, the change in vector y from vector w is for each element y i w i This means a change from . In other words, an element changes from 0 to 1, and from 1 to 0, and as a result, n 10 , and, n 01 This was explained as a change in similarity when y increases. When using fuzzy logic, each element changes continuously, so partial derivatives are used to express each element y i w i The change in the calculated similarity in response to the change from is explained by equations (95) and (96), and the change in numerical similarity is explained in Figure 46. From the above, it can be seen that the properties described here are the same as when calculating similarity using equations (6) and (7), and therefore the equation used to calculate similarity can be replaced with equation (89).

[0389] <Example 9> <Example 9> describes the processing of the learning phase using a division-normalized similarity calculation method with fuzzy logic, the processing of the similarity determination phase without using a noise-added sensitivity characteristic improvement method, and the processing of the similarity determination phase when a noise-added sensitivity characteristic improvement method is used. The learning phase of <Example 9> of the third embodiment is the same as that of <Example 2> of the first embodiment, and the process described in Figure 17 is performed. After the learning phase shown in Figure 17, the similarity determination phase shown in Figure 47 is performed.

[0390] Figure 47 is a flowchart showing the processing of the learning phase using a division-normalized similarity calculation method with fuzzy logic. Steps that perform the same processing as in Figure 17 are denoted by the same reference numerals and their explanations are omitted.

[0391] In step S121, the division-normalized similarity calculation unit 100 calculates the parameter C used in the similarity determination phase as C = Σ N i=1 x i Calculate and set it as follows. After the learning phase shown in Figure 47, the similarity determination phase shown in Figures 48 and 49 is performed.

[0392] Next, we will discuss the similarity determination phase processing of a diffusive learning network. Figure 48 is a flowchart showing the processing in the similarity determination phase of a division-normalized similarity calculation unit when a noise-added sensitivity characteristic improvement method is not used. Steps that perform the same processing as in Figure 18 are denoted by the same reference numerals.

[0393] In step S31, the division-normalized similarity calculation unit 100 calculates the input vector y=(y1,y2,…,y) for the similarity determination phase. N ) T Receive.

[0394] In step S131, the division-normalized similarity calculation unit 100 calculates Y = Σ N i=1 y i Calculate. In step S132, the division-normalized similarity calculation unit 100 calculates z = Σ N i=1 (w i ∧ F y i Calculate ).

[0395] In step S133, the division-normalized similarity calculation unit 100 calculates the similarity as s = 2Z / (C+Y).

[0396] In step S35, the division-normalized similarity calculation unit 100 inputs the calculated similarity s into the activation function f(a) to obtain the output value f(s). This output value f(s) is the output of the division-normalized similarity calculation unit 100 when the noise-added sensitivity characteristic improvement method is not used.

[0397] Figure 49 is a flowchart showing the processing in the similarity determination phase of the division-normalized similarity calculation unit when using the noise-added sensitivity characteristic improvement method. Steps that perform the same processing as in Figure 48 are denoted by the same reference numerals.

[0398] In step S31, the division-normalized similarity calculation unit 100 calculates the input vector y=(y1,y2,…,y) for the similarity determination phase. N ) T Receive.

[0399] In step S131, the division-normalized similarity calculation unit 100 calculates Y = Σ N i=1 y i Calculate.

[0400] In step S132, the division-normalized similarity calculation unit 100 calculates z = Σ N i=1 (w i ∧ F y i Calculate ).

[0401] In step S133, the division-normalized similarity calculation unit 100 calculates the similarity as s = 2Z / (C+Y).

[0402] In step S134, the division-normalized similarity calculation unit 100 generates a random variable X that follows the probability density function P(X), and denotes this as G.

[0403] In step S135, the division-normalized similarity calculation unit 100 calculates f(s+G) as an output value. This output value f(s+G) is the output of the division-normalized similarity calculation unit 100 when the noise-added sensitivity characteristic improvement method is used.

[0404] [Effects of the third embodiment] In the similarity determination method according to the third embodiment (Figures 46-49), the input value is replaced with a value that can take any real number from 0 to 1 using fuzzy logic.

[0405] This approach allows the system to be applied to applications where the input value is not limited to just 0 or 1, such as when dealing with multi-level values ​​rather than just two levels of brightness (like the brightness of an image), or when dealing with infinitely variable values ​​such as real numbers.

[0406] In the similarity determination method according to the third embodiment (Figures 46-49), in the substitution of input values ​​using Fuzzy logic, in the similarity determination phase, w i The sum of the values, w i and y i The sum of the smaller values ​​of y i Calculate the three values ​​of the sum of the values ​​of w i and y i The value representing the sum of the smaller values ​​is w i The value representing the sum of the values ​​and y iThe value obtained by dividing by the sum of the values ​​is calculated and output as the similarity score, which represents the degree of similarity.

[0407] This approach allows the system to handle input values ​​that are not limited to just 0 or 1, such as multi-level values ​​like the brightness of an image (which is not just two levels of light or dark), or non-steppy values ​​like real numbers.

[0408] [Hardware configuration] The division-normalized similarity calculation unit 100 (Figures 1 to 14) according to the first to third embodiments described above is implemented by a computer 900 having a configuration such as that shown in Figure 50. Figure 50 is a hardware configuration diagram showing an example of a computer 900 that implements the functions of the division-normalized similarity calculation unit 100. Computer 900 includes a CPU 901, RAM 902, ROM 903, HDD 904, accelerator 905, input / output interface (I / F) 906, media interface (I / F) 907, and communication interface (I / F) 908. Accelerator 905 corresponds to the division-normalized similarity calculation unit 100 shown in Figures 1 to 14.

[0409] Accelerator 905 is a division-normalized similarity calculation unit 100 (Figures 1 to 14) that processes at high speed data from either the communication interface 908 or the RAM 902. Alternatively, accelerator 905 may be of a type that returns the execution result to the CPU 901 or RAM 902 after processing from the CPU 901 or RAM 902 (look-aside type). Alternatively, accelerator 905 may be of an in-line type that performs processing between the communication interface 908 and the CPU 901 or RAM 902.

[0410] The accelerator 905 is connected to the external device 915 via the communication interface 908. The input / output interface 906 is connected to the input / output device 916. The media interface 907 reads and writes data to the recording medium 917.

[0411] The CPU 901 operates based on a program stored in the ROM 903 or HDD 904, and controls each part of the division-normalized similarity calculation unit 100 shown in Figures 1 to 14 by executing the program (also called an application or app) loaded into the RAM 902. This program can also be distributed via a communication line or by recording it on a recording medium 917 such as a CD-ROM. ROM903 stores boot programs executed by CPU901 when the computer 900 starts up, as well as programs that depend on the computer 900's hardware.

[0412] The CPU 901 controls the input / output device 916, which consists of an input unit such as a mouse or keyboard, and an output unit such as a display or printer, via the input / output interface 906. The CPU 901 acquires data from the input / output device 916 via the input / output interface 906 and outputs generated data to the input / output device 916. In addition to the CPU 901, a GPU (Graphics Processing Unit) or the like may also be used as a processor.

[0413] HDD904 stores programs executed by CPU901 and data used by those programs. Communication I / F908 receives data from other devices via a communication network (e.g., NW (Network)) and outputs it to CPU901, and also transmits data generated by CPU901 to other devices via the communication network.

[0414] The media interface 907 reads a program or data stored in the recording medium 917 and outputs it to the CPU 901 via the RAM 902. The CPU 901 loads the program related to the desired processing from the recording medium 917 onto the RAM 902 via the media interface 907 and executes the loaded program. The recording medium 917 can be an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto Optical Disk), a magnetic recording medium, a conductive memory tape medium, or a semiconductor memory.

[0415] For example, if computer 900 functions as a division-normalized similarity calculation unit 100 configured as one of the first to third embodiments, the CPU 901 of computer 900 performs its function by executing a program loaded onto RAM 902. The HDD 904 stores the data from RAM 902. CPU 901 reads and executes a program related to the desired processing from the recording medium 917. Alternatively, CPU 901 may read a program related to the desired processing from another device via a communication network.

[0416] Furthermore, the first to third embodiments described above are explained in detail for the purpose of clearly illustrating the present invention, and are not necessarily limited to those having all the configurations described. It is also possible to replace a part of the configuration of one first embodiment with the configuration of another first embodiment, and to add the configuration of another first embodiment to a configuration of one first embodiment. Moreover, the first embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These first embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents.

[0417] Furthermore, among the processes described in the first to third embodiments above, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be changed at will unless otherwise specified. Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0418] Furthermore, each of the above configurations, functions, processing units, and processing means may be implemented in hardware, either partially or entirely, by designing them as integrated circuits, for example. Alternatively, each of the above configurations and functions may be implemented in software that allows the processor to interpret and execute programs that implement each function. Information such as programs, tables, and files that implement each function can be stored in memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC (Integrated Circuit) card, an SD (Secure Digital) card, or an optical disc.

[0419] Furthermore, in the first to third embodiments described above, the apparatus was referred to as a division-normalized similarity calculation unit and a spread learning network, but this is for the sake of explanation, and the name may also be similarity calculation unit, similarity calculation unit circuit device, perceptron, spread information network, etc. Also, the method and program were referred to as a division-normalized similarity determination method, but they may also be similarity calculation method, neural network program, etc. [Explanation of symbols]

[0420] 100, 101~106 Division-Normalized Similarity Calculation Unit (Similarity Calculation Unit) 1000 Diffuse Learning Networks

Claims

1. A similarity determination method that calculates the degree of similarity between the input of the learning phase and the input of the similarity determination phase using a perceptron modeled after nerve cells, Accepts one or more input values, Each input value is either the value L or the value H. The i-th input value of the learning phase is x ｉ Expressed as, The i-th input value of the similarity determination phase is y ｉ When expressed as, The i-th input value is w ｉ It has been assigned, value w ｉ Either value L or value H is set to this value. The weight value w assigned to the i-th input value in the aforementioned learning phase. ｉ to x ｉ Set the value to, In the similarity determination phase, w ｉ and y ｉ The number of inputs where both are the value H is divided by the number of inputs where the value of w ｉ is the value H plus the number of inputs where the value of y ｉ is the value H, and the result is calculated as the similarity representing the degree of similarity A method for determining similarity, characterized by the features described above.

2. A similarity determination method that calculates the degree of similarity between the input of the learning phase and the input of the similarity determination phase using a perceptron modeled after nerve cells, In the aforementioned similarity determination phase, when calculating division-normalized similarity using an equation that incorporates the operation caused by a phenomenon called the shunt effect of nerve cells into a perceptron model, As a result of the learning phase, the input value x ｊ The synaptic weights formed in the first perceptron by this are w ｊ The synaptic weights are determined so that, input value y ｊ In the similarity determination phase when the input arrives, the input to the first perceptron is Σ Ｎ j=1 y ｊ ・w ｊ It is calculated as follows: To the second perceptron, where there is no change in synaptic weights, Σ Ｎ j=1 y ｊ The following input is received: the output of the second perceptron generates a shunt effect in the first perceptron through the synapse formed between it and the first perceptron, and the division-normalized similarity is calculated using the following formula. [Math 1] However, x = (x ｊ ) Ｔ And ||x|| represents the norm of the vector x. y = (y ｊ ) Ｔ Therefore, w = (w ｊ ) Ｔ That is The above formula includes the squaring of the norm and the dot product of two vectors as vector operations. A method for determining similarity, characterized by the features described above.

3. Let the input value L be 0 and the value H be 1. In the similarity determination phase, x ｉ The number of inputs for which the value H is, for all input values, x ｉ Calculate as the sum of, lol ｉ and y ｉ The number of inputs where both are value H, for all input values ​​w ｉ and y ｉ The sum of the products of, or w ｉ and y ｉ It is calculated as the sum of the logical ANDs of, y ｉ The number of inputs for which value H is given by y for all i. ｉ Calculate as the sum of The similarity determination method according to claim 1 or 2.

4. The calculated similarity is used as input to the activation function that defines the behavior of the perceptron and the neuron, and the resulting value calculated by the activation function is output as a value representing the degree of similarity. The similarity determination method according to claim 1 or 2.

5. Let the input value L be 0 and the value H be 1. The value X indicates that the i-th input is connected to the j-th similarity calculation unit among multiple similarity calculation units that perform similarity calculation processing. ｉｊ It is represented as such, and the value X when connected. ｉｊ Set to 1, and if not connected, the value X ｉｊ Set X to 0. １ｊ , X ２ｊ A vector whose components are X ｊ , input x of the learning phase １ , x ２ Let x be a vector whose components are ..., and y be the input to the similarity determination phase. １ , y ２ When a vector whose components are ... is expressed in terms of y, lol １ , lol ２ Let w be a vector with components ... set as w = x, When the j-th similarity calculation unit calculates the similarity using the similarity determination method described in claim 1, x ｉ The number of inputs for which this is 1 is given by vector x and X. ｊ The Hadamard product of x○X ｊ It is calculated as the square of the norm, lol ｉ and y ｉ The number of inputs where both values ​​are 1 is measured by the Hadamard product w○X ｊ The dot product of the vector represented by and vector y is calculated, and y ｉ The number of inputs for which is 1 is the Hadamard product y○X ｊ It is calculated as the square of the norm. The similarity determination method according to feature 1.

6. The calculated similarity is then modified by adding a predetermined amount of noise to obtain a new similarity, and the final similarity calculation is performed using this noise-added similarity. The similarity determination method according to claim 1 or 2.

7. The aforementioned noise is a randomly generated number. The similarity determination method according to feature 6.

8. The input value is replaced with a value that can take any real number between 0 and 1 using fuzzy logic. The similarity determination method according to claim 1 or 2.

9. In the substitution of input values ​​using the aforementioned Fuzzy logic, In the similarity determination phase, lol ｉ The sum of the values, w ｉ and y ｉ The sum of the smaller values ​​of y ｉ Calculate the three values ​​that are the sum of the values ​​of, lol ｉ and y ｉ The value representing the sum of the smaller values ​​is w ｉ The value representing the sum of the values ​​and y ｉ The value obtained by dividing by the sum of the values ​​is calculated and output as the similarity score, which represents the degree of similarity. The similarity determination method according to feature 8.

10. A similarity calculation unit characterized by performing similarity calculation based on the similarity determination method described in claim 1 or claim 2.

11. A spreading learning network in which multiple similarity calculation units are connected to a set of inputs, each having some or all of the inputs, and the output of each of the similarity calculation units is input to a perceptron, The similarity calculation unit is, Accepts one or more input values, Each input value is either the value L or the value H. x is the value of the i-th input in the learning phase. ｉ Expressed as, The value of the i-th input in the similarity determination phase is y ｉ When expressed as, The i-th input is value w ｉ It has been assigned, value w ｉ Either value L or value H is set to this value. The weight value w assigned to the i-th input in the aforementioned learning phase. ｉ to x ｉ Set the value to, In the similarity determination phase, lol ｉ and y ｉ The number of inputs where both values ​​are H is given by w ｉ The input number and y whose value is value H ｉ The similarity calculation is performed by dividing the value of by the number of inputs whose value is H, and then calculating the similarity score, which represents the degree of similarity. A diffusive learning network characterized by the following:

12. Multiple similarity calculation units are combined, with one or more inputs from the entire input being used as input to each of the similarity calculation units. Each of the similarity calculation units calculates the similarity, and the sum of the similarities calculated by all the similarity calculation units is output as the final similarity. The diffusive learning network according to feature 11.

13. Instead of using the sum of the similarity scores calculated by all the aforementioned similarity calculation units as the similarity score, the similarity score may be calculated as the sum of the aforementioned similarity scores divided by the total number of aforementioned similarity calculation units, or as the sum of the values ​​obtained by dividing each of the similarity scores calculated by all the aforementioned similarity calculation units by the total number of aforementioned similarity calculation units. The diffusive learning network according to feature 12.

14. For multiple inputs, a computer acting as a similarity calculation unit, which has some or all of those inputs, A procedure for accepting one or more input values. Each input value is either the value L or the value H. x is the i-th input value in the learning phase. ｉ Expressed as, The i-th input value in the similarity determination phase is y ｉ When expressed as, The i-th input value is w ｉ It has been assigned, lol ｉ Either value L or value H is set to this value. The weight value w assigned to the i-th input in the aforementioned learning phase. ｉ to x ｉ The procedure for setting the value, In the similarity determination phase, lol ｉ and y ｉ The number of inputs where both values ​​are H is given by w ｉ The number of inputs and y whose value is H ｉ The procedure for calculating the similarity score, which represents the degree of similarity, is to divide the input number, where H is the value of the input, by the number of inputs. A neural network execution program to perform the following actions.

Images