Pattern recognition device, learning method, and program

Abstract

To provide a technique which contributes to shortening of learning time of a neural network.SOLUTION: A learning data acquisition unit that acquires a set of input data including a plurality of patterns and teacher data, a noise generation unit that generates fluctuation noise which is a value based on a predetermined distribution, an error function value calculation unit that calculates a value of an error function indicating a deviation between final output data which is output data of an output layer based on the fluctuation noise and the input data and the teacher data, and a learning unit that trains a learning model for pattern recognition by optimizing the error function, wherein the learning unit performs relearning of a part of the learning model according to a magnitude of a variance of output data based on the fluctuation noise and the input data after first learning optimized such that a value of the error function is equal to or less than a first threshold value is completed.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a pattern recognition device, a learning method, and a program. [Background technology]

[0002] Patent Document 1 discloses a fluctuation-driven learning method. The fluctuation-driven learning method is a technique in which learning is carried out by adding fluctuation noise with a predetermined distribution to the activation values ​​of an activation function. This learning method makes it possible to learn a model that mixes differentiable continuous functions and discontinuous functions such as threshold functions. Furthermore, because it does not require differentiation or backpropagation processing, problems such as learning stalling due to loss of gradient do not occur. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 8-235146 Summary of the Invention [Problem to be solved by the invention]

[0004] The following analysis was performed by the inventors of the present disclosure.

[0005] However, in the fluctuation-driven learning method of Patent Document 1, learning is performed using normal random numbers generated from a fixed variance as the fluctuation noise given to the activity value of each neuron. During this time, the fluctuation noise is updated every time the learning loop is repeated. This makes learning take a long time.

[0006] The present disclosure has been made in consideration of the above circumstances, and one of its objects is to provide a technology that contributes to shortening the learning time of a neural network. [Means for solving the problem]

[0007] According to a first aspect of the present disclosure, a learning data acquisition unit that acquires a set of input data including a plurality of patterns and training data; a noise generator that generates fluctuation noise, which is a value based on a predetermined distribution; an error function value calculation unit that calculates a value of an error function that indicates a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; a learning unit that learns a learning model for pattern recognition by optimizing the error function, A pattern recognition device is provided in which, after completing a first learning process that optimizes the value of the error function to be equal to or less than a first threshold, the learning unit re-learns a portion of the learning model based on the fluctuation noise and the magnitude of variance of the output data that is based on the input data.

[0008] According to a second aspect of the present disclosure, A learning method in which a computer of a pattern recognition device learns a learning model for pattern recognition, comprising: A set of input data including a plurality of patterns and training data is obtained, generating fluctuation noise, which is a value based on a predetermined distribution; obtaining a value of an error function indicating a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; performing a first learning of the learning model by optimizing the error function so that the value of the error function is equal to or less than a first threshold; There is provided a learning method in which, after the first learning is completed, re-learning of a part of the learning model is performed based on the fluctuation noise and the magnitude of variance of output data based on the input data.

[0009] According to a third aspect of the present disclosure, A computer of a pattern recognition device that learns a learning model for pattern recognition, A procedure for acquiring a set of input data including a plurality of patterns and training data; A procedure for generating fluctuation noise, which is a value based on a predetermined distribution; a step of acquiring a value of an error function indicating a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the training data; performing a first learning of the learning model by optimizing the error function so that the value of the error function is equal to or less than a first threshold; After the first learning is completed, a program is provided that executes a procedure for relearning a portion of the learning model based on the fluctuation noise and the magnitude of variance of the output data based on the input data.

[0010] These programs can be recorded on a computer-readable storage medium. The storage medium can be a non-transitory medium such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. The present disclosure can also be embodied as a computer program product. [Effects of the Invention]

[0011] According to the present disclosure, the learning time of a neural network can be reduced. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1A is a diagram illustrating an example of a neural network according to the present disclosure, and FIG. 1B is a diagram illustrating an example of a neuron according to the present disclosure. [Figure 2] 1A is a functional block diagram of an example of a pattern recognition device according to the present disclosure, and FIG. 1B is a flowchart of an example of a learning process according to the present disclosure. [Figure 3] FIG. 1 is a functional block diagram of an example of a pattern recognition device according to the present disclosure. [Figure 4] FIG. 10 is a diagram for explaining an overview of an update coupling coefficient determination process according to the present disclosure. [Figure 5] 10 is a flowchart illustrating an example of an initial learning process of the present disclosure. [Figure 6]10 is a flowchart illustrating an example of a re-learning process of the present disclosure. [Figure 7] 10 is a flowchart illustrating an example of an update coupling coefficient determination and update process of the present disclosure. [Figure 8] FIG. 1 is a configuration diagram illustrating an example of a hardware configuration of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that reference numerals in the drawings are assigned to each element for convenience as an example to facilitate understanding, and are not intended to limit the present disclosure to the illustrated aspects. Furthermore, connecting lines between blocks in the drawings and the like referred to in the following description include both bidirectional and unidirectional lines. Unidirectional arrows are used to schematically indicate the flow of the main signal (data) and do not exclude bidirectionality. Furthermore, in the following description, "A and / or B" means A or B, or A and B.

[0014] <<First Embodiment>> The pattern recognition device of this embodiment includes a neural network that is trained by inputting input data including a plurality of patterns multiple times.

[0015] A pattern recognition device trains a neural network using input data containing multiple patterns. Specifically, in a problem of learning a group of patterns, the pattern recognition device trains the neural network using a fluctuation-driven learning method so that the output value approximates a teacher signal that has one-to-one correspondence with the patterns. The neural network is, for example, an acyclic hierarchical neural network (an acyclic neural network; hereinafter simply referred to as a neural network).

[0016] First, the neural network included in the pattern recognition device of this embodiment will be described, followed by a description of each component of the pattern recognition device and a description of the learning process performed by the pattern recognition device.

[0017] [Neural Networks] FIG. 1(a) is a configuration diagram illustrating a neural network 500 of a pattern recognition device according to this embodiment. As shown in this figure, the neural network 500 includes a plurality of neurons 510. Note that some reference numerals have been omitted to avoid cluttering the figure. This also applies to the following figures.

[0018] The multiple neurons 510 are divided into layers. The layers of the neural network 500 include an input layer 520, a hidden layer 530, and an output layer 540. In this figure, as an example, the input layer 520 includes four neurons 510, the hidden layer 530 includes six neurons 510, and the output layer 540 includes two neurons 510. Note that the number of neurons 510 included in each layer is not limited to these, and the number may be selected appropriately. Furthermore, the number of hidden layers 530 is not limited to one, and multiple hidden layers 530 may be included.

[0019] 1(b) is a diagram illustrating the input and output of one neuron 510 in the neural network 500 according to this embodiment. As shown in this diagram, the neuron 510 receives input data from multiple neurons 510 in the input layer. The neuron 510 then outputs a neuron output value r(p, t) as shown in the following equation (1). Here, p represents a learning pattern number, and t represents time.

number

[0020] In the above equation (1), f(v(p,t)) on the right side is a sigmoid function shown in equation (2).

number

number

number

[0021] The input data of pattern p is represented as q(p, t), where t indicates time.

[0022] For the error function e(p, t) indicating the deviation between the neuron output value r output from each neuron 510 in the output layer 540 and the training data t, for example, a square error as shown in equation (5) is used.

number

[0023] If the fluctuation noise n(t) is a normal random number, the update formula for the coupling coefficient w to reduce the value (hereinafter also referred to as the error e) obtained by the error function e(p,t) shown in equation (5) is shown in the following equations (6) and (7).

number

number

[0024] In the neural network 500, the coupling coefficient w is sequentially updated according to the above equations (6) and (7) to reduce the error e in equation (5). That is, when input data containing a pattern is input, the neuron output value r from each neuron 510 in the output layer of the neural network 500 can be made to approach the training data t, and the pair of input data q and training data t can be learned.

[0025] In this case, in the fluctuation-driven learning method of Patent Document 1 (hereinafter referred to as the conventional method), the fluctuation noise n k As (t), normal random numbers with the probability density function shown in Equation (8) are used. In the conventional method, the variance σ k 2 is a fixed value.

number

[0026] In the conventional method, all the coupling coefficients w are updated every time the learning loop is repeated during learning, so it takes a long time to converge.

[0027] Here, the learning loop refers to the process of generating fluctuation noise and providing it to each neuron 510, inputting input data to neurons 510 in input layer 520, and inputting the resulting output data, which is neuron output value r from output layer 540, together with training data, into an error function to obtain an error function value. In learning of neural network 500, the connection coefficient w is updated each time the learning loop is repeated, and the error function value is minimized.

[0028] In this embodiment, during training of the neural network 500, the period Tp of pattern p is used to calculate the degree of influence on the output data of each neuron 510 in the group of neurons 510 in the output layer from the neuron 510 of interest. Then, based on this influence, the connection coefficient w to be updated is selected. This shortens the training time compared to updating all the connection coefficients w.

[0029] [Pattern Recognition Device] The pattern recognition device of this embodiment that realizes this will be described below. Fig. 2(a) is a functional block diagram of the pattern recognition device 100 of this embodiment.

[0030] As shown in the figure, the pattern recognition device 100 of this embodiment includes a training data acquisition unit 110, a noise generation unit 120, an error function value calculation unit 130, and a training unit 140.

[0031] The learning data acquisition unit 110 acquires a set of input data including a plurality of patterns and teacher data as learning data. The learning data acquisition unit 110 acquires the learning data from a storage area where the learning data is accumulated, and sends the data to the learning unit 140. The learning data may be acquired, for example, via a network.

[0032] The pattern recognition device 100 of this embodiment uses, for example, an acyclic hierarchical (feedforward) neural network model (neural network 500) as a learning model. Input data and corresponding teacher data are presented to the learning unit 140, and learning is performed. The input data is an input vector input to neurons 510 in an input layer 520 of the neurons 510 of the neural network 500 that constitute the pattern recognition device 100.

[0033] In this embodiment, the learning data acquisition unit 110 acquires q(p, t) as input data for pattern p and acquires t(p) as training data. Then, these pairs are sent to the learning unit 140.

[0034] The noise generator 120 generates fluctuation noise, which is a value based on a predetermined distribution. The fluctuation noise is used in a known fluctuation-driven learning method, and is a random value added to the activation value of each neuron 510 in the neural network 500 constituting the pattern recognition device 100 of this embodiment. The predetermined distribution may be, for example, random values ​​based on a normal distribution.

[0035] The error function value calculation unit 130 calculates an error function value that indicates the deviation between the output data based on the fluctuation noise and the input data and the teacher data.

[0036] The input data is input data from the training data acquired by the training data acquisition unit 110. The output data is an output vector output from the output layer 540 of the neuron 510. This output data is also called, for example, final output data to distinguish it from the output data from the neuron 510 in the intermediate layer 530.

[0037] The error function is a function that outputs a value indicating the deviation between the training data and the final output data as an error function value. For example, as shown in equation (5), the error function value may be the sum of squares of the difference between the training data vector and the output vector.

[0038] The learning unit 140 learns a learning model for pattern recognition by optimizing the error function value. Here, optimization means modifying parameters in a direction that minimizes the error function value. Specifically, the learning unit 140 updates the coupling coefficient w between each neuron 510 based on a predetermined update formula and repeatedly executes a learning loop to minimize the error function value.

[0039] The learning unit 140 of this embodiment performs learning in two stages. That is, a first threshold value α and a second threshold value β (α>β) smaller than the first threshold value α are prepared. The learning unit 140 then performs initial learning (first learning) to optimize the error function value so that it is smaller than the first threshold value α. After the initial learning (first learning) is completed, the learning unit 140 re-learns part of the learning model based on the magnitude of variance of the output data based on the fluctuation noise and the input data. The re-learning ends when the error function value becomes smaller than the second threshold value β, as it has been minimized, i.e., converged.

[0040] The second threshold value β may be, for example, the same as the threshold value used to determine convergence in the conventional method.

[0041] In re-learning, the learning unit 140 defines the magnitude of the variance as the degree of influence, and updates only the coupling coefficient w whose magnitude of variance satisfies a predetermined condition, aiming for convergence.

[0042] [Learning process] Here, a description will be given of the flow of the learning process by the pattern recognition device 100 of this embodiment. Fig. 2(b) shows the processing flow of the learning process by the pattern recognition device 100 of this embodiment. This process is started, for example, in response to a start instruction from the user.

[0043] First, the learning data acquiring unit 110 acquires learning data (step S1101). In this embodiment, as described above, a set of input data including a plurality of patterns and training data is acquired as learning data.

[0044] The learning unit 140 performs initial learning (first learning) using the learning data (step S1102). Here, for example, learning data of the first pattern, pattern 1, is provided to the neural network 500, and the activation values ​​v(p, t) of all neurons 510 are calculated according to the above formula (3). At this time, n(t) is fluctuation noise input to all neurons 510 during the period Tp, and is generated, for example, by normal random numbers having a probability density function represented by the following formula (9):

number

[0045] Then, the learning unit 140 repeats the learning loop while updating the coupling coefficient w until the error function value becomes smaller than a predetermined first threshold α. The learning loop involves calculating the final output data using the above equations and calculating the error function value using the above equation (5).

[0046] Thereafter, the learning unit 140 re-learns a part of the learning model (step S1103) and ends the process. In the re-learning, as described above, from the state where the initial learning is completed, learning is repeated while updating only the coupling coefficient w whose influence degree satisfies a predetermined condition until the error function value becomes smaller than the predetermined second threshold value β. Note that for each repetition, a coupling coefficient w to be updated is selected.

[0047] As described above, the pattern recognition device 100 of this embodiment performs learning in two stages. First, for each iteration of the learning loop, the coupling coefficients w between all neurons 510 are updated, and model learning is performed to converge to a certain extent. After that, only the coupling coefficients w between neurons 510 with the greatest influence are updated, aiming for highly accurate convergence.

[0048] According to the conventional method, the coupling coefficients w between all neurons 510 are updated until learning is complete. In contrast, in the pattern recognition device 100 of this embodiment, the number of times the coupling coefficients w are updated can be significantly reduced, thereby shortening the calculation time.

[0049] According to the pattern recognition device 100 of this embodiment, the learning time of the neural network 500 can be reduced.

[0050] <<Second embodiment>> Next, a second embodiment of the present disclosure will be described. This embodiment is an embodiment that embodies the first embodiment in more detail. In this embodiment, components with the same names as those in the first embodiment basically have the same functions as those in the first embodiment. Hereinafter, this embodiment will be described, focusing on the differences from the first embodiment.

[0051] The pattern recognition device 100 of this embodiment basically has the same configuration as that of the first embodiment. As shown in FIG.

[0052] The learning data acquisition unit 110, the noise generation unit 120, and the error function value calculation unit 130 are the same as those in the first embodiment, and therefore a description thereof will be omitted here.

[0053] The learning unit 140 learns a learning model for pattern recognition by optimizing the error function value. Furthermore, in the learning unit 140 of this embodiment, when re-learning, the determining unit 150 determines an updated coupling coefficient, which is the coupling coefficient w to be updated. Then, by updating only the updated coupling coefficient, a part of the learning model is re-learned.

[0054] The determination unit 150 determines an updated coupling coefficient during relearning. The determination unit 150 applies fluctuation noise only to a selected neuron 510 (first neuron) of the neural network 500. Then, among the group of neurons 510 in the output layer of the first neuron, the neuron 510 with the largest variance of output data is selected as the second neuron, and the coupling coefficient w between the first neuron and the second neuron is determined to be the updated coupling coefficient.

[0055] That is, in the learning loop, before updating the coupling coefficients, the determining unit 150 uses the magnitude of the variance of the output data as the degree of influence of each neuron 510 (first neuron) of the neural network 500 on the group of neurons 510 in the output layer.

[0056] Specifically, before updating the coupling coefficient, that is, in the state at the end of the immediately preceding learning loop, the determination unit 150 selects one neuron 510 from the neural network 500 as a first neuron. Then, the determination unit 150 causes the noise generation unit 120 to generate fluctuation noise n(t) of period Tp to be input to the selected first neuron. Here, the noise generation unit 120 generates the fluctuation noise n(t), for example, using the above equation (9).

[0057] The determination unit 150 inputs the input data acquired by the training data acquisition unit 110 to the neural network 500. At this time, the calculated fluctuation noise n(t) is input only to the selected first neuron, and the variance of the output values ​​of each neuron 510 in the group of neurons 510 in the output layer is calculated.

[0058] Here, the variance is calculated, for example, by the following equation (10).

number

[0059] Equation (10) is the variance of the output of each neuron 510 in the group of neurons 510 in the output layer when the fluctuation noise n(t) is applied only to the first neuron 510, which is the selected neuron, while all values ​​other than the fluctuation noise n(t), i.e., the activity value v, membrane potential u, coupling coefficient w, and input data, are fixed.

[0060] For example, suppose that the hidden layer 530 of the neural network 500 includes a k-layer and a j-layer. The j-layer is the output layer of the k-layer. An image of this case is shown in FIG. 4.

[0061] In FIG. 4, the first neuron is the k-th neuron k in the k-th layer. The group of neurons 510 in the output layer of neuron k is neurons 1 to J. In this figure, σ j 2 Let us denote this as σ_j.

[0062] σ_1 is the variance of the output value of the first neuron 1 in the jth layer when a fluctuation noise n(t) is applied to neuron k. σ_j is the variance of the output value of the jth neuron j in the jth layer. σ_J is the variance of the output value of the jth neuron J in the jth layer.

[0063] The determination unit 150 defines the magnitude of this variance as the degree of influence of the first neuron on each of the groups of neurons 510 in the output layer. Then, the determination unit 150 determines the neuron 510 (neuron j) in the group of neurons 510 in the output layer that has the largest magnitude of variance of its output value as the second neuron, as shown in the following equation (11), and determines the coupling coefficient w between the first neuron and the second neuron as the updated coupling coefficient.

number

[0064] The determination unit 150 performs the above process for all neurons 510 (excluding neurons 510 in the output layer) that make up the neural network 500, and determines and updates the updated coupling coefficients. Thus, according to this embodiment, for each neuron 510, only one coupling coefficient w is determined as the updated coupling coefficient and updated.

[0065] [Learning process] The flow of the learning process of this embodiment will be described. As in the first embodiment, the learning process of this embodiment consists of two stages: an initial learning process and a re-learning process of a part of the learning model.

[0066] [Initial learning process] First, the flow of the initial learning process will be described. The initial learning process basically performs the same processing as the learning loop of the conventional method. Fig. 5 shows the processing flow of the initial learning process of this embodiment.

[0067] First, the learning unit 140 calculates the variance d (=σ k 2 ) is initialized to a fixed value (step S2101).

[0068] Next, the learning unit 140 initializes the coupling coefficient w (step S2102).

[0069] Next, the learning unit 140 causes the noise generating unit 120 to generate fluctuation noise n(t) during the period Tp (step S2103). The fluctuation noise n(t) generated here is input to all neurons 510 that make up the neural network 500.

[0070] The learning unit 140 calculates the neuron output value r according to, for example, the above formulas (1) to (4) (step S2104). This neuron output value r is the final output data, which is the output value from each neuron 510 in the output layer 540.

[0071] The learning unit 140 causes the error function value calculation unit 130 to calculate the error e between the neuron output value r and the training data t (step S2105). The error function value calculation unit 130 calculates the error function value as the error e according to, for example, the error function of the above equation (5).

[0072] The learning unit 140 determines whether the obtained error e (error function value) is smaller than a first threshold value α given in advance for initial learning (step S2106). If it is smaller, the initial learning process ends.

[0073] On the other hand, if the error e is equal to or greater than the first threshold α, the learning unit 140 updates the coupling coefficient w according to equations (6) and (7) (step S2107), returns to step S2103, and repeats the process.

[0074] [Relearning process] Next, the flow of the relearning process of this embodiment will be described. Fig. 6 shows the processing flow of the relearning process of this embodiment. This process is started when the initial learning is completed.

[0075] First, the learning unit 140 causes the determination unit 150 to execute an update coupling coefficient determination and update process for determining and updating an update coupling coefficient (step S2201). The flow of the update coupling coefficient determination and update process will be described later.

[0076] The learning unit 140 calculates the neuron output value r according to, for example, the above formulas (1) to (4) (step S2202). This neuron output value r is the final output data, which is the output value from each neuron 510 in the output layer 540.

[0077] The learning unit 140 causes the error function value calculation unit 130 to calculate the error e between the neuron output value r and the training data t (step S2203). The error function value calculation unit 130 calculates the error function value as the error e according to, for example, the error function of the above equation (5).

[0078] The learning unit 140 determines whether the obtained error e (error function value) is smaller than a second threshold value β given in advance for re-learning (step S2204). If it is smaller, the re-learning process ends.

[0079] On the other hand, if the error e is equal to or greater than the second threshold value β, the learning unit 140 returns to step S2201 and repeats the process.

[0080] [Update coupling coefficient determination and update process] Next, a description will be given of the flow of the updated coupling coefficient determination and update process by the determination unit 150 in step S2201 above. Fig. 7 shows the processing flow of the updated coupling coefficient determination and update process of this embodiment.

[0081] First, the determining unit 150 selects one neuron 510 from among the unselected neurons 510 that constitute the neural network 500 (step S2301). Hereinafter, the selected neuron 510 will be referred to as the first neuron.

[0082] The determining unit 150 causes the noise generating unit 120 to generate fluctuation noise n(t) during the period Tp (step S2302). The fluctuation noise n(t) generated here is input to the first neuron.

[0083] The determining unit 150 inputs the input data to the neural network 500 (step S2303). The input data is data acquired by the learning data acquiring unit 110.

[0084] The determination unit 150 obtains the neuron output value r according to the above formulas (1) to (4) and calculates its variance (step S2304). This neuron output value r is the output value of each neuron 510 of the group of neurons 510 in the output layer of the first neuron. For example, in the example of FIG. 4, if the first neuron is a neuron 510 in the kth layer, the output layer is the jth layer, and the group of neurons 510 is neurons 1 to J in the jth layer. Note that the determination unit 150 calculates the variance of the neuron output value r for each neuron 510 in the group of neurons in the output layer according to formula (10).

[0085] Then, the determination unit 150 determines an updated coupling coefficient based on the calculated variance (step S2305). In this embodiment, the neuron 510 with the largest variance of the neuron output value r among the neurons 510 in the output layer is determined as the second neuron, and the coupling coefficient w between the first neuron and the second neuron is determined as the updated coupling coefficient.

[0086] The determining unit 150 updates the determined updated coupling coefficients (step S2306). The updating is performed according to the above equations (6) and (7).

[0087] The determination unit 150 determines whether all neurons 510 other than the neuron 510 in the output layer have been selected and have been updated as the first neuron (step S2307), and if so, terminates the process.

[0088] As described above, this embodiment has the same configuration as the first embodiment, and therefore has the same effects as the first embodiment.

[0089] Furthermore, according to this embodiment, during relearning, for each neuron 510 constituting the neural network 500, only the coupling coefficient w of that neuron 510 that has the greatest influence on the neuron 510 in the output layer is updated. This further reduces the number of updates to the coupling coefficient w, thereby further shortening the calculation time.

[0090] That is, according to the pattern recognition device 100 of this embodiment, the learning time of the neural network 500 can be reduced.

[0091] <Variation 1> In the second embodiment, the determination unit 150 determines only the coupling coefficient w between the first neuron and the neuron 510 in the output layer whose neuron output value r has the largest variance calculated by equation (10) as the updated coupling coefficient, but this is not limited to this.

[0092] For example, a predetermined number m (m is an integer equal to or greater than 1) of neurons 510 with the largest variances, i.e., the m neurons with the highest variances, may be selected from the group of neurons 510 in the output layer as the second neurons. In this case, too, only the coupling coefficient w between the first neuron and each second neuron is used as the updated coupling coefficient.

[0093] Alternatively, neurons 510 whose variance is equal to or greater than a predetermined threshold may be selected from the group of neurons 510 in the output layer as second neurons. In this case, only the coupling coefficients w between the first neurons and each second neuron are used as updated coupling coefficients.

[0094] Furthermore, any of the update combining coefficient determination methods may be selectable. In this case, the pattern recognition device 100 is capable of executing a plurality of update combining coefficient determination methods. The device may also be provided with a user interface that allows the user to select which update combining coefficient determination method to use.

[0095] <Variation 2> Furthermore, in the second embodiment, the determination section 150 updates the updated combination coefficient immediately after determining the updated combination coefficient, but the present invention is not limited to this.

[0096] For example, the determination unit 150 may only determine the updated coupling coefficients without performing the process of step S2306. That is, the process executed in S2201 of the re-learning process may be the updated coupling coefficient determination process. Then, the learning unit 140 may perform the process of updating the updated coupling coefficients after the updated coupling coefficient determination process of S2201 during the re-learning process.

[0097] <Variation 3> In the above embodiments, the influence of a neuron 510 of interest on a group of neurons 510 in the output layer is measured by the magnitude of the variance of the output data. However, this is not limited to the magnitude of the variance. Various statistical values ​​that indicate the degree of fluctuation in the output data can be used as the influence.

[0098] <Variation 4> In each of the above embodiments, the determiner 150 determines the updated coupling coefficients each time the re-learning learning loop is repeated, but this is not limiting. The determined updated coupling coefficients may be maintained while the learning loop is repeated a predetermined number of times (multiple times).

[0099] Furthermore, the determined updated coupling coefficients may not be changed until the re-learning is completed. In this case, only the determined updated coupling coefficients are updated, and the re-learning learning loop is repeated.

[0100] <Variation 5> In the above embodiment, the activation function is a sigmoid function as shown in Equation (2), but the activation function is not limited to this. The sigmoid function is a differentiable continuous function, but a tanh function (Hyperbolic tangent function) or a discontinuous ReLU may be used, or a mixture of these may be used.

[0101] [Hardware configuration] The pattern recognition device 100 of each of the above embodiments may be realized by, for example, a general-purpose information processing device. The general-purpose information processing device includes, for example, a CPU (Central Processing Unit) 191, a main storage device (memory) 192, an auxiliary storage device 193, a communication I / F 194, and an expansion I / F 195, which are interconnected by an internal bus, as shown in FIG.

[0102] The CPU 191 realizes the above functions and controls the entire device by, for example, loading a program stored in the auxiliary storage device 193 into the main storage device 192 and executing it. Note that the CPU 191 may be replaced by one or more processors such as an MPU (Micro Processing Unit).

[0103] The main storage device 192 is a memory such as a RAM (Random Access Memory), etc. The main storage device 192 is a work area when the CPU 191 processes programs executed by the installed devices.

[0104] The auxiliary storage device 193 is, for example, a read-only memory (ROM), a hard disk drive (HDD), or a solid state drive (SSD). The auxiliary storage device 193 stores various programs executed by the device on which it is installed. In addition, in each of the above embodiments and / or modifications, the storage area of ​​the pattern recognition device 100 may be configured in the auxiliary storage device 193.

[0105] For example, the learning data may be stored in the auxiliary storage device 193. Various data used in the processing, including, for example, the first threshold value α, the second threshold value β, etc., may also be stored in the auxiliary storage device 193. Various data generated during the processing is stored in either the auxiliary storage device 193 or the main storage device 192.

[0106] The auxiliary storage device 193 may include a storage medium such as a flexible disk, a hard disk, an optical disk, a CD-ROM, a CD-R, a magnetic tape, a non-volatile memory card, or a DVD.

[0107] The above-mentioned functions of the pattern recognition device 100 are realized by the CPU 191 loading a program stored in the auxiliary storage device 193 into the main storage device 192 and executing it.

[0108] The programs for realizing the above functions can be recorded on a computer-readable storage medium. The storage medium can be a non-transitory medium such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. The present disclosure can also be embodied as a computer program product. Furthermore, the programs stored in the auxiliary storage device 193 can be provided as a program product recorded on a non-transitory computer-readable recording medium. The auxiliary storage device 193 can be used to store various programs recorded on a non-transitory computer-readable recording medium for the medium to long term.

[0109] The communication I / F 194 is an interface for inputting and outputting signals and data via wired or wireless connections. The communication I / F 194 may also include an antenna, a modem, and the like. For example, when the training data acquisition unit 110 acquires training data from a device external to the pattern recognition device 100, the training data may be acquired from a network via the communication I / F 194.

[0110] The expansion I / F 195 includes, for example, a USB interface and a memory slot, and is connected to an external storage unit, an input device, an output device, and the like. The input device is, for example, a device that accepts user operations, such as a keyboard or a mouse. The output device is a display device such as a monitor. The liquid crystal monitor may have a touch panel function that accepts instructions by touch operation by the user. The expansion I / F 195 may have a plurality of different types of interfaces.

[0111] For example, the user issues an instruction to start processing via an input device or the like connected to the expansion I / F 195. The first threshold value α, the second threshold value β, and the like may also be set.

[0112] The hardware configuration of each device is not limited to this. Each function (server) of each device may be implemented using, for example, an integrated circuit (IC) dedicated to each process, an application specific integrated circuit (ASIC), a system on a chip (SOC), a field programmable gate array (FPGA), or the like.

[0113] In the process flow used in the above explanation, multiple steps (processes) are described in order, but the order in which each step is performed is not limited to the order described. For example, the order of the steps shown in the figure can be changed to the extent that the content is not affected, such as performing each process in parallel.

[0114] Although the embodiments and modifications of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments and can be modified in various ways that would be understandable to a person skilled in the art. Each embodiment and modification can be combined with other embodiments as appropriate. Furthermore, for example, the network configurations and element configurations shown in the drawings are examples intended to aid in understanding the present disclosure and are not limited to the configurations shown in these drawings.

[0115] Finally, preferred embodiments of the present disclosure will be summarized. Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes. (Appendix 1) The pattern recognition device a learning data acquisition unit that acquires a set of input data including a plurality of patterns and training data; a noise generator that generates fluctuation noise, which is a value based on a predetermined distribution; an error function value calculation unit that calculates a value of an error function that indicates a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; and a learning unit that performs learning of a learning model for pattern recognition by optimizing the error function. After completing the first learning in which the value of the error function is optimized to be less than or equal to a first threshold, the learning unit re-learns part of the learning model based on the fluctuation noise and the magnitude of variance of the output data based on the input data. (Appendix 2) 13. The pattern recognition device according to claim 1 or 12, It is desirable that the learning unit determine a re-learning portion, which is a part of the learning model to be re-learned, based on the magnitude of the variance. (Appendix 3) 12. The pattern recognition device according to claim 1 or 11, the learning model is a neural network; the learning unit includes a determination unit that determines an updated coupling coefficient, which is a coupling coefficient to be updated; the determination unit determines, as the updated coupling coefficient, a coupling coefficient between a second neuron and the first neuron, among a group of neurons in an output layer of the first neuron, in a case where the fluctuation noise is applied only to the first neuron of the neural network, and the magnitude of variance of the output data satisfies a predetermined condition; It is desirable that the learning unit re-learns part of the learning model by updating only the updated coupling coefficients determined by the determination unit. (Appendix 4) 4. The pattern recognition device according to claim 3, The predetermined condition is preferably that the magnitude of the variance of the output data is maximum. (Appendix 5) 4. The pattern recognition device according to claim 3, The predetermined condition is preferably a predetermined number of the magnitudes of variance of the output data in descending order of magnitude. (Appendix 6) 4. The pattern recognition device according to claim 3, The predetermined condition is preferably that the magnitude of the variance of the output data is equal to or greater than a predetermined threshold value. (Appendix 7) 7. The pattern recognition device according to claim 3, It is desirable that the learning unit causes the determination unit to determine the updated coupling coefficients each time a learning loop is executed in relearning. (Appendix 8) 7. The pattern recognition device according to claim 1, The second threshold used in the optimization is preferably smaller than the first threshold. (Appendix 9) A learning method in which a computer of a pattern recognition device learns a learning model for pattern recognition is as follows: A set of input data including a plurality of patterns and training data is obtained, generating fluctuation noise, which is a value based on a predetermined distribution; obtaining a value of an error function indicating a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; performing a first learning of the learning model by optimizing the error function so that the value of the error function is equal to or less than a first threshold; After the first learning is completed, a part of the learning model is re-learned depending on the magnitude of variance of the output data based on the fluctuation noise and the input data. (Appendix 10) The program is A computer of a pattern recognition device that learns a learning model for pattern recognition, A procedure for acquiring a set of input data including a plurality of patterns and training data; A procedure for generating fluctuation noise, which is a value based on a predetermined distribution; a step of acquiring a value of an error function indicating a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the training data; performing a first learning of the learning model by optimizing the error function so that the value of the error function is equal to or less than a first threshold; After the first learning is completed, a procedure of re-learning a part of the learning model is executed based on the fluctuation noise and the magnitude of variance of the output data based on the input data. (Appendix 11) The pattern recognition device a learning data acquisition unit that acquires a set of input data including a plurality of patterns and training data; a noise generator that generates fluctuation noise, which is a value based on a predetermined distribution; an error function value calculation unit that calculates a value of an error function that indicates a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; and a learning unit that performs learning of a learning model for pattern recognition by optimizing the error function. After completing the first learning that optimizes the value of the error function so that it is equal to or less than a first threshold, the learning unit re-learns part of the learning model depending on the influence of the fluctuation noise and the input data on the output data. (Appendix 12) In the pattern recognition device according to Supplementary Note 11, it is preferable that the degree of influence is a statistical value indicating the degree of fluctuation in the output data, including the magnitude of the variance of the output data. The forms of Supplements 9 and 10 can be developed into the forms of Supplements 2-8, as in Supplement 1. Furthermore, the form of Supplement 11 can be developed into the forms of Supplements 3-8.

[0116] The disclosures of the above-mentioned patent documents and other documents are incorporated herein by reference. Within the scope of this disclosure (including the claims), modifications and adjustments of the embodiments and variations are possible based on the basic technical concepts. Furthermore, within the scope of this disclosure, various combinations and selections of the various disclosed elements (including each element of each claim, each element of each embodiment or variation, each element of each drawing, etc.) are possible. In other words, this disclosure naturally includes various modifications and alterations that would be possible by a person skilled in the art in accordance with the entire disclosure, including the claims, and the technical concepts. In particular, with regard to the numerical ranges described herein, any numerical value or subrange included within the range should be construed as being specifically described, even if not otherwise specified. [Explanation of symbols]

[0117] 100: Pattern recognition device, 110: Learning data acquisition unit, 120: Noise generation unit, 130: Error function value calculation unit, 140: Learning unit, 150: Determination unit, 191: CPU, 192: Main storage device, 193: Auxiliary storage device, 194: Communication I / F, 195: Expansion I / F, 500: neural network, 510: neuron, 520: input layer, 530: intermediate layer, 540: output layer, Tp: pattern input duration, d: variance, e: error, n: noise, p: pattern, q: input data, r: neuron output value, t: training data, u: membrane potential, v: activity value, w: coupling coefficient, α: first threshold, β: second threshold, σ: variance

Claims

1. a learning data acquisition unit that acquires a set of input data including a plurality of patterns and training data; a noise generator that generates fluctuation noise, which is a value based on a predetermined distribution; an error function value calculation unit that calculates a value of an error function that indicates a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; a learning unit that learns a learning model for pattern recognition by optimizing the error function, A pattern recognition device, wherein after completing a first learning process that optimizes the value of the error function to be equal to or less than a first threshold, the learning unit re-learns a portion of the learning model based on the fluctuation noise and the magnitude of variance of output data based on the input data.

2. 2. The pattern recognition device according to claim 1, The pattern recognition device, wherein the learning unit determines a re-learning portion, which is a part of the learning model to be re-learned, based on the magnitude of the variance.

3. 2. The pattern recognition device according to claim 1, the learning model is a neural network; the learning unit includes a determination unit that determines an updated coupling coefficient, which is a coupling coefficient to be updated; the determination unit determines, as the updated coupling coefficient, a coupling coefficient between a second neuron and the first neuron, among a group of neurons in an output layer of the first neuron, in a case where the fluctuation noise is applied only to the first neuron of the neural network, and the magnitude of variance of the output data satisfies a predetermined condition; The learning unit re-learns a part of the learning model by updating only the updated combination coefficients determined by the determination unit.

4. 4. The pattern recognition device according to claim 3, The pattern recognition device, wherein the predetermined condition is that the magnitude of the variance of the output data is maximum.

5. 4. The pattern recognition device according to claim 3, The pattern recognition device, wherein the predetermined condition is that the magnitude of variance of the output data is a predetermined number in descending order.

6. 4. The pattern recognition device according to claim 3, The pattern recognition device, wherein the predetermined condition is that the magnitude of variance of the output data is equal to or greater than a predetermined threshold.

7. 4. The pattern recognition device according to claim 3, The pattern recognition device, wherein the learning unit causes the determination unit to determine the updated combination coefficients each time a learning loop in relearning is executed.

8. 2. The pattern recognition device according to claim 1, The second threshold used in the optimization is smaller than the first threshold.

9. A learning method in which a computer of a pattern recognition device learns a learning model for pattern recognition, comprising: A set of input data including a plurality of patterns and training data is obtained, generating fluctuation noise, which is a value based on a predetermined distribution; obtaining a value of an error function indicating a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; performing a first learning of the learning model by optimizing the error function so that the value of the error function is equal to or less than a first threshold; A learning method in which, after the first learning is completed, a part of the learning model is re-learned based on the magnitude of variance of output data based on the fluctuation noise and the input data.

10. A computer of a pattern recognition device that learns a learning model for pattern recognition, A procedure for acquiring a set of input data including a plurality of patterns and training data; A procedure for generating fluctuation noise, which is a value based on a predetermined distribution; a step of acquiring a value of an error function indicating a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the training data; performing a first learning of the learning model by optimizing the error function so that the value of the error function is equal to or less than a first threshold; and a program that executes a procedure for re-learning a portion of the learning model based on the magnitude of variance of output data based on the fluctuation noise and the input data after the first learning is completed.

Images