Pattern recognition device, learning method and program

Abstract

To provide a technology that contributes to shortening the learning time of a neural network. [Solution] A pattern recognition device comprising a learning data acquisition unit that acquires a pair of input data including a plurality of patterns and teacher data; a noise generation unit that generates fluctuation noise, which is a value based on a predetermined distribution; an error function value calculation unit that calculates the value of an error function that indicates the deviation between the teacher data and final output data, which is output data of an output layer based on the fluctuation noise and the input data; and a learning unit that performs learning of a learning model for pattern recognition by optimizing the error function, wherein after completing a first learning that is optimized so that the value of the error function is below a first threshold, the learning unit re-learns a part of the learning model based on the magnitude of variance of the output data based on the fluctuation noise and the input data.

Description

[Technical field]

[0001] The present disclosure relates to a pattern recognition device, a learning method, and a program. [Background technology]

[0002] Patent Document 1 discloses a fluctuation-driven learning method. The fluctuation-driven learning method is a technique in which learning is carried out by adding fluctuation noise having a predetermined distribution to the activation value of an activation function. By using this learning method, it is possible to learn a model that mixes a differentiable continuous function and a discontinuous function such as a threshold function. In addition, since no differentiation process or backpropagation process is required, there is no problem of learning not progressing due to disappearance of the gradient. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 8-235146 Summary of the Invention [Problem to be solved by the invention]

[0004] The following analysis was performed by the inventors of the present disclosure.

[0005] However, in the fluctuation-driven learning method of Patent Document 1, learning is performed using normal random numbers generated from a fixed variance value as the fluctuation noise given to the activity value of each neuron. At this time, the fluctuation noise is updated every time the learning loop is repeated. For this reason, learning takes a long time.

[0006] The present disclosure has been made in consideration of the above circumstances, and has as one of its objectives to provide a technique that contributes to shortening the learning time of a neural network. [Means for solving the problem]

[0007] According to a first aspect of the present disclosure, A learning data acquisition unit that acquires a set of input data including a plurality of patterns and teacher data; a noise generating unit that generates a fluctuation noise having a value based on a predetermined distribution; an error function value calculation unit that calculates a value of an error function indicating a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; a learning unit that optimizes the error function to learn a learning model for pattern recognition, A pattern recognition device is provided in which, after completing a first learning process optimized to make the value of the error function equal to or less than a first threshold, the learning unit re-learns a part of the learning model based on the fluctuation noise and the magnitude of variance of output data based on the input data.

[0008] According to a second aspect of the present disclosure, A learning method for a computer of a pattern recognition device to learn a learning model for pattern recognition, comprising the steps of: A set of input data including a plurality of patterns and training data is obtained, generating fluctuating noise, the value of which is based on a predetermined distribution; Obtaining a value of an error function indicating a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; performing a first learning of the learning model by optimizing the error function so that the value of the error function is equal to or less than a first threshold; There is provided a learning method in which, after the first learning is completed, re-learning of a part of the learning model is performed according to the magnitude of variance of output data based on the fluctuation noise and the input data.

[0009] According to a third aspect of the present disclosure, A computer of a pattern recognition device that performs learning of a learning model for pattern recognition, A step of acquiring a pair of input data including a plurality of patterns and training data; A step of generating fluctuation noise, the fluctuation noise being a value based on a predetermined distribution; A step of acquiring a value of an error function indicating a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; performing a first learning of the learning model by optimizing the error function so that the value of the error function is equal to or less than a first threshold; After the first learning is completed, a procedure of re-learning a part of the learning model based on the fluctuation noise and the magnitude of variance of the output data based on the input data is provided.

[0010] These programs can be recorded in a computer-readable storage medium. The storage medium can be a non-transitory medium such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. The present disclosure can also be embodied as a computer program product. Effect of the Invention

[0011] According to the present disclosure, the learning time of a neural network can be reduced. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1A is a diagram for explaining an example of a neural network of the present disclosure, and FIG. 1B is a diagram for explaining an example of a neuron of the present disclosure. [Diagram 2] 1A is a functional block diagram of an example of a pattern recognition device according to the present disclosure, and FIG. 1B is a flowchart of an example of a learning process according to the present disclosure. [Diagram 3] FIG. 2 is a functional block diagram of an example of a pattern recognition device according to the present disclosure. [Figure 4] FIG. 13 is a diagram for explaining an overview of an update combination coefficient determination process according to the present disclosure. [Diagram 5] 13 is a flowchart of an example of an initial learning process of the present disclosure. [Figure 6]13 is a flowchart of an example of a re-learning process of the present disclosure. [Figure 7] 13 is a flowchart of an example of an update coupling coefficient determination / update process of the present disclosure. [Figure 8] FIG. 2 is a configuration diagram showing an example of a hardware configuration of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. Note that the reference symbols in the drawings are added to each element for convenience as an example to facilitate understanding, and are not intended to limit the present disclosure to the illustrated embodiment. In addition, the connecting lines between blocks in the drawings and the like referred to in the following description include both bidirectional and unidirectional. A unidirectional arrow is a schematic representation of the flow of a main signal (data), and does not exclude bidirectionality. In addition, in the following description, "A and / or B" is used to mean A or B, or A and B.

[0014] <<First embodiment>> The pattern recognition device of this embodiment includes a neural network that is trained by inputting input data including a plurality of patterns multiple times.

[0015] The pattern recognition device trains a neural network using input data including a plurality of patterns. Specifically, the pattern recognition device trains the neural network using a fluctuation-driven learning method so that, in a problem of learning a group of patterns, the output value is approximate to a teacher signal that has one-to-one correspondence with the pattern. The neural network is, for example, a non-cyclic hierarchical neural network (non-cyclic neural network; hereinafter, simply referred to as a neural network).

[0016] Hereinafter, first, the neural network included in the pattern recognition device of this embodiment will be described, followed by a description of each component of the pattern recognition device, and then a description of the learning process performed by the pattern recognition device.

[0017] [Neural Networks] 1(a) is a block diagram illustrating a neural network 500 of a pattern recognition device according to this embodiment. As shown in the figure, the neural network 500 includes a plurality of neurons 510. Note that some reference numerals have been omitted to avoid cluttering the figure. The same applies to the following figures.

[0018] The multiple neurons 510 are divided into layers. The layers of the neural network 500 include an input layer 520, an intermediate layer 530, and an output layer 540. In the figure, as an example, the input layer 520 includes four neurons 510, the intermediate layer 530 includes six neurons 510, and the output layer 540 includes two neurons 510. Note that the number of neurons 510 included in each layer is not limited to these, and the number may be selected appropriately. Also, the intermediate layer 530 is not limited to one layer, and multiple intermediate layers 530 may be included.

[0019] 1(b) is a diagram illustrating the input and output of one neuron 510 in the neural network 500 according to this embodiment. As shown in the figure, input data is input to the neuron 510 from multiple neurons 510 in the input side layer. The neuron 510 then outputs a neuron output value r(p, t) shown in the following formula (1). Here, p indicates a learning pattern number, and t indicates time.

number

[0020] In the above formula (1), f(v(p,t)) on the right-hand side is a sigmoid function shown in formula (2).

number

number

number

[0021] The input data of pattern p is represented as q(p, t), where t indicates time.

[0022] For the error function e(p, t) indicating the deviation between the neuron output value r output from each neuron 510 in the output layer 540 and the teacher data t, for example, a square error as shown in equation (5) is used.

number

[0023] If the fluctuation noise n(t) is a normal random number, the update formula for the coupling coefficient w to reduce the value obtained by the error function e(p,t) shown in equation (5) (hereinafter referred to as the error e) is shown in the following equations (6) and (7).

number

number

[0024] In the neural network 500, the coupling coefficient w is sequentially updated according to the above formulas (6) and (7) to reduce the error e in formula (5). In other words, when input data containing a pattern is input, the neuron output value r from each neuron 510 in the output layer of the neural network 500 can be made to approach the teacher data t, and the pair of the input data q and the teacher data t can be learned.

[0025] In this case, in the fluctuation-driven learning method of Patent Document 1 (hereinafter referred to as the conventional method), the fluctuation noise n k For (t), normal random numbers with the probability density function shown in Equation (8) are used. In the conventional method, the variance σ k 2 is a fixed value.

number

[0026] In the conventional method, all the coupling coefficients w are updated every time the learning loop is repeated during learning, so it takes a long time to converge.

[0027] Here, the learning loop refers to a process of generating fluctuation noise and providing it to each neuron 510, inputting input data to neurons 510 in input layer 520, inputting the resulting output data, which is neuron output value r from output layer 540, and teacher data to an error function to obtain an error function value. In learning of neural network 500, the connection coefficient w is updated each time the learning loop is repeated, and the error function value is minimized.

[0028] In this embodiment, when the neural network 500 is trained, the period Tp of the pattern p is used to calculate the degree of influence on the output data of each neuron 510 in the neuron 510 group in the output layer from the neuron 510 of interest. Then, the coupling coefficient w to be updated is selected based on the degree of influence. This shortens the training time compared to updating all the coupling coefficients w.

[0029] [Pattern recognition device] The pattern recognition device of this embodiment that realizes this will be described below. Fig. 2(a) is a functional block diagram of the pattern recognition device 100 of this embodiment.

[0030] As shown in the figure, a pattern recognition device 100 of this embodiment includes a learning data acquisition section 110, a noise generation section 120, an error function value calculation section 130, and a learning section 140.

[0031] The learning data acquiring unit 110 acquires a set of input data including a plurality of patterns and teacher data as learning data. The learning data acquiring unit 110 acquires the learning data from a storage area that accumulates the learning data, and sends the learning data to the learning unit 140. The learning data may be acquired, for example, via a network.

[0032] The pattern recognition device 100 of this embodiment uses, for example, a non-cyclic hierarchical (feed-forward) neural network model (neural network 500) as a learning model. Input data and corresponding teacher data are presented to the learning unit 140, and learning is performed. The input data is an input vector input to neurons 510 in an input layer 520 of neurons 510 in the neural network 500 constituting the pattern recognition device 100.

[0033] In this embodiment, the learning data acquisition unit 110 acquires q(p, t) as input data for pattern p, and acquires t(p) as teacher data. Then, these pairs are sent to the learning unit 140.

[0034] The noise generating unit 120 generates fluctuation noise, which is a value based on a predetermined distribution. The fluctuation noise is used in a known fluctuation-driven learning method, and is a random value added to the activation value of each neuron 510 of the neural network 500 constituting the pattern recognition device 100 of this embodiment. The predetermined distribution may be, for example, a random value based on a normal distribution.

[0035] The error function value calculation section 130 calculates an error function value indicating the deviation between the output data based on the fluctuation noise and the input data and the teacher data.

[0036] The input data is input data among the learning data acquired by the learning data acquiring unit 110. The output data is an output vector output from the output layer 540 of the neuron 510. To distinguish this output data from the output data from the neuron 510 in the intermediate layer 530, this output data is also called, for example, final output data.

[0037] The error function is a function that outputs a value indicating the deviation between the teacher data in the learning data and the final output data as an error function value. For example, as shown in formula (5), the error function value may be the sum of squares of the difference between the teacher data vector and the output vector.

[0038] The learning unit 140 optimizes the error function value to learn the learning model for pattern recognition. Here, optimization means modifying the parameters in a direction that minimizes the error function value. Specifically, the learning unit 140 updates the coupling coefficient w between the neurons 510 based on a predetermined update formula, and repeatedly executes a learning loop so that the error function value is minimized.

[0039] The learning unit 140 of this embodiment performs learning in two stages. That is, a first threshold value α and a second threshold value β (α>β) smaller than the first threshold value α are prepared. Then, the learning unit 140 first performs initial learning (first learning) to optimize the error function value so that it is smaller than the first threshold value α. Then, after the initial learning (first learning) is completed, re-learning is performed on a part of the learning model depending on the magnitude of variance of the output data based on the fluctuation noise and the input data. When the error function value becomes smaller than the second threshold value β, the re-learning is terminated as it has been minimized, that is, converged.

[0040] The second threshold value β may be, for example, the same as a threshold value used to determine convergence in the conventional method.

[0041] In re-learning, the learning unit 140 defines the magnitude of the variance as the degree of influence, and updates only the coupling coefficient w whose magnitude of variance satisfies a predetermined condition, aiming for convergence.

[0042] [Learning process] Here, a flow of the learning process by the pattern recognition device 100 of this embodiment will be described. Fig. 2(b) shows a processing flow of the learning process by the pattern recognition device 100 of this embodiment. This process is started, for example, in response to a start instruction from a user.

[0043] First, the learning data acquiring unit 110 acquires learning data (step S1101). In this embodiment, as described above, a set of input data including a plurality of patterns and teacher data is acquired as the learning data.

[0044] The learning unit 140 performs initial learning (first learning) using the learning data (step S1102). Here, for example, learning data of the first pattern, pattern 1, is provided to the neural network 500, and activation values ​​v(p, t) of all neurons 510 are calculated according to the above formula (3). At this time, n(t) is a fluctuation noise input to all neurons 510 during the period Tp, and is generated by normal random numbers having a probability density function represented by the following formula (9), for example.

number

[0045] Then, the learning unit 140 repeats the learning loop while updating the coupling coefficient w until the error function value becomes smaller than a predetermined first threshold value α. The learning loop is to calculate the final output data using each of the above equations, and to calculate the error function value by the above equation (5).

[0046] Thereafter, the learning unit 140 re-learns a part of the learning model (step S1103) and ends the process. In the re-learning, as described above, from the state where the initial learning is completed, learning is repeated until the error function value becomes smaller than the second threshold value β, while updating only the coupling coefficient w whose influence degree satisfies a predetermined condition. Note that, for each repetition, the coupling coefficient w to be updated is selected.

[0047] As described above, according to the pattern recognition device 100 of this embodiment, learning is performed in two stages. That is, first, the coupling coefficients w between all neurons 510 are updated for each iteration of the learning loop, and the learning model is also learned to a certain extent, leading to convergence. After that, only the coupling coefficients w between neurons 510 with large influence are updated, aiming for highly accurate convergence.

[0048] According to the conventional method, until learning is completed, the coupling coefficients w between all neurons 510 are updated. In contrast, in the pattern recognition device 100 of the present embodiment, the number of updates to the coupling coefficients w can be significantly reduced, and therefore the calculation time can be shortened accordingly.

[0049] According to the pattern recognition device 100 of this embodiment, the learning time of the neural network 500 can be shortened.

[0050] <<Second embodiment>> Next, a second embodiment of the present disclosure will be described. This embodiment is an embodiment in which the first embodiment is embodied in more detail. In this embodiment, configurations having the same names as those in the first embodiment basically have the same functions as those in the first embodiment. Hereinafter, this embodiment will be described with a focus on the differences from the first embodiment.

[0051] The pattern recognition device 100 of this embodiment basically has the same configuration as that of the first embodiment. As shown in FIG.

[0052] The learning data acquiring unit 110, the noise generating unit 120, and the error function value calculating unit 130 are similar to those in the first embodiment, and therefore a description thereof will be omitted here.

[0053] The learning unit 140 optimizes the error function value to learn a learning model for pattern recognition. In addition, the learning unit 140 of this embodiment has the determination unit 150 determine an updated combination coefficient, which is the combination coefficient w to be updated, during re-learning. Then, by updating only the updated combination coefficient, re-learning of a part of the learning model is performed.

[0054] The determination unit 150 determines the updated coupling coefficient during re-learning. The determination unit 150 gives fluctuation noise only to a selected neuron 510 (first neuron) of the neural network 500. Then, among the group of neurons 510 in the output layer of the first neuron, the neuron 510 with the largest variance of output data is selected as the second neuron, and the coupling coefficient w between the first neuron and the second neuron is determined to be the updated coupling coefficient.

[0055] That is, in the learning loop, before updating the coupling coefficients, the determination unit 150 uses the magnitude of variance of the output data described above as the degree of influence of each neuron 510 (first neuron) of the neural network 500 on the group of neurons 510 in the output layer.

[0056] Specifically, before updating the coupling coefficient, that is, in the state at the end of the immediately preceding learning loop, the determination unit 150 selects one neuron 510 from the neural network 500 as a first neuron. Then, the determination unit 150 causes the noise generation unit 120 to generate fluctuation noise n(t) of the period Tp to be input to the selected first neuron. Here, the noise generation unit 120 generates the fluctuation noise n(t), for example, by the above formula (9).

[0057] The determination unit 150 inputs the input data acquired by the learning data acquisition unit 110 to the neural network 500. At this time, the calculated fluctuation noise n(t) is input only to the selected first neuron, and the variance of the output values ​​of each neuron 510 in the group of neurons 510 in the output layer is calculated.

[0058] Here, the variance is calculated, for example, by the following formula (10).

number

[0059] Equation (10) represents the variance of the output of each neuron 510 in the group of neurons 510 in the output layer when the fluctuation noise n(t) is applied only to the first neuron, i.e., the selected neuron 510, while all values ​​other than the fluctuation noise n(t), i.e., the activity value v, membrane potential u, coupling coefficient w, and input data, are fixed.

[0060] For example, suppose that the hidden layer 530 of the neural network 500 includes a k-layer and a j-layer. The j-layer is the output side layer of the k-layer. An image of this case is shown in FIG.

[0061] In FIG. 4, the first neuron is the k-th neuron k in the k-th layer. The neurons 510 in the output layer of neuron k are neurons 1 to J. In this figure, σ j 2 Let us denote this as σ_j.

[0062] σ_1 is the variance of the output value of the first neuron 1 in the jth layer when a fluctuation noise n(t) is given to neuron k. σ_j is the variance of the output value of the jth neuron j in the jth layer. σ_J is the variance of the output value of the jth neuron J in the jth layer.

[0063] The determination unit 150 defines the magnitude of this variance as the degree of influence of the first neuron on each of the group of neurons 510 in the output layer. Then, the determination unit 150 determines, among the group of neurons 510 in the output layer, the neuron 510 (neuron j) having the largest magnitude of variance of the output value as the second neuron, as shown in the following formula (11), and determines the coupling coefficient w between the first neuron and the second neuron as the updated coupling coefficient.

number

[0064] The determination unit 150 performs the above process for all neurons 510 (excluding neurons 510 in the output layer) constituting the neural network 500, determines the updated connection coefficients, and updates them. Thus, according to this embodiment, for each neuron 510, only one connection coefficient w is determined as the updated connection coefficient and updated.

[0065] [Learning process] The flow of the learning process of this embodiment will be described. As in the first embodiment, the learning process of this embodiment has two stages: an initial learning process and a re-learning process of a part of the learning model.

[0066] [Initial learning process] First, the flow of the initial learning process will be described. The initial learning process basically performs the same processing as the learning loop of the conventional method. Fig. 5 shows the processing flow of the initial learning process of this embodiment.

[0067] First, the learning unit 140 calculates the variance d (=σ k 2 ) is initialized to a fixed value (step S2101).

[0068] Next, the learning unit 140 initializes the coupling coefficient w (step S2102).

[0069] Next, the learning unit 140 causes the noise generating unit 120 to generate fluctuation noise n(t) during the period Tp (step S2103). The fluctuation noise n(t) generated here is input to all neurons 510 constituting the neural network 500.

[0070] The learning unit 140 calculates the neuron output value r according to, for example, the above formulas (1) to (4) (step S2104). This neuron output value r is the final output data, which is the output value from each neuron 510 in the output layer 540.

[0071] The learning unit 140 causes the error function value calculation unit 130 to calculate the error e between the neuron output value r and the teacher data t (step S2105). The error function value calculation unit 130 calculates the error function value as the error e according to, for example, the error function of the above formula (5).

[0072] The learning unit 140 determines whether the obtained error e (error function value) is smaller than a first threshold value α given in advance for initial learning (step S2106). If it is smaller, the initial learning process ends.

[0073] On the other hand, if the error e is equal to or greater than the first threshold value α, the learning unit 140 updates the coupling coefficient w according to equations (6) and (7) (step S2107), returns to step S2103, and repeats the process.

[0074] [Relearning process] Next, the flow of the re-learning process of this embodiment will be described. Fig. 6 shows the process flow of the re-learning process of this embodiment. This process is started when the initial learning is completed.

[0075] First, the learning unit 140 causes the determination unit 150 to execute an update coupling coefficient determination / update process for determining and updating an update coupling coefficient (step S2201). The flow of the update coupling coefficient determination / update process will be described later.

[0076] The learning unit 140 calculates the neuron output value r according to, for example, the above formulas (1) to (4) (step S2202). This neuron output value r is the final output data, which is the output value from each neuron 510 in the output layer 540.

[0077] The learning unit 140 causes the error function value calculation unit 130 to calculate the error e between the neuron output value r and the teacher data t (step S2203). The error function value calculation unit 130 calculates the error function value as the error e according to, for example, the error function of the above formula (5).

[0078] The learning unit 140 determines whether the obtained error e (error function value) is smaller than a second threshold value β given in advance for re-learning (step S2204). If it is smaller, the re-learning process ends.

[0079] On the other hand, if the error e is equal to or greater than the second threshold value β, the learning unit 140 returns to step S2201 and repeats the process.

[0080] [Determine and update the coupling coefficient] Next, a description will be given of the flow of the updated coupling coefficient determination / update process by the determination unit 150 in the above step S2201. Fig. 7 shows the process flow of the updated coupling coefficient determination / update process of this embodiment.

[0081] First, the determining unit 150 selects one neuron 510 from among the unselected neurons 510 constituting the neural network 500 (step S2301). Hereinafter, the selected neuron 510 is referred to as a first neuron.

[0082] The determining unit 150 causes the noise generating unit 120 to generate fluctuation noise n(t) during the period Tp (step S2302). The fluctuation noise n(t) generated here is input to a first neuron.

[0083] The determining unit 150 inputs the input data to the neural network 500 (step S2303). The input data is data acquired by the learning data acquiring unit 110.

[0084] The determination unit 150 obtains the neuron output value r according to the above formulas (1) to (4) and calculates its variance (step S2304). This neuron output value r is the output value of each neuron 510 of the group of neurons 510 in the output layer of the first neuron. For example, in the example of FIG. 4, when the first neuron is a neuron 510 in the kth layer, the output layer is the jth layer, and the group of neurons 510 is neurons 1 to J in the jth layer. The determination unit 150 calculates the variance of the neuron output value r for each neuron 510 in the group of neurons in the output layer according to formula (10).

[0085] After that, the determination unit 150 determines the updated coupling coefficient based on the calculated variance (step S2305). In this embodiment, the neuron 510 having the largest variance of the neuron output value r among the neurons 510 in the output layer is determined as the second neuron, and the coupling coefficient w between the first neuron and the second neuron is determined as the updated coupling coefficient.

[0086] The determining unit 150 updates the determined updated coupling coefficients (step S2306). The update is performed according to the above formulas (6) and (7).

[0087] The determination unit 150 determines whether all neurons 510 other than the neuron 510 in the output layer have been selected and this update coupling coefficient determination / update process has been performed as the first neuron (step S2307), and if so, ends the process.

[0088] As described above, the present embodiment has the same configuration as the first embodiment, and therefore provides the same effects as the first embodiment.

[0089] Furthermore, according to this embodiment, during re-learning, for each neuron 510 constituting the neural network 500, only the coupling coefficient w that has the greatest influence on the neuron 510 in the output layer of that neuron 510 is updated. This further reduces the number of updates to the coupling coefficient w, thereby further shortening the calculation time.

[0090] That is, according to the pattern recognition device 100 of this embodiment, the learning time of the neural network 500 can be shortened.

[0091] <Variation 1> In the above second embodiment, the determination unit 150 determines, as the updated coupling coefficient, only the coupling coefficient w between the first neuron and the neuron 510 having the largest variance of the neuron output value r calculated by equation (10) among the neuron group in the output layer, but this is not limited to this.

[0092] For example, a predetermined number m (m is an integer equal to or greater than 1) of neurons 510 with the largest variance, i.e., the top m neurons with the largest variance, may be selected from the group of neurons 510 in the output layer as the second neurons. In this case, too, only the coupling coefficient w between the first neuron and each second neuron is treated as the updated coupling coefficient.

[0093] Alternatively, a neuron 510 whose variance is equal to or greater than a predetermined threshold may be selected from the group of neurons 510 in the output layer as the second neuron. In this case, too, only the coupling coefficient w between the first neuron and each second neuron is set as the updated coupling coefficient.

[0094] Furthermore, any of the update combination coefficient determination methods may be selectable. In this case, the pattern recognition device 100 is capable of executing a plurality of update combination coefficient determination methods. The pattern recognition device 100 may be provided with a user interface that allows the user to select which update combination coefficient determination method to use.

[0095] <Variation 2> In the second embodiment, the determination section 150 updates the updated combination coefficient immediately after determining the updated combination coefficient, but the present invention is not limited to this.

[0096] For example, the determination unit 150 may only determine the updated binding coefficient without performing the process of step S2306. That is, the process executed in S2201 of the re-learning process may be the updated binding coefficient determination process. Then, the learning unit 140 may perform a process of updating the updated binding coefficient after the updated binding coefficient determination process of S2201 during the re-learning process.

[0097] <Variation 3> In the above embodiments, the degree of influence of the neuron 510 of interest on the group of neurons 510 in the output layer is measured by the magnitude of the variance of the output data. However, this is not limited to the magnitude of the variance. The degree of influence can be any of various statistical values ​​that indicate the degree of fluctuation in the output data.

[0098] <Variation 4> In each of the above embodiments, the determination unit 150 determines the updated coupling coefficient every time the re-learning learning loop is repeated, but this is not limiting. The determined updated coupling coefficient may be maintained while the learning loop is repeated a predetermined number of times (multiple times).

[0099] Furthermore, the determined updated coupling coefficients may not be changed until the re-learning is completed. In this case, only the determined updated coupling coefficients are updated, and the re-learning learning loop is repeated.

[0100] <Variation 5> In the above embodiment, the activation function is a sigmoid function as shown in formula (2), but the activation function is not limited to this. The sigmoid function is a differentiable continuous function, but a tanh function (Hyperbolic tangent function) or a discontinuous ReLU may be used, or these may be mixed.

[0101] [Hardware configuration] The pattern recognition device 100 of each of the above embodiments may be realized by, for example, a general-purpose information processing device. The general-purpose information processing device includes, for example, a CPU (Central Processing Unit) 191, a main storage device (memory) 192, an auxiliary storage device 193, a communication I / F 194, and an expansion I / F 195, which are interconnected by an internal bus, as shown in FIG. 8 .

[0102] The CPU 191 realizes the above-mentioned functions and centrally controls the entire device by, for example, loading a program stored in the auxiliary storage device 193 into the main storage device 192 and executing the program. Note that, instead of the CPU 191, one or more processors such as an MPU (Micro Processing Unit) may be used.

[0103] The main storage device 192 is a memory such as a RAM (Random Access Memory), etc. The main storage device 192 is a work area when the CPU 191 processes programs and the like executed by the installed devices.

[0104] The auxiliary storage device 193 is, for example, a read only memory (ROM), a hard disk drive (HDD), a solid state drive (SSD), etc. The auxiliary storage device 193 stores various programs executed by the device on which it is installed. In addition, in each of the above embodiments and / or each of the modified examples, the storage area of ​​the pattern recognition device 100 may be constructed in the auxiliary storage device 193.

[0105] For example, the learning data may be stored in the auxiliary storage device 193. In addition, various data used in the processing, including, for example, the first threshold value α, the second threshold value β, and the like, may also be stored in the auxiliary storage device 193. In addition, various data generated during the processing is stored in either the auxiliary storage device 193 or the main storage device 192.

[0106] The auxiliary storage device 193 may include a storage medium such as a flexible disk, a hard disk, an optical disk, a CD-ROM, a CD-R, a magnetic tape, a non-volatile memory card, a DVD, or the like.

[0107] The above-mentioned functions of the pattern recognition device 100 are realized by the CPU 191 loading a program stored in the auxiliary storage device 193 into the main storage device 192 and executing it.

[0108] The programs for realizing the above-mentioned functions can be recorded in a computer-readable storage medium. The storage medium can be a non-transitory one such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. The present disclosure can also be embodied as a computer program product. In addition, the programs stored in the auxiliary storage device 193 can be provided as a program product recorded in a non-transitory computer-readable recording medium. The auxiliary storage device 193 can be used to store various programs recorded in a non-transitory computer-readable recording medium for a medium to long term.

[0109] The communication I / F 194 is an interface for inputting and outputting signals and data by wire or wirelessly. The communication I / F 194 may also include an antenna, a modem, etc. For example, when the training data acquisition unit 110 acquires training data from a device external to the pattern recognition device 100, the training data may be acquired from a network via the communication I / F 194.

[0110] The expansion I / F 195 includes, for example, a USB interface and a memory slot, and is connected to an external storage unit, an input device, an output device, and the like. The input device is, for example, a device that accepts user operations such as a keyboard or a mouse. The output device is a display device such as a display. The liquid crystal monitor may have a touch panel function that accepts instructions by a user's touch operation. The expansion I / F 195 may have a plurality of different types of interfaces.

[0111] For example, the user issues an instruction to start processing via an input device or the like connected to the expansion I / F 195. In addition, the first threshold value α, the second threshold value β, and the like may be set.

[0112] The hardware configuration of each device is not limited to the above. Each function (server) of each device may be implemented, for example, by an integrated circuit (IC) dedicated to each process, an application specific integrated circuit (ASIC), a system on chip (SOC), a field programmable gate array (FPGA), or the like.

[0113] In the process flow used in the above description, multiple steps (processes) are described in order, but the order of execution of each step is not limited to the order described. For example, the order of the steps shown in the figure can be changed to a range that does not affect the content, such as executing each process in parallel.

[0114] Although the embodiments and modifications of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments, and various modifications that can be understood by those skilled in the art can be made. Each embodiment and modification can be appropriately combined with other embodiments. For example, the network configuration and the configuration of each element shown in each drawing are examples to help understand the present disclosure, and are not limited to the configurations shown in these drawings.

[0115] Finally, preferred embodiments of the present disclosure will be summarized. A part or all of the above-described embodiments can be described as follows, but are not limited to the following. (Appendix 1) The pattern recognition device A learning data acquisition unit that acquires a set of input data including a plurality of patterns and teacher data; a noise generating unit that generates a fluctuation noise having a value based on a predetermined distribution; an error function value calculation unit that calculates a value of an error function indicating a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; and a learning unit that optimizes the error function to learn a learning model for pattern recognition. After completing a first learning process that optimizes the value of the error function so that it is equal to or less than a first threshold, the learning unit re-learns a portion of the learning model based on the fluctuation noise and the magnitude of variance of the output data based on the input data. (Appendix 2) 13. The pattern recognition device according to claim 1, It is preferable that the learning unit determines a re-learning portion, which is a part of the learning model to be re-learned, based on the magnitude of the variance. (Appendix 3) 12. The pattern recognition device according to claim 1, the learning model is a neural network; The learning unit includes a determination unit that determines an updated coupling coefficient, which is a coupling coefficient to be updated; the determination unit determines, as the updated coupling coefficient, a coupling coefficient between a second neuron and the first neuron, among a group of neurons in an output layer of the first neuron, in a case where the fluctuation noise is applied only to the first neuron of the neural network, the magnitude of variance of the output data of the first neuron satisfies a predetermined condition; It is preferable that the learning unit re-learns a part of the learning model by updating only the updated combination coefficients determined by the determination unit. (Appendix 4) 4. The pattern recognition device according to claim 3, It is desirable that the predetermined condition is that the magnitude of variance of the output data is maximized. (Appendix 5) 4. The pattern recognition device according to claim 3, It is desirable that the predetermined condition is a predetermined number in order from the largest magnitude of variance of the output data. (Appendix 6) 4. The pattern recognition device according to claim 3, The predetermined condition is preferably such that the magnitude of variance of the output data is equal to or greater than a predetermined threshold value. (Appendix 7) 7. The pattern recognition device according to claim 3, It is preferable that the learning unit causes the determination unit to determine the updated coupling coefficients each time a learning loop in relearning is executed. (Appendix 8) 7. The pattern recognition device according to claim 1, The second threshold used in the optimization is preferably smaller than the first threshold. (Appendix 9) A learning method in which a computer of a pattern recognition device learns a learning model for pattern recognition is as follows: A set of input data including a plurality of patterns and training data is obtained, generating fluctuating noise, the value of which is based on a predetermined distribution; Obtaining a value of an error function indicating a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; performing a first learning of the learning model by optimizing the error function so that the value of the error function is equal to or less than a first threshold; After the first learning is completed, a part of the learning model is re-learned according to the magnitude of variance of the output data based on the fluctuation noise and the input data. (Appendix 10) The program includes: A computer of a pattern recognition device that performs learning of a learning model for pattern recognition, A step of acquiring a pair of input data including a plurality of patterns and training data; A step of generating fluctuation noise, the fluctuation noise being a value based on a predetermined distribution; A step of acquiring a value of an error function indicating a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; performing a first learning of the learning model by optimizing the error function so that the value of the error function is equal to or less than a first threshold; After the first learning is completed, a procedure of re-learning a part of the learning model is executed based on the fluctuation noise and the magnitude of variance of the output data based on the input data. (Appendix 11) The pattern recognition device A learning data acquisition unit that acquires a set of input data including a plurality of patterns and teacher data; a noise generating unit that generates a fluctuation noise having a value based on a predetermined distribution; an error function value calculation unit that calculates a value of an error function indicating a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; and a learning unit that optimizes the error function to learn a learning model for pattern recognition. After completing a first learning process that optimizes the value of the error function so that it is equal to or less than a first threshold, the learning unit re-learns a portion of the learning model depending on the degree of influence of the fluctuation noise and the input data on the output data. (Appendix 12) In the pattern recognition device described in Supplementary Note 11, it is preferable that the degree of influence is a statistical value indicating a degree of fluctuation in the output data, including a magnitude of variance of the output data. The forms of Supplements 9 and 10 can be expanded into the forms of Supplements 2-8, as in Supplementary Note 1. The form of Supplementary Note 11 can be expanded into the forms of Supplements 3-8.

[0116] The disclosures of the above patent documents and the like are incorporated herein by reference. Within the scope of this disclosure (including the scope of claims), modifications and adjustments of the embodiments and modifications are possible based on the basic technical ideas. Furthermore, within the scope of this disclosure, various combinations and selections of the various disclosed elements (including each element of each claim, each element of each embodiment or modification, each element of each drawing, etc.) are possible. In other words, this disclosure naturally includes various modifications and corrections that a person skilled in the art would be able to make in accordance with the entire disclosure, including the scope of claims, and the technical ideas. In particular, with regard to the numerical ranges described in this specification, any numerical value or subrange included in the range should be interpreted as being specifically described even if not otherwise specified. [Explanation of symbols]

[0117] 100: pattern recognition device, 110: learning data acquisition unit, 120: noise generation unit, 130: error function value calculation unit, 140: learning unit, 150: determination unit, 191: CPU, 192: main storage device, 193: auxiliary storage device, 194: communication I / F, 195: extension I / F, 500: neural network, 510: neurons, 520: input layer, 530: intermediate layer, 540: output layer, Tp: pattern input duration, d: variance, e: error, n: noise, p: pattern, q: input data, r: neuron output value, t: teacher data, u: membrane potential, v: activity value, w: coupling coefficient, α: first threshold, β: second threshold, σ: variance

Claims

1. A learning data acquisition unit that acquires a set of input data including a plurality of patterns and teacher data; a noise generating unit that generates a fluctuation noise having a value based on a predetermined distribution; an error function value calculation unit that calculates a value of an error function indicating a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; a learning unit that optimizes the error function to learn a learning model for pattern recognition, A pattern recognition device, wherein after completing a first learning process optimized to make the value of the error function equal to or less than a first threshold, the learning unit re-learns a portion of the learning model based on the fluctuation noise and the magnitude of variance of output data based on the input data.

2. 2. The pattern recognition device according to claim 1, The pattern recognition device, wherein the learning unit determines a re-learning portion, which is a part of the learning model to be re-learned, based on the magnitude of the variance.

3. 2. The pattern recognition device according to claim 1, the learning model is a neural network; The learning unit includes a determination unit that determines an updated coupling coefficient, which is a coupling coefficient to be updated; the determination unit determines, as the updated coupling coefficient, a coupling coefficient between a second neuron and the first neuron, among a group of neurons in an output layer of the first neuron, in a case where the fluctuation noise is applied only to the first neuron of the neural network, the magnitude of variance of the output data of the first neuron satisfies a predetermined condition; The learning unit re-learns a part of the learning model by updating only the updated combination coefficients determined by the determination unit.

4. 4. The pattern recognition device according to claim 3, A pattern recognition apparatus, wherein the predetermined condition is that the magnitude of variance of the output data is maximum.

5. 4. The pattern recognition device according to claim 3, The pattern recognition device, wherein the predetermined condition is a predetermined number, in order from the largest, of the magnitude of variance of the output data.

6. 4. The pattern recognition device according to claim 3, The pattern recognition device, wherein the predetermined condition is that the magnitude of variance of the output data is equal to or greater than a predetermined threshold.

7. 4. The pattern recognition device according to claim 3, The pattern recognition device, wherein the learning unit causes the determination unit to determine the updated combination coefficients each time a learning loop in re-learning is executed.

8. 2. The pattern recognition device according to claim 1, A pattern recognition apparatus, wherein the second threshold used in the optimization is smaller than the first threshold.

9. A learning method for a computer of a pattern recognition device to learn a learning model for pattern recognition, comprising the steps of: A set of input data including a plurality of patterns and training data is obtained, generating fluctuating noise, the value of which is based on a predetermined distribution; Obtaining a value of an error function indicating a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; performing a first learning of the learning model by optimizing the error function so that the value of the error function is equal to or less than a first threshold; A learning method comprising the steps of: after the first learning is completed, re-learning a part of the learning model based on the fluctuation noise and the magnitude of variance of output data based on the input data.

10. A computer of a pattern recognition device that performs learning of a learning model for pattern recognition, A step of acquiring a pair of input data including a plurality of patterns and training data; A step of generating fluctuation noise, the fluctuation noise being a value based on a predetermined distribution; A step of acquiring a value of an error function indicating a deviation between final output data, which is output data of an output layer based on the fluctuation noise and the input data, and the teacher data; performing a first learning of the learning model by optimizing the error function so that the value of the error function is equal to or less than a first threshold; After the first learning is completed, a procedure of re-learning a part of the learning model based on the fluctuation noise and the magnitude of variance of the output data based on the input data is performed.

Images