Learning control device and inference device

Abstract

Pattern recognition suitable for given data can be suitably used.SOLUTION: The learning control device includes: an acquisition unit that acquires a class number q indicating the number of classes of the type of learning data in a learning stage by supervised learning of a probabilistic neural network that classifies given data; a selection unit that selects, based on a user's selection operation, a formula to be used in calculation of an output value from a plurality of formulas that obtain a radius value σ as a hyperparameter, based on the class of the learning data xtr, the class number q, and a coupling coefficient c between an input layer and an intermediate layer; a configuration update unit that updates a configuration of coupling between an intermediate layer unit and an output layer unit based on a class of the learning data and a class type of the output layer unit; an output calculation unit that calculates an output value of the intermediate layer unit and the output layer unit using the selected formula; and a determination unit that determines a classification result validity of the given learning data based on the calculated output value.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a learning control device and an inference device. [Background technology]

[0002] The recent boom in artificial intelligence (AI) has been brought about by deep learning techniques, and AI is being used in a wide variety of applications, including image and speech recognition, natural language processing, medical diagnostic systems, and autonomous driving-related technologies.

[0003] Deep learning (DL) is a machine learning technique that uses an artificial neural network model (NNM) that mathematically mimics the neural network in the brain. Artificial neural network models used in deep learning are often based on the multi-layer perceptron (MLP) model, which was widely used from the 1980s to the 1990s, or derivative models of MLP, such as recurrent neural networks (RNNs) and convolutional neural networks (CNNs). For example, Patent Document 1 shows an example of generating synthetic images using a convolutional neural network.

[0004] However, MLPs require repeated training, tuning of many hyperparameters, and huge amounts of training data. MLPs also suffer from problems such as local solutions, black-box problems, and catastrophic forgetting during continuous training.

[0005] In contrast, a probabilistic neural network (PNN), which is one of the artificial neural network models, does not require repeated learning and has only one hyperparameter (the radius value of the radial basis function). In addition, since PNNs retain the learning data itself in the connection coefficients between the input layer and the hidden layer, they do not encounter problems such as local solutions, black box problems, and catastrophic forgetting during continuous learning. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2024-071870 Summary of the Invention [Problem to be solved by the invention]

[0007] However, a typical PNN requires training data as coupling coefficients, which can require a huge amount of memory. Also, there is no established method for users to adjust the radius of the radial basis functions appropriately.

[0008] The present invention has been made in view of the above-mentioned points, and has as its object to provide a technique that can suitably utilize pattern recognition suited to given data. [Means for solving the problem]

[0009] One aspect of the present invention is a learning control device that, during a learning stage using supervised learning of a probabilistic neural network that classifies given data, includes: an acquisition unit that acquires a class number q indicating the number of class types of training data; a selection unit that selects, based on a user's selection operation, an equation to be used in calculating an output value from among multiple equations for determining a radius value σ, which is a hyperparameter, based on the training data xtr, the class number q, and a coupling coefficient c between the input layer and the hidden layer; a configuration update unit that updates the coupling configuration of the hidden layer units and the output layer units based on the class of the training data and the class type of the output layer unit; an output calculation unit that calculates output values ​​of the hidden layer units and the output layer units using the selected equation; and a determination unit that determines the validity of the classification result of the given training data based on the calculated output value.

[0010] In addition, in one aspect of the present invention, the learning control device further includes an update unit that updates the coupling coefficient c between the input layer and the intermediate layer when the classification result is determined to be valid as a result of the judgment by the judgment unit.

[0011] In one aspect of the present invention, the configuration update unit further removes at least one of the intermediate layer units and the output layer units corresponding to the class to be removed, selected based on a user operation, and the connections to the units, from the configuration of the probabilistic neural network.

[0012] One aspect of the present invention is an inference device that, in an inference stage using a trained probabilistic neural network, includes an acquisition unit that acquires the number of classes q in the intermediate layer of the probabilistic neural network and the coupling coefficient c between the input layer and the intermediate layer; a selection unit that selects, based on a user's selection operation, an equation to be used in the inference calculation from among multiple equations for determining the radius value σ, which is a hyperparameter, based on the data xts to be inferred and the acquired number of classes q and the coupling coefficient c; and an output calculation unit that calculates output values ​​of the intermediate layer units and the output layer units using the selected equation. [Effects of the Invention]

[0013] According to the present invention, it is possible to suitably utilize pattern recognition suited to given data. [Brief explanation of the drawings]

[0014] [Figure 1] 10 is a flowchart illustrating an example of a processing flow in a learning stage in the first embodiment. [Figure 2] FIG. 10 is a diagram illustrating an example of initial settings for constructing a probabilistic neural network. [Figure 3] FIG. 1 is a diagram illustrating a first example of updating the configuration of a probabilistic neural network. [Figure 4] FIG. 10 is a diagram illustrating a second example of updating the configuration of a probabilistic neural network. [Figure 5] FIG. 1 is a diagram for explaining equivalence of probabilistic neural networks. [Figure 6] 10 is a flowchart illustrating an example of a processing flow of the inference device according to the embodiment. [Figure 7] FIG. 1 is a diagram showing a training dataset used in a validation experiment. [Figure 8] FIG. 10 is a diagram showing the results of a verification experiment in normal learning. [Figure 9] FIG. 10 shows the results of a verification experiment in additional learning of a continuous class. [Figure 10] FIG. 10 shows the results of a verification experiment in continuous data addition learning using radius value setting method 1. [Figure 11] FIG. 10 shows the results of a verification experiment in continuous data addition learning using radius value setting method 2. [Figure 12] 10 is a flowchart illustrating an example of the processing flow in the learning stage in the second embodiment. [Figure 13] 1 is a block diagram illustrating an example of a functional configuration of a learning control device according to an embodiment. [Figure 14]FIG. 2 is a block diagram illustrating an example of the functional configuration of the inference device according to the embodiment. [Figure 15] FIG. 2 is a block diagram showing an example of the internal configuration of a learning control device and an inference device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] [Embodiment] A learning control device and an inference device according to the present embodiment will be described in detail below with reference to the accompanying drawings, showing preferred embodiments. In the drawings, identical or similar parts are designated by identical or similar reference numerals. Note that the present embodiment is not limited to these embodiments and includes various modifications or improvements. In other words, the components described below include those that can be easily imagined by a person skilled in the art and those that are substantially identical, and the components described below can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components of the present embodiment may be made without departing from the spirit of the present invention.

[0016] [Learning stage] FIG. 1 is a flowchart illustrating an example of the processing flow of the learning stage in embodiment 1. The processing of the learning stage is performed by a learning control device 10. The learning control device 10 performs supervised learning to construct a probabilistic neural network PNN that performs class classification on given data. With reference to the same figure, the flow of automatically constructing a probabilistic neural network PNN based on learning data will be described.

[0017] The learning control device 10 receives as input a learning data set Xtr={(xtr(1),t(1)),(xtr(2),t(2)),...,(xtr(Ntr),t(Ntr))}. xtr indicates each piece of learning data, and t indicates the class to which each piece of learning data belongs. The numbers in parentheses identify each piece of learning data xtr and class t. That is, the learning data set Xtr stores a total of Ntr pieces of learning data, from learning data xtr(1) to learning data xtr(Ntr), and a total of Ntr pieces of class data, from t(1) to t(Ntr), which are the classes of each piece of learning data xtr.

[0018] Specifically, take as an example the training data for a task of determining whether an object is a dog or a cat. If the training data for the task of determining whether an object is a dog or a cat includes four images in the order of a dog image, a dog image, a cat image, and a cat image, the training data set Xtr may be, for example, {(xtr(1) = dog image, t(1) = dog), (xtr(2) = dog image, t(2) = dog), (xtr(3) = cat image, t(3) = cat), (xtr(4) = cat image, t(4) = cat)}.

[0019] [Learning stage - Initial settings] 2 is a diagram for explaining an example of initial settings for constructing a probabilistic neural network PNN. Steps S101 to S105 will be described with reference to the diagram.

[0020] (Step S101) The learning control device 10 declares variables i, j, and k and assigns 1 to each variable. i is a variable indicating the number of learning data to be processed among the learning data, and is an integer ranging from 1 to Ntr. j is a variable corresponding to the number of hidden layer units h of the probabilistic neural network PNN constructed by the learning control device 10. k is a variable corresponding to the number of output layer units o of the probabilistic neural network PNN constructed by the learning control device. The probabilistic neural network PNN has a predetermined number of input layer units x, and the number of hidden layer units h and output layer units o is predetermined to be 0. In FIG. 2, the probabilistic neural network PNN includes three input layer units x: a first input layer unit x1, a second input layer unit x2, and a third input layer unit x3; however, this embodiment is not necessarily limited to this example.

[0021] (Step S102) The learning control device 10 adds a first hidden layer unit h1 (j=1) to the probabilistic neural network PNN. That is, the probabilistic neural network PNN initially includes one hidden layer unit h.

[0022] (Step S103) The learning control device 10 sets the coupling coefficient c1 between each input layer unit x included in the probabilistic neural network PNN and the first hidden layer unit h1 added in step S102 as xtr(1). Note that in the hidden layer unit h according to this embodiment, the same coupling coefficient c is set for each input layer unit x. In the following description, the coupling coefficient c between each input layer unit x and one hidden layer unit h may be simply referred to as the coupling coefficient c of the hidden layer unit h.

[0023] (Step S104) The learning control device 10 adds a first output layer unit o1 (k=1) to the probabilistic neural network PNN. That is, the probabilistic neural network PNN initially includes one output layer unit o.

[0024] (Step S105) The learning control device 10 sets the coupling coefficient w11 between the first hidden layer unit h1 added in step S102 and the first output layer unit o1 added in step S104 to 1.

[0025] (Step S106) The learning control device 10 sets the initial number of classes q. The number of classes q is set according to a class number setting method.

[0026] (Step S107) The learning control device 10 increments the count of the variable i indicating the number of the learning data xtr by 1 (i←i+1). That is, the learning control device 10 processes the learning data xtr(1) and then processes the learning data xtr(2).

[0027] [Learning phase - 1st update of configuration] Fig. 3 is a diagram for explaining a first example of updating the configuration of the probabilistic neural network PNN. Fig. 3 shows the configuration of the probabilistic neural network PNN updated after Fig. 2. In Fig. 3, i=2. Steps S108 to S113 will be described with reference to the diagram.

[0028] (Step S108) The learning control device 10 determines whether the probabilistic neural network PNN includes an output layer unit o corresponding to the class t(i) to which the training data xtr(i) belongs. The output layer unit o corresponding to the class t(i) is the output layer unit o that is connected with a hidden layer unit h, whose connection coefficient c is the training data (not limited to the training data xtr(i)) belonging to the class t(i), by a connection coefficient w=1. For example, the output layer unit o corresponding to the class t(i) is connected with a hidden layer unit h, whose connection coefficient c is the training data xtr(i), by a connection coefficient w=1. In the following description, the output layer unit o corresponding to the class t(i) to which xtr(i) belongs may be simply referred to as the output layer unit o corresponding to the class t(i). The learning control device 10 performs the processing from step S109 onwards when it determines that the output layer unit o corresponding to class t(i) exists in the probabilistic neural network PNN, and performs the processing from step S114 onwards when it determines that the output layer unit o corresponding to class t(i) does not exist in the probabilistic neural network PNN.

[0029] (Step S109) If there is no output layer unit o corresponding to class t(i), the learning control device 10 increments the count of variable k, which indicates the number of output layer unit o, by 1 (k←k+1), and adds one output layer unit o. Because the probabilistic neural network PNN in FIG. 2 has one output layer unit o (k=1), the learning control device 10 adds a second output layer unit o2 (k=2) to the probabilistic neural network PNN.

[0030] (Step S110) The learning control device 10 increments the count of the variable j, which indicates the number of the hidden layer unit h, by 1 (j←j+1), and adds one hidden layer unit h. Because the probabilistic neural network PNN in FIG. 2 has one hidden layer unit h (j=1), the learning control device 10 adds a second hidden layer unit h2 (j=2) to the probabilistic neural network PNN.

[0031] (Step S111) The learning control device 10 sets the coupling coefficient cj between each input layer unit x included in the probabilistic neural network PNN and the hidden layer unit h added in step S110 as xtr(i). Specifically, the coupling coefficient c2 between the second hidden layer unit h2 in Fig. 3 and each input layer unit x is set as xtr(2).

[0032] (Step S112) The learning control device 10 sets the coupling coefficient w between the output layer unit o added in step S109 and the hidden layer unit h added in step S110 to 1. In Fig. 3, the coupling coefficient w22 between the second hidden layer unit h2 and the second output layer unit o2 is 1. Note that the first hidden layer unit h1 and the second output layer unit o2, and the second hidden layer unit h2 and the first output layer unit o1 are not coupled (coupling coefficient w12 = 0, coupling coefficient w21 = 0).

[0033] That is, when there is no output layer unit o corresponding to class t(i), the learning control device 10 provides the output layer unit o corresponding to class t(i) to the probabilistic neural network PNN by adding an output layer unit o and an intermediate layer unit h that is connected to the output layer unit o with a connection coefficient w=1.

[0034] (Step S113) The learning control device 10 updates the number of classes q to a value corresponding to the training data set Xtr according to the class number setting method. For example, in the case of learning a general classification task, the learning control device 10 may set the number of classes q to be the same as the number of classes included in the training data set Xtr. In the case of continuous training, the learning control device 10 may also set the number of classes q to be increased by the number of unknown classes that have increased due to continuous training.

[0035] (Step S114) If an output layer unit o corresponding to class t(i) exists, the learning control device 10 sets (calculates) a radius value σ for each hidden layer unit h independently in accordance with a radius value setting method selected by a user operation. The radius value σ is the radius value included in the formula of the radial basis function applied to each hidden layer unit h. The output value of the hidden layer unit h is determined according to the radius value σ of the hidden layer unit h.

[0036] (Step S115) The learning control device 10 calculates the output value of each output layer unit o for the learning data xtr(i). Specifically, the learning control device 10 inputs the learning data xtr(i) to the probabilistic neural network PNN, and calculates the output value of the hidden layer unit h and the output value of the output layer unit o.

[0037] (Step S116) The learning control device 10 determines that the classification result is valid when, among the output layer units o included in the probabilistic neural network PNN, the output layer unit o with the largest output value (hereinafter referred to as the maximum output layer unit omax) corresponds to the class t(i) to which the training data xtr(i) belongs. Specifically, the learning control device 10 determines that the probabilistic neural network PNN has performed valid classification for the training data xtr(i) when the output value of the output layer unit o added corresponding to the class t(i) to which the training data xtr(i) belongs is maximum when the training data xtr(i) is input and the output value of the output layer unit o other than the output layer unit o added corresponding to the class t(i) to which the training data xtr(i) belongs is maximum when the learning data xtr(i) is input and the output value of the output layer unit o other than the output layer unit o added corresponding to the class t(i) to which the training data xtr(i) belongs is maximum, the learning control device 10 determines that the probabilistic neural network PNN has not performed valid classification for the training data xtr(i).

[0038] As described above, the learning control device 10 determines whether the probabilistic neural network PNN has properly classified the learning data, i.e., the validity of the classification, based on the calculated output values. If the learning control device 10 determines that the classification result is valid, it performs the processes from step S117 onwards, and if it determines that the classification result is invalid, it performs the processes from step S118 onwards.

[0039] [Learning stage - updating the connection coefficient c] (Step S117) If it is determined that the classification result is valid, the learning control device 10 updates the coupling coefficient c according to the coupling coefficient update method.

[0040] The learning control device 10 adds an output layer unit o and a hidden layer unit h connected to the output layer unit o only when there is no output layer unit o corresponding to class t(i), thereby enabling the probabilistic neural network PNN to perform appropriate class classification while preventing the configuration from becoming unnecessarily large. Furthermore, when there is no output layer unit o corresponding to class t(i), instead of modifying the existing hidden layer unit h, a new hidden layer unit connected to the output layer unit o is provided, thereby enabling learning of unknown classes while retaining information learned so far. This allows the learning control device 10 of this embodiment to solve the problem of catastrophic forgetting during continuous learning.

[0041] [Learning Phase - Second Update of Configuration] If the classification result is determined to be invalid, the learning control device 10 updates the configuration of the probabilistic neural network PNN. FIG. 4 is a diagram for explaining a second example of updating the configuration of the probabilistic neural network PNN. FIG. 4 shows the configuration of the probabilistic neural network PNN updated after FIG. 3. In FIG. 4, i=3. Steps S118 to S120 will be described with reference to the diagram.

[0042] (Step S118) If it is determined that the classification result is invalid, the count of the variable j, which indicates the number of the hidden layer unit h, is incremented by 1 (j←j+1), and one hidden layer unit h is added. Because the probabilistic neural network PNN in FIG. 3 already has two hidden layer units h (j=2), the learning control device 10 adds a third hidden layer unit h3 (j=3) to the probabilistic neural network PNN. Note that because the probabilistic neural network PNN already has an output layer unit o corresponding to class t(i), the learning control device 10 does not add an output layer unit o.

[0043] (Step S119) The learning control device 10 sets the coupling coefficient cj between each input layer unit x included in the probabilistic neural network PNN and the hidden layer unit h added in step S118 as xtr(i). In Fig. 4, the coupling coefficient c3 between the third hidden layer unit h3 and each input layer unit x is set as xtr(3).

[0044] (Step S120) The learning control device 10 sets the coupling coefficient w between the hidden layer unit h added in step S118 and the output layer unit o corresponding to the class t(i) to which the learning data xtr(i) belongs to as 1. In FIG. 4, if the output layer unit o corresponding to the class t(i) is the second output layer unit o2, the third hidden layer unit h3 and the second output layer unit o2 are coupled with a coupling coefficient w32 = 1. The third hidden layer unit h3 and the first output layer unit o1 are not coupled (coupling coefficient w31 = 0).

[0045] That is, when there is no output layer unit o corresponding to class t(i), the learning control device 10 adds a hidden layer unit h with the learning data xtr(i) as the coupling coefficient c. As a result, when learning data xtr(i) belonging to class t(i) is input, the learning control device 10 increases the output value of the output layer unit o corresponding to class t(i) and updates the probabilistic neural network PNN to enable appropriate classification. Note that the second example of configuration updating shown in FIG. 4 differs from the first example of configuration updating shown in FIG. 3 in that no output layer unit o is added. By not adding an unnecessary output layer unit o in the second example, the learning control device 10 can update the probabilistic neural network PNN to enable appropriate judgment while preventing the addition of unnecessary (excessive) configurations to the probabilistic neural network PNN.

[0046] (Step S121) After performing the processing of step S113, step S117, or step S120, the learning control device 10 determines whether learning has been performed for all the training data xtr. If learning has not been performed for all the training data xtr (i!=Ntr), the learning control device 10 repeats the processing from step S107. If learning has been performed for all the training data xtr (i==Ntr), the learning control device 10 terminates learning. Thus, by providing the training data set Xtr and selecting the method for setting the number of classes, the method for setting the radius value, and the method for updating the coupling coefficients, the learning control device 10 can construct a probabilistic neural network PNN capable of any pattern recognition (classification task). Therefore, the learning control device 10 according to this embodiment can automatically construct an optimal probabilistic neural network PNN without the user having to adjust hyperparameters or determine the number of units.

[0047] According to the learning control device 10 of this embodiment, a probabilistic neural network PNN capable of class classification can be automatically constructed by selecting an equation based on user operation and inputting learning data. The learning control device 10 of this embodiment does not require repeated learning or prior adjustment of hyperparameters.

[0048] The learning control device 10 does not add a unit if the constructed probabilistic neural network PNN can properly classify the training data xtr, and adds a unit only if the constructed probabilistic neural network PNN cannot properly classify the training data xtr. This makes it possible to construct a probabilistic neural network PNN that can classify all training data xtr, while reducing the number of units and the amount of calculation. In other words, the learning control device 10 according to this embodiment can construct a probabilistic neural network PNN in which the number of units and coupling coefficients are appropriately set.

[0049] Next, the method for setting the number of classes, the method for setting the radius value, and the method for setting the coupling coefficient will be described in detail with reference to equations (1) to (5).

[0050] [Class number setting method] There are two main methods for setting the number of classes: class number setting method 1 and class number setting method 2. The method for setting the number of classes to be used is selected based on the user's operation. The user is the person who uses the learning control device 10 or the inference device 20 described below.

[0051] In class number setting method 1, the initial number of classes q is 1. The number of classes q in class number setting method 1 is updated as "q ← q + [number of classes according to the input training data set Xtr]". Class number setting method 1 may be used when the number of classes to be classified in inference is undetermined, for example, when the number of classes to be classified may change through continued training. The number of classes according to the input training data set Xtr is, for example, the number of unknown classes that are difficult to classify using the probabilistic neural network PNN before the number of classes q is updated, i.e., the number of classes that has increased through continued training, etc.

[0052] Specifically, if the first learning had two classes, dog and cat, and the second learning had classes dog, cat, and bird, the number of classes q is updated to 3 so that the added "bird" class can also be classified. Also, if the first learning had two classes, dog and cat, and the second learning had classes cat and bird, the number of classes q is updated to 3 so that the added "bird" class can also be classified, as above.

[0053] In contrast, in number of classes setting method 2, the initial number of classes q is Nc. Nc is the number of classes included in the training dataset Xtr. The number of classes q in number of classes setting method 2 is not updated. Number of classes setting method 2 may be used when the number of classes to be classified in inference has been determined. Number of classes setting method 2 may also be used for general classification tasks that do not require continuous learning.

[0054] Note that the learning control device 10 according to this embodiment is not limited to using only one of class number setting method 1 and class number setting method 2 when setting the number of classes q. After performing learning using the number of classes q set by class setting method 2, the learning control device 10 may set the number of classes q for continuous learning using class number setting method 1.

[0055] [Radius value setting method] In the radius value setting method, the radius value σ is calculated by the following equation (1). In equation (1), "x" represents data sequentially input to the hidden layer unit h. Specifically, "x" represents the training data xtr used in the learning stage and the test data xts used in the inference stage. In equation (1), "c" represents the coupling coefficient between the hidden layer unit h for which the radius value σ is set using the radius value setting method and each input layer unit x. For the j-th hidden layer unit hj, "c" represents the coupling coefficient cj. In equation (1), "q" represents the number of classes set or updated using the class number setting method. Since "x" changes depending on the input data, the radius value σ set by the radius value setting method also changes depending on the input data. This allows the learning control device 10 to automatically set the radius value σ, the only hyperparameter of the probabilistic neural network PNN, to an appropriate value for each input data.

[0056]

number

[0057] Next, three types of formulas, radius value setting method 1, radius value setting method 2, and radius value setting method 3, which are examples of specific calculation methods for radius value setting methods, will be described. Which radius value setting method to use is selected based on a user's operation. Note that radius value setting method 1, radius value setting method 2, and radius value setting method 3 are merely examples, and the radius value setting methods according to this embodiment are not limited to these examples.

[0058] In radius value setting method 1, the radius value σ is calculated by equation (2). Radius value setting method 1 can be selected when number of classes setting method 1 is selected. "j" in equation (2) is a value indicating the number of hidden layer unit h.

[0059]

number

[0060] In radius value setting method 2, the radius value σ is calculated by equation (3). Radius value setting method 2 can be selected when number of classes setting method 1 is selected. "j" in equation (3) is a value indicating the number of hidden layer unit h. Also, "k" in equation (3) is a value indicating the number of output layer unit o.

[0061]

number

[0062] In radius value setting method 3, the radius value σ is calculated by equation (4). Radius value setting method 3 can be selected when number of classes setting method 2 is selected. Radius value setting method 3 is an equation obtained by applying radius value setting method 1 to the case where the number of classes q is fixed (number of classes setting method 2). "m" and "n" in equation (4) are different values ​​and are integers in the range from 1 to Ntr.

[0063]

number

[0064] [Coupling coefficient update method] There are two main types of coupling coefficient update methods: coupling coefficient update method 1 and coupling coefficient update method 2. The coupling coefficient update method to be used is selected based on user operation. Coupling coefficient update method 1 does not update the coupling coefficient c. This allows the learning control device 10 to maintain the coupling coefficient c of the probabilistic neural network PNN that was able to properly classify the classes. In contrast, coupling coefficient update method 2 adjusts the coupling coefficient c using the coupling coefficient c before updating and the data input to the hidden layer unit h. This allows the learning control device 10 to update the coupling coefficient of the probabilistic neural network PNN to match the input data.

[0065] Coupling coefficient update method 2 sets the value calculated by equation (5) as the new coupling coefficient c. The hidden layer unit h (hereinafter sometimes referred to as the maximum hidden layer unit hmax) to be updated in coupling coefficient update method 2 is the hidden layer unit h whose output value, i.e., the radial basis function value, is maximized. Coupling coefficient update method 2 updates the coupling coefficient c (hereinafter referred to as coupling coefficient cmax) between the maximum hidden layer unit hmax and each input layer unit x as the average of the coupling coefficient cmax and the input training data xtr(i). In this way, the learning control device 10 can improve generalization performance by having the probabilistic neural network PNN absorb part of the training data xtr(i).

[0066]

number

[0067] As described above, by using the class t as a teacher signal, the learning control device 10 can automatically determine the structure of the probabilistic neural network PNN, i.e., the number of units in each of the input, hidden, and output layers, in accordance with the input training data xtr. Furthermore, because the automatically constructed probabilistic neural network PNN does not include unnecessary hidden layer units h, the number of units and the amount of calculation can be significantly reduced compared to when the number of hidden layer units is determined in advance by the user.

[0068] Furthermore, by allowing the user to select one of the radius value settings in advance, the learning control device 10 eliminates the need to adjust the radius value σ, which is the only hyperparameter, and can construct an appropriate probabilistic neural network PNN according to the training data xtr. Furthermore, because the learning control device 10 ends learning when all of the training data xtr has been input, there is no need for repeated learning or for the user to adjust the hyperparameters each time, and therefore the enormous amount of calculation required for MLP training is not required.

[0069] [Unlearn] The above describes an example in which the learning control device 10 constructs a probabilistic neural network PNN suitable for the training data set Xtr by adding hidden layer units h and output layer units o and updating the connection coefficients. This embodiment is not limited to this example, and for example, the learning control device 10 may cancel the learning of any class from the constructed probabilistic neural network PNN. Canceling a class means deleting (resetting) the learning content of a specific class after learning multiple classes.

[0070] The MLP model, a hierarchical perceptron-type neural network, stores learned information in a distributed manner in the connection coefficients between the input layer and hidden layer, between hidden layers, and between hidden layers and the output layer (distributed representation), making it difficult to separate classes.

[0071] In contrast, the probabilistic neural network PNN separately holds (locally represents) each piece of input training data xtr as a connection coefficient c between each hidden layer unit h and input layer unit x, and has an output layer unit o for each class t. Therefore, the probabilistic neural network PNN can remove a class by deleting from its configuration the hidden layer unit h that has the training data xtr belonging to the class t to be removed as its connection coefficient c, and the output layer unit o corresponding to class t.

[0072] Class removal is broadly classified into main class removal and subclass removal. A main class is each class t input as training data to a probabilistic neural network (PNN). For example, removing the main class means deleting the learning content about birds from a probabilistic neural network (PNN) that performs pattern recognition for dogs, cats, and birds, and updating it to a probabilistic neural network (PNN) that performs pattern recognition for dogs and cats.

[0073] A subclass is a class defined by further subdividing class t. Canceling a subclass means, for example, deleting the learning content about Shiba Inu from a probabilistic neural network PNN that has been trained about dogs using training data xtr of Shiba Inu, Tosa Inu, and Chihuahua, and updating the probabilistic neural network PNN to one that determines Tosa Inu and Chihuahua as dogs.

[0074] The method for removing the main class will be described with reference to FIG. 5. FIG. 5 is a diagram for explaining the equivalence of probabilistic neural networks (PNNs). FIG. 5(A) shows the arrangement of each unit of the constructed probabilistic neural network (PNN). In the probabilistic neural network (PNN), a hidden layer unit h corresponding to a certain class t is connected to an output layer unit o corresponding to the class t with a connection coefficient w=1, and is connected to an output layer unit o not corresponding to the class t with a connection coefficient w=0. In other words, a hidden layer unit h corresponding to a certain class t is connected only to an output layer unit o corresponding to the class t. Therefore, by rearranging the arrangement of the hidden layer units h, each of the hidden layer units h and output layer units o included in the probabilistic neural network can be regarded as a collection of multiple sub-networks. FIG. 5(B) shows an example of FIG. 5(A) represented as a collection of multiple sub-networks. A sub-network is a collection of one or more hidden layer units h that have learned only on training data xtr belonging to a specific class t, and one output layer unit o. In removing the main class, the subnetwork containing the hidden layer unit h and output layer unit o corresponding to the class t to be removed (hereinafter, sometimes referred to as the subnetwork corresponding to class t) and the connection between the hidden layer unit h and input layer unit x included in the subnetwork are removed from the configuration of the probabilistic neural network PNN.

[0075] Next, we will explain how to remove a subclass. Removing a subclass differs from removing a main class in that the output layer unit o is not deleted because the main class remains. In removing a subclass, the hidden layer unit h, which has the learning data xtr belonging to the subclass as its connection coefficient c, the connection between the hidden layer unit h and each input layer unit x, and the connection between the hidden layer unit h and the output layer unit o are deleted from the configuration of the probabilistic neural network PNN.

[0076] In conventional PNNs, when the configuration is updated due to the release of classes, it is necessary to set the radius value of the hyperparameters to match the configuration after the release of classes. If a new radius value is not set, estimation is performed using the radius value set in the configuration before the release of classes, which can result in a decrease in classification accuracy. In contrast, the probabilistic neural network PNN according to this embodiment sets a radius value according to the input data each time data is input, making it possible to prevent a decrease in classification accuracy before and after the release of classes.

[0077] [Inference stage] Next, the inference device 20 used in the inference (test) stage will be described with reference to Fig. 6. Fig. 6 is a flowchart for explaining an example of the processing flow of the inference device 20 according to the embodiment. The probabilistic neural network PNN used in the inference stage may be constructed by the learning control device 10, or may be one whose number of units is determined by another device or a user.

[0078] A test data set Xts={xts(1), xts(2), ..., xts(Nts)} is input to the inference device 20. xts indicates each piece of test (recognition) data. The numbers in parentheses identify each piece of test data xts. That is, the test data set Xts stores a total of Nts pieces of test data, from test data xts(1) to test data xts(Nts).

[0079] (Step S201) The inference device 20 acquires the number of classes q defined for the probabilistic neural network PNN and the coupling coefficients c of each hidden layer unit h. The number of classes q defined for the probabilistic neural network PNN may be, for example, the number of classes q that can be inferred using the hidden layer units h and output layer units o included in the probabilistic neural network PNN.

[0080] (Step S202) Inference device 20 calculates and determines the radius value σ for each hidden layer unit h in accordance with the radius value setting method selected based on the user's operation. Note that the radius value setting method used in the inference stage and the radius value setting method used in the learning stage may be the same formula, or may be different formulas. If the radius value setting method used in the inference stage and the radius value setting method used in the learning stage are the same formula, the user does not need to select a radius value setting method.

[0081] (Step S203) Inference device 20 calculates the output value of each output layer unit o for test data xts(i). Specifically, inference device 20 calculates the output value (radial basis function value) of each hidden layer unit h based on the coupling coefficient c and the number of classes q of each hidden layer unit h acquired in step S201 and the radius value σ set in step S202. In addition, inference device 20 calculates the output value of each output layer unit o based on the output value of each hidden layer unit h.

[0082] (Step S204) Inference device 20 identifies the maximum output layer unit omax with the largest output value based on the output value of each output layer unit o. Inference device 20 outputs the number of classes q corresponding to the identified maximum output layer unit omax as the pattern recognition result.

[0083] A typical PNN uses the radius value used during training (construction) in the inference stage. In contrast, the inference device 20 according to this embodiment sets a radius value for each piece of test data xts during inference. This allows the inference device 20 to set an appropriate radius value σ for each piece of test data xts.

[0084] [Results of verification experiment] Next, with reference to FIGS. 7 to 11, the results of a verification experiment using the learning control device 10 and the inference device 20 according to this embodiment will be described.

[0085] Figure 7 shows the training datasets used in the validation experiments. Figure 7 shows the number of training data sets, the number of test data sets, the number of classes, and the feature values ​​per data set for eight benchmark datasets for pattern recognition and MNIST. These training datasets can be used for a variety of purposes, including image and speech recognition.

[0086] The verification experiments were conducted on three types of pattern classification tasks: normal learning, continuous class addition learning, and continuous data addition learning. For the method according to this embodiment, class number setting method 2 was used for the normal learning task, and class number setting method 1 was used for the continuous class addition learning and continuous data addition learning. Furthermore, for the method according to this embodiment, coupling coefficient update method 2 was used for all tasks.

[0087] Regular training is a so-called standard pattern recognition task in which all training data is available in advance, i.e., no additional training data is generated later.

[0088] Continuous class addition training is a pattern recognition task in which classes are added and training is repeated. For example, in continuous class addition training, a pattern recognizer that performs two-class classification is first constructed using only training data for classes 1 and 2. Then, a pattern recognizer that performs three-class classification is reconstructed through additional training using only data for class 3. Then, a pattern recognizer that performs four-class classification is reconstructed using only data for class 4... and so on, repeatedly adding classes. In continuous class addition training, it is said that the classification accuracy of MLPs drops during the process of adding classes due to the problem of catastrophic forgetting.

[0089] Continuous data training is the task of reconstructing a pattern recognizer each time to ensure proper class classification in an environment where training data is constantly obtained and the classes to which they belong are not uniform. Continuous data training is relatively easier than continuous class training because training data for all classes required for classification is provided at a certain point in time.

[0090] Comparative experiments were also conducted using the dataset shown in Fig. 7 for a general PNN, an MLP, and a learning method (Deep Neural Network: DNN) designed for continuous data addition learning. The general PNN (hereinafter sometimes referred to as PNN (original)) and the method according to this embodiment were verified when radius value setting method 1 and radius value setting method 2 were used.

[0091] FIG. 8 shows the results of a verification experiment in normal learning. As shown in FIG. 8, in the verification experiment in normal learning, classification accuracy was compared when a PNN (original) was used and when a DNN (two-layered intermediate layer) was used. Note that FIG. 8 only shows the case where radius value setting method 1 was used for the method according to this embodiment. It was found that the method according to this embodiment had slightly lower classification accuracy than the PNN (original), but could significantly reduce the number of intermediate layer units h. Furthermore, the method according to this embodiment had slightly lower classification accuracy than the DNN, and the number of intermediate layer units h was increased. However, there was no significant difference in classification accuracy.

[0092] FIG. 9 shows the results of a verification experiment in continuous class addition learning. FIG. 9 shows the results of two types of radius value setting methods for the PNN (original) and the method according to this embodiment. The "number of units" of the method according to this embodiment shown in FIG. 9 refers to the number of hidden layer units h automatically constructed by the first and second update examples described above. It was confirmed from FIG. 9 that the method according to this embodiment can significantly reduce the number of hidden layer units h while maintaining a certain level of classification accuracy compared to the PNN (original). Note that DNNs are not used for comparison because it is difficult to perform continuous class addition learning without undergoing transfer learning from a trained model.

[0093] FIG. 10 shows the results of a verification experiment in continuous data addition learning using radius setting method 1. FIG. 11 shows the results of a verification experiment in continuous data addition learning using radius setting method 2. In FIGS. 10 and 11, a DL method (DNN_iCaRL) designed for continuous data addition learning was used as one of the comparison targets. It was confirmed that the method according to this embodiment can significantly reduce the number of hidden layer units h without significantly changing the classification accuracy compared to the PNN (original), regardless of whether radius setting method 1 or radius setting method 2 is used. It was also confirmed that the method according to this embodiment can achieve higher classification accuracy than the existing DL method (DNN_iCaRL) in six out of nine datasets.

[0094] [Embodiment 2] FIG. 12 is a flowchart illustrating an example of the processing flow of the learning stage in the second embodiment. The learning (construction) in the second embodiment differs from the learning in the first embodiment in that the processing of step S301 is further performed after the processing of step S120. In the learning in the first embodiment, the probabilistic neural network PNN calculates and updates the radius value σ regardless of whether or not the learning data xtr(i) has been properly classified. In contrast, in the learning in the second embodiment, the radius value σ is updated (newly added step S301) only when it is determined that proper classification has not been performed (step S116; No). The probabilistic neural network PNN updates the radius value σ only when the classification result is not proper, thereby preventing the radius value σ from being changed when the classification result is proper, and thus maintaining the classification accuracy.

[0095] (Step S301) After step S120, i.e., after adding the hidden layer unit h and setting the coupling coefficient c and the coupling coefficient w, the learning control device 10 updates the numerator value in the radius value setting method formula (hereinafter, sometimes simply referred to as the radius value numerator) in accordance with the radius value numerator setting method.

[0096] When the radius numerator setting method is also applied to the inference stage, the radius value of the hidden layer unit h in the probabilistic neural network PNN remains the radius value set in the learning stage. By not changing the radius value in the inference stage, the probabilistic neural network PNN can use the inference method for conventional PNNs as is.

[0097] [Radius value numerator setting method] First, a specific formula for the radius value setting method used in the learning stage in embodiment 2 will be described. In the radius value setting method (formulas (2) to (4)) in embodiment 1, the numerator value differs for each piece of learning data xtr, and therefore the calculated radius value σ also differs for each piece of learning data xtr. In the radius value setting method in embodiment 2, the numerator data is set by a radius value numerator setting method described later, and therefore the same radius value σ is calculated unless the numerator value is updated.

[0098] The radius value σ in the radius value setting method 1 in the second embodiment is calculated by the following equation (6): dmax, which is the numerator in the equation (6), is a value calculated by the radius value numerator setting method 1.

[0099]

number

[0100] The radius value σ in the radius value setting method 2 in the second embodiment is calculated by the following equation (7): dmin,max, which is the numerator in the equation (7), is a value calculated by the radius value numerator setting method 2.

[0101]

number

[0102] There are two main types of radius value numerator setting methods: radius value numerator setting method 1 and radius value numerator setting method 2. Radius value numerator setting method 1 is used when radius value setting method 1 is selected, and radius value numerator setting method 2 is used when radius value setting method is selected. Which radius value numerator setting method (or radius value setting method) to use is selected based on the user's settings.

[0103] Radius value numerator setting method 1 calculates dmax using equation (8). Equation (8) is the same as the numerator of equation (2) in radius value setting method 1 in the first embodiment.

[0104]

number

[0105] The radius value numerator setting method 2 calculates dmin,max by equation (9). Equation (9) is the same as the numerator of equation (3) in the radius value setting method 2 in the first embodiment.

[0106]

number

[0107] [Function Configuration] 13 is a block diagram illustrating an example of the functional configuration of a learning control device 10 according to an embodiment. The learning control device 10 includes an acquisition unit 11, a selection unit 12, a configuration update unit 13, an output calculation unit 14, a determination unit 15, and an update unit 16. The learning control device 10 learns (constructs) a probabilistic neural network PNN that performs class classification based on a learning dataset Xtr.

[0108] The acquiring unit 11 acquires a training data set Xtr. Furthermore, the acquiring unit 11 acquires the number of classes q indicating the number of types of classes t included in the training data xtr, based on the training data set Xtr.

[0109] The selection unit 12 selects one of the multiple formulas for each of the class number setting method, the coupling coefficient determination method, the radius value setting method, and the radius value numerator setting method based on a user's selection operation. The user's selection operation refers to, for example, the user operating an information processing device such as a smartphone, tablet, or computer to select an arbitrary formula.

[0110] When the class t(i) to which the training data xtr(i) belongs does not exist among the classes t of the output layer unit o of the probabilistic neural network PNN, the configuration update unit 13 adds the output layer unit o corresponding to the class t(i) to the probabilistic neural network PNN. The configuration update unit 13 also adds an intermediate layer unit h connected to the added output layer unit o with a connection coefficient w=1 to the probabilistic neural network PNN, and sets the connection coefficient c as the input training data xtr(i).

[0111] If the determination unit 15 determines that the classification result is invalid, the configuration update unit 13 adds an intermediate layer unit h to the probabilistic neural network PNN. Furthermore, the configuration update unit 13 sets the coupling coefficient c between the added intermediate layer unit h and each input layer unit x as training data xtr(i). Furthermore, the configuration update unit 13 sets the coupling coefficient w between the added intermediate layer unit h and the output layer unit o corresponding to class t(i) as w=1. If the classification result is determined to be valid, the configuration update unit 13 does not add a unit to the probabilistic neural network PNN.

[0112] Furthermore, the configuration update unit 13 acquires information indicating the class to be released based on a user operation. If the class to be released is a main class, the configuration update unit 13 updates the configuration of the probabilistic neural network PNN to delete a subnetwork corresponding to the class t to be released and the connections between the hidden layer units h of the subnetwork and each input layer unit x. That is, the configuration update unit 13 updates the configuration of the probabilistic neural network PNN to delete the hidden layer unit h in the subnetwork, the output layer unit o in the subnetwork, the connection between the hidden layer unit h and the output layer unit o, and the connection between the hidden layer unit h and each input layer unit x (the connection to the hidden layer unit h). On the other hand, if the class to be released is a subclass, the configuration update unit 13 updates the configuration of the probabilistic neural network PNN to delete one or more hidden layer units h having the learning data xtr belonging to the subclass to be released as connection coefficients c, and the connection to the hidden layer unit h. The configuration update unit 13 may identify the learning data xtr belonging to the subclass to be released and update the configuration by the user inputting information indicating the learning data xtr belonging to the subclass (for example, a number indicating the order of the learning data xtr).

[0113] The output calculation unit 14 acquires the value of the number of classes q set based on the class number setting method selected by the user. The output calculation unit 14 also acquires the coupling coefficients c of each hidden layer unit h included in the probabilistic neural network PNN constructed by the configuration update unit 13. The output calculation unit 14 also acquires the learning data xtr input to the learning control device 10. For each hidden layer unit h, the output calculation unit 14 calculates a radius value σ from the learning data xtr, the coupling coefficients c, and the number of classes q using the radius value setting method selected based on a user operation. The output calculation unit 14 also calculates radial basis function values ​​as output values ​​of each hidden layer unit h using the calculated radius value σ. The output calculation unit 14 also calculates the output values ​​of the output layer units o connected to the hidden layer unit h based on the output values ​​of each hidden layer unit h.

[0114] The determination unit 15 acquires the output value of each output layer unit o calculated by the output calculation unit 14. The determination unit 15 identifies the class t of the output layer unit o that has the largest output value among the acquired output values. The class t of the output layer unit o is the class t to which the training data xtr that caused the output layer unit o to be added belongs. The determination unit 15 determines that the classification result is valid when the class t of the training data xtr and the class t of the output layer unit o are the same. Furthermore, the determination unit 15 determines that the classification result is invalid when the class t of the training data xtr and the class t of the output layer unit o are not the same.

[0115] If the determination unit 15 determines that the classification result is valid, the update unit 16 updates the connection coefficients c between each input layer unit x and each hidden layer unit h based on the connection coefficient update method selected by the user. Note that the update unit 16 does not necessarily need to update the connection coefficients c.

[0116] 14 is a block diagram illustrating an example of the functional configuration of an inference device 20 according to an embodiment. The inference device 20 includes an acquisition unit 21, a selection unit 22, an output calculation unit 23, and an inference result output unit 24. The inference device 20 performs an inference operation on a class t to which test data xts belongs.

[0117] Each functional unit of the learning control device 10 and the inference device 20 is realized, for example, using a computer including a CPU (Central Processing Unit) and memory, and software. Each functional unit may also be realized using electronic circuits as needed. Furthermore, each functional unit does not have to be included in a single device, and the learning control device 10 and the inference device 20 may be configured from multiple devices.

[0118] The acquisition unit 21 acquires a test data set Xts, i.e., data to be inferred and unknown to the probabilistic neural network PNN. The acquisition unit 21 also acquires the number of types of classes t that can be classified by the probabilistic neural network PNN trained by the learning control device 10, and the coupling coefficients c between the input layer units x and the hidden layer units h. The number of types of classes t that can be classified by the probabilistic neural network PNN may be, for example, the number of output layer units o included in the probabilistic neural network PNN.

[0119] The selection unit 22 selects one of the multiple radius value setting method formulas based on a selection operation by the user. The selection unit 22 may reuse the formula selected in the learning stage, or may select a new formula based on a selection operation by the user in the inference stage.

[0120] The output calculation unit 23 acquires the value of the number of classes q and the coupling coefficients c of each hidden layer unit h included in the probabilistic neural network PNN from the acquisition unit 21. The output calculation unit 23 also acquires test data xts to be input to the inference device 20. For each hidden layer unit h, the output calculation unit 23 calculates a radius value σ from the test data xts, the coupling coefficients c, and the number of classes q using a radius value setting method selected based on a user operation. The output calculation unit 23 also calculates radial basis function values ​​as output values ​​of each hidden layer unit h using the calculated radius value σ. Furthermore, the output calculation unit 23 calculates the output values ​​of the output layer units o coupled to the hidden layer unit h based on the output values ​​of each hidden layer unit h.

[0121] The inference result output unit 24 acquires the output value of each output layer unit o calculated by the output calculation unit 23. The determination unit 15 identifies the class t of the output layer unit o that has the largest output value among the acquired output values. The inference result output unit 24 outputs the identified class t as the class t into which the test data xts has been classified by the inference device 20. The inference result output unit 24 may output the result to an information processing device, or may output the result to a device (not shown) that performs predetermined processing based on the pattern recognition result.

[0122] The learning control device 10 and the inference device 20 according to the embodiments can be used for various classification tasks. For example, the learning control device 10 and the inference device 20 may be used for a task of classifying obstacles and humans in autonomous driving, or a task of determining the ripeness of different apple varieties.

[0123] FIG. 15 is a block diagram showing an example of the internal configuration of the learning control device 10 and the inference device 20 according to an embodiment. At least some of the functions of the learning control device 10 and the inference device 20 can be implemented using a computer. As shown in the figure, the computer includes a central processing unit 901, a RAM 902, an input / output port 903, input / output devices 904 and 905, and a bus 906. The computer itself can be implemented using existing technology. The central processing unit 901 executes instructions contained in a program read from the RAM 902 or the like. In accordance with each instruction, the central processing unit 901 writes data to the RAM 902, reads data from the RAM 902, and performs arithmetic and logical operations. The RAM 902 stores data and programs. Each element included in the RAM 902 has an address and can be accessed using the address. RAM is an abbreviation for "random access memory." The input / output port 903 is a port through which the central processing unit 901 exchanges data with external input / output devices. The input / output devices 904 and 905 are input / output devices. The input / output devices 904 and 905 exchange data with the central processing unit 901 via the input / output port 903. The bus 906 is a common communication path used within the computer. For example, the central processing unit 901 reads and writes data from and to the RAM 902 via the bus 906. Also, for example, the central processing unit 901 accesses the input / output port via the bus 906. Furthermore, all or part of the functional units provided in the learning control device 10 and the inference device 20 may be realized using hardware such as an ASIC, a PLD, or an FPGA. Furthermore, all or part of the functional units may be realized by a combination of software and hardware.

[0124] Note that all or part of the functions of each unit of the learning control device 10 and the inference device 20 in the above-described embodiments may be realized by recording a program for realizing these functions on a computer-readable recording medium, and then loading and executing the program recorded on the recording medium into a computer system. Note that the term "computer system" here includes hardware such as an OS and peripheral devices.

[0125] Although one embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design modifications can be made without departing from the spirit of the present invention. Furthermore, the configurations described in the above-described embodiments and examples can be combined. [Explanation of symbols]

[0126] 10... learning control device, 11... acquisition unit, 12... selection unit, 13... configuration update unit, 14... output calculation unit, 15... judgment unit, 16... update unit, 20... inference device, 21... acquisition unit, 22... selection unit, 23... output calculation unit, 24... inference result output unit, x... input layer unit, h... hidden layer unit, o... output layer unit, t... class, q... number of classes, σ... radius value, Xtr... learning dataset, xtr... learning data, Xts... test dataset, xts... test data,

Claims

1. During the supervised learning phase of a probabilistic neural network that classifies given data, an acquisition unit that acquires a class number q indicating the number of types of classes in the training data; a selection unit that selects, based on a user's selection operation, an equation to be used in calculating an output value from among a plurality of equations for calculating a radius value σ, which is a hyperparameter, based on the training data xtr, the number of classes q, and a coupling coefficient c between the input layer and the intermediate layer; a configuration update unit that updates the configuration of connections between the intermediate layer units and the output layer units based on the class of the training data and the class type of the output layer units; an output calculation unit that calculates output values ​​of the intermediate layer units and the output layer units using the selected equations; a determination unit that determines the validity of the classification result of the given learning data based on the calculated output value; A learning control device comprising:

2. an update unit that updates a coupling coefficient c between the input layer and the intermediate layer when the classification result is determined to be valid as a result of the determination by the determination unit; The learning control device according to claim 1 , further comprising:

3. the configuration update unit further removes at least one of an intermediate layer unit and an output layer unit corresponding to the class to be removed, selected based on a user operation, and connections to the unit from the configuration of the probabilistic neural network; The learning control device according to claim 1 or 2.

4. In the inference stage, a trained probabilistic neural network is used. an acquisition unit that acquires the number of classes q in the intermediate layer of the probabilistic neural network and the coupling coefficient c between the input layer and the intermediate layer; a selection unit that selects, based on a selection operation by a user, an equation to be used in an inference calculation from among a plurality of equations for calculating a radius value σ, which is a hyperparameter, based on the data xts to be inferred, the acquired number of classes q, and the acquired coupling coefficient c; an output calculation unit that calculates output values ​​of the intermediate layer units and the output layer units using the selected equations; An inference device comprising:

Images