Neural network system, learning control device, calculation method, learning control method and program

Abstract

To allow for determining data for use as auxiliary data so as to improve inference accuracy of a neural network.SOLUTION: A neural network system comprises: a manifest data acquisition unit which acquires manifest data which are acquired during both learning and operation; a latent data acquisition unit which acquires latent data which are acquired only during learning; a basic model processing unit which inputs only the manifest data to a basic model to perform operations; an additional model processing unit which trains an additional model so that an operation result from operations using both the manifest data and the latent data and an operation result from operations using only the manifest data are equal to each other during learning, and inputs only the manifest data to the additional model to perform operations during operation; an integration processing unit which calculates an operation result obtained by integrating an operation result obtained by using the basic model and an operation result obtained by using the additional model; and a learning result determination unit which determines whether or not the additional model should be employed on the basis of the operation result of the integration processing unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a neural network system, a learning control device, a calculation method, a learning control method, and a program. [Background technology]

[0002] Patent Document 1 discloses a machine learning method that uses, as training data, main data obtained during both training and operation, and auxiliary data obtained only during training among training and operation. In the machine learning method described in Patent Document 1, training is performed so that the output of a partial network of a neural network is the same when both main data and auxiliary data are used and when only the main data of the main data and auxiliary data is used. It is expected that this training will improve the estimation accuracy of the neural network during operation without requiring auxiliary data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 031305 Summary of the Invention [Problem to be solved by the invention]

[0004] In learning using main data and auxiliary data, it may be unclear what kind of data should be used as auxiliary data to improve the estimation accuracy of the neural network. Even in such a case, it is preferable to be able to determine the data to be used as auxiliary data so that the estimation accuracy of the neural network is improved.

[0005] One example of an object of the present invention is to provide a neural network system, a learning control device, a calculation method, a learning control method, and a program that can determine data to be used as auxiliary data so as to improve the estimation accuracy by a neural network. [Means for solving the problem]

[0006] According to a first aspect of the present invention, a neural network system comprises a manifest data acquisition unit that acquires manifest data, which is data acquired during both learning and operation; a latent data acquisition unit that acquires latent data, which is data acquired only during learning of the learning and operation; a basic model processing unit that inputs only the manifest data of the manifest data and the latent data into a basic model, which is a calculation model, to perform a calculation; an additional model processing unit that learns an additional model, which is a calculation model, so that the calculation results are the same when a calculation is performed using both the manifest data and the latent data during the learning, and when a calculation is performed using only the manifest data of the manifest data and the latent data, and that, during the operation, inputs only the manifest data of the manifest data and the latent data into the additional model to perform a calculation; an integration processing unit that calculates a calculation result that integrates the calculation result using the basic model and the calculation result using the additional model; and a learning result determination unit that decides whether to adopt the additional model based on the calculation result by the integration processing unit.

[0007] The apparatus may further include a switcher unit that sets a non-penetration rate, which is a coefficient by which the latent data output from the latent data acquisition unit to the additional model processing unit is multiplied, so that the influence of the latent data on learning gradually decreases.

[0008] The model configuration control unit further includes a model configuration control unit. Group The model configuration control unit acquires the latent data of For each combination of the groups of latent data, input of latent data included in the combination is received from among additional models provided for each group of latent data. The learning result determination unit causes the additional model to be learned, determining, based on the estimation accuracy based on the calculation result of the integration processing unit for each combination of the groups of the latent data, to adopt any combination of additional models corresponding to the combination of the groups of the latent data, and the additional model processing unit deletes the additional models that were not adopted; This may be done.

[0009] According to a second aspect of the present invention, a learning control device controls learning of a neural network calculation device, the learning control device comprising: a manifest data acquisition unit that acquires manifest data, which is data acquired both during learning and during operation; a latent data acquisition unit that acquires latent data, which is data acquired only during learning among the learning and operation; a basic model processing unit that inputs only the manifest data of the manifest data and the latent data into a basic model, which is a calculation model, and performs a calculation; an additional model processing unit that inputs only the manifest data of the manifest data and the latent data into an additional model, which is a calculation model, during the operation, and performs a calculation; and an integration processing unit that calculates a calculation result that integrates a calculation result by the basic model and a calculation result by the additional model, the learning control device comprising: a connection weight adjustment unit that, during the learning, causes the additional model processing unit to learn the additional model so that the calculation result is the same when performing a calculation using both the manifest data and the latent data and when performing a calculation using only the manifest data of the manifest data and the latent data; The case where the additional model is added is more than the case where the additional model is not added. The integration processing unit of Operation result Determine whether the estimation accuracy has improved a learning result determination unit that determines whether or not to adopt the additional model; a model configuration control unit that causes the additional model processing unit to delete the additional model when it is determined not to adopt the additional model; Equipped with.

[0010] According to a third aspect of the present invention, a calculation method includes: The computeracquiring explicit data which is data acquired during both learning and operation; acquiring latent data which is data acquired only during learning of the learning and operation; inputting only the explicit data of the explicit data and the latent data into a basic model which is a calculation model and performing a calculation during both the learning and operation; learning an additional model which is a calculation model so that the calculation result is the same when performing a calculation using both the explicit data and the latent data during the learning and when performing a calculation using only the explicit data of the explicit data and the latent data; inputting only the explicit data of the explicit data and the latent data into the additional model during the operation and performing a calculation; calculating a calculation result which integrates the calculation result from the basic model and the calculation result from the additional model; The case where the additional model is added is more than the case where the additional model is not added. A calculation result obtained by integrating the calculation result based on the basic model and the calculation result based on the additional model. Determine whether the estimation accuracy has improved determining whether to employ the additional model; and If it is decided not to adopt the additional model, deleting the additional model; and Includes.

[0011] According to a fourth aspect of the present invention, there is provided a learning control method for controlling learning of a neural network calculation device comprising: a manifest data acquisition unit for acquiring manifest data, which is data acquired both during learning and during operation; a latent data acquisition unit for acquiring latent data, which is data acquired only during learning among the learning and operation; a basic model processing unit for inputting only the manifest data of the manifest data and the latent data into a basic model, which is a calculation model, and performing a calculation; an additional model processing unit for inputting only the manifest data of the manifest data and the latent data into an additional model, which is a calculation model, during the operation, and performing a calculation; and an integration processing unit for calculating a calculation result that integrates a calculation result by the basic model and a calculation result by the additional model, wherein, during the learning, the additional model processing unit is caused to learn the additional model so that the calculation result is the same when a calculation is performed using both the manifest data and the latent data and when a calculation is performed using only the manifest data of the manifest data and the latent data; The case where the additional model is added is more than the case where the additional model is not added. The integration processing unit of Operation result Determine whether the estimation accuracy has improved determining whether to employ the additional model; and When it is determined not to adopt the additional model, causing the additional model processing unit to delete the additional model; Includes.

[0012] According to a fifth aspect of the present invention, a program includes the following steps in a computer: acquire explicit data, which is data acquired both during learning and during operation; acquire latent data, which is data acquired only during learning of the learning and operation; input only the explicit data of the explicit data and the latent data into a basic model, which is a calculation model, during both the learning and operation; learn an additional model, which is a calculation model, so that the calculation result is the same when performing a calculation using both the explicit data and the latent data during the learning and when performing a calculation using only the explicit data of the explicit data and the latent data; input only the explicit data of the explicit data and the latent data into the additional model during the operation, and calculate a calculation result that integrates the calculation result from the basic model and the calculation result from the additional model; The case where the additional model is added is more than the case where the additional model is not added. A calculation result obtained by integrating the calculation result based on the basic model and the calculation result based on the additional model. Determine whether the estimation accuracy has improved determining whether to employ the additional model; and If it is decided not to adopt the additional model, deleting the additional model; and This is a program for executing the above.

[0013] According to a sixth aspect of the present invention, a program provides a computer for controlling learning of a neural network calculation device, the computer comprising: a manifest data acquisition unit that acquires manifest data, which is data acquired both during learning and during operation; a latent data acquisition unit that acquires latent data, which is data acquired only during learning of the learning and operation; a basic model processing unit that inputs only the manifest data of the manifest data and the latent data into a basic model, which is a calculation model, and performs a calculation; an additional model processing unit that inputs only the manifest data of the manifest data and the latent data into an additional model, which is a calculation model, during the operation, and performs a calculation result that integrates a calculation result by the basic model and a calculation result by the additional model; the program provides a computer for controlling learning of a neural network calculation device, the computer comprising: a manifest data acquisition unit that acquires manifest data, which is data acquired both during learning and during operation; a latent data acquisition unit that acquires latent data, which is data acquired only during learning of the learning and operation The case where the additional model is added is more than the case where the additional model is not added. The integration processing unit of Operation result Determine whether the estimation accuracy has improved determining whether to employ the additional model; and When it is determined not to adopt the additional model, causing the additional model processing unit to delete the additional model; This is a program for executing the above. Effect of the Invention

[0014] According to the above-mentioned neural network system, learning control device, calculation method, learning control method and program, it is expected that data to be used as auxiliary data can be determined so as to improve the estimation accuracy by the neural network. [Brief description of the drawings]

[0015] [Figure 1] 1 is a schematic block diagram showing a configuration example of a neural network system according to an embodiment. [Diagram 2] FIG. 1 is a diagram illustrating an example of a configuration of a device in a neural network system according to an embodiment. [Diagram 3]FIG. 2 is a diagram illustrating an example of a neural network that is configured by a neural network calculation unit according to the embodiment during pre-learning. [Figure 4] 13 is a diagram showing an example of a neural network configured by a neural network calculation unit when the non-penetration rate value is 0 in the embodiment. FIG. [Diagram 5] FIG. 2 is a diagram showing a first example of a neural network that a neural network calculation unit configures during pre-learning when there are multiple groups of latent data in an embodiment. [Figure 6] 4 is a flowchart showing a first example of a processing procedure in which the neural network system according to the embodiment performs neural network learning. [Figure 7] FIG. 13 is a diagram showing a second example of a neural network that the neural network calculation unit configures during pre-learning when there are multiple groups of latent data in an embodiment. [Figure 8] 11 is a flowchart showing a second example of a processing procedure for the neural network system according to the embodiment to learn the neural network. [Figure 9] FIG. 1 is a schematic block diagram illustrating a configuration of a computer according to at least one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] The following describes embodiments of the present invention, but the following embodiments do not limit the scope of the invention. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. Fig. 1 is a schematic block diagram showing a configuration example of a neural network system according to an embodiment. In the configuration shown in Fig. 1, the neural network system 1 includes a neural network calculation unit 100 and a learning control unit 200. The neural network calculation unit 100 includes a data acquisition unit 110, an explicit data acquisition unit 111, an implicit data acquisition unit 112, a basic model processing unit 121, an additional model processing unit 122, and an integration processing unit 130. The learning control unit 200 includes a permeation data storage unit 210, a model configuration control unit 221, a connection weight adjustment unit 222, a switcher unit 223, and a learning result determination unit 224.

[0017] The neural network system 1 performs machine learning of a neural network and performs calculations using the trained neural network. Machine learning is also simply called learning. The neural network system 1 adjusts the weights (connection loads) between the nodes of the neural network through learning. As the learning algorithm of the neural network system 1, a general learning algorithm such as the Error Backpropagation Method can be used. The neural network configured by the neural network calculation unit 100, or a partial network thereof, is also referred to as a calculation model or a model. The entire neural network configured by the neural network calculation unit 100 is also referred to as an entire model.

[0018] The learning control section 200 controls the neural network calculation section 100. In particular, the learning control section 200 controls the construction of a neural network by the neural network calculation section 100 and the learning of the neural network. The neural network calculation unit 100 and the learning control unit 200 may be configured as one device. For example, one computer such as a personal computer (PC) may execute both the functions of the neural network calculation unit 100 and the learning control unit 200. Alternatively, one device dedicated to the neural network system 1, such as a device using a field-programmable gate array (FPGA) or an application specific integrated circuit (ASIC), may execute both the functions of the neural network calculation unit 100 and the learning control unit 200.

[0019] Alternatively, the neural network calculation section 100 and the learning control section 200 may be configured as separate devices. Fig. 2 is a diagram showing an example of the configuration of devices in the neural network system 1. In the configuration shown in Fig. 2, the neural network system 1 includes a neural network calculation device 101 and a learning control device 201. The neural network calculation device 101 executes the function of the neural network calculation section 100. The learning control device 201 executes the function of the learning control section 200.

[0020] The neural network calculation device 101 may be configured using one computer such as a personal computer, or may be configured using one device dedicated to the neural network calculation device 101, such as a device using an FPGA or an ASIC. The learning control device 201 may be configured using one computer such as a personal computer, or may be configured using one device dedicated to the learning control device 201, such as a device using an FPGA or an ASIC.

[0021] The types of data that the neural network calculation unit 100 obtains during operation are limited to a part of the types of data that are obtained during learning. An example of the type of data that is obtained during operation being limited to a part of the types of data that are obtained during learning is when only a part of the devices that can be used during learning can be used during operation.

[0022] For example, when processing is performed using an existing sensor, a sensor may be temporarily added during learning, learning may be performed, and the temporarily added sensor may be removed after learning is completed. Furthermore, for example, when a state of a certain device is judged using a sensor installed in the device, a sensor may be temporarily added in addition to the existing sensor during learning of the state judgment model. In this case, in order to avoid an increase in the cost of the device or because it is difficult to permanently install the additional sensor due to the structure of the device, the temporarily added sensor may be removed after the learning is completed.

[0023] In addition, in the case of a time series prediction problem, when future data in addition to past data is input to the neural network calculation unit 100 during learning, this is also an example of a case where the types of data obtained during operation are limited to a portion of the types of data obtained during learning. During learning, if a certain point in time in the past is set as the reference time, future data from the reference time as well as past data from the reference time can be input to the neural network calculation unit 100 as learning data. On the other hand, during operation, the current time (the time when the neural network calculation unit 100 is made to perform calculations) may be the reference time. In this case, future data cannot be input to the neural network calculation unit 100.

[0024] Hereinafter, data of a type that is obtained during both learning and operation will be referred to as explicit data, and data of a type that is obtained only during learning will be referred to as implicit data. In order to deal with the fact that the types of data obtained during operation are limited to a portion of the types of data obtained during learning, the neural network system 1 has a mechanism for adjusting the influence of latent data on learning. Specifically, the neural network system 1 adjusts the influence of latent data on learning by multiplying the latent data by a coefficient. The coefficient in this case is called non-permeability.

[0025] The neural network system 1 adjusts the influence of latent data on learning so that the calculation results obtained by the additional model corresponding to the partial network of the overall model are the same when calculations are performed using both manifest and latent data and when calculations are performed using only the manifest data of the manifest and latent data.

[0026] The computation result of the neural network referred to here is the output of the neural network. Here, learning to make the calculation results the same does not necessarily mean that the calculation results are completely the same, but may include that the calculation results are approximately the same. For example, the neural network system 1 may repeat learning until the magnitude of the difference between the calculation results obtained by the additional model when performing calculations using both the visible data and the latent data and when performing calculations using only the visible data out of the visible data and the latent data becomes equal to or smaller than a predetermined threshold.

[0027] When using latent data during learning, it may be unclear what kind of data should be used as latent data to improve the estimation accuracy of the neural network. The estimation accuracy of the neural network here refers to the accuracy of processing according to the purpose of the neural network. For example, when classifying data using a neural network, the accuracy of classifying data is an example of the accuracy of processing.

[0028] For example, in the case of performing processes such as judgment or prediction using existing sensors as described above, in which sensors are temporarily added during learning, it may not be clear what kind of sensors should be installed and where in order to improve estimation accuracy. In addition, in the above-mentioned example of learning time series prediction using future data in addition to past data, it may not be clear what kind of future data should be used to improve the prediction accuracy. For example, in the case of stock price prediction, where various factors are intricately related, it may not be clear what kind of data should be used as latent data to improve the prediction accuracy.

[0029] On the other hand, if data that has no correlation with the target data is added to the training data, the data may act as noise, reducing the estimation accuracy. Therefore, after learning using the latent data, the neural network system 1 determines whether or not the latent data contributes to improving the estimation accuracy.

[0030] For example, the neural network system 1 may determine whether the estimation accuracy after learning of a neural network to which an additional model has been added is improved compared to the estimation accuracy of a neural network to which an additional model has not been added. When it is determined that the estimation accuracy is improved when the additional model is added compared to when the additional model is not added, the neural network system 1 may adopt the neural network to which the additional model is added.

[0031] On the other hand, if it is determined that the estimation accuracy is not improved when the additional model is added compared to when the additional model is not added, the neural network system 1 may return to the neural network without adding the additional model. Furthermore, if there is other data that is a candidate for latent data, the neural network system 1 may similarly determine whether or not to use the candidate as latent data.

[0032] The neural network system 1 learns in two stages so that when it receives input of only explicit data out of explicit and latent data, it outputs the same result as when it receives input of both explicit and latent data. In the first stage of learning, the neural network system 1 learns to calculate the correct answer indicated by the learning data by using both the explicit data and the implicit data. The permeation data storage unit 210 stores the value of the permeation data when the first stage of learning has been sufficiently performed, such as at the end of the first stage of learning. The first stage of learning is called pretraining.

[0033] In the second stage of learning, the additional model configured by the additional model processing unit 122 is the target of learning. The neural network system 1 performs learning to reduce the influence of the latent data on the learning and to calculate the same calculation result as in the case of the additional model pre-learning. For example, the neural network system 1 performs learning of the additional model while gradually decreasing the value of the non-penetration rate multiplied by the latent data from 1 to 0, so that the value of the permeation data becomes the same value as the value stored in the permeation data storage unit 210. The second stage of learning is called percolative learning.

[0034] The data acquisition unit 110 functions as an input layer in the overall model and acquires input data. The manifest data acquisition unit 111 acquires manifest data both during learning and operation, and outputs the acquired manifest data to the basic model processing unit 121 and the additional model processing unit 122. The latent data acquisition unit 112 acquires latent data only during learning, out of during learning and operation, and outputs the acquired latent data to the additional model processing unit 122.

[0035] The basic model processing unit 121, the additional model processing unit 122, and the integration processing unit 130 all constitute a neural network. These neural networks all correspond to partial networks of the overall model. The combination of the basic model processing unit 121, the additional model processing unit 122, and the integration processing unit 130 constitutes the hidden layer (intermediate layer) and the output layer of the overall model.

[0036] The basic model processing unit 121 receives the input of the manifest data, performs calculations using a neural network, and outputs the calculation results to the integration processing unit 130. The calculation results output by the basic model processing unit 121 correspond to intermediate data in the overall model. During learning, the additional model processing unit 122 performs learning so that the output is the same when the input of explicit data and latent data is received as that when the input of explicit data and latent data is received as well as that when the input of explicit data and latent data is received. As a result, the neural network calculation unit 100 performs learning so that the output is the same for the entire model when the input of explicit data and latent data is received as well as that when the input of explicit data and latent data is received as well.

[0037] As described above, the term "learning so that the outputs (calculation results)" referred to here are the same does not necessarily mean that the outputs are completely the same, but may also mean that the outputs are approximately the same. The neural network configured by the additional model processing unit 122 is referred to as an additional model, and the output of the additional model is referred to as percolative data or percolative feature. The additional model processing unit 122 may configure a plurality of additional models. The integration processing unit 130 receives the calculation results of the basic model processing unit 121 and the calculation results of the additional model processing unit 122, performs neural network calculations, and outputs the calculation results. The output of the integration processing unit 130 corresponds to the output (calculation result) of the neural network calculation unit 100.

[0038] The permeation data storage unit 210 stores the permeation data. In particular, the permeation data storage unit 210 stores the permeation data when both the manifest data and the latent data are input to the additional model processing unit 122. The permeation data stored in the permeation data storage unit 210 is referenced in order to make the permeation data when only the manifest data of the manifest data and the latent data is input to the additional model processing unit 122 the same as the permeation data when both the manifest data and the latent data are input to the additional model processing unit 122.

[0039] The model configuration control unit 221 controls the configuration of the model in the additional model processing unit 122. Specifically, the model configuration control unit 221 causes the additional model processing unit 122 to provide an additional model in accordance with the latent data from the latent data acquisition unit 112. Furthermore, when it is determined as a result of learning that the additional model does not improve the estimation accuracy, the model configuration control unit 221 causes the additional model processing unit 122 to delete the additional model. The connection weight adjustment unit 222 controls the learning of the neural network operation unit 100. Specifically, the connection weight adjustment unit 222 controls the neural network operation unit 100 to update the connection weights between the nodes of the neural network.

[0040] The switcher unit 223 multiplies the latent data output by the latent data acquisition unit 112 by the non-penetration rate. As described above, the non-penetration rate is a coefficient for adjusting the degree of influence of latent data on learning. The switcher unit 223 can gradually reduce the degree of influence of the latent data on learning by multiplying the latent data by the non-penetration rate. Here, "gradually" means that there is at least one intermediate state. This is expected to reduce the effect of changes in the presence or absence of latent data in learning, and to enable highly accurate learning. Specifically, it is possible to avoid a situation in which the search space in learning additional models (optimization of connection weights) suddenly changes and the results of pre-learning are not fully utilized in permeative learning.

[0041] Alternatively, when switching from pre-learning to permeation learning, the switcher unit 223 may immediately change the value of the non-penetration rate from 1 to 0. In other words, the switcher unit 223 may switch the value of the non-penetration rate between 1 and 0. This allows the neural network calculation unit 100 to quickly perform learning using only the explicit data out of the explicit and implicit data, which is expected to shorten the learning time for permeative learning.

[0042] When multiple latent data are input to one additional model, the switcher unit 223 may be able to change the value of the non-penetration rate for each latent data. After switching from pre-learning to permeation learning, the switcher unit 223 may set the non-penetration rate so as to gradually decrease the number of latent data input to the additional model.

[0043] Specifically, during pre-learning, the switcher unit 223 sets the non-penetration rate value to 1 for all latent data input to the additional model. During permeation learning, the switcher unit 223 changes the non-penetration rate value from 1 to 0 for only some of the latent data among the latent data input to the additional model. The switcher unit 223 changes all non-penetration rate values ​​to 0 by changing the non-penetration rate multiple times.

[0044] As a result, the switcher unit 223 gradually thins out the latent data to be input to the additional model during pre-learning. By gradually thinning out the latent data in this manner, the switcher unit 223 can avoid a situation in which the search space in learning the additional model suddenly changes and the results of pre-learning are not fully utilized in the permeation learning, as in the case of gradually decreasing the value of the non-penetration rate.

[0045] When the influence of the latent data on learning is reduced, the additional model outputs the same value of permeation data as in pre-learning, so that the neural network calculation unit 100 as a whole is expected to output the same calculation result as in pre-learning using both the manifest data and the latent data. In particular, even when the influence of the latent data becomes 0 in permeation learning, the neural network calculation unit 100 is expected to output the same calculation result as in pre-learning. In this way, through pre-learning and permeation learning, when the neural network calculation unit 100 receives only explicit data out of the explicit and implicit data, it is expected to output the same calculation result as when it receives both explicit and implicit data.

[0046] The learning result determination unit 224 determines whether to adopt an additional model based on the calculation result by the integration processing unit 130. Specifically, the learning result determination unit 224 determines whether the estimation accuracy is improved when there is an additional model compared to when there is no additional model. When it is determined that the estimation accuracy is improved, the learning result determination unit 224 decides to adopt the additional model. On the other hand, if it is determined that the estimation accuracy has not improved, the learning result determination unit 224 decides not to adopt the additional model. In this case, the model configuration control unit 221 controls the additional model processing unit 122 to delete the additional model.

[0047] FIG. 3 is a diagram showing an example of a neural network that the neural network calculation unit 100 configures during pre-learning. The above overall model (the entire neural network configured by the neural network calculation section 100) is denoted as an overall model NT. In the example of Fig. 3, the global model NT includes an input layer NT, a basic model NB, an additional model NA-1, and an integrated circuit NC. The input layer IN includes an explicit data input layer EXP and an implicit data input layer IMP.

[0048] The "1" in additional model NA-1 is an identification number that identifies additional model NA-1. When the overall model includes multiple additional models, they are represented as additional models NA-(i), where (i) is an integer such that 1≦(i)≦NAMAX. NAMAX is an integer such that NAMAX≧1, and indicates the number of additional models included in the overall model NT.

[0049] The input layer IN corresponds to the input layer of the global model NT as a neural network, and accepts input to the global model NT. Specifically, the explicit data input layer EXP accepts input of explicit data, and outputs the input explicit data to the basic model NB and the additional model NA. The data acquisition unit 110 functions as the input layer IN. The explicit data acquisition unit 111 functions as the explicit data input layer EXP. The implicit data acquisition unit 112 functions as the implicit data input layer IMP.

[0050] The input (input data) to the overall model NT and the output (output data) of the overall model NT are represented by variables. Specifically, the manifest variable vector x is expressed as x = (x1, x2, , x xmax ) and the variables x1, x2, . . . , x as input parameters. xmax Each of the parameters takes a value of explicit data. xmax is an integer xmax ≥ 1, and indicates the number of input parameters of the global model NT that take a value of explicit data. The type of the manifest data is not limited to a specific type. Therefore, the type of the elements of the manifest variable vector is not limited to a specific type. For example, the type of the elements of the manifest variable vector may be any of integer, real, or binary. The manifest variable vector may contain elements of different types.

[0051] In addition, the latent variable vector a1 is expressed as a1 = (a11, a12, , a1 a1max ) and variables a11, a12, . . ., a1 as input parameters. a1maxEach of the following takes a value of latent data. a1max is an integer equal to or larger than 1, and indicates the number of input parameters of the total model NT that take the value of latent data input to the additional model NA-1.

[0052] When the overall model includes multiple additional models, the input parameters that take the values ​​of the latent data input to the additional model NA-(i) are defined as the latent variable vector a(i) = (a(i)1,a(i)2,...,a(i) a(i)max a(i)max is an integer a(i)max ≧1, and indicates the number of input parameters of the total model NT that take on the value of the latent data input to the additional model NA-(i). The type of latent data is not limited to a specific type. Therefore, the type of the elements of the latent variable vector is not limited to a specific type. For example, the type of the elements of the latent variable vector may be any of integer, real, or binary. The latent variable vector may contain elements of different types.

[0053] Also, the output variable vector y is expressed as y=(y1,y2,...,y ymax ), where y1, y2, . . . , y ymax Each of the above takes the value of output data. ymax is an integer ymax ≥ 1, and indicates the number of output data of the whole model NT. The type of the output data is not limited to a specific type. Therefore, the type of the elements of the output variable vector is not limited to a specific type. For example, the type of the elements of the output variable vector may be any of integer, real, or binary. The output variable vector may contain elements of different types.

[0054] The basic model NB, the additional model NA1, and the integrated circuit NC are all configured using neural networks. The basic model NB, the additional model NA1, and the integrated circuit NC can be configured as a known neural network or a partial network obtained by removing the input layer from a known neural network. The neural network constituting any of the basic model NB, the additional model NA1, and the integrated circuit NC is not limited to a specific one. The number of nodes included in any of the basic model NB, the additional model NA1, and the integrated circuit NC is not limited to a specific number. The number of layers when any of the basic model NB, the additional model NA1, and the integrated circuit NC is constituted as a hierarchical neural network is also not limited to a specific number.

[0055] 3, the additional model NA-1 and the integrated circuit NC are the targets of pre-learning. Furthermore, the basic model NB may also be the target of pre-learning. The basic model NB receives manifest data as input and performs calculations both during learning and during operation. The values ​​of the internal parameters (connection weights) of the basic model NB may be fixed values. Alternatively, the basic model NB may be the target of learning during pre-learning. The basic model processing unit 121 configures the basic model NB.

[0056] The additional model NA-1 receives input of explicit data and implicit data during pre-learning. For this purpose, the switcher unit 223 sets the non-penetration rate value to 1. When there are multiple latent data, the latent data may be divided into multiple groups, and the additional model processing unit 122 may construct an additional model for each group of latent data. In this case, the learning result determination unit 224 determines for each additional model whether or not to adopt the additional model. The number of latent data included in each group may be one or more. The additional model processing unit 122 configures the additional model NA-1. When multiple additional models are configured, the additional model processing unit 122 also configures the multiple additional models.

[0057] The integrated circuit NC receives the calculation results of the basic model NB and the calculation results of the additional model NA-1, and outputs the calculation results. When there are multiple additional models, the integrated circuit NC receives the calculation results of the basic model NB and the calculation results of each additional model NA-i, and outputs the calculation results. Here, i is an integer such that 1≦i≦NAMAX. The integrated processing unit 130 constitutes an integrated circuit NC. In the pre-learning, the neural network calculation section 100 performs learning under the control of the connection weight adjustment section 222 so that the calculation result of the integrated circuit NC becomes the same value as the correct answer shown in the learning data.

[0058] FIG. 4 is a diagram showing an example of a neural network configured by the neural network calculation unit 100 when the non-penetration rate value is 0. As shown in FIG. In the example of FIG. 4, since no latent data is input to the additional model NA-1, the latent variable vector a1 and the latent input data layer IMP shown in FIG. 3 are not shown. However, the learning data for the permeative learning may include latent data. In particular, the same learning data as that used in the pre-learning can be used as the learning data for the permeative learning, and in that case, there is no need to delete the permeative data from the learning data. There is also no need to delete the latent data acquisition unit 112 in FIG. 1. The fact that no latent data is input to the additional model NA-1 is achieved, for example, by setting the value of the non-permeation rate to 0.

[0059] As described above, in the permeation learning, the switcher unit 223 may gradually decrease the value of the non-penetration rate to 0. Then, the connection weight adjustment unit 222 may control the additional model processing unit 122 to learn the additional model NA-1 every time the switcher unit 223 updates the non-penetration rate. In this case, the connection weight adjustment unit 222 uses the same learning data as the learning data in the case of pre-learning, and causes the additional model processing unit 122 to learn the additional model NA-1 so that the value of the permeation data becomes the same value as the value of the permeation data in the pre-learning stored in the permeation data storage unit 210.

[0060] After the permeation learning is completed, the learning result determination unit 224 determines whether or not to adopt the additional model NA-1 based on the calculation result by the integrated circuit NC. Specifically, the learning result determination unit 224 determines whether or not the estimation accuracy of the overall model is improved when the additional model NA-1 is present compared to when the additional model NA-1 is not present. When it is determined that the estimation accuracy is improved, the learning result determination unit 224 decides to adopt the additional model NA-1. On the other hand, if it is determined that the estimation accuracy has not improved, the learning result determination unit 224 decides not to adopt the additional model NA-1. In this case, the model configuration control unit 221 controls the additional model processing unit 122 to delete the additional model NA-1.

[0061] When there are multiple groups of latent data, the neural network system 1 may generate and learn an additional model corresponding to each group sequentially, and determine whether or not to adopt the additional model. FIG. 5 is a diagram showing a first example of a neural network that the neural network calculation unit 100 configures during pre-learning when there are multiple groups of latent data.

[0062] In the example of Fig. 5, the additional model processing unit 122 constructs an additional model NA-1 for a group of latent data indicated by the latent variable vector a1 in Fig. 3, and after learning, the learning result determination unit 224 decides to adopt the additional model NA-1. The additional model processing unit 122 further constructs an additional model NA-2 for a group of latent data indicated by the latent variable vector a2 in Fig. 5.

[0063] In the pre-training of the additional model NA-2, the additional model NA-2 and the integrated circuit NC are the targets of training. On the other hand, in the pre-training of the additional model NA-2, the basic model NB and the additional model NA-1 are not the targets of training. The neural network calculation unit 100 inputs the manifest data to each of the basic model NB and the additional model NA-1, and inputs the calculation results of each model to the integrated circuit NC. The neural network calculation unit 100 also inputs the manifest data and the latent data represented by the latent variable vector a2 to the additional model NA-2, and inputs the calculation results of the additional model NA-2 to the integrated circuit NC. Then, the neural network calculation unit 100 performs learning of the basic model NB and the additional model NA-1 so that the value of the output data of the integrated circuit NC becomes the same value as the correct value of the learning data. The permeation data storage unit 210 stores the permeation data output by the additional model NA-2.

[0064] In the permeation learning of the additional model NA-2, the additional model NA-2 is the target of learning. On the other hand, in the pre-learning of the additional model NA-2, the basic model NB and the additional model NA-1 are not the targets of learning. The additional model processing unit 122 of the neural network calculation unit 100 learns the additional model NA-2 so that the value of the permeation data output by the additional model NA-2 becomes the same as the value of the permeation data at the end of the pre-learning (the value of the permeation data stored in the permeation data storage unit 210). In this way, when learning a new additional model, additional models that have already been learned are not subject to learning.

[0065] After the permeation learning of the additional model NA-2 is completed, the learning result determination unit 224 determines whether or not to adopt the additional model NA-2 based on the calculation result of the integrated circuit NC. Specifically, the learning result determination unit 224 determines whether or not the estimation accuracy of the overall model is improved when the additional model NA-2 is present compared to when the additional model NA-2 is not present. If it is determined that the estimation accuracy is improved, the learning result determination unit 224 decides to adopt the additional model NA-2. On the other hand, if it is determined that the estimation accuracy has not improved, the learning result determination unit 224 decides not to adopt the additional model NA-2. In this case, the model configuration control unit 221 controls the additional model processing unit 122 to delete the additional model NA-2.

[0066] Fig. 6 is a flowchart showing a first example of a processing procedure for learning a neural network by the neural network system 1. Fig. 6 shows an example in which, when there are multiple groups of latent data, the neural network system 1 sequentially generates and learns an additional model corresponding to each group, and determines whether or not to adopt the additional model for each group.

[0067] In the process of FIG. 6, the neural network calculation unit 100 configures a computation model by a neural network under the control of the learning control unit 200 (step S111). For example, when the basic model NB and the integrated circuit NC are already configured, the additional model processing unit 122 generates one additional model under the control of the model configuration control unit 221. The additional model processing unit 122 sets the input to the generated additional model as manifest data (all manifest data indicated by the manifest variable vector x) and one group of latent data (a group that has not yet been input to the additional model). In addition, the neural network calculation unit 100 causes the output data of the generated additional model to be input to the integrated circuit NC. In the process of FIG. 6, the additional model generated by the additional model processing unit 122 in step S111 is also referred to as the additional model to be learned.

[0068] Next, the neural network calculation unit 100 performs pre-learning under the control of the learning control unit 200 (step S112). In the pre-learning, explicit data and implicit data are input, and learning is performed between the additional model to be learned and the integrated circuit NC so that the value of the output data of the integrated circuit NC becomes the same as the correct value indicated in the learning data. Then, the learning control unit 200 determines whether or not a condition for ending the pre-learning is satisfied (step S113). The condition for ending the pre-learning may be, but is not limited to, a condition that the estimation accuracy of the overall model is equal to or higher than a predetermined threshold.

[0069] If the learning control unit 200 determines that the end condition of the pre-learning is not satisfied (step S113: NO), the process returns to step S112. In this case, the neural network system 1 continues the pre-learning until the end condition is satisfied. On the other hand, if the learning control unit 200 determines that the condition for ending pre-learning is met (step S113: YES), the permeation data storage unit 210 stores the permeation data of the additional model to be learned under the control of the learning control unit 200 (step S121).

[0070] Next, the neural network calculation unit 100 performs permeation learning under the control of the learning control unit 200 (step S122). In the permeation learning, the additional model processing unit 122 performs learning of the additional model to be learned so that the value of the output data of the additional model to be learned is the same as the value of the permeation data stored in the permeation data storage unit 210. Here, the switcher unit 223 gradually reduces the influence of the latent data on the learning, for example by gradually reducing the value of the non-permeation rate. The additional model processing unit 122 performs learning of the additional model to be learned until a predetermined condition is satisfied every time the switcher unit 223 reduces the influence of the latent data on the learning. The predetermined condition here may be, but is not limited to, a condition that the magnitude of the difference between the value of the output data of the additional model to be learned and the value of the permeation data stored in the permeation data storage unit 210 is equal to or less than a predetermined threshold.

[0071] Then, the learning control unit 200 determines whether or not a condition for ending the permeation learning is satisfied (step S123). The condition for ending the permeation learning may be, but is not limited to, a condition that the value of the non-permeation rate is 0 and the magnitude of the difference between the value of the output data of the additional model to be learned and the value of the permeation data stored in the permeation data storage unit 210 is equal to or less than a predetermined threshold.

[0072] If the learning control unit 200 determines that the end condition of the permeative learning is not satisfied (step S123: NO), the process returns to step S122. In this case, the neural network system 1 continues the permeative learning until the end condition is satisfied. On the other hand, if the learning control unit 200 determines that the end condition of the permeation learning is satisfied (step S123: YES), the learning result determination unit 224 evaluates the learning result (step S131). For example, the learning result determination unit 224 determines whether the estimation accuracy of the overall model is higher when there is an additional model to be learned than when there is no additional model to be learned. To achieve this, the learning result determination unit 224 may calculate and store the estimation accuracy of the overall model before the latent data acquisition unit 112 generates the additional model to be learned in step S111.

[0073] Then, the learning result determination unit 224 determines whether to adopt the additional model to be learned based on the evaluation of the learning result (step S132). For example, when the learning result determination unit 224 determines that the estimation accuracy of the overall model is higher when there is an additional model to be learned than when there is no additional model to be learned, the learning result determination unit 224 determines to adopt the additional model to be learned. Alternatively, the learning control unit 200, instead of the learning result determination unit 224, may determine whether or not to adopt the additional model to be learned.

[0074] When the learning result determination unit 224 decides to adopt an additional model to be learned (step S132: YES), the learning control unit 200 determines whether there is any group of latent data that has not been subject to the processing of FIG. 6 (step S151). If the learning control unit 200 determines that there is a group of latent data that has not been subject to the processing in Fig. 6 (step S151: YES), the process returns to step S111. In this case, the neural network system 1 performs the processing in Fig. 6 on one of the groups of latent data that has not been subject to the processing in Fig. 6. On the other hand, if the learning control unit 200 determines that there is no group of latent data that has not been subjected to the processing in FIG. 6 (step S151: NO), the neural network system 1 ends the processing in FIG.

[0075] On the other hand, in step S132, if the learning result determination unit 224 decides not to adopt the additional model to be learned (step S132: NO), the learning control unit 200 controls the additional model processing unit 122 to delete the additional model to be learned (step S141). After step S141, the process proceeds to step S151.

[0076] When there are multiple groups of latent data, the neural network system 1 may generate and learn an additional model for each group included in each combination of groups, and decide whether or not to adopt that additional model. FIG. 7 is a diagram showing a second example of a neural network that the neural network calculation unit 100 configures during pre-learning when there are multiple groups of latent data.

[0077] 7, each of the amax latent variable vectors from a1 to aamax indicates a group of latent data, where amax is an integer greater than or equal to 1. The latent data for each of these amax groups is input to the latent data input layer IMP. The additional model processing unit 122 generates an additional model for each group. These additional models are denoted as additional model NA-1, . . . , additional model NA-amax.

[0078] The learning control unit 200 causes the neural network calculation unit 100 to perform pre-learning and permeation learning for each combination of additional models that receive input of latent data from the groups included in the combination, for all combinations of groups of latent data. The learning result determination unit 224 determines to adopt, from among all combinations of additional models, an additional model included in the combination that results in the highest estimation accuracy of the overall model. The model configuration control unit 221 controls the additional model processing unit 122 to delete the additional models that were not adopted.

[0079] Fig. 8 is a flowchart showing a second example of the processing procedure for learning the neural network by the neural network system 1. Fig. 8 shows an example in which, when there are multiple groups of latent data, the neural network system 1 selects, from among all combinations of groups, a combination that results in the highest estimation accuracy of the overall model.

[0080] In the process of Fig. 8, the model configuration control unit 221 controls the additional model processing unit 122 to generate an additional model for each group of latent data that receives the input of the latent data (step S211). The additional model processing unit 122 inputs the manifest data (all the manifest data indicated by the manifest variable vector x) in addition to the latent data to each of the generated additional models. The neural network calculation unit 100 also inputs the output data of each of the generated additional models to the integrated circuit NC.

[0081] Next, the neural network system 1 starts a loop L11 in which processing is performed for each combination of additional models (step S212). In the processing of Fig. 8, an additional model included in the combination being processed in the loop L11 is referred to as an additional model to be learned.

[0082] In the processing of loop L11, the neural network calculation unit 100 performs pre-learning under the control of the learning control unit 200 (step S221). In the pre-learning, explicit data and implicit data are input, and learning is performed between the additional model to be learned and the integrated circuit NC so that the value of the output data of the integrated circuit NC becomes the same as the correct value indicated in the learning data. For additional models other than the learning target, the additional model processing unit 122 temporarily sets the output to 0, for example, so as not to affect the output of the integrated circuit NC.

[0083] Then, the learning control unit 200 determines whether or not a condition for ending the pre-learning is satisfied (step S113). The condition for ending the pre-learning may be, but is not limited to, a condition that the estimation accuracy of the overall model is equal to or higher than a predetermined threshold.

[0084] If the learning control unit 200 determines that the end condition of the pre-learning is not satisfied (step S222: NO), the process returns to step S221. In this case, the neural network system 1 continues the pre-learning until the end condition is satisfied. On the other hand, if the learning control unit 200 determines that the condition for ending the pre-learning is met (step S222: YES), the permeation data storage unit 210 stores the permeation data of the additional model to be learned under the control of the learning control unit 200 (step S231).

[0085] Next, the neural network calculation unit 100 performs permeation learning under the control of the learning control unit 200 (step S232). In the permeation learning, the additional model processing unit 122 performs learning of the additional model to be learned so that the value of the output data of the additional model to be learned is the same as the value of the permeation data stored in the permeation data storage unit 210. Here, the switcher unit 223 gradually reduces the influence of the latent data on the learning, for example by gradually reducing the value of the non-permeation rate. The additional model processing unit 122 performs learning of the additional model to be learned until a predetermined condition is satisfied every time the switcher unit 223 reduces the influence of the latent data on the learning. The predetermined condition here may be, but is not limited to, a condition that the sum of the magnitude of the difference between the value of the output data of the additional model to be learned and the value of the permeation data stored in the permeation data storage unit 210 for all additional models to be learned is equal to or less than a predetermined threshold value.

[0086] Then, the learning control unit 200 judges whether or not a condition for ending the permeation learning is satisfied (step S233). The condition for ending the permeation learning may be, but is not limited to, a condition that the value of the non-permeation rate is 0, and the sum of the magnitudes of the differences between the values ​​of the output data of the additional models to be learned and the values ​​of the permeation data stored in the permeation data storage unit 210 for all the additional models to be learned is equal to or less than a predetermined threshold value.

[0087] If the learning control unit 200 determines that the end condition of the permeative learning is not satisfied (step S233: NO), the process returns to step S232. In this case, the neural network system 1 continues the permeative learning until the end condition is satisfied. On the other hand, if the learning control unit 200 determines that the end condition of the permeation learning is satisfied (step S233: YES), the learning result determination unit 224 evaluates the learning result and stores the evaluation (step S241). For example, the learning result determination unit 224 calculates and stores the estimation accuracy of the overall model when there is an additional model to be learned.

[0088] After step S241, the neural network system 1 performs a termination process of the loop L11 (step S242). Specifically, the neural network system 1 determines whether or not the process of the loop L11 has been performed for all combinations of the added models. If it is determined that there are unprocessed combinations, the process returns to step S221, and the neural network system 1 continues to process the loop L11 for the unprocessed combinations. On the other hand, if it is determined that there is no unprocessed combination, the neural network system 1 ends the processing of loop L11.

[0089] When the neural network system 1 finishes the process of loop L11 in step S242, the learning result determination unit 224 determines a configuration to be adopted as a configuration of an additional model (step S243). For example, the learning result determination unit 224 determines to adopt a combination of additional models that will maximize the estimation accuracy of the overall model based on the evaluation in step S241. Then, the model configuration control unit 221 deletes the additional models that were not adopted (step S244). That is, the model configuration control unit 221 deletes the additional models other than the additional models included in the combination adopted by the learning result determination unit 224. After step S244, the neural network system 1 ends the process of FIG.

[0090] As described above, the explicit data acquisition unit 111 acquires explicit data, which is data acquired during both learning and operation. The latent data acquisition unit 112 acquires latent data, which is data acquired only during learning among learning and operation. The basic model processing unit 121 inputs only the explicit data of the explicit data and latent data to the basic model, which is a calculation model, and performs a calculation. The additional model processing unit 122 performs learning of the additional model, which is a calculation model, so that the calculation results are the same when performing a calculation using both the explicit data and latent data during learning and when performing a calculation using only the explicit data of the explicit data and latent data, and performs a calculation by inputting only the explicit data of the explicit data and latent data to the additional model during operation. The integration processing unit 130 calculates a calculation result obtained by integrating the calculation result by the basic model and the calculation result by the additional model. The learning result determination unit 224 decides whether or not to adopt the additional model based on the calculation result by the integration processing unit 130.

[0091] According to the neural network system 1, learning can be performed so that the neural network (whole model) outputs the same value whether or not there is latent data, and further, based on the accuracy of the neural network when the latent data is used, it is possible to determine whether or not to adopt the latent data. Therefore, according to the neural network system 1, it is possible to prevent the latent data from acting as noise, which would result in a decrease in the estimation accuracy of the neural network. In this respect, according to the neural network system 1, even if it is not clear what kind of data should be used as latent data (auxiliary data) to improve the estimation accuracy of the neural network, it is possible to determine data to be used as latent data so as to improve the estimation accuracy.

[0092] Furthermore, the switcher unit 223 sets the non-penetration rate, which is a coefficient by which the latent data output from the latent data acquisition unit 112 to the additional model processing unit 122 is multiplied, so that the influence of the latent data on the learning gradually decreases. This makes it possible for the neural network system 1 to avoid a situation in which the search space in the learning of the additional model (optimization of the connection weights) changes suddenly and the results of the pre-learning are not fully utilized in the permeative learning. In this respect, the neural network system 1 makes it possible to perform neural network learning with high accuracy.

[0093] The latent data acquisition unit 112 acquires a plurality of latent data. The model configuration control unit 221 selects one or more latent data from the plurality of latent data, and causes the additional model processing unit 122 to provide an additional model for each selected latent data and to learn the additional model. The learning result determination unit 224 determines whether or not to adopt a combination of the additional models for each selected latent data. According to the neural network system 1, when there are multiple latent data, it is possible to determine which latent data or a combination of latent data to adopt so as to further improve the estimation accuracy of the neural network.

[0094] 9 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. In the configuration shown in FIG. 9, a computer 700 includes a CPU (Central Processing Unit) 710, a main memory device 720, an auxiliary memory device 730, and an interface 740.

[0095] Any one or more of the above-mentioned neural network system 1, neural network calculation device 101, and learning control device 201 may be implemented in a computer 700. In this case, the operation of each of the above-mentioned processing units is stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, loads it in the main storage device 720, and executes the above-mentioned processing according to the program. The CPU 710 also secures a storage area in the main storage device 720 for executing the above-mentioned processing according to the program.

[0096] When the neural network system 1 is implemented in a computer 700, the operations of the neural network calculation unit 100, the learning control unit 200, and each of these units are stored in the form of a program in an auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-mentioned processing according to the program. Furthermore, the CPU 710 allocates a memory area in the main memory 720 for executing processes, such as a memory area corresponding to the permeation data storage unit 210, in accordance with the program. Communication between the neural network system 1 and other devices is performed by the interface 740 having a communication function and performing communication under the control of the CPU 710. Interaction between the neural network system 1 and a user is performed by the interface 740 having an output device such as a display device and input devices such as a keyboard and a mouse, and operating under the control of the CPU 710.

[0097] When the neural network calculation device 101 is implemented in the computer 700, the operation of each part of the neural network calculation device 101 is stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it in the main storage device 720, and executes the above-mentioned processing according to the program. Furthermore, the CPU 710 allocates a storage area in the main storage device 720 for executing the processing of the neural network computing device 101 according to the program. Communication between the neural network calculation device 101 and other devices is performed by the interface 740 having a communication function and performing communication under the control of the CPU 710. Interaction between the neural network calculation device 101 and a user is performed by the interface 740 having an output device such as a display device and input devices such as a keyboard and a mouse, and operating under the control of the CPU 710.

[0098] When the learning control device 201 is implemented in the computer 700, the operation of each part of the learning control device 201 is stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-mentioned processing according to the program. Furthermore, the CPU 710 allocates a storage area in the main storage device 720 for executing the processing of the learning control device 201 according to the program. Communication between the learning control device 201 and other devices is performed by the interface 740 having a communication function and performing communication under the control of the CPU 710. Interaction between the learning control device 201 and a user is performed by the interface 740 having an output device such as a display device and input devices such as a keyboard and a mouse, and operating under the control of the CPU 710.

[0099] Note that the programs for implementing all or part of the functions of the neural network system 1, the neural network calculation device 101, and the learning control device 201 may be recorded on a computer-readable recording medium, and the programs recorded on the recording medium may be read into a computer system and executed to perform processing of each part. Note that the term "computer system" here includes hardware such as an OS (Operating System) and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs (Read Only Memory), and CD-ROMs (Compact Disc Read Only Memory), as well as storage devices such as hard disks built into computer systems. The above-mentioned program may be for realizing part of the above-mentioned functions, or may be capable of realizing the above-mentioned functions in combination with a program already recorded in the computer system.

[0100] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design modifications and the like that do not deviate from the gist of the present invention are also included. [Explanation of symbols]

[0101] 1. Neural network system 100 Neural network calculation unit 101 Neural network computing device 110 Data Acquisition Section 111 Explicit Data Acquisition Section 112 Latent Data Acquisition Department 121 Basic model processing section 122 Additional model processing section 130 Integrated Processing Unit 200 Learning control unit 201 Learning control device 210 Permeation Data Storage Unit 221 Model configuration control section 222 Combined load adjustment section 223 Switcher Section 224 Learning result judgment unit

Claims

1. a tangible data acquisition unit that acquires tangible data, which is data acquired during both learning and operation; a latent data acquisition unit that acquires latent data, which is data acquired only during the learning period out of during the learning period and during the operation period; a basic model processing unit that inputs only the manifest data of the manifest data and the latent data into a basic model that is a calculation model and performs a calculation; an additional model processing unit that learns an additional model, which is a calculation model, so that the calculation results are the same when performing a calculation using both the manifest data and the latent data during the learning process and when performing a calculation using only the manifest data of the manifest data and the latent data, and that performs a calculation by inputting only the manifest data of the manifest data and the latent data into the additional model during the operation process; an integration processing unit that calculates an integrated calculation result by integrating a calculation result based on the basic model and a calculation result based on the additional model; a learning result determination unit that determines whether or not the estimation accuracy based on the calculation result of the integration processing unit is improved when the additional model is added compared to when the additional model is not added, and decides whether or not to adopt the additional model; Equipped with the additional model processing unit deletes the additional model when it is determined not to adopt the additional model. Neural network system.

2. A switcher unit that sets a non-penetration rate, which is a coefficient by which the latent data output from the latent data acquisition unit to the additional model processing unit is multiplied, so that the influence of the latent data on learning gradually decreases. The neural network system of claim 1 further comprising:

3. A model configuration control unit is further provided, The latent data acquisition unit acquires the latent data for a plurality of groups, the model configuration control unit causes, for each combination of the groups of latent data, learning of an additional model that receives input of the latent data included in the combination, among the additional models provided for each group of latent data; the learning result determination unit determines to adopt any one of combinations of additional models corresponding to the combination of the groups of the latent data based on an estimation accuracy based on a calculation result of the integration processing unit for each combination of the groups of the latent data; The additional model processing unit deletes additional models that are not adopted.

3. The neural network system according to claim 1 or 2.

4. a tangible data acquisition unit that acquires tangible data, which is data acquired during both learning and operation; a latent data acquisition unit that acquires latent data, which is data acquired only during the learning period out of during the learning period and during the operation period; a basic model processing unit that inputs only the manifest data of the manifest data and the latent data into a basic model that is a calculation model and performs a calculation; an additional model processing unit that, during the operation, inputs only the manifest data of the manifest data and the latent data into an additional model that is a calculation model and performs a calculation; an integration processing unit that calculates an integrated calculation result by integrating a calculation result based on the basic model and a calculation result based on the additional model; A learning control device for controlling learning of a neural network arithmetic device comprising: a connection weight adjustment unit that causes the additional model processing unit to learn the additional model so that a calculation result is the same when performing a calculation using both the manifest data and the latent data during the learning and when performing a calculation using only the manifest data of the manifest data and the latent data; a learning result determination unit that determines whether or not the estimation accuracy based on the calculation result of the integration processing unit is improved when the additional model is added compared to when the additional model is not added, and decides whether or not to adopt the additional model; a model configuration control unit that causes the additional model processing unit to delete the additional model when it is determined not to adopt the additional model; A learning control device comprising:

5. A computer comprising: Acquiring explicit data, which is data acquired during both learning and operation; acquiring latent data, which is data acquired only during the learning period out of during the learning period and during the operation period; During both the learning and the operation, only the manifest data of the manifest data and the latent data is input to a basic model, which is a calculation model, and a calculation is performed; During the learning, learning an additional model, which is a computation model, is performed so that the computation result is the same when performing a computation using both the manifest data and the latent data and when performing a computation using only the manifest data of the manifest data and the latent data. During the operation, only the manifest data of the manifest data and the latent data is input to the additional model to perform a calculation; Calculating a calculation result by integrating a calculation result based on the basic model and a calculation result based on the additional model; determining whether or not to adopt the additional model by determining whether or not the estimation accuracy based on the calculation result obtained by integrating the calculation result based on the basic model and the calculation result based on the additional model is improved when the additional model is added compared to when the additional model is not added; If it is decided not to adopt the additional model, deleting the additional model; and A calculation method including:

6. a tangible data acquisition unit that acquires tangible data, which is data acquired during both learning and operation; a latent data acquisition unit that acquires latent data, which is data acquired only during the learning period out of during the learning period and during the operation period; a basic model processing unit that inputs only the manifest data of the manifest data and the latent data into a basic model that is a calculation model and performs a calculation; an additional model processing unit that, during the operation, inputs only the manifest data of the manifest data and the latent data into an additional model that is a calculation model and performs a calculation; an integration processing unit that calculates an integrated calculation result by integrating a calculation result based on the basic model and a calculation result based on the additional model; A learning control method for controlling learning of a neural network arithmetic device comprising: During the learning, the additional model processing unit is caused to learn the additional model such that a calculation result is the same when a calculation is performed using both the visible data and the latent data and when a calculation is performed using only the visible data of the visible data and the latent data; determining whether or not the estimation accuracy based on the calculation result of the integration processing unit is improved when the additional model is added compared to when the additional model is not added, and determining whether or not to adopt the additional model; When it is determined not to adopt the additional model, causing the additional model processing unit to delete the additional model; A learning control method comprising:

7. On the computer, Acquiring explicit data, which is data acquired during both learning and operation; acquiring latent data, which is data acquired only during the learning period out of during the learning period and during the operation period; During both the learning and the operation, only the manifest data of the manifest data and the latent data is input to a basic model, which is a calculation model, and a calculation is performed; During the learning, learning an additional model, which is a computation model, is performed so that the computation result is the same when performing a computation using both the manifest data and the latent data and when performing a computation using only the manifest data of the manifest data and the latent data. During the operation, only the manifest data of the manifest data and the latent data is input to the additional model to perform a calculation; Calculating a calculation result by integrating a calculation result based on the basic model and a calculation result based on the additional model; determining whether or not to adopt the additional model by determining whether or not the estimation accuracy based on the calculation result obtained by integrating the calculation result based on the basic model and the calculation result based on the additional model is improved when the additional model is added compared to when the additional model is not added; If it is decided not to adopt the additional model, deleting the additional model; and A program for executing the above.

8. On the computer, a tangible data acquisition unit that acquires tangible data, which is data acquired during both learning and operation; a latent data acquisition unit that acquires latent data, which is data acquired only during the learning period out of during the learning period and during the operation period; a basic model processing unit that inputs only the manifest data of the manifest data and the latent data into a basic model that is a calculation model and performs a calculation; an additional model processing unit that, during the operation, inputs only the manifest data of the manifest data and the latent data into an additional model that is a calculation model and performs a calculation; an integration processing unit that calculates an integrated calculation result by integrating a calculation result based on the basic model and a calculation result based on the additional model; A computer that controls learning of a neural network arithmetic device comprising: During the learning, the additional model processing unit is caused to learn the additional model such that a calculation result is the same when a calculation is performed using both the visible data and the latent data and when a calculation is performed using only the visible data of the visible data and the latent data; determining whether or not the estimation accuracy based on the calculation result of the integration processing unit is improved when the additional model is added compared to when the additional model is not added, and determining whether or not to adopt the additional model; When it is determined not to adopt the additional model, causing the additional model processing unit to delete the additional model; A program for executing the above.

Images