Uncertainty learning device, uncertainty learning program, and uncertainty learning system

Abstract

The present invention includes a training data acquisition unit (51) that acquires training data created on the basis of operation-related data obtained from a machine device (1), a noise imparting unit (52) that creates noise-imparted training data in which noise is imparted to the training data acquired by the training data acquisition unit (51), an outlier value detection unit (53) that calculates an outlier value score from the training data acquired by the training data acquisition unit (51) and the noise-imparted training data created by the noise imparting unit (52), and a model learning unit (54) that calculates, on the basis of the training data and the noise-imparted training data, a loss function that is weighted on the basis of the outlier value score calculated by the outlier value detection unit (53), and performs learning of a machine learning model.

Description

Uncertainty learning device, uncertainty learning program, and uncertainty learning system

[0001] The present disclosure relates to an uncertainty learning device, an uncertainty learning program, and an uncertainty learning system that create a trained model (hereinafter referred to as a "machine learning model") that outputs a predicted value and uncertainty for input data.

[0002] In recent years, techniques for solving various tasks using machine learning models have been known in various situations. For example, when considering the application of machine learning models to devices used in life-threatening situations such as autonomous driving or medical settings, or to mechanical devices such as factory automation (FA) equipment, the machine learning model outputs some data regardless of the input data. It is possible that the machine learning model may sometimes output an output result that is unexpected by humans, in other words, an output result with low reliability. If an output result with low reliability output by a machine learning model is directly used to control the mechanical device, an unexpected situation may occur. Therefore, conventionally, techniques have been known for inferring the degree of uncertainty contained in the output of a machine learning model. Here, uncertainty refers to reliability. For example, Non-Patent Document 1 discloses a technique for expressing the degree of uncertainty contained in the output of a machine learning model as a range, and by adding noise to the training data of the machine learning model, the range of uncertainty output by the machine learning model is increased for data that has not been trained (extrapolated data).

[0003] Danijar Hafner, Noise Contrastive Priors for Functional Uncertainty, 1 Jul 2019

[0004] In conventional techniques for outputting uncertainty, such as the technique disclosed in Non-Patent Document 1, noise is added to all training data to train a machine learning model. Therefore, when making an inference using the machine learning model, even if data similar to the training data is input and a good predicted value is output, there is a problem that the uncertainty output along with the predicted value may be high.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a machine learning model that can infer predicted values ​​and uncertainty, and that can infer uncertainty more accurately than machine learning models created using conventional methods in which noise is added to all training data to train the machine learning model.

[0006] The uncertainty learning device according to the present disclosure is an uncertainty learning device that takes as input data based on driving-related data relating to the operating results of a mechanical device, and creates a machine learning model that outputs a predicted value corresponding to the driving-related data and the uncertainty of the predicted value. The uncertainty learning device includes a learning data acquisition unit that acquires learning data created based on the driving-related data obtained from the mechanical device, a noise addition unit that creates noise-added learning data by adding noise to the learning data acquired by the learning data acquisition unit, an outlier detection unit that calculates an outlier score from the learning data acquired by the learning data acquisition unit and the noise-added learning data created by the noise addition unit, and a model learning unit that calculates a loss function weighted based on the outlier score calculated by the outlier detection unit, based on the learning data and the noise-added learning data, and learns the machine learning model.

[0007] According to the present disclosure, as configured above, it is possible to provide a machine learning model that can infer uncertainty more accurately than a machine learning model created using a conventional method in which noise is added to all training data to train the machine learning model.

[0008] 1A is a diagram for explaining the types of uncertainty inferred by a machine learning model and an example of ideal uncertainty inference by the machine learning model, where FIG. 1A is a diagram for explaining problem uncertainty, FIG. 1B is a diagram for explaining learning uncertainty, and FIG. 1C is a diagram for explaining ideal uncertainty.

[0024] FIG. 1C is a diagram for explaining in more detail problems of the conventional technology that are solved by the uncertainty learning device according to embodiment 1.

[0025] FIG. 1C is a diagram for explaining an example of the configuration of an uncertainty learning system including an uncertainty learning device according to embodiment 1.

[0026] FIG. 1C is a flowchart for explaining the operation of the uncertainty learning device according to embodiment 1 during learning.

[0027] FIG. 1C is a flowchart for explaining the details of the processing of step ST3 of FIG. 4.

[0028] FIG. 1C is a flowchart for explaining the operation of the uncertainty learning device according to embodiment 1 during inference.

[0029] FIG. 7A and FIG. 7B are diagrams showing an example of the hardware configuration of the uncertainty learning device according to embodiment 1.

[0030] FIG. 7B is a diagram showing an example of the configuration of an uncertainty learning system including an uncertainty learning device according to embodiment 2.

[0031] FIG. 7C is a flowchart for explaining the operation of the uncertainty learning device according to embodiment 2 during learning.

[0032] FIG. 7D is a flowchart for explaining the operation of the uncertainty learning device according to embodiment 2 during inference.

[0033] FIG. 7D is a diagram showing an example of the configuration of an uncertainty learning system including an uncertainty learning device according to embodiment 3. 12A is a diagram illustrating an example of Fine Tuning, FIG. 12B is a diagram illustrating an example of Feature Extraction, and FIG. 12C is a diagram illustrating an example of Joint Training. FIG. 12B is a diagram illustrating an example of Feature Extraction, and FIG. 12C is a diagram illustrating an example of Joint Training. FIG. 12C is a flowchart illustrating the operation of the uncertainty learning device according to embodiment 3 during learning. FIG. 12C is a flowchart illustrating the details of the processing of step ST3a in FIG. 13. FIG. 13C is a diagram illustrating an example of the configuration of an uncertainty learning system including an uncertainty learning device according to embodiment 4. FIG. 12D is a flowchart illustrating the operation of the uncertainty learning device according to embodiment 4.

[0009] In order to explain the present disclosure in more detail, embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0010] Embodiment 1. An uncertainty learning device according to embodiment 1 creates a trained model (hereinafter referred to as a "machine learning model"). The machine learning model outputs a predicted value (hereinafter referred to as a "predicted value") and uncertainty based on input data. Uncertainty is an index that indicates the degree of confidence in the predicted value, i.e., how confident the machine learning model is in the predicted value. High uncertainty means that the machine learning model is not confident in the predicted value, in other words, the reliability of the predicted value is low. On the other hand, low uncertainty can be interpreted as the machine learning model being highly confident in the predicted value, in other words, the reliability of the predicted value is high. It is important for users and the like to know the uncertainty of the predicted value output by the machine learning model in order to perform appropriate processing. By using uncertainty information, users and the like can evaluate the risks of predictions made by the machine learning model and make more careful decisions.

[0011] Here, Figure 1 is a diagram illustrating the types of uncertainty inferred by a machine learning model and an example of ideal uncertainty inference by a machine learning model. Examples of uncertainty in predicted values ​​include uncertainty due to variations in data present in training data (hereinafter referred to as "interpolated data") caused by measurement noise or disturbances (see Figure 1A), and uncertainty due to data not present in the training data (hereinafter referred to as "extrapolated data") (see Figure 1B). It is necessary to build a machine learning model that can accurately output uncertainty for both the uncertainty due to variations in the interpolated data (in other words, the uncertainty of the problem) and the uncertainty due to the extrapolated data (in other words, the uncertainty of the training) (see Figure 1C).

[0012] Therefore, for example, in the conventional technology disclosed in the above-mentioned Non-Patent Document 1, noise is added to training data, and training is performed so that the uncertainty of the training data to which the noise has been added increases, in other words, the reliability decreases. However, since this conventional technology trains a machine learning model by adding noise, there is a problem in that although the machine learning model can output the uncertainty of the training with a certain degree of accuracy, it may not be able to output the uncertainty of the problem with high accuracy.

[0013] FIG. 2 is a diagram for explaining in more detail the problems inherent in the above-described conventional technology and the problems solved by the uncertainty learning device according to the first embodiment. For example, if a machine learning model is trained using the KL-divergence method to output uncertainty by expressing the variance of interpolated data as variance, the uncertainty of the problem can be output with a certain degree of accuracy. However, learning using this method makes it difficult to infer appropriate uncertainty for extrapolated data (see the diagram on the left side of FIG. 2). This is because the extrapolated data exists outside the range of the training data, which may result in a large discrepancy between the data and the true distribution. Because machine learning models are trained based on training data, there is a problem in that it is difficult to infer accurate uncertainty for data (extrapolated data) that is outside the range of the training data. In contrast, the conventional technology disclosed in the aforementioned Non-Patent Document 1 adds noise to the training data to increase the uncertainty of the training data to which noise has been added (see the diagram in the middle of FIG. 2). Therefore, this conventional technology can solve the above-described problem of difficulty in inferring accurate uncertainty for extrapolated data. However, in this conventional technology, noise is added to all training data to train the machine learning model. Therefore, when inferencing using the machine learning model, even if data similar to the training data is input and a good predicted value is output, the uncertainty output along with the predicted value may be high. In other words, this conventional technology may not be able to accurately output the uncertainty of the problem.

[0014] The uncertainty learning device according to the first embodiment solves the problems of the conventional technology and provides a machine learning model capable of accurately inferring uncertainty for both the uncertainty of the problem and the uncertainty of the learning, i.e., for both the uncertainty of the interpolated data and the uncertainty of the extrapolated data. Specifically, the uncertainty learning device according to the first embodiment trains a machine learning model to infer uncertainty using the results of inferring the degree of deviation of noise-added learning data using an outlier detection method (see the diagram on the right side of Figure 2). The configuration and operation of the uncertainty learning device will be described in detail later.

[0015] 3 is a diagram showing an example of the configuration of an uncertainty learning system 100 including an uncertainty learning device 2 according to the first embodiment. The uncertainty learning system 100 includes the uncertainty learning device 2 and a machine 1. The uncertainty learning device 2 and the machine 1 are connected via a network. In the first embodiment, the machine 1 is assumed to be, for example, a factory automation (FA) device. The FA device includes, for example, a servo motor, a CNC, a processing machine such as a sheet metal laser processing machine or an electric discharge machine, or a robot.

[0016] The uncertainty learning device 2 is provided in, for example, a server. In embodiment 1, the uncertainty learning device 2 performs "learning" to create a machine learning model and "inference" to infer a predicted value and uncertainty using the machine learning model created by "learning." Details of the "learning" process and the "inference" process will be described later. The uncertainty learning device 2 includes an acquisition unit 3, a preprocessing unit 4, a learning unit 5, and an inference unit 6. The acquisition unit 3 includes a driving result acquisition unit 31 and a driving result storage unit 32. The preprocessing unit 4 includes a preprocessing execution unit 41, a learning data storage unit 42, and a test data storage unit 43. The learning unit 5 includes a learning data acquisition unit 51, a noise addition unit 52, an outlier detection unit 53, a model learning unit 54, and a model storage unit 55. The inference unit 6 includes a model reading unit 61, a prediction unit 62, a test data acquisition unit 63, and an evaluation unit 64. During learning, of the above-mentioned components, the driving result acquisition unit 31, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the model reading unit 61, the test data acquisition unit 63, and the evaluation unit 64 function in the uncertainty learning device 2. During inference, of the above-mentioned components, the driving result acquisition unit 31, the preprocessing execution unit 41, the model reading unit 61, and the prediction unit 62 function in the uncertainty learning device 2.

[0017] The operation result acquisition unit 31 of the acquisition unit 3 acquires data relating to the operation results of the machine 1 (hereinafter referred to as "operation-related data") from the machine 1. The operation results include, for example, parameters set in the machine 1, the configuration of the machine 1, the operation mode, data indicating values ​​specific to the machine 1, or log data of the operation of the machine 1. The operation result acquisition unit 31 acquires the operation-related data from the machine 1 via, for example, an encoder (not shown) or the like. Here, the operation result acquisition unit 31 acquires the operation-related data relating to the operation results of the FA equipment from the FA equipment. For example, if the FA equipment is a CNC (Computer Numerical Control) machining machine, the operation-related data includes a command position, a command speed, a command acceleration, a feedback speed, a feedback acceleration, or a current value. The operation-related data may include, for example, a measured value of a machining position deviation. In the following embodiment 1, when the machine 1 is referred to, this refers to the FA equipment.

[0018] The driving result acquisition unit 31, for example, directly acquires values ​​measured by various sensors installed in the machine 1 as driving-related data. The various sensors include a speed sensor, an acceleration sensor, a gyro sensor, a temperature sensor, and a humidity sensor. The driving result acquisition unit 31 may also acquire, for example, some value calculated based on values ​​measured by the various sensors installed in the machine 1 as driving-related data. The driving result acquisition unit 31 stores the acquired driving-related data in the driving result storage unit 32. For example, the driving result acquisition unit 31 adds the acquisition date and time of the driving-related data to the driving-related data and stores the data in the driving result storage unit 32.

[0019] The driving result storage unit 32 stores the driving-related data acquired by the driving result acquisition unit 31. Note that, although the driving result storage unit 32 is assumed to be provided in the uncertainty learning device 2 here, this is merely an example. The driving result storage unit 32 may be provided in a location outside the uncertainty learning device 2 that can be referenced by the uncertainty learning device 2.

[0020] The preprocessing execution unit 41 of the preprocessing unit 4 acquires the driving-related data acquired by the driving result acquisition unit 31 from the driving result storage unit 32 and performs preprocessing to shape various data based on the driving-related data. In the first embodiment, the preprocessing performed by the preprocessing execution unit 41 includes preprocessing performed when the uncertainty learning device 2 creates a machine learning model and preprocessing performed when the uncertainty learning device 2 performs inference to obtain a predicted value and uncertainty using the created machine learning model. In the first embodiment, the preprocessing performed by the preprocessing execution unit 41 when the uncertainty learning device 2 creates a machine learning model is also referred to as "learning preprocessing," and the preprocessing performed by the preprocessing execution unit 41 when the uncertainty learning device 2 performs inference is also referred to as "inference preprocessing." Note that in the uncertainty learning device 2, the machine learning model is created by the evaluation unit 64 of the learning unit 5 and the inference unit 6, and inference using the machine learning model is performed by the prediction unit 62 of the inference unit 6. Details of the learning unit 5 and the inference unit 6 will be described later. The learning preprocessing and inference preprocessing performed by the preprocessing execution unit 41 will be described below.

[0021] First, the learning preprocessing performed by the preprocessing execution unit 41 will be described. The preprocessing execution unit 41 acquires the driving-related data acquired by the driving result acquisition unit 31 from the driving result storage unit 32, and performs preprocessing, i.e., learning preprocessing, to shape the learning data to be used by the learning unit 5 and the test data to be used by the evaluation unit 64 of the inference unit 6 based on the driving-related data. In the first embodiment, the learning preprocessing performed by the preprocessing execution unit 41 includes, for example, a process of separating data included in the driving-related data into explanatory variables and target variables, a process of converting data included in the driving-related data into feature quantities required for learning a machine learning model, or a process of dividing the driving-related data into learning data and test data.

[0022] Here, explanatory variables and objective variables will be explained. The type of data to be used as the explanatory variables and objective variables is determined in advance by a user or the like. At least one of the data other than the objective variable is used as the explanatory variable. For example, if the objective variable is the deviation of the machining position, operation-related data indicating the machining position or speed can be the explanatory variable. At least one of the data to be predicted by the machine learning model is used as the objective variable. A user or the like may change the objective variable as appropriate depending on the problem; for a regression problem, a numerical value is used as the objective variable, and for a classification problem, a class to be classified is used as the objective variable. For example, if a user or the like wants to predict deviation of the machining position due to friction or the like in the machine device 1, the user or the like can use the measured value of the deviation of the machining position as the objective variable.

[0023] Next, an example of feature transformation in learning preprocessing performed by the preprocessing execution unit 41 will be described. The preprocessing execution unit 41 may transform explanatory variables to create features effective for learning. For example, the preprocessing execution unit 41 may transform explanatory variables by applying standardization to convert the scale of variables, linear transformation using min-max scaling, or nonlinear transformation such as Box-Cox. Alternatively, if some of the data is categorical variables, the preprocessing execution unit 41 may transform explanatory variables by applying one-hot encoding, embedding, or the like.

[0024] Next, an example of the division of training data and test data in the training preprocessing performed by the preprocessing execution unit 41 will be described. When training, it is necessary to divide the data included in the driving-related data into training data used for training and test data for evaluating the performance of the trained machine learning model. For example, the preprocessing execution unit 41 may randomly divide the data included in the driving-related data into training data and test data, or may create training data by aligning the number of data for each class, as in a classification problem, and divide the remaining data into test data. Note that if the driving-related data is time-series data, learning new data in the time direction will result in learning correct data for older data, which can cause data leakage. Therefore, the preprocessing execution unit 41 needs to divide the training data and test data while paying attention to the time axis.

[0025] The preprocessing execution unit 41 stores the learning data created by performing the learning preprocessing in the learning data storage unit 42. The preprocessing execution unit 41 also stores the test data created by performing the learning preprocessing in the test data storage unit 43.

[0026] Next, the inference preprocessing performed by the preprocessing execution unit 41 will be described. The preprocessing execution unit 41 acquires the driving-related data acquired by the driving result acquisition unit 31 from the driving result storage unit 32 and performs preprocessing, i.e., inference preprocessing, to shape data to be used by the prediction unit 62 of the inference unit 6 (hereinafter referred to as "model input data") based on the driving-related data. In the first embodiment, the inference preprocessing performed by the preprocessing execution unit 41 includes, for example, a process of using data included in the driving-related data that is set as an explanatory variable as model input data, or a process of converting the data included in the driving-related data into feature quantities required as model input data for a machine learning model. The preprocessing execution unit 41 may convert the data into feature quantities required as model input data for a machine learning model using the same method as the feature quantity conversion that converts explanatory variables to create feature quantities effective for learning in the learning preprocessing. Note that inference using a machine learning model does not require separation into training data and test data. The preprocessing execution unit 41 outputs the model input data created by performing inference preprocessing to the prediction unit 62.

[0027] The learning data storage unit 42 of the preprocessing unit 4 stores the learning data. Note that, although the learning data storage unit 42 is assumed to be provided in the uncertainty learning device 2 here, this is merely an example. The learning data storage unit 42 may also be provided in a location outside the uncertainty learning device 2 that can be referenced by the uncertainty learning device 2.

[0028] The test data storage unit 43 of the preprocessing unit 4 stores test data. Note that, although the test data storage unit 43 is assumed to be provided in the uncertainty learning device 2 here, this is merely an example. The test data storage unit 43 may also be provided in a location outside the uncertainty learning device 2 that can be referenced by the uncertainty learning device 2.

[0029] The learning unit 5 creates a machine learning model based on the training data stored in the training data storage unit 42. The machine learning model uses one or more machine learning techniques, such as a neural network. The neural network may be of any type, such as a hierarchical neural network, a convolutional neural network, or a recurrent neural network. The neural network only needs to have at least two outputs so that it can output a predicted value and uncertainty. Note that, if the number of dimensions of the input data to the machine learning model, such as the number of parameters, is large, the learning unit 5 may use various dimensionality reduction techniques to reduce the dimensions of the input data and use the dimensionally reduced data as input data to the machine learning model. Examples of various dimensionality reduction techniques include principal component analysis, singular value analysis, tensor analysis, and autoencoder. These dimensionality reduction techniques are well known, and detailed description thereof will be omitted.

[0030] The learning data acquisition unit 51 of the learning unit 5 acquires learning data from the learning data storage unit 42. N pieces of learning data are stored in the learning data storage unit 42. In this case, for the N pieces of learning data, assuming that the explanatory variable is xn and the objective variable is yn, the learning data D Ｎ = {{x １ , y １}, ..., {x ｎ , y ｎ The learning unit 5 uses this learning data D Ｎ The learning unit 50 learns the learning data using a machine learning model. The learning of the machine learning model is performed by the model learning unit 54 of the learning unit 5. Details of the model learning unit 54 will be described later. The learning data acquisition unit 51 outputs the acquired learning data to the noise addition unit 52, the outlier detection unit 53, and the model learning unit 54.

[0031] The noise adding unit 52 of the learning unit 5 creates data (hereinafter referred to as "noise-added learning data") by adding noise to the learning data acquired by the learning data acquiring unit 51. For example, the noise adding unit 52 creates the noise-added learning data by adding values ​​created from a Gaussian distribution, a uniform distribution, a gamma distribution, or a β distribution to the explanatory variables and the objective variables of the learning data.

[0032] For example, when the noise adding unit 52 creates noise-added learning data by adding together values ​​created using a Gaussian distribution, if the explanatory variable is x and the objective variable is y, then: In this case, N(·): Gaussian distribution, μ ｘ , σ ｘ : Mean and standard deviation of noise added to explanatory variables, μ ｙ , σ ｙ : The mean value and standard deviation of the noise added to the objective variable. Note that μ ｙ is μ ｙ = y.

[0033] For example, when the variation in the explanatory variables or the objective variable is known from the characteristics of the machine device 1, the noise adding unit 52 may add noise that simulates the variation. The noise adding unit 52 outputs the noise-added learning data to the outlier detection unit 53 and the model learning unit 54. Note that the noise adding unit 52 may create the noise-added learning data by adding noise to all of the learning data acquired by the learning data acquisition unit 51 from the learning data storage unit 42, or may create the noise-added learning data by adding noise to a portion of the learning data (mini-batch) of the learning data acquired by the learning data acquisition unit 51 from the learning data storage unit 42.

[0034] The outlier detection unit 53 of the learning unit 5 calculates an outlier score from the learning data acquired by the learning data acquisition unit 51 and the noise-added learning data created by the noise adding unit 52 .

[0035] The calculation of the outlier score by the outlier detection unit 53 will be described in detail. First, the outlier detection unit 53 inputs the learning data acquired by the learning data acquisition unit 51 into an outlier detection method and performs learning. Note that the outlier detection unit 53 inputs, for example, all of the learning data acquired by the learning data acquisition unit 51 into the outlier detection method and performs learning. The outlier detection unit 53 may perform learning using a known outlier detection method such as Hotelling's theory, k-nearest neighbor method, Local Outlier Factor (LOF), One-class Support Vector Machine, etc. The outlier detection unit 53 may use one or more of these known outlier detection methods. Furthermore, when the number of dimensions to be input is large, the outlier detection unit 53 may reduce the dimensions of the input data using the known dimension reduction method described above, and use the dimension-reduced data as input to the outlier detection method. The input data that the outlier detection unit 53 inputs to the outlier detection method may be, for example, all or some of the explanatory variables included in the training data, or may be data in which a target variable is added to all or some of the explanatory variables included in the training data.

[0036] After the outlier detection method is trained using all the training data, the outlier detection unit 53 then inputs the noise-added training data created by the noise adding unit 52 into the trained outlier detection method and calculates an outlier score. The outlier detection unit 53 then sets a weight γ for the NCP loss based on the calculated outlier score. In the first embodiment, the second term in the loss function used in training the machine learning model is referred to as "NCP loss." Details of the loss function will be described later.

[0037] The outlier detection unit 53 sets the weight γ of the NCP Loss based on, for example, the magnitude of the outlier score calculated by the outlier detection method. The outlier detection unit 53 sets the weight γ so that it is larger for noise-added learning data that deviates from the learning data, and sets the weight γ so that it is smaller for noise-added learning data that is closer to the learning data. For example, when the outlier score is calculated using a known LOF, which is an outlier detection method that assumes that the greater the difference between the local density of a point and the local density of a neighboring point, the greater the likelihood of it being an outlier, the larger the outlier score, meaning that it deviates more from the learning data. In this case, the outlier detection unit 53 sets the weight γ to, for example, a value normalized so that the value obtained by reversing the positive and negative signs of the outlier score falls within the range of [0, 1]. Furthermore, the outlier detection unit 53 may, for example, set a threshold value (hereinafter referred to as the "outlier determination threshold") and set the weight γ by comparing the outlier score with the outlier determination threshold. For example, the outlier detection unit 53 may set a weight γ=1 to noise-added learning data that deviates from the learning data, such that the outlier score is equal to or greater than the outlier determination threshold, and may set a weight γ=0 to noise-added learning data that is close to the learning data, such that the outlier score is smaller than the outlier determination threshold.

[0038] Note that the above example is merely one example. When deviating from the learning data, whether the outlier score is equal to or greater than the outlier determination threshold or smaller than the outlier determination threshold depends on the outlier detection method. When setting the weight γ by comparing the outlier score with the outlier determination threshold, the outlier detection unit 53 may compare the outlier score with the outlier determination threshold so as to set the weight γ = 1 for noise-added learning data that deviates from the learning data and the weight γ = 0 for noise-added learning data that is close to the learning data.

[0039] The outlier detection unit 53 outputs the set weight γ to the model learning unit 54 .

[0040] The model learning unit 54 of the learning unit 5 calculates a loss function using the NCP Loss weight γ set by the outlier detection unit 53, and learns a machine learning model. More specifically, the model learning unit 54 calculates a loss function using the weight γ set by the outlier detection unit 53, based on the learning data acquired by the learning data acquisition unit 51 and the noise-added learning data created by the noise adding unit 52, and learns a machine learning model. In this way, the model learning unit 54 creates a machine learning model. The model learning unit 54 uses the following loss function L(θ) in learning the machine learning model. Note that the term after Exprior in the calculation formula for the loss function L(θ) below is the second term in the loss function used in learning the machine learning model described above. In this case, pmodel(y|x, θ) is the probability distribution of the output y obtained when the explanatory variable x is input in the parameter θ. ｐｔｒａｉｎ（ｘ，ｙ） [Impmodel(y|x, θ)] is the maximum likelihood inference quantity for the training data, It is. D ＫＬ refers to KL-divergence, which is the distribution of correct data p(y|x) and the output distribution p from the machine learning model. ｍｏｄｅｌ The method is not limited to this as long as it is a method for finding the inter-distribution distance with (y|x, θ). The distribution p(y|x) of the correct data may be regarded as a delta function distribution, or may be a normal distribution, for example. Due to the weight γ of the NCP Loss calculated by the outlier detection unit 53 from the outlier score, the influence of the second term is greater for noise-added learning data that deviates from the learning data, and the influence of the second term is smaller for noise-added learning data that is close to the learning data.

[0041] The model learning unit 54 of the learning unit 5 learns a machine learning model, and when it creates a machine learning model, stores the created machine learning model in the model storage unit 55 .

[0042] The model storage unit 55 stores the machine learning model. Note that, here, the model storage unit 55 is assumed to be provided in the uncertainty learning device 2, but this is merely an example. The model storage unit 55 may also be provided in a location outside the uncertainty learning device 2 that can be referenced by the uncertainty learning device 2.

[0043] The inference unit 6 evaluates the machine learning model created by the learning unit 5 and infers, in other words, makes predictions, of predicted values ​​and uncertainties using the machine learning model based on actual data, in this case, operation-related data acquired from the machine device 1. In the inference unit 6, evaluation of the machine learning model created by the learning unit 5 is performed when the machine learning model is created, i.e., during learning. The evaluation of the machine learning model is performed by an evaluation unit 64 of the inference unit 6. In the inference unit 6, inference of predicted values ​​and uncertainties using the machine learning model based on actual data is performed during inference. The inference using the machine learning model is performed by a prediction unit 62 of the inference unit 6.

[0044] The model reading unit 61 reads a machine learning model by referring to the model storage unit 55. The model reading unit 61 outputs the read machine learning model to the prediction unit 62 and the evaluation unit 64.

[0045] The test data acquisition unit 63 acquires test data from the test data storage unit 43. The test data acquisition unit 63 outputs the acquired test data to the evaluation unit 64.

[0046] The evaluation unit 64 uses the test data acquired by the test data acquisition unit 63 to evaluate the machine learning model read by the model reading unit 61, in other words, the machine learning model created by the model learning unit 54 and stored in the model storage unit 55. More specifically, the evaluation unit 64 evaluates the accuracy of the machine learning model based on the inference result (prediction value and uncertainty) output from the machine learning model and the objective variable of the test data acquired by the test data acquisition unit 63. The evaluation unit 64 may use an evaluation index such as RMSE (root mean square error), MAE (mean absolute error), or a likelihood function. The evaluation unit 64 may acquire the inference result output from the machine learning model from the model learning unit 54. Note that an arrow from the model learning unit 54 to the evaluation unit 64 is omitted in FIG. 3 . The evaluation unit 64 may evaluate the machine learning model using a known machine learning model evaluation method using test data. During learning, the model learning unit 54 inputs explanatory variables into the machine learning model based on the learning data, and creates the machine learning model by learning so that the output inference results (predicted values ​​and uncertainties) match the objective variables corresponding to the explanatory variables. During learning, the evaluation unit 64 evaluates the accuracy of the machine learning model created by the model learning unit 54 to see how much the accuracy of the machine learning model has improved.

[0047] The prediction unit 62 acquires model input data based on driving-related data from the machine 1, inputs the model input data into a machine learning model to obtain a predicted value and uncertainty, and thereby infers the predicted value and uncertainty. As described above, the machine learning model receives input data (here, model input data based on driving-related data) and outputs a predicted value and uncertainty. The model input data input by the prediction unit 62 to the machine learning model is unknown data. The prediction unit 62 inputs the model input data, which is unknown data, to the machine learning model to obtain a predicted value corresponding to the objective variable (position error, control value, etc.) of the unknown data and the uncertainty for the predicted value. Here, the predicted value is, for example, a numerical value in the case of a regression problem, and the probability of each class in the case of a classification problem. The uncertainty is, for example, a variance or standard deviation indicating the variation in data. As described above, the prediction unit 62 acquires model input data to be input to the machine learning model from the machine 1 via the driving result acquisition unit 31 of the acquisition unit 3 and the preprocessing execution unit 41 of the preprocessing unit 4.

[0048] The prediction unit 62 outputs the obtained predicted value and uncertainty to the machine device 1. When the machine device 1 acquires the predicted value and uncertainty from the uncertainty learning device 2, it performs control based on the acquired predicted value and uncertainty.

[0049] Here, a specific example will be given of control based on a predicted value and uncertainty in the machine 1. Here, as an example, the uncertainty learning device 2 outputs, as the predicted value and uncertainty, a correction amount and its uncertainty for a control command for a machining position based on data such as a CAD model when the machine 1 processes (excavates) a workpiece into a certain shape. The machine 1 corrects the control command based on the correction amount and its uncertainty output from the uncertainty learning device 2. For example, if the uncertainty σ output from the uncertainty learning device 2 is equal to or greater than a predetermined threshold value (hereinafter referred to as the "uncertainty determination threshold") th, i.e., if the reliability of the predicted value is low, the machine 1 determines the control command using the following equation: In the following equation, the control command U is the original command value u, and the correction amount f is the correction amount f.

[0050] Note that the above example is merely an example, and the machine 1 may, for example, not correct the control command if the uncertainty is equal to or greater than the uncertainty determination threshold, and may correct the control command if the uncertainty is less than the uncertainty determination threshold. Furthermore, for example, in the uncertainty learning device 2, the prediction unit 62 may determine whether to output the obtained predicted value and uncertainty to the machine 1. For example, the prediction unit 62 may output the predicted value and uncertainty to the machine 1 if the obtained uncertainty is less than the uncertainty determination threshold, and may not output the predicted value and uncertainty to the machine 1 if the obtained uncertainty is greater than or equal to the uncertainty determination threshold. Furthermore, if the obtained uncertainty is high, the prediction unit 62 may collect, as learning data using an active learning technique, the model input data from which the uncertainty was obtained (in other words, the model input data input to the machine learning model).

[0051] A description will be given of the operation of the uncertainty learning device 2 according to embodiment 1. The operation of the uncertainty learning device 2 will be described below, with the operation during learning and the operation during inference being separately described.

[0052] First, the operation of the uncertainty learning device 2 during learning will be described. FIG. 4 is a flowchart for explaining the operation of the uncertainty learning device 2 during learning according to the first embodiment. The uncertainty learning device 2 performs the operation during learning as shown in the flowchart of FIG. 4 based on, for example, instructions from a user. For example, the user operates an input device (not shown) to input an operation start instruction. Upon receiving the operation start instruction, the control unit (not shown) of the uncertainty learning device 2 causes the driving result acquisition unit 31, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the model reading unit 61, the test data acquisition unit 63, and the evaluation unit 64 to start operating. For example, the user may input driving-related data to input the operation start instruction. The uncertainty learning device 2 repeats the learning operation as shown in the flowchart of FIG. 4 until, for example, the machine learning model is evaluated to a certain extent or until the control unit receives an operation end instruction from the user.

[0053] The driving result acquisition unit 31 acquires driving-related data from the machine 1 (step ST1). The driving result acquisition unit 31 stores the acquired driving-related data in the driving result storage unit 32.

[0054] The preprocessing execution unit 41 acquires the driving-related data acquired by the driving result acquisition unit 31 in step ST1 from the driving result storage unit 32, and performs preprocessing, i.e., learning preprocessing, to shape the learning data to be used by the learning unit 5 and the test data to be used by the evaluation unit 64 of the inference unit 6 based on the driving-related data (step ST2). The preprocessing execution unit 41 stores the learning data created by performing the learning preprocessing in the learning data storage unit 42. The preprocessing execution unit 41 also stores the test data created by performing the learning preprocessing in the test data storage unit 43.

[0055] The learning unit 5 performs learning to create a machine learning model based on the learning data stored in the learning data storage unit 42 and the noise-added learning data created based on the learning data (step ST3). The learning unit 5 stores the created machine learning model in the model storage unit 55.

[0056] The evaluation unit 64 of the inference unit 6 evaluates the machine learning model created by the learning unit 5 in step ST3 (step ST4). Specifically, the test data acquisition unit 63 acquires test data from the test data storage unit 43. Then, the evaluation unit 64 evaluates the machine learning model read by the model reading unit 61 using the test data acquired by the test data acquisition unit 63.

[0057] Fig. 5 is a flowchart for explaining the details of the process of step ST3 in Fig. 4. The learning data acquisition unit 51 acquires learning data from the learning data storage unit 42 (step ST11). More specifically, the learning data acquisition unit 51 acquires all learning data from the learning data storage unit 42. The learning data acquisition unit 51 outputs the acquired learning data to the noise addition unit 52, the outlier detection unit 53, and the model learning unit 54.

[0058] The outlier detection unit 53 inputs all the learning data acquired by the learning data acquisition unit 51 in step ST11 into the outlier detection method and performs learning (step ST12).

[0059] The noise adding unit 52 acquires the learning data acquired by the learning data acquiring unit 51 in step ST11 (step ST13). Here, the learning data acquired by the noise adding unit 52 may be all the learning data acquired by the learning data acquiring unit 51, or a part (mini-batch) of the learning data acquired by the learning data acquiring unit 51.

[0060] The noise adding unit 52 adds noise to the training data acquired in step ST13 to create noise-added training data (step ST14). The noise adding unit 52 outputs the created noise-added training data to the outlier detection unit 53 and the model training unit 54.

[0061] The outlier detection unit 53 calculates an outlier score from the learning data acquired in step ST13 and the noise-added learning data created by the noise adding unit 52 in step ST14 (step ST15). More specifically, the outlier detection unit 53 inputs the noise-added learning data created by the noise adding unit 52 to the outlier detection method learned in step ST12, and calculates an outlier score.

[0062] Then, the outlier detection unit 53 sets a weight γ of the NCP Loss based on the calculated outlier score (step ST16). The outlier detection unit 53 outputs the set weight γ to the model learning unit .

[0063] The model learning unit 54 calculates a loss function using the weight γ of the NCP Loss set by the outlier detection unit 53 in step ST16 based on the learning data acquired in step ST13 and the noise-added learning data created in step ST14, and learns a machine learning model (step ST17).

[0064] The model learning unit 54 determines whether learning has been performed a designated number of times (step ST18). Note that the number of times the model learning unit 54 will perform learning is determined in advance by the user or the like.

[0065] If the model learning unit 54 has not performed learning the specified number of times (if "NO" in step ST18), the operation of the learning unit 5 proceeds to the processing of step ST19, and the noise adding unit 52 acquires the learning data acquired by the learning data acquiring unit 51 (step ST19). Here, the learning data acquired by the noise adding unit 52 may be all the learning data acquired by the learning data acquiring unit 51, or may be a portion (mini-batch) of the learning data acquired by the learning data acquiring unit 51. Thereafter, the operation of the learning unit 5 proceeds to the processing of step ST14. Note that in this case, in step ST17, the model learning unit 54 calculates a loss function using the NCP Loss weight γ set by the outlier detection unit 53 in step ST16 based on the learning data acquired in step ST19 and the noise-added learning data created in step ST14, and learns a machine learning model.

[0066] On the other hand, if the model learning unit 54 has learned the specified number of times (if "YES" in step ST18), the model learning unit 54 stores the machine learning model in the model memory unit 55, and the operation of the learning unit 5 ends the processing shown in the flowchart of Figure 5.

[0067] Next, the operation of the uncertainty learning device 2 during inference will be described. FIG. 6 is a flowchart for explaining the operation during inference of the uncertainty learning device 2 according to the first embodiment. The uncertainty learning device 2 performs the operation during inference as shown in the flowchart of FIG. 6 based on, for example, instructions from a user. For example, the user operates an input device to input an instruction to start operation. Upon receiving the instruction to start operation, the control unit of the uncertainty learning device 2 causes the driving result acquisition unit 31, the preprocessing execution unit 41, the model reading unit 61, and the prediction unit 62 to start operation. The uncertainty learning device 2 repeats the operation during inference as shown in the flowchart of FIG. 6 until, for example, the control unit receives an instruction to end operation from the user. Note that the operation during inference of the uncertainty learning device 2 described using the flowchart of FIG. 6 is based on the premise that the operation during learning of the uncertainty learning device 2 described using the flowchart of FIG. 4 has been performed before the operation is performed.

[0068] The driving result acquisition unit 31 acquires driving-related data from the machine 1 (step ST10). The driving result acquisition unit 31 stores the acquired driving-related data in the driving result storage unit 32.

[0069] The preprocessing execution unit 41 acquires the driving-related data acquired by the driving result acquisition unit 31 in step ST10 from the driving result storage unit 32, and performs preprocessing, i.e., inference preprocessing, to shape model input data to be used in the prediction unit 62 of the inference unit 6 based on the driving-related data (step ST20). The preprocessing execution unit 41 outputs the model input data created by performing the inference preprocessing to the prediction unit 62.

[0070] The model reading unit 61 reads the machine learning model by referring to the model storage unit 55 (step ST30). The model reading unit 61 outputs the read machine learning model to the prediction unit 62.

[0071] The prediction unit 62 acquires model input data based on the driving-related data from the machine 1 via the driving result acquisition unit 31 and the preprocessing execution unit 41, and inputs the model input data into a machine learning model to obtain a predicted value and uncertainty, thereby inferring the predicted value and uncertainty (step ST40). The prediction unit 62 then outputs the obtained predicted value and uncertainty to the machine 1 (step ST50). In step ST50, the prediction unit 62 determines whether to output the obtained predicted value and uncertainty to the machine 1, and if it determines to output the obtained predicted value and uncertainty to the machine 1, it may output the predicted value and uncertainty to the machine 1.

[0072] 6, the processing is performed in the order of step ST10, step ST20, and step ST30, but this is merely an example. For example, the processing of step ST30 may be performed before the processing of step ST10, or may be performed in parallel with the processing of steps ST10 to ST20.

[0073] In this way, the uncertainty learning device 2 adds noise to the training data created based on the operation results obtained from the machine device 1, and calculates an outlier score from the training data and the noise-added training data after adding noise. Then, the uncertainty learning device 2 calculates a loss function weighted based on the calculated outlier score based on the training data and the noise-added training data, and trains a machine learning model. When calculating the loss function to train the machine learning model, the uncertainty learning device 2 uses the weight γ set based on the outlier score to increase uncertainty for extrapolated data that is far from the training data and decrease uncertainty for data that is close to the training data. This allows the uncertainty learning device 2 to provide a machine learning model that can more accurately infer uncertainty. In the conventional technology described above, the second term of the loss function is trained using data after adding noise, which increases uncertainty for extrapolated data that is far from the training data, but there is a possibility that uncertainty may also be high for data that is close to the training data or for the training data itself. In contrast, the uncertainty learning device 2 can provide a machine learning model with improved uncertainty inference accuracy. In other words, in a single machine learning model capable of inferring both predicted values ​​and uncertainty, the uncertainty learning device 2 does not learn all data to which noise has been added, but rather controls the data used for learning data using outlier detection technology, thereby providing a machine learning model that can more accurately infer uncertainty for both extrapolated data and data close to the learning data.

[0074] As a technique for inferring predicted values ​​and uncertainties, for example, as disclosed in the following reference document, a technique is known in which Monte Carlo dropout is repeatedly applied to a neural network to obtain predicted values ​​and uncertainties, and calibration is performed using a fitness function that evaluates the accuracy of the predicted values ​​and uncertainties. (Reference Document) Japanese Patent Application Publication No. 2018-200677 However, repeatedly applying Monte Carlo dropout as in the above-mentioned technique lengthens the calculation time for prediction, making it difficult to apply to equipment with a control period of milliseconds, such as factory automation equipment. The uncertainty learning device 2 according to the first embodiment can provide a machine learning model capable of outputting predicted values ​​and uncertainties applicable to the control of a mechanical device 1 with a control period of milliseconds, such as factory automation equipment.

[0075] In the above-described first embodiment, the uncertainty learning device 2 is described as including an acquisition unit 3, a preprocessing unit 4, and an inference unit 6, but this is merely an example. The uncertainty learning device 2 is required to include at least a learning unit 5. For example, the acquisition unit 3, the preprocessing unit 4, and the inference unit 6 may be provided outside the uncertainty learning device 2 in a location that can be referenced by the uncertainty learning device 2. If the uncertainty learning device 2 is configured without the acquisition unit 3, the preprocessing unit 4, and the inference unit 6, the processes of steps ST1 to ST4 can be omitted from the operation of the uncertainty learning device 2 described using the flowchart of FIG. 4. Furthermore, the uncertainty learning device 2 can omit the operation described using the flowchart of FIG. 6.

[0076] In the first embodiment described above, the uncertainty learning device 2 is provided in a server, but this is merely an example. For example, the uncertainty learning device 2 may be provided in the machine 1. In addition, for example, in the uncertainty learning device 2, some or all of the driving result acquisition unit 31, preprocessing execution unit 41, learning data acquisition unit 51, noise addition unit 52, outlier detection unit 53, model learning unit 54, model reading unit 61, prediction unit 62, test data acquisition unit 63, or evaluation unit 64 may be provided in a device external to the server.

[0077] In the first embodiment, the machine 1 is an FA device, but this is merely an example. For example, the machine 1 can be a control device that controls automatic driving of a moving object, or a medical device used in a medical setting, or any other device that solves various tasks using a machine learning model.

[0078] 7A and 7B are diagrams illustrating an example of the hardware configuration of the uncertainty learning device 2 according to the first embodiment. In the first embodiment, the functions of the driving result acquisition unit 31, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, and a control unit (not shown) are realized by a processing circuit 1001. That is, the uncertainty learning device 2 includes the processing circuit 1001 for controlling the creation of a machine learning model capable of more accurate uncertainty inference by learning so that extrapolated data has high uncertainty and data close to the learning data has low uncertainty using a weight γ set based on the outlier score. The processing circuit 1001 may be dedicated hardware as shown in FIG. 7A or a processor 1004 that executes a program stored in memory as shown in FIG. 7B.

[0079] If the processing circuit 1001 is dedicated hardware, the processing circuit 1001 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0080] When the processing circuit is the processor 1004, the functions of the driving result acquisition unit 31, the pre-processing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, and a control unit (not shown) are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 1005. The processor 1004 reads and executes the program stored in the memory 1005, thereby executing the functions of the driving result acquisition unit 31, the pre-processing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, and a control unit (not shown). That is, the uncertainty learning device 2 includes a memory 1005 for storing a program that, when executed by the processor 1004, results in the execution of steps ST1 to ST4 in Fig. 4 described above or steps ST10 to ST50 in Fig. 6 described above. The program stored in the memory 1005 can also be said to cause a computer to execute the procedures or methods of the processing of the driving result acquisition unit 31, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, and a control unit (not shown). Here, the memory 1005 may be, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (registered trademark; hereinafter omitted) (Electrically Erasable Programmable Read-Only Memory), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD (Digital Versatile Disc).

[0081] Note that the functions of the driving result acquisition unit 31, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, and a control unit (not shown) may be partially implemented by dedicated hardware and partially implemented by software or firmware. For example, the functions of the driving result acquisition unit 31 and the preprocessing execution unit 41 may be implemented by a processing circuit 1001 as dedicated hardware, and the functions of the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, and a control unit (not shown) may be implemented by the processor 1004 reading and executing programs stored in the memory 1005. The driving result storage unit 32, the learning data storage unit 42, the test data storage unit 43, and the model storage unit 55 are configured, for example, by a hard disk drive (HDD, not shown) or a solid state drive (SSD, not shown). The uncertainty learning device 2 also includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the machine device 1.

[0082] As described above, the uncertainty learning device 2 according to the first embodiment is configured to include the learning data acquisition unit 51 that acquires learning data created based on operation-related data obtained from the machine device 1, the noise addition unit 52 that creates noise-added learning data by adding noise to the learning data acquired by the learning data acquisition unit 51, the outlier detection unit 53 that calculates an outlier score from the learning data acquired by the learning data acquisition unit 51 and the noise-added learning data created by the noise addition unit 52, and the model learning unit 54 that calculates a loss function weighted based on the outlier score calculated by the outlier detection unit 53 based on the learning data and the noise-added learning data, and learns a machine learning model. Therefore, the uncertainty learning device 2 can provide a machine learning model that is capable of inferring uncertainty more accurately than a machine learning model created by a conventional method of adding noise to all learning data to learn a machine learning model.

[0083] Embodiment 2. Even if multiple pieces of operation-related data obtained from a machine have the same value, the conditions under which the multiple pieces of operation-related data were obtained may be different. For example, when a machine tool processes two different metal sheets made of materials A and B, the machine tool feeds the material at the same speed, but the behavior may differ due to the difference in materials. For example, if a machine learning model trained based on operation-related data obtained when processing material A is used to infer predicted values ​​and uncertainty from operation-related data obtained when processing material B, the inferred uncertainty will be low. This is because the operation-related data obtained when processing material A and the operation-related data obtained when processing material B are obtained when material A or material B is processed at the same speed, the machine learning model erroneously recognizes the data as the same data, i.e., as previously learned data, and infers low uncertainty. Embodiment 1 did not take the above into consideration. Embodiment 2 describes an embodiment in which a machine learning model is created that uses data obtained by an external sensor that cannot be obtained by the machine, enabling inferences with high uncertainty even when certain explanatory variables have the same value because the data obtained by the external sensor are different.

[0084] 8 is a diagram showing an example configuration of an uncertainty learning system 100a including an uncertainty learning device 2a according to embodiment 2. The uncertainty learning system 100a includes an uncertainty learning device 2a, a machine 1, and an external sensor 7. The uncertainty learning device 2a is connected to the machine 1 and the external sensor 7 via a network. In embodiment 2, the machine 1 is assumed to be, for example, an FA device. The external sensor 7 is a variety of sensors such as a temperature sensor, humidity sensor, or vibration sensor that are provided outside the machine 1 and independently of the machine 1.

[0085] The uncertainty learning device 2a according to the second embodiment is provided in, for example, a server. Regarding the configuration example of the uncertainty learning device 2a according to the second embodiment, components similar to those of the configuration example of the uncertainty learning device 2 according to the first embodiment, which have already been described using FIG. 3, are denoted by the same reference numerals, and redundant description will be omitted. The configuration example of the uncertainty learning device 2a differs from the configuration example of the uncertainty learning device 2 according to the first embodiment in that the acquisition unit 3a includes a sensor data acquisition unit 33 and a sensor data storage unit 34 in addition to a driving result acquisition unit 31 and a driving result storage unit 32. The sensor data acquisition unit 33 functions during learning and inference. Note that, in the second embodiment, the uncertainty learning device 2a, like the uncertainty learning device 2 according to the first embodiment, performs "learning" to create a machine learning model and "inference" to infer a predicted value and uncertainty using the machine learning model created by "learning."

[0086] The sensor data acquiring unit 33 acquires data (hereinafter referred to as "sensor data") acquired by the external sensor 7 from the external sensor 7. The sensor data acquired by the sensor data acquiring unit 33 from the external sensor 7 includes, for example, at least one of temperature data of the location where the mechanical device 1 is installed, humidity data of the location where the mechanical device 1 is installed, vibration data of the mechanical device 1, data related to the state of the mechanical device 1, and data related to the task of the mechanical device 1. For example, in the case of a processing machine, examples of sensor data include data indicating the temperature, vibration, shape of the workpiece, etc. of the mechanical device 1, which affect the smoothness of the processed shape or the processing speed.

[0087] The sensor data acquiring unit 33 stores the acquired sensor data in the sensor data storage unit 34. For example, the sensor data acquiring unit 33 adds the acquisition date and time of the sensor data to the sensor data and stores the sensor data in the sensor data storage unit 34.

[0088] The sensor data storage unit 34 stores the sensor data acquired by the sensor data acquisition unit 33. Note that, although the sensor data storage unit 34 is assumed to be provided in the uncertainty learning device 2a here, this is merely an example. The sensor data storage unit 34 may also be provided in a location outside the uncertainty learning device 2 that can be referenced by the uncertainty learning device 2a.

[0089] In the second embodiment, the preprocessing execution unit 41 of the preprocessing unit 4 acquires the driving-related data acquired by the driving result acquisition unit 31 from the driving result storage unit 32, and acquires the sensor data acquired by the sensor data acquisition unit 33 from the sensor data storage unit 34. Then, the preprocessing execution unit 41 performs learning preprocessing to shape learning data to be used by the learning unit 5 and test data to be used by the evaluation unit 64 of the inference unit 6, based on the driving-related data and the sensor data. More specifically, the preprocessing execution unit 41 adds sensor data obtained by the external sensor 7 to the explanatory variables. Furthermore, the preprocessing execution unit 41 acquires the driving-related data acquired by the driving result acquisition unit 31 from the driving result storage unit 32, and acquires the sensor data acquired by the sensor data acquisition unit 33 from the sensor data storage unit 34, and performs preprocessing to shape model input data to be used by the prediction unit 62 of the inference unit 6, i.e., inference preprocessing, based on the driving-related data and the sensor data.

[0090] The learning unit 5 creates a machine learning model based on the learning data that includes explanatory variables based on the sensor data. The specific method for creating the machine learning model by the learning unit 5 is the same as the specific creation method described in embodiment 1, so a duplicated description will be omitted. The learning unit 5 creates one machine learning model based on the learning data created by the preprocessing execution unit 41 based on the driving-related data and the sensor data.

[0091] Note that this is merely an example. For example, if the sensor data acquisition unit 33 acquires several pieces of sensor data with different contents during operation of the machine device 1, the learning unit 5 may create a machine learning model for each piece of sensor data. In this case, for example, the preprocessing execution unit 41 creates learning data and test data grouped based on the contents of the sensor data during preprocessing during learning. The learning unit 5 creates a machine learning model for each group based on the learning data. Note that when creating a machine learning model for each piece of sensor data, the preprocessing execution unit 41 simply uses the sensor data to group the learning data and test data based on the driving-related data, and does not need to add the sensor data to the explanatory variables. As a specific example, assume that data indicating various types of machining shapes (e.g., circles or straight lines) are acquired as sensor data. In this case, the preprocessing execution unit 41 creates learning data and test data based on the driving-related data for each type of machining shape, and associates the created learning data and test data with sensor data indicating the type of machining shape. The learning unit 5 creates a machine learning model for each type of machining shape. For example, in the case where the operation pattern (operation mode) of the mechanical device 1 changes depending on the type of machining shape, by creating a machine learning model for each type of machining shape, the uncertainty learning device 2a can provide a machine learning model corresponding to the operation pattern of the mechanical device 1. In the learning unit 5, the model learning unit 54 associates the created machine learning model with sensor data (in the above example, data indicating the type of machining shape) and stores the associated data in the model storage unit 55.

[0092] In the inference unit 6, the model reading unit 61 reads a machine learning model corresponding to test data grouped based on the content of the sensor data. The model reading unit 61 may, for example, acquire test data from the test data acquisition unit 63 and identify a machine learning model to read from the sensor data linked to the test data. Note that the arrow from the test data acquisition unit 63 to the model reading unit 61 is omitted in FIG. 8 . The evaluation unit 64 of the inference unit 6 evaluates a machine learning model corresponding to the sensor data linked to the test data using the test data acquired by the test data acquisition unit 63. The prediction unit 62 of the inference unit 6 inputs the model input data created by the preprocessing execution unit 41 in the inference preprocessing into the machine learning model corresponding to the sensor data linked to the model input data to obtain a predicted value and uncertainty, thereby inferring the predicted value and uncertainty. Note that when a machine learning model is created for each content of the sensor data, the preprocessing execution unit 41 creates model input data for each content of the sensor data in the inference preprocessing, links the created model input data to the sensor data, and outputs the linked model input data to the prediction unit 62.

[0093] The operation of the uncertainty learning device 2a according to the second embodiment will be described. First, the operation of the uncertainty learning device 2a according to the second embodiment during learning will be described. FIG. 9 is a flowchart for explaining the operation of the uncertainty learning device 2a according to the second embodiment during learning. Note that the operation of the uncertainty learning device 2a during learning shown in FIG. 9 is the operation when the learning unit 5 in the uncertainty learning device 2a creates one machine learning model. The uncertainty learning device 2a performs the operation during learning as shown in the flowchart of FIG. 9 based on, for example, instructions from a user. For example, the user operates an input device (not shown) to input an operation start instruction. Upon receiving the operation start instruction, the control unit (not shown) of the uncertainty learning device 2a causes the driving result acquisition unit 31, the sensor data acquisition unit 33, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the model reading unit 61, the test data acquisition unit 63, and the evaluation unit 64 to start operating. For example, a user may input an operation start instruction by inputting driving-related data. The uncertainty learning device 2a repeats the learning operation shown in the flowchart of FIG. 9, for example, until the machine learning model is evaluated to a certain extent or until the control unit receives an operation end instruction from the user. The specific contents of the processes of steps ST3 to ST4 in FIG. 9 are the same as the specific contents of the processes of steps ST3 to ST4 in FIG. 4, which have been described in embodiment 1, and therefore redundant explanations will be omitted. In addition, the process of step ST3 in FIG. 9 in the uncertainty learning device 2a is, in detail, the process shown in the flowchart of FIG. 5. The operation of the uncertainty learning device 2a, more specifically, the model learning unit 54, shown in the flowchart of FIG. 5 has been described in embodiment 1, and therefore redundant explanations will be omitted.

[0094] The driving result acquisition unit 31 acquires driving-related data from the machine 1 (step ST1a). The driving result acquisition unit 31 stores the acquired driving-related data in the driving result storage unit 32.

[0095] The sensor data acquiring unit 33 acquires sensor data from the external sensor 7 (step ST1b). The sensor data acquiring unit 33 stores the acquired sensor data in the sensor data storage unit .

[0096] The preprocessing execution unit 41 acquires the driving-related data acquired by the driving result acquisition unit 31 from the driving result storage unit 32 in step ST1a, and acquires the sensor data acquired by the sensor data acquisition unit 33 from the sensor data storage unit 34 in step ST1b. The preprocessing execution unit 41 performs preprocessing, i.e., learning preprocessing, to shape the learning data to be used by the learning unit 5 and the test data to be used by the inference unit 6 based on the driving-related data and the sensor data (step ST2a). The preprocessing execution unit 41 stores the learning data created by performing the learning preprocessing in the learning data storage unit 42. The preprocessing execution unit 41 also stores the test data created by performing the learning preprocessing in the test data storage unit 43.

[0097] For example, when the learning unit 5, more specifically the model learning unit 54 of the learning unit 5, creates a machine learning model for each piece of sensor data, the preprocessing execution unit 41 creates learning data and test data grouped based on the type of sensor data in learning preprocessing in step ST2a. The learning unit 5 creates a machine learning model for each piece of sensor data in step ST3. The model reading unit 61 of the inference unit 6 reads the machine learning model corresponding to the sensor data in step ST4, and the evaluation unit 64 evaluates the machine learning model corresponding to the sensor data linked to the test data using the test data acquired by the test data acquisition unit 63.

[0098] Next, the operation of the uncertainty learning device 2a according to the second embodiment during inference will be described. FIG. 10 is a flowchart for explaining the operation of the uncertainty learning device 2a according to the second embodiment during inference. Note that the operation of the uncertainty learning device 2a shown in FIG. 10 during inference is the operation when the learning unit 5 in the uncertainty learning device 2a creates one machine learning model. The uncertainty learning device 2a performs the operation during learning as shown in the flowchart of FIG. 10 based on, for example, a user's instruction. For example, the user operates the input device to input an instruction to start operation. Upon receiving the instruction to start operation, the control unit of the uncertainty learning device 2a causes the driving result acquisition unit 31, the sensor data acquisition unit 33, the preprocessing execution unit 41, the model reading unit 61, and the prediction unit 62 to start operation. The uncertainty learning device 2a repeats the operation during inference as shown in the flowchart of FIG. 10 until, for example, the control unit receives an instruction to end operation from the user. Note that the operation of the uncertainty learning device 2a during inference, which will be explained using the flowchart of Fig. 10, is premised on the fact that the operation of the uncertainty learning device 2a during learning, which was explained using the flowchart of Fig. 9, has been executed before the operation in question is executed. The specific contents of the processing of steps ST30 to ST50 in Fig. 10 are the same as the specific contents of the processing of steps ST30 to ST50 in Fig. 6, which have been explained in the first embodiment, and therefore redundant explanations will be omitted.

[0099] The driving result acquisition unit 31 acquires driving-related data from the machine 1 (step ST10a). The driving result acquisition unit 31 stores the acquired driving-related data in the driving result storage unit 32.

[0100] The sensor data acquiring unit 33 acquires sensor data from the external sensor 7 (step ST10b). The sensor data acquiring unit 33 stores the acquired sensor data in the sensor data storage unit .

[0101] The preprocessing execution unit 41 acquires the driving-related data acquired by the driving result acquisition unit 31 in step ST10a from the driving result storage unit 32, and acquires the sensor data acquired by the sensor data acquisition unit 33 in step ST10b from the sensor data storage unit 34, and performs preprocessing to shape model input data to be used in the prediction unit 62 of the inference unit 6 based on the driving-related data and the sensor data, i.e., inference preprocessing (step ST20a). The preprocessing execution unit 41 outputs the model input data created by performing inference preprocessing to the prediction unit 62.

[0102] 10, the processing is performed in the order of steps ST10a and ST10b, step ST20a, and step ST30, but this is merely an example. For example, the processing of step ST30 may be performed before the processing of steps ST10a and ST10b, or may be performed in parallel with the processing of steps ST10a and ST10b to ST20.

[0103] Furthermore, for example, when the learning unit 5, more specifically the model learning unit 54 of the learning unit 5, creates a machine learning model for each content of sensor data, the preprocessing execution unit 41 creates model input data for each content of sensor data in step ST20a, links the created model input data to the sensor data, and outputs the linked data to the prediction unit 62. In step ST40, the model reading unit 61 of the inference unit 6 reads in a machine learning model corresponding to the sensor data, and in step ST50, the prediction unit 62 inputs the model input data created by the preprocessing execution unit 41 into the machine learning model corresponding to the sensor data linked to the model input data to obtain a predicted value and uncertainty, thereby inferring the predicted value and uncertainty.

[0104] In this way, the uncertainty learning device 2a acquires sensor data acquired by an external sensor 7 provided independently of the machine 1, and acquires learning data created based on the driving-related data and the sensor data obtained from the machine 1. The uncertainty learning device 2a adds noise to the learning data created based on the driving-related data and the sensor data, and calculates an outlier score from the learning data and the noise-added learning data after the noise has been added. The uncertainty learning device 2a then calculates a loss function weighted based on the calculated outlier score based on the learning data and the noise-added learning data, and learns a machine learning model. The uncertainty learning device 2a creates a machine learning model that can recognize differences in driving results, in other words, differences in the driving-related data that arise due to differences in the conditions when the driving-related data was obtained, thereby providing a machine learning model with improved uncertainty inference accuracy.

[0105] In the second embodiment described above, the uncertainty learning device 2a includes the acquisition unit 3a, the preprocessing unit 4, and the inference unit 6, but this is merely an example. The uncertainty learning device 2a is required to include at least the learning unit 5. For example, the acquisition unit 3a, the preprocessing unit 4, and the inference unit 6 may be provided outside the uncertainty learning device 2a in a location that can be referenced by the uncertainty learning device 2a. If the uncertainty learning device 2a is configured without the acquisition unit 3a, the preprocessing unit 4, and the inference unit 6, the processes of steps ST1a, ST1b to ST2, and ST4 can be omitted from the operation of the uncertainty learning device 2a described using the flowchart of FIG. 9. Furthermore, the operation described using the flowchart of FIG. 10 can be omitted from the uncertainty learning device 2a.

[0106] In the second embodiment, the uncertainty learning device 2a is provided in a server, but this is merely an example. For example, the uncertainty learning device 2a may be provided in the machine 1. In addition, for example, in the uncertainty learning device 2a, some or all of the driving result acquisition unit 31, the sensor data acquisition unit 33, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, or the evaluation unit 64 may be provided in a device external to the server.

[0107] In the second embodiment, the machine 1 is an FA device, but this is merely an example. For example, the machine 1 can be a control device that controls the automatic operation of a moving object, or a medical device used in a medical setting, or any other device that solves various tasks using a machine learning model.

[0108] The hardware configuration of the uncertainty learning device 2a according to the second embodiment is the same as that shown in Figures 7A and 7B in the first embodiment, and is therefore not shown in the drawings. In the second embodiment, the functions of the driving result acquisition unit 31, the sensor data acquisition unit 33, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, and a control unit (not shown) are realized by a processing circuit 1001. That is, the uncertainty learning device 2a learns to make the extrapolated data highly uncertain and the data close to the learning data less uncertain using a weight γ set based on the outlier score, thereby enabling more accurate inference of uncertainty. The processing circuit 1001 also controls the creation of a machine learning model that can recognize differences in driving-related data that arise due to differences in the conditions under which the driving-related data were obtained. The processing circuitry 1001 may be dedicated hardware as shown in FIG. 7A, or may be a processor 1004 that executes a program stored in memory as shown in FIG. 7B.

[0109] The processing circuit 1001 reads and executes the programs stored in the memory 1005, thereby executing the functions of the driving result acquisition unit 31, the sensor data acquisition unit 33, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, and a control unit (not shown). That is, the uncertainty learning device 2a includes the memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of step ST1a and step ST1b to step ST4 in FIG. 9 or step ST10a, ST10b to step ST50 in FIG. 10. The program stored in the memory 1005 can also be said to cause a computer to execute the procedures or methods of the driving result acquisition unit 31, the sensor data acquisition unit 33, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, and a control unit (not shown). The driving result storage unit 32, the sensor data storage unit 34, the learning data storage unit 42, the test data storage unit 43, and the model storage unit 55 are configured, for example, with an HDD or SSD. The uncertainty learning device 2a also includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the mechanical device 1.

[0110] As described above, the uncertainty learning device 2a according to the second embodiment includes a sensor data acquisition unit 33 that acquires sensor data acquired by an external sensor 7 provided independently of the machine device 1, and a learning data acquisition unit 51 that acquires learning data created based on driving-related data obtained from the machine device 1 and the sensor data acquired by the sensor data acquisition unit 33. In the uncertainty learning device 2a, the model learning unit 54 calculates a loss function weighted based on the outlier score calculated by the outlier detection unit 53 based on the learning data created based on the driving-related data and the sensor data and the noise-added learning data, and trains a machine learning model. Therefore, the uncertainty learning device 2a can provide a machine learning model that can more accurately infer uncertainty compared to a machine learning model created using a conventional method in which noise is added to all learning data to train the machine learning model. Furthermore, the uncertainty learning device 2a can provide a machine learning model with improved uncertainty inference accuracy by creating a machine learning model that can recognize differences in driving results, in other words, differences in driving-related data that arise due to differences in the conditions under which the driving-related data were obtained.

[0111] Embodiment 3. It may be possible to obtain data obtained in advance by running a simulator for simulation (hereinafter referred to as "simulation data") or driving-related data from past operations of the machine 1 without operating the machine 1. In embodiment 3, an embodiment will be described in which a machine learning model is created using simulation data or past driving-related data that has been acquired in advance, and a machine learning model is created based on the machine learning model by transfer learning to learning data created from driving-related data obtained by operating the machine 1. In the following embodiment 3, the simulation data or past driving-related data acquired in advance is referred to as "pre-data," and the machine learning model created using the pre-data is referred to as a "pre-learning model."

[0112] 11 is a diagram showing an example of the configuration of an uncertainty learning system 100b including an uncertainty learning device 2b according to embodiment 3. The uncertainty learning system 100b includes an uncertainty learning device 2b and a machine 1. The uncertainty learning device 2b is connected to the machine 1 via a network. In embodiment 3, the machine 1 is assumed to be, for example, an FA device.

[0113] The uncertainty learning device 2b according to the third embodiment is provided in, for example, a server. Regarding the configuration example of the uncertainty learning device 2b according to the third embodiment, components similar to those of the configuration example of the uncertainty learning device 2 according to the first embodiment, which have already been described using FIG. 3, are designated by the same reference numerals, and redundant description will be omitted. The configuration example of the uncertainty learning device 2b differs from the configuration example of the uncertainty learning device 2 according to the first embodiment in that it includes a pre-data acquisition unit 8, and in that the learning unit 5a includes a pre-learning unit 56 in addition to the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, and the model storage unit 55. The pre-data acquisition unit 8 and the pre-learning unit 56 function during learning. Note that, in the third embodiment, the uncertainty learning device 2b, like the uncertainty learning device 2 according to the first embodiment, performs "learning" to create a machine learning model and "inference" to infer a predicted value and uncertainty using the machine learning model created by "learning."

[0114] The pre-data acquisition unit 8 acquires pre-data. For example, simulation data obtained by running a simulator (not shown) to perform a simulation operation, or operation-related data acquired when the machine 1 was previously operated, is stored on a cloud (not shown) as pre-data. The pre-data acquisition unit 8 checks whether the pre-data is stored on the cloud, and acquires the pre-data if it is stored. The simulator simulates the operation or behavior of the machine 1. The simulation data acquired by running the simulator to perform a simulation operation includes, for example, parameters set in the machine 1, the configuration of the machine 1, the operation mode, and other data indicating values ​​specific to the machine 1, or log data of the operation of the machine 1. The pre-data acquisition unit 8 outputs the acquired pre-data to the pre-processing execution unit 41 of the pre-processing unit 4.

[0115] In the third embodiment, when prior data is output from the prior data acquisition unit 8, the preprocessing execution unit 41 acquires the prior data and performs learning preprocessing, which shapes learning data to be used by the prior learning unit 56 of the learning unit 5a, based on the prior data. Details of the preprocessing unit 56 will be described later. Furthermore, the preprocessing execution unit 41 acquires driving-related data acquired by the driving result acquisition unit 31 from the driving result storage unit 32, and performs learning preprocessing, which shapes learning data to be used by the model learning unit 54 of the learning unit 5a and test data to be used by the model learning unit 54 of the inference unit 6, based on the driving-related data. Details of the learning preprocessing based on driving-related data performed by the preprocessing execution unit 41 have been described in the first embodiment, so repeated description will be omitted. Furthermore, the preprocessing execution unit 41 may shape learning data based on prior data in a manner similar to the method for shaping learning data based on driving-related data.

[0116] The preprocessing execution unit 41 adds data indicating that the learning data is based on the prior data to the learning data created by performing learning preprocessing based on the prior data, and stores the resulting data in the learning data storage unit 42. In the following third embodiment, learning data based on the prior data is also referred to as "pre-learning data." When the preprocessing execution unit 41 creates learning data by performing learning preprocessing based on driving-related data, the preprocessing execution unit 41 adds data indicating that the learning data is based on driving-related data to the created learning data, and stores the resulting data in the learning data storage unit 42. In addition, the preprocessing execution unit 41 stores test data created by performing learning preprocessing based on driving-related data in the test data storage unit 43.

[0117] Note that details of the inference preprocessing performed by the preprocessing execution unit 41 in embodiment 3 are similar to the details of the inference preprocessing already explained in embodiment 1, and therefore a duplicated explanation will be omitted. In the inference preprocessing, the preprocessing execution unit 41 does not require prior data. The preprocessing execution unit 41 outputs the model input data created by performing the inference preprocessing to the prediction unit 62.

[0118] In the third embodiment, the learning data acquiring unit 51 of the learning unit 5 a checks whether pre-learning data is stored in the learning data storage unit 42. If pre-learning data is stored, the learning data acquiring unit 51 acquires the pre-learning data and outputs the acquired pre-learning data to the pre-learning unit 56. The learning data acquiring unit 51 also acquires learning data from the learning data storage unit 42 and outputs the acquired learning data to the noise adding unit 52, the outlier detection unit 53, and the model learning unit 54.

[0119] The pre-learning unit 56 of the learning unit 5a causes a pre-learning model to learn the relationship between the explanatory variables and the target variable based on the pre-learning data output from the learning data acquisition unit 51. It is preferable that the pre-learning model and the machine learning model learned by the model learning unit 54 are the same type of machine learning model. The pre-learning unit 56 outputs the learned pre-learning model to the model learning unit 54. The model learning unit 54 stores the pre-learning model in an internal buffer or the like.

[0120] In the third embodiment, the model learning unit 54 of the learning unit 5a reads a pre-learning model stored in an internal buffer or the like, calculates a loss function using the NCP Loss weight γ set by the outlier detection unit 53, and transfer learns a machine learning model. More specifically, the model learning unit 54 reads a pre-learning model stored in an internal buffer or the like based on the training data acquired by the training data acquisition unit 51 and the noise-added training data created by the noise adding unit 52, calculates a loss function using the weight γ set by the outlier detection unit 53, and trains a machine learning model. Note that the calculation of the loss function using the NCP Loss weight γ has already been described in the first embodiment, and therefore a duplicated description will be omitted. Furthermore, the details of the noise adding unit 52 and the outlier detection unit 53 are the same as the details of the noise adding unit 52 and the outlier detection unit 53 described in the first embodiment, and therefore a duplicated description will be omitted.

[0121] The transfer learning performed by the model learning unit 54 is a known technique, but a brief description of transfer learning will be given here. FIG. 12 is a diagram for explaining an example of transfer learning. In FIG. 12, FIG. 12A shows an example of Fine Tuning, FIG. 12B shows an example of Feature Extraction, and FIG. 12C shows an example of Joint Training. Transfer learning is a technique for adapting a model trained in one domain to another domain. Specifically, it is an effective method for adapting a model trained in a domain where a wide range of data can be acquired to a domain where there is little data, or for adapting a model trained in a simulator to a real environment. Examples of transfer learning, as shown in FIG. 12, include Fine Tuning, which learns only new data, Feature Extraction, which adds a new layer to a previously trained model and trains it, and Joint Training, which can handle multiple tasks. Note that this is merely an example, and the model learning unit 54 may perform transfer learning using a transfer learning method other than these methods. When learning, the model learning unit 54 can arbitrarily change the layer of the neural network to be learned, and may learn all layers or may focus on one layer.

[0122] Here, the details of the outlier detection unit 53 are the same as those of the outlier detection unit 53 described in the first embodiment. That is, the outlier detection unit 53 inputs the learning data acquired by the learning data acquisition unit 51, more specifically, the learning data created based on driving-related data, into the outlier detection method to learn the outlier detection method. After learning the outlier detection method, the outlier detection unit 53 inputs the noise-added learning data created by the noise addition unit 52 into the learned outlier detection method and calculates an outlier score. However, this is merely an example. For example, the outlier detection unit 53 may use pre-learning data to train the outlier detection method. Specifically, the outlier detection unit 53 trains the outlier detection method using data including the learning data and the pre-learning data. Then, the outlier detection unit 53 inputs the noise-added learning data created by the noise addition unit 52 into the learned outlier detection method and calculates an outlier score. In this case, the learning data acquisition unit 51 outputs the pre-learning data acquired from the learning data storage unit 42 to the pre-learning unit 56 and the outlier detection unit 53.

[0123] In a case where there is a large amount of prior data and a small amount of target data, i.e., driving-related data acquired from the machine device 1, if the outlier detection method is trained using learning data based on a small amount of data, when the model training unit 54 creates a machine learning model, the model will determine that data that is actually close to the learning data (more specifically, the learning data after adding noise) is out of line simply because it is not included in the small amount of learning data based on the small amount of driving-related data used for training, and the learning of uncertainty will not be controlled correctly. The outlier detection unit 53 trains the outlier detection method using data including a large amount of data and a small amount of data, and calculates an outlier score for the data with noise added to the small amount of data, thereby preventing the occurrence of an event such as the above in which the learning of uncertainty is not controlled correctly.

[0124] The operation of the uncertainty learning device 2b according to the third embodiment will be described. First, the operation of the uncertainty learning device 2b according to the third embodiment during learning will be described. FIG. 13 is a flowchart for describing the operation of the uncertainty learning device 2b according to the third embodiment during learning. The uncertainty learning device 2b performs the operation during learning as shown in the flowchart of FIG. 13, for example, based on instructions from a user. For example, the user operates an input device (not shown) to input an operation start instruction. Upon receiving the operation start instruction, the control unit (not shown) of the uncertainty learning device 2b causes the driving result acquisition unit 31, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the pre-learning unit 56, the model reading unit 61, the test data acquisition unit 63, the evaluation unit 64, and the pre-data acquisition unit 8 to start operating. For example, the user may input driving-related data or pre-data to input the operation start instruction. The uncertainty learning device 2b repeats the learning operation shown in the flowchart of Fig. 13 until, for example, the machine learning model is evaluated to a certain extent or until the control unit receives an instruction to end the operation from the user. The specific contents of the processing of steps ST1 to ST2 and step ST4 in Fig. 13 are the same as the specific contents of the processing of steps ST1 to ST2 and step ST4 in Fig. 4, which have been explained in the first embodiment, and therefore, redundant explanations will be omitted.

[0125] The pre-data acquisition unit 8 determines whether a pre-training model has been created (step ST101). For example, a pre-training model presence flag indicating whether a pre-training model has been created is set in a location that can be referenced by each component of the uncertainty learning device 2b. The pre-training model presence flag indicates, for example, "1: pre-training model present" or "0: pre-training model absent," and its initial value is "0." In the uncertainty learning device 2b, when the pre-training unit 56 creates a pre-training model, it sets the pre-training model presence flag to "1." For example, the pre-data acquisition unit 8 can determine whether a pre-training model has been created by referring to the pre-training model presence flag.

[0126] If the prior data acquisition unit 8 determines that the prior learning model has been created ("YES" in step ST101), the operation of the uncertainty learning device 2b proceeds to the processing of step ST1.

[0127] If it is determined that a pre-learning model has not been created ("NO" in step ST101), the pre-data acquisition unit 8 checks whether pre-data is stored on the cloud (step ST102). If pre-data is stored on the cloud ("YES" in step ST102), the pre-data acquisition unit 8 acquires the pre-data (step ST103). The pre-data acquisition unit 8 outputs the acquired pre-data to the pre-processing execution unit 41 of the pre-processing unit 4.

[0128] The pre-processing execution unit 41 acquires the pre-data output from the pre-data acquisition unit 8 in step ST103, and performs pre-processing, i.e., pre-learning pre-processing, to shape pre-learning data to be used by the pre-learning unit 56 of the learning unit 5 a based on the pre-data (step ST104). The pre-processing execution unit 41 adds data indicating that the pre-learning data is based on the pre-data to the pre-learning data, and stores the data in the learning data storage unit 42.

[0129] The learning data acquisition unit 51 acquires the pre-learning data stored in the learning data storage unit 42 by the pre-processing execution unit 41 in step ST104, and the pre-learning unit 56 causes the pre-learning model to learn the relationship between the explanatory variables and the target variable based on the pre-learning data acquired by the learning data acquisition unit 51 (step ST105). The pre-learning unit 56 outputs the learned pre-learning model to the model learning unit 54. The model learning unit 54 stores the pre-learning model in an internal buffer or the like.

[0130] On the other hand, if no pre-data is stored on the cloud ("NO" in step ST102), the pre-data acquisition unit 8 sets the pre-training model creation impossible flag to "1". The pre-training model creation impossible flag is a flag indicating that a pre-training model cannot be created, and is set in a location that can be referenced by each component of the uncertainty learning device 2b. The pre-training model creation impossible flag indicates, for example, "1: pre-training model creation impossible" or "0: pre-training model creation possible", and the initial value of the pre-training model creation impossible flag is "0". Then, the operation of the uncertainty learning device 2b proceeds to the processing of step ST1.

[0131] In step ST3a, the learning unit 5a performs transfer learning to create a machine learning model based on the learning data stored in the learning data storage unit 42, the noise-added learning data created based on the learning data, and the pre-learning model created by the pre-learning unit 56 in step ST105 (step ST3a). The learning unit 5 stores the created machine learning model in the model storage unit 55.

[0132] Fig. 14 is a flowchart for explaining the details of the processing of step ST3a in Fig. 13. The specific contents of the processing of steps ST11 to ST16 and steps ST18 to ST19 in Fig. 14 are the same as the specific contents of the processing of steps ST11 to ST16 and steps ST18 to ST19 in Fig. 5, which have been explained in the first embodiment, and therefore redundant explanations will be omitted.

[0133] If a pre-training model has been created, the model learning unit 54 reads the pre-training model stored in an internal buffer or the like, calculates a loss function using the read pre-training model, the training data acquired in step ST13, the noise-added training data created in step ST14, and the NCP Loss weight γ set by the outlier detection unit 53 in step ST16, and transfer learns the machine learning model (step ST17a). If a pre-training model has not been created, the model learning unit 54 calculates a loss function using the training data acquired in step ST13, the noise-added training data created in step ST14, and the NCP Loss weight γ set by the outlier detection unit 53 in step ST16, and trains the machine learning model. The model learning unit 54 can determine whether a pre-training model has been created, for example, from a pre-training model presence flag and a pre-training model creation impossible flag. For example, when the pre-learning model existence flag is "0" and the pre-learning model creation inability flag is "0", the model learning unit 54 may suspend execution of the processing of step ST17a until the pre-learning model existence flag is set to "1", in other words, until a pre-learning model is created.

[0134] As described above, for example, the outlier detection unit 53 may use pre-learning data to train the outlier detection method. In this case, in step ST12 shown in the flowchart of FIG. 14 , the outlier detection unit 53 trains the outlier detection method using data including the training data and the pre-learning data. Then, in step ST15, the outlier detection unit 53 inputs the noise-added training data created by the noise-adding unit 52 in step ST14 to the outlier detection method trained in step ST12, and calculates an outlier score. In step ST11, the training data acquisition unit 51 outputs the pre-learning data acquired from the training data storage unit 42 to the outlier detection unit 53.

[0135] The operation of the uncertainty learning device 2b in embodiment 3 during inference is similar to the operation of the uncertainty learning device 2 in embodiment 1 during inference, which was explained using the flowchart of Figure 6 in embodiment 1, so duplicate explanations will be omitted.

[0136] In this way, the uncertainty learning device 2b acquires prior data based on simulation data obtained by simulating operation of the machine 1 or driving-related data related to past driving results of the machine 1, and learns a pre-learning model based on pre-learning data created based on the acquired prior data. The uncertainty learning device 2b then calculates a loss function weighted based on the calculated outlier score based on the training data and the noise-added training data, and performs transfer learning on the pre-learning model. A machine learning model may be unable to effectively infer predicted values ​​and uncertainty unless a certain amount of training data is available when the machine learning model is created. As a result, the inference accuracy may be poor when making inferences based on actually unknown data (here, model input data based on driving-related data). In contrast, the uncertainty learning device 2b utilizes past driving-related data obtained in advance or simulation data obtained from a simulator to create a pre-learning model for the machine learning model and perform transfer learning. This enables successful inference of predicted values ​​and uncertainty, even when the driving-related data obtained from the machine 1, which serves as the source of training data for creating the machine learning model, is small, making it possible to provide a highly accurate machine learning model.

[0137] In the third embodiment described above, the uncertainty learning device 2b includes the acquisition unit 3, the preprocessing unit 4, the inference unit 6, and the advance data acquisition unit 8. However, this is merely an example. The uncertainty learning device 2b is required to include at least the learning unit 5a. For example, the acquisition unit 3, the preprocessing unit 4, the inference unit 6, and the advance data acquisition unit 8 may be provided in a location outside the uncertainty learning device 2b that can be referenced by the uncertainty learning device 2b. If the uncertainty learning device 2b does not include the acquisition unit 3, the preprocessing unit 4, the inference unit 6, and the advance data acquisition unit 8, the processes of steps ST101 to ST105, steps ST1 to ST2, and steps ST4 to ST4 can be omitted from the operation of the uncertainty learning device 2b described using the flowchart of FIG. 13. Furthermore, the uncertainty learning device 2b can omit the inference operation shown in the flowchart of FIG. 6.

[0138] In the third embodiment, the uncertainty learning device 2b is provided in a server, but this is merely an example. For example, the uncertainty learning device 2b may be provided in the machine 1. In the uncertainty learning device 2b, for example, some or all of the driving result acquisition unit 31, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the pre-learning unit 56, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, or the pre-data acquisition unit 8 may be provided in a device external to the server.

[0139] In the third embodiment, the machine 1 is an FA device, but this is merely an example. For example, the machine 1 can be a control device that controls automatic driving of a moving object, or a medical device used in a medical setting, or any other device that solves various tasks using a machine learning model.

[0140] The hardware configuration of the uncertainty learning device 2b according to embodiment 3 is the same as that shown in FIGS. 7A and 7B in embodiment 1, and is therefore not shown. In embodiment 3, the functions of the driving result acquisition unit 31, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the pre-learning unit 56, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, the pre-data acquisition unit 8, and a control unit (not shown) are realized by a processing circuit 1001. That is, the uncertainty learning device 2b includes a processing circuit 1001 for controlling the creation of a pre-learning model based on pre-data, and the creation of a machine learning model capable of more accurate uncertainty inference by performing transfer learning using a weight γ set based on the outlier score so that extrapolated data has high uncertainty and data close to the learning data has low uncertainty. The processing circuit 1001 may be dedicated hardware as shown in FIG. 7A, or a processor 1004 that executes a program stored in memory as shown in FIG. 7B.

[0141] The processing circuit 1001 reads and executes the programs stored in the memory 1005, thereby executing the functions of the driving result acquisition unit 31, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the pre-learning unit 56, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, the pre-data acquisition unit 8, and a control unit (not shown). That is, the uncertainty learning device 2b includes the memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of steps ST101 to ST105, ST1 to ST4 in FIG. 13, or steps ST10 to ST50 in FIG. 6. The program stored in the memory 1005 can also be said to cause a computer to execute the processing procedures or methods of the driving result acquisition unit 31, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the pre-learning unit 56, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, the pre-data acquisition unit 8, and a control unit (not shown). The driving result storage unit 32, the learning data storage unit 42, the test data storage unit 43, and the model storage unit 55 are configured, for example, with an HDD or SSD. The uncertainty learning device 2b also includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the mechanical device 1.

[0142] As described above, the uncertainty learning device 2b according to the third embodiment includes a pre-data acquisition unit 8 that acquires pre-data based on simulation data obtained by simulating operation of the machine 1 or driving-related data related to past driving results of the machine 1, and a pre-learning unit 56 that learns a pre-learning model based on pre-learning data created based on the pre-data acquired by the pre-data acquisition unit 8. The model learning unit 54 is configured to calculate a loss function weighted based on the outlier score calculated by the outlier detection unit 53 based on the learning data and the noise-added learning data, and to create a machine learning model by transfer learning the pre-learning model learned by the pre-learning unit 56. The uncertainty learning device 2b creates a pre-learning model for a machine learning model by utilizing past driving-related data obtained in advance or simulation data obtained from a simulator. This enables successful inference of predicted values ​​and uncertainty, even when the driving-related data obtained from the machine 1 that serves as the source of learning data for creating the machine learning model is small, and a highly accurate machine learning model can be provided.

[0143] Embodiment 4. In embodiment 3, the uncertainty learning device creates a pre-learning model based on prior data, and creates a machine learning model by transfer learning using the created pre-learning model. In embodiment 4, an embodiment will be described in which transfer learning is performed using sensor data acquired by an external sensor that cannot be acquired by a mechanical device, to create a machine learning model.

[0144] 15 is a diagram showing an example configuration of an uncertainty learning system 100c including an uncertainty learning device 2c according to embodiment 4. The uncertainty learning system 100c includes an uncertainty learning device 2c, a machine 1, and an external sensor 7. The uncertainty learning device 2c is connected to the machine 1 and the external sensor 7 via a network. Note that the external sensor 7 has already been explained in embodiment 2, so a duplicate explanation will be omitted. In embodiment 4, the machine 1 is assumed to be, for example, an FA device.

[0145] The uncertainty learning device 2c according to the fourth embodiment is provided in, for example, a server. Regarding the exemplary configuration of the uncertainty learning device 2c according to the fourth embodiment, components similar to those of the exemplary configuration of the uncertainty learning device 2b according to the third embodiment, which have already been described using FIG. 11 , are designated by the same reference numerals, and redundant description will be omitted. The exemplary configuration of the uncertainty learning device 2c differs from the exemplary configuration of the uncertainty learning device 2b according to the third embodiment in that the acquisition unit 3a includes a sensor data acquisition unit 33 and a sensor data storage unit 34 in addition to a driving result acquisition unit 31 and a driving result storage unit 32. Details of the acquisition unit 3a are similar to the details of the acquisition unit 3a provided in the uncertainty learning device 2a described in the second embodiment, and therefore redundant description will be omitted. Note that, in the fourth embodiment, like the uncertainty learning device 2b according to the third embodiment, the uncertainty learning device 2c performs "learning" to create a machine learning model and "inference" to infer a predicted value and uncertainty using the machine learning model created by "learning."

[0146] In the fourth embodiment, when prior data is output from the prior data acquisition unit 8, the preprocessing execution unit 41 acquires the prior data and performs learning preprocessing based on the prior data to shape prior learning data to be used by the prior learning unit 56 of the learning unit 5a. The preprocessing execution unit 41 also acquires driving-related data acquired by the driving result acquisition unit 31 from the driving result storage unit 32 and sensor data acquired by the sensor data acquisition unit 33 from the sensor data storage unit 34, and performs learning preprocessing based on the driving-related data and the sensor data to shape learning data to be used by the model learning unit 54 of the learning unit 5 and test data to be used by the evaluation unit 64 of the inference unit 6. More specifically, the preprocessing execution unit 41 adds sensor data acquired by the external sensor 7 to the explanatory variables. Details of the learning preprocessing performed by the preprocessing execution unit 41 to shape the prior learning data based on the prior data and the learning preprocessing to shape the learning data and test data based on the driving-related data and the sensor data have already been described in the second or third embodiment, and therefore will not be described again.

[0147] The preprocessing execution unit 41 adds data indicating that the pre-learning data created by performing learning preprocessing based on the prior data to the pre-learning data, and stores the data in the learning data storage unit 42. Furthermore, when the preprocessing execution unit 41 creates learning data by performing learning preprocessing based on driving-related data, the preprocessing execution unit 41 adds data indicating that the learning data is based on driving-related data to the created learning data, and stores the data in the learning data storage unit 42. Furthermore, the preprocessing execution unit 41 stores test data created by performing learning preprocessing in the test data storage unit 43.

[0148] Note that details of the inference preprocessing performed by the preprocessing execution unit 41 in embodiment 4 are similar to the details of the inference preprocessing already explained in embodiment 2, and therefore redundant explanations will be omitted. In the inference preprocessing, the preprocessing execution unit 41 does not require prior data. The preprocessing execution unit 41 outputs the model input data created by performing the inference preprocessing to the prediction unit 62.

[0149] The model learning unit 54 of the learning unit 5a reads the pre-learning model created by the pre-learning unit 56, which has been stored in an internal buffer or the like, calculates a loss function using the NCP Loss weight γ set by the outlier detection unit 53 based on the learning data including explanatory variables based on the sensor data, and creates a machine learning model by transfer learning the machine learning model. Details of the pre-learning unit 56 have been described in embodiment 3, so a duplicated description will be omitted. Also, details of the transfer learning by the model learning unit 54 have been described in embodiment 3, so a duplicated description will be omitted. The model learning unit 54 creates one machine learning model based on the learning data created by the pre-processing execution unit 41 based on the driving-related data and the sensor data and one pre-learning model created by the pre-learning unit 56.

[0150] Note that this is merely an example. For example, the pre-learning unit 56 may create a pre-learning model for each piece of sensor data, and the model learning unit 54 may create a machine learning model by transfer learning the pre-learning model for each piece of sensor data. For example, if the pre-data acquisition unit 8 can acquire simulation data having the same content as the sensor data acquired by the sensor data acquisition unit 33 from the external sensor 7 as pre-data, the pre-learning unit 56 may create a pre-learning model for each piece of sensor data, and the model learning unit 54 may create a machine learning model by transfer learning the pre-learning model for each piece of sensor data. In this case, for example, the pre-processing execution unit 41 creates pre-learning data grouped based on data (simulation data) having the same content as the sensor data included in the pre-data during learning pre-processing. Hereinafter, the simulation data having the same content as the sensor data included in the pre-data is also referred to as "pre-data for model classification." The pre-processing execution unit 41 links the created pre-learning data to the pre-data for model classification. Furthermore, the pre-processing execution unit 41 creates learning data and test data grouped based on the content of the sensor data during learning pre-processing. The pre-processing execution unit 41 associates the sensor data with the created learning data and test data.

[0151] The pre-learning unit 56 creates a pre-learning model for each of the pre-learning data grouped by the pre-data for model classification. The pre-learning unit 56 associates the created pre-learning model with the pre-data for model classification linked to the pre-learning data from which the pre-learning model was created, and outputs the created pre-learning model to the model learning unit 54.

[0152] The model learning unit 54 loads a pre-learning model corresponding to the content of the sensor data linked to the learning data acquired from the learning data acquisition unit 51 as a pre-learning model to be used in transfer learning, calculates a loss function using the loaded pre-learning model and the NCP Loss weight γ set by the outlier detection unit 53, and transfer learns a machine learning model. The model learning unit 54 links the sensor data to the created machine learning model and stores the sensor data in the model storage unit 55. As described above, the pre-data acquisition unit 8 outputs the pre-learning model to the model learning unit 54 in association with the pre-data for model classification linked to the pre-learning data used to create the pre-learning model. Therefore, the model learning unit 54 can identify the pre-learning model corresponding to the content of the sensor data linked to the learning data based on the pre-learning model, based on the pre-data for model classification associated with the pre-learning model.

[0153] In the inference unit 6, the model reading unit 61 reads machine learning models corresponding to test data grouped based on the content of the sensor data. The evaluation unit 64 of the inference unit 6 evaluates the machine learning models corresponding to the sensor data linked to the test data using the test data acquired by the test data acquisition unit 63. The prediction unit 62 of the inference unit 6 inputs the model input data created by the preprocessing execution unit 41 in the inference preprocessing into the machine learning model corresponding to the sensor data linked to the model input data to obtain a predicted value and uncertainty, thereby inferring the predicted value and uncertainty. Note that when a machine learning model is created for each type of sensor data, the preprocessing execution unit 41 creates model input data for each content of the sensor data in the inference preprocessing, links the created model input data to the sensor data, and outputs the linked model input data to the prediction unit 62.

[0154] Also in the fourth embodiment, similarly to the third embodiment, the outlier detection unit 53 may use the pre-learning data for learning the outlier detection method.

[0155] The operation of the uncertainty learning device 2c according to the fourth embodiment will be described. First, the operation of the uncertainty learning device 2c according to the fourth embodiment during learning will be described. FIG. 16 is a flowchart for explaining the operation of the uncertainty learning device 2c according to the fourth embodiment. Note that the operation of the uncertainty learning device 2c shown in FIG. 16 is the operation when the learning unit 5a in the uncertainty learning device 2c creates one machine learning model. The uncertainty learning device 2c performs the operation during learning as shown in the flowchart of FIG. 16 based on, for example, instructions from a user. For example, the user operates an input device (not shown) to input an operation start instruction. Upon receiving the operation start instruction, the control unit (not shown) of the uncertainty learning device 2a causes the driving result acquisition unit 31, the sensor data acquisition unit 33, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the pre-learning unit 56, the model reading unit 61, the test data acquisition unit 63, the evaluation unit 64, and the pre-data acquisition unit 8 to start their operations. For example, a user may input an operation start instruction by inputting driving-related data. The uncertainty learning device 2c repeats the learning operation shown in the flowchart of FIG. 16, for example, until the machine learning model is evaluated to a certain extent or until the control unit receives an operation end instruction from the user. The specific contents of the processes of steps ST101 to ST105 and step ST3a in FIG. 16 are respectively similar to the specific contents of the processes of steps ST101 to ST105 and step ST3a in FIG. 13, which have been described in embodiment 3, and therefore redundant explanations will be omitted. Furthermore, the specific contents of the processes of steps ST1a, ST1b to ST2a, and step ST4 in FIG. 16 are respectively similar to the specific contents of the processes of steps ST1a, ST1b to ST2a, and step ST4 in FIG. 9, which have been described in embodiment 2, and therefore redundant explanations will be omitted. Note that the process of step ST3a in FIG. 16 in the uncertainty learning device 2c is, in detail, the process shown in the flowchart of FIG. 14.The operation of the uncertainty learning device 2c shown in the flowchart of FIG. 14, more specifically the operation of the model learning unit 54, has already been explained in the third embodiment, so a duplicate explanation will be omitted.

[0156] For example, when the learning unit 5a, more specifically the model learning unit 54 of the learning unit 5a, creates a machine learning model for each piece of sensor data, the preprocessing execution unit 41 creates pre-learning data in step ST104, which is grouped based on data with the same content as the sensor data included in the pre-data, in other words, pre-learning data for model classification, in pre-processing during learning. Furthermore, the preprocessing execution unit 41 creates learning data and test data grouped based on the content of the sensor data in pre-processing during learning, in step ST2a. The pre-learning unit 56 creates a pre-learning model for each group of pre-learning data in step ST105. The pre-learning unit 56 associates the created pre-learning model with the pre-learning data for model classification included in the pre-learning data from which the pre-learning model was created, and outputs the pre-learning model to the model learning unit 54. 14 , the model learning unit 54 loads a pre-learning model corresponding to the sensor data linked to the learning data acquired from the learning data acquisition unit 51 as a pre-learning model to be used in transfer learning, calculates a loss function using the loaded pre-learning model and the NCP Loss weight γ set by the outlier detection unit 53, and transfers the machine learning model. In step ST4, the model reading unit 61 of the inference unit 6 loads a machine learning model corresponding to the sensor data, and the evaluation unit 64 uses the test data acquired by the test data acquisition unit 63 to evaluate the machine learning model corresponding to the content of the sensor data linked to the test data.

[0157] The operation of the uncertainty learning device 2c in embodiment 4 during inference is similar to the operation of the uncertainty learning device 2a in embodiment 2 during inference, which was explained using the flowchart of Figure 10 in embodiment 2, so duplicate explanations will be omitted.

[0158] In this way, the uncertainty learning device 2c acquires simulation data related to the operating results obtained by simulating the operation of the machine 1 or prior data based on operating-related data related to past operating results of the machine 1, and learns a pre-learning model based on pre-learning data created based on the prior data. The uncertainty learning device 2c acquires sensor data acquired by an external sensor 7 provided independently of the machine 1, and acquires learning data created based on the operating-related data and the sensor data obtained from the machine 1. Then, the uncertainty learning device 2 calculates a loss function weighted based on the outlier score based on the learning data and the noise-added learning data, and creates a machine learning model by transfer learning the pre-learning model. The uncertainty learning device 2c can provide a machine learning model with improved accuracy in inferring uncertainty by creating a machine learning model that can recognize differences in driving results, in other words, differences in driving-related data that arise due to differences in the conditions when the driving-related data was obtained.In addition, by creating a pre-learning model for the machine learning model using past driving-related data that has been obtained in advance or simulation data obtained from a simulator, it is possible to successfully infer predicted values ​​and uncertainty even when the amount of driving-related data obtained from the machine device 1 that serves as the source of learning data when creating the machine learning model is small, and a highly accurate machine learning model can be provided.

[0159] Furthermore, the uncertainty learning device 2c may be configured to divide the pre-data into groups corresponding to data with the same content as the sensor data included in the pre-data, create grouped pre-learning data, and create a pre-learning model for each grouped pre-learning data. The uncertainty learning device 2c may then calculate a loss function weighted based on the outlier score based on the learning data created based on the operation-related data obtained from the machine 1 and the sensor data acquired from the external sensor 7, and the noise-added learning data, and create a machine learning model by transfer learning the pre-learning model corresponding to the sensor data. This enables the uncertainty learning device 2c to select a machine learning model appropriate for transfer learning, thereby providing a machine learning model with improved uncertainty inference accuracy.

[0160] In the fourth embodiment described above, the uncertainty learning device 2c includes the acquisition unit 3a, the preprocessing unit 4, the inference unit 6, and the prior data acquisition unit 8. However, this is merely an example. The uncertainty learning device 2c is required to include at least the learning unit 5a. For example, the acquisition unit 3a, the preprocessing unit 4, the inference unit 6, and the prior data acquisition unit 8 may be provided in a location outside the uncertainty learning device 2c that can be referenced by the uncertainty learning device 2c. If the uncertainty learning device 2c does not include the acquisition unit 3a, the preprocessing unit 4, the inference unit 6, and the prior data acquisition unit 8, the processes of steps ST101 to ST105, ST1a, ST1b to ST2a, and ST4 can be omitted from the operation of the uncertainty learning device 2c described using the flowchart of FIG. 16. Furthermore, the uncertainty learning device 2a can omit the inference operation described using the flowchart of FIG. 10.

[0161] In the fourth embodiment, the uncertainty learning device 2c is provided in a server, but this is merely an example. For example, the uncertainty learning device 2c may be provided in the machine 1. In the uncertainty learning device 2c, for example, some or all of the driving result acquisition unit 31, the sensor data acquisition unit 33, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the pre-learning unit 56, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, or the pre-data acquisition unit 8 may be provided in a device external to the server.

[0162] In the fourth embodiment, the machine 1 is an FA device, but this is merely an example. For example, the machine 1 can be a control device that controls the automatic driving of a moving object, or a medical device used in a medical setting, or any other device that solves various tasks using a machine learning model.

[0163] The hardware configuration of the uncertainty learning device 2c according to embodiment 4 is the same as that shown in FIG. 7A and FIG. 7B in embodiment 1, and is therefore not shown in the drawings. In embodiment 4, the functions of the driving result acquisition unit 31, the sensor data acquisition unit 33, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the pre-learning unit 56, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, the pre-data acquisition unit 8, and a control unit (not shown) are realized by a processing circuit 1001. That is, the uncertainty learning device 2c includes the processing circuit 1001 for controlling the creation of a machine learning model that creates a pre-learning model based on pre-data and performs transfer learning using a weight γ set based on the pre-learning model and the outlier score so that the extrapolated data has high uncertainty and the data close to the learning data has low uncertainty, thereby enabling more accurate inference of uncertainty and recognizing differences in the driving-related data that arise due to differences in the conditions under which the driving-related data were obtained. The processing circuitry 1001 may be dedicated hardware as shown in FIG. 7A, or may be a processor 1004 that executes a program stored in memory as shown in FIG. 7B.

[0164] The processing circuit 1001 reads and executes the programs stored in the memory 1005, thereby executing the functions of the driving result acquisition unit 31, the sensor data acquisition unit 33, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the pre-learning unit 56, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, the pre-data acquisition unit 8, and a control unit (not shown). That is, the uncertainty learning device 2c includes the memory 1005 for storing a program that, when executed by the processing circuit 1001, results in the execution of steps ST101 to ST105, ST1a, and ST1b to ST4 in FIG. 16, or steps ST10a, ST10b to ST50 in FIG. 10. The program stored in the memory 1005 can also be said to cause a computer to execute the procedures or methods of the driving result acquisition unit 31, the sensor data acquisition unit 33, the preprocessing execution unit 41, the learning data acquisition unit 51, the noise addition unit 52, the outlier detection unit 53, the model learning unit 54, the pre-learning unit 56, the model reading unit 61, the prediction unit 62, the test data acquisition unit 63, the evaluation unit 64, the pre-data acquisition unit 8, and a control unit (not shown). The driving result storage unit 32, the sensor data storage unit 34, the learning data storage unit 42, the test data storage unit 43, and the model storage unit 55 are configured, for example, with an HDD or SSD. The uncertainty learning device 2c also includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as the mechanical device 1.

[0165] As described above, the uncertainty learning device 2c according to the fourth embodiment includes a pre-data acquisition unit 8 that acquires simulation data relating to the operating results obtained by simulating the operation of the machine 1 or pre-data based on operating-related data relating to the past operating results of the machine 1, a pre-learning unit 56 that learns a pre-learning model based on pre-learning data created based on the pre-data acquired by the pre-data acquisition unit 8, and a sensor data acquisition unit 33 that acquires sensor data acquired by an external sensor 7 that is provided independently of the machine 1, and the learning data acquisition unit 51 acquires learning data created based on the operating-related data obtained from the machine 1 and the sensor data acquired by the sensor data acquisition unit 33, and the model learning unit 54 calculates a loss function weighted based on the outlier score calculated by the outlier detection unit 53 based on the learning data and the noise-added learning data, and creates a machine learning model by transfer learning the pre-learning model learned by the pre-learning unit 56. The uncertainty learning device 2c can provide a machine learning model with improved accuracy in inferring uncertainty by creating a machine learning model that can recognize differences in driving results, in other words, differences in driving-related data that arise due to differences in the conditions when the driving-related data was obtained.In addition, by creating a pre-learning model for the machine learning model using past driving-related data that has been obtained in advance or simulation data obtained from a simulator, it is possible to successfully infer predicted values ​​and uncertainty even when the amount of driving-related data obtained from the machine device 1 that serves as the source of learning data when creating the machine learning model is small, and a highly accurate machine learning model can be provided.

[0166] It should be noted that the embodiments may be freely combined, or any of the components in each embodiment may be modified, or any of the components in each embodiment may be omitted.

[0167] Various aspects of the present disclosure are summarized below as appendices.

[0168] (Supplementary Note 1) An uncertainty learning device that takes as input data based on driving-related data related to operating results of a machine, and creates a machine learning model that outputs a predicted value corresponding to the driving-related data and the uncertainty of the predicted value, the uncertainty learning device comprising: a learning data acquisition unit that acquires learning data created based on the driving-related data obtained from the machine, a noise addition unit that adds noise to the learning data acquired by the learning data acquisition unit to create noise-added learning data, an outlier detection unit that calculates an outlier score from the learning data acquired by the learning data acquisition unit and the noise-added learning data created by the noise addition unit, and a model learning unit that calculates a loss function weighted based on the outlier score calculated by the outlier detection unit, based on the learning data and the noise-added learning data, and learns the machine learning model. (Supplementary Note 2) The uncertainty learning device according to Supplementary Note 1, further comprising: a test data acquisition unit that acquires test data created based on the driving-related data obtained from the machine, and an evaluation unit that evaluates the machine learning model by inferring the predicted value and the uncertainty based on the test data acquired by the test data acquisition unit and the machine learning model created by the model learning unit. (Supplementary Note 3) The uncertainty learning device according to Supplementary Note 1, further comprising: a preprocessing execution unit that creates the learning data based on the driving-related data, wherein the learning data acquisition unit acquires the learning data created by the preprocessing execution unit. (Supplementary Note 4) The uncertainty learning device according to Supplementary Note 2, further comprising: a preprocessing execution unit that creates the learning data and the test data based on the driving-related data, wherein the learning data acquisition unit acquires the learning data created by the preprocessing execution unit, and the test data acquisition unit acquires the test data created by the preprocessing execution unit. (Supplementary Note 5) The uncertainty learning device according to any one of Supplementary Notes 1 to 4, further comprising: a preprocessing execution unit that creates the learning data and the test data based on the driving-related data, wherein the learning data acquisition unit acquires the learning data created by the preprocessing execution unit, and the test data acquisition unit acquires the test data created by the preprocessing execution unit.(Supplementary Note 6) The uncertainty learning device according to any one of Supplementary Note 1 to Supplementary Note 5, further comprising: a preliminary data acquisition unit that acquires preliminary data based on simulation data obtained by simulating operation of the mechanical device or the driving-related data related to the past driving results of the mechanical device; and a preliminary learning unit that learns a pre-learning model based on pre-learning data created based on the preliminary data acquired by the preliminary data acquisition unit, wherein the model learning unit calculates a loss function weighted based on the outlier score calculated by the outlier detection unit based on the learning data and the noise-added learning data, and creates the machine learning model by transfer learning of the pre-learning model learned by the pre-learning unit. (Supplementary Note 7) The uncertainty learning device according to any one of Supplementary Note 1 to Supplementary Note 5, further comprising: a sensor data acquisition unit that acquires sensor data acquired by an external sensor provided independently of the mechanical device, wherein the learning data acquisition unit acquires the learning data created based on the driving-related data obtained from the mechanical device and the sensor data acquired by the sensor data acquisition unit. (Supplementary Note 8) The uncertainty learning device according to any one of Supplementary Note 1 to Supplementary Note 5, comprising: a preliminary data acquisition unit that acquires simulation data on the operating results obtained by simulating the mechanical device, or preliminary data based on the operating-related data on the past operating results of the mechanical device; a preliminary learning unit that learns a pre-learning model based on preliminary learning data created based on the preliminary data acquired by the preliminary data acquisition unit; and a sensor data acquisition unit that acquires sensor data acquired by an external sensor provided independently of the mechanical device, wherein the learning data acquisition unit acquires the learning data created based on the operating-related data obtained from the mechanical device and the sensor data acquired by the sensor data acquisition unit, and the model learning unit calculates a loss function weighted based on the outlier score calculated by the outlier detection unit based on the learning data and the noise-added learning data, and creates the machine learning model by transfer learning the pre-learning model learned by the pre-learning unit.(Supplementary Note 9) The uncertainty learning device according to any one of Supplementary Note 1 to Supplementary Note 8, wherein the mechanical device is an factory automation device, and the operation-related data includes a command position, a command velocity, a command acceleration, a feedback velocity, a feedback acceleration, a current value, or a measured value of a deviation in a machining position. (Supplementary Note 10) An uncertainty learning program for creating a machine learning model that receives as input data based on operation-related data relating to an operation result of a mechanical device, and outputs a predicted value corresponding to the operation-related data and the uncertainty of the predicted value, causing a computer to function as: a learning data acquisition unit that acquires learning data created based on the operation-related data obtained from the mechanical device; a noise addition unit that creates noise-added learning data by adding noise to the learning data acquired by the learning data acquisition unit; an outlier detection unit that calculates an outlier score from the learning data acquired by the learning data acquisition unit and the noise-added learning data created by the noise addition unit; and a model learning unit that calculates a loss function weighted based on the outlier score calculated by the outlier detection unit, based on the learning data and the noise-added learning data, and learns the machine learning model. (Supplementary Note 11) An uncertainty learning system comprising: the uncertainty learning device according to any one of Supplementary Note 1 to Supplementary Note 9; and the machine device.

[0169] The uncertainty learning device of the present disclosure is a machine learning model that can infer predicted values ​​and uncertainty, and can provide a machine learning model that can infer uncertainty more accurately than a machine learning model created using a conventional method of adding noise to all training data to train the machine learning model.

[0170] 1 Mechanical device, 2, 2a, 2b, 2c Uncertainty learning device, 3, 3a Acquisition unit, 31 Driving result acquisition unit, 32 Driving result memory unit, 33 Sensor data acquisition unit, 34 Sensor data memory unit, 4 Preprocessing unit, 41 Preprocessing execution unit, 42 Learning data memory unit, 43 Test data memory unit, 5, 5a Learning unit, 51 Learning data acquisition unit, 52 Noise addition unit, 53 Outlier detection unit, 54 Model learning unit, 55 Model memory unit, 56 Pre-learning unit, 6 Inference unit, 61 Model reading unit, 62 Prediction unit, 63 Test data acquisition unit, 64 Evaluation unit, 7 External sensor, 8 Pre-data acquisition unit, 100, 100a, 100b, 100c Uncertainty learning system, 1001 Processing circuit, 1002 Input interface device, 1003 Output interface device, 1004 Processor, 1005 Memory.

Claims

1. An uncertainty learning device that takes as input data based on driving-related data related to operating results of a machine, and creates a machine learning model that outputs a predicted value corresponding to the driving-related data and the uncertainty of the predicted value, the uncertainty learning device comprising: a learning data acquisition unit that acquires learning data created based on the driving-related data obtained from the machine; a noise addition unit that creates noise-added learning data by adding noise to the learning data acquired by the learning data acquisition unit; an outlier detection unit that calculates an outlier score from the learning data acquired by the learning data acquisition unit and the noise-added learning data created by the noise addition unit; and a model learning unit that calculates a loss function weighted based on the outlier score calculated by the outlier detection unit, based on the learning data and the noise-added learning data, and learns the machine learning model.

2. An uncertainty learning device as described in claim 1, comprising: a test data acquisition unit that acquires test data created based on the operation-related data obtained from the mechanical device; and an evaluation unit that infers the predicted value and the uncertainty based on the test data acquired by the test data acquisition unit and the machine learning model created by the model learning unit, and evaluates the machine learning model.

3. The uncertainty learning device according to claim 1, further comprising a pre-processing execution unit that creates the learning data based on the driving-related data, and the learning data acquisition unit acquires the learning data created by the pre-processing execution unit.

4. The uncertainty learning device according to claim 2, further comprising a pre-processing execution unit that creates the learning data and the test data based on the driving-related data, wherein the learning data acquisition unit acquires the learning data created by the pre-processing execution unit, and the test data acquisition unit acquires the test data created by the pre-processing execution unit.

5. An uncertainty learning device according to any one of claims 1 to 4, characterized in that the uncertainty is expressed by a variance or a standard deviation indicating the dispersion of data.

6. An uncertainty learning device as described in any one of claims 1 to 5, characterized in that it comprises a pre-data acquisition unit that acquires pre-data based on simulation data obtained by simulating operation of the mechanical device or the operation-related data related to the past operating results of the mechanical device, and a pre-learning unit that learns a pre-learning model based on pre-learning data created based on the pre-data acquired by the pre-data acquisition unit, wherein the model learning unit calculates a loss function weighted based on the outlier score calculated by the outlier detection unit based on the learning data and the noise-added learning data, and creates the machine learning model by transfer learning the pre-learning model learned by the pre-learning unit.

7. An uncertainty learning device as described in any one of claims 1 to 5, characterized in that it is provided with a sensor data acquisition unit that acquires sensor data acquired by an external sensor provided independently of the mechanical device, and the learning data acquisition unit acquires the learning data created based on the driving-related data obtained from the mechanical device and the sensor data acquired by the sensor data acquisition unit.

8. An uncertainty learning device as described in any one of claims 1 to 5, characterized in that it comprises: a pre-data acquisition unit that acquires simulation data regarding the operation results obtained by simulating operation of the mechanical device, or pre-data based on the operation-related data regarding the past operation results of the mechanical device; a pre-learning unit that learns a pre-learning model based on pre-learning data created based on the pre-data acquired by the pre-data acquisition unit; and a sensor data acquisition unit that acquires sensor data acquired by an external sensor provided independently of the mechanical device, wherein the learning data acquisition unit acquires the learning data created based on the operation-related data acquired from the mechanical device and the sensor data acquired by the sensor data acquisition unit, and the model learning unit calculates a loss function weighted based on the outlier score calculated by the outlier detection unit based on the learning data and the noise-added learning data, and creates the machine learning model by transfer learning the pre-learning model learned by the pre-learning unit.

9. An uncertainty learning device according to any one of claims 1 to 8, characterized in that the mechanical device is an FA device, and the operation-related data includes a command position, a command speed, a command acceleration, a feedback speed, a feedback acceleration, a current value, or a measured value of a deviation in a machining position.

10. An uncertainty learning program for creating a machine learning model that receives as input data based on driving-related data related to the operation results of a machine, and outputs a predicted value corresponding to the driving-related data and the uncertainty of the predicted value, comprising: a learning data acquisition unit that acquires learning data created based on the driving-related data obtained from the machine; a noise addition unit that creates noise-added learning data by adding noise to the learning data acquired by the learning data acquisition unit; an outlier detection unit that calculates an outlier score from the learning data acquired by the learning data acquisition unit and the noise-added learning data created by the noise addition unit; and a model learning unit that calculates a loss function weighted based on the outlier score calculated by the outlier detection unit based on the learning data and the noise-added learning data, and learns the machine learning model.

11. An uncertainty learning system comprising: an uncertainty learning device according to any one of claims 1 to 9; and the machine device.

Images